[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 13:07:09.593709 21756 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.63.62:35313
I20260709 13:07:09.595276 21756 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 13:07:09.596308 21756 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:09.613531 21763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:09.613539 21765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:09.614904 21756 server_base.cc:1061] running on GCE node
W20260709 13:07:09.613601 21762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:10.873427 21756 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:10.873699 21756 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:10.873869 21756 hybrid_clock.cc:648] HybridClock initialized: now 1783602430873849 us; error 0 us; skew 500 ppm
I20260709 13:07:10.874727 21756 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:10.881520 21756 webserver.cc:533] Webserver started at http://127.21.63.62:45927/ using document root <none> and password file <none>
I20260709 13:07:10.883064 21756 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:10.883414 21756 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:10.883994 21756 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:10.893057 21756 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602429420501-21756-0/minicluster-data/master-0-root/instance:
uuid: "b7fa0401325f4c5e8657495c45164fe4"
format_stamp: "Formatted at 2026-07-09 13:07:10 on dist-test-slave-c6mt"
I20260709 13:07:10.901763 21756 fs_manager.cc:696] Time spent creating directory manager: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:07:10.907544 21771 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:10.908942 21756 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.001s
I20260709 13:07:10.909272 21756 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602429420501-21756-0/minicluster-data/master-0-root
uuid: "b7fa0401325f4c5e8657495c45164fe4"
format_stamp: "Formatted at 2026-07-09 13:07:10 on dist-test-slave-c6mt"
I20260709 13:07:10.909583 21756 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602429420501-21756-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602429420501-21756-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602429420501-21756-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:10.966929 21756 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:10.968531 21756 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 13:07:10.969050 21756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:11.055729 21756 rpc_server.cc:307] RPC server started. Bound to: 127.21.63.62:35313
I20260709 13:07:11.055805 21822 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.63.62:35313 every 8 connection(s)
I20260709 13:07:11.064132 21823 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:11.085800 21823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7fa0401325f4c5e8657495c45164fe4: Bootstrap starting.
I20260709 13:07:11.091681 21823 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7fa0401325f4c5e8657495c45164fe4: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:11.093535 21823 log.cc:826] T 00000000000000000000000000000000 P b7fa0401325f4c5e8657495c45164fe4: Log is configured to *not* fsync() on all Append() calls
I20260709 13:07:11.098119 21823 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7fa0401325f4c5e8657495c45164fe4: No bootstrap required, opened a new log
I20260709 13:07:11.118445 21823 raft_consensus.cc:359] T 00000000000000000000000000000000 P b7fa0401325f4c5e8657495c45164fe4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7fa0401325f4c5e8657495c45164fe4" member_type: VOTER }
I20260709 13:07:11.119014 21823 raft_consensus.cc:385] T 00000000000000000000000000000000 P b7fa0401325f4c5e8657495c45164fe4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:11.119210 21823 raft_consensus.cc:740] T 00000000000000000000000000000000 P b7fa0401325f4c5e8657495c45164fe4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7fa0401325f4c5e8657495c45164fe4, State: Initialized, Role: FOLLOWER
I20260709 13:07:11.120049 21823 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7fa0401325f4c5e8657495c45164fe4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7fa0401325f4c5e8657495c45164fe4" member_type: VOTER }
I20260709 13:07:11.120513 21823 raft_consensus.cc:399] T 00000000000000000000000000000000 P b7fa0401325f4c5e8657495c45164fe4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:07:11.120838 21823 raft_consensus.cc:493] T 00000000000000000000000000000000 P b7fa0401325f4c5e8657495c45164fe4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:07:11.121134 21823 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b7fa0401325f4c5e8657495c45164fe4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:11.125432 21823 raft_consensus.cc:515] T 00000000000000000000000000000000 P b7fa0401325f4c5e8657495c45164fe4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7fa0401325f4c5e8657495c45164fe4" member_type: VOTER }
I20260709 13:07:11.126171 21823 leader_election.cc:304] T 00000000000000000000000000000000 P b7fa0401325f4c5e8657495c45164fe4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b7fa0401325f4c5e8657495c45164fe4; no voters: 
I20260709 13:07:11.127856 21823 leader_election.cc:290] T 00000000000000000000000000000000 P b7fa0401325f4c5e8657495c45164fe4 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:07:11.128259 21826 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b7fa0401325f4c5e8657495c45164fe4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:11.130582 21826 raft_consensus.cc:697] T 00000000000000000000000000000000 P b7fa0401325f4c5e8657495c45164fe4 [term 1 LEADER]: Becoming Leader. State: Replica: b7fa0401325f4c5e8657495c45164fe4, State: Running, Role: LEADER
I20260709 13:07:11.131603 21826 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7fa0401325f4c5e8657495c45164fe4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7fa0401325f4c5e8657495c45164fe4" member_type: VOTER }
I20260709 13:07:11.132583 21823 sys_catalog.cc:565] T 00000000000000000000000000000000 P b7fa0401325f4c5e8657495c45164fe4 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:07:11.144076 21827 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7fa0401325f4c5e8657495c45164fe4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b7fa0401325f4c5e8657495c45164fe4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7fa0401325f4c5e8657495c45164fe4" member_type: VOTER } }
I20260709 13:07:11.144250 21828 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7fa0401325f4c5e8657495c45164fe4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7fa0401325f4c5e8657495c45164fe4. Latest consensus state: current_term: 1 leader_uuid: "b7fa0401325f4c5e8657495c45164fe4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7fa0401325f4c5e8657495c45164fe4" member_type: VOTER } }
I20260709 13:07:11.144922 21827 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7fa0401325f4c5e8657495c45164fe4 [sys.catalog]: This master's current role is: LEADER
I20260709 13:07:11.144995 21828 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7fa0401325f4c5e8657495c45164fe4 [sys.catalog]: This master's current role is: LEADER
I20260709 13:07:11.147603 21836 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:07:11.160312 21836 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:07:11.162854 21756 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:07:11.176551 21836 catalog_manager.cc:1383] Generated new cluster ID: a2a16716d6d140eabf86d060965ef59e
I20260709 13:07:11.176936 21836 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:07:11.202939 21836 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:07:11.204535 21836 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:07:11.221119 21836 catalog_manager.cc:6070] T 00000000000000000000000000000000 P b7fa0401325f4c5e8657495c45164fe4: Generated new TSK 0
I20260709 13:07:11.222095 21836 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:07:11.229878 21756 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:11.239542 21845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:11.241525 21846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:11.244552 21756 server_base.cc:1061] running on GCE node
W20260709 13:07:11.244572 21848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:11.246016 21756 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:11.246225 21756 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:11.246441 21756 hybrid_clock.cc:648] HybridClock initialized: now 1783602431246426 us; error 0 us; skew 500 ppm
I20260709 13:07:11.247161 21756 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:11.252074 21756 webserver.cc:533] Webserver started at http://127.21.63.1:42117/ using document root <none> and password file <none>
I20260709 13:07:11.252642 21756 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:11.252898 21756 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:11.253209 21756 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:11.254319 21756 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602429420501-21756-0/minicluster-data/ts-0-root/instance:
uuid: "01a67819c8a543adafcf1d5fff0822b4"
format_stamp: "Formatted at 2026-07-09 13:07:11 on dist-test-slave-c6mt"
I20260709 13:07:11.261687 21756 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.005s	sys 0.003s
I20260709 13:07:11.266894 21853 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:11.267957 21756 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.004s	sys 0.001s
I20260709 13:07:11.268458 21756 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602429420501-21756-0/minicluster-data/ts-0-root
uuid: "01a67819c8a543adafcf1d5fff0822b4"
format_stamp: "Formatted at 2026-07-09 13:07:11 on dist-test-slave-c6mt"
I20260709 13:07:11.268862 21756 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602429420501-21756-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602429420501-21756-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602429420501-21756-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:11.286906 21756 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:11.288465 21756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:11.291517 21756 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:11.296591 21756 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:11.296968 21756 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:07:11.297282 21756 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:11.297492 21756 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:11.409936 21756 rpc_server.cc:307] RPC server started. Bound to: 127.21.63.1:44917
I20260709 13:07:11.410117 21915 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.63.1:44917 every 8 connection(s)
I20260709 13:07:11.418262 21756 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:11.428615 21920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:11.432209 21921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:11.443439 21923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:11.445627 21756 server_base.cc:1061] running on GCE node
I20260709 13:07:11.445679 21916 heartbeater.cc:344] Connected to a master server at 127.21.63.62:35313
I20260709 13:07:11.446394 21916 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:11.446823 21756 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:11.447132 21756 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:11.447331 21756 hybrid_clock.cc:648] HybridClock initialized: now 1783602431447312 us; error 0 us; skew 500 ppm
I20260709 13:07:11.447556 21916 heartbeater.cc:507] Master 127.21.63.62:35313 requested a full tablet report, sending...
I20260709 13:07:11.448079 21756 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:11.451469 21788 ts_manager.cc:194] Registered new tserver with Master: 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1:44917)
I20260709 13:07:11.452004 21756 webserver.cc:533] Webserver started at http://127.21.63.2:33499/ using document root <none> and password file <none>
I20260709 13:07:11.452802 21756 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:11.453085 21756 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:11.453485 21756 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:11.454747 21788 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57088
I20260709 13:07:11.455173 21756 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602429420501-21756-0/minicluster-data/ts-1-root/instance:
uuid: "07a2cdeeec2b4302a2960391f3825bde"
format_stamp: "Formatted at 2026-07-09 13:07:11 on dist-test-slave-c6mt"
I20260709 13:07:11.461279 21756 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.007s	sys 0.001s
I20260709 13:07:11.465565 21928 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:11.466459 21756 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:07:11.466781 21756 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602429420501-21756-0/minicluster-data/ts-1-root
uuid: "07a2cdeeec2b4302a2960391f3825bde"
format_stamp: "Formatted at 2026-07-09 13:07:11 on dist-test-slave-c6mt"
I20260709 13:07:11.467307 21756 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602429420501-21756-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602429420501-21756-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602429420501-21756-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:11.494875 21756 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:11.496174 21756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:11.498916 21756 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:11.501861 21756 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:11.502039 21756 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:11.502344 21756 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:11.502485 21756 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:11.601996 21756 rpc_server.cc:307] RPC server started. Bound to: 127.21.63.2:36631
I20260709 13:07:11.602172 21990 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.63.2:36631 every 8 connection(s)
I20260709 13:07:11.608067 21756 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:11.621336 21995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:11.625464 21994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:11.626317 21991 heartbeater.cc:344] Connected to a master server at 127.21.63.62:35313
I20260709 13:07:11.626742 21991 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:11.627774 21991 heartbeater.cc:507] Master 127.21.63.62:35313 requested a full tablet report, sending...
I20260709 13:07:11.630453 21756 server_base.cc:1061] running on GCE node
I20260709 13:07:11.630656 21788 ts_manager.cc:194] Registered new tserver with Master: 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2:36631)
W20260709 13:07:11.631673 21997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:11.632311 21788 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57102
I20260709 13:07:11.632889 21756 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:11.633157 21756 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:11.633383 21756 hybrid_clock.cc:648] HybridClock initialized: now 1783602431633364 us; error 0 us; skew 500 ppm
I20260709 13:07:11.634167 21756 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:11.637043 21756 webserver.cc:533] Webserver started at http://127.21.63.3:44783/ using document root <none> and password file <none>
I20260709 13:07:11.637542 21756 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:11.637744 21756 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:11.638037 21756 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:11.639192 21756 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602429420501-21756-0/minicluster-data/ts-2-root/instance:
uuid: "eaf5735be053455e9315e61c44ff7286"
format_stamp: "Formatted at 2026-07-09 13:07:11 on dist-test-slave-c6mt"
I20260709 13:07:11.644197 21756 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.003s
I20260709 13:07:11.648128 22002 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:11.649045 21756 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.003s
I20260709 13:07:11.649345 21756 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602429420501-21756-0/minicluster-data/ts-2-root
uuid: "eaf5735be053455e9315e61c44ff7286"
format_stamp: "Formatted at 2026-07-09 13:07:11 on dist-test-slave-c6mt"
I20260709 13:07:11.649703 21756 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602429420501-21756-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602429420501-21756-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783602429420501-21756-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:11.670403 21756 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:11.671778 21756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:11.673218 21756 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:11.676961 21756 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:11.677219 21756 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:11.677485 21756 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:11.677690 21756 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:11.777511 21756 rpc_server.cc:307] RPC server started. Bound to: 127.21.63.3:41623
I20260709 13:07:11.777668 22064 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.63.3:41623 every 8 connection(s)
I20260709 13:07:11.796444 22065 heartbeater.cc:344] Connected to a master server at 127.21.63.62:35313
I20260709 13:07:11.796883 22065 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:11.797816 22065 heartbeater.cc:507] Master 127.21.63.62:35313 requested a full tablet report, sending...
I20260709 13:07:11.800275 21788 ts_manager.cc:194] Registered new tserver with Master: eaf5735be053455e9315e61c44ff7286 (127.21.63.3:41623)
I20260709 13:07:11.801142 21756 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.018137254s
I20260709 13:07:11.801874 21788 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57108
I20260709 13:07:11.859480 21788 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57120:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:07:11.863232 21788 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:07:11.964648 21956 tablet_service.cc:1511] Processing CreateTablet for tablet 04e73d74096641eab6b132caefd4b28f (DEFAULT_TABLE table=test-workload [id=6b03a85e91104f4da9211df9b7e3ef7e]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:11.966638 21956 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04e73d74096641eab6b132caefd4b28f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:11.975881 21955 tablet_service.cc:1511] Processing CreateTablet for tablet 2f23dcf5479b46908032cb5858ca9fdf (DEFAULT_TABLE table=test-workload [id=6b03a85e91104f4da9211df9b7e3ef7e]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:11.977679 21955 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f23dcf5479b46908032cb5858ca9fdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:11.982892 21881 tablet_service.cc:1511] Processing CreateTablet for tablet 04e73d74096641eab6b132caefd4b28f (DEFAULT_TABLE table=test-workload [id=6b03a85e91104f4da9211df9b7e3ef7e]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:11.983530 21880 tablet_service.cc:1511] Processing CreateTablet for tablet 2f23dcf5479b46908032cb5858ca9fdf (DEFAULT_TABLE table=test-workload [id=6b03a85e91104f4da9211df9b7e3ef7e]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:11.984498 21881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04e73d74096641eab6b132caefd4b28f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:11.985399 21880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f23dcf5479b46908032cb5858ca9fdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:12.010272 21879 tablet_service.cc:1511] Processing CreateTablet for tablet bf2b3d48485a40a4a54a0afcc0bd383b (DEFAULT_TABLE table=test-workload [id=6b03a85e91104f4da9211df9b7e3ef7e]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:12.011744 21879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf2b3d48485a40a4a54a0afcc0bd383b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:12.028508 21954 tablet_service.cc:1511] Processing CreateTablet for tablet bf2b3d48485a40a4a54a0afcc0bd383b (DEFAULT_TABLE table=test-workload [id=6b03a85e91104f4da9211df9b7e3ef7e]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:12.030023 21954 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf2b3d48485a40a4a54a0afcc0bd383b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:12.029474 22030 tablet_service.cc:1511] Processing CreateTablet for tablet 04e73d74096641eab6b132caefd4b28f (DEFAULT_TABLE table=test-workload [id=6b03a85e91104f4da9211df9b7e3ef7e]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:07:12.031664 22030 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 04e73d74096641eab6b132caefd4b28f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:12.035581 22029 tablet_service.cc:1511] Processing CreateTablet for tablet 2f23dcf5479b46908032cb5858ca9fdf (DEFAULT_TABLE table=test-workload [id=6b03a85e91104f4da9211df9b7e3ef7e]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:07:12.037122 22029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2f23dcf5479b46908032cb5858ca9fdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:12.043606 22028 tablet_service.cc:1511] Processing CreateTablet for tablet bf2b3d48485a40a4a54a0afcc0bd383b (DEFAULT_TABLE table=test-workload [id=6b03a85e91104f4da9211df9b7e3ef7e]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:07:12.049309 22028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf2b3d48485a40a4a54a0afcc0bd383b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:12.055204 21956 tablet_service.cc:1511] Processing CreateTablet for tablet 4c1dcbc1199442dd881d67c582108673 (DEFAULT_TABLE table=test-workload [id=6b03a85e91104f4da9211df9b7e3ef7e]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:12.057729 21956 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c1dcbc1199442dd881d67c582108673. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:12.050525 21878 tablet_service.cc:1511] Processing CreateTablet for tablet 4c1dcbc1199442dd881d67c582108673 (DEFAULT_TABLE table=test-workload [id=6b03a85e91104f4da9211df9b7e3ef7e]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:12.065845 21878 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c1dcbc1199442dd881d67c582108673. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:12.089515 22082 tablet_bootstrap.cc:492] T 04e73d74096641eab6b132caefd4b28f P 07a2cdeeec2b4302a2960391f3825bde: Bootstrap starting.
I20260709 13:07:12.099197 22027 tablet_service.cc:1511] Processing CreateTablet for tablet 4c1dcbc1199442dd881d67c582108673 (DEFAULT_TABLE table=test-workload [id=6b03a85e91104f4da9211df9b7e3ef7e]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:07:12.103211 22027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c1dcbc1199442dd881d67c582108673. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:12.105947 21880 tablet_service.cc:1511] Processing CreateTablet for tablet 4dfeb09c8a4d482ab573952200ebab0a (DEFAULT_TABLE table=test-workload [id=6b03a85e91104f4da9211df9b7e3ef7e]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:12.109620 21880 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4dfeb09c8a4d482ab573952200ebab0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:12.117693 21954 tablet_service.cc:1511] Processing CreateTablet for tablet 4dfeb09c8a4d482ab573952200ebab0a (DEFAULT_TABLE table=test-workload [id=6b03a85e91104f4da9211df9b7e3ef7e]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:12.119423 21954 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4dfeb09c8a4d482ab573952200ebab0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:12.130635 22026 tablet_service.cc:1511] Processing CreateTablet for tablet 4dfeb09c8a4d482ab573952200ebab0a (DEFAULT_TABLE table=test-workload [id=6b03a85e91104f4da9211df9b7e3ef7e]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:07:12.132525 22026 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4dfeb09c8a4d482ab573952200ebab0a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:12.134055 21956 tablet_service.cc:1511] Processing CreateTablet for tablet b10d9c93da2c485d96b8ca9058a1b00f (DEFAULT_TABLE table=test-workload [id=6b03a85e91104f4da9211df9b7e3ef7e]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:12.135354 21956 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b10d9c93da2c485d96b8ca9058a1b00f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:12.141681 22083 tablet_bootstrap.cc:492] T 04e73d74096641eab6b132caefd4b28f P 01a67819c8a543adafcf1d5fff0822b4: Bootstrap starting.
I20260709 13:07:12.149865 22082 tablet_bootstrap.cc:654] T 04e73d74096641eab6b132caefd4b28f P 07a2cdeeec2b4302a2960391f3825bde: Neither blocks nor log segments found. Creating new log.
W20260709 13:07:12.150027 21835 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:12.161202 22027 tablet_service.cc:1511] Processing CreateTablet for tablet b10d9c93da2c485d96b8ca9058a1b00f (DEFAULT_TABLE table=test-workload [id=6b03a85e91104f4da9211df9b7e3ef7e]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:12.162582 22027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b10d9c93da2c485d96b8ca9058a1b00f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:12.162397 21878 tablet_service.cc:1511] Processing CreateTablet for tablet b10d9c93da2c485d96b8ca9058a1b00f (DEFAULT_TABLE table=test-workload [id=6b03a85e91104f4da9211df9b7e3ef7e]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:07:12.165457 21878 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b10d9c93da2c485d96b8ca9058a1b00f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:12.165712 21881 tablet_service.cc:1511] Processing CreateTablet for tablet e77b8c40f5024240a36630365043ad11 (DEFAULT_TABLE table=test-workload [id=6b03a85e91104f4da9211df9b7e3ef7e]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:12.166896 21881 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e77b8c40f5024240a36630365043ad11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:12.170792 21955 tablet_service.cc:1511] Processing CreateTablet for tablet e77b8c40f5024240a36630365043ad11 (DEFAULT_TABLE table=test-workload [id=6b03a85e91104f4da9211df9b7e3ef7e]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:12.174360 21955 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e77b8c40f5024240a36630365043ad11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:12.179929 22029 tablet_service.cc:1511] Processing CreateTablet for tablet e77b8c40f5024240a36630365043ad11 (DEFAULT_TABLE table=test-workload [id=6b03a85e91104f4da9211df9b7e3ef7e]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:07:12.181356 22029 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e77b8c40f5024240a36630365043ad11. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:12.188620 21879 tablet_service.cc:1511] Processing CreateTablet for tablet 4a6231da863b48d2943784b1c955cd09 (DEFAULT_TABLE table=test-workload [id=6b03a85e91104f4da9211df9b7e3ef7e]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:12.189874 21879 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a6231da863b48d2943784b1c955cd09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:12.173974 22084 tablet_bootstrap.cc:492] T 2f23dcf5479b46908032cb5858ca9fdf P eaf5735be053455e9315e61c44ff7286: Bootstrap starting.
I20260709 13:07:12.193495 21956 tablet_service.cc:1511] Processing CreateTablet for tablet 4a6231da863b48d2943784b1c955cd09 (DEFAULT_TABLE table=test-workload [id=6b03a85e91104f4da9211df9b7e3ef7e]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:12.194734 21956 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a6231da863b48d2943784b1c955cd09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:12.207451 22028 tablet_service.cc:1511] Processing CreateTablet for tablet 4a6231da863b48d2943784b1c955cd09 (DEFAULT_TABLE table=test-workload [id=6b03a85e91104f4da9211df9b7e3ef7e]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:07:12.212270 21877 tablet_service.cc:1511] Processing CreateTablet for tablet 29b15358296749cc9bd2c0a14af3a408 (DEFAULT_TABLE table=test-workload [id=6b03a85e91104f4da9211df9b7e3ef7e]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:12.217010 22028 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4a6231da863b48d2943784b1c955cd09. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:12.218526 21953 tablet_service.cc:1511] Processing CreateTablet for tablet 29b15358296749cc9bd2c0a14af3a408 (DEFAULT_TABLE table=test-workload [id=6b03a85e91104f4da9211df9b7e3ef7e]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:12.219599 21877 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29b15358296749cc9bd2c0a14af3a408. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:12.221345 21953 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29b15358296749cc9bd2c0a14af3a408. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:12.237102 22083 tablet_bootstrap.cc:654] T 04e73d74096641eab6b132caefd4b28f P 01a67819c8a543adafcf1d5fff0822b4: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:12.238484 22082 tablet_bootstrap.cc:492] T 04e73d74096641eab6b132caefd4b28f P 07a2cdeeec2b4302a2960391f3825bde: No bootstrap required, opened a new log
I20260709 13:07:12.239197 22082 ts_tablet_manager.cc:1403] T 04e73d74096641eab6b132caefd4b28f P 07a2cdeeec2b4302a2960391f3825bde: Time spent bootstrapping tablet: real 0.173s	user 0.011s	sys 0.034s
I20260709 13:07:12.243486 22082 raft_consensus.cc:359] T 04e73d74096641eab6b132caefd4b28f P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.245194 22082 raft_consensus.cc:385] T 04e73d74096641eab6b132caefd4b28f P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:12.245594 22082 raft_consensus.cc:740] T 04e73d74096641eab6b132caefd4b28f P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07a2cdeeec2b4302a2960391f3825bde, State: Initialized, Role: FOLLOWER
I20260709 13:07:12.246773 22082 consensus_queue.cc:260] T 04e73d74096641eab6b132caefd4b28f P 07a2cdeeec2b4302a2960391f3825bde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.246945 22027 tablet_service.cc:1511] Processing CreateTablet for tablet 29b15358296749cc9bd2c0a14af3a408 (DEFAULT_TABLE table=test-workload [id=6b03a85e91104f4da9211df9b7e3ef7e]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:07:12.248701 22027 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29b15358296749cc9bd2c0a14af3a408. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:12.274075 22082 ts_tablet_manager.cc:1434] T 04e73d74096641eab6b132caefd4b28f P 07a2cdeeec2b4302a2960391f3825bde: Time spent starting tablet: real 0.034s	user 0.005s	sys 0.003s
I20260709 13:07:12.274972 21991 heartbeater.cc:499] Master 127.21.63.62:35313 was elected leader, sending a full tablet report...
I20260709 13:07:12.275210 22082 tablet_bootstrap.cc:492] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde: Bootstrap starting.
I20260709 13:07:12.276942 22084 tablet_bootstrap.cc:654] T 2f23dcf5479b46908032cb5858ca9fdf P eaf5735be053455e9315e61c44ff7286: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:12.283385 22083 tablet_bootstrap.cc:492] T 04e73d74096641eab6b132caefd4b28f P 01a67819c8a543adafcf1d5fff0822b4: No bootstrap required, opened a new log
I20260709 13:07:12.283840 22083 ts_tablet_manager.cc:1403] T 04e73d74096641eab6b132caefd4b28f P 01a67819c8a543adafcf1d5fff0822b4: Time spent bootstrapping tablet: real 0.159s	user 0.010s	sys 0.006s
I20260709 13:07:12.284350 22082 tablet_bootstrap.cc:654] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:12.286754 22083 raft_consensus.cc:359] T 04e73d74096641eab6b132caefd4b28f P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.287645 22083 raft_consensus.cc:385] T 04e73d74096641eab6b132caefd4b28f P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:12.288029 22083 raft_consensus.cc:740] T 04e73d74096641eab6b132caefd4b28f P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01a67819c8a543adafcf1d5fff0822b4, State: Initialized, Role: FOLLOWER
I20260709 13:07:12.289171 22083 consensus_queue.cc:260] T 04e73d74096641eab6b132caefd4b28f P 01a67819c8a543adafcf1d5fff0822b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.294970 21916 heartbeater.cc:499] Master 127.21.63.62:35313 was elected leader, sending a full tablet report...
I20260709 13:07:12.297605 22084 tablet_bootstrap.cc:492] T 2f23dcf5479b46908032cb5858ca9fdf P eaf5735be053455e9315e61c44ff7286: No bootstrap required, opened a new log
I20260709 13:07:12.298210 22084 ts_tablet_manager.cc:1403] T 2f23dcf5479b46908032cb5858ca9fdf P eaf5735be053455e9315e61c44ff7286: Time spent bootstrapping tablet: real 0.125s	user 0.012s	sys 0.004s
I20260709 13:07:12.303489 22084 raft_consensus.cc:359] T 2f23dcf5479b46908032cb5858ca9fdf P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.306386 22083 ts_tablet_manager.cc:1434] T 04e73d74096641eab6b132caefd4b28f P 01a67819c8a543adafcf1d5fff0822b4: Time spent starting tablet: real 0.022s	user 0.001s	sys 0.010s
I20260709 13:07:12.307791 22083 tablet_bootstrap.cc:492] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4: Bootstrap starting.
I20260709 13:07:12.304459 22084 raft_consensus.cc:385] T 2f23dcf5479b46908032cb5858ca9fdf P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:12.317010 22084 raft_consensus.cc:740] T 2f23dcf5479b46908032cb5858ca9fdf P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaf5735be053455e9315e61c44ff7286, State: Initialized, Role: FOLLOWER
I20260709 13:07:12.318450 22084 consensus_queue.cc:260] T 2f23dcf5479b46908032cb5858ca9fdf P eaf5735be053455e9315e61c44ff7286 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.316635 22083 tablet_bootstrap.cc:654] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:12.324046 22082 tablet_bootstrap.cc:492] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde: No bootstrap required, opened a new log
I20260709 13:07:12.324569 22082 ts_tablet_manager.cc:1403] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde: Time spent bootstrapping tablet: real 0.050s	user 0.012s	sys 0.027s
I20260709 13:07:12.327809 22082 raft_consensus.cc:359] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.329010 22082 raft_consensus.cc:385] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:12.329831 22082 raft_consensus.cc:740] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07a2cdeeec2b4302a2960391f3825bde, State: Initialized, Role: FOLLOWER
I20260709 13:07:12.330910 22084 ts_tablet_manager.cc:1434] T 2f23dcf5479b46908032cb5858ca9fdf P eaf5735be053455e9315e61c44ff7286: Time spent starting tablet: real 0.031s	user 0.015s	sys 0.000s
I20260709 13:07:12.331936 22065 heartbeater.cc:499] Master 127.21.63.62:35313 was elected leader, sending a full tablet report...
I20260709 13:07:12.332450 22084 tablet_bootstrap.cc:492] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286: Bootstrap starting.
I20260709 13:07:12.335414 22083 tablet_bootstrap.cc:492] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4: No bootstrap required, opened a new log
I20260709 13:07:12.335886 22083 ts_tablet_manager.cc:1403] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4: Time spent bootstrapping tablet: real 0.028s	user 0.006s	sys 0.013s
I20260709 13:07:12.338887 22084 tablet_bootstrap.cc:654] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:12.339056 22083 raft_consensus.cc:359] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.339843 22083 raft_consensus.cc:385] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:12.340145 22083 raft_consensus.cc:740] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01a67819c8a543adafcf1d5fff0822b4, State: Initialized, Role: FOLLOWER
I20260709 13:07:12.340898 22083 consensus_queue.cc:260] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.343290 22083 ts_tablet_manager.cc:1434] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:07:12.344187 22083 tablet_bootstrap.cc:492] T bf2b3d48485a40a4a54a0afcc0bd383b P 01a67819c8a543adafcf1d5fff0822b4: Bootstrap starting.
I20260709 13:07:12.346303 22090 raft_consensus.cc:493] T 04e73d74096641eab6b132caefd4b28f P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.330964 22082 consensus_queue.cc:260] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.346792 22090 raft_consensus.cc:515] T 04e73d74096641eab6b132caefd4b28f P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.348403 22082 ts_tablet_manager.cc:1434] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde: Time spent starting tablet: real 0.023s	user 0.002s	sys 0.004s
I20260709 13:07:12.349776 22090 leader_election.cc:290] T 04e73d74096641eab6b132caefd4b28f P 01a67819c8a543adafcf1d5fff0822b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2:36631), eaf5735be053455e9315e61c44ff7286 (127.21.63.3:41623)
I20260709 13:07:12.350853 22086 raft_consensus.cc:493] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.351293 22086 raft_consensus.cc:515] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.353865 22086 leader_election.cc:290] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1:44917), eaf5735be053455e9315e61c44ff7286 (127.21.63.3:41623)
W20260709 13:07:12.359654 21992 tablet.cc:2406] T 04e73d74096641eab6b132caefd4b28f P 07a2cdeeec2b4302a2960391f3825bde: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:07:12.359136 22083 tablet_bootstrap.cc:654] T bf2b3d48485a40a4a54a0afcc0bd383b P 01a67819c8a543adafcf1d5fff0822b4: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:12.349290 22082 tablet_bootstrap.cc:492] T 4c1dcbc1199442dd881d67c582108673 P 07a2cdeeec2b4302a2960391f3825bde: Bootstrap starting.
I20260709 13:07:12.368484 22083 tablet_bootstrap.cc:492] T bf2b3d48485a40a4a54a0afcc0bd383b P 01a67819c8a543adafcf1d5fff0822b4: No bootstrap required, opened a new log
I20260709 13:07:12.369088 22083 ts_tablet_manager.cc:1403] T bf2b3d48485a40a4a54a0afcc0bd383b P 01a67819c8a543adafcf1d5fff0822b4: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.001s
I20260709 13:07:12.373385 22083 raft_consensus.cc:359] T bf2b3d48485a40a4a54a0afcc0bd383b P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.374274 22083 raft_consensus.cc:385] T bf2b3d48485a40a4a54a0afcc0bd383b P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:12.374624 22083 raft_consensus.cc:740] T bf2b3d48485a40a4a54a0afcc0bd383b P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01a67819c8a543adafcf1d5fff0822b4, State: Initialized, Role: FOLLOWER
I20260709 13:07:12.375464 22083 consensus_queue.cc:260] T bf2b3d48485a40a4a54a0afcc0bd383b P 01a67819c8a543adafcf1d5fff0822b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.376788 22084 tablet_bootstrap.cc:492] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286: No bootstrap required, opened a new log
I20260709 13:07:12.377295 22084 ts_tablet_manager.cc:1403] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286: Time spent bootstrapping tablet: real 0.045s	user 0.008s	sys 0.006s
I20260709 13:07:12.378378 22083 ts_tablet_manager.cc:1434] T bf2b3d48485a40a4a54a0afcc0bd383b P 01a67819c8a543adafcf1d5fff0822b4: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.005s
I20260709 13:07:12.379482 22083 tablet_bootstrap.cc:492] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4: Bootstrap starting.
I20260709 13:07:12.380275 22084 raft_consensus.cc:359] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.381088 22084 raft_consensus.cc:385] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:12.381453 22084 raft_consensus.cc:740] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaf5735be053455e9315e61c44ff7286, State: Initialized, Role: FOLLOWER
I20260709 13:07:12.382203 22084 consensus_queue.cc:260] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.384573 22084 ts_tablet_manager.cc:1434] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.004s
I20260709 13:07:12.387609 22082 tablet_bootstrap.cc:654] T 4c1dcbc1199442dd881d67c582108673 P 07a2cdeeec2b4302a2960391f3825bde: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:12.387863 22083 tablet_bootstrap.cc:654] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:12.392009 22084 tablet_bootstrap.cc:492] T 04e73d74096641eab6b132caefd4b28f P eaf5735be053455e9315e61c44ff7286: Bootstrap starting.
I20260709 13:07:12.403482 22084 tablet_bootstrap.cc:654] T 04e73d74096641eab6b132caefd4b28f P eaf5735be053455e9315e61c44ff7286: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:12.403940 21966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04e73d74096641eab6b132caefd4b28f" candidate_uuid: "01a67819c8a543adafcf1d5fff0822b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a2cdeeec2b4302a2960391f3825bde" is_pre_election: true
I20260709 13:07:12.404875 21966 raft_consensus.cc:2468] T 04e73d74096641eab6b132caefd4b28f P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01a67819c8a543adafcf1d5fff0822b4 in term 0.
I20260709 13:07:12.406349 21857 leader_election.cc:304] T 04e73d74096641eab6b132caefd4b28f P 01a67819c8a543adafcf1d5fff0822b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a67819c8a543adafcf1d5fff0822b4, 07a2cdeeec2b4302a2960391f3825bde; no voters: 
I20260709 13:07:12.407393 22090 raft_consensus.cc:2804] T 04e73d74096641eab6b132caefd4b28f P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:12.407992 22090 raft_consensus.cc:493] T 04e73d74096641eab6b132caefd4b28f P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:12.408520 22090 raft_consensus.cc:3060] T 04e73d74096641eab6b132caefd4b28f P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:12.419052 22090 raft_consensus.cc:515] T 04e73d74096641eab6b132caefd4b28f P 01a67819c8a543adafcf1d5fff0822b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.422359 22090 leader_election.cc:290] T 04e73d74096641eab6b132caefd4b28f P 01a67819c8a543adafcf1d5fff0822b4 [CANDIDATE]: Term 1 election: Requested vote from peers 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2:36631), eaf5735be053455e9315e61c44ff7286 (127.21.63.3:41623)
I20260709 13:07:12.424436 21966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04e73d74096641eab6b132caefd4b28f" candidate_uuid: "01a67819c8a543adafcf1d5fff0822b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a2cdeeec2b4302a2960391f3825bde"
I20260709 13:07:12.425215 21966 raft_consensus.cc:3060] T 04e73d74096641eab6b132caefd4b28f P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:12.440719 22082 tablet_bootstrap.cc:492] T 4c1dcbc1199442dd881d67c582108673 P 07a2cdeeec2b4302a2960391f3825bde: No bootstrap required, opened a new log
I20260709 13:07:12.441506 22082 ts_tablet_manager.cc:1403] T 4c1dcbc1199442dd881d67c582108673 P 07a2cdeeec2b4302a2960391f3825bde: Time spent bootstrapping tablet: real 0.092s	user 0.030s	sys 0.016s
I20260709 13:07:12.444124 21966 raft_consensus.cc:2468] T 04e73d74096641eab6b132caefd4b28f P 07a2cdeeec2b4302a2960391f3825bde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01a67819c8a543adafcf1d5fff0822b4 in term 1.
I20260709 13:07:12.446161 22082 raft_consensus.cc:359] T 4c1dcbc1199442dd881d67c582108673 P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.455147 22082 raft_consensus.cc:385] T 4c1dcbc1199442dd881d67c582108673 P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:12.455700 22082 raft_consensus.cc:740] T 4c1dcbc1199442dd881d67c582108673 P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07a2cdeeec2b4302a2960391f3825bde, State: Initialized, Role: FOLLOWER
I20260709 13:07:12.457825 22040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04e73d74096641eab6b132caefd4b28f" candidate_uuid: "01a67819c8a543adafcf1d5fff0822b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaf5735be053455e9315e61c44ff7286" is_pre_election: true
I20260709 13:07:12.460393 22039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04e73d74096641eab6b132caefd4b28f" candidate_uuid: "01a67819c8a543adafcf1d5fff0822b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaf5735be053455e9315e61c44ff7286"
W20260709 13:07:12.460650 21854 leader_election.cc:343] T 04e73d74096641eab6b132caefd4b28f P 01a67819c8a543adafcf1d5fff0822b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eaf5735be053455e9315e61c44ff7286 (127.21.63.3:41623): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:07:12.462924 21854 leader_election.cc:343] T 04e73d74096641eab6b132caefd4b28f P 01a67819c8a543adafcf1d5fff0822b4 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer eaf5735be053455e9315e61c44ff7286 (127.21.63.3:41623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:12.464365 21857 leader_election.cc:304] T 04e73d74096641eab6b132caefd4b28f P 01a67819c8a543adafcf1d5fff0822b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01a67819c8a543adafcf1d5fff0822b4, 07a2cdeeec2b4302a2960391f3825bde; no voters: eaf5735be053455e9315e61c44ff7286
I20260709 13:07:12.465596 22090 raft_consensus.cc:2804] T 04e73d74096641eab6b132caefd4b28f P 01a67819c8a543adafcf1d5fff0822b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:12.465109 22082 consensus_queue.cc:260] T 4c1dcbc1199442dd881d67c582108673 P 07a2cdeeec2b4302a2960391f3825bde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.467598 21891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f23dcf5479b46908032cb5858ca9fdf" candidate_uuid: "07a2cdeeec2b4302a2960391f3825bde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01a67819c8a543adafcf1d5fff0822b4" is_pre_election: true
I20260709 13:07:12.468540 21891 raft_consensus.cc:2468] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07a2cdeeec2b4302a2960391f3825bde in term 0.
I20260709 13:07:12.470188 21929 leader_election.cc:304] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a67819c8a543adafcf1d5fff0822b4, 07a2cdeeec2b4302a2960391f3825bde; no voters: 
I20260709 13:07:12.471313 22086 raft_consensus.cc:2804] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:12.471504 22082 ts_tablet_manager.cc:1434] T 4c1dcbc1199442dd881d67c582108673 P 07a2cdeeec2b4302a2960391f3825bde: Time spent starting tablet: real 0.030s	user 0.003s	sys 0.007s
I20260709 13:07:12.471768 22086 raft_consensus.cc:493] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:12.472198 22086 raft_consensus.cc:3060] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:12.472703 22082 tablet_bootstrap.cc:492] T bf2b3d48485a40a4a54a0afcc0bd383b P 07a2cdeeec2b4302a2960391f3825bde: Bootstrap starting.
I20260709 13:07:12.476956 22091 raft_consensus.cc:493] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.477885 22084 tablet_bootstrap.cc:492] T 04e73d74096641eab6b132caefd4b28f P eaf5735be053455e9315e61c44ff7286: No bootstrap required, opened a new log
I20260709 13:07:12.477555 22091 raft_consensus.cc:515] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.478562 22084 ts_tablet_manager.cc:1403] T 04e73d74096641eab6b132caefd4b28f P eaf5735be053455e9315e61c44ff7286: Time spent bootstrapping tablet: real 0.087s	user 0.017s	sys 0.053s
I20260709 13:07:12.480984 22091 leader_election.cc:290] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1:44917), 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2:36631)
I20260709 13:07:12.482426 22084 raft_consensus.cc:359] T 04e73d74096641eab6b132caefd4b28f P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.483232 22084 raft_consensus.cc:385] T 04e73d74096641eab6b132caefd4b28f P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:12.483680 22084 raft_consensus.cc:740] T 04e73d74096641eab6b132caefd4b28f P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaf5735be053455e9315e61c44ff7286, State: Initialized, Role: FOLLOWER
I20260709 13:07:12.485004 22084 consensus_queue.cc:260] T 04e73d74096641eab6b132caefd4b28f P eaf5735be053455e9315e61c44ff7286 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.487989 22090 raft_consensus.cc:697] T 04e73d74096641eab6b132caefd4b28f P 01a67819c8a543adafcf1d5fff0822b4 [term 1 LEADER]: Becoming Leader. State: Replica: 01a67819c8a543adafcf1d5fff0822b4, State: Running, Role: LEADER
I20260709 13:07:12.488300 22084 ts_tablet_manager.cc:1434] T 04e73d74096641eab6b132caefd4b28f P eaf5735be053455e9315e61c44ff7286: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:07:12.489012 22090 consensus_queue.cc:237] T 04e73d74096641eab6b132caefd4b28f P 01a67819c8a543adafcf1d5fff0822b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.489749 22084 tablet_bootstrap.cc:492] T 4c1dcbc1199442dd881d67c582108673 P eaf5735be053455e9315e61c44ff7286: Bootstrap starting.
I20260709 13:07:12.491804 22082 tablet_bootstrap.cc:654] T bf2b3d48485a40a4a54a0afcc0bd383b P 07a2cdeeec2b4302a2960391f3825bde: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:12.500046 22083 tablet_bootstrap.cc:492] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4: No bootstrap required, opened a new log
I20260709 13:07:12.500631 22083 ts_tablet_manager.cc:1403] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4: Time spent bootstrapping tablet: real 0.121s	user 0.036s	sys 0.032s
I20260709 13:07:12.505550 22083 raft_consensus.cc:359] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.506289 22083 raft_consensus.cc:385] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:12.506603 22083 raft_consensus.cc:740] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01a67819c8a543adafcf1d5fff0822b4, State: Initialized, Role: FOLLOWER
I20260709 13:07:12.507391 22083 consensus_queue.cc:260] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.508769 22086 raft_consensus.cc:515] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.511449 22091 raft_consensus.cc:493] T 2f23dcf5479b46908032cb5858ca9fdf P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.512171 22091 raft_consensus.cc:515] T 2f23dcf5479b46908032cb5858ca9fdf P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.516918 21891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f23dcf5479b46908032cb5858ca9fdf" candidate_uuid: "07a2cdeeec2b4302a2960391f3825bde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01a67819c8a543adafcf1d5fff0822b4"
I20260709 13:07:12.516835 22086 leader_election.cc:290] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [CANDIDATE]: Term 1 election: Requested vote from peers 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1:44917), eaf5735be053455e9315e61c44ff7286 (127.21.63.3:41623)
I20260709 13:07:12.518452 21891 raft_consensus.cc:3060] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:12.518368 22091 leader_election.cc:290] T 2f23dcf5479b46908032cb5858ca9fdf P eaf5735be053455e9315e61c44ff7286 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1:44917), 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2:36631)
I20260709 13:07:12.535312 22039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f23dcf5479b46908032cb5858ca9fdf" candidate_uuid: "07a2cdeeec2b4302a2960391f3825bde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaf5735be053455e9315e61c44ff7286" is_pre_election: true
I20260709 13:07:12.536864 22084 tablet_bootstrap.cc:654] T 4c1dcbc1199442dd881d67c582108673 P eaf5735be053455e9315e61c44ff7286: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:12.536405 22039 raft_consensus.cc:2468] T 2f23dcf5479b46908032cb5858ca9fdf P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07a2cdeeec2b4302a2960391f3825bde in term 0.
I20260709 13:07:12.539480 22083 ts_tablet_manager.cc:1434] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4: Time spent starting tablet: real 0.038s	user 0.003s	sys 0.005s
I20260709 13:07:12.540529 22083 tablet_bootstrap.cc:492] T 4dfeb09c8a4d482ab573952200ebab0a P 01a67819c8a543adafcf1d5fff0822b4: Bootstrap starting.
I20260709 13:07:12.549350 22039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f23dcf5479b46908032cb5858ca9fdf" candidate_uuid: "07a2cdeeec2b4302a2960391f3825bde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaf5735be053455e9315e61c44ff7286"
I20260709 13:07:12.550060 22039 raft_consensus.cc:3060] T 2f23dcf5479b46908032cb5858ca9fdf P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:12.542511 21786 catalog_manager.cc:5697] T 04e73d74096641eab6b132caefd4b28f P 01a67819c8a543adafcf1d5fff0822b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1). New cstate: current_term: 1 leader_uuid: "01a67819c8a543adafcf1d5fff0822b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:12.558975 21891 raft_consensus.cc:2468] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07a2cdeeec2b4302a2960391f3825bde in term 1.
I20260709 13:07:12.560523 21929 leader_election.cc:304] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a67819c8a543adafcf1d5fff0822b4, 07a2cdeeec2b4302a2960391f3825bde; no voters: 
I20260709 13:07:12.561595 22086 raft_consensus.cc:2804] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:12.565569 22083 tablet_bootstrap.cc:654] T 4dfeb09c8a4d482ab573952200ebab0a P 01a67819c8a543adafcf1d5fff0822b4: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:12.566514 22090 raft_consensus.cc:493] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.568428 21890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f23dcf5479b46908032cb5858ca9fdf" candidate_uuid: "eaf5735be053455e9315e61c44ff7286" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01a67819c8a543adafcf1d5fff0822b4" is_pre_election: true
I20260709 13:07:12.569411 22090 raft_consensus.cc:515] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.570189 21890 raft_consensus.cc:2393] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eaf5735be053455e9315e61c44ff7286 in current term 1: Already voted for candidate 07a2cdeeec2b4302a2960391f3825bde in this term.
I20260709 13:07:12.567497 21891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf2b3d48485a40a4a54a0afcc0bd383b" candidate_uuid: "eaf5735be053455e9315e61c44ff7286" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01a67819c8a543adafcf1d5fff0822b4" is_pre_election: true
I20260709 13:07:12.571579 21891 raft_consensus.cc:2468] T bf2b3d48485a40a4a54a0afcc0bd383b P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eaf5735be053455e9315e61c44ff7286 in term 0.
I20260709 13:07:12.572695 21966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c1dcbc1199442dd881d67c582108673" candidate_uuid: "01a67819c8a543adafcf1d5fff0822b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a2cdeeec2b4302a2960391f3825bde" is_pre_election: true
I20260709 13:07:12.572785 22090 leader_election.cc:290] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2:36631), eaf5735be053455e9315e61c44ff7286 (127.21.63.3:41623)
I20260709 13:07:12.580865 21966 raft_consensus.cc:2468] T 4c1dcbc1199442dd881d67c582108673 P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01a67819c8a543adafcf1d5fff0822b4 in term 0.
I20260709 13:07:12.581331 22040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c1dcbc1199442dd881d67c582108673" candidate_uuid: "01a67819c8a543adafcf1d5fff0822b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaf5735be053455e9315e61c44ff7286" is_pre_election: true
I20260709 13:07:12.582231 22003 leader_election.cc:304] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a67819c8a543adafcf1d5fff0822b4, eaf5735be053455e9315e61c44ff7286; no voters: 
W20260709 13:07:12.583088 21854 leader_election.cc:343] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer eaf5735be053455e9315e61c44ff7286 (127.21.63.3:41623): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:12.584215 22091 raft_consensus.cc:2804] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:12.584560 22091 raft_consensus.cc:493] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:12.584582 21857 leader_election.cc:304] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 01a67819c8a543adafcf1d5fff0822b4, 07a2cdeeec2b4302a2960391f3825bde; no voters: eaf5735be053455e9315e61c44ff7286
I20260709 13:07:12.585028 22091 raft_consensus.cc:3060] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:12.585314 22039 raft_consensus.cc:2468] T 2f23dcf5479b46908032cb5858ca9fdf P eaf5735be053455e9315e61c44ff7286 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07a2cdeeec2b4302a2960391f3825bde in term 1.
I20260709 13:07:12.583724 22082 tablet_bootstrap.cc:492] T bf2b3d48485a40a4a54a0afcc0bd383b P 07a2cdeeec2b4302a2960391f3825bde: No bootstrap required, opened a new log
I20260709 13:07:12.586299 22090 raft_consensus.cc:2804] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:12.586647 22090 raft_consensus.cc:493] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:12.582747 22086 raft_consensus.cc:697] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [term 1 LEADER]: Becoming Leader. State: Replica: 07a2cdeeec2b4302a2960391f3825bde, State: Running, Role: LEADER
I20260709 13:07:12.586961 22090 raft_consensus.cc:3060] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:12.596949 21965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f23dcf5479b46908032cb5858ca9fdf" candidate_uuid: "eaf5735be053455e9315e61c44ff7286" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a2cdeeec2b4302a2960391f3825bde" is_pre_election: true
I20260709 13:07:12.599552 22006 leader_election.cc:304] T 2f23dcf5479b46908032cb5858ca9fdf P eaf5735be053455e9315e61c44ff7286 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eaf5735be053455e9315e61c44ff7286; no voters: 01a67819c8a543adafcf1d5fff0822b4, 07a2cdeeec2b4302a2960391f3825bde
I20260709 13:07:12.606746 22091 raft_consensus.cc:515] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.610066 21891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf2b3d48485a40a4a54a0afcc0bd383b" candidate_uuid: "eaf5735be053455e9315e61c44ff7286" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01a67819c8a543adafcf1d5fff0822b4"
I20260709 13:07:12.610623 21891 raft_consensus.cc:3060] T bf2b3d48485a40a4a54a0afcc0bd383b P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:12.612411 22082 ts_tablet_manager.cc:1403] T bf2b3d48485a40a4a54a0afcc0bd383b P 07a2cdeeec2b4302a2960391f3825bde: Time spent bootstrapping tablet: real 0.140s	user 0.035s	sys 0.023s
I20260709 13:07:12.612977 21966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf2b3d48485a40a4a54a0afcc0bd383b" candidate_uuid: "eaf5735be053455e9315e61c44ff7286" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a2cdeeec2b4302a2960391f3825bde" is_pre_election: true
I20260709 13:07:12.613183 22091 leader_election.cc:290] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [CANDIDATE]: Term 1 election: Requested vote from peers 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1:44917), 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2:36631)
I20260709 13:07:12.613703 22091 raft_consensus.cc:2749] T 2f23dcf5479b46908032cb5858ca9fdf P eaf5735be053455e9315e61c44ff7286 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:12.614740 22086 consensus_queue.cc:237] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.615280 22082 raft_consensus.cc:359] T bf2b3d48485a40a4a54a0afcc0bd383b P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.615990 22082 raft_consensus.cc:385] T bf2b3d48485a40a4a54a0afcc0bd383b P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:12.616267 22082 raft_consensus.cc:740] T bf2b3d48485a40a4a54a0afcc0bd383b P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07a2cdeeec2b4302a2960391f3825bde, State: Initialized, Role: FOLLOWER
I20260709 13:07:12.616704 22090 raft_consensus.cc:515] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.616988 22082 consensus_queue.cc:260] T bf2b3d48485a40a4a54a0afcc0bd383b P 07a2cdeeec2b4302a2960391f3825bde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.619151 22082 ts_tablet_manager.cc:1434] T bf2b3d48485a40a4a54a0afcc0bd383b P 07a2cdeeec2b4302a2960391f3825bde: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:07:12.619997 22090 leader_election.cc:290] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4 [CANDIDATE]: Term 1 election: Requested vote from peers 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2:36631), eaf5735be053455e9315e61c44ff7286 (127.21.63.3:41623)
I20260709 13:07:12.620694 21966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c1dcbc1199442dd881d67c582108673" candidate_uuid: "01a67819c8a543adafcf1d5fff0822b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a2cdeeec2b4302a2960391f3825bde"
I20260709 13:07:12.621229 21966 raft_consensus.cc:3060] T 4c1dcbc1199442dd881d67c582108673 P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:12.623740 21891 raft_consensus.cc:2468] T bf2b3d48485a40a4a54a0afcc0bd383b P 01a67819c8a543adafcf1d5fff0822b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaf5735be053455e9315e61c44ff7286 in term 1.
I20260709 13:07:12.625355 22003 leader_election.cc:304] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a67819c8a543adafcf1d5fff0822b4, eaf5735be053455e9315e61c44ff7286; no voters: 
I20260709 13:07:12.626281 22091 raft_consensus.cc:2804] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:12.626717 22091 raft_consensus.cc:697] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [term 1 LEADER]: Becoming Leader. State: Replica: eaf5735be053455e9315e61c44ff7286, State: Running, Role: LEADER
I20260709 13:07:12.627617 22091 consensus_queue.cc:237] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.628744 21966 raft_consensus.cc:2468] T 4c1dcbc1199442dd881d67c582108673 P 07a2cdeeec2b4302a2960391f3825bde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01a67819c8a543adafcf1d5fff0822b4 in term 1.
I20260709 13:07:12.629873 21857 leader_election.cc:304] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a67819c8a543adafcf1d5fff0822b4, 07a2cdeeec2b4302a2960391f3825bde; no voters: 
I20260709 13:07:12.631084 22090 raft_consensus.cc:2804] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:12.631795 22090 raft_consensus.cc:697] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4 [term 1 LEADER]: Becoming Leader. State: Replica: 01a67819c8a543adafcf1d5fff0822b4, State: Running, Role: LEADER
I20260709 13:07:12.632151 22084 tablet_bootstrap.cc:492] T 4c1dcbc1199442dd881d67c582108673 P eaf5735be053455e9315e61c44ff7286: No bootstrap required, opened a new log
I20260709 13:07:12.632601 22084 ts_tablet_manager.cc:1403] T 4c1dcbc1199442dd881d67c582108673 P eaf5735be053455e9315e61c44ff7286: Time spent bootstrapping tablet: real 0.143s	user 0.011s	sys 0.019s
I20260709 13:07:12.632864 22090 consensus_queue.cc:237] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.635720 22084 raft_consensus.cc:359] T 4c1dcbc1199442dd881d67c582108673 P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.636755 22084 raft_consensus.cc:385] T 4c1dcbc1199442dd881d67c582108673 P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:12.637068 22084 raft_consensus.cc:740] T 4c1dcbc1199442dd881d67c582108673 P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaf5735be053455e9315e61c44ff7286, State: Initialized, Role: FOLLOWER
I20260709 13:07:12.638052 22084 consensus_queue.cc:260] T 4c1dcbc1199442dd881d67c582108673 P eaf5735be053455e9315e61c44ff7286 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.643476 22082 tablet_bootstrap.cc:492] T b10d9c93da2c485d96b8ca9058a1b00f P 07a2cdeeec2b4302a2960391f3825bde: Bootstrap starting.
W20260709 13:07:12.660040 22006 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:36631, user_credentials={real_user=slave}} blocked reactor thread for 60750.2us
I20260709 13:07:12.662127 22084 ts_tablet_manager.cc:1434] T 4c1dcbc1199442dd881d67c582108673 P eaf5735be053455e9315e61c44ff7286: Time spent starting tablet: real 0.029s	user 0.009s	sys 0.002s
I20260709 13:07:12.665553 22084 tablet_bootstrap.cc:492] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286: Bootstrap starting.
I20260709 13:07:12.668789 21966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf2b3d48485a40a4a54a0afcc0bd383b" candidate_uuid: "eaf5735be053455e9315e61c44ff7286" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a2cdeeec2b4302a2960391f3825bde"
I20260709 13:07:12.671176 21966 raft_consensus.cc:3060] T bf2b3d48485a40a4a54a0afcc0bd383b P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:12.673398 22117 raft_consensus.cc:493] T 04e73d74096641eab6b132caefd4b28f P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.674458 22117 raft_consensus.cc:515] T 04e73d74096641eab6b132caefd4b28f P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.676575 22039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c1dcbc1199442dd881d67c582108673" candidate_uuid: "01a67819c8a543adafcf1d5fff0822b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaf5735be053455e9315e61c44ff7286"
I20260709 13:07:12.675179 22083 tablet_bootstrap.cc:492] T 4dfeb09c8a4d482ab573952200ebab0a P 01a67819c8a543adafcf1d5fff0822b4: No bootstrap required, opened a new log
I20260709 13:07:12.677323 22039 raft_consensus.cc:3060] T 4c1dcbc1199442dd881d67c582108673 P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:12.679951 22083 ts_tablet_manager.cc:1403] T 4dfeb09c8a4d482ab573952200ebab0a P 01a67819c8a543adafcf1d5fff0822b4: Time spent bootstrapping tablet: real 0.140s	user 0.017s	sys 0.006s
W20260709 13:07:12.679919 22006 leader_election.cc:343] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2:36631): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:12.680383 22117 leader_election.cc:290] T 04e73d74096641eab6b132caefd4b28f P eaf5735be053455e9315e61c44ff7286 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2:36631), 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1:44917)
I20260709 13:07:12.681546 21891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04e73d74096641eab6b132caefd4b28f" candidate_uuid: "eaf5735be053455e9315e61c44ff7286" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01a67819c8a543adafcf1d5fff0822b4" is_pre_election: true
I20260709 13:07:12.681098 21787 catalog_manager.cc:5697] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 reported cstate change: term changed from 0 to 1, leader changed from <none> to eaf5735be053455e9315e61c44ff7286 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "eaf5735be053455e9315e61c44ff7286" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:12.689419 21965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04e73d74096641eab6b132caefd4b28f" candidate_uuid: "eaf5735be053455e9315e61c44ff7286" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a2cdeeec2b4302a2960391f3825bde" is_pre_election: true
I20260709 13:07:12.691039 21965 raft_consensus.cc:2393] T 04e73d74096641eab6b132caefd4b28f P 07a2cdeeec2b4302a2960391f3825bde [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate eaf5735be053455e9315e61c44ff7286 in current term 1: Already voted for candidate 01a67819c8a543adafcf1d5fff0822b4 in this term.
I20260709 13:07:12.694201 22006 leader_election.cc:304] T 04e73d74096641eab6b132caefd4b28f P eaf5735be053455e9315e61c44ff7286 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: eaf5735be053455e9315e61c44ff7286; no voters: 01a67819c8a543adafcf1d5fff0822b4, 07a2cdeeec2b4302a2960391f3825bde
I20260709 13:07:12.692853 21966 raft_consensus.cc:2468] T bf2b3d48485a40a4a54a0afcc0bd383b P 07a2cdeeec2b4302a2960391f3825bde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaf5735be053455e9315e61c44ff7286 in term 1.
I20260709 13:07:12.696194 22117 raft_consensus.cc:3060] T 04e73d74096641eab6b132caefd4b28f P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:12.694005 21785 catalog_manager.cc:5697] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde reported cstate change: term changed from 0 to 1, leader changed from <none> to 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2). New cstate: current_term: 1 leader_uuid: "07a2cdeeec2b4302a2960391f3825bde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:12.693079 22039 raft_consensus.cc:2468] T 4c1dcbc1199442dd881d67c582108673 P eaf5735be053455e9315e61c44ff7286 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01a67819c8a543adafcf1d5fff0822b4 in term 1.
I20260709 13:07:12.696360 22082 tablet_bootstrap.cc:654] T b10d9c93da2c485d96b8ca9058a1b00f P 07a2cdeeec2b4302a2960391f3825bde: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:12.701411 22083 raft_consensus.cc:359] T 4dfeb09c8a4d482ab573952200ebab0a P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.703092 22083 raft_consensus.cc:385] T 4dfeb09c8a4d482ab573952200ebab0a P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:12.704155 22083 raft_consensus.cc:740] T 4dfeb09c8a4d482ab573952200ebab0a P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01a67819c8a543adafcf1d5fff0822b4, State: Initialized, Role: FOLLOWER
I20260709 13:07:12.713816 22117 raft_consensus.cc:2749] T 04e73d74096641eab6b132caefd4b28f P eaf5735be053455e9315e61c44ff7286 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:12.706744 22083 consensus_queue.cc:260] T 4dfeb09c8a4d482ab573952200ebab0a P 01a67819c8a543adafcf1d5fff0822b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.716547 22083 ts_tablet_manager.cc:1434] T 4dfeb09c8a4d482ab573952200ebab0a P 01a67819c8a543adafcf1d5fff0822b4: Time spent starting tablet: real 0.034s	user 0.000s	sys 0.005s
I20260709 13:07:12.717523 22083 tablet_bootstrap.cc:492] T 4a6231da863b48d2943784b1c955cd09 P 01a67819c8a543adafcf1d5fff0822b4: Bootstrap starting.
I20260709 13:07:12.719661 22084 tablet_bootstrap.cc:654] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:12.723894 22083 tablet_bootstrap.cc:654] T 4a6231da863b48d2943784b1c955cd09 P 01a67819c8a543adafcf1d5fff0822b4: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:12.733767 21786 catalog_manager.cc:5697] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1). New cstate: current_term: 1 leader_uuid: "01a67819c8a543adafcf1d5fff0822b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:12.744724 22082 tablet_bootstrap.cc:492] T b10d9c93da2c485d96b8ca9058a1b00f P 07a2cdeeec2b4302a2960391f3825bde: No bootstrap required, opened a new log
I20260709 13:07:12.745210 22082 ts_tablet_manager.cc:1403] T b10d9c93da2c485d96b8ca9058a1b00f P 07a2cdeeec2b4302a2960391f3825bde: Time spent bootstrapping tablet: real 0.104s	user 0.010s	sys 0.008s
I20260709 13:07:12.748327 22082 raft_consensus.cc:359] T b10d9c93da2c485d96b8ca9058a1b00f P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.753268 22084 tablet_bootstrap.cc:492] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286: No bootstrap required, opened a new log
I20260709 13:07:12.753268 22082 raft_consensus.cc:385] T b10d9c93da2c485d96b8ca9058a1b00f P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:12.753882 22084 ts_tablet_manager.cc:1403] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286: Time spent bootstrapping tablet: real 0.089s	user 0.011s	sys 0.009s
I20260709 13:07:12.753877 22082 raft_consensus.cc:740] T b10d9c93da2c485d96b8ca9058a1b00f P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07a2cdeeec2b4302a2960391f3825bde, State: Initialized, Role: FOLLOWER
I20260709 13:07:12.754868 22082 consensus_queue.cc:260] T b10d9c93da2c485d96b8ca9058a1b00f P 07a2cdeeec2b4302a2960391f3825bde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.755625 22083 tablet_bootstrap.cc:492] T 4a6231da863b48d2943784b1c955cd09 P 01a67819c8a543adafcf1d5fff0822b4: No bootstrap required, opened a new log
I20260709 13:07:12.756210 22083 ts_tablet_manager.cc:1403] T 4a6231da863b48d2943784b1c955cd09 P 01a67819c8a543adafcf1d5fff0822b4: Time spent bootstrapping tablet: real 0.039s	user 0.007s	sys 0.010s
I20260709 13:07:12.757129 22084 raft_consensus.cc:359] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.757853 22084 raft_consensus.cc:385] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:12.758154 22084 raft_consensus.cc:740] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaf5735be053455e9315e61c44ff7286, State: Initialized, Role: FOLLOWER
I20260709 13:07:12.758888 22084 consensus_queue.cc:260] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.759267 22083 raft_consensus.cc:359] T 4a6231da863b48d2943784b1c955cd09 P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.760061 22083 raft_consensus.cc:385] T 4a6231da863b48d2943784b1c955cd09 P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:12.760422 22083 raft_consensus.cc:740] T 4a6231da863b48d2943784b1c955cd09 P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01a67819c8a543adafcf1d5fff0822b4, State: Initialized, Role: FOLLOWER
I20260709 13:07:12.761323 22083 consensus_queue.cc:260] T 4a6231da863b48d2943784b1c955cd09 P 01a67819c8a543adafcf1d5fff0822b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.763936 22083 ts_tablet_manager.cc:1434] T 4a6231da863b48d2943784b1c955cd09 P 01a67819c8a543adafcf1d5fff0822b4: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.005s
I20260709 13:07:12.764976 22083 tablet_bootstrap.cc:492] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4: Bootstrap starting.
I20260709 13:07:12.765326 22084 ts_tablet_manager.cc:1434] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.002s
I20260709 13:07:12.767961 22082 ts_tablet_manager.cc:1434] T b10d9c93da2c485d96b8ca9058a1b00f P 07a2cdeeec2b4302a2960391f3825bde: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.001s
I20260709 13:07:12.769331 22082 tablet_bootstrap.cc:492] T e77b8c40f5024240a36630365043ad11 P 07a2cdeeec2b4302a2960391f3825bde: Bootstrap starting.
I20260709 13:07:12.772912 22084 tablet_bootstrap.cc:492] T b10d9c93da2c485d96b8ca9058a1b00f P eaf5735be053455e9315e61c44ff7286: Bootstrap starting.
I20260709 13:07:12.777824 22082 tablet_bootstrap.cc:654] T e77b8c40f5024240a36630365043ad11 P 07a2cdeeec2b4302a2960391f3825bde: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:12.783776 22083 tablet_bootstrap.cc:654] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:12.787386 22084 tablet_bootstrap.cc:654] T b10d9c93da2c485d96b8ca9058a1b00f P eaf5735be053455e9315e61c44ff7286: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:12.793144 22082 tablet_bootstrap.cc:492] T e77b8c40f5024240a36630365043ad11 P 07a2cdeeec2b4302a2960391f3825bde: No bootstrap required, opened a new log
I20260709 13:07:12.793637 22082 ts_tablet_manager.cc:1403] T e77b8c40f5024240a36630365043ad11 P 07a2cdeeec2b4302a2960391f3825bde: Time spent bootstrapping tablet: real 0.025s	user 0.011s	sys 0.008s
I20260709 13:07:12.796650 22083 tablet_bootstrap.cc:492] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4: No bootstrap required, opened a new log
I20260709 13:07:12.796613 22082 raft_consensus.cc:359] T e77b8c40f5024240a36630365043ad11 P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.797379 22083 ts_tablet_manager.cc:1403] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4: Time spent bootstrapping tablet: real 0.033s	user 0.006s	sys 0.014s
I20260709 13:07:12.797426 22082 raft_consensus.cc:385] T e77b8c40f5024240a36630365043ad11 P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:12.798008 22082 raft_consensus.cc:740] T e77b8c40f5024240a36630365043ad11 P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07a2cdeeec2b4302a2960391f3825bde, State: Initialized, Role: FOLLOWER
I20260709 13:07:12.798755 22082 consensus_queue.cc:260] T e77b8c40f5024240a36630365043ad11 P 07a2cdeeec2b4302a2960391f3825bde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.800930 22083 raft_consensus.cc:359] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.801301 22082 ts_tablet_manager.cc:1434] T e77b8c40f5024240a36630365043ad11 P 07a2cdeeec2b4302a2960391f3825bde: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:07:12.801638 22083 raft_consensus.cc:385] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:12.802044 22083 raft_consensus.cc:740] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01a67819c8a543adafcf1d5fff0822b4, State: Initialized, Role: FOLLOWER
I20260709 13:07:12.802428 22082 tablet_bootstrap.cc:492] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde: Bootstrap starting.
I20260709 13:07:12.802882 22083 consensus_queue.cc:260] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.809648 22082 tablet_bootstrap.cc:654] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:12.811789 22083 ts_tablet_manager.cc:1434] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:07:12.815706 22083 tablet_bootstrap.cc:492] T b10d9c93da2c485d96b8ca9058a1b00f P 01a67819c8a543adafcf1d5fff0822b4: Bootstrap starting.
I20260709 13:07:12.821895 22084 tablet_bootstrap.cc:492] T b10d9c93da2c485d96b8ca9058a1b00f P eaf5735be053455e9315e61c44ff7286: No bootstrap required, opened a new log
I20260709 13:07:12.821976 22082 tablet_bootstrap.cc:492] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde: No bootstrap required, opened a new log
I20260709 13:07:12.822415 22084 ts_tablet_manager.cc:1403] T b10d9c93da2c485d96b8ca9058a1b00f P eaf5735be053455e9315e61c44ff7286: Time spent bootstrapping tablet: real 0.053s	user 0.021s	sys 0.001s
I20260709 13:07:12.822521 22082 ts_tablet_manager.cc:1403] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde: Time spent bootstrapping tablet: real 0.020s	user 0.014s	sys 0.003s
I20260709 13:07:12.823212 22083 tablet_bootstrap.cc:654] T b10d9c93da2c485d96b8ca9058a1b00f P 01a67819c8a543adafcf1d5fff0822b4: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:12.825865 22082 raft_consensus.cc:359] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.826582 22082 raft_consensus.cc:385] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:12.826900 22082 raft_consensus.cc:740] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07a2cdeeec2b4302a2960391f3825bde, State: Initialized, Role: FOLLOWER
I20260709 13:07:12.826680 22084 raft_consensus.cc:359] T b10d9c93da2c485d96b8ca9058a1b00f P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.827735 22084 raft_consensus.cc:385] T b10d9c93da2c485d96b8ca9058a1b00f P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:12.827678 22082 consensus_queue.cc:260] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.828230 22084 raft_consensus.cc:740] T b10d9c93da2c485d96b8ca9058a1b00f P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaf5735be053455e9315e61c44ff7286, State: Initialized, Role: FOLLOWER
I20260709 13:07:12.829169 22084 consensus_queue.cc:260] T b10d9c93da2c485d96b8ca9058a1b00f P eaf5735be053455e9315e61c44ff7286 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.830335 22082 ts_tablet_manager.cc:1434] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:07:12.831290 22082 tablet_bootstrap.cc:492] T 4dfeb09c8a4d482ab573952200ebab0a P 07a2cdeeec2b4302a2960391f3825bde: Bootstrap starting.
I20260709 13:07:12.831661 22084 ts_tablet_manager.cc:1434] T b10d9c93da2c485d96b8ca9058a1b00f P eaf5735be053455e9315e61c44ff7286: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:07:12.832700 22084 tablet_bootstrap.cc:492] T e77b8c40f5024240a36630365043ad11 P eaf5735be053455e9315e61c44ff7286: Bootstrap starting.
I20260709 13:07:12.838235 22082 tablet_bootstrap.cc:654] T 4dfeb09c8a4d482ab573952200ebab0a P 07a2cdeeec2b4302a2960391f3825bde: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:12.843348 22083 tablet_bootstrap.cc:492] T b10d9c93da2c485d96b8ca9058a1b00f P 01a67819c8a543adafcf1d5fff0822b4: No bootstrap required, opened a new log
I20260709 13:07:12.843780 22083 ts_tablet_manager.cc:1403] T b10d9c93da2c485d96b8ca9058a1b00f P 01a67819c8a543adafcf1d5fff0822b4: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.000s
I20260709 13:07:12.847150 22083 raft_consensus.cc:359] T b10d9c93da2c485d96b8ca9058a1b00f P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.847875 22083 raft_consensus.cc:385] T b10d9c93da2c485d96b8ca9058a1b00f P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:12.848178 22083 raft_consensus.cc:740] T b10d9c93da2c485d96b8ca9058a1b00f P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01a67819c8a543adafcf1d5fff0822b4, State: Initialized, Role: FOLLOWER
I20260709 13:07:12.849331 22082 tablet_bootstrap.cc:492] T 4dfeb09c8a4d482ab573952200ebab0a P 07a2cdeeec2b4302a2960391f3825bde: No bootstrap required, opened a new log
I20260709 13:07:12.848837 22083 consensus_queue.cc:260] T b10d9c93da2c485d96b8ca9058a1b00f P 01a67819c8a543adafcf1d5fff0822b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.849766 22082 ts_tablet_manager.cc:1403] T 4dfeb09c8a4d482ab573952200ebab0a P 07a2cdeeec2b4302a2960391f3825bde: Time spent bootstrapping tablet: real 0.019s	user 0.014s	sys 0.003s
I20260709 13:07:12.852258 22082 raft_consensus.cc:359] T 4dfeb09c8a4d482ab573952200ebab0a P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.852905 22082 raft_consensus.cc:385] T 4dfeb09c8a4d482ab573952200ebab0a P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:12.853175 22082 raft_consensus.cc:740] T 4dfeb09c8a4d482ab573952200ebab0a P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07a2cdeeec2b4302a2960391f3825bde, State: Initialized, Role: FOLLOWER
I20260709 13:07:12.854107 22084 tablet_bootstrap.cc:654] T e77b8c40f5024240a36630365043ad11 P eaf5735be053455e9315e61c44ff7286: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:12.853794 22082 consensus_queue.cc:260] T 4dfeb09c8a4d482ab573952200ebab0a P 07a2cdeeec2b4302a2960391f3825bde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.856050 22082 ts_tablet_manager.cc:1434] T 4dfeb09c8a4d482ab573952200ebab0a P 07a2cdeeec2b4302a2960391f3825bde: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:07:12.856863 22082 tablet_bootstrap.cc:492] T 29b15358296749cc9bd2c0a14af3a408 P 07a2cdeeec2b4302a2960391f3825bde: Bootstrap starting.
I20260709 13:07:12.860949 22083 ts_tablet_manager.cc:1434] T b10d9c93da2c485d96b8ca9058a1b00f P 01a67819c8a543adafcf1d5fff0822b4: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.002s
I20260709 13:07:12.862054 22083 tablet_bootstrap.cc:492] T 29b15358296749cc9bd2c0a14af3a408 P 01a67819c8a543adafcf1d5fff0822b4: Bootstrap starting.
I20260709 13:07:12.862174 22082 tablet_bootstrap.cc:654] T 29b15358296749cc9bd2c0a14af3a408 P 07a2cdeeec2b4302a2960391f3825bde: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:12.868266 22084 tablet_bootstrap.cc:492] T e77b8c40f5024240a36630365043ad11 P eaf5735be053455e9315e61c44ff7286: No bootstrap required, opened a new log
I20260709 13:07:12.868793 22084 ts_tablet_manager.cc:1403] T e77b8c40f5024240a36630365043ad11 P eaf5735be053455e9315e61c44ff7286: Time spent bootstrapping tablet: real 0.036s	user 0.014s	sys 0.000s
I20260709 13:07:12.871564 22082 tablet_bootstrap.cc:492] T 29b15358296749cc9bd2c0a14af3a408 P 07a2cdeeec2b4302a2960391f3825bde: No bootstrap required, opened a new log
I20260709 13:07:12.871999 22082 ts_tablet_manager.cc:1403] T 29b15358296749cc9bd2c0a14af3a408 P 07a2cdeeec2b4302a2960391f3825bde: Time spent bootstrapping tablet: real 0.015s	user 0.002s	sys 0.010s
I20260709 13:07:12.873329 22084 raft_consensus.cc:359] T e77b8c40f5024240a36630365043ad11 P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.874050 22084 raft_consensus.cc:385] T e77b8c40f5024240a36630365043ad11 P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:12.874358 22084 raft_consensus.cc:740] T e77b8c40f5024240a36630365043ad11 P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaf5735be053455e9315e61c44ff7286, State: Initialized, Role: FOLLOWER
I20260709 13:07:12.875047 22082 raft_consensus.cc:359] T 29b15358296749cc9bd2c0a14af3a408 P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.875144 22084 consensus_queue.cc:260] T e77b8c40f5024240a36630365043ad11 P eaf5735be053455e9315e61c44ff7286 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.875743 22082 raft_consensus.cc:385] T 29b15358296749cc9bd2c0a14af3a408 P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:12.876075 22082 raft_consensus.cc:740] T 29b15358296749cc9bd2c0a14af3a408 P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07a2cdeeec2b4302a2960391f3825bde, State: Initialized, Role: FOLLOWER
I20260709 13:07:12.876734 22082 consensus_queue.cc:260] T 29b15358296749cc9bd2c0a14af3a408 P 07a2cdeeec2b4302a2960391f3825bde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.878768 22082 ts_tablet_manager.cc:1434] T 29b15358296749cc9bd2c0a14af3a408 P 07a2cdeeec2b4302a2960391f3825bde: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.002s
I20260709 13:07:12.882686 22084 ts_tablet_manager.cc:1434] T e77b8c40f5024240a36630365043ad11 P eaf5735be053455e9315e61c44ff7286: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:07:12.883669 22084 tablet_bootstrap.cc:492] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286: Bootstrap starting.
I20260709 13:07:12.893098 22084 tablet_bootstrap.cc:654] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:12.894431 22083 tablet_bootstrap.cc:654] T 29b15358296749cc9bd2c0a14af3a408 P 01a67819c8a543adafcf1d5fff0822b4: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:12.899114 22083 tablet_bootstrap.cc:492] T 29b15358296749cc9bd2c0a14af3a408 P 01a67819c8a543adafcf1d5fff0822b4: No bootstrap required, opened a new log
I20260709 13:07:12.899621 22083 ts_tablet_manager.cc:1403] T 29b15358296749cc9bd2c0a14af3a408 P 01a67819c8a543adafcf1d5fff0822b4: Time spent bootstrapping tablet: real 0.038s	user 0.010s	sys 0.000s
I20260709 13:07:12.900923 22084 tablet_bootstrap.cc:492] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286: No bootstrap required, opened a new log
I20260709 13:07:12.901360 22084 ts_tablet_manager.cc:1403] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286: Time spent bootstrapping tablet: real 0.018s	user 0.008s	sys 0.002s
I20260709 13:07:12.902326 22083 raft_consensus.cc:359] T 29b15358296749cc9bd2c0a14af3a408 P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.903041 22083 raft_consensus.cc:385] T 29b15358296749cc9bd2c0a14af3a408 P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:12.903362 22083 raft_consensus.cc:740] T 29b15358296749cc9bd2c0a14af3a408 P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01a67819c8a543adafcf1d5fff0822b4, State: Initialized, Role: FOLLOWER
I20260709 13:07:12.904191 22083 consensus_queue.cc:260] T 29b15358296749cc9bd2c0a14af3a408 P 01a67819c8a543adafcf1d5fff0822b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.904305 22084 raft_consensus.cc:359] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.905185 22084 raft_consensus.cc:385] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:12.905524 22084 raft_consensus.cc:740] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaf5735be053455e9315e61c44ff7286, State: Initialized, Role: FOLLOWER
I20260709 13:07:12.906668 22083 ts_tablet_manager.cc:1434] T 29b15358296749cc9bd2c0a14af3a408 P 01a67819c8a543adafcf1d5fff0822b4: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:07:12.906340 22084 consensus_queue.cc:260] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.908721 22084 ts_tablet_manager.cc:1434] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:07:12.909451 22117 raft_consensus.cc:493] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.909884 22117 raft_consensus.cc:515] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.919215 22117 leader_election.cc:290] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1:44917), 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2:36631)
I20260709 13:07:12.920171 21966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a6231da863b48d2943784b1c955cd09" candidate_uuid: "eaf5735be053455e9315e61c44ff7286" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a2cdeeec2b4302a2960391f3825bde" is_pre_election: true
I20260709 13:07:12.920786 21966 raft_consensus.cc:2468] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eaf5735be053455e9315e61c44ff7286 in term 0.
I20260709 13:07:12.920790 21891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a6231da863b48d2943784b1c955cd09" candidate_uuid: "eaf5735be053455e9315e61c44ff7286" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01a67819c8a543adafcf1d5fff0822b4" is_pre_election: true
I20260709 13:07:12.921515 21891 raft_consensus.cc:2468] T 4a6231da863b48d2943784b1c955cd09 P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eaf5735be053455e9315e61c44ff7286 in term 0.
I20260709 13:07:12.922704 22003 leader_election.cc:304] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a67819c8a543adafcf1d5fff0822b4, eaf5735be053455e9315e61c44ff7286; no voters: 
I20260709 13:07:12.923691 22117 raft_consensus.cc:2804] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:12.924044 22117 raft_consensus.cc:493] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:12.924424 22117 raft_consensus.cc:3060] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:12.926164 22086 raft_consensus.cc:493] T b10d9c93da2c485d96b8ca9058a1b00f P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.926622 22086 raft_consensus.cc:515] T b10d9c93da2c485d96b8ca9058a1b00f P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.931077 22117 raft_consensus.cc:515] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.931409 21891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10d9c93da2c485d96b8ca9058a1b00f" candidate_uuid: "07a2cdeeec2b4302a2960391f3825bde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01a67819c8a543adafcf1d5fff0822b4" is_pre_election: true
I20260709 13:07:12.932065 21891 raft_consensus.cc:2468] T b10d9c93da2c485d96b8ca9058a1b00f P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07a2cdeeec2b4302a2960391f3825bde in term 0.
I20260709 13:07:12.933086 22086 leader_election.cc:290] T b10d9c93da2c485d96b8ca9058a1b00f P 07a2cdeeec2b4302a2960391f3825bde [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1:44917), eaf5735be053455e9315e61c44ff7286 (127.21.63.3:41623)
I20260709 13:07:12.933208 21929 leader_election.cc:304] T b10d9c93da2c485d96b8ca9058a1b00f P 07a2cdeeec2b4302a2960391f3825bde [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a67819c8a543adafcf1d5fff0822b4, 07a2cdeeec2b4302a2960391f3825bde; no voters: 
I20260709 13:07:12.933106 22084 tablet_bootstrap.cc:492] T 29b15358296749cc9bd2c0a14af3a408 P eaf5735be053455e9315e61c44ff7286: Bootstrap starting.
I20260709 13:07:12.933632 22117 leader_election.cc:290] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [CANDIDATE]: Term 1 election: Requested vote from peers 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1:44917), 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2:36631)
I20260709 13:07:12.935518 21891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a6231da863b48d2943784b1c955cd09" candidate_uuid: "eaf5735be053455e9315e61c44ff7286" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01a67819c8a543adafcf1d5fff0822b4"
I20260709 13:07:12.936161 21891 raft_consensus.cc:3060] T 4a6231da863b48d2943784b1c955cd09 P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:12.936889 22086 raft_consensus.cc:2804] T b10d9c93da2c485d96b8ca9058a1b00f P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:12.937258 22086 raft_consensus.cc:493] T b10d9c93da2c485d96b8ca9058a1b00f P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:12.937598 22086 raft_consensus.cc:3060] T b10d9c93da2c485d96b8ca9058a1b00f P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:12.937736 22039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10d9c93da2c485d96b8ca9058a1b00f" candidate_uuid: "07a2cdeeec2b4302a2960391f3825bde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaf5735be053455e9315e61c44ff7286" is_pre_election: true
I20260709 13:07:12.938143 21966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a6231da863b48d2943784b1c955cd09" candidate_uuid: "eaf5735be053455e9315e61c44ff7286" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a2cdeeec2b4302a2960391f3825bde"
I20260709 13:07:12.938411 22039 raft_consensus.cc:2468] T b10d9c93da2c485d96b8ca9058a1b00f P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 07a2cdeeec2b4302a2960391f3825bde in term 0.
I20260709 13:07:12.938678 21966 raft_consensus.cc:3060] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:12.944272 22090 raft_consensus.cc:493] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.944833 22090 raft_consensus.cc:515] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.945192 21966 raft_consensus.cc:2468] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaf5735be053455e9315e61c44ff7286 in term 1.
I20260709 13:07:12.946372 22006 leader_election.cc:304] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07a2cdeeec2b4302a2960391f3825bde, eaf5735be053455e9315e61c44ff7286; no voters: 
I20260709 13:07:12.946919 22090 leader_election.cc:290] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2:36631), eaf5735be053455e9315e61c44ff7286 (127.21.63.3:41623)
I20260709 13:07:12.948215 21965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b8c40f5024240a36630365043ad11" candidate_uuid: "01a67819c8a543adafcf1d5fff0822b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a2cdeeec2b4302a2960391f3825bde" is_pre_election: true
I20260709 13:07:12.948621 22039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b8c40f5024240a36630365043ad11" candidate_uuid: "01a67819c8a543adafcf1d5fff0822b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaf5735be053455e9315e61c44ff7286" is_pre_election: true
I20260709 13:07:12.948945 21965 raft_consensus.cc:2468] T e77b8c40f5024240a36630365043ad11 P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01a67819c8a543adafcf1d5fff0822b4 in term 0.
I20260709 13:07:12.949290 22039 raft_consensus.cc:2468] T e77b8c40f5024240a36630365043ad11 P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01a67819c8a543adafcf1d5fff0822b4 in term 0.
I20260709 13:07:12.950037 22084 tablet_bootstrap.cc:654] T 29b15358296749cc9bd2c0a14af3a408 P eaf5735be053455e9315e61c44ff7286: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:12.950182 21857 leader_election.cc:304] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a67819c8a543adafcf1d5fff0822b4, 07a2cdeeec2b4302a2960391f3825bde; no voters: 
I20260709 13:07:12.946179 21891 raft_consensus.cc:2468] T 4a6231da863b48d2943784b1c955cd09 P 01a67819c8a543adafcf1d5fff0822b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaf5735be053455e9315e61c44ff7286 in term 1.
I20260709 13:07:12.953428 22090 raft_consensus.cc:2804] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:12.953814 22090 raft_consensus.cc:493] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:12.954136 22090 raft_consensus.cc:3060] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:12.954552 22084 tablet_bootstrap.cc:492] T 29b15358296749cc9bd2c0a14af3a408 P eaf5735be053455e9315e61c44ff7286: No bootstrap required, opened a new log
I20260709 13:07:12.954933 22084 ts_tablet_manager.cc:1403] T 29b15358296749cc9bd2c0a14af3a408 P eaf5735be053455e9315e61c44ff7286: Time spent bootstrapping tablet: real 0.022s	user 0.008s	sys 0.001s
I20260709 13:07:12.956740 22084 raft_consensus.cc:359] T 29b15358296749cc9bd2c0a14af3a408 P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.957226 22084 raft_consensus.cc:385] T 29b15358296749cc9bd2c0a14af3a408 P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:12.957439 22084 raft_consensus.cc:740] T 29b15358296749cc9bd2c0a14af3a408 P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaf5735be053455e9315e61c44ff7286, State: Initialized, Role: FOLLOWER
I20260709 13:07:12.958062 22084 consensus_queue.cc:260] T 29b15358296749cc9bd2c0a14af3a408 P eaf5735be053455e9315e61c44ff7286 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.959738 22084 ts_tablet_manager.cc:1434] T 29b15358296749cc9bd2c0a14af3a408 P eaf5735be053455e9315e61c44ff7286: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.000s
I20260709 13:07:12.960939 22090 raft_consensus.cc:515] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.963785 21966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b8c40f5024240a36630365043ad11" candidate_uuid: "01a67819c8a543adafcf1d5fff0822b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a2cdeeec2b4302a2960391f3825bde"
I20260709 13:07:12.964201 22039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b8c40f5024240a36630365043ad11" candidate_uuid: "01a67819c8a543adafcf1d5fff0822b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaf5735be053455e9315e61c44ff7286"
I20260709 13:07:12.964498 21966 raft_consensus.cc:3060] T e77b8c40f5024240a36630365043ad11 P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:12.964905 22039 raft_consensus.cc:3060] T e77b8c40f5024240a36630365043ad11 P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:12.970670 22091 raft_consensus.cc:493] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.971158 22091 raft_consensus.cc:515] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.971675 22103 raft_consensus.cc:493] T 29b15358296749cc9bd2c0a14af3a408 P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:12.972115 22103 raft_consensus.cc:515] T 29b15358296749cc9bd2c0a14af3a408 P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.973268 21966 raft_consensus.cc:2468] T e77b8c40f5024240a36630365043ad11 P 07a2cdeeec2b4302a2960391f3825bde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01a67819c8a543adafcf1d5fff0822b4 in term 1.
I20260709 13:07:12.975346 21964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b15358296749cc9bd2c0a14af3a408" candidate_uuid: "01a67819c8a543adafcf1d5fff0822b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a2cdeeec2b4302a2960391f3825bde" is_pre_election: true
I20260709 13:07:12.974175 22103 leader_election.cc:290] T 29b15358296749cc9bd2c0a14af3a408 P 01a67819c8a543adafcf1d5fff0822b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2:36631), eaf5735be053455e9315e61c44ff7286 (127.21.63.3:41623)
I20260709 13:07:12.975399 21857 leader_election.cc:304] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a67819c8a543adafcf1d5fff0822b4, 07a2cdeeec2b4302a2960391f3825bde; no voters: 
I20260709 13:07:12.976133 21964 raft_consensus.cc:2468] T 29b15358296749cc9bd2c0a14af3a408 P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01a67819c8a543adafcf1d5fff0822b4 in term 0.
I20260709 13:07:12.976480 22103 raft_consensus.cc:2804] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:12.977352 21857 leader_election.cc:304] T 29b15358296749cc9bd2c0a14af3a408 P 01a67819c8a543adafcf1d5fff0822b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a67819c8a543adafcf1d5fff0822b4, 07a2cdeeec2b4302a2960391f3825bde; no voters: 
I20260709 13:07:12.977470 22040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b15358296749cc9bd2c0a14af3a408" candidate_uuid: "01a67819c8a543adafcf1d5fff0822b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaf5735be053455e9315e61c44ff7286" is_pre_election: true
I20260709 13:07:12.977682 21965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dfeb09c8a4d482ab573952200ebab0a" candidate_uuid: "eaf5735be053455e9315e61c44ff7286" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a2cdeeec2b4302a2960391f3825bde" is_pre_election: true
I20260709 13:07:12.978118 22040 raft_consensus.cc:2468] T 29b15358296749cc9bd2c0a14af3a408 P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 01a67819c8a543adafcf1d5fff0822b4 in term 0.
I20260709 13:07:12.978531 21965 raft_consensus.cc:2468] T 4dfeb09c8a4d482ab573952200ebab0a P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eaf5735be053455e9315e61c44ff7286 in term 0.
I20260709 13:07:12.979805 21891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dfeb09c8a4d482ab573952200ebab0a" candidate_uuid: "eaf5735be053455e9315e61c44ff7286" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01a67819c8a543adafcf1d5fff0822b4" is_pre_election: true
I20260709 13:07:12.979888 22006 leader_election.cc:304] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07a2cdeeec2b4302a2960391f3825bde, eaf5735be053455e9315e61c44ff7286; no voters: 
I20260709 13:07:12.980455 21891 raft_consensus.cc:2468] T 4dfeb09c8a4d482ab573952200ebab0a P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eaf5735be053455e9315e61c44ff7286 in term 0.
I20260709 13:07:12.980826 22111 raft_consensus.cc:2804] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:12.973388 22091 leader_election.cc:290] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1:44917), 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2:36631)
I20260709 13:07:12.981277 22111 raft_consensus.cc:493] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:12.982191 22111 raft_consensus.cc:3060] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:12.983505 22039 raft_consensus.cc:2468] T e77b8c40f5024240a36630365043ad11 P eaf5735be053455e9315e61c44ff7286 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01a67819c8a543adafcf1d5fff0822b4 in term 1.
I20260709 13:07:12.947597 22117 raft_consensus.cc:2804] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:12.985641 22117 raft_consensus.cc:697] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [term 1 LEADER]: Becoming Leader. State: Replica: eaf5735be053455e9315e61c44ff7286, State: Running, Role: LEADER
I20260709 13:07:12.986853 22117 consensus_queue.cc:237] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.988900 22111 raft_consensus.cc:515] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.944271 22086 raft_consensus.cc:515] T b10d9c93da2c485d96b8ca9058a1b00f P 07a2cdeeec2b4302a2960391f3825bde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:12.991703 21891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dfeb09c8a4d482ab573952200ebab0a" candidate_uuid: "eaf5735be053455e9315e61c44ff7286" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01a67819c8a543adafcf1d5fff0822b4"
I20260709 13:07:12.992259 21891 raft_consensus.cc:3060] T 4dfeb09c8a4d482ab573952200ebab0a P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:12.993173 22086 leader_election.cc:290] T b10d9c93da2c485d96b8ca9058a1b00f P 07a2cdeeec2b4302a2960391f3825bde [CANDIDATE]: Term 1 election: Requested vote from peers 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1:44917), eaf5735be053455e9315e61c44ff7286 (127.21.63.3:41623)
I20260709 13:07:12.995160 22090 leader_election.cc:290] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4 [CANDIDATE]: Term 1 election: Requested vote from peers 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2:36631), eaf5735be053455e9315e61c44ff7286 (127.21.63.3:41623)
I20260709 13:07:12.997155 22090 raft_consensus.cc:2804] T 29b15358296749cc9bd2c0a14af3a408 P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:12.997695 22090 raft_consensus.cc:493] T 29b15358296749cc9bd2c0a14af3a408 P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:12.998208 22090 raft_consensus.cc:3060] T 29b15358296749cc9bd2c0a14af3a408 P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:12.998907 21891 raft_consensus.cc:2468] T 4dfeb09c8a4d482ab573952200ebab0a P 01a67819c8a543adafcf1d5fff0822b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaf5735be053455e9315e61c44ff7286 in term 1.
I20260709 13:07:13.000119 22003 leader_election.cc:304] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a67819c8a543adafcf1d5fff0822b4, eaf5735be053455e9315e61c44ff7286; no voters: 
I20260709 13:07:13.000885 22091 raft_consensus.cc:2804] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:13.002795 21890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10d9c93da2c485d96b8ca9058a1b00f" candidate_uuid: "07a2cdeeec2b4302a2960391f3825bde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01a67819c8a543adafcf1d5fff0822b4"
I20260709 13:07:13.003417 21890 raft_consensus.cc:3060] T b10d9c93da2c485d96b8ca9058a1b00f P 01a67819c8a543adafcf1d5fff0822b4 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:13.009874 22039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b10d9c93da2c485d96b8ca9058a1b00f" candidate_uuid: "07a2cdeeec2b4302a2960391f3825bde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaf5735be053455e9315e61c44ff7286"
I20260709 13:07:13.010540 22039 raft_consensus.cc:3060] T b10d9c93da2c485d96b8ca9058a1b00f P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:13.018818 21890 raft_consensus.cc:2468] T b10d9c93da2c485d96b8ca9058a1b00f P 01a67819c8a543adafcf1d5fff0822b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07a2cdeeec2b4302a2960391f3825bde in term 1.
I20260709 13:07:13.019176 22039 raft_consensus.cc:2468] T b10d9c93da2c485d96b8ca9058a1b00f P eaf5735be053455e9315e61c44ff7286 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07a2cdeeec2b4302a2960391f3825bde in term 1.
I20260709 13:07:13.018997 22090 raft_consensus.cc:515] T 29b15358296749cc9bd2c0a14af3a408 P 01a67819c8a543adafcf1d5fff0822b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:13.020172 21929 leader_election.cc:304] T b10d9c93da2c485d96b8ca9058a1b00f P 07a2cdeeec2b4302a2960391f3825bde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a67819c8a543adafcf1d5fff0822b4, 07a2cdeeec2b4302a2960391f3825bde; no voters: 
I20260709 13:07:13.021080 22086 raft_consensus.cc:2804] T b10d9c93da2c485d96b8ca9058a1b00f P 07a2cdeeec2b4302a2960391f3825bde [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:13.021693 22086 raft_consensus.cc:697] T b10d9c93da2c485d96b8ca9058a1b00f P 07a2cdeeec2b4302a2960391f3825bde [term 1 LEADER]: Becoming Leader. State: Replica: 07a2cdeeec2b4302a2960391f3825bde, State: Running, Role: LEADER
I20260709 13:07:13.022747 22086 consensus_queue.cc:237] T b10d9c93da2c485d96b8ca9058a1b00f P 07a2cdeeec2b4302a2960391f3825bde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:13.024959 21965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dfeb09c8a4d482ab573952200ebab0a" candidate_uuid: "eaf5735be053455e9315e61c44ff7286" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a2cdeeec2b4302a2960391f3825bde"
I20260709 13:07:13.025542 21965 raft_consensus.cc:3060] T 4dfeb09c8a4d482ab573952200ebab0a P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:13.033156 21965 raft_consensus.cc:2468] T 4dfeb09c8a4d482ab573952200ebab0a P 07a2cdeeec2b4302a2960391f3825bde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaf5735be053455e9315e61c44ff7286 in term 1.
W20260709 13:07:13.034911 21857 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:36631, user_credentials={real_user=slave}} blocked reactor thread for 57638.1us
I20260709 13:07:13.034991 22103 raft_consensus.cc:697] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4 [term 1 LEADER]: Becoming Leader. State: Replica: 01a67819c8a543adafcf1d5fff0822b4, State: Running, Role: LEADER
I20260709 13:07:13.036262 22111 leader_election.cc:290] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 [CANDIDATE]: Term 1 election: Requested vote from peers 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1:44917), 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2:36631)
I20260709 13:07:13.036183 22103 consensus_queue.cc:237] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:13.044193 22110 consensus_queue.cc:1048] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [LEADER]: Connected to new peer: Peer: permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:13.048779 22091 raft_consensus.cc:697] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 [term 1 LEADER]: Becoming Leader. State: Replica: eaf5735be053455e9315e61c44ff7286, State: Running, Role: LEADER
I20260709 13:07:13.049651 22091 consensus_queue.cc:237] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:13.053777 21965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b15358296749cc9bd2c0a14af3a408" candidate_uuid: "01a67819c8a543adafcf1d5fff0822b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07a2cdeeec2b4302a2960391f3825bde"
I20260709 13:07:13.054097 22039 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b15358296749cc9bd2c0a14af3a408" candidate_uuid: "01a67819c8a543adafcf1d5fff0822b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eaf5735be053455e9315e61c44ff7286"
I20260709 13:07:13.054391 21965 raft_consensus.cc:3060] T 29b15358296749cc9bd2c0a14af3a408 P 07a2cdeeec2b4302a2960391f3825bde [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:13.054796 22039 raft_consensus.cc:3060] T 29b15358296749cc9bd2c0a14af3a408 P eaf5735be053455e9315e61c44ff7286 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:13.055310 22090 leader_election.cc:290] T 29b15358296749cc9bd2c0a14af3a408 P 01a67819c8a543adafcf1d5fff0822b4 [CANDIDATE]: Term 1 election: Requested vote from peers 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2:36631), eaf5735be053455e9315e61c44ff7286 (127.21.63.3:41623)
I20260709 13:07:13.062526 22134 consensus_queue.cc:1048] T 04e73d74096641eab6b132caefd4b28f P 01a67819c8a543adafcf1d5fff0822b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:07:13.076285 22137 consensus_queue.cc:1048] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.033s
I20260709 13:07:13.080065 22103 consensus_queue.cc:1048] T 04e73d74096641eab6b132caefd4b28f P 01a67819c8a543adafcf1d5fff0822b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:07:13.084957 21965 raft_consensus.cc:2468] T 29b15358296749cc9bd2c0a14af3a408 P 07a2cdeeec2b4302a2960391f3825bde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01a67819c8a543adafcf1d5fff0822b4 in term 1.
I20260709 13:07:13.079407 22039 raft_consensus.cc:2468] T 29b15358296749cc9bd2c0a14af3a408 P eaf5735be053455e9315e61c44ff7286 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01a67819c8a543adafcf1d5fff0822b4 in term 1.
I20260709 13:07:13.097029 21785 catalog_manager.cc:5697] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 reported cstate change: term changed from 0 to 1, leader changed from <none> to eaf5735be053455e9315e61c44ff7286 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "eaf5735be053455e9315e61c44ff7286" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:13.090796 21857 leader_election.cc:304] T 29b15358296749cc9bd2c0a14af3a408 P 01a67819c8a543adafcf1d5fff0822b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a67819c8a543adafcf1d5fff0822b4, 07a2cdeeec2b4302a2960391f3825bde; no voters: 
I20260709 13:07:13.100039 22103 raft_consensus.cc:2804] T 29b15358296749cc9bd2c0a14af3a408 P 01a67819c8a543adafcf1d5fff0822b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:13.102911 22103 raft_consensus.cc:697] T 29b15358296749cc9bd2c0a14af3a408 P 01a67819c8a543adafcf1d5fff0822b4 [term 1 LEADER]: Becoming Leader. State: Replica: 01a67819c8a543adafcf1d5fff0822b4, State: Running, Role: LEADER
I20260709 13:07:13.105115 22103 consensus_queue.cc:237] T 29b15358296749cc9bd2c0a14af3a408 P 01a67819c8a543adafcf1d5fff0822b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:13.116266 21787 catalog_manager.cc:5697] T b10d9c93da2c485d96b8ca9058a1b00f P 07a2cdeeec2b4302a2960391f3825bde reported cstate change: term changed from 0 to 1, leader changed from <none> to 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2). New cstate: current_term: 1 leader_uuid: "07a2cdeeec2b4302a2960391f3825bde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:13.121080 21786 catalog_manager.cc:5697] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1). New cstate: current_term: 1 leader_uuid: "01a67819c8a543adafcf1d5fff0822b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:13.131492 21785 catalog_manager.cc:5697] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 reported cstate change: term changed from 0 to 1, leader changed from <none> to eaf5735be053455e9315e61c44ff7286 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "eaf5735be053455e9315e61c44ff7286" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:13.148694 22134 consensus_queue.cc:1048] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:13.174834 22086 consensus_queue.cc:1048] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:07:13.195595 22117 consensus_queue.cc:1048] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:13.185447 21786 catalog_manager.cc:5697] T 29b15358296749cc9bd2c0a14af3a408 P 01a67819c8a543adafcf1d5fff0822b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1). New cstate: current_term: 1 leader_uuid: "01a67819c8a543adafcf1d5fff0822b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:13.234114 22134 consensus_queue.cc:1048] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:13.387135 21966 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2f23dcf5479b46908032cb5858ca9fdf"
dest_uuid: "07a2cdeeec2b4302a2960391f3825bde"
mode: GRACEFUL
new_leader_uuid: "01a67819c8a543adafcf1d5fff0822b4"
 from {username='slave'} at 127.0.0.1:53108
I20260709 13:07:13.387862 21966 raft_consensus.cc:606] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [term 1 LEADER]: Received request to transfer leadership to TS 01a67819c8a543adafcf1d5fff0822b4
I20260709 13:07:13.404317 22038 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bf2b3d48485a40a4a54a0afcc0bd383b"
dest_uuid: "eaf5735be053455e9315e61c44ff7286"
mode: GRACEFUL
new_leader_uuid: "01a67819c8a543adafcf1d5fff0822b4"
 from {username='slave'} at 127.0.0.1:35884
I20260709 13:07:13.404917 22038 raft_consensus.cc:606] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [term 1 LEADER]: Received request to transfer leadership to TS 01a67819c8a543adafcf1d5fff0822b4
I20260709 13:07:13.415014 22038 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4dfeb09c8a4d482ab573952200ebab0a"
dest_uuid: "eaf5735be053455e9315e61c44ff7286"
mode: GRACEFUL
new_leader_uuid: "07a2cdeeec2b4302a2960391f3825bde"
 from {username='slave'} at 127.0.0.1:35884
I20260709 13:07:13.415598 22038 raft_consensus.cc:606] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 [term 1 LEADER]: Received request to transfer leadership to TS 07a2cdeeec2b4302a2960391f3825bde
I20260709 13:07:13.432053 22117 consensus_queue.cc:1048] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:13.441092 21890 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e77b8c40f5024240a36630365043ad11"
dest_uuid: "01a67819c8a543adafcf1d5fff0822b4"
mode: GRACEFUL
new_leader_uuid: "07a2cdeeec2b4302a2960391f3825bde"
 from {username='slave'} at 127.0.0.1:59342
I20260709 13:07:13.441702 21890 raft_consensus.cc:606] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4 [term 1 LEADER]: Received request to transfer leadership to TS 07a2cdeeec2b4302a2960391f3825bde
I20260709 13:07:13.444504 22117 consensus_queue.cc:1048] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:13.446648 22110 consensus_queue.cc:1048] T b10d9c93da2c485d96b8ca9058a1b00f P 07a2cdeeec2b4302a2960391f3825bde [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:13.452590 22039 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4a6231da863b48d2943784b1c955cd09"
dest_uuid: "eaf5735be053455e9315e61c44ff7286"
mode: GRACEFUL
new_leader_uuid: "07a2cdeeec2b4302a2960391f3825bde"
 from {username='slave'} at 127.0.0.1:35884
I20260709 13:07:13.456481 22039 raft_consensus.cc:606] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [term 1 LEADER]: Received request to transfer leadership to TS 07a2cdeeec2b4302a2960391f3825bde
I20260709 13:07:13.457566 22137 consensus_queue.cc:1048] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:13.466063 21891 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "29b15358296749cc9bd2c0a14af3a408"
dest_uuid: "01a67819c8a543adafcf1d5fff0822b4"
mode: GRACEFUL
new_leader_uuid: "eaf5735be053455e9315e61c44ff7286"
 from {username='slave'} at 127.0.0.1:59342
I20260709 13:07:13.466737 21891 raft_consensus.cc:606] T 29b15358296749cc9bd2c0a14af3a408 P 01a67819c8a543adafcf1d5fff0822b4 [term 1 LEADER]: Received request to transfer leadership to TS eaf5735be053455e9315e61c44ff7286
I20260709 13:07:13.469678 22137 raft_consensus.cc:993] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286: : Instructing follower 07a2cdeeec2b4302a2960391f3825bde to start an election
I20260709 13:07:13.470186 22137 raft_consensus.cc:1081] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 [term 1 LEADER]: Signalling peer 07a2cdeeec2b4302a2960391f3825bde to start an election
I20260709 13:07:13.470497 22091 raft_consensus.cc:993] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286: : Instructing follower 07a2cdeeec2b4302a2960391f3825bde to start an election
I20260709 13:07:13.470917 22091 raft_consensus.cc:1081] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [term 1 LEADER]: Signalling peer 07a2cdeeec2b4302a2960391f3825bde to start an election
I20260709 13:07:13.473822 21964 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4dfeb09c8a4d482ab573952200ebab0a"
dest_uuid: "07a2cdeeec2b4302a2960391f3825bde"
 from {username='slave'} at 127.0.0.1:53094
I20260709 13:07:13.474436 21964 raft_consensus.cc:493] T 4dfeb09c8a4d482ab573952200ebab0a P 07a2cdeeec2b4302a2960391f3825bde [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:13.474865 21964 raft_consensus.cc:3060] T 4dfeb09c8a4d482ab573952200ebab0a P 07a2cdeeec2b4302a2960391f3825bde [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:13.475438 22139 consensus_queue.cc:1048] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:13.491571 21964 raft_consensus.cc:515] T 4dfeb09c8a4d482ab573952200ebab0a P 07a2cdeeec2b4302a2960391f3825bde [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:13.497419 21966 raft_consensus.cc:1240] T 4dfeb09c8a4d482ab573952200ebab0a P 07a2cdeeec2b4302a2960391f3825bde [term 2 FOLLOWER]: Rejecting Update request from peer eaf5735be053455e9315e61c44ff7286 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:13.497835 21965 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4a6231da863b48d2943784b1c955cd09"
dest_uuid: "07a2cdeeec2b4302a2960391f3825bde"
 from {username='slave'} at 127.0.0.1:53094
I20260709 13:07:13.498425 21965 raft_consensus.cc:493] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:13.498915 21965 raft_consensus.cc:3060] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:13.503692 21891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dfeb09c8a4d482ab573952200ebab0a" candidate_uuid: "07a2cdeeec2b4302a2960391f3825bde" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "01a67819c8a543adafcf1d5fff0822b4"
I20260709 13:07:13.504071 21964 leader_election.cc:290] T 4dfeb09c8a4d482ab573952200ebab0a P 07a2cdeeec2b4302a2960391f3825bde [CANDIDATE]: Term 2 election: Requested vote from peers 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1:44917), eaf5735be053455e9315e61c44ff7286 (127.21.63.3:41623)
I20260709 13:07:13.504304 21891 raft_consensus.cc:3060] T 4dfeb09c8a4d482ab573952200ebab0a P 01a67819c8a543adafcf1d5fff0822b4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:13.507028 21965 raft_consensus.cc:515] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:13.513095 21891 raft_consensus.cc:2468] T 4dfeb09c8a4d482ab573952200ebab0a P 01a67819c8a543adafcf1d5fff0822b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07a2cdeeec2b4302a2960391f3825bde in term 2.
I20260709 13:07:13.514532 21929 leader_election.cc:304] T 4dfeb09c8a4d482ab573952200ebab0a P 07a2cdeeec2b4302a2960391f3825bde [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a67819c8a543adafcf1d5fff0822b4, 07a2cdeeec2b4302a2960391f3825bde; no voters: 
I20260709 13:07:13.515460 22110 raft_consensus.cc:2804] T 4dfeb09c8a4d482ab573952200ebab0a P 07a2cdeeec2b4302a2960391f3825bde [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:13.515900 22110 raft_consensus.cc:697] T 4dfeb09c8a4d482ab573952200ebab0a P 07a2cdeeec2b4302a2960391f3825bde [term 2 LEADER]: Becoming Leader. State: Replica: 07a2cdeeec2b4302a2960391f3825bde, State: Running, Role: LEADER
I20260709 13:07:13.516778 22110 consensus_queue.cc:237] T 4dfeb09c8a4d482ab573952200ebab0a P 07a2cdeeec2b4302a2960391f3825bde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:13.517922 22139 consensus_queue.cc:1059] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:13.518814 22139 raft_consensus.cc:3055] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:13.519719 22139 raft_consensus.cc:740] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: eaf5735be053455e9315e61c44ff7286, State: Running, Role: LEADER
I20260709 13:07:13.521481 22139 consensus_queue.cc:260] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:13.522972 22139 raft_consensus.cc:3060] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:13.525200 22038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dfeb09c8a4d482ab573952200ebab0a" candidate_uuid: "07a2cdeeec2b4302a2960391f3825bde" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eaf5735be053455e9315e61c44ff7286"
I20260709 13:07:13.530942 22110 consensus_queue.cc:1048] T b10d9c93da2c485d96b8ca9058a1b00f P 07a2cdeeec2b4302a2960391f3825bde [LEADER]: Connected to new peer: Peer: permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:13.531193 21785 catalog_manager.cc:5697] T 4dfeb09c8a4d482ab573952200ebab0a P 07a2cdeeec2b4302a2960391f3825bde reported cstate change: term changed from 1 to 2, leader changed from eaf5735be053455e9315e61c44ff7286 (127.21.63.3) to 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2). New cstate: current_term: 2 leader_uuid: "07a2cdeeec2b4302a2960391f3825bde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:13.533622 22135 consensus_queue.cc:1048] T 29b15358296749cc9bd2c0a14af3a408 P 01a67819c8a543adafcf1d5fff0822b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:13.536314 21891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a6231da863b48d2943784b1c955cd09" candidate_uuid: "07a2cdeeec2b4302a2960391f3825bde" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "01a67819c8a543adafcf1d5fff0822b4"
I20260709 13:07:13.537052 21891 raft_consensus.cc:3060] T 4a6231da863b48d2943784b1c955cd09 P 01a67819c8a543adafcf1d5fff0822b4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:13.547588 22134 consensus_queue.cc:1048] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:13.549466 22134 raft_consensus.cc:993] T 29b15358296749cc9bd2c0a14af3a408 P 01a67819c8a543adafcf1d5fff0822b4: : Instructing follower eaf5735be053455e9315e61c44ff7286 to start an election
I20260709 13:07:13.549970 22134 raft_consensus.cc:1081] T 29b15358296749cc9bd2c0a14af3a408 P 01a67819c8a543adafcf1d5fff0822b4 [term 1 LEADER]: Signalling peer eaf5735be053455e9315e61c44ff7286 to start an election
I20260709 13:07:13.558614 22038 raft_consensus.cc:2468] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07a2cdeeec2b4302a2960391f3825bde in term 2.
I20260709 13:07:13.559303 21965 leader_election.cc:290] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde [CANDIDATE]: Term 2 election: Requested vote from peers 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1:44917), eaf5735be053455e9315e61c44ff7286 (127.21.63.3:41623)
I20260709 13:07:13.560312 22039 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "29b15358296749cc9bd2c0a14af3a408"
dest_uuid: "eaf5735be053455e9315e61c44ff7286"
 from {username='slave'} at 127.0.0.1:35866
I20260709 13:07:13.560904 22039 raft_consensus.cc:493] T 29b15358296749cc9bd2c0a14af3a408 P eaf5735be053455e9315e61c44ff7286 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:13.561269 22039 raft_consensus.cc:3060] T 29b15358296749cc9bd2c0a14af3a408 P eaf5735be053455e9315e61c44ff7286 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:13.566912 22040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a6231da863b48d2943784b1c955cd09" candidate_uuid: "07a2cdeeec2b4302a2960391f3825bde" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eaf5735be053455e9315e61c44ff7286"
I20260709 13:07:13.567603 22040 raft_consensus.cc:3055] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:13.567946 22040 raft_consensus.cc:740] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: eaf5735be053455e9315e61c44ff7286, State: Running, Role: LEADER
I20260709 13:07:13.568819 22040 consensus_queue.cc:260] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:13.569093 22039 raft_consensus.cc:515] T 29b15358296749cc9bd2c0a14af3a408 P eaf5735be053455e9315e61c44ff7286 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:13.570164 22040 raft_consensus.cc:3060] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:13.571221 22039 leader_election.cc:290] T 29b15358296749cc9bd2c0a14af3a408 P eaf5735be053455e9315e61c44ff7286 [CANDIDATE]: Term 2 election: Requested vote from peers 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1:44917), 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2:36631)
I20260709 13:07:13.573837 21890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b15358296749cc9bd2c0a14af3a408" candidate_uuid: "eaf5735be053455e9315e61c44ff7286" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "01a67819c8a543adafcf1d5fff0822b4"
I20260709 13:07:13.574445 21890 raft_consensus.cc:3055] T 29b15358296749cc9bd2c0a14af3a408 P 01a67819c8a543adafcf1d5fff0822b4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:13.574759 21890 raft_consensus.cc:740] T 29b15358296749cc9bd2c0a14af3a408 P 01a67819c8a543adafcf1d5fff0822b4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01a67819c8a543adafcf1d5fff0822b4, State: Running, Role: LEADER
I20260709 13:07:13.575542 21890 consensus_queue.cc:260] T 29b15358296749cc9bd2c0a14af3a408 P 01a67819c8a543adafcf1d5fff0822b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:13.558614 21891 raft_consensus.cc:2468] T 4a6231da863b48d2943784b1c955cd09 P 01a67819c8a543adafcf1d5fff0822b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07a2cdeeec2b4302a2960391f3825bde in term 2.
I20260709 13:07:13.579149 21929 leader_election.cc:304] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a67819c8a543adafcf1d5fff0822b4, 07a2cdeeec2b4302a2960391f3825bde; no voters: 
I20260709 13:07:13.579973 22110 raft_consensus.cc:2804] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:13.580366 22040 raft_consensus.cc:2468] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07a2cdeeec2b4302a2960391f3825bde in term 2.
I20260709 13:07:13.580456 22110 raft_consensus.cc:697] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde [term 2 LEADER]: Becoming Leader. State: Replica: 07a2cdeeec2b4302a2960391f3825bde, State: Running, Role: LEADER
I20260709 13:07:13.576493 21890 raft_consensus.cc:3060] T 29b15358296749cc9bd2c0a14af3a408 P 01a67819c8a543adafcf1d5fff0822b4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:13.582922 21965 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29b15358296749cc9bd2c0a14af3a408" candidate_uuid: "eaf5735be053455e9315e61c44ff7286" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "07a2cdeeec2b4302a2960391f3825bde"
I20260709 13:07:13.585089 22134 consensus_queue.cc:1048] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:13.587601 21965 raft_consensus.cc:3060] T 29b15358296749cc9bd2c0a14af3a408 P 07a2cdeeec2b4302a2960391f3825bde [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:13.591418 22090 raft_consensus.cc:993] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4: : Instructing follower 07a2cdeeec2b4302a2960391f3825bde to start an election
I20260709 13:07:13.591845 22090 raft_consensus.cc:1081] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4 [term 1 LEADER]: Signalling peer 07a2cdeeec2b4302a2960391f3825bde to start an election
I20260709 13:07:13.593235 22110 consensus_queue.cc:237] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:13.594511 21965 raft_consensus.cc:2468] T 29b15358296749cc9bd2c0a14af3a408 P 07a2cdeeec2b4302a2960391f3825bde [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaf5735be053455e9315e61c44ff7286 in term 2.
I20260709 13:07:13.594630 21966 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e77b8c40f5024240a36630365043ad11"
dest_uuid: "07a2cdeeec2b4302a2960391f3825bde"
 from {username='slave'} at 127.0.0.1:53080
I20260709 13:07:13.595186 21966 raft_consensus.cc:493] T e77b8c40f5024240a36630365043ad11 P 07a2cdeeec2b4302a2960391f3825bde [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:13.595589 21966 raft_consensus.cc:3060] T e77b8c40f5024240a36630365043ad11 P 07a2cdeeec2b4302a2960391f3825bde [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:13.597509 21890 raft_consensus.cc:2468] T 29b15358296749cc9bd2c0a14af3a408 P 01a67819c8a543adafcf1d5fff0822b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaf5735be053455e9315e61c44ff7286 in term 2.
I20260709 13:07:13.599345 22003 leader_election.cc:304] T 29b15358296749cc9bd2c0a14af3a408 P eaf5735be053455e9315e61c44ff7286 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a67819c8a543adafcf1d5fff0822b4, eaf5735be053455e9315e61c44ff7286; no voters: 
I20260709 13:07:13.600214 22139 raft_consensus.cc:2804] T 29b15358296749cc9bd2c0a14af3a408 P eaf5735be053455e9315e61c44ff7286 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:13.600677 22139 raft_consensus.cc:697] T 29b15358296749cc9bd2c0a14af3a408 P eaf5735be053455e9315e61c44ff7286 [term 2 LEADER]: Becoming Leader. State: Replica: eaf5735be053455e9315e61c44ff7286, State: Running, Role: LEADER
I20260709 13:07:13.601851 22139 consensus_queue.cc:237] T 29b15358296749cc9bd2c0a14af3a408 P eaf5735be053455e9315e61c44ff7286 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:13.603106 21966 raft_consensus.cc:515] T e77b8c40f5024240a36630365043ad11 P 07a2cdeeec2b4302a2960391f3825bde [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:13.604238 21964 raft_consensus.cc:1240] T e77b8c40f5024240a36630365043ad11 P 07a2cdeeec2b4302a2960391f3825bde [term 2 FOLLOWER]: Rejecting Update request from peer 01a67819c8a543adafcf1d5fff0822b4 for earlier term 1. Current term is 2. Ops: []
I20260709 13:07:13.610344 22090 consensus_queue.cc:1059] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:13.611218 22103 raft_consensus.cc:3055] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:13.611629 22103 raft_consensus.cc:740] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01a67819c8a543adafcf1d5fff0822b4, State: Running, Role: LEADER
I20260709 13:07:13.612407 22103 consensus_queue.cc:260] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:13.613471 22103 raft_consensus.cc:3060] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:13.614146 21966 leader_election.cc:290] T e77b8c40f5024240a36630365043ad11 P 07a2cdeeec2b4302a2960391f3825bde [CANDIDATE]: Term 2 election: Requested vote from peers 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1:44917), eaf5735be053455e9315e61c44ff7286 (127.21.63.3:41623)
I20260709 13:07:13.617938 21890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b8c40f5024240a36630365043ad11" candidate_uuid: "07a2cdeeec2b4302a2960391f3825bde" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "01a67819c8a543adafcf1d5fff0822b4"
I20260709 13:07:13.619249 21786 catalog_manager.cc:5697] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde reported cstate change: term changed from 1 to 2, leader changed from eaf5735be053455e9315e61c44ff7286 (127.21.63.3) to 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2). New cstate: current_term: 2 leader_uuid: "07a2cdeeec2b4302a2960391f3825bde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:13.619346 21785 catalog_manager.cc:5697] T 29b15358296749cc9bd2c0a14af3a408 P eaf5735be053455e9315e61c44ff7286 reported cstate change: term changed from 1 to 2, leader changed from 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1) to eaf5735be053455e9315e61c44ff7286 (127.21.63.3). New cstate: current_term: 2 leader_uuid: "eaf5735be053455e9315e61c44ff7286" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:13.623224 22040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b8c40f5024240a36630365043ad11" candidate_uuid: "07a2cdeeec2b4302a2960391f3825bde" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eaf5735be053455e9315e61c44ff7286"
I20260709 13:07:13.623826 22040 raft_consensus.cc:3060] T e77b8c40f5024240a36630365043ad11 P eaf5735be053455e9315e61c44ff7286 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:13.628409 21890 raft_consensus.cc:2468] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07a2cdeeec2b4302a2960391f3825bde in term 2.
I20260709 13:07:13.629988 21929 leader_election.cc:304] T e77b8c40f5024240a36630365043ad11 P 07a2cdeeec2b4302a2960391f3825bde [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a67819c8a543adafcf1d5fff0822b4, 07a2cdeeec2b4302a2960391f3825bde; no voters: 
I20260709 13:07:13.630358 22040 raft_consensus.cc:2468] T e77b8c40f5024240a36630365043ad11 P eaf5735be053455e9315e61c44ff7286 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07a2cdeeec2b4302a2960391f3825bde in term 2.
I20260709 13:07:13.630810 22110 raft_consensus.cc:2804] T e77b8c40f5024240a36630365043ad11 P 07a2cdeeec2b4302a2960391f3825bde [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:13.631304 22110 raft_consensus.cc:697] T e77b8c40f5024240a36630365043ad11 P 07a2cdeeec2b4302a2960391f3825bde [term 2 LEADER]: Becoming Leader. State: Replica: 07a2cdeeec2b4302a2960391f3825bde, State: Running, Role: LEADER
I20260709 13:07:13.632125 22110 consensus_queue.cc:237] T e77b8c40f5024240a36630365043ad11 P 07a2cdeeec2b4302a2960391f3825bde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:13.643381 21785 catalog_manager.cc:5697] T e77b8c40f5024240a36630365043ad11 P 07a2cdeeec2b4302a2960391f3825bde reported cstate change: term changed from 1 to 2, leader changed from 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1) to 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2). New cstate: current_term: 2 leader_uuid: "07a2cdeeec2b4302a2960391f3825bde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:13.715507 22110 raft_consensus.cc:993] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde: : Instructing follower 01a67819c8a543adafcf1d5fff0822b4 to start an election
I20260709 13:07:13.716145 22166 raft_consensus.cc:1081] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [term 1 LEADER]: Signalling peer 01a67819c8a543adafcf1d5fff0822b4 to start an election
I20260709 13:07:13.718153 21890 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2f23dcf5479b46908032cb5858ca9fdf"
dest_uuid: "01a67819c8a543adafcf1d5fff0822b4"
 from {username='slave'} at 127.0.0.1:59322
I20260709 13:07:13.718686 21890 raft_consensus.cc:493] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:13.718999 21890 raft_consensus.cc:3060] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:13.723418 21890 raft_consensus.cc:515] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:13.725224 21890 leader_election.cc:290] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4 [CANDIDATE]: Term 2 election: Requested vote from peers 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2:36631), eaf5735be053455e9315e61c44ff7286 (127.21.63.3:41623)
I20260709 13:07:13.726017 21966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f23dcf5479b46908032cb5858ca9fdf" candidate_uuid: "01a67819c8a543adafcf1d5fff0822b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "07a2cdeeec2b4302a2960391f3825bde"
I20260709 13:07:13.726320 22040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f23dcf5479b46908032cb5858ca9fdf" candidate_uuid: "01a67819c8a543adafcf1d5fff0822b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eaf5735be053455e9315e61c44ff7286"
I20260709 13:07:13.726778 21966 raft_consensus.cc:3055] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:13.726997 22040 raft_consensus.cc:3060] T 2f23dcf5479b46908032cb5858ca9fdf P eaf5735be053455e9315e61c44ff7286 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:13.727195 21966 raft_consensus.cc:740] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 07a2cdeeec2b4302a2960391f3825bde, State: Running, Role: LEADER
I20260709 13:07:13.728130 21966 consensus_queue.cc:260] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:13.729419 21966 raft_consensus.cc:3060] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:13.732280 22040 raft_consensus.cc:2468] T 2f23dcf5479b46908032cb5858ca9fdf P eaf5735be053455e9315e61c44ff7286 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01a67819c8a543adafcf1d5fff0822b4 in term 2.
I20260709 13:07:13.733516 21854 leader_election.cc:304] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a67819c8a543adafcf1d5fff0822b4, eaf5735be053455e9315e61c44ff7286; no voters: 
I20260709 13:07:13.734335 22103 raft_consensus.cc:2804] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:13.734907 22103 raft_consensus.cc:697] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4 [term 2 LEADER]: Becoming Leader. State: Replica: 01a67819c8a543adafcf1d5fff0822b4, State: Running, Role: LEADER
I20260709 13:07:13.736268 21966 raft_consensus.cc:2468] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01a67819c8a543adafcf1d5fff0822b4 in term 2.
I20260709 13:07:13.736009 22103 consensus_queue.cc:237] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:13.742914 21785 catalog_manager.cc:5697] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4 reported cstate change: term changed from 1 to 2, leader changed from 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2) to 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1). New cstate: current_term: 2 leader_uuid: "01a67819c8a543adafcf1d5fff0822b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:13.783836 22117 raft_consensus.cc:993] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286: : Instructing follower 01a67819c8a543adafcf1d5fff0822b4 to start an election
I20260709 13:07:13.784354 22139 raft_consensus.cc:1081] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [term 1 LEADER]: Signalling peer 01a67819c8a543adafcf1d5fff0822b4 to start an election
I20260709 13:07:13.785965 21890 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bf2b3d48485a40a4a54a0afcc0bd383b"
dest_uuid: "01a67819c8a543adafcf1d5fff0822b4"
 from {username='slave'} at 127.0.0.1:59338
I20260709 13:07:13.786475 21890 raft_consensus.cc:493] T bf2b3d48485a40a4a54a0afcc0bd383b P 01a67819c8a543adafcf1d5fff0822b4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:13.786827 21890 raft_consensus.cc:3060] T bf2b3d48485a40a4a54a0afcc0bd383b P 01a67819c8a543adafcf1d5fff0822b4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:13.791823 21890 raft_consensus.cc:515] T bf2b3d48485a40a4a54a0afcc0bd383b P 01a67819c8a543adafcf1d5fff0822b4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:13.793601 21890 leader_election.cc:290] T bf2b3d48485a40a4a54a0afcc0bd383b P 01a67819c8a543adafcf1d5fff0822b4 [CANDIDATE]: Term 2 election: Requested vote from peers 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2:36631), eaf5735be053455e9315e61c44ff7286 (127.21.63.3:41623)
I20260709 13:07:13.794529 21966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf2b3d48485a40a4a54a0afcc0bd383b" candidate_uuid: "01a67819c8a543adafcf1d5fff0822b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "07a2cdeeec2b4302a2960391f3825bde"
I20260709 13:07:13.794660 22040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf2b3d48485a40a4a54a0afcc0bd383b" candidate_uuid: "01a67819c8a543adafcf1d5fff0822b4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eaf5735be053455e9315e61c44ff7286"
I20260709 13:07:13.795130 21966 raft_consensus.cc:3060] T bf2b3d48485a40a4a54a0afcc0bd383b P 07a2cdeeec2b4302a2960391f3825bde [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:13.795306 22040 raft_consensus.cc:3055] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:13.795786 22040 raft_consensus.cc:740] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: eaf5735be053455e9315e61c44ff7286, State: Running, Role: LEADER
I20260709 13:07:13.796547 22040 consensus_queue.cc:260] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:13.797685 22040 raft_consensus.cc:3060] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:13.801851 21966 raft_consensus.cc:2468] T bf2b3d48485a40a4a54a0afcc0bd383b P 07a2cdeeec2b4302a2960391f3825bde [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01a67819c8a543adafcf1d5fff0822b4 in term 2.
I20260709 13:07:13.803176 21857 leader_election.cc:304] T bf2b3d48485a40a4a54a0afcc0bd383b P 01a67819c8a543adafcf1d5fff0822b4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a67819c8a543adafcf1d5fff0822b4, 07a2cdeeec2b4302a2960391f3825bde; no voters: 
I20260709 13:07:13.803225 22040 raft_consensus.cc:2468] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01a67819c8a543adafcf1d5fff0822b4 in term 2.
I20260709 13:07:13.804098 22103 raft_consensus.cc:2804] T bf2b3d48485a40a4a54a0afcc0bd383b P 01a67819c8a543adafcf1d5fff0822b4 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:13.804594 22103 raft_consensus.cc:697] T bf2b3d48485a40a4a54a0afcc0bd383b P 01a67819c8a543adafcf1d5fff0822b4 [term 2 LEADER]: Becoming Leader. State: Replica: 01a67819c8a543adafcf1d5fff0822b4, State: Running, Role: LEADER
I20260709 13:07:13.805516 22103 consensus_queue.cc:237] T bf2b3d48485a40a4a54a0afcc0bd383b P 01a67819c8a543adafcf1d5fff0822b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:13.812445 21785 catalog_manager.cc:5697] T bf2b3d48485a40a4a54a0afcc0bd383b P 01a67819c8a543adafcf1d5fff0822b4 reported cstate change: term changed from 1 to 2, leader changed from eaf5735be053455e9315e61c44ff7286 (127.21.63.3) to 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1). New cstate: current_term: 2 leader_uuid: "01a67819c8a543adafcf1d5fff0822b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:13.982739 21890 raft_consensus.cc:1275] T 4a6231da863b48d2943784b1c955cd09 P 01a67819c8a543adafcf1d5fff0822b4 [term 2 FOLLOWER]: Refusing update from remote peer 07a2cdeeec2b4302a2960391f3825bde: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:13.983338 21891 raft_consensus.cc:1275] T 29b15358296749cc9bd2c0a14af3a408 P 01a67819c8a543adafcf1d5fff0822b4 [term 2 FOLLOWER]: Refusing update from remote peer eaf5735be053455e9315e61c44ff7286: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:13.984217 22166 consensus_queue.cc:1048] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde [LEADER]: Connected to new peer: Peer: permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:13.985071 22139 consensus_queue.cc:1048] T 29b15358296749cc9bd2c0a14af3a408 P eaf5735be053455e9315e61c44ff7286 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:13.998975 21966 raft_consensus.cc:1275] T 29b15358296749cc9bd2c0a14af3a408 P 07a2cdeeec2b4302a2960391f3825bde [term 2 FOLLOWER]: Refusing update from remote peer eaf5735be053455e9315e61c44ff7286: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:14.000079 22040 raft_consensus.cc:1275] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [term 2 FOLLOWER]: Refusing update from remote peer 07a2cdeeec2b4302a2960391f3825bde: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:14.000588 22139 consensus_queue.cc:1048] T 29b15358296749cc9bd2c0a14af3a408 P eaf5735be053455e9315e61c44ff7286 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:14.005569 22086 consensus_queue.cc:1048] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:14.023772 22038 raft_consensus.cc:1275] T e77b8c40f5024240a36630365043ad11 P eaf5735be053455e9315e61c44ff7286 [term 2 FOLLOWER]: Refusing update from remote peer 07a2cdeeec2b4302a2960391f3825bde: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:14.027954 22166 consensus_queue.cc:1048] T e77b8c40f5024240a36630365043ad11 P 07a2cdeeec2b4302a2960391f3825bde [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:14.040426 22040 raft_consensus.cc:1275] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 [term 2 FOLLOWER]: Refusing update from remote peer 07a2cdeeec2b4302a2960391f3825bde: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:14.043227 21891 raft_consensus.cc:1275] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4 [term 2 FOLLOWER]: Refusing update from remote peer 07a2cdeeec2b4302a2960391f3825bde: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:14.045044 22086 consensus_queue.cc:1048] T e77b8c40f5024240a36630365043ad11 P 07a2cdeeec2b4302a2960391f3825bde [LEADER]: Connected to new peer: Peer: permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:14.045565 22166 consensus_queue.cc:1048] T 4dfeb09c8a4d482ab573952200ebab0a P 07a2cdeeec2b4302a2960391f3825bde [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:14.048130 21890 raft_consensus.cc:1275] T 4dfeb09c8a4d482ab573952200ebab0a P 01a67819c8a543adafcf1d5fff0822b4 [term 2 FOLLOWER]: Refusing update from remote peer 07a2cdeeec2b4302a2960391f3825bde: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:14.056492 22086 consensus_queue.cc:1048] T 4dfeb09c8a4d482ab573952200ebab0a P 07a2cdeeec2b4302a2960391f3825bde [LEADER]: Connected to new peer: Peer: permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:14.267134 22038 raft_consensus.cc:1275] T 2f23dcf5479b46908032cb5858ca9fdf P eaf5735be053455e9315e61c44ff7286 [term 2 FOLLOWER]: Refusing update from remote peer 01a67819c8a543adafcf1d5fff0822b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:14.269042 22103 consensus_queue.cc:1048] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:14.273720 22038 raft_consensus.cc:1275] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [term 2 FOLLOWER]: Refusing update from remote peer 01a67819c8a543adafcf1d5fff0822b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:14.275242 22103 consensus_queue.cc:1048] T bf2b3d48485a40a4a54a0afcc0bd383b P 01a67819c8a543adafcf1d5fff0822b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:14.285352 21966 raft_consensus.cc:1275] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [term 2 FOLLOWER]: Refusing update from remote peer 01a67819c8a543adafcf1d5fff0822b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:14.290341 22134 consensus_queue.cc:1048] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:14.297559 21964 raft_consensus.cc:1275] T bf2b3d48485a40a4a54a0afcc0bd383b P 07a2cdeeec2b4302a2960391f3825bde [term 2 FOLLOWER]: Refusing update from remote peer 01a67819c8a543adafcf1d5fff0822b4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:14.299374 22134 consensus_queue.cc:1048] T bf2b3d48485a40a4a54a0afcc0bd383b P 01a67819c8a543adafcf1d5fff0822b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:16.499238 21756 auto_leader_rebalancer-test.cc:316] The leader distribution is 
I20260709 13:07:16.500192 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:16.521699 21966 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4a6231da863b48d2943784b1c955cd09"
dest_uuid: "07a2cdeeec2b4302a2960391f3825bde"
mode: GRACEFUL
new_leader_uuid: "eaf5735be053455e9315e61c44ff7286"
 from {username='slave'} at 127.0.0.1:53122
I20260709 13:07:16.522289 21966 raft_consensus.cc:606] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde [term 2 LEADER]: Received request to transfer leadership to TS eaf5735be053455e9315e61c44ff7286
I20260709 13:07:16.533753 21891 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bf2b3d48485a40a4a54a0afcc0bd383b"
dest_uuid: "01a67819c8a543adafcf1d5fff0822b4"
mode: GRACEFUL
new_leader_uuid: "eaf5735be053455e9315e61c44ff7286"
 from {username='slave'} at 127.0.0.1:59348
I20260709 13:07:16.534406 21891 raft_consensus.cc:606] T bf2b3d48485a40a4a54a0afcc0bd383b P 01a67819c8a543adafcf1d5fff0822b4 [term 2 LEADER]: Received request to transfer leadership to TS eaf5735be053455e9315e61c44ff7286
I20260709 13:07:16.535908 21756 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:07:16.536402 21756 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:16.670887 22183 raft_consensus.cc:993] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde: : Instructing follower eaf5735be053455e9315e61c44ff7286 to start an election
I20260709 13:07:16.671492 22188 raft_consensus.cc:1081] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde [term 2 LEADER]: Signalling peer eaf5735be053455e9315e61c44ff7286 to start an election
I20260709 13:07:16.673185 22038 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4a6231da863b48d2943784b1c955cd09"
dest_uuid: "eaf5735be053455e9315e61c44ff7286"
 from {username='slave'} at 127.0.0.1:35880
I20260709 13:07:16.673800 22038 raft_consensus.cc:493] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:16.674211 22038 raft_consensus.cc:3060] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:16.680941 22038 raft_consensus.cc:515] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:16.682957 22038 leader_election.cc:290] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [CANDIDATE]: Term 3 election: Requested vote from peers 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1:44917), 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2:36631)
I20260709 13:07:16.683755 21891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a6231da863b48d2943784b1c955cd09" candidate_uuid: "eaf5735be053455e9315e61c44ff7286" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "01a67819c8a543adafcf1d5fff0822b4"
I20260709 13:07:16.683976 21966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a6231da863b48d2943784b1c955cd09" candidate_uuid: "eaf5735be053455e9315e61c44ff7286" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "07a2cdeeec2b4302a2960391f3825bde"
I20260709 13:07:16.684424 21891 raft_consensus.cc:3060] T 4a6231da863b48d2943784b1c955cd09 P 01a67819c8a543adafcf1d5fff0822b4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:16.684624 21966 raft_consensus.cc:3055] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:16.685060 21966 raft_consensus.cc:740] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 07a2cdeeec2b4302a2960391f3825bde, State: Running, Role: LEADER
I20260709 13:07:16.685916 21966 consensus_queue.cc:260] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:16.687191 21966 raft_consensus.cc:3060] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:16.691727 21891 raft_consensus.cc:2468] T 4a6231da863b48d2943784b1c955cd09 P 01a67819c8a543adafcf1d5fff0822b4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaf5735be053455e9315e61c44ff7286 in term 3.
I20260709 13:07:16.693158 22003 leader_election.cc:304] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a67819c8a543adafcf1d5fff0822b4, eaf5735be053455e9315e61c44ff7286; no voters: 
I20260709 13:07:16.693938 22180 raft_consensus.cc:2804] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:16.693964 21966 raft_consensus.cc:2468] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaf5735be053455e9315e61c44ff7286 in term 3.
I20260709 13:07:16.695314 22180 raft_consensus.cc:697] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [term 3 LEADER]: Becoming Leader. State: Replica: eaf5735be053455e9315e61c44ff7286, State: Running, Role: LEADER
I20260709 13:07:16.696125 22180 consensus_queue.cc:237] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:16.703253 21787 catalog_manager.cc:5697] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 reported cstate change: term changed from 2 to 3, leader changed from 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2) to eaf5735be053455e9315e61c44ff7286 (127.21.63.3). New cstate: current_term: 3 leader_uuid: "eaf5735be053455e9315e61c44ff7286" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:16.791545 22184 raft_consensus.cc:993] T bf2b3d48485a40a4a54a0afcc0bd383b P 01a67819c8a543adafcf1d5fff0822b4: : Instructing follower eaf5735be053455e9315e61c44ff7286 to start an election
I20260709 13:07:16.792037 22195 raft_consensus.cc:1081] T bf2b3d48485a40a4a54a0afcc0bd383b P 01a67819c8a543adafcf1d5fff0822b4 [term 2 LEADER]: Signalling peer eaf5735be053455e9315e61c44ff7286 to start an election
I20260709 13:07:16.793663 22038 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bf2b3d48485a40a4a54a0afcc0bd383b"
dest_uuid: "eaf5735be053455e9315e61c44ff7286"
 from {username='slave'} at 127.0.0.1:35866
I20260709 13:07:16.794158 22038 raft_consensus.cc:493] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:16.794448 22038 raft_consensus.cc:3060] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:16.799220 22038 raft_consensus.cc:515] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:16.800925 22038 leader_election.cc:290] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [CANDIDATE]: Term 3 election: Requested vote from peers 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1:44917), 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2:36631)
I20260709 13:07:16.801789 21891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf2b3d48485a40a4a54a0afcc0bd383b" candidate_uuid: "eaf5735be053455e9315e61c44ff7286" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "01a67819c8a543adafcf1d5fff0822b4"
I20260709 13:07:16.802151 21966 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf2b3d48485a40a4a54a0afcc0bd383b" candidate_uuid: "eaf5735be053455e9315e61c44ff7286" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "07a2cdeeec2b4302a2960391f3825bde"
I20260709 13:07:16.802953 21966 raft_consensus.cc:3060] T bf2b3d48485a40a4a54a0afcc0bd383b P 07a2cdeeec2b4302a2960391f3825bde [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:16.802546 21891 raft_consensus.cc:3055] T bf2b3d48485a40a4a54a0afcc0bd383b P 01a67819c8a543adafcf1d5fff0822b4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:16.807356 21891 raft_consensus.cc:740] T bf2b3d48485a40a4a54a0afcc0bd383b P 01a67819c8a543adafcf1d5fff0822b4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 01a67819c8a543adafcf1d5fff0822b4, State: Running, Role: LEADER
I20260709 13:07:16.808717 21891 consensus_queue.cc:260] T bf2b3d48485a40a4a54a0afcc0bd383b P 01a67819c8a543adafcf1d5fff0822b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:16.809801 21891 raft_consensus.cc:3060] T bf2b3d48485a40a4a54a0afcc0bd383b P 01a67819c8a543adafcf1d5fff0822b4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:16.810853 21966 raft_consensus.cc:2468] T bf2b3d48485a40a4a54a0afcc0bd383b P 07a2cdeeec2b4302a2960391f3825bde [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaf5735be053455e9315e61c44ff7286 in term 3.
I20260709 13:07:16.812108 22006 leader_election.cc:304] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07a2cdeeec2b4302a2960391f3825bde, eaf5735be053455e9315e61c44ff7286; no voters: 
I20260709 13:07:16.812902 22180 raft_consensus.cc:2804] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:16.813359 22180 raft_consensus.cc:697] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [term 3 LEADER]: Becoming Leader. State: Replica: eaf5735be053455e9315e61c44ff7286, State: Running, Role: LEADER
I20260709 13:07:16.814147 22180 consensus_queue.cc:237] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:16.814790 21891 raft_consensus.cc:2468] T bf2b3d48485a40a4a54a0afcc0bd383b P 01a67819c8a543adafcf1d5fff0822b4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaf5735be053455e9315e61c44ff7286 in term 3.
I20260709 13:07:16.823534 21787 catalog_manager.cc:5697] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 reported cstate change: term changed from 2 to 3, leader changed from 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1) to eaf5735be053455e9315e61c44ff7286 (127.21.63.3). New cstate: current_term: 3 leader_uuid: "eaf5735be053455e9315e61c44ff7286" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:17.197888 21891 raft_consensus.cc:1275] T bf2b3d48485a40a4a54a0afcc0bd383b P 01a67819c8a543adafcf1d5fff0822b4 [term 3 FOLLOWER]: Refusing update from remote peer eaf5735be053455e9315e61c44ff7286: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:17.199139 22180 consensus_queue.cc:1048] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:17.209936 21966 raft_consensus.cc:1275] T bf2b3d48485a40a4a54a0afcc0bd383b P 07a2cdeeec2b4302a2960391f3825bde [term 3 FOLLOWER]: Refusing update from remote peer eaf5735be053455e9315e61c44ff7286: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:17.211583 22189 consensus_queue.cc:1048] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:17.221798 21891 raft_consensus.cc:1275] T 4a6231da863b48d2943784b1c955cd09 P 01a67819c8a543adafcf1d5fff0822b4 [term 3 FOLLOWER]: Refusing update from remote peer eaf5735be053455e9315e61c44ff7286: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:17.223135 22189 consensus_queue.cc:1048] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:17.234898 21966 raft_consensus.cc:1275] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde [term 3 FOLLOWER]: Refusing update from remote peer eaf5735be053455e9315e61c44ff7286: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:17.236205 22199 consensus_queue.cc:1048] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:17.537338 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:17.548162 21891 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "04e73d74096641eab6b132caefd4b28f"
dest_uuid: "01a67819c8a543adafcf1d5fff0822b4"
mode: GRACEFUL
new_leader_uuid: "07a2cdeeec2b4302a2960391f3825bde"
 from {username='slave'} at 127.0.0.1:59342
I20260709 13:07:17.548848 21891 raft_consensus.cc:606] T 04e73d74096641eab6b132caefd4b28f P 01a67819c8a543adafcf1d5fff0822b4 [term 1 LEADER]: Received request to transfer leadership to TS 07a2cdeeec2b4302a2960391f3825bde
I20260709 13:07:17.554668 21891 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2f23dcf5479b46908032cb5858ca9fdf"
dest_uuid: "01a67819c8a543adafcf1d5fff0822b4"
mode: GRACEFUL
new_leader_uuid: "07a2cdeeec2b4302a2960391f3825bde"
 from {username='slave'} at 127.0.0.1:59342
I20260709 13:07:17.555241 21891 raft_consensus.cc:606] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4 [term 2 LEADER]: Received request to transfer leadership to TS 07a2cdeeec2b4302a2960391f3825bde
I20260709 13:07:17.560689 22038 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bf2b3d48485a40a4a54a0afcc0bd383b"
dest_uuid: "eaf5735be053455e9315e61c44ff7286"
mode: GRACEFUL
new_leader_uuid: "07a2cdeeec2b4302a2960391f3825bde"
 from {username='slave'} at 127.0.0.1:35884
I20260709 13:07:17.561206 22038 raft_consensus.cc:606] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [term 3 LEADER]: Received request to transfer leadership to TS 07a2cdeeec2b4302a2960391f3825bde
I20260709 13:07:17.566234 21891 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4c1dcbc1199442dd881d67c582108673"
dest_uuid: "01a67819c8a543adafcf1d5fff0822b4"
mode: GRACEFUL
new_leader_uuid: "07a2cdeeec2b4302a2960391f3825bde"
 from {username='slave'} at 127.0.0.1:59342
I20260709 13:07:17.566839 21891 raft_consensus.cc:606] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4 [term 1 LEADER]: Received request to transfer leadership to TS 07a2cdeeec2b4302a2960391f3825bde
I20260709 13:07:17.579722 22038 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4a6231da863b48d2943784b1c955cd09"
dest_uuid: "eaf5735be053455e9315e61c44ff7286"
mode: GRACEFUL
new_leader_uuid: "07a2cdeeec2b4302a2960391f3825bde"
 from {username='slave'} at 127.0.0.1:35884
I20260709 13:07:17.580298 22038 raft_consensus.cc:606] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [term 3 LEADER]: Received request to transfer leadership to TS 07a2cdeeec2b4302a2960391f3825bde
I20260709 13:07:17.630298 22195 raft_consensus.cc:993] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4: : Instructing follower 07a2cdeeec2b4302a2960391f3825bde to start an election
I20260709 13:07:17.630759 22195 raft_consensus.cc:1081] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4 [term 2 LEADER]: Signalling peer 07a2cdeeec2b4302a2960391f3825bde to start an election
I20260709 13:07:17.632498 21966 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2f23dcf5479b46908032cb5858ca9fdf"
dest_uuid: "07a2cdeeec2b4302a2960391f3825bde"
 from {username='slave'} at 127.0.0.1:53080
I20260709 13:07:17.632997 21966 raft_consensus.cc:493] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:17.633286 21966 raft_consensus.cc:3060] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:17.637255 21966 raft_consensus.cc:515] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:17.638721 21966 leader_election.cc:290] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [CANDIDATE]: Term 3 election: Requested vote from peers 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1:44917), eaf5735be053455e9315e61c44ff7286 (127.21.63.3:41623)
I20260709 13:07:17.639604 21891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f23dcf5479b46908032cb5858ca9fdf" candidate_uuid: "07a2cdeeec2b4302a2960391f3825bde" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "01a67819c8a543adafcf1d5fff0822b4"
I20260709 13:07:17.639880 22038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f23dcf5479b46908032cb5858ca9fdf" candidate_uuid: "07a2cdeeec2b4302a2960391f3825bde" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "eaf5735be053455e9315e61c44ff7286"
I20260709 13:07:17.640197 21891 raft_consensus.cc:3055] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:17.640395 22038 raft_consensus.cc:3060] T 2f23dcf5479b46908032cb5858ca9fdf P eaf5735be053455e9315e61c44ff7286 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:17.640542 21891 raft_consensus.cc:740] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 01a67819c8a543adafcf1d5fff0822b4, State: Running, Role: LEADER
I20260709 13:07:17.641408 21891 consensus_queue.cc:260] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:17.642591 21891 raft_consensus.cc:3060] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:17.647042 22199 raft_consensus.cc:993] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286: : Instructing follower 07a2cdeeec2b4302a2960391f3825bde to start an election
I20260709 13:07:17.647238 22038 raft_consensus.cc:2468] T 2f23dcf5479b46908032cb5858ca9fdf P eaf5735be053455e9315e61c44ff7286 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07a2cdeeec2b4302a2960391f3825bde in term 3.
I20260709 13:07:17.647585 22199 raft_consensus.cc:1081] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [term 3 LEADER]: Signalling peer 07a2cdeeec2b4302a2960391f3825bde to start an election
I20260709 13:07:17.649322 21891 raft_consensus.cc:2468] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07a2cdeeec2b4302a2960391f3825bde in term 3.
I20260709 13:07:17.649652 21966 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4a6231da863b48d2943784b1c955cd09"
dest_uuid: "07a2cdeeec2b4302a2960391f3825bde"
 from {username='slave'} at 127.0.0.1:53094
I20260709 13:07:17.649566 21929 leader_election.cc:304] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07a2cdeeec2b4302a2960391f3825bde, eaf5735be053455e9315e61c44ff7286; no voters: 
I20260709 13:07:17.650265 21966 raft_consensus.cc:493] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:17.650642 22188 raft_consensus.cc:2804] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:17.650676 21966 raft_consensus.cc:3060] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:17.652268 22188 raft_consensus.cc:697] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [term 3 LEADER]: Becoming Leader. State: Replica: 07a2cdeeec2b4302a2960391f3825bde, State: Running, Role: LEADER
I20260709 13:07:17.653529 22188 consensus_queue.cc:237] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:17.655887 21966 raft_consensus.cc:515] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:17.657771 21966 leader_election.cc:290] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde [CANDIDATE]: Term 4 election: Requested vote from peers 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1:44917), eaf5735be053455e9315e61c44ff7286 (127.21.63.3:41623)
I20260709 13:07:17.658845 22038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a6231da863b48d2943784b1c955cd09" candidate_uuid: "07a2cdeeec2b4302a2960391f3825bde" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "eaf5735be053455e9315e61c44ff7286"
I20260709 13:07:17.659333 22038 raft_consensus.cc:3055] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:17.659552 22038 raft_consensus.cc:740] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: eaf5735be053455e9315e61c44ff7286, State: Running, Role: LEADER
I20260709 13:07:17.660204 22038 consensus_queue.cc:260] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:17.659972 21787 catalog_manager.cc:5697] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde reported cstate change: term changed from 2 to 3, leader changed from 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1) to 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2). New cstate: current_term: 3 leader_uuid: "07a2cdeeec2b4302a2960391f3825bde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:17.661269 22038 raft_consensus.cc:3060] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:17.665117 21891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4a6231da863b48d2943784b1c955cd09" candidate_uuid: "07a2cdeeec2b4302a2960391f3825bde" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "01a67819c8a543adafcf1d5fff0822b4"
I20260709 13:07:17.665897 21891 raft_consensus.cc:3060] T 4a6231da863b48d2943784b1c955cd09 P 01a67819c8a543adafcf1d5fff0822b4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:17.670871 22038 raft_consensus.cc:2468] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07a2cdeeec2b4302a2960391f3825bde in term 4.
I20260709 13:07:17.672187 21929 leader_election.cc:304] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07a2cdeeec2b4302a2960391f3825bde, eaf5735be053455e9315e61c44ff7286; no voters: 
I20260709 13:07:17.673162 22188 raft_consensus.cc:2804] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:17.673224 21891 raft_consensus.cc:2468] T 4a6231da863b48d2943784b1c955cd09 P 01a67819c8a543adafcf1d5fff0822b4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07a2cdeeec2b4302a2960391f3825bde in term 4.
I20260709 13:07:17.673707 22188 raft_consensus.cc:697] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde [term 4 LEADER]: Becoming Leader. State: Replica: 07a2cdeeec2b4302a2960391f3825bde, State: Running, Role: LEADER
I20260709 13:07:17.674638 22188 consensus_queue.cc:237] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:17.681348 21787 catalog_manager.cc:5697] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde reported cstate change: term changed from 3 to 4, leader changed from eaf5735be053455e9315e61c44ff7286 (127.21.63.3) to 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2). New cstate: current_term: 4 leader_uuid: "07a2cdeeec2b4302a2960391f3825bde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:17.834494 22180 raft_consensus.cc:993] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286: : Instructing follower 07a2cdeeec2b4302a2960391f3825bde to start an election
I20260709 13:07:17.834966 22189 raft_consensus.cc:1081] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [term 3 LEADER]: Signalling peer 07a2cdeeec2b4302a2960391f3825bde to start an election
I20260709 13:07:17.836326 21966 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bf2b3d48485a40a4a54a0afcc0bd383b"
dest_uuid: "07a2cdeeec2b4302a2960391f3825bde"
 from {username='slave'} at 127.0.0.1:53094
I20260709 13:07:17.836795 21966 raft_consensus.cc:493] T bf2b3d48485a40a4a54a0afcc0bd383b P 07a2cdeeec2b4302a2960391f3825bde [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:17.837114 21966 raft_consensus.cc:3060] T bf2b3d48485a40a4a54a0afcc0bd383b P 07a2cdeeec2b4302a2960391f3825bde [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:17.841578 21966 raft_consensus.cc:515] T bf2b3d48485a40a4a54a0afcc0bd383b P 07a2cdeeec2b4302a2960391f3825bde [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:17.842875 21966 leader_election.cc:290] T bf2b3d48485a40a4a54a0afcc0bd383b P 07a2cdeeec2b4302a2960391f3825bde [CANDIDATE]: Term 4 election: Requested vote from peers 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1:44917), eaf5735be053455e9315e61c44ff7286 (127.21.63.3:41623)
I20260709 13:07:17.843794 21891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf2b3d48485a40a4a54a0afcc0bd383b" candidate_uuid: "07a2cdeeec2b4302a2960391f3825bde" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "01a67819c8a543adafcf1d5fff0822b4"
I20260709 13:07:17.844166 22038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf2b3d48485a40a4a54a0afcc0bd383b" candidate_uuid: "07a2cdeeec2b4302a2960391f3825bde" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "eaf5735be053455e9315e61c44ff7286"
I20260709 13:07:17.844388 21891 raft_consensus.cc:3060] T bf2b3d48485a40a4a54a0afcc0bd383b P 01a67819c8a543adafcf1d5fff0822b4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:17.844717 22038 raft_consensus.cc:3055] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:17.845055 22038 raft_consensus.cc:740] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: eaf5735be053455e9315e61c44ff7286, State: Running, Role: LEADER
I20260709 13:07:17.845842 22038 consensus_queue.cc:260] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:17.847060 22038 raft_consensus.cc:3060] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:17.850930 21891 raft_consensus.cc:2468] T bf2b3d48485a40a4a54a0afcc0bd383b P 01a67819c8a543adafcf1d5fff0822b4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07a2cdeeec2b4302a2960391f3825bde in term 4.
I20260709 13:07:17.852531 21929 leader_election.cc:304] T bf2b3d48485a40a4a54a0afcc0bd383b P 07a2cdeeec2b4302a2960391f3825bde [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a67819c8a543adafcf1d5fff0822b4, 07a2cdeeec2b4302a2960391f3825bde; no voters: 
I20260709 13:07:17.853602 22188 raft_consensus.cc:2804] T bf2b3d48485a40a4a54a0afcc0bd383b P 07a2cdeeec2b4302a2960391f3825bde [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:17.854002 22038 raft_consensus.cc:2468] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07a2cdeeec2b4302a2960391f3825bde in term 4.
I20260709 13:07:17.854135 22188 raft_consensus.cc:697] T bf2b3d48485a40a4a54a0afcc0bd383b P 07a2cdeeec2b4302a2960391f3825bde [term 4 LEADER]: Becoming Leader. State: Replica: 07a2cdeeec2b4302a2960391f3825bde, State: Running, Role: LEADER
I20260709 13:07:17.854985 22188 consensus_queue.cc:237] T bf2b3d48485a40a4a54a0afcc0bd383b P 07a2cdeeec2b4302a2960391f3825bde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:17.860713 21787 catalog_manager.cc:5697] T bf2b3d48485a40a4a54a0afcc0bd383b P 07a2cdeeec2b4302a2960391f3825bde reported cstate change: term changed from 3 to 4, leader changed from eaf5735be053455e9315e61c44ff7286 (127.21.63.3) to 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2). New cstate: current_term: 4 leader_uuid: "07a2cdeeec2b4302a2960391f3825bde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:17.905822 22205 raft_consensus.cc:993] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4: : Instructing follower 07a2cdeeec2b4302a2960391f3825bde to start an election
I20260709 13:07:17.906301 22195 raft_consensus.cc:1081] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4 [term 1 LEADER]: Signalling peer 07a2cdeeec2b4302a2960391f3825bde to start an election
I20260709 13:07:17.907713 21966 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4c1dcbc1199442dd881d67c582108673"
dest_uuid: "07a2cdeeec2b4302a2960391f3825bde"
 from {username='slave'} at 127.0.0.1:53080
I20260709 13:07:17.908242 21966 raft_consensus.cc:493] T 4c1dcbc1199442dd881d67c582108673 P 07a2cdeeec2b4302a2960391f3825bde [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:17.908602 21966 raft_consensus.cc:3060] T 4c1dcbc1199442dd881d67c582108673 P 07a2cdeeec2b4302a2960391f3825bde [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:17.912457 22205 raft_consensus.cc:993] T 04e73d74096641eab6b132caefd4b28f P 01a67819c8a543adafcf1d5fff0822b4: : Instructing follower 07a2cdeeec2b4302a2960391f3825bde to start an election
I20260709 13:07:17.912909 22195 raft_consensus.cc:1081] T 04e73d74096641eab6b132caefd4b28f P 01a67819c8a543adafcf1d5fff0822b4 [term 1 LEADER]: Signalling peer 07a2cdeeec2b4302a2960391f3825bde to start an election
I20260709 13:07:17.914439 21964 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "04e73d74096641eab6b132caefd4b28f"
dest_uuid: "07a2cdeeec2b4302a2960391f3825bde"
 from {username='slave'} at 127.0.0.1:53080
I20260709 13:07:17.914393 21966 raft_consensus.cc:515] T 4c1dcbc1199442dd881d67c582108673 P 07a2cdeeec2b4302a2960391f3825bde [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:17.915099 21964 raft_consensus.cc:493] T 04e73d74096641eab6b132caefd4b28f P 07a2cdeeec2b4302a2960391f3825bde [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:17.915535 21964 raft_consensus.cc:3060] T 04e73d74096641eab6b132caefd4b28f P 07a2cdeeec2b4302a2960391f3825bde [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:17.916481 21966 leader_election.cc:290] T 4c1dcbc1199442dd881d67c582108673 P 07a2cdeeec2b4302a2960391f3825bde [CANDIDATE]: Term 2 election: Requested vote from peers 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1:44917), eaf5735be053455e9315e61c44ff7286 (127.21.63.3:41623)
I20260709 13:07:17.917196 21891 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c1dcbc1199442dd881d67c582108673" candidate_uuid: "07a2cdeeec2b4302a2960391f3825bde" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "01a67819c8a543adafcf1d5fff0822b4"
I20260709 13:07:17.917543 22038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c1dcbc1199442dd881d67c582108673" candidate_uuid: "07a2cdeeec2b4302a2960391f3825bde" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eaf5735be053455e9315e61c44ff7286"
I20260709 13:07:17.917902 21891 raft_consensus.cc:3055] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:17.918157 22038 raft_consensus.cc:3060] T 4c1dcbc1199442dd881d67c582108673 P eaf5735be053455e9315e61c44ff7286 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:17.918286 21891 raft_consensus.cc:740] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01a67819c8a543adafcf1d5fff0822b4, State: Running, Role: LEADER
I20260709 13:07:17.919100 21891 consensus_queue.cc:260] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:17.919888 21891 raft_consensus.cc:3060] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:17.922622 21964 raft_consensus.cc:515] T 04e73d74096641eab6b132caefd4b28f P 07a2cdeeec2b4302a2960391f3825bde [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:17.924073 21891 raft_consensus.cc:2468] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07a2cdeeec2b4302a2960391f3825bde in term 2.
I20260709 13:07:17.925127 21964 leader_election.cc:290] T 04e73d74096641eab6b132caefd4b28f P 07a2cdeeec2b4302a2960391f3825bde [CANDIDATE]: Term 2 election: Requested vote from peers 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1:44917), eaf5735be053455e9315e61c44ff7286 (127.21.63.3:41623)
I20260709 13:07:17.925289 21929 leader_election.cc:304] T 4c1dcbc1199442dd881d67c582108673 P 07a2cdeeec2b4302a2960391f3825bde [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a67819c8a543adafcf1d5fff0822b4, 07a2cdeeec2b4302a2960391f3825bde; no voters: 
I20260709 13:07:17.925529 22038 raft_consensus.cc:2468] T 4c1dcbc1199442dd881d67c582108673 P eaf5735be053455e9315e61c44ff7286 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07a2cdeeec2b4302a2960391f3825bde in term 2.
I20260709 13:07:17.925768 21890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04e73d74096641eab6b132caefd4b28f" candidate_uuid: "07a2cdeeec2b4302a2960391f3825bde" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "01a67819c8a543adafcf1d5fff0822b4"
I20260709 13:07:17.926434 21890 raft_consensus.cc:3055] T 04e73d74096641eab6b132caefd4b28f P 01a67819c8a543adafcf1d5fff0822b4 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:17.926751 21890 raft_consensus.cc:740] T 04e73d74096641eab6b132caefd4b28f P 01a67819c8a543adafcf1d5fff0822b4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 01a67819c8a543adafcf1d5fff0822b4, State: Running, Role: LEADER
I20260709 13:07:17.927430 22188 raft_consensus.cc:2804] T 4c1dcbc1199442dd881d67c582108673 P 07a2cdeeec2b4302a2960391f3825bde [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:17.927471 22038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "04e73d74096641eab6b132caefd4b28f" candidate_uuid: "07a2cdeeec2b4302a2960391f3825bde" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "eaf5735be053455e9315e61c44ff7286"
I20260709 13:07:17.927517 21890 consensus_queue.cc:260] T 04e73d74096641eab6b132caefd4b28f P 01a67819c8a543adafcf1d5fff0822b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:17.928097 22188 raft_consensus.cc:697] T 4c1dcbc1199442dd881d67c582108673 P 07a2cdeeec2b4302a2960391f3825bde [term 2 LEADER]: Becoming Leader. State: Replica: 07a2cdeeec2b4302a2960391f3825bde, State: Running, Role: LEADER
I20260709 13:07:17.928194 22038 raft_consensus.cc:3060] T 04e73d74096641eab6b132caefd4b28f P eaf5735be053455e9315e61c44ff7286 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:17.928871 21890 raft_consensus.cc:3060] T 04e73d74096641eab6b132caefd4b28f P 01a67819c8a543adafcf1d5fff0822b4 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:17.928920 22188 consensus_queue.cc:237] T 4c1dcbc1199442dd881d67c582108673 P 07a2cdeeec2b4302a2960391f3825bde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:17.936271 22038 raft_consensus.cc:2468] T 04e73d74096641eab6b132caefd4b28f P eaf5735be053455e9315e61c44ff7286 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07a2cdeeec2b4302a2960391f3825bde in term 2.
I20260709 13:07:17.937958 21929 leader_election.cc:304] T 04e73d74096641eab6b132caefd4b28f P 07a2cdeeec2b4302a2960391f3825bde [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07a2cdeeec2b4302a2960391f3825bde, eaf5735be053455e9315e61c44ff7286; no voters: 
I20260709 13:07:17.938277 21890 raft_consensus.cc:2468] T 04e73d74096641eab6b132caefd4b28f P 01a67819c8a543adafcf1d5fff0822b4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 07a2cdeeec2b4302a2960391f3825bde in term 2.
I20260709 13:07:17.938764 22206 raft_consensus.cc:2804] T 04e73d74096641eab6b132caefd4b28f P 07a2cdeeec2b4302a2960391f3825bde [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:17.941833 22206 raft_consensus.cc:697] T 04e73d74096641eab6b132caefd4b28f P 07a2cdeeec2b4302a2960391f3825bde [term 2 LEADER]: Becoming Leader. State: Replica: 07a2cdeeec2b4302a2960391f3825bde, State: Running, Role: LEADER
I20260709 13:07:17.942667 22206 consensus_queue.cc:237] T 04e73d74096641eab6b132caefd4b28f P 07a2cdeeec2b4302a2960391f3825bde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:17.944087 21787 catalog_manager.cc:5697] T 4c1dcbc1199442dd881d67c582108673 P 07a2cdeeec2b4302a2960391f3825bde reported cstate change: term changed from 1 to 2, leader changed from 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1) to 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2). New cstate: current_term: 2 leader_uuid: "07a2cdeeec2b4302a2960391f3825bde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:17.956257 21787 catalog_manager.cc:5697] T 04e73d74096641eab6b132caefd4b28f P 07a2cdeeec2b4302a2960391f3825bde reported cstate change: term changed from 1 to 2, leader changed from 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1) to 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2). New cstate: current_term: 2 leader_uuid: "07a2cdeeec2b4302a2960391f3825bde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:18.168560 21890 raft_consensus.cc:1275] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4 [term 3 FOLLOWER]: Refusing update from remote peer 07a2cdeeec2b4302a2960391f3825bde: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:18.169978 22206 consensus_queue.cc:1048] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [LEADER]: Connected to new peer: Peer: permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:18.181200 22038 raft_consensus.cc:1275] T 2f23dcf5479b46908032cb5858ca9fdf P eaf5735be053455e9315e61c44ff7286 [term 3 FOLLOWER]: Refusing update from remote peer 07a2cdeeec2b4302a2960391f3825bde: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:18.182587 22212 consensus_queue.cc:1048] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:18.259800 21890 raft_consensus.cc:1275] T 4a6231da863b48d2943784b1c955cd09 P 01a67819c8a543adafcf1d5fff0822b4 [term 4 FOLLOWER]: Refusing update from remote peer 07a2cdeeec2b4302a2960391f3825bde: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:07:18.261155 22212 consensus_queue.cc:1048] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde [LEADER]: Connected to new peer: Peer: permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:18.270186 22038 raft_consensus.cc:1275] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [term 4 FOLLOWER]: Refusing update from remote peer 07a2cdeeec2b4302a2960391f3825bde: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:07:18.272248 22206 consensus_queue.cc:1048] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:18.276376 22040 raft_consensus.cc:1275] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [term 4 FOLLOWER]: Refusing update from remote peer 07a2cdeeec2b4302a2960391f3825bde: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:07:18.278088 22206 consensus_queue.cc:1048] T bf2b3d48485a40a4a54a0afcc0bd383b P 07a2cdeeec2b4302a2960391f3825bde [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:18.288174 21890 raft_consensus.cc:1275] T bf2b3d48485a40a4a54a0afcc0bd383b P 01a67819c8a543adafcf1d5fff0822b4 [term 4 FOLLOWER]: Refusing update from remote peer 07a2cdeeec2b4302a2960391f3825bde: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:07:18.289389 22188 consensus_queue.cc:1048] T bf2b3d48485a40a4a54a0afcc0bd383b P 07a2cdeeec2b4302a2960391f3825bde [LEADER]: Connected to new peer: Peer: permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:18.392287 21890 raft_consensus.cc:1275] T 04e73d74096641eab6b132caefd4b28f P 01a67819c8a543adafcf1d5fff0822b4 [term 2 FOLLOWER]: Refusing update from remote peer 07a2cdeeec2b4302a2960391f3825bde: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:18.393996 22188 consensus_queue.cc:1048] T 04e73d74096641eab6b132caefd4b28f P 07a2cdeeec2b4302a2960391f3825bde [LEADER]: Connected to new peer: Peer: permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:18.395982 21890 raft_consensus.cc:1275] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4 [term 2 FOLLOWER]: Refusing update from remote peer 07a2cdeeec2b4302a2960391f3825bde: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:18.397270 22188 consensus_queue.cc:1048] T 4c1dcbc1199442dd881d67c582108673 P 07a2cdeeec2b4302a2960391f3825bde [LEADER]: Connected to new peer: Peer: permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:18.404326 22038 raft_consensus.cc:1275] T 04e73d74096641eab6b132caefd4b28f P eaf5735be053455e9315e61c44ff7286 [term 2 FOLLOWER]: Refusing update from remote peer 07a2cdeeec2b4302a2960391f3825bde: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:18.406839 22212 consensus_queue.cc:1048] T 04e73d74096641eab6b132caefd4b28f P 07a2cdeeec2b4302a2960391f3825bde [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:18.407958 22038 raft_consensus.cc:1275] T 4c1dcbc1199442dd881d67c582108673 P eaf5735be053455e9315e61c44ff7286 [term 2 FOLLOWER]: Refusing update from remote peer 07a2cdeeec2b4302a2960391f3825bde: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:18.410823 22212 consensus_queue.cc:1048] T 4c1dcbc1199442dd881d67c582108673 P 07a2cdeeec2b4302a2960391f3825bde [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:20.585198 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:20.592834 21964 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2f23dcf5479b46908032cb5858ca9fdf"
dest_uuid: "07a2cdeeec2b4302a2960391f3825bde"
mode: GRACEFUL
new_leader_uuid: "01a67819c8a543adafcf1d5fff0822b4"
 from {username='slave'} at 127.0.0.1:53122
I20260709 13:07:20.593622 21964 raft_consensus.cc:606] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [term 3 LEADER]: Received request to transfer leadership to TS 01a67819c8a543adafcf1d5fff0822b4
I20260709 13:07:20.596231 21964 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4c1dcbc1199442dd881d67c582108673"
dest_uuid: "07a2cdeeec2b4302a2960391f3825bde"
mode: GRACEFUL
new_leader_uuid: "01a67819c8a543adafcf1d5fff0822b4"
 from {username='slave'} at 127.0.0.1:53122
I20260709 13:07:20.596729 21964 raft_consensus.cc:606] T 4c1dcbc1199442dd881d67c582108673 P 07a2cdeeec2b4302a2960391f3825bde [term 2 LEADER]: Received request to transfer leadership to TS 01a67819c8a543adafcf1d5fff0822b4
I20260709 13:07:20.598989 21964 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4dfeb09c8a4d482ab573952200ebab0a"
dest_uuid: "07a2cdeeec2b4302a2960391f3825bde"
mode: GRACEFUL
new_leader_uuid: "eaf5735be053455e9315e61c44ff7286"
 from {username='slave'} at 127.0.0.1:53122
I20260709 13:07:20.599529 21964 raft_consensus.cc:606] T 4dfeb09c8a4d482ab573952200ebab0a P 07a2cdeeec2b4302a2960391f3825bde [term 2 LEADER]: Received request to transfer leadership to TS eaf5735be053455e9315e61c44ff7286
I20260709 13:07:20.602102 21964 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bf2b3d48485a40a4a54a0afcc0bd383b"
dest_uuid: "07a2cdeeec2b4302a2960391f3825bde"
mode: GRACEFUL
new_leader_uuid: "eaf5735be053455e9315e61c44ff7286"
 from {username='slave'} at 127.0.0.1:53122
I20260709 13:07:20.602645 21964 raft_consensus.cc:606] T bf2b3d48485a40a4a54a0afcc0bd383b P 07a2cdeeec2b4302a2960391f3825bde [term 4 LEADER]: Received request to transfer leadership to TS eaf5735be053455e9315e61c44ff7286
I20260709 13:07:20.605371 21964 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e77b8c40f5024240a36630365043ad11"
dest_uuid: "07a2cdeeec2b4302a2960391f3825bde"
mode: GRACEFUL
new_leader_uuid: "01a67819c8a543adafcf1d5fff0822b4"
 from {username='slave'} at 127.0.0.1:53122
I20260709 13:07:20.605940 22212 raft_consensus.cc:993] T 4c1dcbc1199442dd881d67c582108673 P 07a2cdeeec2b4302a2960391f3825bde: : Instructing follower 01a67819c8a543adafcf1d5fff0822b4 to start an election
I20260709 13:07:20.606050 21964 raft_consensus.cc:606] T e77b8c40f5024240a36630365043ad11 P 07a2cdeeec2b4302a2960391f3825bde [term 2 LEADER]: Received request to transfer leadership to TS 01a67819c8a543adafcf1d5fff0822b4
I20260709 13:07:20.606537 22230 raft_consensus.cc:1081] T 4c1dcbc1199442dd881d67c582108673 P 07a2cdeeec2b4302a2960391f3825bde [term 2 LEADER]: Signalling peer 01a67819c8a543adafcf1d5fff0822b4 to start an election
I20260709 13:07:20.607494 21756 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:07:20.608071 21756 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:20.608557 21890 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4c1dcbc1199442dd881d67c582108673"
dest_uuid: "01a67819c8a543adafcf1d5fff0822b4"
 from {username='slave'} at 127.0.0.1:59322
I20260709 13:07:20.609053 21890 raft_consensus.cc:493] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:20.609375 21890 raft_consensus.cc:3060] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:20.614425 21890 raft_consensus.cc:515] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:20.616259 21890 leader_election.cc:290] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4 [CANDIDATE]: Term 3 election: Requested vote from peers 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2:36631), eaf5735be053455e9315e61c44ff7286 (127.21.63.3:41623)
I20260709 13:07:20.617143 21964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c1dcbc1199442dd881d67c582108673" candidate_uuid: "01a67819c8a543adafcf1d5fff0822b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "07a2cdeeec2b4302a2960391f3825bde"
I20260709 13:07:20.617415 22040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c1dcbc1199442dd881d67c582108673" candidate_uuid: "01a67819c8a543adafcf1d5fff0822b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "eaf5735be053455e9315e61c44ff7286"
I20260709 13:07:20.617969 21964 raft_consensus.cc:3055] T 4c1dcbc1199442dd881d67c582108673 P 07a2cdeeec2b4302a2960391f3825bde [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:20.618031 22040 raft_consensus.cc:3060] T 4c1dcbc1199442dd881d67c582108673 P eaf5735be053455e9315e61c44ff7286 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:20.618294 21964 raft_consensus.cc:740] T 4c1dcbc1199442dd881d67c582108673 P 07a2cdeeec2b4302a2960391f3825bde [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 07a2cdeeec2b4302a2960391f3825bde, State: Running, Role: LEADER
I20260709 13:07:20.619088 21964 consensus_queue.cc:260] T 4c1dcbc1199442dd881d67c582108673 P 07a2cdeeec2b4302a2960391f3825bde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:20.620069 21964 raft_consensus.cc:3060] T 4c1dcbc1199442dd881d67c582108673 P 07a2cdeeec2b4302a2960391f3825bde [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:20.622833 22040 raft_consensus.cc:2468] T 4c1dcbc1199442dd881d67c582108673 P eaf5735be053455e9315e61c44ff7286 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01a67819c8a543adafcf1d5fff0822b4 in term 3.
I20260709 13:07:20.624063 21854 leader_election.cc:304] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a67819c8a543adafcf1d5fff0822b4, eaf5735be053455e9315e61c44ff7286; no voters: 
I20260709 13:07:20.625104 21964 raft_consensus.cc:2468] T 4c1dcbc1199442dd881d67c582108673 P 07a2cdeeec2b4302a2960391f3825bde [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01a67819c8a543adafcf1d5fff0822b4 in term 3.
I20260709 13:07:20.625605 22231 raft_consensus.cc:2804] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:20.626827 22231 raft_consensus.cc:697] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4 [term 3 LEADER]: Becoming Leader. State: Replica: 01a67819c8a543adafcf1d5fff0822b4, State: Running, Role: LEADER
I20260709 13:07:20.627767 22231 consensus_queue.cc:237] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:20.635596 21787 catalog_manager.cc:5697] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4 reported cstate change: term changed from 2 to 3, leader changed from 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2) to 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1). New cstate: current_term: 3 leader_uuid: "01a67819c8a543adafcf1d5fff0822b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:20.870939 22212 raft_consensus.cc:993] T 4dfeb09c8a4d482ab573952200ebab0a P 07a2cdeeec2b4302a2960391f3825bde: : Instructing follower eaf5735be053455e9315e61c44ff7286 to start an election
I20260709 13:07:20.871549 22230 raft_consensus.cc:1081] T 4dfeb09c8a4d482ab573952200ebab0a P 07a2cdeeec2b4302a2960391f3825bde [term 2 LEADER]: Signalling peer eaf5735be053455e9315e61c44ff7286 to start an election
I20260709 13:07:20.872957 22040 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4dfeb09c8a4d482ab573952200ebab0a"
dest_uuid: "eaf5735be053455e9315e61c44ff7286"
 from {username='slave'} at 127.0.0.1:35880
I20260709 13:07:20.873421 22040 raft_consensus.cc:493] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:20.873677 22040 raft_consensus.cc:3060] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:20.878154 22212 raft_consensus.cc:993] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde: : Instructing follower 01a67819c8a543adafcf1d5fff0822b4 to start an election
I20260709 13:07:20.878577 22230 raft_consensus.cc:1081] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [term 3 LEADER]: Signalling peer 01a67819c8a543adafcf1d5fff0822b4 to start an election
I20260709 13:07:20.879628 22040 raft_consensus.cc:515] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:20.880100 21890 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2f23dcf5479b46908032cb5858ca9fdf"
dest_uuid: "01a67819c8a543adafcf1d5fff0822b4"
 from {username='slave'} at 127.0.0.1:59322
I20260709 13:07:20.880579 21890 raft_consensus.cc:493] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:20.880962 21890 raft_consensus.cc:3060] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:20.882624 21889 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dfeb09c8a4d482ab573952200ebab0a" candidate_uuid: "eaf5735be053455e9315e61c44ff7286" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "01a67819c8a543adafcf1d5fff0822b4"
I20260709 13:07:20.882658 22040 leader_election.cc:290] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 [CANDIDATE]: Term 3 election: Requested vote from peers 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1:44917), 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2:36631)
I20260709 13:07:20.886088 21964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4dfeb09c8a4d482ab573952200ebab0a" candidate_uuid: "eaf5735be053455e9315e61c44ff7286" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "07a2cdeeec2b4302a2960391f3825bde"
I20260709 13:07:20.886662 21964 raft_consensus.cc:3055] T 4dfeb09c8a4d482ab573952200ebab0a P 07a2cdeeec2b4302a2960391f3825bde [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:20.886970 21964 raft_consensus.cc:740] T 4dfeb09c8a4d482ab573952200ebab0a P 07a2cdeeec2b4302a2960391f3825bde [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 07a2cdeeec2b4302a2960391f3825bde, State: Running, Role: LEADER
I20260709 13:07:20.887753 21964 consensus_queue.cc:260] T 4dfeb09c8a4d482ab573952200ebab0a P 07a2cdeeec2b4302a2960391f3825bde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:20.888896 21889 raft_consensus.cc:3060] T 4dfeb09c8a4d482ab573952200ebab0a P 01a67819c8a543adafcf1d5fff0822b4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:20.889072 21964 raft_consensus.cc:3060] T 4dfeb09c8a4d482ab573952200ebab0a P 07a2cdeeec2b4302a2960391f3825bde [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:20.894007 21889 raft_consensus.cc:2468] T 4dfeb09c8a4d482ab573952200ebab0a P 01a67819c8a543adafcf1d5fff0822b4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaf5735be053455e9315e61c44ff7286 in term 3.
I20260709 13:07:20.894069 21964 raft_consensus.cc:2468] T 4dfeb09c8a4d482ab573952200ebab0a P 07a2cdeeec2b4302a2960391f3825bde [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaf5735be053455e9315e61c44ff7286 in term 3.
I20260709 13:07:20.888142 21890 raft_consensus.cc:515] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:20.895224 22003 leader_election.cc:304] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a67819c8a543adafcf1d5fff0822b4, eaf5735be053455e9315e61c44ff7286; no voters: 
I20260709 13:07:20.896111 21890 leader_election.cc:290] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4 [CANDIDATE]: Term 4 election: Requested vote from peers 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2:36631), eaf5735be053455e9315e61c44ff7286 (127.21.63.3:41623)
I20260709 13:07:20.896436 22189 raft_consensus.cc:2804] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:20.897249 22040 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f23dcf5479b46908032cb5858ca9fdf" candidate_uuid: "01a67819c8a543adafcf1d5fff0822b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "eaf5735be053455e9315e61c44ff7286"
I20260709 13:07:20.898459 22040 raft_consensus.cc:3060] T 2f23dcf5479b46908032cb5858ca9fdf P eaf5735be053455e9315e61c44ff7286 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:20.898802 22189 raft_consensus.cc:697] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 [term 3 LEADER]: Becoming Leader. State: Replica: eaf5735be053455e9315e61c44ff7286, State: Running, Role: LEADER
I20260709 13:07:20.899487 22189 consensus_queue.cc:237] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:20.902844 21964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2f23dcf5479b46908032cb5858ca9fdf" candidate_uuid: "01a67819c8a543adafcf1d5fff0822b4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "07a2cdeeec2b4302a2960391f3825bde"
I20260709 13:07:20.903587 21964 raft_consensus.cc:3055] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:20.903899 22040 raft_consensus.cc:2468] T 2f23dcf5479b46908032cb5858ca9fdf P eaf5735be053455e9315e61c44ff7286 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01a67819c8a543adafcf1d5fff0822b4 in term 4.
I20260709 13:07:20.904012 21964 raft_consensus.cc:740] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 07a2cdeeec2b4302a2960391f3825bde, State: Running, Role: LEADER
I20260709 13:07:20.905218 21854 leader_election.cc:304] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a67819c8a543adafcf1d5fff0822b4, eaf5735be053455e9315e61c44ff7286; no voters: 
I20260709 13:07:20.905220 21964 consensus_queue.cc:260] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:20.906169 22231 raft_consensus.cc:2804] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:20.906601 22231 raft_consensus.cc:697] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4 [term 4 LEADER]: Becoming Leader. State: Replica: 01a67819c8a543adafcf1d5fff0822b4, State: Running, Role: LEADER
I20260709 13:07:20.906687 21964 raft_consensus.cc:3060] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:20.907413 21787 catalog_manager.cc:5697] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 reported cstate change: term changed from 2 to 3, leader changed from 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2) to eaf5735be053455e9315e61c44ff7286 (127.21.63.3). New cstate: current_term: 3 leader_uuid: "eaf5735be053455e9315e61c44ff7286" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:20.907589 22231 consensus_queue.cc:237] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:20.914054 21964 raft_consensus.cc:2468] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01a67819c8a543adafcf1d5fff0822b4 in term 4.
I20260709 13:07:20.916994 21787 catalog_manager.cc:5697] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4 reported cstate change: term changed from 3 to 4, leader changed from 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2) to 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1). New cstate: current_term: 4 leader_uuid: "01a67819c8a543adafcf1d5fff0822b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:21.026094 22212 raft_consensus.cc:993] T bf2b3d48485a40a4a54a0afcc0bd383b P 07a2cdeeec2b4302a2960391f3825bde: : Instructing follower eaf5735be053455e9315e61c44ff7286 to start an election
I20260709 13:07:21.026531 22230 raft_consensus.cc:1081] T bf2b3d48485a40a4a54a0afcc0bd383b P 07a2cdeeec2b4302a2960391f3825bde [term 4 LEADER]: Signalling peer eaf5735be053455e9315e61c44ff7286 to start an election
I20260709 13:07:21.027873 22040 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bf2b3d48485a40a4a54a0afcc0bd383b"
dest_uuid: "eaf5735be053455e9315e61c44ff7286"
 from {username='slave'} at 127.0.0.1:35880
I20260709 13:07:21.028348 22040 raft_consensus.cc:493] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:21.028612 22040 raft_consensus.cc:3060] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:21.032326 22040 raft_consensus.cc:515] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:21.033680 22040 leader_election.cc:290] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [CANDIDATE]: Term 5 election: Requested vote from peers 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1:44917), 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2:36631)
I20260709 13:07:21.034495 21890 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf2b3d48485a40a4a54a0afcc0bd383b" candidate_uuid: "eaf5735be053455e9315e61c44ff7286" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "01a67819c8a543adafcf1d5fff0822b4"
I20260709 13:07:21.034801 21964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf2b3d48485a40a4a54a0afcc0bd383b" candidate_uuid: "eaf5735be053455e9315e61c44ff7286" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "07a2cdeeec2b4302a2960391f3825bde"
I20260709 13:07:21.035037 21890 raft_consensus.cc:3060] T bf2b3d48485a40a4a54a0afcc0bd383b P 01a67819c8a543adafcf1d5fff0822b4 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:21.035386 21964 raft_consensus.cc:3055] T bf2b3d48485a40a4a54a0afcc0bd383b P 07a2cdeeec2b4302a2960391f3825bde [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:21.035665 21964 raft_consensus.cc:740] T bf2b3d48485a40a4a54a0afcc0bd383b P 07a2cdeeec2b4302a2960391f3825bde [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 07a2cdeeec2b4302a2960391f3825bde, State: Running, Role: LEADER
I20260709 13:07:21.036350 21964 consensus_queue.cc:260] T bf2b3d48485a40a4a54a0afcc0bd383b P 07a2cdeeec2b4302a2960391f3825bde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:21.037366 21964 raft_consensus.cc:3060] T bf2b3d48485a40a4a54a0afcc0bd383b P 07a2cdeeec2b4302a2960391f3825bde [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:21.039119 21890 raft_consensus.cc:2468] T bf2b3d48485a40a4a54a0afcc0bd383b P 01a67819c8a543adafcf1d5fff0822b4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaf5735be053455e9315e61c44ff7286 in term 5.
I20260709 13:07:21.040186 22003 leader_election.cc:304] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a67819c8a543adafcf1d5fff0822b4, eaf5735be053455e9315e61c44ff7286; no voters: 
I20260709 13:07:21.041070 22238 raft_consensus.cc:2804] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:21.041450 22238 raft_consensus.cc:697] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [term 5 LEADER]: Becoming Leader. State: Replica: eaf5735be053455e9315e61c44ff7286, State: Running, Role: LEADER
I20260709 13:07:21.042095 21964 raft_consensus.cc:2468] T bf2b3d48485a40a4a54a0afcc0bd383b P 07a2cdeeec2b4302a2960391f3825bde [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eaf5735be053455e9315e61c44ff7286 in term 5.
I20260709 13:07:21.042235 22238 consensus_queue.cc:237] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:21.049011 21787 catalog_manager.cc:5697] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 reported cstate change: term changed from 4 to 5, leader changed from 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2) to eaf5735be053455e9315e61c44ff7286 (127.21.63.3). New cstate: current_term: 5 leader_uuid: "eaf5735be053455e9315e61c44ff7286" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:21.109390 22040 raft_consensus.cc:1275] T 4c1dcbc1199442dd881d67c582108673 P eaf5735be053455e9315e61c44ff7286 [term 3 FOLLOWER]: Refusing update from remote peer 01a67819c8a543adafcf1d5fff0822b4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:21.110651 22231 consensus_queue.cc:1048] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:21.122432 21964 raft_consensus.cc:1275] T 4c1dcbc1199442dd881d67c582108673 P 07a2cdeeec2b4302a2960391f3825bde [term 3 FOLLOWER]: Refusing update from remote peer 01a67819c8a543adafcf1d5fff0822b4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:21.125128 22231 consensus_queue.cc:1048] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:21.139632 22212 raft_consensus.cc:993] T e77b8c40f5024240a36630365043ad11 P 07a2cdeeec2b4302a2960391f3825bde: : Instructing follower 01a67819c8a543adafcf1d5fff0822b4 to start an election
I20260709 13:07:21.140113 22230 raft_consensus.cc:1081] T e77b8c40f5024240a36630365043ad11 P 07a2cdeeec2b4302a2960391f3825bde [term 2 LEADER]: Signalling peer 01a67819c8a543adafcf1d5fff0822b4 to start an election
I20260709 13:07:21.141595 21890 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e77b8c40f5024240a36630365043ad11"
dest_uuid: "01a67819c8a543adafcf1d5fff0822b4"
 from {username='slave'} at 127.0.0.1:59322
I20260709 13:07:21.141999 21890 raft_consensus.cc:493] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:21.142230 21890 raft_consensus.cc:3060] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:21.146454 21890 raft_consensus.cc:515] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:21.148149 21890 leader_election.cc:290] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4 [CANDIDATE]: Term 3 election: Requested vote from peers 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2:36631), eaf5735be053455e9315e61c44ff7286 (127.21.63.3:41623)
I20260709 13:07:21.148953 21964 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b8c40f5024240a36630365043ad11" candidate_uuid: "01a67819c8a543adafcf1d5fff0822b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "07a2cdeeec2b4302a2960391f3825bde"
I20260709 13:07:21.149281 22038 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e77b8c40f5024240a36630365043ad11" candidate_uuid: "01a67819c8a543adafcf1d5fff0822b4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "eaf5735be053455e9315e61c44ff7286"
I20260709 13:07:21.149570 21964 raft_consensus.cc:3055] T e77b8c40f5024240a36630365043ad11 P 07a2cdeeec2b4302a2960391f3825bde [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:21.149909 21964 raft_consensus.cc:740] T e77b8c40f5024240a36630365043ad11 P 07a2cdeeec2b4302a2960391f3825bde [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 07a2cdeeec2b4302a2960391f3825bde, State: Running, Role: LEADER
I20260709 13:07:21.149883 22038 raft_consensus.cc:3060] T e77b8c40f5024240a36630365043ad11 P eaf5735be053455e9315e61c44ff7286 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:21.150683 21964 consensus_queue.cc:260] T e77b8c40f5024240a36630365043ad11 P 07a2cdeeec2b4302a2960391f3825bde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:21.151674 21964 raft_consensus.cc:3060] T e77b8c40f5024240a36630365043ad11 P 07a2cdeeec2b4302a2960391f3825bde [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:21.154549 22038 raft_consensus.cc:2468] T e77b8c40f5024240a36630365043ad11 P eaf5735be053455e9315e61c44ff7286 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01a67819c8a543adafcf1d5fff0822b4 in term 3.
I20260709 13:07:21.155541 21854 leader_election.cc:304] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01a67819c8a543adafcf1d5fff0822b4, eaf5735be053455e9315e61c44ff7286; no voters: 
I20260709 13:07:21.156057 21964 raft_consensus.cc:2468] T e77b8c40f5024240a36630365043ad11 P 07a2cdeeec2b4302a2960391f3825bde [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01a67819c8a543adafcf1d5fff0822b4 in term 3.
I20260709 13:07:21.156262 22231 raft_consensus.cc:2804] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:21.156728 22231 raft_consensus.cc:697] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4 [term 3 LEADER]: Becoming Leader. State: Replica: 01a67819c8a543adafcf1d5fff0822b4, State: Running, Role: LEADER
I20260709 13:07:21.157630 22231 consensus_queue.cc:237] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } }
I20260709 13:07:21.164655 21786 catalog_manager.cc:5697] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4 reported cstate change: term changed from 2 to 3, leader changed from 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2) to 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1). New cstate: current_term: 3 leader_uuid: "01a67819c8a543adafcf1d5fff0822b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:21.301270 21890 raft_consensus.cc:1275] T 4dfeb09c8a4d482ab573952200ebab0a P 01a67819c8a543adafcf1d5fff0822b4 [term 3 FOLLOWER]: Refusing update from remote peer eaf5735be053455e9315e61c44ff7286: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:21.302425 22238 consensus_queue.cc:1048] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:21.307170 21964 raft_consensus.cc:1275] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [term 4 FOLLOWER]: Refusing update from remote peer 01a67819c8a543adafcf1d5fff0822b4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:07:21.308451 22231 consensus_queue.cc:1048] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:21.311908 21966 raft_consensus.cc:1275] T 4dfeb09c8a4d482ab573952200ebab0a P 07a2cdeeec2b4302a2960391f3825bde [term 3 FOLLOWER]: Refusing update from remote peer eaf5735be053455e9315e61c44ff7286: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:21.313342 22249 consensus_queue.cc:1048] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:21.322196 22038 raft_consensus.cc:1275] T 2f23dcf5479b46908032cb5858ca9fdf P eaf5735be053455e9315e61c44ff7286 [term 4 FOLLOWER]: Refusing update from remote peer 01a67819c8a543adafcf1d5fff0822b4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:07:21.323464 22231 consensus_queue.cc:1048] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:21.514963 21964 raft_consensus.cc:1275] T bf2b3d48485a40a4a54a0afcc0bd383b P 07a2cdeeec2b4302a2960391f3825bde [term 5 FOLLOWER]: Refusing update from remote peer eaf5735be053455e9315e61c44ff7286: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:07:21.516214 22249 consensus_queue.cc:1048] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:21.525055 21890 raft_consensus.cc:1275] T bf2b3d48485a40a4a54a0afcc0bd383b P 01a67819c8a543adafcf1d5fff0822b4 [term 5 FOLLOWER]: Refusing update from remote peer eaf5735be053455e9315e61c44ff7286: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:07:21.526528 22189 consensus_queue.cc:1048] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01a67819c8a543adafcf1d5fff0822b4" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 44917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:21.609074 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:21.613291 21964 raft_consensus.cc:1275] T e77b8c40f5024240a36630365043ad11 P 07a2cdeeec2b4302a2960391f3825bde [term 3 FOLLOWER]: Refusing update from remote peer 01a67819c8a543adafcf1d5fff0822b4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:21.614496 22231 consensus_queue.cc:1048] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "07a2cdeeec2b4302a2960391f3825bde" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 36631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:21.615367 21756 tablet_server.cc:179] TabletServer@127.21.63.1:0 shutting down...
W20260709 13:07:21.626281 21929 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:07:21.637559 22038 raft_consensus.cc:1275] T e77b8c40f5024240a36630365043ad11 P eaf5735be053455e9315e61c44ff7286 [term 3 FOLLOWER]: Refusing update from remote peer 01a67819c8a543adafcf1d5fff0822b4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:21.639430 22231 consensus_queue.cc:1048] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaf5735be053455e9315e61c44ff7286" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 13:07:21.647334 21929 consensus_peers.cc:597] T b10d9c93da2c485d96b8ca9058a1b00f P 07a2cdeeec2b4302a2960391f3825bde -> Peer 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1:44917): Couldn't send request to peer 01a67819c8a543adafcf1d5fff0822b4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:21.684259 21756 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:21.685143 21756 tablet_replica.cc:333] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4: stopping tablet replica
I20260709 13:07:21.685997 21756 raft_consensus.cc:2243] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:21.686865 21756 raft_consensus.cc:2272] T e77b8c40f5024240a36630365043ad11 P 01a67819c8a543adafcf1d5fff0822b4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.690445 21756 tablet_replica.cc:333] T 4dfeb09c8a4d482ab573952200ebab0a P 01a67819c8a543adafcf1d5fff0822b4: stopping tablet replica
I20260709 13:07:21.691010 21756 raft_consensus.cc:2243] T 4dfeb09c8a4d482ab573952200ebab0a P 01a67819c8a543adafcf1d5fff0822b4 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.691459 21756 raft_consensus.cc:2272] T 4dfeb09c8a4d482ab573952200ebab0a P 01a67819c8a543adafcf1d5fff0822b4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.693557 21756 tablet_replica.cc:333] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4: stopping tablet replica
I20260709 13:07:21.694108 21756 raft_consensus.cc:2243] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:21.694860 21756 raft_consensus.cc:2272] T 4c1dcbc1199442dd881d67c582108673 P 01a67819c8a543adafcf1d5fff0822b4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.697338 21756 tablet_replica.cc:333] T 4a6231da863b48d2943784b1c955cd09 P 01a67819c8a543adafcf1d5fff0822b4: stopping tablet replica
I20260709 13:07:21.697880 21756 raft_consensus.cc:2243] T 4a6231da863b48d2943784b1c955cd09 P 01a67819c8a543adafcf1d5fff0822b4 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.698329 21756 raft_consensus.cc:2272] T 4a6231da863b48d2943784b1c955cd09 P 01a67819c8a543adafcf1d5fff0822b4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.699914 21756 tablet_replica.cc:333] T bf2b3d48485a40a4a54a0afcc0bd383b P 01a67819c8a543adafcf1d5fff0822b4: stopping tablet replica
I20260709 13:07:21.700397 21756 raft_consensus.cc:2243] T bf2b3d48485a40a4a54a0afcc0bd383b P 01a67819c8a543adafcf1d5fff0822b4 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.700881 21756 raft_consensus.cc:2272] T bf2b3d48485a40a4a54a0afcc0bd383b P 01a67819c8a543adafcf1d5fff0822b4 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.702813 21756 tablet_replica.cc:333] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4: stopping tablet replica
I20260709 13:07:21.703338 21756 raft_consensus.cc:2243] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:07:21.704062 21756 raft_consensus.cc:2272] T 2f23dcf5479b46908032cb5858ca9fdf P 01a67819c8a543adafcf1d5fff0822b4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.706084 21756 tablet_replica.cc:333] T 29b15358296749cc9bd2c0a14af3a408 P 01a67819c8a543adafcf1d5fff0822b4: stopping tablet replica
I20260709 13:07:21.706581 21756 raft_consensus.cc:2243] T 29b15358296749cc9bd2c0a14af3a408 P 01a67819c8a543adafcf1d5fff0822b4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.707135 21756 raft_consensus.cc:2272] T 29b15358296749cc9bd2c0a14af3a408 P 01a67819c8a543adafcf1d5fff0822b4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.708896 21756 tablet_replica.cc:333] T b10d9c93da2c485d96b8ca9058a1b00f P 01a67819c8a543adafcf1d5fff0822b4: stopping tablet replica
I20260709 13:07:21.709378 21756 raft_consensus.cc:2243] T b10d9c93da2c485d96b8ca9058a1b00f P 01a67819c8a543adafcf1d5fff0822b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.709764 21756 raft_consensus.cc:2272] T b10d9c93da2c485d96b8ca9058a1b00f P 01a67819c8a543adafcf1d5fff0822b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.711337 21756 tablet_replica.cc:333] T 04e73d74096641eab6b132caefd4b28f P 01a67819c8a543adafcf1d5fff0822b4: stopping tablet replica
I20260709 13:07:21.711814 21756 raft_consensus.cc:2243] T 04e73d74096641eab6b132caefd4b28f P 01a67819c8a543adafcf1d5fff0822b4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.712231 21756 raft_consensus.cc:2272] T 04e73d74096641eab6b132caefd4b28f P 01a67819c8a543adafcf1d5fff0822b4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.813627 21756 tablet_server.cc:196] TabletServer@127.21.63.1:0 shutdown complete.
I20260709 13:07:21.833961 21756 tablet_server.cc:179] TabletServer@127.21.63.2:0 shutting down...
I20260709 13:07:21.858415 21756 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:21.859207 21756 tablet_replica.cc:333] T 4dfeb09c8a4d482ab573952200ebab0a P 07a2cdeeec2b4302a2960391f3825bde: stopping tablet replica
I20260709 13:07:21.859891 21756 raft_consensus.cc:2243] T 4dfeb09c8a4d482ab573952200ebab0a P 07a2cdeeec2b4302a2960391f3825bde [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.860387 21756 raft_consensus.cc:2272] T 4dfeb09c8a4d482ab573952200ebab0a P 07a2cdeeec2b4302a2960391f3825bde [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.863523 21756 tablet_replica.cc:333] T e77b8c40f5024240a36630365043ad11 P 07a2cdeeec2b4302a2960391f3825bde: stopping tablet replica
I20260709 13:07:21.864315 21756 raft_consensus.cc:2243] T e77b8c40f5024240a36630365043ad11 P 07a2cdeeec2b4302a2960391f3825bde [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.864827 21756 raft_consensus.cc:2272] T e77b8c40f5024240a36630365043ad11 P 07a2cdeeec2b4302a2960391f3825bde [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:21.869913 22006 consensus_peers.cc:597] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 -> Peer 07a2cdeeec2b4302a2960391f3825bde (127.21.63.2:36631): Couldn't send request to peer 07a2cdeeec2b4302a2960391f3825bde. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:21.868038 21756 tablet_replica.cc:333] T 29b15358296749cc9bd2c0a14af3a408 P 07a2cdeeec2b4302a2960391f3825bde: stopping tablet replica
I20260709 13:07:21.872220 21756 raft_consensus.cc:2243] T 29b15358296749cc9bd2c0a14af3a408 P 07a2cdeeec2b4302a2960391f3825bde [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.872819 21756 raft_consensus.cc:2272] T 29b15358296749cc9bd2c0a14af3a408 P 07a2cdeeec2b4302a2960391f3825bde [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:07:21.874089 22003 consensus_peers.cc:597] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 -> Peer 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1:44917): Couldn't send request to peer 01a67819c8a543adafcf1d5fff0822b4. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:44917: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:21.874979 21756 tablet_replica.cc:333] T 04e73d74096641eab6b132caefd4b28f P 07a2cdeeec2b4302a2960391f3825bde: stopping tablet replica
W20260709 13:07:21.875190 21929 consensus_peers.cc:597] T 04e73d74096641eab6b132caefd4b28f P 07a2cdeeec2b4302a2960391f3825bde -> Peer 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1:44917): Couldn't send request to peer 01a67819c8a543adafcf1d5fff0822b4. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:44917: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:21.875653 21756 raft_consensus.cc:2243] T 04e73d74096641eab6b132caefd4b28f P 07a2cdeeec2b4302a2960391f3825bde [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:21.876384 21756 raft_consensus.cc:2272] T 04e73d74096641eab6b132caefd4b28f P 07a2cdeeec2b4302a2960391f3825bde [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.878062 21756 tablet_replica.cc:333] T b10d9c93da2c485d96b8ca9058a1b00f P 07a2cdeeec2b4302a2960391f3825bde: stopping tablet replica
I20260709 13:07:21.878490 21756 raft_consensus.cc:2243] T b10d9c93da2c485d96b8ca9058a1b00f P 07a2cdeeec2b4302a2960391f3825bde [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:21.879113 21756 raft_consensus.cc:2272] T b10d9c93da2c485d96b8ca9058a1b00f P 07a2cdeeec2b4302a2960391f3825bde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.880645 21756 tablet_replica.cc:333] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde: stopping tablet replica
I20260709 13:07:21.881106 21756 raft_consensus.cc:2243] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:07:21.881719 21756 raft_consensus.cc:2272] T 4a6231da863b48d2943784b1c955cd09 P 07a2cdeeec2b4302a2960391f3825bde [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.883230 21756 tablet_replica.cc:333] T bf2b3d48485a40a4a54a0afcc0bd383b P 07a2cdeeec2b4302a2960391f3825bde: stopping tablet replica
I20260709 13:07:21.883708 21756 raft_consensus.cc:2243] T bf2b3d48485a40a4a54a0afcc0bd383b P 07a2cdeeec2b4302a2960391f3825bde [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.884034 21756 raft_consensus.cc:2272] T bf2b3d48485a40a4a54a0afcc0bd383b P 07a2cdeeec2b4302a2960391f3825bde [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.885797 21756 tablet_replica.cc:333] T 4c1dcbc1199442dd881d67c582108673 P 07a2cdeeec2b4302a2960391f3825bde: stopping tablet replica
I20260709 13:07:21.886188 21756 raft_consensus.cc:2243] T 4c1dcbc1199442dd881d67c582108673 P 07a2cdeeec2b4302a2960391f3825bde [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.886509 21756 raft_consensus.cc:2272] T 4c1dcbc1199442dd881d67c582108673 P 07a2cdeeec2b4302a2960391f3825bde [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.888311 21756 tablet_replica.cc:333] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde: stopping tablet replica
I20260709 13:07:21.888813 21756 raft_consensus.cc:2243] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.889223 21756 raft_consensus.cc:2272] T 2f23dcf5479b46908032cb5858ca9fdf P 07a2cdeeec2b4302a2960391f3825bde [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.915506 21756 tablet_server.cc:196] TabletServer@127.21.63.2:0 shutdown complete.
I20260709 13:07:21.931711 21756 tablet_server.cc:179] TabletServer@127.21.63.3:0 shutting down...
W20260709 13:07:21.955718 22003 consensus_peers.cc:597] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 -> Peer 01a67819c8a543adafcf1d5fff0822b4 (127.21.63.1:44917): Couldn't send request to peer 01a67819c8a543adafcf1d5fff0822b4. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:44917: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:21.960407 21756 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:21.961278 21756 tablet_replica.cc:333] T e77b8c40f5024240a36630365043ad11 P eaf5735be053455e9315e61c44ff7286: stopping tablet replica
I20260709 13:07:21.962005 21756 raft_consensus.cc:2243] T e77b8c40f5024240a36630365043ad11 P eaf5735be053455e9315e61c44ff7286 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.962517 21756 raft_consensus.cc:2272] T e77b8c40f5024240a36630365043ad11 P eaf5735be053455e9315e61c44ff7286 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.964852 21756 tablet_replica.cc:333] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286: stopping tablet replica
I20260709 13:07:21.965405 21756 raft_consensus.cc:2243] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:21.966207 21756 raft_consensus.cc:2272] T 4dfeb09c8a4d482ab573952200ebab0a P eaf5735be053455e9315e61c44ff7286 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.969055 21756 tablet_replica.cc:333] T 29b15358296749cc9bd2c0a14af3a408 P eaf5735be053455e9315e61c44ff7286: stopping tablet replica
I20260709 13:07:21.969616 21756 raft_consensus.cc:2243] T 29b15358296749cc9bd2c0a14af3a408 P eaf5735be053455e9315e61c44ff7286 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:21.970393 21756 raft_consensus.cc:2272] T 29b15358296749cc9bd2c0a14af3a408 P eaf5735be053455e9315e61c44ff7286 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.972111 21756 tablet_replica.cc:333] T b10d9c93da2c485d96b8ca9058a1b00f P eaf5735be053455e9315e61c44ff7286: stopping tablet replica
I20260709 13:07:21.972637 21756 raft_consensus.cc:2243] T b10d9c93da2c485d96b8ca9058a1b00f P eaf5735be053455e9315e61c44ff7286 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.973138 21756 raft_consensus.cc:2272] T b10d9c93da2c485d96b8ca9058a1b00f P eaf5735be053455e9315e61c44ff7286 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.974856 21756 tablet_replica.cc:333] T 04e73d74096641eab6b132caefd4b28f P eaf5735be053455e9315e61c44ff7286: stopping tablet replica
I20260709 13:07:21.975373 21756 raft_consensus.cc:2243] T 04e73d74096641eab6b132caefd4b28f P eaf5735be053455e9315e61c44ff7286 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.975841 21756 raft_consensus.cc:2272] T 04e73d74096641eab6b132caefd4b28f P eaf5735be053455e9315e61c44ff7286 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.977684 21756 tablet_replica.cc:333] T 4c1dcbc1199442dd881d67c582108673 P eaf5735be053455e9315e61c44ff7286: stopping tablet replica
I20260709 13:07:21.978233 21756 raft_consensus.cc:2243] T 4c1dcbc1199442dd881d67c582108673 P eaf5735be053455e9315e61c44ff7286 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.978662 21756 raft_consensus.cc:2272] T 4c1dcbc1199442dd881d67c582108673 P eaf5735be053455e9315e61c44ff7286 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.980880 21756 tablet_replica.cc:333] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286: stopping tablet replica
I20260709 13:07:21.981451 21756 raft_consensus.cc:2243] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.981935 21756 raft_consensus.cc:2272] T 4a6231da863b48d2943784b1c955cd09 P eaf5735be053455e9315e61c44ff7286 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.983645 21756 tablet_replica.cc:333] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286: stopping tablet replica
I20260709 13:07:21.984123 21756 raft_consensus.cc:2243] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:07:21.984963 21756 raft_consensus.cc:2272] T bf2b3d48485a40a4a54a0afcc0bd383b P eaf5735be053455e9315e61c44ff7286 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:21.986995 21756 tablet_replica.cc:333] T 2f23dcf5479b46908032cb5858ca9fdf P eaf5735be053455e9315e61c44ff7286: stopping tablet replica
I20260709 13:07:21.987494 21756 raft_consensus.cc:2243] T 2f23dcf5479b46908032cb5858ca9fdf P eaf5735be053455e9315e61c44ff7286 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:21.987944 21756 raft_consensus.cc:2272] T 2f23dcf5479b46908032cb5858ca9fdf P eaf5735be053455e9315e61c44ff7286 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.017503 21756 tablet_server.cc:196] TabletServer@127.21.63.3:0 shutdown complete.
I20260709 13:07:22.033665 21756 master.cc:562] Master@127.21.63.62:35313 shutting down...
I20260709 13:07:22.142506 21756 raft_consensus.cc:2243] T 00000000000000000000000000000000 P b7fa0401325f4c5e8657495c45164fe4 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:22.143076 21756 raft_consensus.cc:2272] T 00000000000000000000000000000000 P b7fa0401325f4c5e8657495c45164fe4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:22.143431 21756 tablet_replica.cc:333] T 00000000000000000000000000000000 P b7fa0401325f4c5e8657495c45164fe4: stopping tablet replica
I20260709 13:07:22.164072 21756 master.cc:584] Master@127.21.63.62:35313 shutdown complete.
01a67819c8a543adafcf1d5fff0822b4  4
07a2cdeeec2b4302a2960391f3825bde  4
eaf5735be053455e9315e61c44ff7286  1
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (12614 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 13:07:22.205591 21756 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.63.62:46883
I20260709 13:07:22.206709 21756 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:22.211920 22262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:22.212132 22263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:22.214491 22265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:22.214829 21756 server_base.cc:1061] running on GCE node
I20260709 13:07:22.215622 21756 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:22.215796 21756 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:22.215912 21756 hybrid_clock.cc:648] HybridClock initialized: now 1783602442215901 us; error 0 us; skew 500 ppm
I20260709 13:07:22.216365 21756 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:22.218915 21756 webserver.cc:533] Webserver started at http://127.21.63.62:39437/ using document root <none> and password file <none>
I20260709 13:07:22.219386 21756 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:22.219553 21756 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:22.219789 21756 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:22.220960 21756 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602429420501-21756-0/minicluster-data/master-0-root/instance:
uuid: "fb83fe4dd9d6412ca3df972b6f2106cb"
format_stamp: "Formatted at 2026-07-09 13:07:22 on dist-test-slave-c6mt"
I20260709 13:07:22.225523 21756 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:07:22.228806 22270 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:22.229658 21756 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:07:22.230001 21756 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602429420501-21756-0/minicluster-data/master-0-root
uuid: "fb83fe4dd9d6412ca3df972b6f2106cb"
format_stamp: "Formatted at 2026-07-09 13:07:22 on dist-test-slave-c6mt"
I20260709 13:07:22.230365 21756 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602429420501-21756-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602429420501-21756-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602429420501-21756-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:22.252997 21756 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:22.254181 21756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:22.290995 21756 rpc_server.cc:307] RPC server started. Bound to: 127.21.63.62:46883
I20260709 13:07:22.291100 22321 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.63.62:46883 every 8 connection(s)
I20260709 13:07:22.295054 22322 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:22.306411 22322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb83fe4dd9d6412ca3df972b6f2106cb: Bootstrap starting.
I20260709 13:07:22.311362 22322 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb83fe4dd9d6412ca3df972b6f2106cb: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:22.315659 22322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb83fe4dd9d6412ca3df972b6f2106cb: No bootstrap required, opened a new log
I20260709 13:07:22.317926 22322 raft_consensus.cc:359] T 00000000000000000000000000000000 P fb83fe4dd9d6412ca3df972b6f2106cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb83fe4dd9d6412ca3df972b6f2106cb" member_type: VOTER }
I20260709 13:07:22.318347 22322 raft_consensus.cc:385] T 00000000000000000000000000000000 P fb83fe4dd9d6412ca3df972b6f2106cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:22.318603 22322 raft_consensus.cc:740] T 00000000000000000000000000000000 P fb83fe4dd9d6412ca3df972b6f2106cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb83fe4dd9d6412ca3df972b6f2106cb, State: Initialized, Role: FOLLOWER
I20260709 13:07:22.319228 22322 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb83fe4dd9d6412ca3df972b6f2106cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb83fe4dd9d6412ca3df972b6f2106cb" member_type: VOTER }
I20260709 13:07:22.319691 22322 raft_consensus.cc:399] T 00000000000000000000000000000000 P fb83fe4dd9d6412ca3df972b6f2106cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:07:22.319909 22322 raft_consensus.cc:493] T 00000000000000000000000000000000 P fb83fe4dd9d6412ca3df972b6f2106cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:07:22.320240 22322 raft_consensus.cc:3060] T 00000000000000000000000000000000 P fb83fe4dd9d6412ca3df972b6f2106cb [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:22.324502 22322 raft_consensus.cc:515] T 00000000000000000000000000000000 P fb83fe4dd9d6412ca3df972b6f2106cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb83fe4dd9d6412ca3df972b6f2106cb" member_type: VOTER }
I20260709 13:07:22.325098 22322 leader_election.cc:304] T 00000000000000000000000000000000 P fb83fe4dd9d6412ca3df972b6f2106cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb83fe4dd9d6412ca3df972b6f2106cb; no voters: 
I20260709 13:07:22.326231 22322 leader_election.cc:290] T 00000000000000000000000000000000 P fb83fe4dd9d6412ca3df972b6f2106cb [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:07:22.326613 22325 raft_consensus.cc:2804] T 00000000000000000000000000000000 P fb83fe4dd9d6412ca3df972b6f2106cb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:22.327868 22325 raft_consensus.cc:697] T 00000000000000000000000000000000 P fb83fe4dd9d6412ca3df972b6f2106cb [term 1 LEADER]: Becoming Leader. State: Replica: fb83fe4dd9d6412ca3df972b6f2106cb, State: Running, Role: LEADER
I20260709 13:07:22.328547 22325 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb83fe4dd9d6412ca3df972b6f2106cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb83fe4dd9d6412ca3df972b6f2106cb" member_type: VOTER }
I20260709 13:07:22.329411 22322 sys_catalog.cc:565] T 00000000000000000000000000000000 P fb83fe4dd9d6412ca3df972b6f2106cb [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:07:22.333043 22326 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb83fe4dd9d6412ca3df972b6f2106cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb83fe4dd9d6412ca3df972b6f2106cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb83fe4dd9d6412ca3df972b6f2106cb" member_type: VOTER } }
I20260709 13:07:22.333083 22327 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb83fe4dd9d6412ca3df972b6f2106cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb83fe4dd9d6412ca3df972b6f2106cb. Latest consensus state: current_term: 1 leader_uuid: "fb83fe4dd9d6412ca3df972b6f2106cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb83fe4dd9d6412ca3df972b6f2106cb" member_type: VOTER } }
I20260709 13:07:22.333877 22326 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb83fe4dd9d6412ca3df972b6f2106cb [sys.catalog]: This master's current role is: LEADER
I20260709 13:07:22.334158 22327 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb83fe4dd9d6412ca3df972b6f2106cb [sys.catalog]: This master's current role is: LEADER
I20260709 13:07:22.338874 22331 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:07:22.343763 22331 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:07:22.347493 21756 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:07:22.352239 22331 catalog_manager.cc:1383] Generated new cluster ID: 82ae3e2ae71b4c5280fbbab992ebad2f
I20260709 13:07:22.352468 22331 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:07:22.375022 22331 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:07:22.376425 22331 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:07:22.392108 22331 catalog_manager.cc:6070] T 00000000000000000000000000000000 P fb83fe4dd9d6412ca3df972b6f2106cb: Generated new TSK 0
I20260709 13:07:22.392755 22331 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:07:22.413836 21756 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:22.419392 22343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:22.420789 22344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:22.422418 21756 server_base.cc:1061] running on GCE node
W20260709 13:07:22.422946 22346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:22.423823 21756 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:22.424069 21756 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:22.424203 21756 hybrid_clock.cc:648] HybridClock initialized: now 1783602442424192 us; error 0 us; skew 500 ppm
I20260709 13:07:22.424751 21756 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:22.427048 21756 webserver.cc:533] Webserver started at http://127.21.63.1:33161/ using document root <none> and password file <none>
I20260709 13:07:22.427544 21756 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:22.427717 21756 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:22.427964 21756 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:22.429087 21756 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602429420501-21756-0/minicluster-data/ts-0-root/instance:
uuid: "75d0a416460746daa0e458ca38689e3b"
format_stamp: "Formatted at 2026-07-09 13:07:22 on dist-test-slave-c6mt"
I20260709 13:07:22.434058 21756 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:07:22.437326 22351 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:22.438223 21756 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 13:07:22.438484 21756 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602429420501-21756-0/minicluster-data/ts-0-root
uuid: "75d0a416460746daa0e458ca38689e3b"
format_stamp: "Formatted at 2026-07-09 13:07:22 on dist-test-slave-c6mt"
I20260709 13:07:22.438740 21756 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602429420501-21756-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602429420501-21756-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602429420501-21756-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:22.452998 21756 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:22.454142 21756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:22.455530 21756 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:22.457679 21756 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:22.457852 21756 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:22.458158 21756 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:22.458310 21756 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:22.498958 21756 rpc_server.cc:307] RPC server started. Bound to: 127.21.63.1:37185
I20260709 13:07:22.499073 22413 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.63.1:37185 every 8 connection(s)
I20260709 13:07:22.503739 21756 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:22.510707 22418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:22.512425 22419 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:22.514115 22421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:22.515344 21756 server_base.cc:1061] running on GCE node
I20260709 13:07:22.516394 21756 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:22.516573 21756 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:22.516759 21756 hybrid_clock.cc:648] HybridClock initialized: now 1783602442516745 us; error 0 us; skew 500 ppm
I20260709 13:07:22.517509 21756 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:22.520309 21756 webserver.cc:533] Webserver started at http://127.21.63.2:37467/ using document root <none> and password file <none>
I20260709 13:07:22.520656 22414 heartbeater.cc:344] Connected to a master server at 127.21.63.62:46883
I20260709 13:07:22.520910 21756 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:22.521047 22414 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:22.521188 21756 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:22.521554 21756 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:22.521955 22414 heartbeater.cc:507] Master 127.21.63.62:46883 requested a full tablet report, sending...
I20260709 13:07:22.522710 21756 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602429420501-21756-0/minicluster-data/ts-1-root/instance:
uuid: "396253bba01d4eb382f848b21f804d02"
format_stamp: "Formatted at 2026-07-09 13:07:22 on dist-test-slave-c6mt"
I20260709 13:07:22.523907 22287 ts_manager.cc:194] Registered new tserver with Master: 75d0a416460746daa0e458ca38689e3b (127.21.63.1:37185)
I20260709 13:07:22.525628 22287 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34346
I20260709 13:07:22.527598 21756 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 13:07:22.530931 22426 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:22.531708 21756 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.002s
I20260709 13:07:22.532006 21756 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602429420501-21756-0/minicluster-data/ts-1-root
uuid: "396253bba01d4eb382f848b21f804d02"
format_stamp: "Formatted at 2026-07-09 13:07:22 on dist-test-slave-c6mt"
I20260709 13:07:22.532317 21756 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602429420501-21756-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602429420501-21756-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602429420501-21756-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:22.542949 21756 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:22.544188 21756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:22.545660 21756 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:22.547693 21756 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:22.547909 21756 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:22.548161 21756 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:22.548331 21756 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:22.589571 21756 rpc_server.cc:307] RPC server started. Bound to: 127.21.63.2:43161
I20260709 13:07:22.589696 22488 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.63.2:43161 every 8 connection(s)
I20260709 13:07:22.594515 21756 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:22.601512 22492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:22.603013 22493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:22.606077 22495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:22.606194 21756 server_base.cc:1061] running on GCE node
I20260709 13:07:22.607270 21756 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:22.607470 21756 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:22.607586 21756 hybrid_clock.cc:648] HybridClock initialized: now 1783602442607575 us; error 0 us; skew 500 ppm
I20260709 13:07:22.608207 21756 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:22.608940 22489 heartbeater.cc:344] Connected to a master server at 127.21.63.62:46883
I20260709 13:07:22.609308 22489 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:22.610082 22489 heartbeater.cc:507] Master 127.21.63.62:46883 requested a full tablet report, sending...
I20260709 13:07:22.610932 21756 webserver.cc:533] Webserver started at http://127.21.63.3:44547/ using document root <none> and password file <none>
I20260709 13:07:22.611567 21756 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:22.611764 21756 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:22.612035 21756 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:22.612000 22287 ts_manager.cc:194] Registered new tserver with Master: 396253bba01d4eb382f848b21f804d02 (127.21.63.2:43161)
I20260709 13:07:22.613314 21756 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602429420501-21756-0/minicluster-data/ts-2-root/instance:
uuid: "cb201644020643b485d68b0cd44e1b74"
format_stamp: "Formatted at 2026-07-09 13:07:22 on dist-test-slave-c6mt"
I20260709 13:07:22.613494 22287 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34358
I20260709 13:07:22.617740 21756 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.000s	sys 0.004s
I20260709 13:07:22.621138 22500 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:22.622036 21756 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:07:22.622334 21756 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602429420501-21756-0/minicluster-data/ts-2-root
uuid: "cb201644020643b485d68b0cd44e1b74"
format_stamp: "Formatted at 2026-07-09 13:07:22 on dist-test-slave-c6mt"
I20260709 13:07:22.622681 21756 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602429420501-21756-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602429420501-21756-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783602429420501-21756-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:22.637257 21756 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:22.638912 21756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:22.640545 21756 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:22.642792 21756 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:22.643033 21756 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:22.643270 21756 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:22.643450 21756 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:22.682313 21756 rpc_server.cc:307] RPC server started. Bound to: 127.21.63.3:41079
I20260709 13:07:22.682444 22562 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.63.3:41079 every 8 connection(s)
I20260709 13:07:22.697171 22563 heartbeater.cc:344] Connected to a master server at 127.21.63.62:46883
I20260709 13:07:22.697506 22563 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:22.698275 22563 heartbeater.cc:507] Master 127.21.63.62:46883 requested a full tablet report, sending...
I20260709 13:07:22.700340 22287 ts_manager.cc:194] Registered new tserver with Master: cb201644020643b485d68b0cd44e1b74 (127.21.63.3:41079)
I20260709 13:07:22.701314 21756 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015868978s
I20260709 13:07:22.701790 22287 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34362
I20260709 13:07:22.726449 22287 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34366:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:07:22.729746 22287 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:07:22.811985 22379 tablet_service.cc:1511] Processing CreateTablet for tablet c80fcb082aa04657987302b6666fe031 (DEFAULT_TABLE table=test-workload [id=d5c7a0fdea294d3da0e065fc0013c015]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:07:22.813602 22379 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c80fcb082aa04657987302b6666fe031. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:22.816448 22454 tablet_service.cc:1511] Processing CreateTablet for tablet c80fcb082aa04657987302b6666fe031 (DEFAULT_TABLE table=test-workload [id=d5c7a0fdea294d3da0e065fc0013c015]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:07:22.817800 22454 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c80fcb082aa04657987302b6666fe031. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:22.817955 22451 tablet_service.cc:1511] Processing CreateTablet for tablet 9b21d45dd8e84d6cba59e68212962e65 (DEFAULT_TABLE table=test-workload [id=d5c7a0fdea294d3da0e065fc0013c015]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:07:22.819347 22451 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b21d45dd8e84d6cba59e68212962e65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:22.819410 22377 tablet_service.cc:1511] Processing CreateTablet for tablet 101b2ac61327416ea7a64aa0261d2306 (DEFAULT_TABLE table=test-workload [id=d5c7a0fdea294d3da0e065fc0013c015]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:07:22.820973 22377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 101b2ac61327416ea7a64aa0261d2306. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:22.821135 22528 tablet_service.cc:1511] Processing CreateTablet for tablet c80fcb082aa04657987302b6666fe031 (DEFAULT_TABLE table=test-workload [id=d5c7a0fdea294d3da0e065fc0013c015]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:07:22.822429 22528 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c80fcb082aa04657987302b6666fe031. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:22.825371 22376 tablet_service.cc:1511] Processing CreateTablet for tablet 9b21d45dd8e84d6cba59e68212962e65 (DEFAULT_TABLE table=test-workload [id=d5c7a0fdea294d3da0e065fc0013c015]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:07:22.825829 22453 tablet_service.cc:1511] Processing CreateTablet for tablet c19b5a8350d74917bbc57cd93f2c23e2 (DEFAULT_TABLE table=test-workload [id=d5c7a0fdea294d3da0e065fc0013c015]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:07:22.826295 22527 tablet_service.cc:1511] Processing CreateTablet for tablet c19b5a8350d74917bbc57cd93f2c23e2 (DEFAULT_TABLE table=test-workload [id=d5c7a0fdea294d3da0e065fc0013c015]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:07:22.826851 22376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b21d45dd8e84d6cba59e68212962e65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:22.827101 22453 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c19b5a8350d74917bbc57cd93f2c23e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:22.827697 22527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c19b5a8350d74917bbc57cd93f2c23e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:22.829840 22375 tablet_service.cc:1511] Processing CreateTablet for tablet dd07de1f75c54b4bba2d31d81e28ce54 (DEFAULT_TABLE table=test-workload [id=d5c7a0fdea294d3da0e065fc0013c015]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:07:22.831301 22375 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd07de1f75c54b4bba2d31d81e28ce54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:22.832911 22452 tablet_service.cc:1511] Processing CreateTablet for tablet 101b2ac61327416ea7a64aa0261d2306 (DEFAULT_TABLE table=test-workload [id=d5c7a0fdea294d3da0e065fc0013c015]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:07:22.833199 22526 tablet_service.cc:1511] Processing CreateTablet for tablet 101b2ac61327416ea7a64aa0261d2306 (DEFAULT_TABLE table=test-workload [id=d5c7a0fdea294d3da0e065fc0013c015]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:07:22.817857 22378 tablet_service.cc:1511] Processing CreateTablet for tablet c19b5a8350d74917bbc57cd93f2c23e2 (DEFAULT_TABLE table=test-workload [id=d5c7a0fdea294d3da0e065fc0013c015]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:07:22.834254 22452 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 101b2ac61327416ea7a64aa0261d2306. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:22.834652 22526 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 101b2ac61327416ea7a64aa0261d2306. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:22.834776 22378 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c19b5a8350d74917bbc57cd93f2c23e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:22.839402 22525 tablet_service.cc:1511] Processing CreateTablet for tablet 9b21d45dd8e84d6cba59e68212962e65 (DEFAULT_TABLE table=test-workload [id=d5c7a0fdea294d3da0e065fc0013c015]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:07:22.841085 22525 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b21d45dd8e84d6cba59e68212962e65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:22.851317 22450 tablet_service.cc:1511] Processing CreateTablet for tablet dd07de1f75c54b4bba2d31d81e28ce54 (DEFAULT_TABLE table=test-workload [id=d5c7a0fdea294d3da0e065fc0013c015]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:07:22.854432 22449 tablet_service.cc:1511] Processing CreateTablet for tablet fcc967d7308b4cb3a5d93bc77dfa4681 (DEFAULT_TABLE table=test-workload [id=d5c7a0fdea294d3da0e065fc0013c015]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:07:22.852757 22450 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd07de1f75c54b4bba2d31d81e28ce54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:22.852769 22374 tablet_service.cc:1511] Processing CreateTablet for tablet fcc967d7308b4cb3a5d93bc77dfa4681 (DEFAULT_TABLE table=test-workload [id=d5c7a0fdea294d3da0e065fc0013c015]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:07:22.858423 22373 tablet_service.cc:1511] Processing CreateTablet for tablet aadfca82d29a4b5e89d49830fbb46b5e (DEFAULT_TABLE table=test-workload [id=d5c7a0fdea294d3da0e065fc0013c015]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:07:22.861766 22374 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcc967d7308b4cb3a5d93bc77dfa4681. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:22.862785 22373 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aadfca82d29a4b5e89d49830fbb46b5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:22.865022 22449 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcc967d7308b4cb3a5d93bc77dfa4681. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:22.875653 22578 tablet_bootstrap.cc:492] T c80fcb082aa04657987302b6666fe031 P 396253bba01d4eb382f848b21f804d02: Bootstrap starting.
I20260709 13:07:22.886381 22524 tablet_service.cc:1511] Processing CreateTablet for tablet dd07de1f75c54b4bba2d31d81e28ce54 (DEFAULT_TABLE table=test-workload [id=d5c7a0fdea294d3da0e065fc0013c015]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:07:22.888556 22524 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd07de1f75c54b4bba2d31d81e28ce54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:22.896426 22371 tablet_service.cc:1511] Processing CreateTablet for tablet a9e5138d2b904033978bb1ac8ab3d39f (DEFAULT_TABLE table=test-workload [id=d5c7a0fdea294d3da0e065fc0013c015]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:07:22.896553 22447 tablet_service.cc:1511] Processing CreateTablet for tablet 0eeeed324f544d639dab976cc4795bb3 (DEFAULT_TABLE table=test-workload [id=d5c7a0fdea294d3da0e065fc0013c015]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:07:22.898756 22446 tablet_service.cc:1511] Processing CreateTablet for tablet a9e5138d2b904033978bb1ac8ab3d39f (DEFAULT_TABLE table=test-workload [id=d5c7a0fdea294d3da0e065fc0013c015]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:07:22.890688 22377 tablet_service.cc:1511] Processing CreateTablet for tablet 5aba9ad2b64a42ad90bd6f5c11cf2272 (DEFAULT_TABLE table=test-workload [id=d5c7a0fdea294d3da0e065fc0013c015]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:07:22.896901 22448 tablet_service.cc:1511] Processing CreateTablet for tablet 5aba9ad2b64a42ad90bd6f5c11cf2272 (DEFAULT_TABLE table=test-workload [id=d5c7a0fdea294d3da0e065fc0013c015]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:07:22.902105 22372 tablet_service.cc:1511] Processing CreateTablet for tablet 0eeeed324f544d639dab976cc4795bb3 (DEFAULT_TABLE table=test-workload [id=d5c7a0fdea294d3da0e065fc0013c015]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:07:22.903434 22447 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0eeeed324f544d639dab976cc4795bb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:22.913733 22446 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9e5138d2b904033978bb1ac8ab3d39f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:22.920756 22448 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5aba9ad2b64a42ad90bd6f5c11cf2272. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:22.922081 22371 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9e5138d2b904033978bb1ac8ab3d39f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:22.931321 22377 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5aba9ad2b64a42ad90bd6f5c11cf2272. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:22.931788 22578 tablet_bootstrap.cc:654] T c80fcb082aa04657987302b6666fe031 P 396253bba01d4eb382f848b21f804d02: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:22.934434 22372 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0eeeed324f544d639dab976cc4795bb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:22.934281 22523 tablet_service.cc:1511] Processing CreateTablet for tablet fcc967d7308b4cb3a5d93bc77dfa4681 (DEFAULT_TABLE table=test-workload [id=d5c7a0fdea294d3da0e065fc0013c015]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:07:22.935771 22523 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fcc967d7308b4cb3a5d93bc77dfa4681. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:22.946436 22522 tablet_service.cc:1511] Processing CreateTablet for tablet aadfca82d29a4b5e89d49830fbb46b5e (DEFAULT_TABLE table=test-workload [id=d5c7a0fdea294d3da0e065fc0013c015]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:07:22.947861 22522 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aadfca82d29a4b5e89d49830fbb46b5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:22.953244 22521 tablet_service.cc:1511] Processing CreateTablet for tablet 5aba9ad2b64a42ad90bd6f5c11cf2272 (DEFAULT_TABLE table=test-workload [id=d5c7a0fdea294d3da0e065fc0013c015]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:07:22.953564 22520 tablet_service.cc:1511] Processing CreateTablet for tablet 0eeeed324f544d639dab976cc4795bb3 (DEFAULT_TABLE table=test-workload [id=d5c7a0fdea294d3da0e065fc0013c015]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:07:22.954037 22519 tablet_service.cc:1511] Processing CreateTablet for tablet a9e5138d2b904033978bb1ac8ab3d39f (DEFAULT_TABLE table=test-workload [id=d5c7a0fdea294d3da0e065fc0013c015]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:07:22.955322 22521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5aba9ad2b64a42ad90bd6f5c11cf2272. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:22.899204 22454 tablet_service.cc:1511] Processing CreateTablet for tablet aadfca82d29a4b5e89d49830fbb46b5e (DEFAULT_TABLE table=test-workload [id=d5c7a0fdea294d3da0e065fc0013c015]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:07:22.959566 22579 tablet_bootstrap.cc:492] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b: Bootstrap starting.
I20260709 13:07:22.959954 22520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0eeeed324f544d639dab976cc4795bb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:22.962355 22519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a9e5138d2b904033978bb1ac8ab3d39f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:23.024341 22454 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aadfca82d29a4b5e89d49830fbb46b5e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:23.054596 22580 tablet_bootstrap.cc:492] T fcc967d7308b4cb3a5d93bc77dfa4681 P cb201644020643b485d68b0cd44e1b74: Bootstrap starting.
I20260709 13:07:23.056394 22579 tablet_bootstrap.cc:654] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.064431 22580 tablet_bootstrap.cc:654] T fcc967d7308b4cb3a5d93bc77dfa4681 P cb201644020643b485d68b0cd44e1b74: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.069443 22578 tablet_bootstrap.cc:492] T c80fcb082aa04657987302b6666fe031 P 396253bba01d4eb382f848b21f804d02: No bootstrap required, opened a new log
I20260709 13:07:23.069870 22578 ts_tablet_manager.cc:1403] T c80fcb082aa04657987302b6666fe031 P 396253bba01d4eb382f848b21f804d02: Time spent bootstrapping tablet: real 0.195s	user 0.004s	sys 0.010s
I20260709 13:07:23.071928 22578 raft_consensus.cc:359] T c80fcb082aa04657987302b6666fe031 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.074761 22578 raft_consensus.cc:385] T c80fcb082aa04657987302b6666fe031 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.075064 22578 raft_consensus.cc:740] T c80fcb082aa04657987302b6666fe031 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 396253bba01d4eb382f848b21f804d02, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.075191 22579 tablet_bootstrap.cc:492] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b: No bootstrap required, opened a new log
I20260709 13:07:23.075539 22580 tablet_bootstrap.cc:492] T fcc967d7308b4cb3a5d93bc77dfa4681 P cb201644020643b485d68b0cd44e1b74: No bootstrap required, opened a new log
I20260709 13:07:23.075711 22579 ts_tablet_manager.cc:1403] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b: Time spent bootstrapping tablet: real 0.117s	user 0.013s	sys 0.011s
I20260709 13:07:23.075971 22580 ts_tablet_manager.cc:1403] T fcc967d7308b4cb3a5d93bc77dfa4681 P cb201644020643b485d68b0cd44e1b74: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.000s
I20260709 13:07:23.075836 22578 consensus_queue.cc:260] T c80fcb082aa04657987302b6666fe031 P 396253bba01d4eb382f848b21f804d02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.078048 22489 heartbeater.cc:499] Master 127.21.63.62:46883 was elected leader, sending a full tablet report...
I20260709 13:07:23.078547 22578 ts_tablet_manager.cc:1434] T c80fcb082aa04657987302b6666fe031 P 396253bba01d4eb382f848b21f804d02: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:07:23.078600 22580 raft_consensus.cc:359] T fcc967d7308b4cb3a5d93bc77dfa4681 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.079315 22580 raft_consensus.cc:385] T fcc967d7308b4cb3a5d93bc77dfa4681 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.079576 22580 raft_consensus.cc:740] T fcc967d7308b4cb3a5d93bc77dfa4681 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb201644020643b485d68b0cd44e1b74, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.079588 22578 tablet_bootstrap.cc:492] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02: Bootstrap starting.
I20260709 13:07:23.080313 22580 consensus_queue.cc:260] T fcc967d7308b4cb3a5d93bc77dfa4681 P cb201644020643b485d68b0cd44e1b74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.082448 22563 heartbeater.cc:499] Master 127.21.63.62:46883 was elected leader, sending a full tablet report...
I20260709 13:07:23.082973 22580 ts_tablet_manager.cc:1434] T fcc967d7308b4cb3a5d93bc77dfa4681 P cb201644020643b485d68b0cd44e1b74: Time spent starting tablet: real 0.007s	user 0.001s	sys 0.007s
I20260709 13:07:23.083444 22579 raft_consensus.cc:359] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.083935 22580 tablet_bootstrap.cc:492] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74: Bootstrap starting.
I20260709 13:07:23.084152 22579 raft_consensus.cc:385] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.084465 22579 raft_consensus.cc:740] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75d0a416460746daa0e458ca38689e3b, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.085374 22579 consensus_queue.cc:260] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.090466 22580 tablet_bootstrap.cc:654] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.095567 22578 tablet_bootstrap.cc:654] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.095686 22579 ts_tablet_manager.cc:1434] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b: Time spent starting tablet: real 0.020s	user 0.010s	sys 0.003s
I20260709 13:07:23.096385 22414 heartbeater.cc:499] Master 127.21.63.62:46883 was elected leader, sending a full tablet report...
I20260709 13:07:23.103814 22580 tablet_bootstrap.cc:492] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74: No bootstrap required, opened a new log
I20260709 13:07:23.104323 22580 ts_tablet_manager.cc:1403] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.007s
I20260709 13:07:23.106720 22578 tablet_bootstrap.cc:492] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02: No bootstrap required, opened a new log
I20260709 13:07:23.107213 22578 ts_tablet_manager.cc:1403] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.000s
I20260709 13:07:23.107177 22580 raft_consensus.cc:359] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.107889 22580 raft_consensus.cc:385] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.108220 22580 raft_consensus.cc:740] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb201644020643b485d68b0cd44e1b74, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.108897 22580 consensus_queue.cc:260] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.105479 22579 tablet_bootstrap.cc:492] T 101b2ac61327416ea7a64aa0261d2306 P 75d0a416460746daa0e458ca38689e3b: Bootstrap starting.
I20260709 13:07:23.110860 22580 ts_tablet_manager.cc:1434] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:07:23.110577 22578 raft_consensus.cc:359] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.111265 22578 raft_consensus.cc:385] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.111652 22578 raft_consensus.cc:740] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 396253bba01d4eb382f848b21f804d02, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.111917 22580 tablet_bootstrap.cc:492] T 9b21d45dd8e84d6cba59e68212962e65 P cb201644020643b485d68b0cd44e1b74: Bootstrap starting.
I20260709 13:07:23.112556 22578 consensus_queue.cc:260] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.114954 22578 ts_tablet_manager.cc:1434] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.003s
I20260709 13:07:23.115912 22578 tablet_bootstrap.cc:492] T a9e5138d2b904033978bb1ac8ab3d39f P 396253bba01d4eb382f848b21f804d02: Bootstrap starting.
I20260709 13:07:23.117257 22579 tablet_bootstrap.cc:654] T 101b2ac61327416ea7a64aa0261d2306 P 75d0a416460746daa0e458ca38689e3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.119138 22580 tablet_bootstrap.cc:654] T 9b21d45dd8e84d6cba59e68212962e65 P cb201644020643b485d68b0cd44e1b74: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.122764 22578 tablet_bootstrap.cc:654] T a9e5138d2b904033978bb1ac8ab3d39f P 396253bba01d4eb382f848b21f804d02: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.129561 22579 tablet_bootstrap.cc:492] T 101b2ac61327416ea7a64aa0261d2306 P 75d0a416460746daa0e458ca38689e3b: No bootstrap required, opened a new log
I20260709 13:07:23.129715 22584 raft_consensus.cc:493] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.130071 22579 ts_tablet_manager.cc:1403] T 101b2ac61327416ea7a64aa0261d2306 P 75d0a416460746daa0e458ca38689e3b: Time spent bootstrapping tablet: real 0.025s	user 0.014s	sys 0.004s
I20260709 13:07:23.130177 22584 raft_consensus.cc:515] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.132772 22584 leader_election.cc:290] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75d0a416460746daa0e458ca38689e3b (127.21.63.1:37185), cb201644020643b485d68b0cd44e1b74 (127.21.63.3:41079)
I20260709 13:07:23.142747 22579 raft_consensus.cc:359] T 101b2ac61327416ea7a64aa0261d2306 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.143522 22579 raft_consensus.cc:385] T 101b2ac61327416ea7a64aa0261d2306 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.143852 22579 raft_consensus.cc:740] T 101b2ac61327416ea7a64aa0261d2306 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75d0a416460746daa0e458ca38689e3b, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.144574 22579 consensus_queue.cc:260] T 101b2ac61327416ea7a64aa0261d2306 P 75d0a416460746daa0e458ca38689e3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.147174 22578 tablet_bootstrap.cc:492] T a9e5138d2b904033978bb1ac8ab3d39f P 396253bba01d4eb382f848b21f804d02: No bootstrap required, opened a new log
I20260709 13:07:23.147497 22580 tablet_bootstrap.cc:492] T 9b21d45dd8e84d6cba59e68212962e65 P cb201644020643b485d68b0cd44e1b74: No bootstrap required, opened a new log
I20260709 13:07:23.147682 22578 ts_tablet_manager.cc:1403] T a9e5138d2b904033978bb1ac8ab3d39f P 396253bba01d4eb382f848b21f804d02: Time spent bootstrapping tablet: real 0.032s	user 0.010s	sys 0.003s
I20260709 13:07:23.148252 22580 ts_tablet_manager.cc:1403] T 9b21d45dd8e84d6cba59e68212962e65 P cb201644020643b485d68b0cd44e1b74: Time spent bootstrapping tablet: real 0.037s	user 0.025s	sys 0.008s
I20260709 13:07:23.151026 22579 ts_tablet_manager.cc:1434] T 101b2ac61327416ea7a64aa0261d2306 P 75d0a416460746daa0e458ca38689e3b: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 13:07:23.150689 22578 raft_consensus.cc:359] T a9e5138d2b904033978bb1ac8ab3d39f P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.151407 22578 raft_consensus.cc:385] T a9e5138d2b904033978bb1ac8ab3d39f P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.151701 22578 raft_consensus.cc:740] T a9e5138d2b904033978bb1ac8ab3d39f P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 396253bba01d4eb382f848b21f804d02, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.151927 22579 tablet_bootstrap.cc:492] T 9b21d45dd8e84d6cba59e68212962e65 P 75d0a416460746daa0e458ca38689e3b: Bootstrap starting.
I20260709 13:07:23.151726 22580 raft_consensus.cc:359] T 9b21d45dd8e84d6cba59e68212962e65 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.152655 22580 raft_consensus.cc:385] T 9b21d45dd8e84d6cba59e68212962e65 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.153102 22580 raft_consensus.cc:740] T 9b21d45dd8e84d6cba59e68212962e65 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb201644020643b485d68b0cd44e1b74, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.152779 22578 consensus_queue.cc:260] T a9e5138d2b904033978bb1ac8ab3d39f P 396253bba01d4eb382f848b21f804d02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.154101 22580 consensus_queue.cc:260] T 9b21d45dd8e84d6cba59e68212962e65 P cb201644020643b485d68b0cd44e1b74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.156957 22578 ts_tablet_manager.cc:1434] T a9e5138d2b904033978bb1ac8ab3d39f P 396253bba01d4eb382f848b21f804d02: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.004s
I20260709 13:07:23.157879 22578 tablet_bootstrap.cc:492] T 9b21d45dd8e84d6cba59e68212962e65 P 396253bba01d4eb382f848b21f804d02: Bootstrap starting.
I20260709 13:07:23.161789 22580 ts_tablet_manager.cc:1434] T 9b21d45dd8e84d6cba59e68212962e65 P cb201644020643b485d68b0cd44e1b74: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:07:23.162858 22580 tablet_bootstrap.cc:492] T a9e5138d2b904033978bb1ac8ab3d39f P cb201644020643b485d68b0cd44e1b74: Bootstrap starting.
I20260709 13:07:23.164291 22578 tablet_bootstrap.cc:654] T 9b21d45dd8e84d6cba59e68212962e65 P 396253bba01d4eb382f848b21f804d02: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.170182 22580 tablet_bootstrap.cc:654] T a9e5138d2b904033978bb1ac8ab3d39f P cb201644020643b485d68b0cd44e1b74: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.172436 22579 tablet_bootstrap.cc:654] T 9b21d45dd8e84d6cba59e68212962e65 P 75d0a416460746daa0e458ca38689e3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.180889 22578 tablet_bootstrap.cc:492] T 9b21d45dd8e84d6cba59e68212962e65 P 396253bba01d4eb382f848b21f804d02: No bootstrap required, opened a new log
I20260709 13:07:23.181392 22578 ts_tablet_manager.cc:1403] T 9b21d45dd8e84d6cba59e68212962e65 P 396253bba01d4eb382f848b21f804d02: Time spent bootstrapping tablet: real 0.024s	user 0.003s	sys 0.014s
I20260709 13:07:23.184365 22578 raft_consensus.cc:359] T 9b21d45dd8e84d6cba59e68212962e65 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.187052 22578 raft_consensus.cc:385] T 9b21d45dd8e84d6cba59e68212962e65 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.187394 22578 raft_consensus.cc:740] T 9b21d45dd8e84d6cba59e68212962e65 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 396253bba01d4eb382f848b21f804d02, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.188117 22578 consensus_queue.cc:260] T 9b21d45dd8e84d6cba59e68212962e65 P 396253bba01d4eb382f848b21f804d02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.189932 22580 tablet_bootstrap.cc:492] T a9e5138d2b904033978bb1ac8ab3d39f P cb201644020643b485d68b0cd44e1b74: No bootstrap required, opened a new log
I20260709 13:07:23.190536 22580 ts_tablet_manager.cc:1403] T a9e5138d2b904033978bb1ac8ab3d39f P cb201644020643b485d68b0cd44e1b74: Time spent bootstrapping tablet: real 0.028s	user 0.013s	sys 0.011s
I20260709 13:07:23.194751 22580 raft_consensus.cc:359] T a9e5138d2b904033978bb1ac8ab3d39f P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.195775 22580 raft_consensus.cc:385] T a9e5138d2b904033978bb1ac8ab3d39f P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.196165 22580 raft_consensus.cc:740] T a9e5138d2b904033978bb1ac8ab3d39f P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb201644020643b485d68b0cd44e1b74, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.197183 22580 consensus_queue.cc:260] T a9e5138d2b904033978bb1ac8ab3d39f P cb201644020643b485d68b0cd44e1b74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.197609 22578 ts_tablet_manager.cc:1434] T 9b21d45dd8e84d6cba59e68212962e65 P 396253bba01d4eb382f848b21f804d02: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260709 13:07:23.197576 22538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c19b5a8350d74917bbc57cd93f2c23e2" candidate_uuid: "396253bba01d4eb382f848b21f804d02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb201644020643b485d68b0cd44e1b74" is_pre_election: true
I20260709 13:07:23.197803 22579 tablet_bootstrap.cc:492] T 9b21d45dd8e84d6cba59e68212962e65 P 75d0a416460746daa0e458ca38689e3b: No bootstrap required, opened a new log
I20260709 13:07:23.198549 22579 ts_tablet_manager.cc:1403] T 9b21d45dd8e84d6cba59e68212962e65 P 75d0a416460746daa0e458ca38689e3b: Time spent bootstrapping tablet: real 0.047s	user 0.013s	sys 0.001s
I20260709 13:07:23.199911 22580 ts_tablet_manager.cc:1434] T a9e5138d2b904033978bb1ac8ab3d39f P cb201644020643b485d68b0cd44e1b74: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:07:23.201027 22580 tablet_bootstrap.cc:492] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74: Bootstrap starting.
I20260709 13:07:23.201637 22579 raft_consensus.cc:359] T 9b21d45dd8e84d6cba59e68212962e65 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.202420 22579 raft_consensus.cc:385] T 9b21d45dd8e84d6cba59e68212962e65 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.202754 22579 raft_consensus.cc:740] T 9b21d45dd8e84d6cba59e68212962e65 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75d0a416460746daa0e458ca38689e3b, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.203619 22579 consensus_queue.cc:260] T 9b21d45dd8e84d6cba59e68212962e65 P 75d0a416460746daa0e458ca38689e3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.204384 22389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c19b5a8350d74917bbc57cd93f2c23e2" candidate_uuid: "396253bba01d4eb382f848b21f804d02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75d0a416460746daa0e458ca38689e3b" is_pre_election: true
I20260709 13:07:23.205317 22578 tablet_bootstrap.cc:492] T dd07de1f75c54b4bba2d31d81e28ce54 P 396253bba01d4eb382f848b21f804d02: Bootstrap starting.
I20260709 13:07:23.205180 22389 raft_consensus.cc:2468] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 396253bba01d4eb382f848b21f804d02 in term 0.
I20260709 13:07:23.206224 22579 ts_tablet_manager.cc:1434] T 9b21d45dd8e84d6cba59e68212962e65 P 75d0a416460746daa0e458ca38689e3b: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:07:23.207217 22579 tablet_bootstrap.cc:492] T a9e5138d2b904033978bb1ac8ab3d39f P 75d0a416460746daa0e458ca38689e3b: Bootstrap starting.
I20260709 13:07:23.208495 22580 tablet_bootstrap.cc:654] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.212150 22578 tablet_bootstrap.cc:654] T dd07de1f75c54b4bba2d31d81e28ce54 P 396253bba01d4eb382f848b21f804d02: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.213824 22579 tablet_bootstrap.cc:654] T a9e5138d2b904033978bb1ac8ab3d39f P 75d0a416460746daa0e458ca38689e3b: Neither blocks nor log segments found. Creating new log.
W20260709 13:07:23.219126 22429 leader_election.cc:343] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb201644020643b485d68b0cd44e1b74 (127.21.63.3:41079): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:23.219915 22429 leader_election.cc:304] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 396253bba01d4eb382f848b21f804d02, 75d0a416460746daa0e458ca38689e3b; no voters: cb201644020643b485d68b0cd44e1b74
I20260709 13:07:23.220929 22584 raft_consensus.cc:2804] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:23.221450 22584 raft_consensus.cc:493] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:23.221901 22584 raft_consensus.cc:3060] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.225322 22580 tablet_bootstrap.cc:492] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74: No bootstrap required, opened a new log
I20260709 13:07:23.225829 22580 ts_tablet_manager.cc:1403] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74: Time spent bootstrapping tablet: real 0.025s	user 0.014s	sys 0.003s
I20260709 13:07:23.228919 22580 raft_consensus.cc:359] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.229664 22580 raft_consensus.cc:385] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.229993 22580 raft_consensus.cc:740] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb201644020643b485d68b0cd44e1b74, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.230607 22580 consensus_queue.cc:260] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.231747 22584 raft_consensus.cc:515] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.233070 22580 ts_tablet_manager.cc:1434] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:07:23.234494 22580 tablet_bootstrap.cc:492] T c80fcb082aa04657987302b6666fe031 P cb201644020643b485d68b0cd44e1b74: Bootstrap starting.
I20260709 13:07:23.235144 22585 raft_consensus.cc:493] T 9b21d45dd8e84d6cba59e68212962e65 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.235661 22585 raft_consensus.cc:515] T 9b21d45dd8e84d6cba59e68212962e65 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.236397 22584 leader_election.cc:290] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02 [CANDIDATE]: Term 1 election: Requested vote from peers 75d0a416460746daa0e458ca38689e3b (127.21.63.1:37185), cb201644020643b485d68b0cd44e1b74 (127.21.63.3:41079)
I20260709 13:07:23.239562 22578 tablet_bootstrap.cc:492] T dd07de1f75c54b4bba2d31d81e28ce54 P 396253bba01d4eb382f848b21f804d02: No bootstrap required, opened a new log
I20260709 13:07:23.240223 22578 ts_tablet_manager.cc:1403] T dd07de1f75c54b4bba2d31d81e28ce54 P 396253bba01d4eb382f848b21f804d02: Time spent bootstrapping tablet: real 0.035s	user 0.009s	sys 0.007s
I20260709 13:07:23.240561 22580 tablet_bootstrap.cc:654] T c80fcb082aa04657987302b6666fe031 P cb201644020643b485d68b0cd44e1b74: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.241099 22585 leader_election.cc:290] T 9b21d45dd8e84d6cba59e68212962e65 P cb201644020643b485d68b0cd44e1b74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75d0a416460746daa0e458ca38689e3b (127.21.63.1:37185), 396253bba01d4eb382f848b21f804d02 (127.21.63.2:43161)
I20260709 13:07:23.244181 22578 raft_consensus.cc:359] T dd07de1f75c54b4bba2d31d81e28ce54 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.245241 22578 raft_consensus.cc:385] T dd07de1f75c54b4bba2d31d81e28ce54 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.245607 22578 raft_consensus.cc:740] T dd07de1f75c54b4bba2d31d81e28ce54 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 396253bba01d4eb382f848b21f804d02, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.246622 22584 raft_consensus.cc:493] T c80fcb082aa04657987302b6666fe031 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.247239 22584 raft_consensus.cc:515] T c80fcb082aa04657987302b6666fe031 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.249761 22584 leader_election.cc:290] T c80fcb082aa04657987302b6666fe031 P 396253bba01d4eb382f848b21f804d02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75d0a416460746daa0e458ca38689e3b (127.21.63.1:37185), cb201644020643b485d68b0cd44e1b74 (127.21.63.3:41079)
I20260709 13:07:23.251704 22578 consensus_queue.cc:260] T dd07de1f75c54b4bba2d31d81e28ce54 P 396253bba01d4eb382f848b21f804d02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.252336 22389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c19b5a8350d74917bbc57cd93f2c23e2" candidate_uuid: "396253bba01d4eb382f848b21f804d02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75d0a416460746daa0e458ca38689e3b"
I20260709 13:07:23.252862 22580 tablet_bootstrap.cc:492] T c80fcb082aa04657987302b6666fe031 P cb201644020643b485d68b0cd44e1b74: No bootstrap required, opened a new log
I20260709 13:07:23.252939 22389 raft_consensus.cc:3060] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.253369 22580 ts_tablet_manager.cc:1403] T c80fcb082aa04657987302b6666fe031 P cb201644020643b485d68b0cd44e1b74: Time spent bootstrapping tablet: real 0.019s	user 0.006s	sys 0.011s
I20260709 13:07:23.256069 22538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c19b5a8350d74917bbc57cd93f2c23e2" candidate_uuid: "396253bba01d4eb382f848b21f804d02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb201644020643b485d68b0cd44e1b74"
I20260709 13:07:23.256268 22580 raft_consensus.cc:359] T c80fcb082aa04657987302b6666fe031 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.257010 22580 raft_consensus.cc:385] T c80fcb082aa04657987302b6666fe031 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.257298 22580 raft_consensus.cc:740] T c80fcb082aa04657987302b6666fe031 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb201644020643b485d68b0cd44e1b74, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.258024 22580 consensus_queue.cc:260] T c80fcb082aa04657987302b6666fe031 P cb201644020643b485d68b0cd44e1b74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.258762 22388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80fcb082aa04657987302b6666fe031" candidate_uuid: "396253bba01d4eb382f848b21f804d02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75d0a416460746daa0e458ca38689e3b" is_pre_election: true
I20260709 13:07:23.260289 22580 ts_tablet_manager.cc:1434] T c80fcb082aa04657987302b6666fe031 P cb201644020643b485d68b0cd44e1b74: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:07:23.260715 22389 raft_consensus.cc:2468] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 396253bba01d4eb382f848b21f804d02 in term 1.
I20260709 13:07:23.252935 22579 tablet_bootstrap.cc:492] T a9e5138d2b904033978bb1ac8ab3d39f P 75d0a416460746daa0e458ca38689e3b: No bootstrap required, opened a new log
I20260709 13:07:23.261817 22579 ts_tablet_manager.cc:1403] T a9e5138d2b904033978bb1ac8ab3d39f P 75d0a416460746daa0e458ca38689e3b: Time spent bootstrapping tablet: real 0.055s	user 0.033s	sys 0.012s
I20260709 13:07:23.261294 22580 tablet_bootstrap.cc:492] T aadfca82d29a4b5e89d49830fbb46b5e P cb201644020643b485d68b0cd44e1b74: Bootstrap starting.
I20260709 13:07:23.262630 22578 ts_tablet_manager.cc:1434] T dd07de1f75c54b4bba2d31d81e28ce54 P 396253bba01d4eb382f848b21f804d02: Time spent starting tablet: real 0.022s	user 0.003s	sys 0.004s
I20260709 13:07:23.265055 22579 raft_consensus.cc:359] T a9e5138d2b904033978bb1ac8ab3d39f P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.265859 22579 raft_consensus.cc:385] T a9e5138d2b904033978bb1ac8ab3d39f P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.266177 22579 raft_consensus.cc:740] T a9e5138d2b904033978bb1ac8ab3d39f P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75d0a416460746daa0e458ca38689e3b, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.266981 22579 consensus_queue.cc:260] T a9e5138d2b904033978bb1ac8ab3d39f P 75d0a416460746daa0e458ca38689e3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.268210 22578 tablet_bootstrap.cc:492] T 101b2ac61327416ea7a64aa0261d2306 P 396253bba01d4eb382f848b21f804d02: Bootstrap starting.
I20260709 13:07:23.269505 22580 tablet_bootstrap.cc:654] T aadfca82d29a4b5e89d49830fbb46b5e P cb201644020643b485d68b0cd44e1b74: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.274380 22579 ts_tablet_manager.cc:1434] T a9e5138d2b904033978bb1ac8ab3d39f P 75d0a416460746daa0e458ca38689e3b: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.004s
I20260709 13:07:23.275489 22579 tablet_bootstrap.cc:492] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b: Bootstrap starting.
I20260709 13:07:23.276789 22578 tablet_bootstrap.cc:654] T 101b2ac61327416ea7a64aa0261d2306 P 396253bba01d4eb382f848b21f804d02: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.278349 22538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80fcb082aa04657987302b6666fe031" candidate_uuid: "396253bba01d4eb382f848b21f804d02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb201644020643b485d68b0cd44e1b74" is_pre_election: true
I20260709 13:07:23.279047 22538 raft_consensus.cc:2468] T c80fcb082aa04657987302b6666fe031 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 396253bba01d4eb382f848b21f804d02 in term 0.
W20260709 13:07:23.280714 22429 leader_election.cc:343] T c80fcb082aa04657987302b6666fe031 P 396253bba01d4eb382f848b21f804d02 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75d0a416460746daa0e458ca38689e3b (127.21.63.1:37185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:23.282315 22429 leader_election.cc:304] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 396253bba01d4eb382f848b21f804d02, 75d0a416460746daa0e458ca38689e3b; no voters: 
I20260709 13:07:23.286554 22584 raft_consensus.cc:2804] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 13:07:23.286684 22429 leader_election.cc:343] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cb201644020643b485d68b0cd44e1b74 (127.21.63.3:41079): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:23.288482 22429 leader_election.cc:304] T c80fcb082aa04657987302b6666fe031 P 396253bba01d4eb382f848b21f804d02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 396253bba01d4eb382f848b21f804d02, cb201644020643b485d68b0cd44e1b74; no voters: 75d0a416460746daa0e458ca38689e3b
I20260709 13:07:23.287035 22579 tablet_bootstrap.cc:654] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.292079 22584 raft_consensus.cc:697] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02 [term 1 LEADER]: Becoming Leader. State: Replica: 396253bba01d4eb382f848b21f804d02, State: Running, Role: LEADER
I20260709 13:07:23.293243 22584 consensus_queue.cc:237] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.295706 22606 raft_consensus.cc:2804] T c80fcb082aa04657987302b6666fe031 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:23.296170 22606 raft_consensus.cc:493] T c80fcb082aa04657987302b6666fe031 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:23.296700 22606 raft_consensus.cc:3060] T c80fcb082aa04657987302b6666fe031 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.300338 22578 tablet_bootstrap.cc:492] T 101b2ac61327416ea7a64aa0261d2306 P 396253bba01d4eb382f848b21f804d02: No bootstrap required, opened a new log
I20260709 13:07:23.300855 22578 ts_tablet_manager.cc:1403] T 101b2ac61327416ea7a64aa0261d2306 P 396253bba01d4eb382f848b21f804d02: Time spent bootstrapping tablet: real 0.033s	user 0.015s	sys 0.006s
I20260709 13:07:23.303782 22578 raft_consensus.cc:359] T 101b2ac61327416ea7a64aa0261d2306 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.304446 22389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b21d45dd8e84d6cba59e68212962e65" candidate_uuid: "cb201644020643b485d68b0cd44e1b74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75d0a416460746daa0e458ca38689e3b" is_pre_election: true
I20260709 13:07:23.305094 22389 raft_consensus.cc:2468] T 9b21d45dd8e84d6cba59e68212962e65 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb201644020643b485d68b0cd44e1b74 in term 0.
I20260709 13:07:23.306483 22503 leader_election.cc:304] T 9b21d45dd8e84d6cba59e68212962e65 P cb201644020643b485d68b0cd44e1b74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75d0a416460746daa0e458ca38689e3b, cb201644020643b485d68b0cd44e1b74; no voters: 
I20260709 13:07:23.307367 22585 raft_consensus.cc:2804] T 9b21d45dd8e84d6cba59e68212962e65 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:23.307452 22580 tablet_bootstrap.cc:492] T aadfca82d29a4b5e89d49830fbb46b5e P cb201644020643b485d68b0cd44e1b74: No bootstrap required, opened a new log
I20260709 13:07:23.307758 22585 raft_consensus.cc:493] T 9b21d45dd8e84d6cba59e68212962e65 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:23.307837 22579 tablet_bootstrap.cc:492] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b: No bootstrap required, opened a new log
I20260709 13:07:23.308285 22580 ts_tablet_manager.cc:1403] T aadfca82d29a4b5e89d49830fbb46b5e P cb201644020643b485d68b0cd44e1b74: Time spent bootstrapping tablet: real 0.047s	user 0.020s	sys 0.010s
I20260709 13:07:23.308507 22579 ts_tablet_manager.cc:1403] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b: Time spent bootstrapping tablet: real 0.033s	user 0.007s	sys 0.009s
I20260709 13:07:23.308228 22585 raft_consensus.cc:3060] T 9b21d45dd8e84d6cba59e68212962e65 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.311556 22586 raft_consensus.cc:493] T a9e5138d2b904033978bb1ac8ab3d39f P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.311842 22464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b21d45dd8e84d6cba59e68212962e65" candidate_uuid: "cb201644020643b485d68b0cd44e1b74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "396253bba01d4eb382f848b21f804d02" is_pre_election: true
I20260709 13:07:23.312122 22586 raft_consensus.cc:515] T a9e5138d2b904033978bb1ac8ab3d39f P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.312525 22464 raft_consensus.cc:2468] T 9b21d45dd8e84d6cba59e68212962e65 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb201644020643b485d68b0cd44e1b74 in term 0.
I20260709 13:07:23.315367 22586 leader_election.cc:290] T a9e5138d2b904033978bb1ac8ab3d39f P 75d0a416460746daa0e458ca38689e3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 396253bba01d4eb382f848b21f804d02 (127.21.63.2:43161), cb201644020643b485d68b0cd44e1b74 (127.21.63.3:41079)
I20260709 13:07:23.311555 22579 raft_consensus.cc:359] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.316537 22579 raft_consensus.cc:385] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.316854 22579 raft_consensus.cc:740] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75d0a416460746daa0e458ca38689e3b, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.317677 22579 consensus_queue.cc:260] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.318229 22606 raft_consensus.cc:515] T c80fcb082aa04657987302b6666fe031 P 396253bba01d4eb382f848b21f804d02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.318830 22585 raft_consensus.cc:515] T 9b21d45dd8e84d6cba59e68212962e65 P cb201644020643b485d68b0cd44e1b74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.321058 22606 leader_election.cc:290] T c80fcb082aa04657987302b6666fe031 P 396253bba01d4eb382f848b21f804d02 [CANDIDATE]: Term 1 election: Requested vote from peers 75d0a416460746daa0e458ca38689e3b (127.21.63.1:37185), cb201644020643b485d68b0cd44e1b74 (127.21.63.3:41079)
I20260709 13:07:23.312913 22580 raft_consensus.cc:359] T aadfca82d29a4b5e89d49830fbb46b5e P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.321734 22388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80fcb082aa04657987302b6666fe031" candidate_uuid: "396253bba01d4eb382f848b21f804d02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75d0a416460746daa0e458ca38689e3b"
I20260709 13:07:23.322038 22580 raft_consensus.cc:385] T aadfca82d29a4b5e89d49830fbb46b5e P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.322296 22464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b21d45dd8e84d6cba59e68212962e65" candidate_uuid: "cb201644020643b485d68b0cd44e1b74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "396253bba01d4eb382f848b21f804d02"
I20260709 13:07:23.322362 22580 raft_consensus.cc:740] T aadfca82d29a4b5e89d49830fbb46b5e P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb201644020643b485d68b0cd44e1b74, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.322988 22464 raft_consensus.cc:3060] T 9b21d45dd8e84d6cba59e68212962e65 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.323521 22580 consensus_queue.cc:260] T aadfca82d29a4b5e89d49830fbb46b5e P cb201644020643b485d68b0cd44e1b74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.324405 22538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80fcb082aa04657987302b6666fe031" candidate_uuid: "396253bba01d4eb382f848b21f804d02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb201644020643b485d68b0cd44e1b74"
I20260709 13:07:23.325017 22538 raft_consensus.cc:3060] T c80fcb082aa04657987302b6666fe031 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.325943 22389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b21d45dd8e84d6cba59e68212962e65" candidate_uuid: "cb201644020643b485d68b0cd44e1b74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75d0a416460746daa0e458ca38689e3b"
I20260709 13:07:23.326478 22389 raft_consensus.cc:3060] T 9b21d45dd8e84d6cba59e68212962e65 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.330713 22464 raft_consensus.cc:2468] T 9b21d45dd8e84d6cba59e68212962e65 P 396253bba01d4eb382f848b21f804d02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb201644020643b485d68b0cd44e1b74 in term 1.
W20260709 13:07:23.331689 22429 leader_election.cc:343] T c80fcb082aa04657987302b6666fe031 P 396253bba01d4eb382f848b21f804d02 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 75d0a416460746daa0e458ca38689e3b (127.21.63.1:37185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:23.331897 22503 leader_election.cc:304] T 9b21d45dd8e84d6cba59e68212962e65 P cb201644020643b485d68b0cd44e1b74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 396253bba01d4eb382f848b21f804d02, cb201644020643b485d68b0cd44e1b74; no voters: 
I20260709 13:07:23.334735 22578 raft_consensus.cc:385] T 101b2ac61327416ea7a64aa0261d2306 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.335276 22578 raft_consensus.cc:740] T 101b2ac61327416ea7a64aa0261d2306 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 396253bba01d4eb382f848b21f804d02, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.336059 22578 consensus_queue.cc:260] T 101b2ac61327416ea7a64aa0261d2306 P 396253bba01d4eb382f848b21f804d02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.337539 22389 raft_consensus.cc:2468] T 9b21d45dd8e84d6cba59e68212962e65 P 75d0a416460746daa0e458ca38689e3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb201644020643b485d68b0cd44e1b74 in term 1.
I20260709 13:07:23.338299 22578 ts_tablet_manager.cc:1434] T 101b2ac61327416ea7a64aa0261d2306 P 396253bba01d4eb382f848b21f804d02: Time spent starting tablet: real 0.037s	user 0.005s	sys 0.000s
I20260709 13:07:23.338924 22538 raft_consensus.cc:2468] T c80fcb082aa04657987302b6666fe031 P cb201644020643b485d68b0cd44e1b74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 396253bba01d4eb382f848b21f804d02 in term 1.
I20260709 13:07:23.339746 22578 tablet_bootstrap.cc:492] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 396253bba01d4eb382f848b21f804d02: Bootstrap starting.
I20260709 13:07:23.332823 22585 leader_election.cc:290] T 9b21d45dd8e84d6cba59e68212962e65 P cb201644020643b485d68b0cd44e1b74 [CANDIDATE]: Term 1 election: Requested vote from peers 75d0a416460746daa0e458ca38689e3b (127.21.63.1:37185), 396253bba01d4eb382f848b21f804d02 (127.21.63.2:43161)
I20260709 13:07:23.346747 22612 raft_consensus.cc:493] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.346886 22579 ts_tablet_manager.cc:1434] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b: Time spent starting tablet: real 0.038s	user 0.006s	sys 0.000s
I20260709 13:07:23.351215 22579 tablet_bootstrap.cc:492] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b: Bootstrap starting.
I20260709 13:07:23.348079 22429 leader_election.cc:304] T c80fcb082aa04657987302b6666fe031 P 396253bba01d4eb382f848b21f804d02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 396253bba01d4eb382f848b21f804d02, cb201644020643b485d68b0cd44e1b74; no voters: 75d0a416460746daa0e458ca38689e3b
W20260709 13:07:23.349254 22335 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:23.349624 22612 raft_consensus.cc:515] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.358444 22585 raft_consensus.cc:2804] T 9b21d45dd8e84d6cba59e68212962e65 P cb201644020643b485d68b0cd44e1b74 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:23.358987 22585 raft_consensus.cc:697] T 9b21d45dd8e84d6cba59e68212962e65 P cb201644020643b485d68b0cd44e1b74 [term 1 LEADER]: Becoming Leader. State: Replica: cb201644020643b485d68b0cd44e1b74, State: Running, Role: LEADER
I20260709 13:07:23.359902 22585 consensus_queue.cc:237] T 9b21d45dd8e84d6cba59e68212962e65 P cb201644020643b485d68b0cd44e1b74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.356233 22606 raft_consensus.cc:2804] T c80fcb082aa04657987302b6666fe031 P 396253bba01d4eb382f848b21f804d02 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:23.369067 22586 raft_consensus.cc:493] T 101b2ac61327416ea7a64aa0261d2306 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.369801 22580 ts_tablet_manager.cc:1434] T aadfca82d29a4b5e89d49830fbb46b5e P cb201644020643b485d68b0cd44e1b74: Time spent starting tablet: real 0.061s	user 0.011s	sys 0.012s
I20260709 13:07:23.369627 22586 raft_consensus.cc:515] T 101b2ac61327416ea7a64aa0261d2306 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.370846 22580 tablet_bootstrap.cc:492] T c19b5a8350d74917bbc57cd93f2c23e2 P cb201644020643b485d68b0cd44e1b74: Bootstrap starting.
I20260709 13:07:23.373013 22586 leader_election.cc:290] T 101b2ac61327416ea7a64aa0261d2306 P 75d0a416460746daa0e458ca38689e3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 396253bba01d4eb382f848b21f804d02 (127.21.63.2:43161), cb201644020643b485d68b0cd44e1b74 (127.21.63.3:41079)
I20260709 13:07:23.374262 22286 catalog_manager.cc:5697] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 396253bba01d4eb382f848b21f804d02 (127.21.63.2). New cstate: current_term: 1 leader_uuid: "396253bba01d4eb382f848b21f804d02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:23.371555 22606 raft_consensus.cc:697] T c80fcb082aa04657987302b6666fe031 P 396253bba01d4eb382f848b21f804d02 [term 1 LEADER]: Becoming Leader. State: Replica: 396253bba01d4eb382f848b21f804d02, State: Running, Role: LEADER
I20260709 13:07:23.379477 22606 consensus_queue.cc:237] T c80fcb082aa04657987302b6666fe031 P 396253bba01d4eb382f848b21f804d02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.381727 22578 tablet_bootstrap.cc:654] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 396253bba01d4eb382f848b21f804d02: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.382457 22580 tablet_bootstrap.cc:654] T c19b5a8350d74917bbc57cd93f2c23e2 P cb201644020643b485d68b0cd44e1b74: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.386166 22389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd07de1f75c54b4bba2d31d81e28ce54" candidate_uuid: "cb201644020643b485d68b0cd44e1b74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75d0a416460746daa0e458ca38689e3b" is_pre_election: true
I20260709 13:07:23.387825 22389 raft_consensus.cc:2468] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb201644020643b485d68b0cd44e1b74 in term 0.
I20260709 13:07:23.387837 22464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd07de1f75c54b4bba2d31d81e28ce54" candidate_uuid: "cb201644020643b485d68b0cd44e1b74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "396253bba01d4eb382f848b21f804d02" is_pre_election: true
I20260709 13:07:23.388934 22464 raft_consensus.cc:2468] T dd07de1f75c54b4bba2d31d81e28ce54 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb201644020643b485d68b0cd44e1b74 in term 0.
I20260709 13:07:23.389477 22503 leader_election.cc:304] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75d0a416460746daa0e458ca38689e3b, cb201644020643b485d68b0cd44e1b74; no voters: 
I20260709 13:07:23.393306 22606 raft_consensus.cc:493] T a9e5138d2b904033978bb1ac8ab3d39f P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.393824 22606 raft_consensus.cc:515] T a9e5138d2b904033978bb1ac8ab3d39f P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.394134 22610 raft_consensus.cc:2804] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:23.394642 22610 raft_consensus.cc:493] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:23.395077 22610 raft_consensus.cc:3060] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.395870 22579 tablet_bootstrap.cc:654] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.384531 22612 leader_election.cc:290] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75d0a416460746daa0e458ca38689e3b (127.21.63.1:37185), 396253bba01d4eb382f848b21f804d02 (127.21.63.2:43161)
I20260709 13:07:23.399518 22538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e5138d2b904033978bb1ac8ab3d39f" candidate_uuid: "396253bba01d4eb382f848b21f804d02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb201644020643b485d68b0cd44e1b74" is_pre_election: true
I20260709 13:07:23.400159 22538 raft_consensus.cc:2468] T a9e5138d2b904033978bb1ac8ab3d39f P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 396253bba01d4eb382f848b21f804d02 in term 0.
I20260709 13:07:23.401836 22464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e5138d2b904033978bb1ac8ab3d39f" candidate_uuid: "75d0a416460746daa0e458ca38689e3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "396253bba01d4eb382f848b21f804d02" is_pre_election: true
I20260709 13:07:23.402072 22463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "101b2ac61327416ea7a64aa0261d2306" candidate_uuid: "75d0a416460746daa0e458ca38689e3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "396253bba01d4eb382f848b21f804d02" is_pre_election: true
I20260709 13:07:23.402499 22464 raft_consensus.cc:2468] T a9e5138d2b904033978bb1ac8ab3d39f P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75d0a416460746daa0e458ca38689e3b in term 0.
I20260709 13:07:23.403806 22463 raft_consensus.cc:2468] T 101b2ac61327416ea7a64aa0261d2306 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75d0a416460746daa0e458ca38689e3b in term 0.
I20260709 13:07:23.396111 22606 leader_election.cc:290] T a9e5138d2b904033978bb1ac8ab3d39f P 396253bba01d4eb382f848b21f804d02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75d0a416460746daa0e458ca38689e3b (127.21.63.1:37185), cb201644020643b485d68b0cd44e1b74 (127.21.63.3:41079)
I20260709 13:07:23.405647 22389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e5138d2b904033978bb1ac8ab3d39f" candidate_uuid: "396253bba01d4eb382f848b21f804d02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75d0a416460746daa0e458ca38689e3b" is_pre_election: true
I20260709 13:07:23.408058 22429 leader_election.cc:304] T a9e5138d2b904033978bb1ac8ab3d39f P 396253bba01d4eb382f848b21f804d02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 396253bba01d4eb382f848b21f804d02, cb201644020643b485d68b0cd44e1b74; no voters: 
I20260709 13:07:23.408788 22389 raft_consensus.cc:2468] T a9e5138d2b904033978bb1ac8ab3d39f P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 396253bba01d4eb382f848b21f804d02 in term 0.
I20260709 13:07:23.409533 22606 raft_consensus.cc:2804] T a9e5138d2b904033978bb1ac8ab3d39f P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:23.409922 22606 raft_consensus.cc:493] T a9e5138d2b904033978bb1ac8ab3d39f P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:23.410249 22606 raft_consensus.cc:3060] T a9e5138d2b904033978bb1ac8ab3d39f P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.412122 22354 leader_election.cc:304] T a9e5138d2b904033978bb1ac8ab3d39f P 75d0a416460746daa0e458ca38689e3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 396253bba01d4eb382f848b21f804d02, 75d0a416460746daa0e458ca38689e3b; no voters: 
I20260709 13:07:23.419006 22586 raft_consensus.cc:2804] T a9e5138d2b904033978bb1ac8ab3d39f P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:23.419109 22354 leader_election.cc:304] T 101b2ac61327416ea7a64aa0261d2306 P 75d0a416460746daa0e458ca38689e3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 396253bba01d4eb382f848b21f804d02, 75d0a416460746daa0e458ca38689e3b; no voters: 
I20260709 13:07:23.419682 22586 raft_consensus.cc:493] T a9e5138d2b904033978bb1ac8ab3d39f P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:23.421152 22586 raft_consensus.cc:3060] T a9e5138d2b904033978bb1ac8ab3d39f P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.425463 22580 tablet_bootstrap.cc:492] T c19b5a8350d74917bbc57cd93f2c23e2 P cb201644020643b485d68b0cd44e1b74: No bootstrap required, opened a new log
I20260709 13:07:23.426002 22580 ts_tablet_manager.cc:1403] T c19b5a8350d74917bbc57cd93f2c23e2 P cb201644020643b485d68b0cd44e1b74: Time spent bootstrapping tablet: real 0.055s	user 0.013s	sys 0.006s
I20260709 13:07:23.428879 22580 raft_consensus.cc:359] T c19b5a8350d74917bbc57cd93f2c23e2 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.429677 22580 raft_consensus.cc:385] T c19b5a8350d74917bbc57cd93f2c23e2 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.429848 22578 tablet_bootstrap.cc:492] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 396253bba01d4eb382f848b21f804d02: No bootstrap required, opened a new log
I20260709 13:07:23.429975 22580 raft_consensus.cc:740] T c19b5a8350d74917bbc57cd93f2c23e2 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb201644020643b485d68b0cd44e1b74, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.430435 22578 ts_tablet_manager.cc:1403] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 396253bba01d4eb382f848b21f804d02: Time spent bootstrapping tablet: real 0.091s	user 0.014s	sys 0.008s
I20260709 13:07:23.432493 22610 raft_consensus.cc:515] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.430914 22580 consensus_queue.cc:260] T c19b5a8350d74917bbc57cd93f2c23e2 P cb201644020643b485d68b0cd44e1b74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.434105 22284 catalog_manager.cc:5697] T 9b21d45dd8e84d6cba59e68212962e65 P cb201644020643b485d68b0cd44e1b74 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb201644020643b485d68b0cd44e1b74 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "cb201644020643b485d68b0cd44e1b74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:23.436156 22580 ts_tablet_manager.cc:1434] T c19b5a8350d74917bbc57cd93f2c23e2 P cb201644020643b485d68b0cd44e1b74: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.001s
I20260709 13:07:23.438925 22389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd07de1f75c54b4bba2d31d81e28ce54" candidate_uuid: "cb201644020643b485d68b0cd44e1b74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75d0a416460746daa0e458ca38689e3b"
I20260709 13:07:23.439733 22389 raft_consensus.cc:3060] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.444545 22580 tablet_bootstrap.cc:492] T 0eeeed324f544d639dab976cc4795bb3 P cb201644020643b485d68b0cd44e1b74: Bootstrap starting.
I20260709 13:07:23.434574 22578 raft_consensus.cc:359] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.446904 22538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e5138d2b904033978bb1ac8ab3d39f" candidate_uuid: "75d0a416460746daa0e458ca38689e3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb201644020643b485d68b0cd44e1b74" is_pre_election: true
I20260709 13:07:23.447584 22538 raft_consensus.cc:2468] T a9e5138d2b904033978bb1ac8ab3d39f P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75d0a416460746daa0e458ca38689e3b in term 0.
I20260709 13:07:23.447712 22606 raft_consensus.cc:515] T a9e5138d2b904033978bb1ac8ab3d39f P 396253bba01d4eb382f848b21f804d02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.449576 22389 raft_consensus.cc:2468] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb201644020643b485d68b0cd44e1b74 in term 1.
I20260709 13:07:23.445573 22578 raft_consensus.cc:385] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.450608 22578 raft_consensus.cc:740] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 396253bba01d4eb382f848b21f804d02, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.450881 22503 leader_election.cc:304] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75d0a416460746daa0e458ca38689e3b, cb201644020643b485d68b0cd44e1b74; no voters: 
I20260709 13:07:23.451474 22586 raft_consensus.cc:515] T a9e5138d2b904033978bb1ac8ab3d39f P 75d0a416460746daa0e458ca38689e3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.451972 22585 raft_consensus.cc:2804] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:23.451531 22578 consensus_queue.cc:260] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 396253bba01d4eb382f848b21f804d02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.452560 22585 raft_consensus.cc:697] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [term 1 LEADER]: Becoming Leader. State: Replica: cb201644020643b485d68b0cd44e1b74, State: Running, Role: LEADER
I20260709 13:07:23.452842 22538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e5138d2b904033978bb1ac8ab3d39f" candidate_uuid: "396253bba01d4eb382f848b21f804d02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb201644020643b485d68b0cd44e1b74"
I20260709 13:07:23.454586 22538 raft_consensus.cc:3060] T a9e5138d2b904033978bb1ac8ab3d39f P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.454418 22580 tablet_bootstrap.cc:654] T 0eeeed324f544d639dab976cc4795bb3 P cb201644020643b485d68b0cd44e1b74: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.453387 22389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e5138d2b904033978bb1ac8ab3d39f" candidate_uuid: "396253bba01d4eb382f848b21f804d02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75d0a416460746daa0e458ca38689e3b"
I20260709 13:07:23.454010 22606 leader_election.cc:290] T a9e5138d2b904033978bb1ac8ab3d39f P 396253bba01d4eb382f848b21f804d02 [CANDIDATE]: Term 1 election: Requested vote from peers 75d0a416460746daa0e458ca38689e3b (127.21.63.1:37185), cb201644020643b485d68b0cd44e1b74 (127.21.63.3:41079)
I20260709 13:07:23.456269 22537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "101b2ac61327416ea7a64aa0261d2306" candidate_uuid: "75d0a416460746daa0e458ca38689e3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb201644020643b485d68b0cd44e1b74" is_pre_election: true
I20260709 13:07:23.455096 22612 raft_consensus.cc:493] T fcc967d7308b4cb3a5d93bc77dfa4681 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.454972 22585 consensus_queue.cc:237] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.461171 22612 raft_consensus.cc:515] T fcc967d7308b4cb3a5d93bc77dfa4681 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.462965 22538 raft_consensus.cc:2468] T a9e5138d2b904033978bb1ac8ab3d39f P cb201644020643b485d68b0cd44e1b74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 396253bba01d4eb382f848b21f804d02 in term 1.
I20260709 13:07:23.454797 22586 leader_election.cc:290] T a9e5138d2b904033978bb1ac8ab3d39f P 75d0a416460746daa0e458ca38689e3b [CANDIDATE]: Term 1 election: Requested vote from peers 396253bba01d4eb382f848b21f804d02 (127.21.63.2:43161), cb201644020643b485d68b0cd44e1b74 (127.21.63.3:41079)
I20260709 13:07:23.458055 22389 raft_consensus.cc:2393] T a9e5138d2b904033978bb1ac8ab3d39f P 75d0a416460746daa0e458ca38689e3b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 396253bba01d4eb382f848b21f804d02 in current term 1: Already voted for candidate 75d0a416460746daa0e458ca38689e3b in this term.
I20260709 13:07:23.464617 22463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e5138d2b904033978bb1ac8ab3d39f" candidate_uuid: "75d0a416460746daa0e458ca38689e3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "396253bba01d4eb382f848b21f804d02"
I20260709 13:07:23.464763 22429 leader_election.cc:304] T a9e5138d2b904033978bb1ac8ab3d39f P 396253bba01d4eb382f848b21f804d02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 396253bba01d4eb382f848b21f804d02, cb201644020643b485d68b0cd44e1b74; no voters: 
I20260709 13:07:23.465404 22463 raft_consensus.cc:2393] T a9e5138d2b904033978bb1ac8ab3d39f P 396253bba01d4eb382f848b21f804d02 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 75d0a416460746daa0e458ca38689e3b in current term 1: Already voted for candidate 396253bba01d4eb382f848b21f804d02 in this term.
I20260709 13:07:23.464082 22610 leader_election.cc:290] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [CANDIDATE]: Term 1 election: Requested vote from peers 75d0a416460746daa0e458ca38689e3b (127.21.63.1:37185), 396253bba01d4eb382f848b21f804d02 (127.21.63.2:43161)
I20260709 13:07:23.466480 22464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd07de1f75c54b4bba2d31d81e28ce54" candidate_uuid: "cb201644020643b485d68b0cd44e1b74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "396253bba01d4eb382f848b21f804d02"
W20260709 13:07:23.468362 22354 leader_election.cc:343] T 101b2ac61327416ea7a64aa0261d2306 P 75d0a416460746daa0e458ca38689e3b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cb201644020643b485d68b0cd44e1b74 (127.21.63.3:41079): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:23.466404 22536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e5138d2b904033978bb1ac8ab3d39f" candidate_uuid: "75d0a416460746daa0e458ca38689e3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb201644020643b485d68b0cd44e1b74"
I20260709 13:07:23.467782 22388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcc967d7308b4cb3a5d93bc77dfa4681" candidate_uuid: "cb201644020643b485d68b0cd44e1b74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75d0a416460746daa0e458ca38689e3b" is_pre_election: true
I20260709 13:07:23.468243 22584 raft_consensus.cc:2804] T a9e5138d2b904033978bb1ac8ab3d39f P 396253bba01d4eb382f848b21f804d02 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:23.469975 22579 tablet_bootstrap.cc:492] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b: No bootstrap required, opened a new log
I20260709 13:07:23.470388 22584 raft_consensus.cc:697] T a9e5138d2b904033978bb1ac8ab3d39f P 396253bba01d4eb382f848b21f804d02 [term 1 LEADER]: Becoming Leader. State: Replica: 396253bba01d4eb382f848b21f804d02, State: Running, Role: LEADER
I20260709 13:07:23.471330 22584 consensus_queue.cc:237] T a9e5138d2b904033978bb1ac8ab3d39f P 396253bba01d4eb382f848b21f804d02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.468796 22464 raft_consensus.cc:3060] T dd07de1f75c54b4bba2d31d81e28ce54 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.470528 22579 ts_tablet_manager.cc:1403] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b: Time spent bootstrapping tablet: real 0.120s	user 0.013s	sys 0.010s
I20260709 13:07:23.469893 22536 raft_consensus.cc:2393] T a9e5138d2b904033978bb1ac8ab3d39f P cb201644020643b485d68b0cd44e1b74 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 75d0a416460746daa0e458ca38689e3b in current term 1: Already voted for candidate 396253bba01d4eb382f848b21f804d02 in this term.
W20260709 13:07:23.477874 22503 leader_election.cc:343] T fcc967d7308b4cb3a5d93bc77dfa4681 P cb201644020643b485d68b0cd44e1b74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75d0a416460746daa0e458ca38689e3b (127.21.63.1:37185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:23.468763 22578 ts_tablet_manager.cc:1434] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 396253bba01d4eb382f848b21f804d02: Time spent starting tablet: real 0.038s	user 0.009s	sys 0.001s
I20260709 13:07:23.478804 22354 leader_election.cc:304] T a9e5138d2b904033978bb1ac8ab3d39f P 75d0a416460746daa0e458ca38689e3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75d0a416460746daa0e458ca38689e3b; no voters: 396253bba01d4eb382f848b21f804d02, cb201644020643b485d68b0cd44e1b74
I20260709 13:07:23.480373 22578 tablet_bootstrap.cc:492] T fcc967d7308b4cb3a5d93bc77dfa4681 P 396253bba01d4eb382f848b21f804d02: Bootstrap starting.
I20260709 13:07:23.480511 22586 raft_consensus.cc:2749] T a9e5138d2b904033978bb1ac8ab3d39f P 75d0a416460746daa0e458ca38689e3b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:07:23.481022 22464 raft_consensus.cc:2468] T dd07de1f75c54b4bba2d31d81e28ce54 P 396253bba01d4eb382f848b21f804d02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb201644020643b485d68b0cd44e1b74 in term 1.
I20260709 13:07:23.482458 22621 raft_consensus.cc:2804] T 101b2ac61327416ea7a64aa0261d2306 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:23.482860 22621 raft_consensus.cc:493] T 101b2ac61327416ea7a64aa0261d2306 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:23.483349 22610 raft_consensus.cc:493] T c19b5a8350d74917bbc57cd93f2c23e2 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.483994 22610 raft_consensus.cc:515] T c19b5a8350d74917bbc57cd93f2c23e2 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.484957 22612 leader_election.cc:290] T fcc967d7308b4cb3a5d93bc77dfa4681 P cb201644020643b485d68b0cd44e1b74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75d0a416460746daa0e458ca38689e3b (127.21.63.1:37185), 396253bba01d4eb382f848b21f804d02 (127.21.63.2:43161)
I20260709 13:07:23.487000 22464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcc967d7308b4cb3a5d93bc77dfa4681" candidate_uuid: "cb201644020643b485d68b0cd44e1b74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "396253bba01d4eb382f848b21f804d02" is_pre_election: true
I20260709 13:07:23.487319 22388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c19b5a8350d74917bbc57cd93f2c23e2" candidate_uuid: "cb201644020643b485d68b0cd44e1b74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75d0a416460746daa0e458ca38689e3b" is_pre_election: true
I20260709 13:07:23.487843 22621 raft_consensus.cc:3060] T 101b2ac61327416ea7a64aa0261d2306 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.488096 22388 raft_consensus.cc:2393] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb201644020643b485d68b0cd44e1b74 in current term 1: Already voted for candidate 396253bba01d4eb382f848b21f804d02 in this term.
W20260709 13:07:23.488291 22503 leader_election.cc:343] T fcc967d7308b4cb3a5d93bc77dfa4681 P cb201644020643b485d68b0cd44e1b74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 396253bba01d4eb382f848b21f804d02 (127.21.63.2:43161): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:23.489678 22610 leader_election.cc:290] T c19b5a8350d74917bbc57cd93f2c23e2 P cb201644020643b485d68b0cd44e1b74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75d0a416460746daa0e458ca38689e3b (127.21.63.1:37185), 396253bba01d4eb382f848b21f804d02 (127.21.63.2:43161)
I20260709 13:07:23.488768 22503 leader_election.cc:304] T fcc967d7308b4cb3a5d93bc77dfa4681 P cb201644020643b485d68b0cd44e1b74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb201644020643b485d68b0cd44e1b74; no voters: 396253bba01d4eb382f848b21f804d02, 75d0a416460746daa0e458ca38689e3b
I20260709 13:07:23.489073 22579 raft_consensus.cc:359] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.492244 22579 raft_consensus.cc:385] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.492700 22579 raft_consensus.cc:740] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75d0a416460746daa0e458ca38689e3b, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.493194 22464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c19b5a8350d74917bbc57cd93f2c23e2" candidate_uuid: "cb201644020643b485d68b0cd44e1b74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "396253bba01d4eb382f848b21f804d02" is_pre_election: true
I20260709 13:07:23.494067 22585 raft_consensus.cc:2749] T fcc967d7308b4cb3a5d93bc77dfa4681 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:23.497520 22580 tablet_bootstrap.cc:492] T 0eeeed324f544d639dab976cc4795bb3 P cb201644020643b485d68b0cd44e1b74: No bootstrap required, opened a new log
I20260709 13:07:23.497376 22503 leader_election.cc:304] T c19b5a8350d74917bbc57cd93f2c23e2 P cb201644020643b485d68b0cd44e1b74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb201644020643b485d68b0cd44e1b74; no voters: 396253bba01d4eb382f848b21f804d02, 75d0a416460746daa0e458ca38689e3b
I20260709 13:07:23.498075 22580 ts_tablet_manager.cc:1403] T 0eeeed324f544d639dab976cc4795bb3 P cb201644020643b485d68b0cd44e1b74: Time spent bootstrapping tablet: real 0.059s	user 0.013s	sys 0.002s
I20260709 13:07:23.499634 22585 raft_consensus.cc:3060] T c19b5a8350d74917bbc57cd93f2c23e2 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.500960 22580 raft_consensus.cc:359] T 0eeeed324f544d639dab976cc4795bb3 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.501641 22580 raft_consensus.cc:385] T 0eeeed324f544d639dab976cc4795bb3 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.502063 22578 tablet_bootstrap.cc:654] T fcc967d7308b4cb3a5d93bc77dfa4681 P 396253bba01d4eb382f848b21f804d02: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.501973 22580 raft_consensus.cc:740] T 0eeeed324f544d639dab976cc4795bb3 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb201644020643b485d68b0cd44e1b74, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.501984 22621 raft_consensus.cc:515] T 101b2ac61327416ea7a64aa0261d2306 P 75d0a416460746daa0e458ca38689e3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.503072 22580 consensus_queue.cc:260] T 0eeeed324f544d639dab976cc4795bb3 P cb201644020643b485d68b0cd44e1b74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.495151 22579 consensus_queue.cc:260] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.509214 22579 ts_tablet_manager.cc:1434] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b: Time spent starting tablet: real 0.036s	user 0.009s	sys 0.001s
I20260709 13:07:23.510187 22579 tablet_bootstrap.cc:492] T aadfca82d29a4b5e89d49830fbb46b5e P 75d0a416460746daa0e458ca38689e3b: Bootstrap starting.
I20260709 13:07:23.504657 22621 leader_election.cc:290] T 101b2ac61327416ea7a64aa0261d2306 P 75d0a416460746daa0e458ca38689e3b [CANDIDATE]: Term 1 election: Requested vote from peers 396253bba01d4eb382f848b21f804d02 (127.21.63.2:43161), cb201644020643b485d68b0cd44e1b74 (127.21.63.3:41079)
I20260709 13:07:23.518671 22579 tablet_bootstrap.cc:654] T aadfca82d29a4b5e89d49830fbb46b5e P 75d0a416460746daa0e458ca38689e3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.519093 22585 raft_consensus.cc:2749] T c19b5a8350d74917bbc57cd93f2c23e2 P cb201644020643b485d68b0cd44e1b74 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:23.523914 22285 catalog_manager.cc:5697] T c80fcb082aa04657987302b6666fe031 P 396253bba01d4eb382f848b21f804d02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 396253bba01d4eb382f848b21f804d02 (127.21.63.2). New cstate: current_term: 1 leader_uuid: "396253bba01d4eb382f848b21f804d02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:23.506168 22537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "101b2ac61327416ea7a64aa0261d2306" candidate_uuid: "75d0a416460746daa0e458ca38689e3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb201644020643b485d68b0cd44e1b74"
I20260709 13:07:23.506840 22464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "101b2ac61327416ea7a64aa0261d2306" candidate_uuid: "75d0a416460746daa0e458ca38689e3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "396253bba01d4eb382f848b21f804d02"
I20260709 13:07:23.530014 22464 raft_consensus.cc:3060] T 101b2ac61327416ea7a64aa0261d2306 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.527452 22283 catalog_manager.cc:5697] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb201644020643b485d68b0cd44e1b74 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "cb201644020643b485d68b0cd44e1b74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } health_report { overall_health: HEALTHY } } }
W20260709 13:07:23.533182 22354 leader_election.cc:343] T 101b2ac61327416ea7a64aa0261d2306 P 75d0a416460746daa0e458ca38689e3b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cb201644020643b485d68b0cd44e1b74 (127.21.63.3:41079): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:23.537698 22285 catalog_manager.cc:5697] T a9e5138d2b904033978bb1ac8ab3d39f P 396253bba01d4eb382f848b21f804d02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 396253bba01d4eb382f848b21f804d02 (127.21.63.2). New cstate: current_term: 1 leader_uuid: "396253bba01d4eb382f848b21f804d02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:23.539438 22585 raft_consensus.cc:493] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.540027 22585 raft_consensus.cc:515] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.505267 22580 ts_tablet_manager.cc:1434] T 0eeeed324f544d639dab976cc4795bb3 P cb201644020643b485d68b0cd44e1b74: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:07:23.542413 22585 leader_election.cc:290] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75d0a416460746daa0e458ca38689e3b (127.21.63.1:37185), 396253bba01d4eb382f848b21f804d02 (127.21.63.2:43161)
I20260709 13:07:23.543512 22388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5aba9ad2b64a42ad90bd6f5c11cf2272" candidate_uuid: "cb201644020643b485d68b0cd44e1b74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75d0a416460746daa0e458ca38689e3b" is_pre_election: true
I20260709 13:07:23.543991 22463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5aba9ad2b64a42ad90bd6f5c11cf2272" candidate_uuid: "cb201644020643b485d68b0cd44e1b74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "396253bba01d4eb382f848b21f804d02" is_pre_election: true
W20260709 13:07:23.545003 22503 leader_election.cc:343] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75d0a416460746daa0e458ca38689e3b (127.21.63.1:37185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:23.544602 22463 raft_consensus.cc:2468] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb201644020643b485d68b0cd44e1b74 in term 0.
I20260709 13:07:23.547027 22503 leader_election.cc:304] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 396253bba01d4eb382f848b21f804d02, cb201644020643b485d68b0cd44e1b74; no voters: 75d0a416460746daa0e458ca38689e3b
I20260709 13:07:23.548274 22585 raft_consensus.cc:2804] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:23.549504 22585 raft_consensus.cc:493] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:23.550797 22585 raft_consensus.cc:3060] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.549196 22580 tablet_bootstrap.cc:492] T 101b2ac61327416ea7a64aa0261d2306 P cb201644020643b485d68b0cd44e1b74: Bootstrap starting.
I20260709 13:07:23.539551 22464 raft_consensus.cc:2468] T 101b2ac61327416ea7a64aa0261d2306 P 396253bba01d4eb382f848b21f804d02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75d0a416460746daa0e458ca38689e3b in term 1.
I20260709 13:07:23.565913 22354 leader_election.cc:304] T 101b2ac61327416ea7a64aa0261d2306 P 75d0a416460746daa0e458ca38689e3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 396253bba01d4eb382f848b21f804d02, 75d0a416460746daa0e458ca38689e3b; no voters: cb201644020643b485d68b0cd44e1b74
I20260709 13:07:23.566744 22621 raft_consensus.cc:2804] T 101b2ac61327416ea7a64aa0261d2306 P 75d0a416460746daa0e458ca38689e3b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:23.567479 22621 raft_consensus.cc:697] T 101b2ac61327416ea7a64aa0261d2306 P 75d0a416460746daa0e458ca38689e3b [term 1 LEADER]: Becoming Leader. State: Replica: 75d0a416460746daa0e458ca38689e3b, State: Running, Role: LEADER
I20260709 13:07:23.568652 22621 consensus_queue.cc:237] T 101b2ac61327416ea7a64aa0261d2306 P 75d0a416460746daa0e458ca38689e3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.572394 22585 raft_consensus.cc:515] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.575307 22388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5aba9ad2b64a42ad90bd6f5c11cf2272" candidate_uuid: "cb201644020643b485d68b0cd44e1b74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75d0a416460746daa0e458ca38689e3b"
W20260709 13:07:23.576736 22503 leader_election.cc:343] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 75d0a416460746daa0e458ca38689e3b (127.21.63.1:37185): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:23.578892 22464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5aba9ad2b64a42ad90bd6f5c11cf2272" candidate_uuid: "cb201644020643b485d68b0cd44e1b74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "396253bba01d4eb382f848b21f804d02"
I20260709 13:07:23.579468 22464 raft_consensus.cc:3060] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.587307 22578 tablet_bootstrap.cc:492] T fcc967d7308b4cb3a5d93bc77dfa4681 P 396253bba01d4eb382f848b21f804d02: No bootstrap required, opened a new log
I20260709 13:07:23.587760 22464 raft_consensus.cc:2468] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 396253bba01d4eb382f848b21f804d02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb201644020643b485d68b0cd44e1b74 in term 1.
I20260709 13:07:23.587994 22578 ts_tablet_manager.cc:1403] T fcc967d7308b4cb3a5d93bc77dfa4681 P 396253bba01d4eb382f848b21f804d02: Time spent bootstrapping tablet: real 0.108s	user 0.036s	sys 0.029s
I20260709 13:07:23.588935 22503 leader_election.cc:304] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 396253bba01d4eb382f848b21f804d02, cb201644020643b485d68b0cd44e1b74; no voters: 75d0a416460746daa0e458ca38689e3b
I20260709 13:07:23.589918 22610 raft_consensus.cc:2804] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:23.590428 22610 raft_consensus.cc:697] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [term 1 LEADER]: Becoming Leader. State: Replica: cb201644020643b485d68b0cd44e1b74, State: Running, Role: LEADER
I20260709 13:07:23.591275 22610 consensus_queue.cc:237] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.592228 22580 tablet_bootstrap.cc:654] T 101b2ac61327416ea7a64aa0261d2306 P cb201644020643b485d68b0cd44e1b74: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.584388 22585 leader_election.cc:290] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [CANDIDATE]: Term 1 election: Requested vote from peers 75d0a416460746daa0e458ca38689e3b (127.21.63.1:37185), 396253bba01d4eb382f848b21f804d02 (127.21.63.2:43161)
I20260709 13:07:23.593986 22578 raft_consensus.cc:359] T fcc967d7308b4cb3a5d93bc77dfa4681 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.594906 22578 raft_consensus.cc:385] T fcc967d7308b4cb3a5d93bc77dfa4681 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.595255 22578 raft_consensus.cc:740] T fcc967d7308b4cb3a5d93bc77dfa4681 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 396253bba01d4eb382f848b21f804d02, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.596199 22578 consensus_queue.cc:260] T fcc967d7308b4cb3a5d93bc77dfa4681 P 396253bba01d4eb382f848b21f804d02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.598603 22578 ts_tablet_manager.cc:1434] T fcc967d7308b4cb3a5d93bc77dfa4681 P 396253bba01d4eb382f848b21f804d02: Time spent starting tablet: real 0.010s	user 0.010s	sys 0.000s
I20260709 13:07:23.599510 22578 tablet_bootstrap.cc:492] T 0eeeed324f544d639dab976cc4795bb3 P 396253bba01d4eb382f848b21f804d02: Bootstrap starting.
I20260709 13:07:23.601056 22579 tablet_bootstrap.cc:492] T aadfca82d29a4b5e89d49830fbb46b5e P 75d0a416460746daa0e458ca38689e3b: No bootstrap required, opened a new log
I20260709 13:07:23.601521 22579 ts_tablet_manager.cc:1403] T aadfca82d29a4b5e89d49830fbb46b5e P 75d0a416460746daa0e458ca38689e3b: Time spent bootstrapping tablet: real 0.092s	user 0.047s	sys 0.038s
I20260709 13:07:23.605511 22578 tablet_bootstrap.cc:654] T 0eeeed324f544d639dab976cc4795bb3 P 396253bba01d4eb382f848b21f804d02: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.605742 22579 raft_consensus.cc:359] T aadfca82d29a4b5e89d49830fbb46b5e P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.606523 22579 raft_consensus.cc:385] T aadfca82d29a4b5e89d49830fbb46b5e P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.606837 22579 raft_consensus.cc:740] T aadfca82d29a4b5e89d49830fbb46b5e P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75d0a416460746daa0e458ca38689e3b, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.607190 22580 tablet_bootstrap.cc:492] T 101b2ac61327416ea7a64aa0261d2306 P cb201644020643b485d68b0cd44e1b74: No bootstrap required, opened a new log
I20260709 13:07:23.605338 22621 raft_consensus.cc:493] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.607635 22579 consensus_queue.cc:260] T aadfca82d29a4b5e89d49830fbb46b5e P 75d0a416460746daa0e458ca38689e3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.607957 22621 raft_consensus.cc:515] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.610769 22579 ts_tablet_manager.cc:1434] T aadfca82d29a4b5e89d49830fbb46b5e P 75d0a416460746daa0e458ca38689e3b: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:07:23.611049 22464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcc967d7308b4cb3a5d93bc77dfa4681" candidate_uuid: "75d0a416460746daa0e458ca38689e3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "396253bba01d4eb382f848b21f804d02" is_pre_election: true
I20260709 13:07:23.611716 22464 raft_consensus.cc:2468] T fcc967d7308b4cb3a5d93bc77dfa4681 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75d0a416460746daa0e458ca38689e3b in term 0.
I20260709 13:07:23.612141 22579 tablet_bootstrap.cc:492] T c80fcb082aa04657987302b6666fe031 P 75d0a416460746daa0e458ca38689e3b: Bootstrap starting.
I20260709 13:07:23.613106 22354 leader_election.cc:304] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 396253bba01d4eb382f848b21f804d02, 75d0a416460746daa0e458ca38689e3b; no voters: 
I20260709 13:07:23.607643 22580 ts_tablet_manager.cc:1403] T 101b2ac61327416ea7a64aa0261d2306 P cb201644020643b485d68b0cd44e1b74: Time spent bootstrapping tablet: real 0.059s	user 0.009s	sys 0.004s
I20260709 13:07:23.613907 22586 raft_consensus.cc:2804] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:23.614393 22586 raft_consensus.cc:493] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:23.614805 22586 raft_consensus.cc:3060] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.616689 22578 tablet_bootstrap.cc:492] T 0eeeed324f544d639dab976cc4795bb3 P 396253bba01d4eb382f848b21f804d02: No bootstrap required, opened a new log
I20260709 13:07:23.617131 22578 ts_tablet_manager.cc:1403] T 0eeeed324f544d639dab976cc4795bb3 P 396253bba01d4eb382f848b21f804d02: Time spent bootstrapping tablet: real 0.018s	user 0.013s	sys 0.003s
I20260709 13:07:23.617632 22621 leader_election.cc:290] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 396253bba01d4eb382f848b21f804d02 (127.21.63.2:43161), cb201644020643b485d68b0cd44e1b74 (127.21.63.3:41079)
I20260709 13:07:23.618366 22580 raft_consensus.cc:359] T 101b2ac61327416ea7a64aa0261d2306 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.619073 22580 raft_consensus.cc:385] T 101b2ac61327416ea7a64aa0261d2306 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.619351 22580 raft_consensus.cc:740] T 101b2ac61327416ea7a64aa0261d2306 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb201644020643b485d68b0cd44e1b74, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.619473 22537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcc967d7308b4cb3a5d93bc77dfa4681" candidate_uuid: "75d0a416460746daa0e458ca38689e3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb201644020643b485d68b0cd44e1b74" is_pre_election: true
I20260709 13:07:23.619712 22579 tablet_bootstrap.cc:654] T c80fcb082aa04657987302b6666fe031 P 75d0a416460746daa0e458ca38689e3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.619660 22578 raft_consensus.cc:359] T 0eeeed324f544d639dab976cc4795bb3 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.620426 22578 raft_consensus.cc:385] T 0eeeed324f544d639dab976cc4795bb3 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.620117 22537 raft_consensus.cc:2468] T fcc967d7308b4cb3a5d93bc77dfa4681 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75d0a416460746daa0e458ca38689e3b in term 0.
I20260709 13:07:23.620079 22580 consensus_queue.cc:260] T 101b2ac61327416ea7a64aa0261d2306 P cb201644020643b485d68b0cd44e1b74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.620811 22578 raft_consensus.cc:740] T 0eeeed324f544d639dab976cc4795bb3 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 396253bba01d4eb382f848b21f804d02, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.621788 22578 consensus_queue.cc:260] T 0eeeed324f544d639dab976cc4795bb3 P 396253bba01d4eb382f848b21f804d02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.622814 22580 ts_tablet_manager.cc:1434] T 101b2ac61327416ea7a64aa0261d2306 P cb201644020643b485d68b0cd44e1b74: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:07:23.627566 22578 ts_tablet_manager.cc:1434] T 0eeeed324f544d639dab976cc4795bb3 P 396253bba01d4eb382f848b21f804d02: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:07:23.627941 22586 raft_consensus.cc:515] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.612253 22285 catalog_manager.cc:5697] T 101b2ac61327416ea7a64aa0261d2306 P 75d0a416460746daa0e458ca38689e3b reported cstate change: term changed from 0 to 1, leader changed from <none> to 75d0a416460746daa0e458ca38689e3b (127.21.63.1). New cstate: current_term: 1 leader_uuid: "75d0a416460746daa0e458ca38689e3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:23.629076 22578 tablet_bootstrap.cc:492] T aadfca82d29a4b5e89d49830fbb46b5e P 396253bba01d4eb382f848b21f804d02: Bootstrap starting.
I20260709 13:07:23.632421 22586 leader_election.cc:290] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [CANDIDATE]: Term 1 election: Requested vote from peers 396253bba01d4eb382f848b21f804d02 (127.21.63.2:43161), cb201644020643b485d68b0cd44e1b74 (127.21.63.3:41079)
I20260709 13:07:23.635813 22464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcc967d7308b4cb3a5d93bc77dfa4681" candidate_uuid: "75d0a416460746daa0e458ca38689e3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "396253bba01d4eb382f848b21f804d02"
I20260709 13:07:23.636432 22464 raft_consensus.cc:3060] T fcc967d7308b4cb3a5d93bc77dfa4681 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.638019 22578 tablet_bootstrap.cc:654] T aadfca82d29a4b5e89d49830fbb46b5e P 396253bba01d4eb382f848b21f804d02: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.639066 22579 tablet_bootstrap.cc:492] T c80fcb082aa04657987302b6666fe031 P 75d0a416460746daa0e458ca38689e3b: No bootstrap required, opened a new log
I20260709 13:07:23.639508 22579 ts_tablet_manager.cc:1403] T c80fcb082aa04657987302b6666fe031 P 75d0a416460746daa0e458ca38689e3b: Time spent bootstrapping tablet: real 0.028s	user 0.010s	sys 0.015s
I20260709 13:07:23.641812 22537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcc967d7308b4cb3a5d93bc77dfa4681" candidate_uuid: "75d0a416460746daa0e458ca38689e3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb201644020643b485d68b0cd44e1b74"
I20260709 13:07:23.642431 22537 raft_consensus.cc:3060] T fcc967d7308b4cb3a5d93bc77dfa4681 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.642561 22579 raft_consensus.cc:359] T c80fcb082aa04657987302b6666fe031 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.643499 22579 raft_consensus.cc:385] T c80fcb082aa04657987302b6666fe031 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.643819 22579 raft_consensus.cc:740] T c80fcb082aa04657987302b6666fe031 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75d0a416460746daa0e458ca38689e3b, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.644361 22464 raft_consensus.cc:2468] T fcc967d7308b4cb3a5d93bc77dfa4681 P 396253bba01d4eb382f848b21f804d02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75d0a416460746daa0e458ca38689e3b in term 1.
I20260709 13:07:23.644807 22579 consensus_queue.cc:260] T c80fcb082aa04657987302b6666fe031 P 75d0a416460746daa0e458ca38689e3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.645591 22354 leader_election.cc:304] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 396253bba01d4eb382f848b21f804d02, 75d0a416460746daa0e458ca38689e3b; no voters: 
I20260709 13:07:23.647289 22579 ts_tablet_manager.cc:1434] T c80fcb082aa04657987302b6666fe031 P 75d0a416460746daa0e458ca38689e3b: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:07:23.648144 22579 tablet_bootstrap.cc:492] T 0eeeed324f544d639dab976cc4795bb3 P 75d0a416460746daa0e458ca38689e3b: Bootstrap starting.
I20260709 13:07:23.648725 22578 tablet_bootstrap.cc:492] T aadfca82d29a4b5e89d49830fbb46b5e P 396253bba01d4eb382f848b21f804d02: No bootstrap required, opened a new log
I20260709 13:07:23.649135 22578 ts_tablet_manager.cc:1403] T aadfca82d29a4b5e89d49830fbb46b5e P 396253bba01d4eb382f848b21f804d02: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.007s
I20260709 13:07:23.649523 22283 catalog_manager.cc:5697] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb201644020643b485d68b0cd44e1b74 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "cb201644020643b485d68b0cd44e1b74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:23.652113 22578 raft_consensus.cc:359] T aadfca82d29a4b5e89d49830fbb46b5e P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.652798 22578 raft_consensus.cc:385] T aadfca82d29a4b5e89d49830fbb46b5e P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.653072 22578 raft_consensus.cc:740] T aadfca82d29a4b5e89d49830fbb46b5e P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 396253bba01d4eb382f848b21f804d02, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.653766 22579 tablet_bootstrap.cc:654] T 0eeeed324f544d639dab976cc4795bb3 P 75d0a416460746daa0e458ca38689e3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.653926 22578 consensus_queue.cc:260] T aadfca82d29a4b5e89d49830fbb46b5e P 396253bba01d4eb382f848b21f804d02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.656225 22578 ts_tablet_manager.cc:1434] T aadfca82d29a4b5e89d49830fbb46b5e P 396253bba01d4eb382f848b21f804d02: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:07:23.664824 22579 tablet_bootstrap.cc:492] T 0eeeed324f544d639dab976cc4795bb3 P 75d0a416460746daa0e458ca38689e3b: No bootstrap required, opened a new log
I20260709 13:07:23.665275 22579 ts_tablet_manager.cc:1403] T 0eeeed324f544d639dab976cc4795bb3 P 75d0a416460746daa0e458ca38689e3b: Time spent bootstrapping tablet: real 0.017s	user 0.009s	sys 0.007s
I20260709 13:07:23.665958 22586 raft_consensus.cc:2804] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:23.667219 22586 raft_consensus.cc:697] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [term 1 LEADER]: Becoming Leader. State: Replica: 75d0a416460746daa0e458ca38689e3b, State: Running, Role: LEADER
I20260709 13:07:23.667447 22579 raft_consensus.cc:359] T 0eeeed324f544d639dab976cc4795bb3 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.668053 22579 raft_consensus.cc:385] T 0eeeed324f544d639dab976cc4795bb3 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.668282 22579 raft_consensus.cc:740] T 0eeeed324f544d639dab976cc4795bb3 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75d0a416460746daa0e458ca38689e3b, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.668203 22586 consensus_queue.cc:237] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.669046 22579 consensus_queue.cc:260] T 0eeeed324f544d639dab976cc4795bb3 P 75d0a416460746daa0e458ca38689e3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.670431 22537 raft_consensus.cc:2468] T fcc967d7308b4cb3a5d93bc77dfa4681 P cb201644020643b485d68b0cd44e1b74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75d0a416460746daa0e458ca38689e3b in term 1.
I20260709 13:07:23.671255 22579 ts_tablet_manager.cc:1434] T 0eeeed324f544d639dab976cc4795bb3 P 75d0a416460746daa0e458ca38689e3b: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.004s
I20260709 13:07:23.672142 22579 tablet_bootstrap.cc:492] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 75d0a416460746daa0e458ca38689e3b: Bootstrap starting.
I20260709 13:07:23.677088 22579 tablet_bootstrap.cc:654] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 75d0a416460746daa0e458ca38689e3b: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:23.679157 22283 catalog_manager.cc:5697] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b reported cstate change: term changed from 0 to 1, leader changed from <none> to 75d0a416460746daa0e458ca38689e3b (127.21.63.1). New cstate: current_term: 1 leader_uuid: "75d0a416460746daa0e458ca38689e3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:23.681860 22579 tablet_bootstrap.cc:492] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 75d0a416460746daa0e458ca38689e3b: No bootstrap required, opened a new log
I20260709 13:07:23.682237 22579 ts_tablet_manager.cc:1403] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 75d0a416460746daa0e458ca38689e3b: Time spent bootstrapping tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:07:23.684078 22579 raft_consensus.cc:359] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.684719 22579 raft_consensus.cc:385] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:23.685009 22579 raft_consensus.cc:740] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75d0a416460746daa0e458ca38689e3b, State: Initialized, Role: FOLLOWER
I20260709 13:07:23.685602 22579 consensus_queue.cc:260] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 75d0a416460746daa0e458ca38689e3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.687161 22579 ts_tablet_manager.cc:1434] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 75d0a416460746daa0e458ca38689e3b: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:07:23.689002 22610 raft_consensus.cc:493] T 101b2ac61327416ea7a64aa0261d2306 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.689507 22610 raft_consensus.cc:515] T 101b2ac61327416ea7a64aa0261d2306 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.691742 22610 leader_election.cc:290] T 101b2ac61327416ea7a64aa0261d2306 P cb201644020643b485d68b0cd44e1b74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75d0a416460746daa0e458ca38689e3b (127.21.63.1:37185), 396253bba01d4eb382f848b21f804d02 (127.21.63.2:43161)
I20260709 13:07:23.692467 22388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "101b2ac61327416ea7a64aa0261d2306" candidate_uuid: "cb201644020643b485d68b0cd44e1b74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75d0a416460746daa0e458ca38689e3b" is_pre_election: true
I20260709 13:07:23.694490 22464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "101b2ac61327416ea7a64aa0261d2306" candidate_uuid: "cb201644020643b485d68b0cd44e1b74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "396253bba01d4eb382f848b21f804d02" is_pre_election: true
I20260709 13:07:23.695293 22464 raft_consensus.cc:2393] T 101b2ac61327416ea7a64aa0261d2306 P 396253bba01d4eb382f848b21f804d02 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb201644020643b485d68b0cd44e1b74 in current term 1: Already voted for candidate 75d0a416460746daa0e458ca38689e3b in this term.
I20260709 13:07:23.696396 22503 leader_election.cc:304] T 101b2ac61327416ea7a64aa0261d2306 P cb201644020643b485d68b0cd44e1b74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb201644020643b485d68b0cd44e1b74; no voters: 396253bba01d4eb382f848b21f804d02, 75d0a416460746daa0e458ca38689e3b
I20260709 13:07:23.697297 22610 raft_consensus.cc:3060] T 101b2ac61327416ea7a64aa0261d2306 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.702826 22610 raft_consensus.cc:2749] T 101b2ac61327416ea7a64aa0261d2306 P cb201644020643b485d68b0cd44e1b74 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:23.779966 22610 raft_consensus.cc:493] T aadfca82d29a4b5e89d49830fbb46b5e P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.780419 22610 raft_consensus.cc:515] T aadfca82d29a4b5e89d49830fbb46b5e P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.782097 22610 leader_election.cc:290] T aadfca82d29a4b5e89d49830fbb46b5e P cb201644020643b485d68b0cd44e1b74 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75d0a416460746daa0e458ca38689e3b (127.21.63.1:37185), 396253bba01d4eb382f848b21f804d02 (127.21.63.2:43161)
I20260709 13:07:23.783000 22388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadfca82d29a4b5e89d49830fbb46b5e" candidate_uuid: "cb201644020643b485d68b0cd44e1b74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75d0a416460746daa0e458ca38689e3b" is_pre_election: true
I20260709 13:07:23.783121 22464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadfca82d29a4b5e89d49830fbb46b5e" candidate_uuid: "cb201644020643b485d68b0cd44e1b74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "396253bba01d4eb382f848b21f804d02" is_pre_election: true
I20260709 13:07:23.783628 22388 raft_consensus.cc:2468] T aadfca82d29a4b5e89d49830fbb46b5e P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb201644020643b485d68b0cd44e1b74 in term 0.
I20260709 13:07:23.783741 22464 raft_consensus.cc:2468] T aadfca82d29a4b5e89d49830fbb46b5e P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb201644020643b485d68b0cd44e1b74 in term 0.
I20260709 13:07:23.784714 22503 leader_election.cc:304] T aadfca82d29a4b5e89d49830fbb46b5e P cb201644020643b485d68b0cd44e1b74 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75d0a416460746daa0e458ca38689e3b, cb201644020643b485d68b0cd44e1b74; no voters: 
I20260709 13:07:23.785442 22610 raft_consensus.cc:2804] T aadfca82d29a4b5e89d49830fbb46b5e P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:23.785768 22610 raft_consensus.cc:493] T aadfca82d29a4b5e89d49830fbb46b5e P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:23.786068 22610 raft_consensus.cc:3060] T aadfca82d29a4b5e89d49830fbb46b5e P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.790375 22610 raft_consensus.cc:515] T aadfca82d29a4b5e89d49830fbb46b5e P cb201644020643b485d68b0cd44e1b74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.791915 22610 leader_election.cc:290] T aadfca82d29a4b5e89d49830fbb46b5e P cb201644020643b485d68b0cd44e1b74 [CANDIDATE]: Term 1 election: Requested vote from peers 75d0a416460746daa0e458ca38689e3b (127.21.63.1:37185), 396253bba01d4eb382f848b21f804d02 (127.21.63.2:43161)
I20260709 13:07:23.792843 22388 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadfca82d29a4b5e89d49830fbb46b5e" candidate_uuid: "cb201644020643b485d68b0cd44e1b74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75d0a416460746daa0e458ca38689e3b"
I20260709 13:07:23.793081 22464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadfca82d29a4b5e89d49830fbb46b5e" candidate_uuid: "cb201644020643b485d68b0cd44e1b74" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "396253bba01d4eb382f848b21f804d02"
I20260709 13:07:23.793311 22388 raft_consensus.cc:3060] T aadfca82d29a4b5e89d49830fbb46b5e P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.793694 22464 raft_consensus.cc:3060] T aadfca82d29a4b5e89d49830fbb46b5e P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.797816 22388 raft_consensus.cc:2468] T aadfca82d29a4b5e89d49830fbb46b5e P 75d0a416460746daa0e458ca38689e3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb201644020643b485d68b0cd44e1b74 in term 1.
I20260709 13:07:23.798503 22464 raft_consensus.cc:2468] T aadfca82d29a4b5e89d49830fbb46b5e P 396253bba01d4eb382f848b21f804d02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb201644020643b485d68b0cd44e1b74 in term 1.
I20260709 13:07:23.798866 22503 leader_election.cc:304] T aadfca82d29a4b5e89d49830fbb46b5e P cb201644020643b485d68b0cd44e1b74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75d0a416460746daa0e458ca38689e3b, cb201644020643b485d68b0cd44e1b74; no voters: 
I20260709 13:07:23.799891 22610 raft_consensus.cc:2804] T aadfca82d29a4b5e89d49830fbb46b5e P cb201644020643b485d68b0cd44e1b74 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:23.800343 22610 raft_consensus.cc:697] T aadfca82d29a4b5e89d49830fbb46b5e P cb201644020643b485d68b0cd44e1b74 [term 1 LEADER]: Becoming Leader. State: Replica: cb201644020643b485d68b0cd44e1b74, State: Running, Role: LEADER
I20260709 13:07:23.801133 22610 consensus_queue.cc:237] T aadfca82d29a4b5e89d49830fbb46b5e P cb201644020643b485d68b0cd44e1b74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.809342 22285 catalog_manager.cc:5697] T aadfca82d29a4b5e89d49830fbb46b5e P cb201644020643b485d68b0cd44e1b74 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb201644020643b485d68b0cd44e1b74 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "cb201644020643b485d68b0cd44e1b74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:23.825583 22586 raft_consensus.cc:493] T 0eeeed324f544d639dab976cc4795bb3 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.826007 22586 raft_consensus.cc:515] T 0eeeed324f544d639dab976cc4795bb3 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.827790 22586 leader_election.cc:290] T 0eeeed324f544d639dab976cc4795bb3 P 75d0a416460746daa0e458ca38689e3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 396253bba01d4eb382f848b21f804d02 (127.21.63.2:43161), cb201644020643b485d68b0cd44e1b74 (127.21.63.3:41079)
I20260709 13:07:23.828824 22464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eeeed324f544d639dab976cc4795bb3" candidate_uuid: "75d0a416460746daa0e458ca38689e3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "396253bba01d4eb382f848b21f804d02" is_pre_election: true
I20260709 13:07:23.828862 22537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eeeed324f544d639dab976cc4795bb3" candidate_uuid: "75d0a416460746daa0e458ca38689e3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb201644020643b485d68b0cd44e1b74" is_pre_election: true
I20260709 13:07:23.829720 22537 raft_consensus.cc:2468] T 0eeeed324f544d639dab976cc4795bb3 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75d0a416460746daa0e458ca38689e3b in term 0.
I20260709 13:07:23.829723 22464 raft_consensus.cc:2468] T 0eeeed324f544d639dab976cc4795bb3 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75d0a416460746daa0e458ca38689e3b in term 0.
I20260709 13:07:23.831342 22354 leader_election.cc:304] T 0eeeed324f544d639dab976cc4795bb3 P 75d0a416460746daa0e458ca38689e3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75d0a416460746daa0e458ca38689e3b, cb201644020643b485d68b0cd44e1b74; no voters: 
I20260709 13:07:23.832005 22586 raft_consensus.cc:2804] T 0eeeed324f544d639dab976cc4795bb3 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:23.832355 22586 raft_consensus.cc:493] T 0eeeed324f544d639dab976cc4795bb3 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:23.832711 22621 raft_consensus.cc:493] T c80fcb082aa04657987302b6666fe031 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.832756 22586 raft_consensus.cc:3060] T 0eeeed324f544d639dab976cc4795bb3 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.833241 22621 raft_consensus.cc:515] T c80fcb082aa04657987302b6666fe031 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.834936 22621 leader_election.cc:290] T c80fcb082aa04657987302b6666fe031 P 75d0a416460746daa0e458ca38689e3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 396253bba01d4eb382f848b21f804d02 (127.21.63.2:43161), cb201644020643b485d68b0cd44e1b74 (127.21.63.3:41079)
I20260709 13:07:23.835695 22464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80fcb082aa04657987302b6666fe031" candidate_uuid: "75d0a416460746daa0e458ca38689e3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "396253bba01d4eb382f848b21f804d02" is_pre_election: true
I20260709 13:07:23.836102 22537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80fcb082aa04657987302b6666fe031" candidate_uuid: "75d0a416460746daa0e458ca38689e3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb201644020643b485d68b0cd44e1b74" is_pre_election: true
I20260709 13:07:23.836818 22537 raft_consensus.cc:2393] T c80fcb082aa04657987302b6666fe031 P cb201644020643b485d68b0cd44e1b74 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 75d0a416460746daa0e458ca38689e3b in current term 1: Already voted for candidate 396253bba01d4eb382f848b21f804d02 in this term.
I20260709 13:07:23.837710 22354 leader_election.cc:304] T c80fcb082aa04657987302b6666fe031 P 75d0a416460746daa0e458ca38689e3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75d0a416460746daa0e458ca38689e3b; no voters: 396253bba01d4eb382f848b21f804d02, cb201644020643b485d68b0cd44e1b74
I20260709 13:07:23.838791 22621 raft_consensus.cc:3060] T c80fcb082aa04657987302b6666fe031 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.839202 22586 raft_consensus.cc:515] T 0eeeed324f544d639dab976cc4795bb3 P 75d0a416460746daa0e458ca38689e3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.841924 22586 leader_election.cc:290] T 0eeeed324f544d639dab976cc4795bb3 P 75d0a416460746daa0e458ca38689e3b [CANDIDATE]: Term 1 election: Requested vote from peers 396253bba01d4eb382f848b21f804d02 (127.21.63.2:43161), cb201644020643b485d68b0cd44e1b74 (127.21.63.3:41079)
I20260709 13:07:23.842579 22537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eeeed324f544d639dab976cc4795bb3" candidate_uuid: "75d0a416460746daa0e458ca38689e3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb201644020643b485d68b0cd44e1b74"
I20260709 13:07:23.842955 22464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eeeed324f544d639dab976cc4795bb3" candidate_uuid: "75d0a416460746daa0e458ca38689e3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "396253bba01d4eb382f848b21f804d02"
I20260709 13:07:23.843539 22464 raft_consensus.cc:3060] T 0eeeed324f544d639dab976cc4795bb3 P 396253bba01d4eb382f848b21f804d02 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.843180 22537 raft_consensus.cc:3060] T 0eeeed324f544d639dab976cc4795bb3 P cb201644020643b485d68b0cd44e1b74 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.844835 22584 consensus_queue.cc:1048] T c80fcb082aa04657987302b6666fe031 P 396253bba01d4eb382f848b21f804d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:23.845429 22621 raft_consensus.cc:2749] T c80fcb082aa04657987302b6666fe031 P 75d0a416460746daa0e458ca38689e3b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:23.850946 22537 raft_consensus.cc:2468] T 0eeeed324f544d639dab976cc4795bb3 P cb201644020643b485d68b0cd44e1b74 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75d0a416460746daa0e458ca38689e3b in term 1.
I20260709 13:07:23.852135 22354 leader_election.cc:304] T 0eeeed324f544d639dab976cc4795bb3 P 75d0a416460746daa0e458ca38689e3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75d0a416460746daa0e458ca38689e3b, cb201644020643b485d68b0cd44e1b74; no voters: 
I20260709 13:07:23.853082 22621 raft_consensus.cc:2804] T 0eeeed324f544d639dab976cc4795bb3 P 75d0a416460746daa0e458ca38689e3b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:23.853583 22621 raft_consensus.cc:697] T 0eeeed324f544d639dab976cc4795bb3 P 75d0a416460746daa0e458ca38689e3b [term 1 LEADER]: Becoming Leader. State: Replica: 75d0a416460746daa0e458ca38689e3b, State: Running, Role: LEADER
I20260709 13:07:23.853659 22610 consensus_queue.cc:1048] T 9b21d45dd8e84d6cba59e68212962e65 P cb201644020643b485d68b0cd44e1b74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:23.854197 22464 raft_consensus.cc:2468] T 0eeeed324f544d639dab976cc4795bb3 P 396253bba01d4eb382f848b21f804d02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75d0a416460746daa0e458ca38689e3b in term 1.
I20260709 13:07:23.854566 22621 consensus_queue.cc:237] T 0eeeed324f544d639dab976cc4795bb3 P 75d0a416460746daa0e458ca38689e3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.864328 22285 catalog_manager.cc:5697] T 0eeeed324f544d639dab976cc4795bb3 P 75d0a416460746daa0e458ca38689e3b reported cstate change: term changed from 0 to 1, leader changed from <none> to 75d0a416460746daa0e458ca38689e3b (127.21.63.1). New cstate: current_term: 1 leader_uuid: "75d0a416460746daa0e458ca38689e3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:23.876447 22612 consensus_queue.cc:1048] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:23.878738 22612 consensus_queue.cc:1048] T 9b21d45dd8e84d6cba59e68212962e65 P cb201644020643b485d68b0cd44e1b74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:23.892877 22586 raft_consensus.cc:493] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:23.893402 22586 raft_consensus.cc:515] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.895227 22586 leader_election.cc:290] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 75d0a416460746daa0e458ca38689e3b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 396253bba01d4eb382f848b21f804d02 (127.21.63.2:43161), cb201644020643b485d68b0cd44e1b74 (127.21.63.3:41079)
I20260709 13:07:23.896266 22536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5aba9ad2b64a42ad90bd6f5c11cf2272" candidate_uuid: "75d0a416460746daa0e458ca38689e3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb201644020643b485d68b0cd44e1b74" is_pre_election: true
I20260709 13:07:23.896436 22462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5aba9ad2b64a42ad90bd6f5c11cf2272" candidate_uuid: "75d0a416460746daa0e458ca38689e3b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "396253bba01d4eb382f848b21f804d02" is_pre_election: true
I20260709 13:07:23.897162 22462 raft_consensus.cc:2393] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 396253bba01d4eb382f848b21f804d02 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 75d0a416460746daa0e458ca38689e3b in current term 1: Already voted for candidate cb201644020643b485d68b0cd44e1b74 in this term.
I20260709 13:07:23.898124 22606 consensus_queue.cc:1048] T c80fcb082aa04657987302b6666fe031 P 396253bba01d4eb382f848b21f804d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:23.898608 22354 leader_election.cc:304] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 75d0a416460746daa0e458ca38689e3b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75d0a416460746daa0e458ca38689e3b; no voters: 396253bba01d4eb382f848b21f804d02, cb201644020643b485d68b0cd44e1b74
I20260709 13:07:23.901038 22621 raft_consensus.cc:3060] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 75d0a416460746daa0e458ca38689e3b [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:23.911590 22621 raft_consensus.cc:2749] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 75d0a416460746daa0e458ca38689e3b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:23.928990 22645 consensus_queue.cc:1048] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:07:23.932725 22645 consensus_queue.cc:1048] T a9e5138d2b904033978bb1ac8ab3d39f P 396253bba01d4eb382f848b21f804d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:23.942973 22463 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c80fcb082aa04657987302b6666fe031"
dest_uuid: "396253bba01d4eb382f848b21f804d02"
mode: GRACEFUL
new_leader_uuid: "75d0a416460746daa0e458ca38689e3b"
 from {username='slave'} at 127.0.0.1:57308
I20260709 13:07:23.943624 22463 raft_consensus.cc:606] T c80fcb082aa04657987302b6666fe031 P 396253bba01d4eb382f848b21f804d02 [term 1 LEADER]: Received request to transfer leadership to TS 75d0a416460746daa0e458ca38689e3b
I20260709 13:07:23.943583 22606 consensus_queue.cc:1048] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:23.945633 22584 raft_consensus.cc:993] T c80fcb082aa04657987302b6666fe031 P 396253bba01d4eb382f848b21f804d02: : Instructing follower 75d0a416460746daa0e458ca38689e3b to start an election
I20260709 13:07:23.946102 22584 raft_consensus.cc:1081] T c80fcb082aa04657987302b6666fe031 P 396253bba01d4eb382f848b21f804d02 [term 1 LEADER]: Signalling peer 75d0a416460746daa0e458ca38689e3b to start an election
I20260709 13:07:23.950141 22389 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c80fcb082aa04657987302b6666fe031"
dest_uuid: "75d0a416460746daa0e458ca38689e3b"
 from {username='slave'} at 127.0.0.1:54200
I20260709 13:07:23.950716 22389 raft_consensus.cc:493] T c80fcb082aa04657987302b6666fe031 P 75d0a416460746daa0e458ca38689e3b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:23.950886 22463 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c19b5a8350d74917bbc57cd93f2c23e2"
dest_uuid: "396253bba01d4eb382f848b21f804d02"
mode: GRACEFUL
new_leader_uuid: "75d0a416460746daa0e458ca38689e3b"
 from {username='slave'} at 127.0.0.1:57308
I20260709 13:07:23.951121 22389 raft_consensus.cc:3060] T c80fcb082aa04657987302b6666fe031 P 75d0a416460746daa0e458ca38689e3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:23.951464 22463 raft_consensus.cc:606] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02 [term 1 LEADER]: Received request to transfer leadership to TS 75d0a416460746daa0e458ca38689e3b
I20260709 13:07:23.958475 22621 consensus_queue.cc:1048] T 101b2ac61327416ea7a64aa0261d2306 P 75d0a416460746daa0e458ca38689e3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:23.959188 22389 raft_consensus.cc:515] T c80fcb082aa04657987302b6666fe031 P 75d0a416460746daa0e458ca38689e3b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.962037 22389 leader_election.cc:290] T c80fcb082aa04657987302b6666fe031 P 75d0a416460746daa0e458ca38689e3b [CANDIDATE]: Term 2 election: Requested vote from peers 396253bba01d4eb382f848b21f804d02 (127.21.63.2:43161), cb201644020643b485d68b0cd44e1b74 (127.21.63.3:41079)
I20260709 13:07:23.969759 22463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80fcb082aa04657987302b6666fe031" candidate_uuid: "75d0a416460746daa0e458ca38689e3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "396253bba01d4eb382f848b21f804d02"
I20260709 13:07:23.970392 22463 raft_consensus.cc:3055] T c80fcb082aa04657987302b6666fe031 P 396253bba01d4eb382f848b21f804d02 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:23.967828 22538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c80fcb082aa04657987302b6666fe031" candidate_uuid: "75d0a416460746daa0e458ca38689e3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cb201644020643b485d68b0cd44e1b74"
I20260709 13:07:23.970737 22463 raft_consensus.cc:740] T c80fcb082aa04657987302b6666fe031 P 396253bba01d4eb382f848b21f804d02 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 396253bba01d4eb382f848b21f804d02, State: Running, Role: LEADER
I20260709 13:07:23.971328 22538 raft_consensus.cc:3060] T c80fcb082aa04657987302b6666fe031 P cb201644020643b485d68b0cd44e1b74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:23.972325 22463 consensus_queue.cc:260] T c80fcb082aa04657987302b6666fe031 P 396253bba01d4eb382f848b21f804d02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.973605 22463 raft_consensus.cc:3060] T c80fcb082aa04657987302b6666fe031 P 396253bba01d4eb382f848b21f804d02 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:23.986928 22585 consensus_queue.cc:1048] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:23.991467 22538 raft_consensus.cc:2468] T c80fcb082aa04657987302b6666fe031 P cb201644020643b485d68b0cd44e1b74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75d0a416460746daa0e458ca38689e3b in term 2.
I20260709 13:07:23.993927 22354 leader_election.cc:304] T c80fcb082aa04657987302b6666fe031 P 75d0a416460746daa0e458ca38689e3b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75d0a416460746daa0e458ca38689e3b, cb201644020643b485d68b0cd44e1b74; no voters: 
I20260709 13:07:23.995472 22621 raft_consensus.cc:2804] T c80fcb082aa04657987302b6666fe031 P 75d0a416460746daa0e458ca38689e3b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:23.996076 22621 raft_consensus.cc:697] T c80fcb082aa04657987302b6666fe031 P 75d0a416460746daa0e458ca38689e3b [term 2 LEADER]: Becoming Leader. State: Replica: 75d0a416460746daa0e458ca38689e3b, State: Running, Role: LEADER
I20260709 13:07:23.997001 22621 consensus_queue.cc:237] T c80fcb082aa04657987302b6666fe031 P 75d0a416460746daa0e458ca38689e3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:23.999042 22463 raft_consensus.cc:2468] T c80fcb082aa04657987302b6666fe031 P 396253bba01d4eb382f848b21f804d02 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75d0a416460746daa0e458ca38689e3b in term 2.
I20260709 13:07:24.011282 22585 consensus_queue.cc:1048] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:24.022776 22621 consensus_queue.cc:1048] T 101b2ac61327416ea7a64aa0261d2306 P 75d0a416460746daa0e458ca38689e3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:24.028137 22645 consensus_queue.cc:1048] T a9e5138d2b904033978bb1ac8ab3d39f P 396253bba01d4eb382f848b21f804d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:24.050297 22584 raft_consensus.cc:993] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02: : Instructing follower 75d0a416460746daa0e458ca38689e3b to start an election
I20260709 13:07:24.050927 22584 raft_consensus.cc:1081] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02 [term 1 LEADER]: Signalling peer 75d0a416460746daa0e458ca38689e3b to start an election
I20260709 13:07:24.052114 22537 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9b21d45dd8e84d6cba59e68212962e65"
dest_uuid: "cb201644020643b485d68b0cd44e1b74"
mode: GRACEFUL
new_leader_uuid: "75d0a416460746daa0e458ca38689e3b"
 from {username='slave'} at 127.0.0.1:47024
I20260709 13:07:24.052826 22537 raft_consensus.cc:606] T 9b21d45dd8e84d6cba59e68212962e65 P cb201644020643b485d68b0cd44e1b74 [term 1 LEADER]: Received request to transfer leadership to TS 75d0a416460746daa0e458ca38689e3b
I20260709 13:07:24.061980 22585 consensus_queue.cc:1048] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:24.063350 22659 consensus_queue.cc:1048] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:24.064924 22537 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dd07de1f75c54b4bba2d31d81e28ce54"
dest_uuid: "cb201644020643b485d68b0cd44e1b74"
mode: GRACEFUL
new_leader_uuid: "75d0a416460746daa0e458ca38689e3b"
 from {username='slave'} at 127.0.0.1:47024
I20260709 13:07:24.066514 22389 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c19b5a8350d74917bbc57cd93f2c23e2"
dest_uuid: "75d0a416460746daa0e458ca38689e3b"
 from {username='slave'} at 127.0.0.1:54200
I20260709 13:07:24.071381 22389 raft_consensus.cc:493] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:24.071353 22537 raft_consensus.cc:606] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [term 1 LEADER]: Received request to transfer leadership to TS 75d0a416460746daa0e458ca38689e3b
I20260709 13:07:24.072078 22389 raft_consensus.cc:3060] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:24.078032 22389 raft_consensus.cc:515] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:24.079937 22389 leader_election.cc:290] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b [CANDIDATE]: Term 2 election: Requested vote from peers 396253bba01d4eb382f848b21f804d02 (127.21.63.2:43161), cb201644020643b485d68b0cd44e1b74 (127.21.63.3:41079)
I20260709 13:07:24.082095 22536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c19b5a8350d74917bbc57cd93f2c23e2" candidate_uuid: "75d0a416460746daa0e458ca38689e3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cb201644020643b485d68b0cd44e1b74"
I20260709 13:07:24.082684 22536 raft_consensus.cc:3060] T c19b5a8350d74917bbc57cd93f2c23e2 P cb201644020643b485d68b0cd44e1b74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:24.086076 22462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c19b5a8350d74917bbc57cd93f2c23e2" candidate_uuid: "75d0a416460746daa0e458ca38689e3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "396253bba01d4eb382f848b21f804d02"
I20260709 13:07:24.086807 22462 raft_consensus.cc:3055] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:24.087154 22462 raft_consensus.cc:740] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 396253bba01d4eb382f848b21f804d02, State: Running, Role: LEADER
I20260709 13:07:24.088115 22462 consensus_queue.cc:260] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:24.089681 22462 raft_consensus.cc:3060] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:24.090385 22536 raft_consensus.cc:2468] T c19b5a8350d74917bbc57cd93f2c23e2 P cb201644020643b485d68b0cd44e1b74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75d0a416460746daa0e458ca38689e3b in term 2.
I20260709 13:07:24.090406 22658 consensus_queue.cc:1048] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:24.091538 22354 leader_election.cc:304] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75d0a416460746daa0e458ca38689e3b, cb201644020643b485d68b0cd44e1b74; no voters: 
I20260709 13:07:24.092350 22658 raft_consensus.cc:2804] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:24.092840 22658 raft_consensus.cc:697] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b [term 2 LEADER]: Becoming Leader. State: Replica: 75d0a416460746daa0e458ca38689e3b, State: Running, Role: LEADER
I20260709 13:07:24.093650 22658 consensus_queue.cc:237] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:24.099421 22286 catalog_manager.cc:5697] T c80fcb082aa04657987302b6666fe031 P 75d0a416460746daa0e458ca38689e3b reported cstate change: term changed from 1 to 2, leader changed from 396253bba01d4eb382f848b21f804d02 (127.21.63.2) to 75d0a416460746daa0e458ca38689e3b (127.21.63.1). New cstate: current_term: 2 leader_uuid: "75d0a416460746daa0e458ca38689e3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:24.107509 22462 raft_consensus.cc:2468] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75d0a416460746daa0e458ca38689e3b in term 2.
I20260709 13:07:24.127256 22386 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fcc967d7308b4cb3a5d93bc77dfa4681"
dest_uuid: "75d0a416460746daa0e458ca38689e3b"
mode: GRACEFUL
new_leader_uuid: "396253bba01d4eb382f848b21f804d02"
 from {username='slave'} at 127.0.0.1:54204
I20260709 13:07:24.127955 22386 raft_consensus.cc:606] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [term 1 LEADER]: Received request to transfer leadership to TS 396253bba01d4eb382f848b21f804d02
I20260709 13:07:24.131567 22283 catalog_manager.cc:5697] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b reported cstate change: term changed from 1 to 2, leader changed from 396253bba01d4eb382f848b21f804d02 (127.21.63.2) to 75d0a416460746daa0e458ca38689e3b (127.21.63.1). New cstate: current_term: 2 leader_uuid: "75d0a416460746daa0e458ca38689e3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:24.134354 22536 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aadfca82d29a4b5e89d49830fbb46b5e"
dest_uuid: "cb201644020643b485d68b0cd44e1b74"
mode: GRACEFUL
new_leader_uuid: "396253bba01d4eb382f848b21f804d02"
 from {username='slave'} at 127.0.0.1:47024
I20260709 13:07:24.134865 22536 raft_consensus.cc:606] T aadfca82d29a4b5e89d49830fbb46b5e P cb201644020643b485d68b0cd44e1b74 [term 1 LEADER]: Received request to transfer leadership to TS 396253bba01d4eb382f848b21f804d02
I20260709 13:07:24.140385 22536 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5aba9ad2b64a42ad90bd6f5c11cf2272"
dest_uuid: "cb201644020643b485d68b0cd44e1b74"
mode: GRACEFUL
new_leader_uuid: "396253bba01d4eb382f848b21f804d02"
 from {username='slave'} at 127.0.0.1:47024
I20260709 13:07:24.140887 22536 raft_consensus.cc:606] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [term 1 LEADER]: Received request to transfer leadership to TS 396253bba01d4eb382f848b21f804d02
I20260709 13:07:24.145948 22386 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0eeeed324f544d639dab976cc4795bb3"
dest_uuid: "75d0a416460746daa0e458ca38689e3b"
mode: GRACEFUL
new_leader_uuid: "396253bba01d4eb382f848b21f804d02"
 from {username='slave'} at 127.0.0.1:54204
I20260709 13:07:24.146432 22386 raft_consensus.cc:606] T 0eeeed324f544d639dab976cc4795bb3 P 75d0a416460746daa0e458ca38689e3b [term 1 LEADER]: Received request to transfer leadership to TS 396253bba01d4eb382f848b21f804d02
I20260709 13:07:24.151319 22462 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a9e5138d2b904033978bb1ac8ab3d39f"
dest_uuid: "396253bba01d4eb382f848b21f804d02"
mode: GRACEFUL
new_leader_uuid: "cb201644020643b485d68b0cd44e1b74"
 from {username='slave'} at 127.0.0.1:57308
I20260709 13:07:24.151821 22462 raft_consensus.cc:606] T a9e5138d2b904033978bb1ac8ab3d39f P 396253bba01d4eb382f848b21f804d02 [term 1 LEADER]: Received request to transfer leadership to TS cb201644020643b485d68b0cd44e1b74
I20260709 13:07:24.299106 22658 consensus_queue.cc:1048] T 0eeeed324f544d639dab976cc4795bb3 P 75d0a416460746daa0e458ca38689e3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:24.312520 22586 consensus_queue.cc:1048] T 0eeeed324f544d639dab976cc4795bb3 P 75d0a416460746daa0e458ca38689e3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:24.320133 22647 raft_consensus.cc:993] T 0eeeed324f544d639dab976cc4795bb3 P 75d0a416460746daa0e458ca38689e3b: : Instructing follower 396253bba01d4eb382f848b21f804d02 to start an election
I20260709 13:07:24.320546 22586 raft_consensus.cc:1081] T 0eeeed324f544d639dab976cc4795bb3 P 75d0a416460746daa0e458ca38689e3b [term 1 LEADER]: Signalling peer 396253bba01d4eb382f848b21f804d02 to start an election
I20260709 13:07:24.322242 22462 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0eeeed324f544d639dab976cc4795bb3"
dest_uuid: "396253bba01d4eb382f848b21f804d02"
 from {username='slave'} at 127.0.0.1:57306
I20260709 13:07:24.322844 22462 raft_consensus.cc:493] T 0eeeed324f544d639dab976cc4795bb3 P 396253bba01d4eb382f848b21f804d02 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:24.323258 22462 raft_consensus.cc:3060] T 0eeeed324f544d639dab976cc4795bb3 P 396253bba01d4eb382f848b21f804d02 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:24.328399 22462 raft_consensus.cc:515] T 0eeeed324f544d639dab976cc4795bb3 P 396253bba01d4eb382f848b21f804d02 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:24.330554 22462 leader_election.cc:290] T 0eeeed324f544d639dab976cc4795bb3 P 396253bba01d4eb382f848b21f804d02 [CANDIDATE]: Term 2 election: Requested vote from peers 75d0a416460746daa0e458ca38689e3b (127.21.63.1:37185), cb201644020643b485d68b0cd44e1b74 (127.21.63.3:41079)
I20260709 13:07:24.331231 22386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eeeed324f544d639dab976cc4795bb3" candidate_uuid: "396253bba01d4eb382f848b21f804d02" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75d0a416460746daa0e458ca38689e3b"
I20260709 13:07:24.331889 22386 raft_consensus.cc:3055] T 0eeeed324f544d639dab976cc4795bb3 P 75d0a416460746daa0e458ca38689e3b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:24.332016 22536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0eeeed324f544d639dab976cc4795bb3" candidate_uuid: "396253bba01d4eb382f848b21f804d02" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cb201644020643b485d68b0cd44e1b74"
I20260709 13:07:24.332250 22386 raft_consensus.cc:740] T 0eeeed324f544d639dab976cc4795bb3 P 75d0a416460746daa0e458ca38689e3b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 75d0a416460746daa0e458ca38689e3b, State: Running, Role: LEADER
I20260709 13:07:24.332757 22536 raft_consensus.cc:3060] T 0eeeed324f544d639dab976cc4795bb3 P cb201644020643b485d68b0cd44e1b74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:24.333144 22386 consensus_queue.cc:260] T 0eeeed324f544d639dab976cc4795bb3 P 75d0a416460746daa0e458ca38689e3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:24.334424 22386 raft_consensus.cc:3060] T 0eeeed324f544d639dab976cc4795bb3 P 75d0a416460746daa0e458ca38689e3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:24.338824 22536 raft_consensus.cc:2468] T 0eeeed324f544d639dab976cc4795bb3 P cb201644020643b485d68b0cd44e1b74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 396253bba01d4eb382f848b21f804d02 in term 2.
I20260709 13:07:24.339924 22429 leader_election.cc:304] T 0eeeed324f544d639dab976cc4795bb3 P 396253bba01d4eb382f848b21f804d02 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 396253bba01d4eb382f848b21f804d02, cb201644020643b485d68b0cd44e1b74; no voters: 
I20260709 13:07:24.340723 22584 raft_consensus.cc:2804] T 0eeeed324f544d639dab976cc4795bb3 P 396253bba01d4eb382f848b21f804d02 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:24.340710 22386 raft_consensus.cc:2468] T 0eeeed324f544d639dab976cc4795bb3 P 75d0a416460746daa0e458ca38689e3b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 396253bba01d4eb382f848b21f804d02 in term 2.
I20260709 13:07:24.341413 22584 raft_consensus.cc:697] T 0eeeed324f544d639dab976cc4795bb3 P 396253bba01d4eb382f848b21f804d02 [term 2 LEADER]: Becoming Leader. State: Replica: 396253bba01d4eb382f848b21f804d02, State: Running, Role: LEADER
I20260709 13:07:24.342406 22584 consensus_queue.cc:237] T 0eeeed324f544d639dab976cc4795bb3 P 396253bba01d4eb382f848b21f804d02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:24.349424 22283 catalog_manager.cc:5697] T 0eeeed324f544d639dab976cc4795bb3 P 396253bba01d4eb382f848b21f804d02 reported cstate change: term changed from 1 to 2, leader changed from 75d0a416460746daa0e458ca38689e3b (127.21.63.1) to 396253bba01d4eb382f848b21f804d02 (127.21.63.2). New cstate: current_term: 2 leader_uuid: "396253bba01d4eb382f848b21f804d02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:24.366246 22612 consensus_queue.cc:1048] T aadfca82d29a4b5e89d49830fbb46b5e P cb201644020643b485d68b0cd44e1b74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:24.372562 22606 raft_consensus.cc:993] T a9e5138d2b904033978bb1ac8ab3d39f P 396253bba01d4eb382f848b21f804d02: : Instructing follower cb201644020643b485d68b0cd44e1b74 to start an election
I20260709 13:07:24.373132 22606 raft_consensus.cc:1081] T a9e5138d2b904033978bb1ac8ab3d39f P 396253bba01d4eb382f848b21f804d02 [term 1 LEADER]: Signalling peer cb201644020643b485d68b0cd44e1b74 to start an election
I20260709 13:07:24.378221 22536 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a9e5138d2b904033978bb1ac8ab3d39f"
dest_uuid: "cb201644020643b485d68b0cd44e1b74"
 from {username='slave'} at 127.0.0.1:46994
I20260709 13:07:24.378871 22536 raft_consensus.cc:493] T a9e5138d2b904033978bb1ac8ab3d39f P cb201644020643b485d68b0cd44e1b74 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:24.379386 22536 raft_consensus.cc:3060] T a9e5138d2b904033978bb1ac8ab3d39f P cb201644020643b485d68b0cd44e1b74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:24.380962 22585 consensus_queue.cc:1048] T aadfca82d29a4b5e89d49830fbb46b5e P cb201644020643b485d68b0cd44e1b74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:24.388365 22585 raft_consensus.cc:993] T 9b21d45dd8e84d6cba59e68212962e65 P cb201644020643b485d68b0cd44e1b74: : Instructing follower 75d0a416460746daa0e458ca38689e3b to start an election
I20260709 13:07:24.388859 22585 raft_consensus.cc:1081] T 9b21d45dd8e84d6cba59e68212962e65 P cb201644020643b485d68b0cd44e1b74 [term 1 LEADER]: Signalling peer 75d0a416460746daa0e458ca38689e3b to start an election
I20260709 13:07:24.389832 22464 raft_consensus.cc:1275] T c80fcb082aa04657987302b6666fe031 P 396253bba01d4eb382f848b21f804d02 [term 2 FOLLOWER]: Refusing update from remote peer 75d0a416460746daa0e458ca38689e3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:24.390609 22389 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9b21d45dd8e84d6cba59e68212962e65"
dest_uuid: "75d0a416460746daa0e458ca38689e3b"
 from {username='slave'} at 127.0.0.1:54202
I20260709 13:07:24.391238 22389 raft_consensus.cc:493] T 9b21d45dd8e84d6cba59e68212962e65 P 75d0a416460746daa0e458ca38689e3b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:24.391614 22389 raft_consensus.cc:3060] T 9b21d45dd8e84d6cba59e68212962e65 P 75d0a416460746daa0e458ca38689e3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:24.391525 22586 consensus_queue.cc:1048] T c80fcb082aa04657987302b6666fe031 P 75d0a416460746daa0e458ca38689e3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:24.388077 22536 raft_consensus.cc:515] T a9e5138d2b904033978bb1ac8ab3d39f P cb201644020643b485d68b0cd44e1b74 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:24.394237 22536 leader_election.cc:290] T a9e5138d2b904033978bb1ac8ab3d39f P cb201644020643b485d68b0cd44e1b74 [CANDIDATE]: Term 2 election: Requested vote from peers 75d0a416460746daa0e458ca38689e3b (127.21.63.1:37185), 396253bba01d4eb382f848b21f804d02 (127.21.63.2:43161)
I20260709 13:07:24.396178 22463 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e5138d2b904033978bb1ac8ab3d39f" candidate_uuid: "cb201644020643b485d68b0cd44e1b74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "396253bba01d4eb382f848b21f804d02"
I20260709 13:07:24.396808 22463 raft_consensus.cc:3055] T a9e5138d2b904033978bb1ac8ab3d39f P 396253bba01d4eb382f848b21f804d02 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:24.397127 22463 raft_consensus.cc:740] T a9e5138d2b904033978bb1ac8ab3d39f P 396253bba01d4eb382f848b21f804d02 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 396253bba01d4eb382f848b21f804d02, State: Running, Role: LEADER
I20260709 13:07:24.397889 22463 consensus_queue.cc:260] T a9e5138d2b904033978bb1ac8ab3d39f P 396253bba01d4eb382f848b21f804d02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:24.399048 22463 raft_consensus.cc:3060] T a9e5138d2b904033978bb1ac8ab3d39f P 396253bba01d4eb382f848b21f804d02 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:24.399307 22389 raft_consensus.cc:515] T 9b21d45dd8e84d6cba59e68212962e65 P 75d0a416460746daa0e458ca38689e3b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:24.401374 22389 leader_election.cc:290] T 9b21d45dd8e84d6cba59e68212962e65 P 75d0a416460746daa0e458ca38689e3b [CANDIDATE]: Term 2 election: Requested vote from peers 396253bba01d4eb382f848b21f804d02 (127.21.63.2:43161), cb201644020643b485d68b0cd44e1b74 (127.21.63.3:41079)
I20260709 13:07:24.403903 22612 raft_consensus.cc:993] T aadfca82d29a4b5e89d49830fbb46b5e P cb201644020643b485d68b0cd44e1b74: : Instructing follower 396253bba01d4eb382f848b21f804d02 to start an election
I20260709 13:07:24.404286 22612 raft_consensus.cc:1081] T aadfca82d29a4b5e89d49830fbb46b5e P cb201644020643b485d68b0cd44e1b74 [term 1 LEADER]: Signalling peer 396253bba01d4eb382f848b21f804d02 to start an election
I20260709 13:07:24.406005 22464 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aadfca82d29a4b5e89d49830fbb46b5e"
dest_uuid: "396253bba01d4eb382f848b21f804d02"
 from {username='slave'} at 127.0.0.1:57304
I20260709 13:07:24.406731 22537 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b21d45dd8e84d6cba59e68212962e65" candidate_uuid: "75d0a416460746daa0e458ca38689e3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cb201644020643b485d68b0cd44e1b74"
I20260709 13:07:24.407290 22537 raft_consensus.cc:3055] T 9b21d45dd8e84d6cba59e68212962e65 P cb201644020643b485d68b0cd44e1b74 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:24.407603 22537 raft_consensus.cc:740] T 9b21d45dd8e84d6cba59e68212962e65 P cb201644020643b485d68b0cd44e1b74 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cb201644020643b485d68b0cd44e1b74, State: Running, Role: LEADER
I20260709 13:07:24.407773 22464 raft_consensus.cc:493] T aadfca82d29a4b5e89d49830fbb46b5e P 396253bba01d4eb382f848b21f804d02 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:24.408192 22464 raft_consensus.cc:3060] T aadfca82d29a4b5e89d49830fbb46b5e P 396253bba01d4eb382f848b21f804d02 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:24.408432 22537 consensus_queue.cc:260] T 9b21d45dd8e84d6cba59e68212962e65 P cb201644020643b485d68b0cd44e1b74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:24.409684 22537 raft_consensus.cc:3060] T 9b21d45dd8e84d6cba59e68212962e65 P cb201644020643b485d68b0cd44e1b74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:24.409736 22463 raft_consensus.cc:2468] T a9e5138d2b904033978bb1ac8ab3d39f P 396253bba01d4eb382f848b21f804d02 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb201644020643b485d68b0cd44e1b74 in term 2.
I20260709 13:07:24.396178 22386 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a9e5138d2b904033978bb1ac8ab3d39f" candidate_uuid: "cb201644020643b485d68b0cd44e1b74" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75d0a416460746daa0e458ca38689e3b"
I20260709 13:07:24.411046 22503 leader_election.cc:304] T a9e5138d2b904033978bb1ac8ab3d39f P cb201644020643b485d68b0cd44e1b74 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 396253bba01d4eb382f848b21f804d02, cb201644020643b485d68b0cd44e1b74; no voters: 
I20260709 13:07:24.411768 22536 raft_consensus.cc:1275] T c80fcb082aa04657987302b6666fe031 P cb201644020643b485d68b0cd44e1b74 [term 2 FOLLOWER]: Refusing update from remote peer 75d0a416460746daa0e458ca38689e3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:24.412247 22585 raft_consensus.cc:2804] T a9e5138d2b904033978bb1ac8ab3d39f P cb201644020643b485d68b0cd44e1b74 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:24.413422 22585 raft_consensus.cc:697] T a9e5138d2b904033978bb1ac8ab3d39f P cb201644020643b485d68b0cd44e1b74 [term 2 LEADER]: Becoming Leader. State: Replica: cb201644020643b485d68b0cd44e1b74, State: Running, Role: LEADER
I20260709 13:07:24.415118 22585 consensus_queue.cc:237] T a9e5138d2b904033978bb1ac8ab3d39f P cb201644020643b485d68b0cd44e1b74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:24.411069 22386 raft_consensus.cc:3060] T a9e5138d2b904033978bb1ac8ab3d39f P 75d0a416460746daa0e458ca38689e3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:24.418503 22658 consensus_queue.cc:1048] T c80fcb082aa04657987302b6666fe031 P 75d0a416460746daa0e458ca38689e3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:24.419261 22537 raft_consensus.cc:2468] T 9b21d45dd8e84d6cba59e68212962e65 P cb201644020643b485d68b0cd44e1b74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75d0a416460746daa0e458ca38689e3b in term 2.
I20260709 13:07:24.422648 22462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b21d45dd8e84d6cba59e68212962e65" candidate_uuid: "75d0a416460746daa0e458ca38689e3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "396253bba01d4eb382f848b21f804d02"
I20260709 13:07:24.423206 22462 raft_consensus.cc:3060] T 9b21d45dd8e84d6cba59e68212962e65 P 396253bba01d4eb382f848b21f804d02 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:24.423494 22354 leader_election.cc:304] T 9b21d45dd8e84d6cba59e68212962e65 P 75d0a416460746daa0e458ca38689e3b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75d0a416460746daa0e458ca38689e3b, cb201644020643b485d68b0cd44e1b74; no voters: 
I20260709 13:07:24.423156 22283 catalog_manager.cc:5697] T a9e5138d2b904033978bb1ac8ab3d39f P cb201644020643b485d68b0cd44e1b74 reported cstate change: term changed from 1 to 2, leader changed from 396253bba01d4eb382f848b21f804d02 (127.21.63.2) to cb201644020643b485d68b0cd44e1b74 (127.21.63.3). New cstate: current_term: 2 leader_uuid: "cb201644020643b485d68b0cd44e1b74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:24.425686 22386 raft_consensus.cc:2468] T a9e5138d2b904033978bb1ac8ab3d39f P 75d0a416460746daa0e458ca38689e3b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb201644020643b485d68b0cd44e1b74 in term 2.
I20260709 13:07:24.416268 22464 raft_consensus.cc:515] T aadfca82d29a4b5e89d49830fbb46b5e P 396253bba01d4eb382f848b21f804d02 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:24.428334 22464 leader_election.cc:290] T aadfca82d29a4b5e89d49830fbb46b5e P 396253bba01d4eb382f848b21f804d02 [CANDIDATE]: Term 2 election: Requested vote from peers 75d0a416460746daa0e458ca38689e3b (127.21.63.1:37185), cb201644020643b485d68b0cd44e1b74 (127.21.63.3:41079)
I20260709 13:07:24.428823 22621 raft_consensus.cc:2804] T 9b21d45dd8e84d6cba59e68212962e65 P 75d0a416460746daa0e458ca38689e3b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:24.429324 22621 raft_consensus.cc:697] T 9b21d45dd8e84d6cba59e68212962e65 P 75d0a416460746daa0e458ca38689e3b [term 2 LEADER]: Becoming Leader. State: Replica: 75d0a416460746daa0e458ca38689e3b, State: Running, Role: LEADER
I20260709 13:07:24.430115 22621 consensus_queue.cc:237] T 9b21d45dd8e84d6cba59e68212962e65 P 75d0a416460746daa0e458ca38689e3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:24.433286 22536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadfca82d29a4b5e89d49830fbb46b5e" candidate_uuid: "396253bba01d4eb382f848b21f804d02" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cb201644020643b485d68b0cd44e1b74"
I20260709 13:07:24.433849 22536 raft_consensus.cc:3055] T aadfca82d29a4b5e89d49830fbb46b5e P cb201644020643b485d68b0cd44e1b74 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:24.434126 22536 raft_consensus.cc:740] T aadfca82d29a4b5e89d49830fbb46b5e P cb201644020643b485d68b0cd44e1b74 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cb201644020643b485d68b0cd44e1b74, State: Running, Role: LEADER
I20260709 13:07:24.434900 22536 consensus_queue.cc:260] T aadfca82d29a4b5e89d49830fbb46b5e P cb201644020643b485d68b0cd44e1b74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:24.436018 22536 raft_consensus.cc:3060] T aadfca82d29a4b5e89d49830fbb46b5e P cb201644020643b485d68b0cd44e1b74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:24.437121 22389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadfca82d29a4b5e89d49830fbb46b5e" candidate_uuid: "396253bba01d4eb382f848b21f804d02" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75d0a416460746daa0e458ca38689e3b"
I20260709 13:07:24.437789 22389 raft_consensus.cc:3060] T aadfca82d29a4b5e89d49830fbb46b5e P 75d0a416460746daa0e458ca38689e3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:24.439832 22283 catalog_manager.cc:5697] T 9b21d45dd8e84d6cba59e68212962e65 P 75d0a416460746daa0e458ca38689e3b reported cstate change: term changed from 1 to 2, leader changed from cb201644020643b485d68b0cd44e1b74 (127.21.63.3) to 75d0a416460746daa0e458ca38689e3b (127.21.63.1). New cstate: current_term: 2 leader_uuid: "75d0a416460746daa0e458ca38689e3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:24.443996 22536 raft_consensus.cc:2468] T aadfca82d29a4b5e89d49830fbb46b5e P cb201644020643b485d68b0cd44e1b74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 396253bba01d4eb382f848b21f804d02 in term 2.
I20260709 13:07:24.444594 22389 raft_consensus.cc:2468] T aadfca82d29a4b5e89d49830fbb46b5e P 75d0a416460746daa0e458ca38689e3b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 396253bba01d4eb382f848b21f804d02 in term 2.
I20260709 13:07:24.445233 22429 leader_election.cc:304] T aadfca82d29a4b5e89d49830fbb46b5e P 396253bba01d4eb382f848b21f804d02 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 396253bba01d4eb382f848b21f804d02, cb201644020643b485d68b0cd44e1b74; no voters: 
I20260709 13:07:24.446018 22606 raft_consensus.cc:2804] T aadfca82d29a4b5e89d49830fbb46b5e P 396253bba01d4eb382f848b21f804d02 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:24.446473 22606 raft_consensus.cc:697] T aadfca82d29a4b5e89d49830fbb46b5e P 396253bba01d4eb382f848b21f804d02 [term 2 LEADER]: Becoming Leader. State: Replica: 396253bba01d4eb382f848b21f804d02, State: Running, Role: LEADER
I20260709 13:07:24.447352 22606 consensus_queue.cc:237] T aadfca82d29a4b5e89d49830fbb46b5e P 396253bba01d4eb382f848b21f804d02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:24.448511 22462 raft_consensus.cc:2468] T 9b21d45dd8e84d6cba59e68212962e65 P 396253bba01d4eb382f848b21f804d02 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75d0a416460746daa0e458ca38689e3b in term 2.
I20260709 13:07:24.454567 22283 catalog_manager.cc:5697] T aadfca82d29a4b5e89d49830fbb46b5e P 396253bba01d4eb382f848b21f804d02 reported cstate change: term changed from 1 to 2, leader changed from cb201644020643b485d68b0cd44e1b74 (127.21.63.3) to 396253bba01d4eb382f848b21f804d02 (127.21.63.2). New cstate: current_term: 2 leader_uuid: "396253bba01d4eb382f848b21f804d02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:24.475428 22462 raft_consensus.cc:1275] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02 [term 2 FOLLOWER]: Refusing update from remote peer 75d0a416460746daa0e458ca38689e3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:24.476809 22621 consensus_queue.cc:1048] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:24.484972 22536 raft_consensus.cc:1275] T c19b5a8350d74917bbc57cd93f2c23e2 P cb201644020643b485d68b0cd44e1b74 [term 2 FOLLOWER]: Refusing update from remote peer 75d0a416460746daa0e458ca38689e3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:24.486594 22658 consensus_queue.cc:1048] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:24.495293 22612 raft_consensus.cc:993] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74: : Instructing follower 396253bba01d4eb382f848b21f804d02 to start an election
I20260709 13:07:24.495848 22612 raft_consensus.cc:1081] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [term 1 LEADER]: Signalling peer 396253bba01d4eb382f848b21f804d02 to start an election
I20260709 13:07:24.497691 22462 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5aba9ad2b64a42ad90bd6f5c11cf2272"
dest_uuid: "396253bba01d4eb382f848b21f804d02"
 from {username='slave'} at 127.0.0.1:57304
I20260709 13:07:24.498157 22462 raft_consensus.cc:493] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 396253bba01d4eb382f848b21f804d02 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:24.498497 22462 raft_consensus.cc:3060] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 396253bba01d4eb382f848b21f804d02 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:24.504211 22462 raft_consensus.cc:515] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 396253bba01d4eb382f848b21f804d02 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:24.506201 22462 leader_election.cc:290] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 396253bba01d4eb382f848b21f804d02 [CANDIDATE]: Term 2 election: Requested vote from peers 75d0a416460746daa0e458ca38689e3b (127.21.63.1:37185), cb201644020643b485d68b0cd44e1b74 (127.21.63.3:41079)
I20260709 13:07:24.506825 22389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5aba9ad2b64a42ad90bd6f5c11cf2272" candidate_uuid: "396253bba01d4eb382f848b21f804d02" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75d0a416460746daa0e458ca38689e3b"
I20260709 13:07:24.507130 22536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5aba9ad2b64a42ad90bd6f5c11cf2272" candidate_uuid: "396253bba01d4eb382f848b21f804d02" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cb201644020643b485d68b0cd44e1b74"
I20260709 13:07:24.507638 22389 raft_consensus.cc:3060] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 75d0a416460746daa0e458ca38689e3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:24.508029 22536 raft_consensus.cc:3055] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:24.508343 22536 raft_consensus.cc:740] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cb201644020643b485d68b0cd44e1b74, State: Running, Role: LEADER
I20260709 13:07:24.509057 22536 consensus_queue.cc:260] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:24.510218 22536 raft_consensus.cc:3060] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:24.512949 22389 raft_consensus.cc:2468] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 75d0a416460746daa0e458ca38689e3b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 396253bba01d4eb382f848b21f804d02 in term 2.
I20260709 13:07:24.514139 22429 leader_election.cc:304] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 396253bba01d4eb382f848b21f804d02 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 396253bba01d4eb382f848b21f804d02, 75d0a416460746daa0e458ca38689e3b; no voters: 
I20260709 13:07:24.515035 22606 raft_consensus.cc:2804] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 396253bba01d4eb382f848b21f804d02 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:24.515502 22606 raft_consensus.cc:697] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 396253bba01d4eb382f848b21f804d02 [term 2 LEADER]: Becoming Leader. State: Replica: 396253bba01d4eb382f848b21f804d02, State: Running, Role: LEADER
I20260709 13:07:24.516173 22536 raft_consensus.cc:2468] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 396253bba01d4eb382f848b21f804d02 in term 2.
I20260709 13:07:24.516273 22606 consensus_queue.cc:237] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 396253bba01d4eb382f848b21f804d02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:24.520606 22647 raft_consensus.cc:993] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b: : Instructing follower 396253bba01d4eb382f848b21f804d02 to start an election
I20260709 13:07:24.521090 22658 raft_consensus.cc:1081] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [term 1 LEADER]: Signalling peer 396253bba01d4eb382f848b21f804d02 to start an election
I20260709 13:07:24.522580 22462 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fcc967d7308b4cb3a5d93bc77dfa4681"
dest_uuid: "396253bba01d4eb382f848b21f804d02"
 from {username='slave'} at 127.0.0.1:57306
I20260709 13:07:24.523172 22462 raft_consensus.cc:493] T fcc967d7308b4cb3a5d93bc77dfa4681 P 396253bba01d4eb382f848b21f804d02 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:24.523551 22462 raft_consensus.cc:3060] T fcc967d7308b4cb3a5d93bc77dfa4681 P 396253bba01d4eb382f848b21f804d02 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:24.524351 22284 catalog_manager.cc:5697] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 396253bba01d4eb382f848b21f804d02 reported cstate change: term changed from 1 to 2, leader changed from cb201644020643b485d68b0cd44e1b74 (127.21.63.3) to 396253bba01d4eb382f848b21f804d02 (127.21.63.2). New cstate: current_term: 2 leader_uuid: "396253bba01d4eb382f848b21f804d02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:24.529395 22462 raft_consensus.cc:515] T fcc967d7308b4cb3a5d93bc77dfa4681 P 396253bba01d4eb382f848b21f804d02 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:24.531369 22462 leader_election.cc:290] T fcc967d7308b4cb3a5d93bc77dfa4681 P 396253bba01d4eb382f848b21f804d02 [CANDIDATE]: Term 2 election: Requested vote from peers 75d0a416460746daa0e458ca38689e3b (127.21.63.1:37185), cb201644020643b485d68b0cd44e1b74 (127.21.63.3:41079)
I20260709 13:07:24.532250 22389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcc967d7308b4cb3a5d93bc77dfa4681" candidate_uuid: "396253bba01d4eb382f848b21f804d02" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75d0a416460746daa0e458ca38689e3b"
I20260709 13:07:24.532935 22389 raft_consensus.cc:3055] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:24.532943 22536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcc967d7308b4cb3a5d93bc77dfa4681" candidate_uuid: "396253bba01d4eb382f848b21f804d02" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cb201644020643b485d68b0cd44e1b74"
I20260709 13:07:24.533483 22389 raft_consensus.cc:740] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 75d0a416460746daa0e458ca38689e3b, State: Running, Role: LEADER
I20260709 13:07:24.533680 22536 raft_consensus.cc:3060] T fcc967d7308b4cb3a5d93bc77dfa4681 P cb201644020643b485d68b0cd44e1b74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:24.534523 22389 consensus_queue.cc:260] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:24.535517 22389 raft_consensus.cc:3060] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:24.538630 22536 raft_consensus.cc:2468] T fcc967d7308b4cb3a5d93bc77dfa4681 P cb201644020643b485d68b0cd44e1b74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 396253bba01d4eb382f848b21f804d02 in term 2.
I20260709 13:07:24.539566 22429 leader_election.cc:304] T fcc967d7308b4cb3a5d93bc77dfa4681 P 396253bba01d4eb382f848b21f804d02 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 396253bba01d4eb382f848b21f804d02, cb201644020643b485d68b0cd44e1b74; no voters: 
I20260709 13:07:24.540227 22606 raft_consensus.cc:2804] T fcc967d7308b4cb3a5d93bc77dfa4681 P 396253bba01d4eb382f848b21f804d02 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:24.540488 22389 raft_consensus.cc:2468] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 396253bba01d4eb382f848b21f804d02 in term 2.
I20260709 13:07:24.540982 22606 raft_consensus.cc:697] T fcc967d7308b4cb3a5d93bc77dfa4681 P 396253bba01d4eb382f848b21f804d02 [term 2 LEADER]: Becoming Leader. State: Replica: 396253bba01d4eb382f848b21f804d02, State: Running, Role: LEADER
I20260709 13:07:24.541868 22606 consensus_queue.cc:237] T fcc967d7308b4cb3a5d93bc77dfa4681 P 396253bba01d4eb382f848b21f804d02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:24.548346 22284 catalog_manager.cc:5697] T fcc967d7308b4cb3a5d93bc77dfa4681 P 396253bba01d4eb382f848b21f804d02 reported cstate change: term changed from 1 to 2, leader changed from 75d0a416460746daa0e458ca38689e3b (127.21.63.1) to 396253bba01d4eb382f848b21f804d02 (127.21.63.2). New cstate: current_term: 2 leader_uuid: "396253bba01d4eb382f848b21f804d02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:24.610620 22610 raft_consensus.cc:993] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74: : Instructing follower 75d0a416460746daa0e458ca38689e3b to start an election
I20260709 13:07:24.611156 22585 raft_consensus.cc:1081] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [term 1 LEADER]: Signalling peer 75d0a416460746daa0e458ca38689e3b to start an election
I20260709 13:07:24.612727 22389 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dd07de1f75c54b4bba2d31d81e28ce54"
dest_uuid: "75d0a416460746daa0e458ca38689e3b"
 from {username='slave'} at 127.0.0.1:54202
I20260709 13:07:24.613297 22389 raft_consensus.cc:493] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:24.613678 22389 raft_consensus.cc:3060] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:24.618753 22389 raft_consensus.cc:515] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:24.620493 22389 leader_election.cc:290] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [CANDIDATE]: Term 2 election: Requested vote from peers 396253bba01d4eb382f848b21f804d02 (127.21.63.2:43161), cb201644020643b485d68b0cd44e1b74 (127.21.63.3:41079)
I20260709 13:07:24.621740 22462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd07de1f75c54b4bba2d31d81e28ce54" candidate_uuid: "75d0a416460746daa0e458ca38689e3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "396253bba01d4eb382f848b21f804d02"
I20260709 13:07:24.621938 22536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd07de1f75c54b4bba2d31d81e28ce54" candidate_uuid: "75d0a416460746daa0e458ca38689e3b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cb201644020643b485d68b0cd44e1b74"
I20260709 13:07:24.622270 22462 raft_consensus.cc:3060] T dd07de1f75c54b4bba2d31d81e28ce54 P 396253bba01d4eb382f848b21f804d02 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:24.622610 22536 raft_consensus.cc:3055] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:24.622956 22536 raft_consensus.cc:740] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cb201644020643b485d68b0cd44e1b74, State: Running, Role: LEADER
I20260709 13:07:24.623692 22536 consensus_queue.cc:260] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:24.624961 22536 raft_consensus.cc:3060] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:24.628372 22462 raft_consensus.cc:2468] T dd07de1f75c54b4bba2d31d81e28ce54 P 396253bba01d4eb382f848b21f804d02 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75d0a416460746daa0e458ca38689e3b in term 2.
I20260709 13:07:24.629674 22354 leader_election.cc:304] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 396253bba01d4eb382f848b21f804d02, 75d0a416460746daa0e458ca38689e3b; no voters: 
I20260709 13:07:24.630383 22536 raft_consensus.cc:2468] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75d0a416460746daa0e458ca38689e3b in term 2.
I20260709 13:07:24.630571 22658 raft_consensus.cc:2804] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:24.631381 22658 raft_consensus.cc:697] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [term 2 LEADER]: Becoming Leader. State: Replica: 75d0a416460746daa0e458ca38689e3b, State: Running, Role: LEADER
I20260709 13:07:24.632300 22658 consensus_queue.cc:237] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:24.639730 22284 catalog_manager.cc:5697] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b reported cstate change: term changed from 1 to 2, leader changed from cb201644020643b485d68b0cd44e1b74 (127.21.63.3) to 75d0a416460746daa0e458ca38689e3b (127.21.63.1). New cstate: current_term: 2 leader_uuid: "75d0a416460746daa0e458ca38689e3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:24.786856 22389 raft_consensus.cc:1275] T 0eeeed324f544d639dab976cc4795bb3 P 75d0a416460746daa0e458ca38689e3b [term 2 FOLLOWER]: Refusing update from remote peer 396253bba01d4eb382f848b21f804d02: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:24.788265 22606 consensus_queue.cc:1048] T 0eeeed324f544d639dab976cc4795bb3 P 396253bba01d4eb382f848b21f804d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:24.797479 22536 raft_consensus.cc:1275] T 0eeeed324f544d639dab976cc4795bb3 P cb201644020643b485d68b0cd44e1b74 [term 2 FOLLOWER]: Refusing update from remote peer 396253bba01d4eb382f848b21f804d02: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:24.799010 22584 consensus_queue.cc:1048] T 0eeeed324f544d639dab976cc4795bb3 P 396253bba01d4eb382f848b21f804d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:24.839957 22462 raft_consensus.cc:1275] T a9e5138d2b904033978bb1ac8ab3d39f P 396253bba01d4eb382f848b21f804d02 [term 2 FOLLOWER]: Refusing update from remote peer cb201644020643b485d68b0cd44e1b74: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:24.841348 22585 consensus_queue.cc:1048] T a9e5138d2b904033978bb1ac8ab3d39f P cb201644020643b485d68b0cd44e1b74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:24.849925 22389 raft_consensus.cc:1275] T a9e5138d2b904033978bb1ac8ab3d39f P 75d0a416460746daa0e458ca38689e3b [term 2 FOLLOWER]: Refusing update from remote peer cb201644020643b485d68b0cd44e1b74: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:24.851853 22612 consensus_queue.cc:1048] T a9e5138d2b904033978bb1ac8ab3d39f P cb201644020643b485d68b0cd44e1b74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:24.897605 22536 raft_consensus.cc:1275] T aadfca82d29a4b5e89d49830fbb46b5e P cb201644020643b485d68b0cd44e1b74 [term 2 FOLLOWER]: Refusing update from remote peer 396253bba01d4eb382f848b21f804d02: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:24.898898 22584 consensus_queue.cc:1048] T aadfca82d29a4b5e89d49830fbb46b5e P 396253bba01d4eb382f848b21f804d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:24.906787 22389 raft_consensus.cc:1275] T aadfca82d29a4b5e89d49830fbb46b5e P 75d0a416460746daa0e458ca38689e3b [term 2 FOLLOWER]: Refusing update from remote peer 396253bba01d4eb382f848b21f804d02: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:24.909315 22584 consensus_queue.cc:1048] T aadfca82d29a4b5e89d49830fbb46b5e P 396253bba01d4eb382f848b21f804d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:24.945256 22462 raft_consensus.cc:1275] T 9b21d45dd8e84d6cba59e68212962e65 P 396253bba01d4eb382f848b21f804d02 [term 2 FOLLOWER]: Refusing update from remote peer 75d0a416460746daa0e458ca38689e3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:24.946388 22658 consensus_queue.cc:1048] T 9b21d45dd8e84d6cba59e68212962e65 P 75d0a416460746daa0e458ca38689e3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:24.955535 22538 raft_consensus.cc:1275] T 9b21d45dd8e84d6cba59e68212962e65 P cb201644020643b485d68b0cd44e1b74 [term 2 FOLLOWER]: Refusing update from remote peer 75d0a416460746daa0e458ca38689e3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:24.956979 22647 consensus_queue.cc:1048] T 9b21d45dd8e84d6cba59e68212962e65 P 75d0a416460746daa0e458ca38689e3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:24.993412 22389 raft_consensus.cc:1275] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 75d0a416460746daa0e458ca38689e3b [term 2 FOLLOWER]: Refusing update from remote peer 396253bba01d4eb382f848b21f804d02: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:24.994959 22584 consensus_queue.cc:1048] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 396253bba01d4eb382f848b21f804d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:25.005239 22538 raft_consensus.cc:1275] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [term 2 FOLLOWER]: Refusing update from remote peer 396253bba01d4eb382f848b21f804d02: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:25.007923 22584 consensus_queue.cc:1048] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 396253bba01d4eb382f848b21f804d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:25.079308 22538 raft_consensus.cc:1275] T fcc967d7308b4cb3a5d93bc77dfa4681 P cb201644020643b485d68b0cd44e1b74 [term 2 FOLLOWER]: Refusing update from remote peer 396253bba01d4eb382f848b21f804d02: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:25.080817 22584 consensus_queue.cc:1048] T fcc967d7308b4cb3a5d93bc77dfa4681 P 396253bba01d4eb382f848b21f804d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:25.088241 22389 raft_consensus.cc:1275] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [term 2 FOLLOWER]: Refusing update from remote peer 396253bba01d4eb382f848b21f804d02: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:25.090307 22606 consensus_queue.cc:1048] T fcc967d7308b4cb3a5d93bc77dfa4681 P 396253bba01d4eb382f848b21f804d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:25.164328 22538 raft_consensus.cc:1275] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [term 2 FOLLOWER]: Refusing update from remote peer 75d0a416460746daa0e458ca38689e3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:25.165627 22658 consensus_queue.cc:1048] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:25.174247 22462 raft_consensus.cc:1275] T dd07de1f75c54b4bba2d31d81e28ce54 P 396253bba01d4eb382f848b21f804d02 [term 2 FOLLOWER]: Refusing update from remote peer 75d0a416460746daa0e458ca38689e3b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:25.176013 22621 consensus_queue.cc:1048] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:27.183856 21756 auto_leader_rebalancer-test.cc:348] The leader distribution is 
I20260709 13:07:27.184573 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:27.200994 22389 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c19b5a8350d74917bbc57cd93f2c23e2"
dest_uuid: "75d0a416460746daa0e458ca38689e3b"
mode: GRACEFUL
new_leader_uuid: "cb201644020643b485d68b0cd44e1b74"
 from {username='slave'} at 127.0.0.1:54282
I20260709 13:07:27.201478 22389 raft_consensus.cc:606] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b [term 2 LEADER]: Received request to transfer leadership to TS cb201644020643b485d68b0cd44e1b74
I20260709 13:07:27.203730 22389 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dd07de1f75c54b4bba2d31d81e28ce54"
dest_uuid: "75d0a416460746daa0e458ca38689e3b"
mode: GRACEFUL
new_leader_uuid: "cb201644020643b485d68b0cd44e1b74"
 from {username='slave'} at 127.0.0.1:54282
I20260709 13:07:27.204159 22389 raft_consensus.cc:606] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [term 2 LEADER]: Received request to transfer leadership to TS cb201644020643b485d68b0cd44e1b74
I20260709 13:07:27.205154 21756 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:07:27.205711 21756 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:27.405915 22684 raft_consensus.cc:993] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b: : Instructing follower cb201644020643b485d68b0cd44e1b74 to start an election
I20260709 13:07:27.406388 22684 raft_consensus.cc:1081] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b [term 2 LEADER]: Signalling peer cb201644020643b485d68b0cd44e1b74 to start an election
I20260709 13:07:27.408512 22536 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c19b5a8350d74917bbc57cd93f2c23e2"
dest_uuid: "cb201644020643b485d68b0cd44e1b74"
 from {username='slave'} at 127.0.0.1:47008
I20260709 13:07:27.409011 22536 raft_consensus.cc:493] T c19b5a8350d74917bbc57cd93f2c23e2 P cb201644020643b485d68b0cd44e1b74 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:27.409320 22536 raft_consensus.cc:3060] T c19b5a8350d74917bbc57cd93f2c23e2 P cb201644020643b485d68b0cd44e1b74 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:27.413836 22536 raft_consensus.cc:515] T c19b5a8350d74917bbc57cd93f2c23e2 P cb201644020643b485d68b0cd44e1b74 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:27.415469 22536 leader_election.cc:290] T c19b5a8350d74917bbc57cd93f2c23e2 P cb201644020643b485d68b0cd44e1b74 [CANDIDATE]: Term 3 election: Requested vote from peers 75d0a416460746daa0e458ca38689e3b (127.21.63.1:37185), 396253bba01d4eb382f848b21f804d02 (127.21.63.2:43161)
I20260709 13:07:27.416301 22389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c19b5a8350d74917bbc57cd93f2c23e2" candidate_uuid: "cb201644020643b485d68b0cd44e1b74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "75d0a416460746daa0e458ca38689e3b"
I20260709 13:07:27.416867 22389 raft_consensus.cc:3055] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:27.417201 22389 raft_consensus.cc:740] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 75d0a416460746daa0e458ca38689e3b, State: Running, Role: LEADER
I20260709 13:07:27.416716 22464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c19b5a8350d74917bbc57cd93f2c23e2" candidate_uuid: "cb201644020643b485d68b0cd44e1b74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "396253bba01d4eb382f848b21f804d02"
I20260709 13:07:27.418165 22464 raft_consensus.cc:3060] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:27.417896 22389 consensus_queue.cc:260] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:27.419054 22389 raft_consensus.cc:3060] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:27.424657 22464 raft_consensus.cc:2468] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb201644020643b485d68b0cd44e1b74 in term 3.
I20260709 13:07:27.424837 22389 raft_consensus.cc:2468] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb201644020643b485d68b0cd44e1b74 in term 3.
I20260709 13:07:27.425949 22503 leader_election.cc:304] T c19b5a8350d74917bbc57cd93f2c23e2 P cb201644020643b485d68b0cd44e1b74 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 396253bba01d4eb382f848b21f804d02, cb201644020643b485d68b0cd44e1b74; no voters: 
I20260709 13:07:27.426614 22679 raft_consensus.cc:2804] T c19b5a8350d74917bbc57cd93f2c23e2 P cb201644020643b485d68b0cd44e1b74 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:27.427698 22679 raft_consensus.cc:697] T c19b5a8350d74917bbc57cd93f2c23e2 P cb201644020643b485d68b0cd44e1b74 [term 3 LEADER]: Becoming Leader. State: Replica: cb201644020643b485d68b0cd44e1b74, State: Running, Role: LEADER
I20260709 13:07:27.428620 22679 consensus_queue.cc:237] T c19b5a8350d74917bbc57cd93f2c23e2 P cb201644020643b485d68b0cd44e1b74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:27.435684 22283 catalog_manager.cc:5697] T c19b5a8350d74917bbc57cd93f2c23e2 P cb201644020643b485d68b0cd44e1b74 reported cstate change: term changed from 2 to 3, leader changed from 75d0a416460746daa0e458ca38689e3b (127.21.63.1) to cb201644020643b485d68b0cd44e1b74 (127.21.63.3). New cstate: current_term: 3 leader_uuid: "cb201644020643b485d68b0cd44e1b74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:27.601145 22658 raft_consensus.cc:993] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b: : Instructing follower cb201644020643b485d68b0cd44e1b74 to start an election
I20260709 13:07:27.601572 22684 raft_consensus.cc:1081] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [term 2 LEADER]: Signalling peer cb201644020643b485d68b0cd44e1b74 to start an election
I20260709 13:07:27.602934 22536 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dd07de1f75c54b4bba2d31d81e28ce54"
dest_uuid: "cb201644020643b485d68b0cd44e1b74"
 from {username='slave'} at 127.0.0.1:47008
I20260709 13:07:27.603412 22536 raft_consensus.cc:493] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:27.603685 22536 raft_consensus.cc:3060] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:27.608263 22536 raft_consensus.cc:515] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:27.609673 22536 leader_election.cc:290] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [CANDIDATE]: Term 3 election: Requested vote from peers 75d0a416460746daa0e458ca38689e3b (127.21.63.1:37185), 396253bba01d4eb382f848b21f804d02 (127.21.63.2:43161)
I20260709 13:07:27.610500 22389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd07de1f75c54b4bba2d31d81e28ce54" candidate_uuid: "cb201644020643b485d68b0cd44e1b74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "75d0a416460746daa0e458ca38689e3b"
I20260709 13:07:27.610705 22464 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd07de1f75c54b4bba2d31d81e28ce54" candidate_uuid: "cb201644020643b485d68b0cd44e1b74" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "396253bba01d4eb382f848b21f804d02"
I20260709 13:07:27.611048 22389 raft_consensus.cc:3055] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:27.611223 22464 raft_consensus.cc:3060] T dd07de1f75c54b4bba2d31d81e28ce54 P 396253bba01d4eb382f848b21f804d02 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:27.611356 22389 raft_consensus.cc:740] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 75d0a416460746daa0e458ca38689e3b, State: Running, Role: LEADER
I20260709 13:07:27.612067 22389 consensus_queue.cc:260] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:27.612957 22389 raft_consensus.cc:3060] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:27.615648 22464 raft_consensus.cc:2468] T dd07de1f75c54b4bba2d31d81e28ce54 P 396253bba01d4eb382f848b21f804d02 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb201644020643b485d68b0cd44e1b74 in term 3.
I20260709 13:07:27.616605 22503 leader_election.cc:304] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 396253bba01d4eb382f848b21f804d02, cb201644020643b485d68b0cd44e1b74; no voters: 
I20260709 13:07:27.616866 22389 raft_consensus.cc:2468] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb201644020643b485d68b0cd44e1b74 in term 3.
I20260709 13:07:27.617386 22679 raft_consensus.cc:2804] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:27.617882 22679 raft_consensus.cc:697] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [term 3 LEADER]: Becoming Leader. State: Replica: cb201644020643b485d68b0cd44e1b74, State: Running, Role: LEADER
I20260709 13:07:27.618562 22679 consensus_queue.cc:237] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:27.626339 22283 catalog_manager.cc:5697] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 reported cstate change: term changed from 2 to 3, leader changed from 75d0a416460746daa0e458ca38689e3b (127.21.63.1) to cb201644020643b485d68b0cd44e1b74 (127.21.63.3). New cstate: current_term: 3 leader_uuid: "cb201644020643b485d68b0cd44e1b74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:27.847767 22389 raft_consensus.cc:1275] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b [term 3 FOLLOWER]: Refusing update from remote peer cb201644020643b485d68b0cd44e1b74: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:27.848050 22464 raft_consensus.cc:1275] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02 [term 3 FOLLOWER]: Refusing update from remote peer cb201644020643b485d68b0cd44e1b74: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:27.849217 22685 consensus_queue.cc:1048] T c19b5a8350d74917bbc57cd93f2c23e2 P cb201644020643b485d68b0cd44e1b74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:27.850010 22679 consensus_queue.cc:1048] T c19b5a8350d74917bbc57cd93f2c23e2 P cb201644020643b485d68b0cd44e1b74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:28.077064 22389 raft_consensus.cc:1275] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [term 3 FOLLOWER]: Refusing update from remote peer cb201644020643b485d68b0cd44e1b74: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:28.078508 22679 consensus_queue.cc:1048] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:28.087050 22464 raft_consensus.cc:1275] T dd07de1f75c54b4bba2d31d81e28ce54 P 396253bba01d4eb382f848b21f804d02 [term 3 FOLLOWER]: Refusing update from remote peer cb201644020643b485d68b0cd44e1b74: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:28.088825 22685 consensus_queue.cc:1048] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:28.206571 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:28.220032 22536 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c19b5a8350d74917bbc57cd93f2c23e2"
dest_uuid: "cb201644020643b485d68b0cd44e1b74"
mode: GRACEFUL
new_leader_uuid: "75d0a416460746daa0e458ca38689e3b"
 from {username='slave'} at 127.0.0.1:47024
I20260709 13:07:28.220495 22536 raft_consensus.cc:606] T c19b5a8350d74917bbc57cd93f2c23e2 P cb201644020643b485d68b0cd44e1b74 [term 3 LEADER]: Received request to transfer leadership to TS 75d0a416460746daa0e458ca38689e3b
I20260709 13:07:28.231065 22536 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dd07de1f75c54b4bba2d31d81e28ce54"
dest_uuid: "cb201644020643b485d68b0cd44e1b74"
mode: GRACEFUL
new_leader_uuid: "75d0a416460746daa0e458ca38689e3b"
 from {username='slave'} at 127.0.0.1:47024
I20260709 13:07:28.231753 22536 raft_consensus.cc:606] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [term 3 LEADER]: Received request to transfer leadership to TS 75d0a416460746daa0e458ca38689e3b
I20260709 13:07:28.236603 22462 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fcc967d7308b4cb3a5d93bc77dfa4681"
dest_uuid: "396253bba01d4eb382f848b21f804d02"
mode: GRACEFUL
new_leader_uuid: "75d0a416460746daa0e458ca38689e3b"
 from {username='slave'} at 127.0.0.1:57308
I20260709 13:07:28.237149 22462 raft_consensus.cc:606] T fcc967d7308b4cb3a5d93bc77dfa4681 P 396253bba01d4eb382f848b21f804d02 [term 2 LEADER]: Received request to transfer leadership to TS 75d0a416460746daa0e458ca38689e3b
I20260709 13:07:28.242105 22462 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aadfca82d29a4b5e89d49830fbb46b5e"
dest_uuid: "396253bba01d4eb382f848b21f804d02"
mode: GRACEFUL
new_leader_uuid: "75d0a416460746daa0e458ca38689e3b"
 from {username='slave'} at 127.0.0.1:57308
I20260709 13:07:28.242693 22462 raft_consensus.cc:606] T aadfca82d29a4b5e89d49830fbb46b5e P 396253bba01d4eb382f848b21f804d02 [term 2 LEADER]: Received request to transfer leadership to TS 75d0a416460746daa0e458ca38689e3b
I20260709 13:07:28.247745 22462 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5aba9ad2b64a42ad90bd6f5c11cf2272"
dest_uuid: "396253bba01d4eb382f848b21f804d02"
mode: GRACEFUL
new_leader_uuid: "75d0a416460746daa0e458ca38689e3b"
 from {username='slave'} at 127.0.0.1:57308
I20260709 13:07:28.248271 22462 raft_consensus.cc:606] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 396253bba01d4eb382f848b21f804d02 [term 2 LEADER]: Received request to transfer leadership to TS 75d0a416460746daa0e458ca38689e3b
I20260709 13:07:28.328722 22701 raft_consensus.cc:993] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 396253bba01d4eb382f848b21f804d02: : Instructing follower 75d0a416460746daa0e458ca38689e3b to start an election
I20260709 13:07:28.329314 22683 raft_consensus.cc:1081] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 396253bba01d4eb382f848b21f804d02 [term 2 LEADER]: Signalling peer 75d0a416460746daa0e458ca38689e3b to start an election
I20260709 13:07:28.330856 22389 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5aba9ad2b64a42ad90bd6f5c11cf2272"
dest_uuid: "75d0a416460746daa0e458ca38689e3b"
 from {username='slave'} at 127.0.0.1:54200
I20260709 13:07:28.331332 22389 raft_consensus.cc:493] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 75d0a416460746daa0e458ca38689e3b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:28.331703 22389 raft_consensus.cc:3060] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 75d0a416460746daa0e458ca38689e3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:28.336125 22389 raft_consensus.cc:515] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 75d0a416460746daa0e458ca38689e3b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:28.337621 22389 leader_election.cc:290] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 75d0a416460746daa0e458ca38689e3b [CANDIDATE]: Term 3 election: Requested vote from peers 396253bba01d4eb382f848b21f804d02 (127.21.63.2:43161), cb201644020643b485d68b0cd44e1b74 (127.21.63.3:41079)
I20260709 13:07:28.338871 22462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5aba9ad2b64a42ad90bd6f5c11cf2272" candidate_uuid: "75d0a416460746daa0e458ca38689e3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "396253bba01d4eb382f848b21f804d02"
I20260709 13:07:28.338941 22538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5aba9ad2b64a42ad90bd6f5c11cf2272" candidate_uuid: "75d0a416460746daa0e458ca38689e3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cb201644020643b485d68b0cd44e1b74"
I20260709 13:07:28.339493 22462 raft_consensus.cc:3055] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 396253bba01d4eb382f848b21f804d02 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:28.339623 22538 raft_consensus.cc:3060] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:28.339918 22462 raft_consensus.cc:740] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 396253bba01d4eb382f848b21f804d02 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 396253bba01d4eb382f848b21f804d02, State: Running, Role: LEADER
I20260709 13:07:28.341122 22462 consensus_queue.cc:260] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 396253bba01d4eb382f848b21f804d02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:28.342223 22462 raft_consensus.cc:3060] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 396253bba01d4eb382f848b21f804d02 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:28.345019 22538 raft_consensus.cc:2468] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75d0a416460746daa0e458ca38689e3b in term 3.
I20260709 13:07:28.346184 22354 leader_election.cc:304] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 75d0a416460746daa0e458ca38689e3b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75d0a416460746daa0e458ca38689e3b, cb201644020643b485d68b0cd44e1b74; no voters: 
I20260709 13:07:28.347023 22684 raft_consensus.cc:2804] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 75d0a416460746daa0e458ca38689e3b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:28.347433 22462 raft_consensus.cc:2468] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 396253bba01d4eb382f848b21f804d02 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75d0a416460746daa0e458ca38689e3b in term 3.
I20260709 13:07:28.347488 22684 raft_consensus.cc:697] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 75d0a416460746daa0e458ca38689e3b [term 3 LEADER]: Becoming Leader. State: Replica: 75d0a416460746daa0e458ca38689e3b, State: Running, Role: LEADER
I20260709 13:07:28.348568 22684 consensus_queue.cc:237] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 75d0a416460746daa0e458ca38689e3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:28.355811 22284 catalog_manager.cc:5697] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 75d0a416460746daa0e458ca38689e3b reported cstate change: term changed from 2 to 3, leader changed from 396253bba01d4eb382f848b21f804d02 (127.21.63.2) to 75d0a416460746daa0e458ca38689e3b (127.21.63.1). New cstate: current_term: 3 leader_uuid: "75d0a416460746daa0e458ca38689e3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:28.419762 22701 raft_consensus.cc:993] T aadfca82d29a4b5e89d49830fbb46b5e P 396253bba01d4eb382f848b21f804d02: : Instructing follower 75d0a416460746daa0e458ca38689e3b to start an election
I20260709 13:07:28.420228 22683 raft_consensus.cc:1081] T aadfca82d29a4b5e89d49830fbb46b5e P 396253bba01d4eb382f848b21f804d02 [term 2 LEADER]: Signalling peer 75d0a416460746daa0e458ca38689e3b to start an election
I20260709 13:07:28.422055 22389 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aadfca82d29a4b5e89d49830fbb46b5e"
dest_uuid: "75d0a416460746daa0e458ca38689e3b"
 from {username='slave'} at 127.0.0.1:54200
I20260709 13:07:28.422603 22389 raft_consensus.cc:493] T aadfca82d29a4b5e89d49830fbb46b5e P 75d0a416460746daa0e458ca38689e3b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:28.422950 22389 raft_consensus.cc:3060] T aadfca82d29a4b5e89d49830fbb46b5e P 75d0a416460746daa0e458ca38689e3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:28.427325 22389 raft_consensus.cc:515] T aadfca82d29a4b5e89d49830fbb46b5e P 75d0a416460746daa0e458ca38689e3b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:28.429211 22389 leader_election.cc:290] T aadfca82d29a4b5e89d49830fbb46b5e P 75d0a416460746daa0e458ca38689e3b [CANDIDATE]: Term 3 election: Requested vote from peers 396253bba01d4eb382f848b21f804d02 (127.21.63.2:43161), cb201644020643b485d68b0cd44e1b74 (127.21.63.3:41079)
I20260709 13:07:28.431062 22462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadfca82d29a4b5e89d49830fbb46b5e" candidate_uuid: "75d0a416460746daa0e458ca38689e3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "396253bba01d4eb382f848b21f804d02"
I20260709 13:07:28.431159 22538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aadfca82d29a4b5e89d49830fbb46b5e" candidate_uuid: "75d0a416460746daa0e458ca38689e3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cb201644020643b485d68b0cd44e1b74"
I20260709 13:07:28.431818 22538 raft_consensus.cc:3060] T aadfca82d29a4b5e89d49830fbb46b5e P cb201644020643b485d68b0cd44e1b74 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:28.431753 22462 raft_consensus.cc:3055] T aadfca82d29a4b5e89d49830fbb46b5e P 396253bba01d4eb382f848b21f804d02 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:28.432871 22462 raft_consensus.cc:740] T aadfca82d29a4b5e89d49830fbb46b5e P 396253bba01d4eb382f848b21f804d02 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 396253bba01d4eb382f848b21f804d02, State: Running, Role: LEADER
I20260709 13:07:28.433727 22462 consensus_queue.cc:260] T aadfca82d29a4b5e89d49830fbb46b5e P 396253bba01d4eb382f848b21f804d02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:28.434772 22462 raft_consensus.cc:3060] T aadfca82d29a4b5e89d49830fbb46b5e P 396253bba01d4eb382f848b21f804d02 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:28.437129 22538 raft_consensus.cc:2468] T aadfca82d29a4b5e89d49830fbb46b5e P cb201644020643b485d68b0cd44e1b74 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75d0a416460746daa0e458ca38689e3b in term 3.
I20260709 13:07:28.438232 22354 leader_election.cc:304] T aadfca82d29a4b5e89d49830fbb46b5e P 75d0a416460746daa0e458ca38689e3b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75d0a416460746daa0e458ca38689e3b, cb201644020643b485d68b0cd44e1b74; no voters: 
I20260709 13:07:28.439042 22684 raft_consensus.cc:2804] T aadfca82d29a4b5e89d49830fbb46b5e P 75d0a416460746daa0e458ca38689e3b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:28.439438 22684 raft_consensus.cc:697] T aadfca82d29a4b5e89d49830fbb46b5e P 75d0a416460746daa0e458ca38689e3b [term 3 LEADER]: Becoming Leader. State: Replica: 75d0a416460746daa0e458ca38689e3b, State: Running, Role: LEADER
I20260709 13:07:28.439865 22462 raft_consensus.cc:2468] T aadfca82d29a4b5e89d49830fbb46b5e P 396253bba01d4eb382f848b21f804d02 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75d0a416460746daa0e458ca38689e3b in term 3.
I20260709 13:07:28.440274 22684 consensus_queue.cc:237] T aadfca82d29a4b5e89d49830fbb46b5e P 75d0a416460746daa0e458ca38689e3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:28.448027 22284 catalog_manager.cc:5697] T aadfca82d29a4b5e89d49830fbb46b5e P 75d0a416460746daa0e458ca38689e3b reported cstate change: term changed from 2 to 3, leader changed from 396253bba01d4eb382f848b21f804d02 (127.21.63.2) to 75d0a416460746daa0e458ca38689e3b (127.21.63.1). New cstate: current_term: 3 leader_uuid: "75d0a416460746daa0e458ca38689e3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:28.470230 22696 raft_consensus.cc:993] T c19b5a8350d74917bbc57cd93f2c23e2 P cb201644020643b485d68b0cd44e1b74: : Instructing follower 75d0a416460746daa0e458ca38689e3b to start an election
I20260709 13:07:28.470690 22685 raft_consensus.cc:1081] T c19b5a8350d74917bbc57cd93f2c23e2 P cb201644020643b485d68b0cd44e1b74 [term 3 LEADER]: Signalling peer 75d0a416460746daa0e458ca38689e3b to start an election
I20260709 13:07:28.471997 22389 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c19b5a8350d74917bbc57cd93f2c23e2"
dest_uuid: "75d0a416460746daa0e458ca38689e3b"
 from {username='slave'} at 127.0.0.1:54202
I20260709 13:07:28.472378 22389 raft_consensus.cc:493] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:28.472591 22389 raft_consensus.cc:3060] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:28.476365 22389 raft_consensus.cc:515] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:28.477701 22389 leader_election.cc:290] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b [CANDIDATE]: Term 4 election: Requested vote from peers 396253bba01d4eb382f848b21f804d02 (127.21.63.2:43161), cb201644020643b485d68b0cd44e1b74 (127.21.63.3:41079)
I20260709 13:07:28.478582 22462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c19b5a8350d74917bbc57cd93f2c23e2" candidate_uuid: "75d0a416460746daa0e458ca38689e3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "396253bba01d4eb382f848b21f804d02"
I20260709 13:07:28.479030 22462 raft_consensus.cc:3060] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:28.479429 22538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c19b5a8350d74917bbc57cd93f2c23e2" candidate_uuid: "75d0a416460746daa0e458ca38689e3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "cb201644020643b485d68b0cd44e1b74"
I20260709 13:07:28.479960 22538 raft_consensus.cc:3055] T c19b5a8350d74917bbc57cd93f2c23e2 P cb201644020643b485d68b0cd44e1b74 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:28.480247 22538 raft_consensus.cc:740] T c19b5a8350d74917bbc57cd93f2c23e2 P cb201644020643b485d68b0cd44e1b74 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cb201644020643b485d68b0cd44e1b74, State: Running, Role: LEADER
I20260709 13:07:28.481480 22696 raft_consensus.cc:993] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74: : Instructing follower 75d0a416460746daa0e458ca38689e3b to start an election
I20260709 13:07:28.481137 22538 consensus_queue.cc:260] T c19b5a8350d74917bbc57cd93f2c23e2 P cb201644020643b485d68b0cd44e1b74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:28.481899 22685 raft_consensus.cc:1081] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [term 3 LEADER]: Signalling peer 75d0a416460746daa0e458ca38689e3b to start an election
I20260709 13:07:28.482502 22538 raft_consensus.cc:3060] T c19b5a8350d74917bbc57cd93f2c23e2 P cb201644020643b485d68b0cd44e1b74 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:28.483273 22389 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dd07de1f75c54b4bba2d31d81e28ce54"
dest_uuid: "75d0a416460746daa0e458ca38689e3b"
 from {username='slave'} at 127.0.0.1:54202
I20260709 13:07:28.483695 22389 raft_consensus.cc:493] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:28.483966 22389 raft_consensus.cc:3060] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:28.485857 22462 raft_consensus.cc:2468] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75d0a416460746daa0e458ca38689e3b in term 4.
I20260709 13:07:28.487181 22354 leader_election.cc:304] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 396253bba01d4eb382f848b21f804d02, 75d0a416460746daa0e458ca38689e3b; no voters: 
I20260709 13:07:28.488170 22684 raft_consensus.cc:2804] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:28.488099 22389 raft_consensus.cc:515] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:28.488703 22684 raft_consensus.cc:697] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b [term 4 LEADER]: Becoming Leader. State: Replica: 75d0a416460746daa0e458ca38689e3b, State: Running, Role: LEADER
I20260709 13:07:28.489548 22538 raft_consensus.cc:2468] T c19b5a8350d74917bbc57cd93f2c23e2 P cb201644020643b485d68b0cd44e1b74 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75d0a416460746daa0e458ca38689e3b in term 4.
I20260709 13:07:28.489464 22684 consensus_queue.cc:237] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:28.489974 22389 leader_election.cc:290] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [CANDIDATE]: Term 4 election: Requested vote from peers 396253bba01d4eb382f848b21f804d02 (127.21.63.2:43161), cb201644020643b485d68b0cd44e1b74 (127.21.63.3:41079)
I20260709 13:07:28.491261 22536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd07de1f75c54b4bba2d31d81e28ce54" candidate_uuid: "75d0a416460746daa0e458ca38689e3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "cb201644020643b485d68b0cd44e1b74"
I20260709 13:07:28.491763 22536 raft_consensus.cc:3055] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:28.492046 22536 raft_consensus.cc:740] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: cb201644020643b485d68b0cd44e1b74, State: Running, Role: LEADER
I20260709 13:07:28.492192 22462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd07de1f75c54b4bba2d31d81e28ce54" candidate_uuid: "75d0a416460746daa0e458ca38689e3b" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "396253bba01d4eb382f848b21f804d02"
I20260709 13:07:28.492915 22462 raft_consensus.cc:3060] T dd07de1f75c54b4bba2d31d81e28ce54 P 396253bba01d4eb382f848b21f804d02 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:28.492746 22536 consensus_queue.cc:260] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:28.493930 22536 raft_consensus.cc:3060] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:28.496461 22284 catalog_manager.cc:5697] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b reported cstate change: term changed from 3 to 4, leader changed from cb201644020643b485d68b0cd44e1b74 (127.21.63.3) to 75d0a416460746daa0e458ca38689e3b (127.21.63.1). New cstate: current_term: 4 leader_uuid: "75d0a416460746daa0e458ca38689e3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:28.498131 22536 raft_consensus.cc:2468] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75d0a416460746daa0e458ca38689e3b in term 4.
I20260709 13:07:28.498970 22462 raft_consensus.cc:2468] T dd07de1f75c54b4bba2d31d81e28ce54 P 396253bba01d4eb382f848b21f804d02 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75d0a416460746daa0e458ca38689e3b in term 4.
I20260709 13:07:28.499045 22354 leader_election.cc:304] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75d0a416460746daa0e458ca38689e3b, cb201644020643b485d68b0cd44e1b74; no voters: 
I20260709 13:07:28.500391 22684 raft_consensus.cc:2804] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:28.500883 22684 raft_consensus.cc:697] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [term 4 LEADER]: Becoming Leader. State: Replica: 75d0a416460746daa0e458ca38689e3b, State: Running, Role: LEADER
I20260709 13:07:28.501847 22684 consensus_queue.cc:237] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:28.510509 22284 catalog_manager.cc:5697] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b reported cstate change: term changed from 3 to 4, leader changed from cb201644020643b485d68b0cd44e1b74 (127.21.63.3) to 75d0a416460746daa0e458ca38689e3b (127.21.63.1). New cstate: current_term: 4 leader_uuid: "75d0a416460746daa0e458ca38689e3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:28.610891 22701 raft_consensus.cc:993] T fcc967d7308b4cb3a5d93bc77dfa4681 P 396253bba01d4eb382f848b21f804d02: : Instructing follower 75d0a416460746daa0e458ca38689e3b to start an election
I20260709 13:07:28.611281 22701 raft_consensus.cc:1081] T fcc967d7308b4cb3a5d93bc77dfa4681 P 396253bba01d4eb382f848b21f804d02 [term 2 LEADER]: Signalling peer 75d0a416460746daa0e458ca38689e3b to start an election
I20260709 13:07:28.612831 22389 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fcc967d7308b4cb3a5d93bc77dfa4681"
dest_uuid: "75d0a416460746daa0e458ca38689e3b"
 from {username='slave'} at 127.0.0.1:54200
I20260709 13:07:28.613277 22389 raft_consensus.cc:493] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:28.613540 22389 raft_consensus.cc:3060] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:28.617334 22389 raft_consensus.cc:515] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:28.618605 22389 leader_election.cc:290] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [CANDIDATE]: Term 3 election: Requested vote from peers 396253bba01d4eb382f848b21f804d02 (127.21.63.2:43161), cb201644020643b485d68b0cd44e1b74 (127.21.63.3:41079)
I20260709 13:07:28.619429 22462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcc967d7308b4cb3a5d93bc77dfa4681" candidate_uuid: "75d0a416460746daa0e458ca38689e3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "396253bba01d4eb382f848b21f804d02"
I20260709 13:07:28.619648 22536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcc967d7308b4cb3a5d93bc77dfa4681" candidate_uuid: "75d0a416460746daa0e458ca38689e3b" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cb201644020643b485d68b0cd44e1b74"
I20260709 13:07:28.619894 22462 raft_consensus.cc:3055] T fcc967d7308b4cb3a5d93bc77dfa4681 P 396253bba01d4eb382f848b21f804d02 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:07:28.620178 22536 raft_consensus.cc:3060] T fcc967d7308b4cb3a5d93bc77dfa4681 P cb201644020643b485d68b0cd44e1b74 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:28.620196 22462 raft_consensus.cc:740] T fcc967d7308b4cb3a5d93bc77dfa4681 P 396253bba01d4eb382f848b21f804d02 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 396253bba01d4eb382f848b21f804d02, State: Running, Role: LEADER
I20260709 13:07:28.621016 22462 consensus_queue.cc:260] T fcc967d7308b4cb3a5d93bc77dfa4681 P 396253bba01d4eb382f848b21f804d02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:28.621794 22462 raft_consensus.cc:3060] T fcc967d7308b4cb3a5d93bc77dfa4681 P 396253bba01d4eb382f848b21f804d02 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:07:28.624200 22536 raft_consensus.cc:2468] T fcc967d7308b4cb3a5d93bc77dfa4681 P cb201644020643b485d68b0cd44e1b74 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75d0a416460746daa0e458ca38689e3b in term 3.
I20260709 13:07:28.625136 22354 leader_election.cc:304] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75d0a416460746daa0e458ca38689e3b, cb201644020643b485d68b0cd44e1b74; no voters: 
I20260709 13:07:28.625787 22684 raft_consensus.cc:2804] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:07:28.625963 22462 raft_consensus.cc:2468] T fcc967d7308b4cb3a5d93bc77dfa4681 P 396253bba01d4eb382f848b21f804d02 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75d0a416460746daa0e458ca38689e3b in term 3.
I20260709 13:07:28.626266 22684 raft_consensus.cc:697] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [term 3 LEADER]: Becoming Leader. State: Replica: 75d0a416460746daa0e458ca38689e3b, State: Running, Role: LEADER
I20260709 13:07:28.627074 22684 consensus_queue.cc:237] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:28.633955 22284 catalog_manager.cc:5697] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b reported cstate change: term changed from 2 to 3, leader changed from 396253bba01d4eb382f848b21f804d02 (127.21.63.2) to 75d0a416460746daa0e458ca38689e3b (127.21.63.1). New cstate: current_term: 3 leader_uuid: "75d0a416460746daa0e458ca38689e3b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:28.817994 22536 raft_consensus.cc:1275] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [term 3 FOLLOWER]: Refusing update from remote peer 75d0a416460746daa0e458ca38689e3b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:28.819219 22684 consensus_queue.cc:1048] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 75d0a416460746daa0e458ca38689e3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:28.832276 22462 raft_consensus.cc:1275] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 396253bba01d4eb382f848b21f804d02 [term 3 FOLLOWER]: Refusing update from remote peer 75d0a416460746daa0e458ca38689e3b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:28.833630 22464 raft_consensus.cc:1275] T aadfca82d29a4b5e89d49830fbb46b5e P 396253bba01d4eb382f848b21f804d02 [term 3 FOLLOWER]: Refusing update from remote peer 75d0a416460746daa0e458ca38689e3b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:28.835022 22684 consensus_queue.cc:1048] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 75d0a416460746daa0e458ca38689e3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:28.836154 22711 consensus_queue.cc:1048] T aadfca82d29a4b5e89d49830fbb46b5e P 75d0a416460746daa0e458ca38689e3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:28.842979 22536 raft_consensus.cc:1275] T aadfca82d29a4b5e89d49830fbb46b5e P cb201644020643b485d68b0cd44e1b74 [term 3 FOLLOWER]: Refusing update from remote peer 75d0a416460746daa0e458ca38689e3b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:28.844800 22711 consensus_queue.cc:1048] T aadfca82d29a4b5e89d49830fbb46b5e P 75d0a416460746daa0e458ca38689e3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:28.898451 22536 raft_consensus.cc:1275] T c19b5a8350d74917bbc57cd93f2c23e2 P cb201644020643b485d68b0cd44e1b74 [term 4 FOLLOWER]: Refusing update from remote peer 75d0a416460746daa0e458ca38689e3b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:07:28.899691 22684 consensus_queue.cc:1048] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:28.909559 22462 raft_consensus.cc:1275] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02 [term 4 FOLLOWER]: Refusing update from remote peer 75d0a416460746daa0e458ca38689e3b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:07:28.911666 22684 consensus_queue.cc:1048] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:29.057279 22536 raft_consensus.cc:1275] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [term 4 FOLLOWER]: Refusing update from remote peer 75d0a416460746daa0e458ca38689e3b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:07:29.058521 22684 consensus_queue.cc:1048] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:29.067098 22462 raft_consensus.cc:1275] T dd07de1f75c54b4bba2d31d81e28ce54 P 396253bba01d4eb382f848b21f804d02 [term 4 FOLLOWER]: Refusing update from remote peer 75d0a416460746daa0e458ca38689e3b: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:07:29.068545 22684 consensus_queue.cc:1048] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:29.086669 22462 raft_consensus.cc:1275] T fcc967d7308b4cb3a5d93bc77dfa4681 P 396253bba01d4eb382f848b21f804d02 [term 3 FOLLOWER]: Refusing update from remote peer 75d0a416460746daa0e458ca38689e3b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:29.087908 22684 consensus_queue.cc:1048] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:29.096346 22536 raft_consensus.cc:1275] T fcc967d7308b4cb3a5d93bc77dfa4681 P cb201644020643b485d68b0cd44e1b74 [term 3 FOLLOWER]: Refusing update from remote peer 75d0a416460746daa0e458ca38689e3b: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:07:29.098424 22715 consensus_queue.cc:1048] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:31.255930 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:31.263348 22389 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "101b2ac61327416ea7a64aa0261d2306"
dest_uuid: "75d0a416460746daa0e458ca38689e3b"
mode: GRACEFUL
new_leader_uuid: "396253bba01d4eb382f848b21f804d02"
 from {username='slave'} at 127.0.0.1:54282
I20260709 13:07:31.263940 22389 raft_consensus.cc:606] T 101b2ac61327416ea7a64aa0261d2306 P 75d0a416460746daa0e458ca38689e3b [term 1 LEADER]: Received request to transfer leadership to TS 396253bba01d4eb382f848b21f804d02
I20260709 13:07:31.266527 22389 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5aba9ad2b64a42ad90bd6f5c11cf2272"
dest_uuid: "75d0a416460746daa0e458ca38689e3b"
mode: GRACEFUL
new_leader_uuid: "cb201644020643b485d68b0cd44e1b74"
 from {username='slave'} at 127.0.0.1:54282
I20260709 13:07:31.266921 22389 raft_consensus.cc:606] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 75d0a416460746daa0e458ca38689e3b [term 3 LEADER]: Received request to transfer leadership to TS cb201644020643b485d68b0cd44e1b74
I20260709 13:07:31.269095 22389 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dd07de1f75c54b4bba2d31d81e28ce54"
dest_uuid: "75d0a416460746daa0e458ca38689e3b"
mode: GRACEFUL
new_leader_uuid: "396253bba01d4eb382f848b21f804d02"
 from {username='slave'} at 127.0.0.1:54282
I20260709 13:07:31.269486 22389 raft_consensus.cc:606] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [term 4 LEADER]: Received request to transfer leadership to TS 396253bba01d4eb382f848b21f804d02
I20260709 13:07:31.271763 22389 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fcc967d7308b4cb3a5d93bc77dfa4681"
dest_uuid: "75d0a416460746daa0e458ca38689e3b"
mode: GRACEFUL
new_leader_uuid: "cb201644020643b485d68b0cd44e1b74"
 from {username='slave'} at 127.0.0.1:54282
I20260709 13:07:31.272224 22389 raft_consensus.cc:606] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [term 3 LEADER]: Received request to transfer leadership to TS cb201644020643b485d68b0cd44e1b74
I20260709 13:07:31.273422 21756 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:07:31.273882 21756 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:07:31.332271 22715 raft_consensus.cc:993] T 101b2ac61327416ea7a64aa0261d2306 P 75d0a416460746daa0e458ca38689e3b: : Instructing follower 396253bba01d4eb382f848b21f804d02 to start an election
I20260709 13:07:31.333016 22723 raft_consensus.cc:1081] T 101b2ac61327416ea7a64aa0261d2306 P 75d0a416460746daa0e458ca38689e3b [term 1 LEADER]: Signalling peer 396253bba01d4eb382f848b21f804d02 to start an election
I20260709 13:07:31.334986 22462 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "101b2ac61327416ea7a64aa0261d2306"
dest_uuid: "396253bba01d4eb382f848b21f804d02"
 from {username='slave'} at 127.0.0.1:57306
I20260709 13:07:31.335466 22462 raft_consensus.cc:493] T 101b2ac61327416ea7a64aa0261d2306 P 396253bba01d4eb382f848b21f804d02 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:31.335826 22462 raft_consensus.cc:3060] T 101b2ac61327416ea7a64aa0261d2306 P 396253bba01d4eb382f848b21f804d02 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:31.342797 22462 raft_consensus.cc:515] T 101b2ac61327416ea7a64aa0261d2306 P 396253bba01d4eb382f848b21f804d02 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:31.344602 22462 leader_election.cc:290] T 101b2ac61327416ea7a64aa0261d2306 P 396253bba01d4eb382f848b21f804d02 [CANDIDATE]: Term 2 election: Requested vote from peers 75d0a416460746daa0e458ca38689e3b (127.21.63.1:37185), cb201644020643b485d68b0cd44e1b74 (127.21.63.3:41079)
I20260709 13:07:31.345435 22389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "101b2ac61327416ea7a64aa0261d2306" candidate_uuid: "396253bba01d4eb382f848b21f804d02" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "75d0a416460746daa0e458ca38689e3b"
I20260709 13:07:31.345914 22536 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "101b2ac61327416ea7a64aa0261d2306" candidate_uuid: "396253bba01d4eb382f848b21f804d02" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cb201644020643b485d68b0cd44e1b74"
I20260709 13:07:31.346298 22389 raft_consensus.cc:3055] T 101b2ac61327416ea7a64aa0261d2306 P 75d0a416460746daa0e458ca38689e3b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:07:31.346558 22536 raft_consensus.cc:3060] T 101b2ac61327416ea7a64aa0261d2306 P cb201644020643b485d68b0cd44e1b74 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:31.346714 22389 raft_consensus.cc:740] T 101b2ac61327416ea7a64aa0261d2306 P 75d0a416460746daa0e458ca38689e3b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 75d0a416460746daa0e458ca38689e3b, State: Running, Role: LEADER
I20260709 13:07:31.347663 22389 consensus_queue.cc:260] T 101b2ac61327416ea7a64aa0261d2306 P 75d0a416460746daa0e458ca38689e3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:31.348810 22389 raft_consensus.cc:3060] T 101b2ac61327416ea7a64aa0261d2306 P 75d0a416460746daa0e458ca38689e3b [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:31.351438 22536 raft_consensus.cc:2468] T 101b2ac61327416ea7a64aa0261d2306 P cb201644020643b485d68b0cd44e1b74 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 396253bba01d4eb382f848b21f804d02 in term 2.
I20260709 13:07:31.352617 22429 leader_election.cc:304] T 101b2ac61327416ea7a64aa0261d2306 P 396253bba01d4eb382f848b21f804d02 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 396253bba01d4eb382f848b21f804d02, cb201644020643b485d68b0cd44e1b74; no voters: 
I20260709 13:07:31.353520 22701 raft_consensus.cc:2804] T 101b2ac61327416ea7a64aa0261d2306 P 396253bba01d4eb382f848b21f804d02 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:31.353888 22389 raft_consensus.cc:2468] T 101b2ac61327416ea7a64aa0261d2306 P 75d0a416460746daa0e458ca38689e3b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 396253bba01d4eb382f848b21f804d02 in term 2.
I20260709 13:07:31.354605 22701 raft_consensus.cc:697] T 101b2ac61327416ea7a64aa0261d2306 P 396253bba01d4eb382f848b21f804d02 [term 2 LEADER]: Becoming Leader. State: Replica: 396253bba01d4eb382f848b21f804d02, State: Running, Role: LEADER
I20260709 13:07:31.355760 22701 consensus_queue.cc:237] T 101b2ac61327416ea7a64aa0261d2306 P 396253bba01d4eb382f848b21f804d02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:31.363862 22283 catalog_manager.cc:5697] T 101b2ac61327416ea7a64aa0261d2306 P 396253bba01d4eb382f848b21f804d02 reported cstate change: term changed from 1 to 2, leader changed from 75d0a416460746daa0e458ca38689e3b (127.21.63.1) to 396253bba01d4eb382f848b21f804d02 (127.21.63.2). New cstate: current_term: 2 leader_uuid: "396253bba01d4eb382f848b21f804d02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:31.664851 22715 raft_consensus.cc:993] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b: : Instructing follower cb201644020643b485d68b0cd44e1b74 to start an election
I20260709 13:07:31.665371 22723 raft_consensus.cc:1081] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [term 3 LEADER]: Signalling peer cb201644020643b485d68b0cd44e1b74 to start an election
I20260709 13:07:31.666793 22536 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fcc967d7308b4cb3a5d93bc77dfa4681"
dest_uuid: "cb201644020643b485d68b0cd44e1b74"
 from {username='slave'} at 127.0.0.1:47008
I20260709 13:07:31.667287 22536 raft_consensus.cc:493] T fcc967d7308b4cb3a5d93bc77dfa4681 P cb201644020643b485d68b0cd44e1b74 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:31.667591 22536 raft_consensus.cc:3060] T fcc967d7308b4cb3a5d93bc77dfa4681 P cb201644020643b485d68b0cd44e1b74 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:31.672147 22536 raft_consensus.cc:515] T fcc967d7308b4cb3a5d93bc77dfa4681 P cb201644020643b485d68b0cd44e1b74 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:31.673878 22536 leader_election.cc:290] T fcc967d7308b4cb3a5d93bc77dfa4681 P cb201644020643b485d68b0cd44e1b74 [CANDIDATE]: Term 4 election: Requested vote from peers 75d0a416460746daa0e458ca38689e3b (127.21.63.1:37185), 396253bba01d4eb382f848b21f804d02 (127.21.63.2:43161)
I20260709 13:07:31.674602 22389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcc967d7308b4cb3a5d93bc77dfa4681" candidate_uuid: "cb201644020643b485d68b0cd44e1b74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "75d0a416460746daa0e458ca38689e3b"
I20260709 13:07:31.674933 22462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fcc967d7308b4cb3a5d93bc77dfa4681" candidate_uuid: "cb201644020643b485d68b0cd44e1b74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "396253bba01d4eb382f848b21f804d02"
I20260709 13:07:31.675176 22389 raft_consensus.cc:3055] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:31.675426 22462 raft_consensus.cc:3060] T fcc967d7308b4cb3a5d93bc77dfa4681 P 396253bba01d4eb382f848b21f804d02 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:31.675598 22389 raft_consensus.cc:740] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 75d0a416460746daa0e458ca38689e3b, State: Running, Role: LEADER
I20260709 13:07:31.676364 22389 consensus_queue.cc:260] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:31.677115 22715 raft_consensus.cc:993] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 75d0a416460746daa0e458ca38689e3b: : Instructing follower cb201644020643b485d68b0cd44e1b74 to start an election
I20260709 13:07:31.677588 22723 raft_consensus.cc:1081] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 75d0a416460746daa0e458ca38689e3b [term 3 LEADER]: Signalling peer cb201644020643b485d68b0cd44e1b74 to start an election
I20260709 13:07:31.677668 22389 raft_consensus.cc:3060] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:31.679241 22538 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5aba9ad2b64a42ad90bd6f5c11cf2272"
dest_uuid: "cb201644020643b485d68b0cd44e1b74"
 from {username='slave'} at 127.0.0.1:47008
I20260709 13:07:31.679735 22538 raft_consensus.cc:493] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:31.680069 22538 raft_consensus.cc:3060] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:31.681892 22462 raft_consensus.cc:2468] T fcc967d7308b4cb3a5d93bc77dfa4681 P 396253bba01d4eb382f848b21f804d02 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb201644020643b485d68b0cd44e1b74 in term 4.
I20260709 13:07:31.683147 22503 leader_election.cc:304] T fcc967d7308b4cb3a5d93bc77dfa4681 P cb201644020643b485d68b0cd44e1b74 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 396253bba01d4eb382f848b21f804d02, cb201644020643b485d68b0cd44e1b74; no voters: 
I20260709 13:07:31.683892 22389 raft_consensus.cc:2468] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb201644020643b485d68b0cd44e1b74 in term 4.
I20260709 13:07:31.684010 22685 raft_consensus.cc:2804] T fcc967d7308b4cb3a5d93bc77dfa4681 P cb201644020643b485d68b0cd44e1b74 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:31.685407 22685 raft_consensus.cc:697] T fcc967d7308b4cb3a5d93bc77dfa4681 P cb201644020643b485d68b0cd44e1b74 [term 4 LEADER]: Becoming Leader. State: Replica: cb201644020643b485d68b0cd44e1b74, State: Running, Role: LEADER
I20260709 13:07:31.686550 22685 consensus_queue.cc:237] T fcc967d7308b4cb3a5d93bc77dfa4681 P cb201644020643b485d68b0cd44e1b74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:31.687067 22538 raft_consensus.cc:515] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:31.689370 22538 leader_election.cc:290] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [CANDIDATE]: Term 4 election: Requested vote from peers 75d0a416460746daa0e458ca38689e3b (127.21.63.1:37185), 396253bba01d4eb382f848b21f804d02 (127.21.63.2:43161)
I20260709 13:07:31.690080 22389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5aba9ad2b64a42ad90bd6f5c11cf2272" candidate_uuid: "cb201644020643b485d68b0cd44e1b74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "75d0a416460746daa0e458ca38689e3b"
I20260709 13:07:31.690430 22462 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5aba9ad2b64a42ad90bd6f5c11cf2272" candidate_uuid: "cb201644020643b485d68b0cd44e1b74" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "396253bba01d4eb382f848b21f804d02"
I20260709 13:07:31.690697 22389 raft_consensus.cc:3055] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 75d0a416460746daa0e458ca38689e3b [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:07:31.691017 22389 raft_consensus.cc:740] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 75d0a416460746daa0e458ca38689e3b [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 75d0a416460746daa0e458ca38689e3b, State: Running, Role: LEADER
I20260709 13:07:31.690968 22462 raft_consensus.cc:3060] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 396253bba01d4eb382f848b21f804d02 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:31.693900 22389 consensus_queue.cc:260] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 75d0a416460746daa0e458ca38689e3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:31.695036 22389 raft_consensus.cc:3060] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 75d0a416460746daa0e458ca38689e3b [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:07:31.697044 22283 catalog_manager.cc:5697] T fcc967d7308b4cb3a5d93bc77dfa4681 P cb201644020643b485d68b0cd44e1b74 reported cstate change: term changed from 3 to 4, leader changed from 75d0a416460746daa0e458ca38689e3b (127.21.63.1) to cb201644020643b485d68b0cd44e1b74 (127.21.63.3). New cstate: current_term: 4 leader_uuid: "cb201644020643b485d68b0cd44e1b74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:31.699560 22389 raft_consensus.cc:2468] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 75d0a416460746daa0e458ca38689e3b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb201644020643b485d68b0cd44e1b74 in term 4.
I20260709 13:07:31.700780 22503 leader_election.cc:304] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75d0a416460746daa0e458ca38689e3b, cb201644020643b485d68b0cd44e1b74; no voters: 
I20260709 13:07:31.699599 22462 raft_consensus.cc:2468] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 396253bba01d4eb382f848b21f804d02 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb201644020643b485d68b0cd44e1b74 in term 4.
I20260709 13:07:31.701682 22685 raft_consensus.cc:2804] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:07:31.702239 22685 raft_consensus.cc:697] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [term 4 LEADER]: Becoming Leader. State: Replica: cb201644020643b485d68b0cd44e1b74, State: Running, Role: LEADER
I20260709 13:07:31.704499 22723 raft_consensus.cc:993] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b: : Instructing follower 396253bba01d4eb382f848b21f804d02 to start an election
I20260709 13:07:31.703329 22685 consensus_queue.cc:237] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:31.705008 22723 raft_consensus.cc:1081] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [term 4 LEADER]: Signalling peer 396253bba01d4eb382f848b21f804d02 to start an election
I20260709 13:07:31.706773 22464 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "dd07de1f75c54b4bba2d31d81e28ce54"
dest_uuid: "396253bba01d4eb382f848b21f804d02"
 from {username='slave'} at 127.0.0.1:57306
I20260709 13:07:31.707378 22464 raft_consensus.cc:493] T dd07de1f75c54b4bba2d31d81e28ce54 P 396253bba01d4eb382f848b21f804d02 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:07:31.707684 22464 raft_consensus.cc:3060] T dd07de1f75c54b4bba2d31d81e28ce54 P 396253bba01d4eb382f848b21f804d02 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:31.713671 22464 raft_consensus.cc:515] T dd07de1f75c54b4bba2d31d81e28ce54 P 396253bba01d4eb382f848b21f804d02 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:31.715591 22464 leader_election.cc:290] T dd07de1f75c54b4bba2d31d81e28ce54 P 396253bba01d4eb382f848b21f804d02 [CANDIDATE]: Term 5 election: Requested vote from peers 75d0a416460746daa0e458ca38689e3b (127.21.63.1:37185), cb201644020643b485d68b0cd44e1b74 (127.21.63.3:41079)
I20260709 13:07:31.716495 22389 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd07de1f75c54b4bba2d31d81e28ce54" candidate_uuid: "396253bba01d4eb382f848b21f804d02" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "75d0a416460746daa0e458ca38689e3b"
I20260709 13:07:31.717139 22389 raft_consensus.cc:3055] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:07:31.717237 22538 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd07de1f75c54b4bba2d31d81e28ce54" candidate_uuid: "396253bba01d4eb382f848b21f804d02" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "cb201644020643b485d68b0cd44e1b74"
I20260709 13:07:31.717482 22389 raft_consensus.cc:740] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 75d0a416460746daa0e458ca38689e3b, State: Running, Role: LEADER
I20260709 13:07:31.717976 22538 raft_consensus.cc:3060] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:31.718442 22389 consensus_queue.cc:260] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:31.719651 22389 raft_consensus.cc:3060] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:07:31.719156 22283 catalog_manager.cc:5697] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 reported cstate change: term changed from 3 to 4, leader changed from 75d0a416460746daa0e458ca38689e3b (127.21.63.1) to cb201644020643b485d68b0cd44e1b74 (127.21.63.3). New cstate: current_term: 4 leader_uuid: "cb201644020643b485d68b0cd44e1b74" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:31.722899 22538 raft_consensus.cc:2468] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 396253bba01d4eb382f848b21f804d02 in term 5.
I20260709 13:07:31.723991 22429 leader_election.cc:304] T dd07de1f75c54b4bba2d31d81e28ce54 P 396253bba01d4eb382f848b21f804d02 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 396253bba01d4eb382f848b21f804d02, cb201644020643b485d68b0cd44e1b74; no voters: 
I20260709 13:07:31.724763 22701 raft_consensus.cc:2804] T dd07de1f75c54b4bba2d31d81e28ce54 P 396253bba01d4eb382f848b21f804d02 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:07:31.725267 22701 raft_consensus.cc:697] T dd07de1f75c54b4bba2d31d81e28ce54 P 396253bba01d4eb382f848b21f804d02 [term 5 LEADER]: Becoming Leader. State: Replica: 396253bba01d4eb382f848b21f804d02, State: Running, Role: LEADER
I20260709 13:07:31.726334 22389 raft_consensus.cc:2468] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 396253bba01d4eb382f848b21f804d02 in term 5.
I20260709 13:07:31.726224 22701 consensus_queue.cc:237] T dd07de1f75c54b4bba2d31d81e28ce54 P 396253bba01d4eb382f848b21f804d02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } }
I20260709 13:07:31.734145 22283 catalog_manager.cc:5697] T dd07de1f75c54b4bba2d31d81e28ce54 P 396253bba01d4eb382f848b21f804d02 reported cstate change: term changed from 4 to 5, leader changed from 75d0a416460746daa0e458ca38689e3b (127.21.63.1) to 396253bba01d4eb382f848b21f804d02 (127.21.63.2). New cstate: current_term: 5 leader_uuid: "396253bba01d4eb382f848b21f804d02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:31.798796 22538 raft_consensus.cc:1275] T 101b2ac61327416ea7a64aa0261d2306 P cb201644020643b485d68b0cd44e1b74 [term 2 FOLLOWER]: Refusing update from remote peer 396253bba01d4eb382f848b21f804d02: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:31.800061 22701 consensus_queue.cc:1048] T 101b2ac61327416ea7a64aa0261d2306 P 396253bba01d4eb382f848b21f804d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:31.811185 22389 raft_consensus.cc:1275] T 101b2ac61327416ea7a64aa0261d2306 P 75d0a416460746daa0e458ca38689e3b [term 2 FOLLOWER]: Refusing update from remote peer 396253bba01d4eb382f848b21f804d02: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:07:31.812848 22724 consensus_queue.cc:1048] T 101b2ac61327416ea7a64aa0261d2306 P 396253bba01d4eb382f848b21f804d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:32.084049 22389 raft_consensus.cc:1275] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 75d0a416460746daa0e458ca38689e3b [term 4 FOLLOWER]: Refusing update from remote peer cb201644020643b485d68b0cd44e1b74: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:07:32.085217 22685 consensus_queue.cc:1048] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:32.094550 22464 raft_consensus.cc:1275] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 396253bba01d4eb382f848b21f804d02 [term 4 FOLLOWER]: Refusing update from remote peer cb201644020643b485d68b0cd44e1b74: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:07:32.096714 22730 consensus_queue.cc:1048] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:07:32.223101 22389 raft_consensus.cc:1275] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [term 4 FOLLOWER]: Refusing update from remote peer cb201644020643b485d68b0cd44e1b74: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:07:32.224061 22464 raft_consensus.cc:1275] T fcc967d7308b4cb3a5d93bc77dfa4681 P 396253bba01d4eb382f848b21f804d02 [term 4 FOLLOWER]: Refusing update from remote peer cb201644020643b485d68b0cd44e1b74: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:07:32.224502 22730 consensus_queue.cc:1048] T fcc967d7308b4cb3a5d93bc77dfa4681 P cb201644020643b485d68b0cd44e1b74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:32.225317 22741 consensus_queue.cc:1048] T fcc967d7308b4cb3a5d93bc77dfa4681 P cb201644020643b485d68b0cd44e1b74 [LEADER]: Connected to new peer: Peer: permanent_uuid: "396253bba01d4eb382f848b21f804d02" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43161 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:32.258348 22389 raft_consensus.cc:1275] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [term 5 FOLLOWER]: Refusing update from remote peer 396253bba01d4eb382f848b21f804d02: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:07:32.259390 22724 consensus_queue.cc:1048] T dd07de1f75c54b4bba2d31d81e28ce54 P 396253bba01d4eb382f848b21f804d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75d0a416460746daa0e458ca38689e3b" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:07:32.267848 22538 raft_consensus.cc:1275] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [term 5 FOLLOWER]: Refusing update from remote peer 396253bba01d4eb382f848b21f804d02: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 13:07:32.271068 22736 consensus_queue.cc:1048] T dd07de1f75c54b4bba2d31d81e28ce54 P 396253bba01d4eb382f848b21f804d02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb201644020643b485d68b0cd44e1b74" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 41079 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:07:32.274853 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:07:32.281958 21756 tablet_server.cc:179] TabletServer@127.21.63.1:0 shutting down...
W20260709 13:07:32.294574 22429 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 4 similar messages]
W20260709 13:07:32.301396 22429 consensus_peers.cc:597] T 0eeeed324f544d639dab976cc4795bb3 P 396253bba01d4eb382f848b21f804d02 -> Peer 75d0a416460746daa0e458ca38689e3b (127.21.63.1:37185): Couldn't send request to peer 75d0a416460746daa0e458ca38689e3b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:32.321678 21756 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:07:32.322492 22503 consensus_peers.cc:597] T a9e5138d2b904033978bb1ac8ab3d39f P cb201644020643b485d68b0cd44e1b74 -> Peer 75d0a416460746daa0e458ca38689e3b (127.21.63.1:37185): Couldn't send request to peer 75d0a416460746daa0e458ca38689e3b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:32.322654 21756 tablet_replica.cc:333] T c80fcb082aa04657987302b6666fe031 P 75d0a416460746daa0e458ca38689e3b: stopping tablet replica
I20260709 13:07:32.323439 21756 raft_consensus.cc:2243] T c80fcb082aa04657987302b6666fe031 P 75d0a416460746daa0e458ca38689e3b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:32.324373 21756 raft_consensus.cc:2272] T c80fcb082aa04657987302b6666fe031 P 75d0a416460746daa0e458ca38689e3b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.326264 21756 tablet_replica.cc:333] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 75d0a416460746daa0e458ca38689e3b: stopping tablet replica
I20260709 13:07:32.326809 21756 raft_consensus.cc:2243] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 75d0a416460746daa0e458ca38689e3b [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.327234 21756 raft_consensus.cc:2272] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 75d0a416460746daa0e458ca38689e3b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.329700 21756 tablet_replica.cc:333] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b: stopping tablet replica
I20260709 13:07:32.330260 21756 raft_consensus.cc:2243] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.330677 21756 raft_consensus.cc:2272] T fcc967d7308b4cb3a5d93bc77dfa4681 P 75d0a416460746daa0e458ca38689e3b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.333190 21756 tablet_replica.cc:333] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b: stopping tablet replica
I20260709 13:07:32.333758 21756 raft_consensus.cc:2243] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.334231 21756 raft_consensus.cc:2272] T dd07de1f75c54b4bba2d31d81e28ce54 P 75d0a416460746daa0e458ca38689e3b [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.336464 21756 tablet_replica.cc:333] T aadfca82d29a4b5e89d49830fbb46b5e P 75d0a416460746daa0e458ca38689e3b: stopping tablet replica
I20260709 13:07:32.337052 21756 raft_consensus.cc:2243] T aadfca82d29a4b5e89d49830fbb46b5e P 75d0a416460746daa0e458ca38689e3b [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:07:32.337746 21756 raft_consensus.cc:2272] T aadfca82d29a4b5e89d49830fbb46b5e P 75d0a416460746daa0e458ca38689e3b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.339534 21756 tablet_replica.cc:333] T a9e5138d2b904033978bb1ac8ab3d39f P 75d0a416460746daa0e458ca38689e3b: stopping tablet replica
I20260709 13:07:32.340008 21756 raft_consensus.cc:2243] T a9e5138d2b904033978bb1ac8ab3d39f P 75d0a416460746daa0e458ca38689e3b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.340378 21756 raft_consensus.cc:2272] T a9e5138d2b904033978bb1ac8ab3d39f P 75d0a416460746daa0e458ca38689e3b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.342811 21756 tablet_replica.cc:333] T 9b21d45dd8e84d6cba59e68212962e65 P 75d0a416460746daa0e458ca38689e3b: stopping tablet replica
I20260709 13:07:32.343591 21756 raft_consensus.cc:2243] T 9b21d45dd8e84d6cba59e68212962e65 P 75d0a416460746daa0e458ca38689e3b [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:32.344493 21756 raft_consensus.cc:2272] T 9b21d45dd8e84d6cba59e68212962e65 P 75d0a416460746daa0e458ca38689e3b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.346402 21756 tablet_replica.cc:333] T 101b2ac61327416ea7a64aa0261d2306 P 75d0a416460746daa0e458ca38689e3b: stopping tablet replica
I20260709 13:07:32.346931 21756 raft_consensus.cc:2243] T 101b2ac61327416ea7a64aa0261d2306 P 75d0a416460746daa0e458ca38689e3b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.347363 21756 raft_consensus.cc:2272] T 101b2ac61327416ea7a64aa0261d2306 P 75d0a416460746daa0e458ca38689e3b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.349639 21756 tablet_replica.cc:333] T 0eeeed324f544d639dab976cc4795bb3 P 75d0a416460746daa0e458ca38689e3b: stopping tablet replica
I20260709 13:07:32.350176 21756 raft_consensus.cc:2243] T 0eeeed324f544d639dab976cc4795bb3 P 75d0a416460746daa0e458ca38689e3b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.350739 21756 raft_consensus.cc:2272] T 0eeeed324f544d639dab976cc4795bb3 P 75d0a416460746daa0e458ca38689e3b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.352350 21756 tablet_replica.cc:333] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b: stopping tablet replica
I20260709 13:07:32.352831 21756 raft_consensus.cc:2243] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:07:32.353540 21756 raft_consensus.cc:2272] T c19b5a8350d74917bbc57cd93f2c23e2 P 75d0a416460746daa0e458ca38689e3b [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.384229 21756 tablet_server.cc:196] TabletServer@127.21.63.1:0 shutdown complete.
I20260709 13:07:32.403942 21756 tablet_server.cc:179] TabletServer@127.21.63.2:0 shutting down...
I20260709 13:07:32.428280 21756 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:32.429121 21756 tablet_replica.cc:333] T fcc967d7308b4cb3a5d93bc77dfa4681 P 396253bba01d4eb382f848b21f804d02: stopping tablet replica
I20260709 13:07:32.430168 21756 raft_consensus.cc:2243] T fcc967d7308b4cb3a5d93bc77dfa4681 P 396253bba01d4eb382f848b21f804d02 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.430845 21756 raft_consensus.cc:2272] T fcc967d7308b4cb3a5d93bc77dfa4681 P 396253bba01d4eb382f848b21f804d02 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.433508 21756 tablet_replica.cc:333] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 396253bba01d4eb382f848b21f804d02: stopping tablet replica
I20260709 13:07:32.434199 21756 raft_consensus.cc:2243] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 396253bba01d4eb382f848b21f804d02 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.434616 21756 raft_consensus.cc:2272] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P 396253bba01d4eb382f848b21f804d02 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.436789 21756 tablet_replica.cc:333] T 101b2ac61327416ea7a64aa0261d2306 P 396253bba01d4eb382f848b21f804d02: stopping tablet replica
I20260709 13:07:32.437268 21756 raft_consensus.cc:2243] T 101b2ac61327416ea7a64aa0261d2306 P 396253bba01d4eb382f848b21f804d02 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:32.437911 21756 raft_consensus.cc:2272] T 101b2ac61327416ea7a64aa0261d2306 P 396253bba01d4eb382f848b21f804d02 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.439888 21756 tablet_replica.cc:333] T dd07de1f75c54b4bba2d31d81e28ce54 P 396253bba01d4eb382f848b21f804d02: stopping tablet replica
I20260709 13:07:32.440380 21756 raft_consensus.cc:2243] T dd07de1f75c54b4bba2d31d81e28ce54 P 396253bba01d4eb382f848b21f804d02 [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:07:32.441184 21756 raft_consensus.cc:2272] T dd07de1f75c54b4bba2d31d81e28ce54 P 396253bba01d4eb382f848b21f804d02 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.443080 21756 tablet_replica.cc:333] T 9b21d45dd8e84d6cba59e68212962e65 P 396253bba01d4eb382f848b21f804d02: stopping tablet replica
I20260709 13:07:32.443547 21756 raft_consensus.cc:2243] T 9b21d45dd8e84d6cba59e68212962e65 P 396253bba01d4eb382f848b21f804d02 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.443908 21756 raft_consensus.cc:2272] T 9b21d45dd8e84d6cba59e68212962e65 P 396253bba01d4eb382f848b21f804d02 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.445747 21756 tablet_replica.cc:333] T aadfca82d29a4b5e89d49830fbb46b5e P 396253bba01d4eb382f848b21f804d02: stopping tablet replica
I20260709 13:07:32.446201 21756 raft_consensus.cc:2243] T aadfca82d29a4b5e89d49830fbb46b5e P 396253bba01d4eb382f848b21f804d02 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.446575 21756 raft_consensus.cc:2272] T aadfca82d29a4b5e89d49830fbb46b5e P 396253bba01d4eb382f848b21f804d02 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.448175 21756 tablet_replica.cc:333] T a9e5138d2b904033978bb1ac8ab3d39f P 396253bba01d4eb382f848b21f804d02: stopping tablet replica
I20260709 13:07:32.448625 21756 raft_consensus.cc:2243] T a9e5138d2b904033978bb1ac8ab3d39f P 396253bba01d4eb382f848b21f804d02 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.449074 21756 raft_consensus.cc:2272] T a9e5138d2b904033978bb1ac8ab3d39f P 396253bba01d4eb382f848b21f804d02 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.450743 21756 tablet_replica.cc:333] T 0eeeed324f544d639dab976cc4795bb3 P 396253bba01d4eb382f848b21f804d02: stopping tablet replica
I20260709 13:07:32.451169 21756 raft_consensus.cc:2243] T 0eeeed324f544d639dab976cc4795bb3 P 396253bba01d4eb382f848b21f804d02 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:32.451912 21756 raft_consensus.cc:2272] T 0eeeed324f544d639dab976cc4795bb3 P 396253bba01d4eb382f848b21f804d02 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.453732 21756 tablet_replica.cc:333] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02: stopping tablet replica
I20260709 13:07:32.454170 21756 raft_consensus.cc:2243] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.454556 21756 raft_consensus.cc:2272] T c19b5a8350d74917bbc57cd93f2c23e2 P 396253bba01d4eb382f848b21f804d02 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.456136 21756 tablet_replica.cc:333] T c80fcb082aa04657987302b6666fe031 P 396253bba01d4eb382f848b21f804d02: stopping tablet replica
I20260709 13:07:32.456573 21756 raft_consensus.cc:2243] T c80fcb082aa04657987302b6666fe031 P 396253bba01d4eb382f848b21f804d02 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.457036 21756 raft_consensus.cc:2272] T c80fcb082aa04657987302b6666fe031 P 396253bba01d4eb382f848b21f804d02 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.485920 21756 tablet_server.cc:196] TabletServer@127.21.63.2:0 shutdown complete.
W20260709 13:07:32.491856 22503 consensus_peers.cc:597] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 -> Peer 75d0a416460746daa0e458ca38689e3b (127.21.63.1:37185): Couldn't send request to peer 75d0a416460746daa0e458ca38689e3b. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:37185: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:32.506750 21756 tablet_server.cc:179] TabletServer@127.21.63.3:0 shutting down...
I20260709 13:07:32.530902 21756 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:07:32.531677 21756 tablet_replica.cc:333] T 101b2ac61327416ea7a64aa0261d2306 P cb201644020643b485d68b0cd44e1b74: stopping tablet replica
I20260709 13:07:32.532305 21756 raft_consensus.cc:2243] T 101b2ac61327416ea7a64aa0261d2306 P cb201644020643b485d68b0cd44e1b74 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.532963 21756 raft_consensus.cc:2272] T 101b2ac61327416ea7a64aa0261d2306 P cb201644020643b485d68b0cd44e1b74 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.535157 21756 tablet_replica.cc:333] T 0eeeed324f544d639dab976cc4795bb3 P cb201644020643b485d68b0cd44e1b74: stopping tablet replica
I20260709 13:07:32.535704 21756 raft_consensus.cc:2243] T 0eeeed324f544d639dab976cc4795bb3 P cb201644020643b485d68b0cd44e1b74 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.536136 21756 raft_consensus.cc:2272] T 0eeeed324f544d639dab976cc4795bb3 P cb201644020643b485d68b0cd44e1b74 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.537923 21756 tablet_replica.cc:333] T c19b5a8350d74917bbc57cd93f2c23e2 P cb201644020643b485d68b0cd44e1b74: stopping tablet replica
I20260709 13:07:32.538478 21756 raft_consensus.cc:2243] T c19b5a8350d74917bbc57cd93f2c23e2 P cb201644020643b485d68b0cd44e1b74 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.538913 21756 raft_consensus.cc:2272] T c19b5a8350d74917bbc57cd93f2c23e2 P cb201644020643b485d68b0cd44e1b74 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.540633 21756 tablet_replica.cc:333] T c80fcb082aa04657987302b6666fe031 P cb201644020643b485d68b0cd44e1b74: stopping tablet replica
I20260709 13:07:32.541843 21756 raft_consensus.cc:2243] T c80fcb082aa04657987302b6666fe031 P cb201644020643b485d68b0cd44e1b74 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.542336 21756 raft_consensus.cc:2272] T c80fcb082aa04657987302b6666fe031 P cb201644020643b485d68b0cd44e1b74 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.544371 21756 tablet_replica.cc:333] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74: stopping tablet replica
I20260709 13:07:32.544996 21756 raft_consensus.cc:2243] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.545463 21756 raft_consensus.cc:2272] T dd07de1f75c54b4bba2d31d81e28ce54 P cb201644020643b485d68b0cd44e1b74 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.548125 21756 tablet_replica.cc:333] T aadfca82d29a4b5e89d49830fbb46b5e P cb201644020643b485d68b0cd44e1b74: stopping tablet replica
I20260709 13:07:32.548715 21756 raft_consensus.cc:2243] T aadfca82d29a4b5e89d49830fbb46b5e P cb201644020643b485d68b0cd44e1b74 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.549150 21756 raft_consensus.cc:2272] T aadfca82d29a4b5e89d49830fbb46b5e P cb201644020643b485d68b0cd44e1b74 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.550712 21756 tablet_replica.cc:333] T a9e5138d2b904033978bb1ac8ab3d39f P cb201644020643b485d68b0cd44e1b74: stopping tablet replica
I20260709 13:07:32.551167 21756 raft_consensus.cc:2243] T a9e5138d2b904033978bb1ac8ab3d39f P cb201644020643b485d68b0cd44e1b74 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:07:32.551887 21756 raft_consensus.cc:2272] T a9e5138d2b904033978bb1ac8ab3d39f P cb201644020643b485d68b0cd44e1b74 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.553542 21756 tablet_replica.cc:333] T 9b21d45dd8e84d6cba59e68212962e65 P cb201644020643b485d68b0cd44e1b74: stopping tablet replica
I20260709 13:07:32.554003 21756 raft_consensus.cc:2243] T 9b21d45dd8e84d6cba59e68212962e65 P cb201644020643b485d68b0cd44e1b74 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:32.554396 21756 raft_consensus.cc:2272] T 9b21d45dd8e84d6cba59e68212962e65 P cb201644020643b485d68b0cd44e1b74 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.555873 21756 tablet_replica.cc:333] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74: stopping tablet replica
I20260709 13:07:32.556332 21756 raft_consensus.cc:2243] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:07:32.557092 21756 raft_consensus.cc:2272] T 5aba9ad2b64a42ad90bd6f5c11cf2272 P cb201644020643b485d68b0cd44e1b74 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.558970 21756 tablet_replica.cc:333] T fcc967d7308b4cb3a5d93bc77dfa4681 P cb201644020643b485d68b0cd44e1b74: stopping tablet replica
I20260709 13:07:32.559465 21756 raft_consensus.cc:2243] T fcc967d7308b4cb3a5d93bc77dfa4681 P cb201644020643b485d68b0cd44e1b74 [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:07:32.560127 21756 raft_consensus.cc:2272] T fcc967d7308b4cb3a5d93bc77dfa4681 P cb201644020643b485d68b0cd44e1b74 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.589264 21756 tablet_server.cc:196] TabletServer@127.21.63.3:0 shutdown complete.
I20260709 13:07:32.607118 21756 master.cc:562] Master@127.21.63.62:46883 shutting down...
I20260709 13:07:32.625164 21756 raft_consensus.cc:2243] T 00000000000000000000000000000000 P fb83fe4dd9d6412ca3df972b6f2106cb [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:07:32.625689 21756 raft_consensus.cc:2272] T 00000000000000000000000000000000 P fb83fe4dd9d6412ca3df972b6f2106cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:32.626017 21756 tablet_replica.cc:333] T 00000000000000000000000000000000 P fb83fe4dd9d6412ca3df972b6f2106cb: stopping tablet replica
I20260709 13:07:32.647378 21756 master.cc:584] Master@127.21.63.62:46883 shutdown complete.
396253bba01d4eb382f848b21f804d02  4
75d0a416460746daa0e458ca38689e3b  5
cb201644020643b485d68b0cd44e1b74  1
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (10482 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 13:07:32.688628 21756 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.63.62:33399
I20260709 13:07:32.689911 21756 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:32.696338 22753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:32.696594 22754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:32.698251 22756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:32.698710 21756 server_base.cc:1061] running on GCE node
I20260709 13:07:32.699625 21756 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:32.699863 21756 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:32.700039 21756 hybrid_clock.cc:648] HybridClock initialized: now 1783602452700020 us; error 0 us; skew 500 ppm
I20260709 13:07:32.700590 21756 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:32.703192 21756 webserver.cc:533] Webserver started at http://127.21.63.62:44861/ using document root <none> and password file <none>
I20260709 13:07:32.703785 21756 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:32.703994 21756 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:32.704335 21756 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:32.705598 21756 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/master-0-root/instance:
uuid: "8d175b4ae72f4fc48971c2f31668971c"
format_stamp: "Formatted at 2026-07-09 13:07:32 on dist-test-slave-c6mt"
I20260709 13:07:32.710350 21756 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:07:32.713909 22761 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:32.714742 21756 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 13:07:32.715019 21756 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/master-0-root
uuid: "8d175b4ae72f4fc48971c2f31668971c"
format_stamp: "Formatted at 2026-07-09 13:07:32 on dist-test-slave-c6mt"
I20260709 13:07:32.715299 21756 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:32.743155 21756 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:32.744340 21756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:32.783262 21756 rpc_server.cc:307] RPC server started. Bound to: 127.21.63.62:33399
I20260709 13:07:32.783394 22812 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.63.62:33399 every 8 connection(s)
I20260709 13:07:32.787072 22813 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:32.796572 22813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d175b4ae72f4fc48971c2f31668971c: Bootstrap starting.
I20260709 13:07:32.801260 22813 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d175b4ae72f4fc48971c2f31668971c: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:32.805336 22813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d175b4ae72f4fc48971c2f31668971c: No bootstrap required, opened a new log
I20260709 13:07:32.807435 22813 raft_consensus.cc:359] T 00000000000000000000000000000000 P 8d175b4ae72f4fc48971c2f31668971c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d175b4ae72f4fc48971c2f31668971c" member_type: VOTER }
I20260709 13:07:32.807819 22813 raft_consensus.cc:385] T 00000000000000000000000000000000 P 8d175b4ae72f4fc48971c2f31668971c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:32.808076 22813 raft_consensus.cc:740] T 00000000000000000000000000000000 P 8d175b4ae72f4fc48971c2f31668971c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d175b4ae72f4fc48971c2f31668971c, State: Initialized, Role: FOLLOWER
I20260709 13:07:32.808792 22813 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d175b4ae72f4fc48971c2f31668971c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d175b4ae72f4fc48971c2f31668971c" member_type: VOTER }
I20260709 13:07:32.809242 22813 raft_consensus.cc:399] T 00000000000000000000000000000000 P 8d175b4ae72f4fc48971c2f31668971c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:07:32.809471 22813 raft_consensus.cc:493] T 00000000000000000000000000000000 P 8d175b4ae72f4fc48971c2f31668971c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:07:32.809746 22813 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 8d175b4ae72f4fc48971c2f31668971c [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:32.814028 22813 raft_consensus.cc:515] T 00000000000000000000000000000000 P 8d175b4ae72f4fc48971c2f31668971c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d175b4ae72f4fc48971c2f31668971c" member_type: VOTER }
I20260709 13:07:32.814531 22813 leader_election.cc:304] T 00000000000000000000000000000000 P 8d175b4ae72f4fc48971c2f31668971c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8d175b4ae72f4fc48971c2f31668971c; no voters: 
I20260709 13:07:32.815599 22813 leader_election.cc:290] T 00000000000000000000000000000000 P 8d175b4ae72f4fc48971c2f31668971c [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:07:32.816049 22816 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 8d175b4ae72f4fc48971c2f31668971c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:32.817445 22816 raft_consensus.cc:697] T 00000000000000000000000000000000 P 8d175b4ae72f4fc48971c2f31668971c [term 1 LEADER]: Becoming Leader. State: Replica: 8d175b4ae72f4fc48971c2f31668971c, State: Running, Role: LEADER
I20260709 13:07:32.818105 22816 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d175b4ae72f4fc48971c2f31668971c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d175b4ae72f4fc48971c2f31668971c" member_type: VOTER }
I20260709 13:07:32.818496 22813 sys_catalog.cc:565] T 00000000000000000000000000000000 P 8d175b4ae72f4fc48971c2f31668971c [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:07:32.822885 22818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d175b4ae72f4fc48971c2f31668971c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d175b4ae72f4fc48971c2f31668971c. Latest consensus state: current_term: 1 leader_uuid: "8d175b4ae72f4fc48971c2f31668971c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d175b4ae72f4fc48971c2f31668971c" member_type: VOTER } }
I20260709 13:07:32.823474 22818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d175b4ae72f4fc48971c2f31668971c [sys.catalog]: This master's current role is: LEADER
I20260709 13:07:32.824713 22817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d175b4ae72f4fc48971c2f31668971c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8d175b4ae72f4fc48971c2f31668971c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d175b4ae72f4fc48971c2f31668971c" member_type: VOTER } }
I20260709 13:07:32.825477 22817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d175b4ae72f4fc48971c2f31668971c [sys.catalog]: This master's current role is: LEADER
I20260709 13:07:32.826372 22824 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:07:32.835006 22824 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:07:32.835996 21756 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:07:32.844036 22824 catalog_manager.cc:1383] Generated new cluster ID: 2de93034bb054160a9ebc22a8921a7ec
I20260709 13:07:32.844409 22824 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:07:32.874054 22824 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:07:32.875378 22824 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:07:32.894120 22824 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 8d175b4ae72f4fc48971c2f31668971c: Generated new TSK 0
I20260709 13:07:32.894749 22824 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:07:32.902244 21756 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:32.908124 22834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:32.909106 22835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:32.911412 22837 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:32.911803 21756 server_base.cc:1061] running on GCE node
I20260709 13:07:32.913025 21756 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:32.913221 21756 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:32.913430 21756 hybrid_clock.cc:648] HybridClock initialized: now 1783602452913385 us; error 0 us; skew 500 ppm
I20260709 13:07:32.913965 21756 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:32.916175 21756 webserver.cc:533] Webserver started at http://127.21.63.1:38797/ using document root <none> and password file <none>
I20260709 13:07:32.916638 21756 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:32.916898 21756 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:32.917232 21756 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:32.918206 21756 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-0-root/instance:
uuid: "0a407504ed1741bf89a276558adf4c0f"
format_stamp: "Formatted at 2026-07-09 13:07:32 on dist-test-slave-c6mt"
I20260709 13:07:32.922683 21756 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:07:32.926007 22842 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:32.926800 21756 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:07:32.927188 21756 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-0-root
uuid: "0a407504ed1741bf89a276558adf4c0f"
format_stamp: "Formatted at 2026-07-09 13:07:32 on dist-test-slave-c6mt"
I20260709 13:07:32.927575 21756 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:32.953051 21756 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:32.954298 21756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:32.956310 21756 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:32.958760 21756 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:32.958985 21756 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:32.959256 21756 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:32.959437 21756 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:33.002910 21756 rpc_server.cc:307] RPC server started. Bound to: 127.21.63.1:33929
I20260709 13:07:33.003000 22904 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.63.1:33929 every 8 connection(s)
I20260709 13:07:33.008596 21756 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:33.016328 22909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:33.017273 22910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:33.019376 22912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:33.020879 21756 server_base.cc:1061] running on GCE node
I20260709 13:07:33.021919 21756 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:33.022269 21756 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:33.022511 21756 hybrid_clock.cc:648] HybridClock initialized: now 1783602453022491 us; error 0 us; skew 500 ppm
I20260709 13:07:33.023110 21756 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:33.023751 22905 heartbeater.cc:344] Connected to a master server at 127.21.63.62:33399
I20260709 13:07:33.024188 22905 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:33.025373 22905 heartbeater.cc:507] Master 127.21.63.62:33399 requested a full tablet report, sending...
I20260709 13:07:33.027240 21756 webserver.cc:533] Webserver started at http://127.21.63.2:43533/ using document root <none> and password file <none>
I20260709 13:07:33.027909 21756 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:33.028204 21756 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:33.028577 21756 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:33.028421 22778 ts_manager.cc:194] Registered new tserver with Master: 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929)
I20260709 13:07:33.030076 21756 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-1-root/instance:
uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
format_stamp: "Formatted at 2026-07-09 13:07:33 on dist-test-slave-c6mt"
I20260709 13:07:33.031067 22778 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33466
I20260709 13:07:33.035501 21756 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:07:33.038897 22917 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:33.039844 21756 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:07:33.040148 21756 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-1-root
uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
format_stamp: "Formatted at 2026-07-09 13:07:33 on dist-test-slave-c6mt"
I20260709 13:07:33.040457 21756 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:33.052276 21756 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:33.053480 21756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:33.054927 21756 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:33.057212 21756 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:33.057436 21756 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:33.057670 21756 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:33.057847 21756 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:33.097689 21756 rpc_server.cc:307] RPC server started. Bound to: 127.21.63.2:33309
I20260709 13:07:33.097808 22979 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.63.2:33309 every 8 connection(s)
I20260709 13:07:33.102375 21756 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:33.110332 22984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:33.112125 22983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:33.113147 22986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:33.114986 22980 heartbeater.cc:344] Connected to a master server at 127.21.63.62:33399
I20260709 13:07:33.115460 22980 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:33.116472 21756 server_base.cc:1061] running on GCE node
I20260709 13:07:33.116624 22980 heartbeater.cc:507] Master 127.21.63.62:33399 requested a full tablet report, sending...
I20260709 13:07:33.117624 21756 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:33.117883 21756 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:33.118139 21756 hybrid_clock.cc:648] HybridClock initialized: now 1783602453118117 us; error 0 us; skew 500 ppm
I20260709 13:07:33.118798 21756 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:33.118949 22778 ts_manager.cc:194] Registered new tserver with Master: aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:33.120460 22778 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33478
I20260709 13:07:33.121778 21756 webserver.cc:533] Webserver started at http://127.21.63.3:32909/ using document root <none> and password file <none>
I20260709 13:07:33.122315 21756 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:33.122541 21756 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:33.122828 21756 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:33.123980 21756 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-2-root/instance:
uuid: "f3c799b1249e40119d2a5864231c6c75"
format_stamp: "Formatted at 2026-07-09 13:07:33 on dist-test-slave-c6mt"
I20260709 13:07:33.128708 21756 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:07:33.132257 22991 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:33.133097 21756 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:07:33.133427 21756 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-2-root
uuid: "f3c799b1249e40119d2a5864231c6c75"
format_stamp: "Formatted at 2026-07-09 13:07:33 on dist-test-slave-c6mt"
I20260709 13:07:33.133733 21756 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:33.156872 21756 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:33.158027 21756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:33.159416 21756 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:33.161602 21756 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:33.161823 21756 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:33.162077 21756 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:33.162249 21756 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:33.204536 21756 rpc_server.cc:307] RPC server started. Bound to: 127.21.63.3:42499
I20260709 13:07:33.204659 23053 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.63.3:42499 every 8 connection(s)
I20260709 13:07:33.219185 23054 heartbeater.cc:344] Connected to a master server at 127.21.63.62:33399
I20260709 13:07:33.219599 23054 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:33.220507 23054 heartbeater.cc:507] Master 127.21.63.62:33399 requested a full tablet report, sending...
I20260709 13:07:33.222605 22778 ts_manager.cc:194] Registered new tserver with Master: f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:33.222713 21756 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014767427s
I20260709 13:07:33.224496 22778 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33488
I20260709 13:07:33.251078 22778 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33504:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:07:33.257149 22778 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:07:33.473142 22867 tablet_service.cc:1511] Processing CreateTablet for tablet dd740f20d2d74669b0fd10f1ec5b44f0 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:07:33.475332 22867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd740f20d2d74669b0fd10f1ec5b44f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.477415 22865 tablet_service.cc:1511] Processing CreateTablet for tablet ec334666b7964c969125af95d9c797ed (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:07:33.478971 22865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec334666b7964c969125af95d9c797ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.479566 22869 tablet_service.cc:1511] Processing CreateTablet for tablet 85d33f0fd80a49b8bd8cb73571ed3191 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:07:33.480902 22869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85d33f0fd80a49b8bd8cb73571ed3191. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.482206 22868 tablet_service.cc:1511] Processing CreateTablet for tablet 67e7fdd613434ffaacb8a030d4693490 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:07:33.483536 22868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67e7fdd613434ffaacb8a030d4693490. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.485046 22864 tablet_service.cc:1511] Processing CreateTablet for tablet edc3e3e3969d4c17b76da48ab861a54b (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:07:33.485831 22870 tablet_service.cc:1511] Processing CreateTablet for tablet 0c4ecd16880043f4a75006a3da4717bf (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:07:33.486457 22864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edc3e3e3969d4c17b76da48ab861a54b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.487133 22870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c4ecd16880043f4a75006a3da4717bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.489873 22863 tablet_service.cc:1511] Processing CreateTablet for tablet 7426119f4da44ffaa1513b481a866a01 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:07:33.478189 22866 tablet_service.cc:1511] Processing CreateTablet for tablet cbf52b91ec054ac6a2f8503462b1b7a6 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:07:33.491302 22863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7426119f4da44ffaa1513b481a866a01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.492177 22866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cbf52b91ec054ac6a2f8503462b1b7a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.498844 22862 tablet_service.cc:1511] Processing CreateTablet for tablet a0b6723ad2db4fccb7a1c219ff04e99a (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:07:33.500352 22862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0b6723ad2db4fccb7a1c219ff04e99a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.502859 22861 tablet_service.cc:1511] Processing CreateTablet for tablet b7633b7b4c334b039db57d77774df345 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:07:33.504397 23069 tablet_bootstrap.cc:492] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:33.506242 22869 tablet_service.cc:1511] Processing CreateTablet for tablet 5ab4700268fc4c67b9930087c5da71c8 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:07:33.513585 23019 tablet_service.cc:1511] Processing CreateTablet for tablet 0c4ecd16880043f4a75006a3da4717bf (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:07:33.515054 23019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c4ecd16880043f4a75006a3da4717bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.522509 22868 tablet_service.cc:1511] Processing CreateTablet for tablet f511a9ca0d9b4f54833aa7d0c092ccc1 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:07:33.525151 22944 tablet_service.cc:1511] Processing CreateTablet for tablet 85d33f0fd80a49b8bd8cb73571ed3191 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:07:33.526458 22944 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85d33f0fd80a49b8bd8cb73571ed3191. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.527583 22945 tablet_service.cc:1511] Processing CreateTablet for tablet 0c4ecd16880043f4a75006a3da4717bf (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:07:33.529057 22945 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c4ecd16880043f4a75006a3da4717bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.529840 22942 tablet_service.cc:1511] Processing CreateTablet for tablet dd740f20d2d74669b0fd10f1ec5b44f0 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:07:33.531100 22942 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd740f20d2d74669b0fd10f1ec5b44f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.533548 22866 tablet_service.cc:1511] Processing CreateTablet for tablet 104e2273f37843cbb5245981c817dfe0 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:07:33.534987 22939 tablet_service.cc:1511] Processing CreateTablet for tablet edc3e3e3969d4c17b76da48ab861a54b (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:07:33.535190 22936 tablet_service.cc:1511] Processing CreateTablet for tablet b7633b7b4c334b039db57d77774df345 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:07:33.536326 22939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edc3e3e3969d4c17b76da48ab861a54b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.538283 22936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7633b7b4c334b039db57d77774df345. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.537869 22938 tablet_service.cc:1511] Processing CreateTablet for tablet 7426119f4da44ffaa1513b481a866a01 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:07:33.539292 22938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7426119f4da44ffaa1513b481a866a01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.540403 22941 tablet_service.cc:1511] Processing CreateTablet for tablet cbf52b91ec054ac6a2f8503462b1b7a6 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:07:33.541838 22941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cbf52b91ec054ac6a2f8503462b1b7a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.542189 22870 tablet_service.cc:1511] Processing CreateTablet for tablet 616e98b5f7534394ae8790b08aec10e6 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:07:33.544178 22943 tablet_service.cc:1511] Processing CreateTablet for tablet 67e7fdd613434ffaacb8a030d4693490 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:07:33.545751 22943 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67e7fdd613434ffaacb8a030d4693490. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.533772 22937 tablet_service.cc:1511] Processing CreateTablet for tablet a0b6723ad2db4fccb7a1c219ff04e99a (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:07:33.550032 22937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0b6723ad2db4fccb7a1c219ff04e99a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.533772 22940 tablet_service.cc:1511] Processing CreateTablet for tablet ec334666b7964c969125af95d9c797ed (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:07:33.554070 22940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec334666b7964c969125af95d9c797ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.569842 22861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7633b7b4c334b039db57d77774df345. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.570729 22868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f511a9ca0d9b4f54833aa7d0c092ccc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.577111 22869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ab4700268fc4c67b9930087c5da71c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.580974 22870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 616e98b5f7534394ae8790b08aec10e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.584977 22866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 104e2273f37843cbb5245981c817dfe0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.586674 23070 tablet_bootstrap.cc:492] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:33.588399 22945 tablet_service.cc:1511] Processing CreateTablet for tablet 5ab4700268fc4c67b9930087c5da71c8 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:07:33.589812 22945 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ab4700268fc4c67b9930087c5da71c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.602828 23070 tablet_bootstrap.cc:654] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:33.603845 22868 tablet_service.cc:1511] Processing CreateTablet for tablet 2af6865189dc47369064cfad3fd137e1 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:07:33.605269 22868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2af6865189dc47369064cfad3fd137e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.605650 23018 tablet_service.cc:1511] Processing CreateTablet for tablet 85d33f0fd80a49b8bd8cb73571ed3191 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:07:33.606906 23018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85d33f0fd80a49b8bd8cb73571ed3191. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.616173 23017 tablet_service.cc:1511] Processing CreateTablet for tablet 67e7fdd613434ffaacb8a030d4693490 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:07:33.617569 23017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67e7fdd613434ffaacb8a030d4693490. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.620442 22945 tablet_service.cc:1511] Processing CreateTablet for tablet f511a9ca0d9b4f54833aa7d0c092ccc1 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:07:33.620965 23016 tablet_service.cc:1511] Processing CreateTablet for tablet dd740f20d2d74669b0fd10f1ec5b44f0 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:07:33.621822 22945 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f511a9ca0d9b4f54833aa7d0c092ccc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.622252 23016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd740f20d2d74669b0fd10f1ec5b44f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.625806 23015 tablet_service.cc:1511] Processing CreateTablet for tablet cbf52b91ec054ac6a2f8503462b1b7a6 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:07:33.626113 23011 tablet_service.cc:1511] Processing CreateTablet for tablet a0b6723ad2db4fccb7a1c219ff04e99a (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:07:33.628751 23015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cbf52b91ec054ac6a2f8503462b1b7a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.634224 23012 tablet_service.cc:1511] Processing CreateTablet for tablet 7426119f4da44ffaa1513b481a866a01 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:07:33.635584 23012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7426119f4da44ffaa1513b481a866a01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.635264 23013 tablet_service.cc:1511] Processing CreateTablet for tablet edc3e3e3969d4c17b76da48ab861a54b (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:07:33.636525 23013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet edc3e3e3969d4c17b76da48ab861a54b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.649082 23011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0b6723ad2db4fccb7a1c219ff04e99a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.625813 23010 tablet_service.cc:1511] Processing CreateTablet for tablet b7633b7b4c334b039db57d77774df345 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:07:33.655689 23010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7633b7b4c334b039db57d77774df345. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.641646 23069 tablet_bootstrap.cc:654] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:33.634227 23014 tablet_service.cc:1511] Processing CreateTablet for tablet ec334666b7964c969125af95d9c797ed (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:07:33.662272 22868 tablet_service.cc:1511] Processing CreateTablet for tablet 0f076a0e47a14c2b933d3058063c2bcd (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:07:33.663695 22868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f076a0e47a14c2b933d3058063c2bcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.669051 23014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec334666b7964c969125af95d9c797ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.680262 22939 tablet_service.cc:1511] Processing CreateTablet for tablet 616e98b5f7534394ae8790b08aec10e6 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:07:33.681705 22939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 616e98b5f7534394ae8790b08aec10e6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:07:33.684866 22764 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 52719.7us
I20260709 13:07:33.686479 22943 tablet_service.cc:1511] Processing CreateTablet for tablet 2af6865189dc47369064cfad3fd137e1 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:07:33.693840 22941 tablet_service.cc:1511] Processing CreateTablet for tablet 0f076a0e47a14c2b933d3058063c2bcd (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:07:33.694605 22940 tablet_service.cc:1511] Processing CreateTablet for tablet 7f35ebcf93ad423ea8bd0b777bee23ee (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:07:33.698097 22936 tablet_service.cc:1511] Processing CreateTablet for tablet 13e53d9e00624dfcb87aca9b055fdc9a (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:07:33.699150 22943 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2af6865189dc47369064cfad3fd137e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.700975 22940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f35ebcf93ad423ea8bd0b777bee23ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.703994 22863 tablet_service.cc:1511] Processing CreateTablet for tablet 13e53d9e00624dfcb87aca9b055fdc9a (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:07:33.705232 22863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13e53d9e00624dfcb87aca9b055fdc9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.709748 22942 tablet_service.cc:1511] Processing CreateTablet for tablet 392f5a2324d74a499e6baad745c941fd (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:07:33.671661 22938 tablet_service.cc:1511] Processing CreateTablet for tablet 104e2273f37843cbb5245981c817dfe0 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:07:33.716045 22936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13e53d9e00624dfcb87aca9b055fdc9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.717265 22938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 104e2273f37843cbb5245981c817dfe0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.745777 22941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f076a0e47a14c2b933d3058063c2bcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.758193 23070 tablet_bootstrap.cc:492] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:33.760075 23070 ts_tablet_manager.cc:1403] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.174s	user 0.018s	sys 0.004s
I20260709 13:07:33.763152 23070 raft_consensus.cc:359] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:33.763947 23070 raft_consensus.cc:385] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:33.764292 23070 raft_consensus.cc:740] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:33.766525 22867 tablet_service.cc:1511] Processing CreateTablet for tablet 392f5a2324d74a499e6baad745c941fd (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:07:33.767982 22867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 392f5a2324d74a499e6baad745c941fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.769353 23011 tablet_service.cc:1511] Processing CreateTablet for tablet 2af6865189dc47369064cfad3fd137e1 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:07:33.780992 23011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2af6865189dc47369064cfad3fd137e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.772641 22940 tablet_service.cc:1511] Processing CreateTablet for tablet aa6540758938493d98d7852fdbe3ffab (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:07:33.773797 22869 tablet_service.cc:1511] Processing CreateTablet for tablet 8a996de6f02a4723aef91074b3d56714 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:07:33.787309 22869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a996de6f02a4723aef91074b3d56714. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.776453 23069 tablet_bootstrap.cc:492] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:33.777593 22939 tablet_service.cc:1511] Processing CreateTablet for tablet 8217046fd4e14cc4a0194da4f6ad3887 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:07:33.778900 22870 tablet_service.cc:1511] Processing CreateTablet for tablet f3b08335f3564868a8160dc1b5ade501 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:07:33.789022 23069 ts_tablet_manager.cc:1403] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.285s	user 0.027s	sys 0.007s
I20260709 13:07:33.789853 22870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3b08335f3564868a8160dc1b5ade501. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.791185 23012 tablet_service.cc:1511] Processing CreateTablet for tablet 5ab4700268fc4c67b9930087c5da71c8 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:07:33.751748 22864 tablet_service.cc:1511] Processing CreateTablet for tablet 8217046fd4e14cc4a0194da4f6ad3887 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:07:33.800024 22866 tablet_service.cc:1511] Processing CreateTablet for tablet 7f35ebcf93ad423ea8bd0b777bee23ee (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:07:33.801260 22864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8217046fd4e14cc4a0194da4f6ad3887. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.805253 22866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f35ebcf93ad423ea8bd0b777bee23ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.807381 23014 tablet_service.cc:1511] Processing CreateTablet for tablet 616e98b5f7534394ae8790b08aec10e6 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:07:33.807847 22868 tablet_service.cc:1511] Processing CreateTablet for tablet 579096ba88474cfbbe40e8bf769879ec (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:07:33.809145 22868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 579096ba88474cfbbe40e8bf769879ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.819384 22863 tablet_service.cc:1511] Processing CreateTablet for tablet 612dc49c81064f4faa79f65fcf1cfc02 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:07:33.821545 22863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 612dc49c81064f4faa79f65fcf1cfc02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.791929 23069 raft_consensus.cc:359] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:33.770256 22862 tablet_service.cc:1511] Processing CreateTablet for tablet aa6540758938493d98d7852fdbe3ffab (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:07:33.823469 23069 raft_consensus.cc:385] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:33.823860 23069 raft_consensus.cc:740] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:33.798882 23015 tablet_service.cc:1511] Processing CreateTablet for tablet 104e2273f37843cbb5245981c817dfe0 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:07:33.824051 22862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa6540758938493d98d7852fdbe3ffab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.822786 22870 tablet_service.cc:1511] Processing CreateTablet for tablet 91c88873d54149c58a20f4cf6194e5fc (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:07:33.827731 22870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91c88873d54149c58a20f4cf6194e5fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.794499 23016 tablet_service.cc:1511] Processing CreateTablet for tablet f511a9ca0d9b4f54833aa7d0c092ccc1 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
W20260709 13:07:33.830185 22826 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:33.834350 22865 tablet_service.cc:1511] Processing CreateTablet for tablet b7e4ee754b7a4408b5fc55259d431f39 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:07:33.835853 22865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7e4ee754b7a4408b5fc55259d431f39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.842484 22939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8217046fd4e14cc4a0194da4f6ad3887. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.843678 22938 tablet_service.cc:1511] Processing CreateTablet for tablet f3b08335f3564868a8160dc1b5ade501 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:07:33.845058 22938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3b08335f3564868a8160dc1b5ade501. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.847635 22944 tablet_service.cc:1511] Processing CreateTablet for tablet 579096ba88474cfbbe40e8bf769879ec (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:07:33.846570 23069 consensus_queue.cc:260] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:33.848991 22944 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 579096ba88474cfbbe40e8bf769879ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.851346 22937 tablet_service.cc:1511] Processing CreateTablet for tablet 8a996de6f02a4723aef91074b3d56714 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:07:33.853245 23070 consensus_queue.cc:260] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:33.853528 22868 tablet_service.cc:1511] Processing CreateTablet for tablet 7ca8b9a08f4146c386bc06d03c497038 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:07:33.854992 22868 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ca8b9a08f4146c386bc06d03c497038. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.853027 22869 tablet_service.cc:1511] Processing CreateTablet for tablet 1120807254f5444997e16f6a7feece4d (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:07:33.860276 23010 tablet_service.cc:1511] Processing CreateTablet for tablet 0f076a0e47a14c2b933d3058063c2bcd (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:07:33.861511 23014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 616e98b5f7534394ae8790b08aec10e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.862187 23015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 104e2273f37843cbb5245981c817dfe0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.861507 22869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1120807254f5444997e16f6a7feece4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.863629 22945 tablet_service.cc:1511] Processing CreateTablet for tablet ac5cc94926a742e6a0c1340ec33c0ff0 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:07:33.863912 22943 tablet_service.cc:1511] Processing CreateTablet for tablet ec7dd25342084943aacb4b9830f5f1ea (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:07:33.862187 22940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa6540758938493d98d7852fdbe3ffab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.865356 22943 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec7dd25342084943aacb4b9830f5f1ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.865868 23016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f511a9ca0d9b4f54833aa7d0c092ccc1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.868343 22945 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac5cc94926a742e6a0c1340ec33c0ff0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.869942 23013 tablet_service.cc:1511] Processing CreateTablet for tablet 7f35ebcf93ad423ea8bd0b777bee23ee (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:07:33.870983 23012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ab4700268fc4c67b9930087c5da71c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.851567 22867 tablet_service.cc:1511] Processing CreateTablet for tablet ec7dd25342084943aacb4b9830f5f1ea (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:07:33.872836 22867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec7dd25342084943aacb4b9830f5f1ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.873005 23019 tablet_service.cc:1511] Processing CreateTablet for tablet 13e53d9e00624dfcb87aca9b055fdc9a (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:07:33.819286 22861 tablet_service.cc:1511] Processing CreateTablet for tablet ac5cc94926a742e6a0c1340ec33c0ff0 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:07:33.874954 22861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac5cc94926a742e6a0c1340ec33c0ff0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.883220 22936 tablet_service.cc:1511] Processing CreateTablet for tablet 91c88873d54149c58a20f4cf6194e5fc (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:07:33.888358 22942 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 392f5a2324d74a499e6baad745c941fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.893198 22937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a996de6f02a4723aef91074b3d56714. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.893543 23073 tablet_bootstrap.cc:492] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:33.902655 23070 ts_tablet_manager.cc:1434] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.142s	user 0.005s	sys 0.002s
I20260709 13:07:33.903452 22980 heartbeater.cc:499] Master 127.21.63.62:33399 was elected leader, sending a full tablet report...
I20260709 13:07:33.903784 23070 tablet_bootstrap.cc:492] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:33.907171 23018 tablet_service.cc:1511] Processing CreateTablet for tablet 91c88873d54149c58a20f4cf6194e5fc (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:07:33.892359 22863 tablet_service.cc:1511] Processing CreateTablet for tablet 2959101199a44a4583a4ee456ecf044d (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:07:33.909794 22863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2959101199a44a4583a4ee456ecf044d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.926023 23017 tablet_service.cc:1511] Processing CreateTablet for tablet ac5cc94926a742e6a0c1340ec33c0ff0 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:07:33.932767 22936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91c88873d54149c58a20f4cf6194e5fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.936081 23013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f35ebcf93ad423ea8bd0b777bee23ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.950614 23010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f076a0e47a14c2b933d3058063c2bcd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.956960 23018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91c88873d54149c58a20f4cf6194e5fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.956880 22939 tablet_service.cc:1511] Processing CreateTablet for tablet b7e4ee754b7a4408b5fc55259d431f39 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:07:33.959245 22939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7e4ee754b7a4408b5fc55259d431f39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.961738 23070 tablet_bootstrap.cc:654] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:33.961524 22937 tablet_service.cc:1511] Processing CreateTablet for tablet 7ca8b9a08f4146c386bc06d03c497038 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:07:33.963476 22941 tablet_service.cc:1511] Processing CreateTablet for tablet 2959101199a44a4583a4ee456ecf044d (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:07:33.963945 22870 tablet_service.cc:1511] Processing CreateTablet for tablet a424231ef7b4470bbd575564909a365e (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:07:33.966310 23019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13e53d9e00624dfcb87aca9b055fdc9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.932432 22938 tablet_service.cc:1511] Processing CreateTablet for tablet 612dc49c81064f4faa79f65fcf1cfc02 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:07:33.974249 23074 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:33.974779 22862 tablet_service.cc:1511] Processing CreateTablet for tablet c26347a647b240d7a3bc5cd449b7993b (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:07:33.975551 22905 heartbeater.cc:499] Master 127.21.63.62:33399 was elected leader, sending a full tablet report...
I20260709 13:07:33.977757 23017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac5cc94926a742e6a0c1340ec33c0ff0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.974783 23074 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:33.987231 22870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a424231ef7b4470bbd575564909a365e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:33.991843 23073 tablet_bootstrap.cc:654] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:33.995041 22867 tablet_service.cc:1511] Processing CreateTablet for tablet fab4388fa7ea49bda9fa070cf542577f (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:07:34.004958 22945 tablet_service.cc:1511] Processing CreateTablet for tablet 1120807254f5444997e16f6a7feece4d (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:07:34.005177 22942 tablet_service.cc:1511] Processing CreateTablet for tablet c26347a647b240d7a3bc5cd449b7993b (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:07:33.981511 22937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ca8b9a08f4146c386bc06d03c497038. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.008589 23011 tablet_service.cc:1511] Processing CreateTablet for tablet 392f5a2324d74a499e6baad745c941fd (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:07:33.984582 23074 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:34.028419 23015 tablet_service.cc:1511] Processing CreateTablet for tablet f3b08335f3564868a8160dc1b5ade501 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:07:34.029044 23016 tablet_service.cc:1511] Processing CreateTablet for tablet 8217046fd4e14cc4a0194da4f6ad3887 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:07:34.029789 23015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f3b08335f3564868a8160dc1b5ade501. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.030416 23016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8217046fd4e14cc4a0194da4f6ad3887. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.041294 22940 tablet_service.cc:1511] Processing CreateTablet for tablet a424231ef7b4470bbd575564909a365e (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:07:34.041422 23014 tablet_service.cc:1511] Processing CreateTablet for tablet aa6540758938493d98d7852fdbe3ffab (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:07:34.045131 23069 ts_tablet_manager.cc:1434] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.256s	user 0.015s	sys 0.001s
I20260709 13:07:34.047398 23011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 392f5a2324d74a499e6baad745c941fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.053066 23014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa6540758938493d98d7852fdbe3ffab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.044891 22866 tablet_service.cc:1511] Processing CreateTablet for tablet f603724630e44ef78014e7719f096cb3 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:07:34.054685 22866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f603724630e44ef78014e7719f096cb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.056860 23069 tablet_bootstrap.cc:492] T 67e7fdd613434ffaacb8a030d4693490 P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:34.054978 22864 tablet_service.cc:1511] Processing CreateTablet for tablet 6bdc53dc56134e28bcc43e68e76055fc (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:07:34.057403 22867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fab4388fa7ea49bda9fa070cf542577f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.055817 22941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2959101199a44a4583a4ee456ecf044d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.060043 22862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c26347a647b240d7a3bc5cd449b7993b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.062052 22942 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c26347a647b240d7a3bc5cd449b7993b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.077236 22940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a424231ef7b4470bbd575564909a365e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.088862 23012 tablet_service.cc:1511] Processing CreateTablet for tablet 8a996de6f02a4723aef91074b3d56714 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:07:34.092486 22945 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1120807254f5444997e16f6a7feece4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.091033 23012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a996de6f02a4723aef91074b3d56714. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.103030 22861 tablet_service.cc:1511] Processing CreateTablet for tablet cdebaf291e5d4ae9814a80ce81ee19fe (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:07:34.116353 23070 tablet_bootstrap.cc:492] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:34.117041 23070 ts_tablet_manager.cc:1403] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.214s	user 0.012s	sys 0.002s
I20260709 13:07:34.120214 23070 raft_consensus.cc:359] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:34.121039 23070 raft_consensus.cc:385] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:34.121628 23070 raft_consensus.cc:740] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:34.122596 23070 consensus_queue.cc:260] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:34.125345 23070 ts_tablet_manager.cc:1434] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:07:34.126387 23070 tablet_bootstrap.cc:492] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:34.136107 23073 tablet_bootstrap.cc:492] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:34.138212 23073 ts_tablet_manager.cc:1403] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.245s	user 0.012s	sys 0.007s
I20260709 13:07:34.139997 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:34.131007 22869 tablet_service.cc:1511] Processing CreateTablet for tablet b41a09964104486c8e50b539ec2b16f6 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:07:34.144419 23073 raft_consensus.cc:359] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:34.145292 23073 raft_consensus.cc:385] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:34.145889 23073 raft_consensus.cc:740] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:34.146688 23073 consensus_queue.cc:260] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:34.136118 22864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bdc53dc56134e28bcc43e68e76055fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.151513 22869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b41a09964104486c8e50b539ec2b16f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.158977 22861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdebaf291e5d4ae9814a80ce81ee19fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.159724 23075 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.160254 23075 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:34.132047 22870 tablet_service.cc:1511] Processing CreateTablet for tablet 7cd8492f63fb4193a47cf9fbc574ef58 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:07:34.153949 22938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 612dc49c81064f4faa79f65fcf1cfc02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.169869 23075 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:34.119285 22943 tablet_service.cc:1511] Processing CreateTablet for tablet fab4388fa7ea49bda9fa070cf542577f (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:07:34.194870 23010 tablet_service.cc:1511] Processing CreateTablet for tablet b7e4ee754b7a4408b5fc55259d431f39 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:07:34.197852 22865 tablet_service.cc:1511] Processing CreateTablet for tablet 698093275c264d4d99fb8c1a49e3b28f (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:07:34.199860 22865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 698093275c264d4d99fb8c1a49e3b28f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.128700 23019 tablet_service.cc:1511] Processing CreateTablet for tablet 579096ba88474cfbbe40e8bf769879ec (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:07:34.214257 22943 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fab4388fa7ea49bda9fa070cf542577f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.218631 23011 tablet_service.cc:1511] Processing CreateTablet for tablet 612dc49c81064f4faa79f65fcf1cfc02 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:07:34.219307 22944 tablet_service.cc:1511] Processing CreateTablet for tablet cdebaf291e5d4ae9814a80ce81ee19fe (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:07:34.226958 22870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7cd8492f63fb4193a47cf9fbc574ef58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.227404 23054 heartbeater.cc:499] Master 127.21.63.62:33399 was elected leader, sending a full tablet report...
I20260709 13:07:34.227768 23010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7e4ee754b7a4408b5fc55259d431f39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.205724 22942 tablet_service.cc:1511] Processing CreateTablet for tablet 6bdc53dc56134e28bcc43e68e76055fc (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
W20260709 13:07:34.235502 22843 leader_election.cc:343] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:34.236523 22936 tablet_service.cc:1511] Processing CreateTablet for tablet f603724630e44ef78014e7719f096cb3 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:07:34.240851 23069 tablet_bootstrap.cc:654] T 67e7fdd613434ffaacb8a030d4693490 P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:34.244285 22938 tablet_service.cc:1511] Processing CreateTablet for tablet b41a09964104486c8e50b539ec2b16f6 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:07:34.247750 22945 tablet_service.cc:1511] Processing CreateTablet for tablet 7cd8492f63fb4193a47cf9fbc574ef58 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
W20260709 13:07:34.250620 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 96790us
I20260709 13:07:34.261852 23013 tablet_service.cc:1511] Processing CreateTablet for tablet 7ca8b9a08f4146c386bc06d03c497038 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:07:34.264427 23070 tablet_bootstrap.cc:654] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:34.258173 22944 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdebaf291e5d4ae9814a80ce81ee19fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.307833 23011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 612dc49c81064f4faa79f65fcf1cfc02. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.321030 23073 ts_tablet_manager.cc:1434] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.182s	user 0.026s	sys 0.018s
I20260709 13:07:34.322204 23086 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.324069 22936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f603724630e44ef78014e7719f096cb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.328281 22864 tablet_service.cc:1511] Processing CreateTablet for tablet 3047c357228240a2aa4d84e8fbf437fc (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:07:34.329589 22945 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7cd8492f63fb4193a47cf9fbc574ef58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.327355 23086 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:34.334597 22944 tablet_service.cc:1511] Processing CreateTablet for tablet 698093275c264d4d99fb8c1a49e3b28f (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:07:34.335525 23086 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:34.335937 22944 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 698093275c264d4d99fb8c1a49e3b28f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.343575 22940 tablet_service.cc:1511] Processing CreateTablet for tablet 3047c357228240a2aa4d84e8fbf437fc (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:07:34.349463 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:34.355732 23019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 579096ba88474cfbbe40e8bf769879ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.355544 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:34.357349 23013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ca8b9a08f4146c386bc06d03c497038. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.358307 23016 tablet_service.cc:1511] Processing CreateTablet for tablet ec7dd25342084943aacb4b9830f5f1ea (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:07:34.359705 23016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec7dd25342084943aacb4b9830f5f1ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.362941 22864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3047c357228240a2aa4d84e8fbf437fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.362342 23017 tablet_service.cc:1511] Processing CreateTablet for tablet 2959101199a44a4583a4ee456ecf044d (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:07:34.363910 23017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2959101199a44a4583a4ee456ecf044d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.364593 23015 tablet_service.cc:1511] Processing CreateTablet for tablet 1120807254f5444997e16f6a7feece4d (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:07:34.366315 23015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1120807254f5444997e16f6a7feece4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.371336 23018 tablet_service.cc:1511] Processing CreateTablet for tablet c26347a647b240d7a3bc5cd449b7993b (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:07:34.372593 23018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c26347a647b240d7a3bc5cd449b7993b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:07:34.385299 22920 leader_election.cc:343] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:34.384999 22861 tablet_service.cc:1511] Processing CreateTablet for tablet e3b39c18a21f448783cdd998acfb2f80 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:07:34.389742 22861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3b39c18a21f448783cdd998acfb2f80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.392692 22865 tablet_service.cc:1511] Processing CreateTablet for tablet 4e689c91ad7343b5b6ba36fa648a06ac (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:07:34.394083 22865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e689c91ad7343b5b6ba36fa648a06ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.396785 22939 tablet_service.cc:1511] Processing CreateTablet for tablet 4e689c91ad7343b5b6ba36fa648a06ac (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:07:34.402277 23069 tablet_bootstrap.cc:492] T 67e7fdd613434ffaacb8a030d4693490 P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:34.402845 23069 ts_tablet_manager.cc:1403] T 67e7fdd613434ffaacb8a030d4693490 P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.346s	user 0.051s	sys 0.011s
I20260709 13:07:34.405851 23069 raft_consensus.cc:359] T 67e7fdd613434ffaacb8a030d4693490 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:34.413700 23073 tablet_bootstrap.cc:492] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:34.415697 23014 tablet_service.cc:1511] Processing CreateTablet for tablet f603724630e44ef78014e7719f096cb3 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:07:34.413766 22866 tablet_service.cc:1511] Processing CreateTablet for tablet 5cbb1e724f1d4e3f97c0d29d6b2ce944 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:07:34.418203 23069 raft_consensus.cc:385] T 67e7fdd613434ffaacb8a030d4693490 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:07:34.421770 22843 leader_election.cc:343] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:34.423023 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:34.416210 23011 tablet_service.cc:1511] Processing CreateTablet for tablet fab4388fa7ea49bda9fa070cf542577f (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:07:34.438207 23014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f603724630e44ef78014e7719f096cb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.439500 22941 tablet_service.cc:1511] Processing CreateTablet for tablet 5cbb1e724f1d4e3f97c0d29d6b2ce944 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:07:34.431038 23012 tablet_service.cc:1511] Processing CreateTablet for tablet a424231ef7b4470bbd575564909a365e (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:07:34.436981 23029 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 0.
I20260709 13:07:34.438625 22843 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:34.356503 22940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3047c357228240a2aa4d84e8fbf437fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.446360 22866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5cbb1e724f1d4e3f97c0d29d6b2ce944. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.446743 23011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fab4388fa7ea49bda9fa070cf542577f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.449069 23074 raft_consensus.cc:2749] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:34.450136 22918 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:07:34.450977 23075 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:34.451361 23075 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:34.451756 23075 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:34.452567 22941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5cbb1e724f1d4e3f97c0d29d6b2ce944. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.457401 23012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a424231ef7b4470bbd575564909a365e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.458290 22938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b41a09964104486c8e50b539ec2b16f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.435107 23069 raft_consensus.cc:740] T 67e7fdd613434ffaacb8a030d4693490 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:34.485702 23073 tablet_bootstrap.cc:654] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:34.494309 23070 tablet_bootstrap.cc:492] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:34.494912 23070 ts_tablet_manager.cc:1403] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.369s	user 0.013s	sys 0.036s
I20260709 13:07:34.464429 23075 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:34.502853 23019 tablet_service.cc:1511] Processing CreateTablet for tablet 6bdc53dc56134e28bcc43e68e76055fc (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:07:34.504145 23019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bdc53dc56134e28bcc43e68e76055fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.506997 23013 tablet_service.cc:1511] Processing CreateTablet for tablet cdebaf291e5d4ae9814a80ce81ee19fe (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:07:34.508417 23013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdebaf291e5d4ae9814a80ce81ee19fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.486889 22939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e689c91ad7343b5b6ba36fa648a06ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.512094 22869 tablet_service.cc:1511] Processing CreateTablet for tablet b65d9dbf257d434d86d4656dabfd1b70 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:07:34.513476 22869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b65d9dbf257d434d86d4656dabfd1b70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.513185 23015 tablet_service.cc:1511] Processing CreateTablet for tablet b41a09964104486c8e50b539ec2b16f6 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:07:34.514737 23015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b41a09964104486c8e50b539ec2b16f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.520771 22943 tablet_service.cc:1511] Processing CreateTablet for tablet e3b39c18a21f448783cdd998acfb2f80 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:07:34.522279 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:07:34.524109 22920 leader_election.cc:343] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:34.532996 22943 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3b39c18a21f448783cdd998acfb2f80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.534554 23070 raft_consensus.cc:359] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:34.535514 23070 raft_consensus.cc:385] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:34.535874 23070 raft_consensus.cc:740] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:34.540285 22942 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bdc53dc56134e28bcc43e68e76055fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.555588 23017 tablet_service.cc:1511] Processing CreateTablet for tablet 7cd8492f63fb4193a47cf9fbc574ef58 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:07:34.565670 23075 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:34.566332 23069 consensus_queue.cc:260] T 67e7fdd613434ffaacb8a030d4693490 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:34.567245 23074 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.567701 23074 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:07:34.568456 22762 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 57938.7us
I20260709 13:07:34.573711 22945 tablet_service.cc:1511] Processing CreateTablet for tablet b65d9dbf257d434d86d4656dabfd1b70 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:07:34.575529 23070 consensus_queue.cc:260] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:34.575935 23074 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:34.577750 23070 ts_tablet_manager.cc:1434] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.082s	user 0.007s	sys 0.000s
I20260709 13:07:34.581027 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:07:34.591058 22994 leader_election.cc:343] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:34.588923 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:07:34.593209 22843 leader_election.cc:343] T 67e7fdd613434ffaacb8a030d4693490 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:34.603482 23075 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.607919 23069 ts_tablet_manager.cc:1434] T 67e7fdd613434ffaacb8a030d4693490 P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.205s	user 0.017s	sys 0.001s
I20260709 13:07:34.607537 23075 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:34.607721 22865 tablet_service.cc:1511] Processing CreateTablet for tablet 80ab3f0c16e747e3b1917985c734eed4 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:07:34.602496 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:07:34.608912 23069 tablet_bootstrap.cc:492] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:34.609050 22865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80ab3f0c16e747e3b1917985c734eed4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.609133 23029 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:34.610428 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:07:34.611802 22920 leader_election.cc:343] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:34.614375 23075 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:34.620747 23017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7cd8492f63fb4193a47cf9fbc574ef58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.623675 23011 tablet_service.cc:1511] Processing CreateTablet for tablet 698093275c264d4d99fb8c1a49e3b28f (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:07:34.623932 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:34.623817 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:07:34.625782 22918 leader_election.cc:343] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:07:34.626477 22843 leader_election.cc:343] T 67e7fdd613434ffaacb8a030d4693490 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:34.626919 22843 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:34.626220 22918 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:34.629088 23093 raft_consensus.cc:2749] T 67e7fdd613434ffaacb8a030d4693490 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:34.626196 23016 tablet_service.cc:1511] Processing CreateTablet for tablet 3047c357228240a2aa4d84e8fbf437fc (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:07:34.631181 23015 tablet_service.cc:1511] Processing CreateTablet for tablet 5cbb1e724f1d4e3f97c0d29d6b2ce944 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:07:34.631585 23012 tablet_service.cc:1511] Processing CreateTablet for tablet e3b39c18a21f448783cdd998acfb2f80 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:07:34.632913 23070 tablet_bootstrap.cc:492] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:34.637661 22941 tablet_service.cc:1511] Processing CreateTablet for tablet 80ab3f0c16e747e3b1917985c734eed4 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:07:34.644579 23029 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 1.
I20260709 13:07:34.645658 22918 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:07:34.623263 22945 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b65d9dbf257d434d86d4656dabfd1b70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.645557 23010 tablet_service.cc:1511] Processing CreateTablet for tablet 4e689c91ad7343b5b6ba36fa648a06ac (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:07:34.656980 23016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3047c357228240a2aa4d84e8fbf437fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.659981 23075 raft_consensus.cc:2749] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:34.659471 23011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 698093275c264d4d99fb8c1a49e3b28f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.662675 23018 tablet_service.cc:1511] Processing CreateTablet for tablet b65d9dbf257d434d86d4656dabfd1b70 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:07:34.664461 23012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3b39c18a21f448783cdd998acfb2f80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.669072 23010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e689c91ad7343b5b6ba36fa648a06ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.672979 23018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b65d9dbf257d434d86d4656dabfd1b70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.675661 22864 tablet_service.cc:1511] Processing CreateTablet for tablet abd5e7d435ca4afd9727e37b2ab31589 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:07:34.669893 22941 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80ab3f0c16e747e3b1917985c734eed4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.662725 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:34.676968 23015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5cbb1e724f1d4e3f97c0d29d6b2ce944. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.676753 22944 tablet_service.cc:1511] Processing CreateTablet for tablet abd5e7d435ca4afd9727e37b2ab31589 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:07:34.682118 22870 tablet_service.cc:1511] Processing CreateTablet for tablet c3d2a83be5024e50818057526cbaf822 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:07:34.684713 22940 tablet_service.cc:1511] Processing CreateTablet for tablet c3d2a83be5024e50818057526cbaf822 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:07:34.690145 22936 tablet_service.cc:1511] Processing CreateTablet for tablet 4c0bea78fb2a40ef90cff914c5c6e250 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:07:34.692085 22936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c0bea78fb2a40ef90cff914c5c6e250. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.694911 22940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3d2a83be5024e50818057526cbaf822. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.699604 22944 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abd5e7d435ca4afd9727e37b2ab31589. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.700498 22867 tablet_service.cc:1511] Processing CreateTablet for tablet 6eba8b8b84ca4a9aa5d9eaca94670686 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:07:34.701921 22867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6eba8b8b84ca4a9aa5d9eaca94670686. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.663280 23075 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:34.705219 23075 raft_consensus.cc:697] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:07:34.706133 22861 tablet_service.cc:1511] Processing CreateTablet for tablet 4c0bea78fb2a40ef90cff914c5c6e250 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:07:34.706517 23075 consensus_queue.cc:237] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:34.707934 22861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c0bea78fb2a40ef90cff914c5c6e250. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.721868 23073 tablet_bootstrap.cc:492] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:34.722426 23073 ts_tablet_manager.cc:1403] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.309s	user 0.013s	sys 0.013s
I20260709 13:07:34.729857 22864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abd5e7d435ca4afd9727e37b2ab31589. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.733258 22870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3d2a83be5024e50818057526cbaf822. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.739066 23070 tablet_bootstrap.cc:654] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:34.752990 23014 tablet_service.cc:1511] Processing CreateTablet for tablet 80ab3f0c16e747e3b1917985c734eed4 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:07:34.754346 23014 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80ab3f0c16e747e3b1917985c734eed4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.756714 23073 raft_consensus.cc:359] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:34.757454 23073 raft_consensus.cc:385] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:34.757759 23073 raft_consensus.cc:740] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:34.758631 23073 consensus_queue.cc:260] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:34.759989 23069 tablet_bootstrap.cc:654] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:34.761979 22939 tablet_service.cc:1511] Processing CreateTablet for tablet 6eba8b8b84ca4a9aa5d9eaca94670686 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:07:34.764081 22939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6eba8b8b84ca4a9aa5d9eaca94670686. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.778359 23013 tablet_service.cc:1511] Processing CreateTablet for tablet abd5e7d435ca4afd9727e37b2ab31589 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:07:34.780746 22866 tablet_service.cc:1511] Processing CreateTablet for tablet 3f1c9e3f9b244b0695775e91f28d67d7 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:07:34.782040 22866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f1c9e3f9b244b0695775e91f28d67d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.782922 22942 tablet_service.cc:1511] Processing CreateTablet for tablet 3f1c9e3f9b244b0695775e91f28d67d7 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:07:34.774322 23073 ts_tablet_manager.cc:1434] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.052s	user 0.006s	sys 0.001s
I20260709 13:07:34.794368 22942 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f1c9e3f9b244b0695775e91f28d67d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.796979 22992 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:07:34.798729 23069 tablet_bootstrap.cc:492] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:34.802917 23069 ts_tablet_manager.cc:1403] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.194s	user 0.011s	sys 0.004s
I20260709 13:07:34.806407 23069 raft_consensus.cc:359] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:34.806015 22776 catalog_manager.cc:5697] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 1 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:34.807166 23069 raft_consensus.cc:385] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:34.807528 23069 raft_consensus.cc:740] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:34.802947 23088 raft_consensus.cc:2749] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:34.808426 23069 consensus_queue.cc:260] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:34.811215 23017 tablet_service.cc:1511] Processing CreateTablet for tablet 4c0bea78fb2a40ef90cff914c5c6e250 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:07:34.812644 23019 tablet_service.cc:1511] Processing CreateTablet for tablet 6eba8b8b84ca4a9aa5d9eaca94670686 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:07:34.812602 23017 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c0bea78fb2a40ef90cff914c5c6e250. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.814126 23019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6eba8b8b84ca4a9aa5d9eaca94670686. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.815521 23013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abd5e7d435ca4afd9727e37b2ab31589. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.819810 23073 tablet_bootstrap.cc:492] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:34.824350 23011 tablet_service.cc:1511] Processing CreateTablet for tablet 3f1c9e3f9b244b0695775e91f28d67d7 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:07:34.816989 23016 tablet_service.cc:1511] Processing CreateTablet for tablet ced80ef275e04ef8bf4c071883b8284e (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:07:34.825738 23011 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f1c9e3f9b244b0695775e91f28d67d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.826534 23016 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ced80ef275e04ef8bf4c071883b8284e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.799057 23018 tablet_service.cc:1511] Processing CreateTablet for tablet c3d2a83be5024e50818057526cbaf822 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:07:34.830263 22870 tablet_service.cc:1511] Processing CreateTablet for tablet f11454b577ab42d7bfe7c09154b00817 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:07:34.834710 22870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f11454b577ab42d7bfe7c09154b00817. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.835263 23070 tablet_bootstrap.cc:492] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:34.835836 23070 ts_tablet_manager.cc:1403] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.213s	user 0.014s	sys 0.003s
I20260709 13:07:34.818843 23069 ts_tablet_manager.cc:1434] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.015s	user 0.002s	sys 0.004s
W20260709 13:07:34.832563 22826 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:34.842434 22865 tablet_service.cc:1511] Processing CreateTablet for tablet 3ec25afe61174e01a75a21ab33afe2e1 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:07:34.844029 23069 tablet_bootstrap.cc:492] T 0c4ecd16880043f4a75006a3da4717bf P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:34.844590 22865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ec25afe61174e01a75a21ab33afe2e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.847409 23070 raft_consensus.cc:359] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:34.848186 23070 raft_consensus.cc:385] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:34.832635 22864 tablet_service.cc:1511] Processing CreateTablet for tablet ced80ef275e04ef8bf4c071883b8284e (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:07:34.848496 23070 raft_consensus.cc:740] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:34.855610 23018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3d2a83be5024e50818057526cbaf822. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.865173 22870 tablet_service.cc:1511] Processing CreateTablet for tablet 421f71937d5e4cef9084068ce92d12b7 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:07:34.868637 22869 tablet_service.cc:1511] Processing CreateTablet for tablet 778fe3ade80844c48f8408313fc86a26 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:07:34.869939 22936 tablet_service.cc:1511] Processing CreateTablet for tablet 3ec25afe61174e01a75a21ab33afe2e1 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:07:34.869890 23010 tablet_service.cc:1511] Processing CreateTablet for tablet 3ec25afe61174e01a75a21ab33afe2e1 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:07:34.871367 22936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ec25afe61174e01a75a21ab33afe2e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.871783 23010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3ec25afe61174e01a75a21ab33afe2e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.869982 23088 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.873229 23088 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:34.875285 23088 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:34.875116 23012 tablet_service.cc:1511] Processing CreateTablet for tablet f11454b577ab42d7bfe7c09154b00817 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:07:34.876375 23012 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f11454b577ab42d7bfe7c09154b00817. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.876446 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:34.878034 22940 tablet_service.cc:1511] Processing CreateTablet for tablet 421f71937d5e4cef9084068ce92d12b7 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:07:34.878937 22943 tablet_service.cc:1511] Processing CreateTablet for tablet 778fe3ade80844c48f8408313fc86a26 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:07:34.879698 23073 tablet_bootstrap.cc:654] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:34.879776 22940 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 421f71937d5e4cef9084068ce92d12b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.887189 23069 tablet_bootstrap.cc:654] T 0c4ecd16880043f4a75006a3da4717bf P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:34.887526 22943 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 778fe3ade80844c48f8408313fc86a26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.889079 23070 consensus_queue.cc:260] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:34.891721 23070 ts_tablet_manager.cc:1434] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.056s	user 0.005s	sys 0.000s
I20260709 13:07:34.895697 23093 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.896275 23093 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:07:34.897259 22992 leader_election.cc:343] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:34.897357 22944 tablet_service.cc:1511] Processing CreateTablet for tablet ced80ef275e04ef8bf4c071883b8284e (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:07:34.887212 23075 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.898588 23093 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:34.898432 23075 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:34.899978 23015 tablet_service.cc:1511] Processing CreateTablet for tablet 421f71937d5e4cef9084068ce92d12b7 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:07:34.901031 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:34.901469 23015 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 421f71937d5e4cef9084068ce92d12b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.902082 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:07:34.903668 22918 leader_election.cc:343] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:34.903767 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:34.904029 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:34.904323 22864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ced80ef275e04ef8bf4c071883b8284e. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:07:34.904467 22843 leader_election.cc:343] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:34.904780 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:07:34.906525 22920 leader_election.cc:343] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:34.906976 22920 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:34.908016 23094 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:07:34.909102 22843 leader_election.cc:343] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:34.909605 22843 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:07:34.910940 22994 leader_election.cc:343] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:34.911420 22994 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:07:34.912024 23075 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:34.912628 23088 raft_consensus.cc:2749] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:34.913094 22944 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ced80ef275e04ef8bf4c071883b8284e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.913455 23093 raft_consensus.cc:2749] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:34.922310 22869 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 778fe3ade80844c48f8408313fc86a26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.923475 23075 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:34.923307 22945 tablet_service.cc:1511] Processing CreateTablet for tablet f11454b577ab42d7bfe7c09154b00817 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:07:34.923988 23075 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:34.924691 22945 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f11454b577ab42d7bfe7c09154b00817. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.926954 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:07:34.928467 22920 leader_election.cc:343] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:34.931182 22870 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 421f71937d5e4cef9084068ce92d12b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.937264 22938 tablet_service.cc:1511] Processing CreateTablet for tablet 61adbc0c57034ac28a0b50af8a59ed07 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:07:34.938072 22866 tablet_service.cc:1511] Processing CreateTablet for tablet 0fa29d11409a4d209411116a7ef8af04 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:07:34.938562 22938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61adbc0c57034ac28a0b50af8a59ed07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.939235 22866 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fa29d11409a4d209411116a7ef8af04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.939687 22937 tablet_service.cc:1511] Processing CreateTablet for tablet 0fa29d11409a4d209411116a7ef8af04 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:07:34.941115 22937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fa29d11409a4d209411116a7ef8af04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.941294 23013 tablet_service.cc:1511] Processing CreateTablet for tablet 61adbc0c57034ac28a0b50af8a59ed07 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:07:34.942472 23013 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61adbc0c57034ac28a0b50af8a59ed07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.945897 23070 tablet_bootstrap.cc:492] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:34.950435 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:07:34.953326 22918 leader_election.cc:343] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:34.953812 22918 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:34.954694 23094 raft_consensus.cc:2749] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:34.947790 23075 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:34.978756 23019 tablet_service.cc:1511] Processing CreateTablet for tablet 0fa29d11409a4d209411116a7ef8af04 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:07:34.980165 23019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fa29d11409a4d209411116a7ef8af04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:34.999047 23073 tablet_bootstrap.cc:492] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:34.999606 23073 ts_tablet_manager.cc:1403] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.180s	user 0.013s	sys 0.007s
I20260709 13:07:35.002844 23073 raft_consensus.cc:359] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.003576 23073 raft_consensus.cc:385] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:35.003963 23073 raft_consensus.cc:740] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:35.005055 23073 consensus_queue.cc:260] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.007753 23073 ts_tablet_manager.cc:1434] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.007s
I20260709 13:07:35.008956 23073 tablet_bootstrap.cc:492] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:35.009675 23010 tablet_service.cc:1511] Processing CreateTablet for tablet 778fe3ade80844c48f8408313fc86a26 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:07:35.011184 23010 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 778fe3ade80844c48f8408313fc86a26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:35.022513 23073 tablet_bootstrap.cc:654] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:35.034386 23070 tablet_bootstrap.cc:654] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:35.054747 23069 tablet_bootstrap.cc:492] T 0c4ecd16880043f4a75006a3da4717bf P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:35.055267 23069 ts_tablet_manager.cc:1403] T 0c4ecd16880043f4a75006a3da4717bf P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.211s	user 0.011s	sys 0.044s
I20260709 13:07:35.057057 22867 tablet_service.cc:1511] Processing CreateTablet for tablet 61adbc0c57034ac28a0b50af8a59ed07 (DEFAULT_TABLE table=test-workload [id=a42f2ccfbe484baca3019323a490c64a]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:07:35.058415 22867 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 61adbc0c57034ac28a0b50af8a59ed07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:35.058101 23069 raft_consensus.cc:359] T 0c4ecd16880043f4a75006a3da4717bf P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.065382 23069 raft_consensus.cc:385] T 0c4ecd16880043f4a75006a3da4717bf P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:35.065956 23069 raft_consensus.cc:740] T 0c4ecd16880043f4a75006a3da4717bf P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:35.067462 23073 tablet_bootstrap.cc:492] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:35.067022 23069 consensus_queue.cc:260] T 0c4ecd16880043f4a75006a3da4717bf P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.067971 23073 ts_tablet_manager.cc:1403] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.059s	user 0.015s	sys 0.010s
I20260709 13:07:35.069767 23069 ts_tablet_manager.cc:1434] T 0c4ecd16880043f4a75006a3da4717bf P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 13:07:35.071110 23069 tablet_bootstrap.cc:492] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:35.070775 23073 raft_consensus.cc:359] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.071491 23073 raft_consensus.cc:385] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:35.071815 23073 raft_consensus.cc:740] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:35.083361 23070 tablet_bootstrap.cc:492] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:35.083925 23070 ts_tablet_manager.cc:1403] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.191s	user 0.009s	sys 0.006s
I20260709 13:07:35.089110 23069 tablet_bootstrap.cc:654] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:35.089973 23070 raft_consensus.cc:359] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.091068 23070 raft_consensus.cc:385] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:35.091506 23070 raft_consensus.cc:740] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:35.072525 23073 consensus_queue.cc:260] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.094954 23073 ts_tablet_manager.cc:1434] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.027s	user 0.006s	sys 0.000s
I20260709 13:07:35.096362 23073 tablet_bootstrap.cc:492] T cbf52b91ec054ac6a2f8503462b1b7a6 P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:35.101027 23070 consensus_queue.cc:260] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.105780 23070 ts_tablet_manager.cc:1434] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.001s
I20260709 13:07:35.107168 23070 tablet_bootstrap.cc:492] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:35.115769 23073 tablet_bootstrap.cc:654] T cbf52b91ec054ac6a2f8503462b1b7a6 P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:35.119583 23069 tablet_bootstrap.cc:492] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:35.120154 23069 ts_tablet_manager.cc:1403] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.049s	user 0.016s	sys 0.020s
I20260709 13:07:35.121629 23070 tablet_bootstrap.cc:654] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:35.123344 23069 raft_consensus.cc:359] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.124181 23069 raft_consensus.cc:385] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:35.124528 23069 raft_consensus.cc:740] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:35.125344 23069 consensus_queue.cc:260] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.127701 23069 ts_tablet_manager.cc:1434] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.004s
I20260709 13:07:35.128788 23069 tablet_bootstrap.cc:492] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:35.131882 23070 tablet_bootstrap.cc:492] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:35.132416 23070 ts_tablet_manager.cc:1403] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.026s	user 0.008s	sys 0.004s
I20260709 13:07:35.135733 23069 tablet_bootstrap.cc:654] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:35.137032 23073 tablet_bootstrap.cc:492] T cbf52b91ec054ac6a2f8503462b1b7a6 P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:35.137626 23073 ts_tablet_manager.cc:1403] T cbf52b91ec054ac6a2f8503462b1b7a6 P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.042s	user 0.013s	sys 0.004s
I20260709 13:07:35.138005 23075 consensus_queue.cc:1048] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:35.141827 23073 raft_consensus.cc:359] T cbf52b91ec054ac6a2f8503462b1b7a6 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.147967 23073 raft_consensus.cc:385] T cbf52b91ec054ac6a2f8503462b1b7a6 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:35.148526 23073 raft_consensus.cc:740] T cbf52b91ec054ac6a2f8503462b1b7a6 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:35.149717 23073 consensus_queue.cc:260] T cbf52b91ec054ac6a2f8503462b1b7a6 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.153213 23073 ts_tablet_manager.cc:1434] T cbf52b91ec054ac6a2f8503462b1b7a6 P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 13:07:35.146818 23070 raft_consensus.cc:359] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.153893 23070 raft_consensus.cc:385] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:35.154393 23070 raft_consensus.cc:740] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:35.155507 23070 consensus_queue.cc:260] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.158898 23070 ts_tablet_manager.cc:1434] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:07:35.161734 23070 tablet_bootstrap.cc:492] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:35.160392 23073 tablet_bootstrap.cc:492] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:35.166980 23069 tablet_bootstrap.cc:492] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:35.167498 23069 ts_tablet_manager.cc:1403] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.039s	user 0.016s	sys 0.020s
I20260709 13:07:35.169922 23070 tablet_bootstrap.cc:654] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:35.171077 23094 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.171047 23069 raft_consensus.cc:359] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.172111 23069 raft_consensus.cc:385] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:35.171782 23094 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.172271 23073 tablet_bootstrap.cc:654] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:35.172472 23069 raft_consensus.cc:740] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:35.173601 23069 consensus_queue.cc:260] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.174198 23094 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:35.175926 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:35.176272 23069 ts_tablet_manager.cc:1434] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.005s
I20260709 13:07:35.176630 22880 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 0.
I20260709 13:07:35.178922 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:35.179859 22920 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:07:35.181186 23094 raft_consensus.cc:2804] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:07:35.181249 22918 leader_election.cc:343] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:35.181790 23094 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:35.184528 23094 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:35.187958 23070 tablet_bootstrap.cc:492] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:35.188728 23070 ts_tablet_manager.cc:1403] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.027s	user 0.014s	sys 0.007s
I20260709 13:07:35.194123 23070 raft_consensus.cc:359] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.194851 23069 tablet_bootstrap.cc:492] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:35.194948 23070 raft_consensus.cc:385] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:35.195447 23070 raft_consensus.cc:740] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:35.195648 23073 tablet_bootstrap.cc:492] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:35.196138 23073 ts_tablet_manager.cc:1403] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.036s	user 0.008s	sys 0.021s
I20260709 13:07:35.196385 23070 consensus_queue.cc:260] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.200795 23115 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.201465 23115 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.203639 23069 tablet_bootstrap.cc:654] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:35.205665 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:35.207093 22880 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 0.
I20260709 13:07:35.199873 23073 raft_consensus.cc:359] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.208912 22920 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:07:35.211084 23115 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:35.209944 23073 raft_consensus.cc:385] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:35.216537 23073 raft_consensus.cc:740] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:35.217114 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:35.217772 23028 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 0.
I20260709 13:07:35.217684 23073 consensus_queue.cc:260] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.220546 23115 raft_consensus.cc:2804] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:35.221247 23115 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:35.208927 23094 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.224959 23115 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:35.227535 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:07:35.227830 23073 ts_tablet_manager.cc:1434] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.003s
I20260709 13:07:35.228228 22880 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:35.229678 23093 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.230980 23093 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.231926 22879 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:35.232026 23094 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:35.233191 23093 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:35.235913 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:07:35.238668 22918 leader_election.cc:343] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:35.236627 23115 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.239746 22880 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 1.
I20260709 13:07:35.240370 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:35.229485 23073 tablet_bootstrap.cc:492] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:35.242190 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:07:35.241878 22920 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:35.242799 23028 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:35.244184 23117 raft_consensus.cc:2804] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:35.244793 23117 raft_consensus.cc:697] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:07:35.247164 23070 ts_tablet_manager.cc:1434] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.058s	user 0.027s	sys 0.008s
I20260709 13:07:35.247355 23075 consensus_queue.cc:1048] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:35.248376 23070 tablet_bootstrap.cc:492] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:35.252014 23028 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 1.
I20260709 13:07:35.253140 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:07:35.253736 22880 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:35.253777 22918 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:35.254773 23094 raft_consensus.cc:2804] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:35.255101 23073 tablet_bootstrap.cc:654] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:35.255259 23094 raft_consensus.cc:697] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:07:35.256217 23094 consensus_queue.cc:237] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.246644 23117 consensus_queue.cc:237] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.252372 23115 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:35.259594 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:35.264631 22843 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:35.267192 23070 tablet_bootstrap.cc:654] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:35.278852 22880 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 1.
I20260709 13:07:35.281893 23069 tablet_bootstrap.cc:492] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:35.282608 23069 ts_tablet_manager.cc:1403] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.088s	user 0.016s	sys 0.000s
I20260709 13:07:35.291395 22778 catalog_manager.cc:5697] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 1 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:35.297781 23069 raft_consensus.cc:359] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.325760 23069 raft_consensus.cc:385] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:35.326318 23069 raft_consensus.cc:740] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:35.323807 22775 catalog_manager.cc:5697] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 1 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:35.342090 23069 consensus_queue.cc:260] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.342828 23073 tablet_bootstrap.cc:492] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:35.345050 23073 ts_tablet_manager.cc:1403] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.116s	user 0.025s	sys 0.047s
I20260709 13:07:35.348413 23070 tablet_bootstrap.cc:492] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:35.349099 23070 ts_tablet_manager.cc:1403] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.101s	user 0.038s	sys 0.040s
I20260709 13:07:35.350170 23073 raft_consensus.cc:359] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.351264 23073 raft_consensus.cc:385] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:35.351675 23073 raft_consensus.cc:740] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:35.352938 23070 raft_consensus.cc:359] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.354148 23070 raft_consensus.cc:385] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:35.352972 23073 consensus_queue.cc:260] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.354645 23070 raft_consensus.cc:740] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:35.355697 23069 ts_tablet_manager.cc:1434] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.073s	user 0.015s	sys 0.001s
I20260709 13:07:35.355741 23070 consensus_queue.cc:260] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.356333 23093 raft_consensus.cc:2749] T 0c4ecd16880043f4a75006a3da4717bf P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:35.358493 23070 ts_tablet_manager.cc:1434] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 13:07:35.359472 23070 tablet_bootstrap.cc:492] T dd740f20d2d74669b0fd10f1ec5b44f0 P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:35.359555 23069 tablet_bootstrap.cc:492] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:35.364765 23073 ts_tablet_manager.cc:1434] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 13:07:35.365931 23073 tablet_bootstrap.cc:492] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:35.367164 23070 tablet_bootstrap.cc:654] T dd740f20d2d74669b0fd10f1ec5b44f0 P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:35.373728 23069 tablet_bootstrap.cc:654] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:35.380981 23088 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.381569 23088 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.383318 23070 tablet_bootstrap.cc:492] T dd740f20d2d74669b0fd10f1ec5b44f0 P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:35.383749 23070 ts_tablet_manager.cc:1403] T dd740f20d2d74669b0fd10f1ec5b44f0 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.024s	user 0.014s	sys 0.007s
I20260709 13:07:35.384882 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:35.385223 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:35.385638 23073 tablet_bootstrap.cc:654] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:35.385984 22955 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 0.
W20260709 13:07:35.386338 22994 leader_election.cc:343] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:35.386482 23070 raft_consensus.cc:359] T dd740f20d2d74669b0fd10f1ec5b44f0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.387228 23070 raft_consensus.cc:385] T dd740f20d2d74669b0fd10f1ec5b44f0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:35.387508 23070 raft_consensus.cc:740] T dd740f20d2d74669b0fd10f1ec5b44f0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:35.387439 22992 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:07:35.388309 23070 consensus_queue.cc:260] T dd740f20d2d74669b0fd10f1ec5b44f0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.388823 23088 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:35.389341 23088 raft_consensus.cc:2804] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:35.389686 23088 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:35.390033 23088 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:35.390427 23070 ts_tablet_manager.cc:1434] T dd740f20d2d74669b0fd10f1ec5b44f0 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:07:35.391372 23070 tablet_bootstrap.cc:492] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:35.394869 23069 tablet_bootstrap.cc:492] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:35.401060 23069 ts_tablet_manager.cc:1403] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.042s	user 0.006s	sys 0.007s
I20260709 13:07:35.397248 23070 tablet_bootstrap.cc:654] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:35.403817 23088 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.403808 23069 raft_consensus.cc:359] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.407457 23069 raft_consensus.cc:385] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:35.407327 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:07:35.407769 23069 raft_consensus.cc:740] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:35.408468 23069 consensus_queue.cc:260] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:07:35.408939 22994 leader_election.cc:343] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:35.409996 23088 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:35.413170 23069 ts_tablet_manager.cc:1434] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260709 13:07:35.413946 23069 tablet_bootstrap.cc:492] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:35.418489 23069 tablet_bootstrap.cc:654] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:35.418941 23070 tablet_bootstrap.cc:492] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:35.419610 23070 ts_tablet_manager.cc:1403] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.028s	user 0.011s	sys 0.000s
I20260709 13:07:35.422971 23070 raft_consensus.cc:359] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.424037 23070 raft_consensus.cc:385] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:35.424487 23070 raft_consensus.cc:740] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:35.425513 23070 consensus_queue.cc:260] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.428718 23070 ts_tablet_manager.cc:1434] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:07:35.430439 23070 tablet_bootstrap.cc:492] T 7f35ebcf93ad423ea8bd0b777bee23ee P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:35.431373 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:35.431784 22955 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:35.432263 23093 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.432754 23093 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.434969 23093 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:35.436595 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:35.437492 23070 tablet_bootstrap.cc:654] T 7f35ebcf93ad423ea8bd0b777bee23ee P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:35.437829 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:07:35.438482 22843 leader_election.cc:343] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:35.437826 22955 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
W20260709 13:07:35.439153 22843 leader_election.cc:343] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:35.439683 22843 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:35.440418 23093 raft_consensus.cc:2749] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:35.441114 22992 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:07:35.441488 23069 tablet_bootstrap.cc:492] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:35.442148 23069 ts_tablet_manager.cc:1403] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.028s	user 0.019s	sys 0.004s
I20260709 13:07:35.444082 23073 tablet_bootstrap.cc:492] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:35.444525 23073 ts_tablet_manager.cc:1403] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.079s	user 0.027s	sys 0.022s
I20260709 13:07:35.446033 23069 raft_consensus.cc:359] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.447090 23069 raft_consensus.cc:385] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:35.447106 23073 raft_consensus.cc:359] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.447722 23069 raft_consensus.cc:740] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:35.448071 23073 raft_consensus.cc:385] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:35.448789 23117 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.449295 23117 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.449003 23069 consensus_queue.cc:260] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.451329 23117 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:35.451936 23129 raft_consensus.cc:2804] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:35.448484 23073 raft_consensus.cc:740] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:35.452967 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:35.453815 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:35.453831 23073 consensus_queue.cc:260] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:07:35.454576 22920 leader_election.cc:343] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:07:35.455204 22918 leader_election.cc:343] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:35.455588 22918 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:35.455862 23073 ts_tablet_manager.cc:1434] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.000s
I20260709 13:07:35.456382 23069 ts_tablet_manager.cc:1434] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.001s
I20260709 13:07:35.456710 23073 tablet_bootstrap.cc:492] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:35.456952 23117 raft_consensus.cc:2749] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:35.458259 23069 tablet_bootstrap.cc:492] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:35.458266 23088 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.458873 23088 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.459429 23070 tablet_bootstrap.cc:492] T 7f35ebcf93ad423ea8bd0b777bee23ee P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:35.459875 23070 ts_tablet_manager.cc:1403] T 7f35ebcf93ad423ea8bd0b777bee23ee P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.030s	user 0.017s	sys 0.006s
I20260709 13:07:35.460901 23088 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:35.462037 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:35.462458 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:35.462682 22955 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 0.
I20260709 13:07:35.463080 22879 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 0.
I20260709 13:07:35.462764 23070 raft_consensus.cc:359] T 7f35ebcf93ad423ea8bd0b777bee23ee P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.463593 23070 raft_consensus.cc:385] T 7f35ebcf93ad423ea8bd0b777bee23ee P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:35.463948 22992 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:35.464234 23070 raft_consensus.cc:740] T 7f35ebcf93ad423ea8bd0b777bee23ee P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:35.464552 23069 tablet_bootstrap.cc:654] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:35.464777 23088 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:35.465129 23088 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:35.465698 23088 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:35.465085 23070 consensus_queue.cc:260] T 7f35ebcf93ad423ea8bd0b777bee23ee P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.467695 23073 tablet_bootstrap.cc:654] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:35.467764 23070 ts_tablet_manager.cc:1434] T 7f35ebcf93ad423ea8bd0b777bee23ee P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:07:35.469561 23070 tablet_bootstrap.cc:492] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:35.477736 23070 tablet_bootstrap.cc:654] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:35.490578 23129 raft_consensus.cc:697] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:07:35.493683 23070 tablet_bootstrap.cc:492] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:35.494421 23070 ts_tablet_manager.cc:1403] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.025s	user 0.021s	sys 0.004s
I20260709 13:07:35.491699 23129 consensus_queue.cc:237] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.502907 23093 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.503405 23093 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.505425 23093 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:35.506062 23088 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.507278 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:35.509248 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:35.509765 22955 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:35.498919 23070 raft_consensus.cc:359] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.510566 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:35.510633 23070 raft_consensus.cc:385] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:35.511145 23070 raft_consensus.cc:740] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
W20260709 13:07:35.511672 22843 leader_election.cc:343] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:35.509645 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:07:35.512460 22879 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:35.512095 23070 consensus_queue.cc:260] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:07:35.512984 22843 leader_election.cc:343] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:35.513417 22843 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:35.514433 23093 raft_consensus.cc:2749] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:35.515784 23070 ts_tablet_manager.cc:1434] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 13:07:35.517074 23070 tablet_bootstrap.cc:492] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:35.525409 23070 tablet_bootstrap.cc:654] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:35.532135 22879 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:07:35.533805 22994 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:35.538455 23070 tablet_bootstrap.cc:492] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:35.539112 23070 ts_tablet_manager.cc:1403] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.004s
I20260709 13:07:35.542835 23070 raft_consensus.cc:359] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.543825 23070 raft_consensus.cc:385] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:35.544204 23070 raft_consensus.cc:740] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:35.545511 23070 consensus_queue.cc:260] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.546706 22955 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:07:35.548828 23088 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:35.548862 23070 ts_tablet_manager.cc:1434] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.009s	user 0.011s	sys 0.000s
I20260709 13:07:35.549698 23088 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:35.550307 23088 raft_consensus.cc:697] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:07:35.550557 23070 tablet_bootstrap.cc:492] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:35.551424 23088 consensus_queue.cc:237] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.559505 23070 tablet_bootstrap.cc:654] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:35.563241 23139 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.563880 23139 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.569506 23070 tablet_bootstrap.cc:492] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:35.573428 23093 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.578222 23093 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.579443 23073 tablet_bootstrap.cc:492] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:35.578958 22776 catalog_manager.cc:5697] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:35.580528 23093 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:35.579941 23073 ts_tablet_manager.cc:1403] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.123s	user 0.040s	sys 0.054s
I20260709 13:07:35.574276 23069 tablet_bootstrap.cc:492] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:35.581279 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:35.583350 22879 raft_consensus.cc:2468] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 0.
I20260709 13:07:35.581934 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:35.584151 23139 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:35.581750 23069 ts_tablet_manager.cc:1403] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.124s	user 0.031s	sys 0.082s
I20260709 13:07:35.584049 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:35.584285 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:35.596926 22954 raft_consensus.cc:2468] T dd740f20d2d74669b0fd10f1ec5b44f0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 0.
I20260709 13:07:35.585606 22994 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:07:35.597431 22843 leader_election.cc:343] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:35.581410 23070 ts_tablet_manager.cc:1403] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.031s	user 0.015s	sys 0.003s
W20260709 13:07:35.598353 22843 leader_election.cc:343] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:35.596302 23073 raft_consensus.cc:359] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.598763 22843 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:35.599182 23073 raft_consensus.cc:385] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:35.599004 23069 raft_consensus.cc:359] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.599485 23073 raft_consensus.cc:740] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:35.599640 23069 raft_consensus.cc:385] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:35.599781 23093 raft_consensus.cc:2749] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:35.599967 23069 raft_consensus.cc:740] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:35.600452 23073 consensus_queue.cc:260] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.600889 23070 raft_consensus.cc:359] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.601537 23070 raft_consensus.cc:385] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:35.600955 23069 consensus_queue.cc:260] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.601812 23070 raft_consensus.cc:740] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:35.602548 23070 consensus_queue.cc:260] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.603106 23073 ts_tablet_manager.cc:1434] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.022s	user 0.001s	sys 0.005s
I20260709 13:07:35.603633 23069 ts_tablet_manager.cc:1434] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:07:35.605165 23069 tablet_bootstrap.cc:492] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:35.608332 23070 ts_tablet_manager.cc:1434] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:07:35.609258 23070 tablet_bootstrap.cc:492] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:35.611564 23129 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.612076 23129 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.615257 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:35.600953 23139 raft_consensus.cc:2804] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:35.615875 22954 raft_consensus.cc:2468] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 0.
I20260709 13:07:35.616055 23139 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:35.616551 23139 raft_consensus.cc:3060] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:35.616861 22992 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:35.617121 23070 tablet_bootstrap.cc:654] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:35.617923 23140 raft_consensus.cc:2804] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:35.618382 23140 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:35.618744 23140 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:35.619349 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:35.620162 23069 tablet_bootstrap.cc:654] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
W20260709 13:07:35.620906 22994 leader_election.cc:343] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:35.624436 22775 catalog_manager.cc:5697] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:35.626895 23139 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.633586 23129 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:35.636818 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:07:35.637159 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:35.637511 23139 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:35.637553 22879 raft_consensus.cc:3060] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:35.637966 22954 raft_consensus.cc:3060] T dd740f20d2d74669b0fd10f1ec5b44f0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:35.603957 23073 tablet_bootstrap.cc:492] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:35.650498 22879 raft_consensus.cc:2468] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:07:35.651959 22994 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:35.653081 23139 raft_consensus.cc:2804] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:35.653549 23139 raft_consensus.cc:697] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:07:35.654361 23139 consensus_queue.cc:237] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.655069 23094 consensus_queue.cc:1048] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:35.678593 23073 tablet_bootstrap.cc:654] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:35.680269 23140 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.668170 22954 raft_consensus.cc:2468] T dd740f20d2d74669b0fd10f1ec5b44f0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:07:35.681950 23070 tablet_bootstrap.cc:492] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:35.682427 23070 ts_tablet_manager.cc:1403] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.073s	user 0.012s	sys 0.044s
I20260709 13:07:35.683529 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:07:35.684808 22994 leader_election.cc:343] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:35.684938 23070 raft_consensus.cc:359] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.685998 23070 raft_consensus.cc:385] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:35.686276 23070 raft_consensus.cc:740] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:35.686331 23140 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:35.686853 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:35.687599 22954 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:35.687894 23070 consensus_queue.cc:260] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.689919 23070 ts_tablet_manager.cc:1434] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.002s
I20260709 13:07:35.690770 23070 tablet_bootstrap.cc:492] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:35.694943 22954 raft_consensus.cc:2468] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:07:35.686874 23069 tablet_bootstrap.cc:492] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:35.695318 22778 catalog_manager.cc:5697] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:35.696103 23069 ts_tablet_manager.cc:1403] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.092s	user 0.016s	sys 0.030s
I20260709 13:07:35.696002 22992 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:07:35.697559 23070 tablet_bootstrap.cc:654] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:35.699401 23075 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.699883 23075 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.701363 23073 tablet_bootstrap.cc:492] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:35.701786 23073 ts_tablet_manager.cc:1403] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.098s	user 0.002s	sys 0.015s
I20260709 13:07:35.702004 23075 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:35.703474 23075 consensus_queue.cc:1048] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:35.703567 23069 raft_consensus.cc:359] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.704267 23069 raft_consensus.cc:385] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:35.704541 23069 raft_consensus.cc:740] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:35.705340 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:35.705273 23069 consensus_queue.cc:260] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.705955 23029 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 0.
I20260709 13:07:35.707212 23069 ts_tablet_manager.cc:1434] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.011s	user 0.002s	sys 0.003s
I20260709 13:07:35.707283 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:35.707345 23073 raft_consensus.cc:359] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.708361 23073 raft_consensus.cc:385] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:35.708753 23073 raft_consensus.cc:740] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:35.710145 23073 consensus_queue.cc:260] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.713419 23073 ts_tablet_manager.cc:1434] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.002s
I20260709 13:07:35.714637 23073 tablet_bootstrap.cc:492] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:35.715340 22918 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:35.716171 23075 raft_consensus.cc:2804] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:35.716540 23075 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:35.710860 23069 tablet_bootstrap.cc:492] T a0b6723ad2db4fccb7a1c219ff04e99a P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:35.716918 23075 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:35.719022 23070 tablet_bootstrap.cc:492] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:35.719466 23070 ts_tablet_manager.cc:1403] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.029s	user 0.007s	sys 0.014s
I20260709 13:07:35.720743 23093 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:35.720973 22920 leader_election.cc:343] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:35.721254 23093 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.723608 23093 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:35.723728 23069 tablet_bootstrap.cc:654] T a0b6723ad2db4fccb7a1c219ff04e99a P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:35.724152 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:35.724864 22954 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 0.
I20260709 13:07:35.729139 23073 tablet_bootstrap.cc:654] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:35.729036 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:35.731027 22843 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:07:35.731976 22843 leader_election.cc:343] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:35.732425 23125 raft_consensus.cc:2804] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:35.732807 23125 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:35.733181 23125 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:35.732839 23070 raft_consensus.cc:359] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.733896 23070 raft_consensus.cc:385] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:35.734311 23070 raft_consensus.cc:740] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:35.735213 23070 consensus_queue.cc:260] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.740244 23070 ts_tablet_manager.cc:1434] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.003s
I20260709 13:07:35.741560 23070 tablet_bootstrap.cc:492] T 13e53d9e00624dfcb87aca9b055fdc9a P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:35.699129 23140 raft_consensus.cc:2804] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:35.743368 23116 consensus_queue.cc:1048] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:07:35.771256 23075 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.756927 23140 raft_consensus.cc:697] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:07:35.773484 23140 consensus_queue.cc:237] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.788039 23069 tablet_bootstrap.cc:492] T a0b6723ad2db4fccb7a1c219ff04e99a P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:35.788599 23069 ts_tablet_manager.cc:1403] T a0b6723ad2db4fccb7a1c219ff04e99a P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.078s	user 0.028s	sys 0.032s
I20260709 13:07:35.792451 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:07:35.793656 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:07:35.793977 23029 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:35.794143 23070 tablet_bootstrap.cc:654] T 13e53d9e00624dfcb87aca9b055fdc9a P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:35.791348 23069 raft_consensus.cc:359] T a0b6723ad2db4fccb7a1c219ff04e99a P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.798202 23069 raft_consensus.cc:385] T a0b6723ad2db4fccb7a1c219ff04e99a P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:35.798491 23069 raft_consensus.cc:740] T a0b6723ad2db4fccb7a1c219ff04e99a P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:35.799289 23069 consensus_queue.cc:260] T a0b6723ad2db4fccb7a1c219ff04e99a P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.800113 22878 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:35.799618 22778 catalog_manager.cc:5697] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:35.801599 23028 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:35.810865 22878 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 1.
I20260709 13:07:35.814245 23069 ts_tablet_manager.cc:1434] T a0b6723ad2db4fccb7a1c219ff04e99a P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.025s	user 0.005s	sys 0.000s
I20260709 13:07:35.815917 23069 tablet_bootstrap.cc:492] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:35.821404 22920 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:07:35.822265 23115 raft_consensus.cc:2804] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:35.822736 23115 raft_consensus.cc:697] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:07:35.823571 23115 consensus_queue.cc:237] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.825763 23069 tablet_bootstrap.cc:654] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:35.825692 23116 consensus_queue.cc:1048] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:35.825858 23029 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 1.
I20260709 13:07:35.828060 23075 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:35.827615 23140 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.827288 23125 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.829599 23140 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.832603 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:35.833153 22954 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:35.835048 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:07:35.837088 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:35.837705 22878 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 0.
I20260709 13:07:35.839025 22994 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:35.839903 23139 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:35.840152 23069 tablet_bootstrap.cc:492] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:35.840278 23139 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:35.840760 23069 ts_tablet_manager.cc:1403] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.006s
I20260709 13:07:35.840806 23139 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:35.840214 22954 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 1.
I20260709 13:07:35.844408 23069 raft_consensus.cc:359] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.845438 23069 raft_consensus.cc:385] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:35.845798 23069 raft_consensus.cc:740] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:35.846591 23069 consensus_queue.cc:260] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.847487 23125 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:35.846354 23140 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:35.853276 23069 ts_tablet_manager.cc:1434] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:07:35.855772 23075 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.856352 23075 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.858492 23075 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:35.853114 22843 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:07:35.859876 23069 tablet_bootstrap.cc:492] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
W20260709 13:07:35.859889 22843 leader_election.cc:343] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:35.860913 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:35.862413 23093 raft_consensus.cc:2804] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:35.863765 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:35.863871 23093 raft_consensus.cc:697] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
W20260709 13:07:35.862404 22920 leader_election.cc:343] T 7f35ebcf93ad423ea8bd0b777bee23ee P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:35.865077 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:35.864921 23093 consensus_queue.cc:237] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:07:35.866248 22918 leader_election.cc:343] T 7f35ebcf93ad423ea8bd0b777bee23ee P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:35.866786 22918 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:35.862527 23140 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.867672 23140 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.869532 22954 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 0.
W20260709 13:07:35.859074 22826 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:35.872174 23075 raft_consensus.cc:2749] T 7f35ebcf93ad423ea8bd0b777bee23ee P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:35.862335 23139 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.874773 23139 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:35.875779 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:07:35.876542 22878 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:35.879355 23070 tablet_bootstrap.cc:492] T 13e53d9e00624dfcb87aca9b055fdc9a P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:35.879866 23070 ts_tablet_manager.cc:1403] T 13e53d9e00624dfcb87aca9b055fdc9a P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.139s	user 0.014s	sys 0.020s
I20260709 13:07:35.880614 23140 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:35.881146 23073 tablet_bootstrap.cc:492] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:35.881649 23073 ts_tablet_manager.cc:1403] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.167s	user 0.029s	sys 0.010s
I20260709 13:07:35.881207 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:35.883566 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:35.883818 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:35.884177 22879 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 0.
I20260709 13:07:35.884403 22955 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 0.
I20260709 13:07:35.883132 22954 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:35.885705 22994 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:35.886186 22776 catalog_manager.cc:5697] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 1 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:35.887408 23140 raft_consensus.cc:2804] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:35.889685 23140 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:35.889955 22994 consensus_peers.cc:597] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:35.890142 23140 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:35.890821 23070 raft_consensus.cc:359] T 13e53d9e00624dfcb87aca9b055fdc9a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.891461 23070 raft_consensus.cc:385] T 13e53d9e00624dfcb87aca9b055fdc9a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:35.891746 23070 raft_consensus.cc:740] T 13e53d9e00624dfcb87aca9b055fdc9a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:35.891937 22878 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:07:35.892414 23070 consensus_queue.cc:260] T 13e53d9e00624dfcb87aca9b055fdc9a P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.893222 22994 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:35.894767 23139 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:35.895480 23139 raft_consensus.cc:697] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:07:35.896471 22954 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:07:35.896327 23139 consensus_queue.cc:237] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.897991 23073 raft_consensus.cc:359] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.898633 23073 raft_consensus.cc:385] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:35.898906 23073 raft_consensus.cc:740] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:35.900064 23073 consensus_queue.cc:260] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.905990 23073 ts_tablet_manager.cc:1434] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.000s
I20260709 13:07:35.907289 23073 tablet_bootstrap.cc:492] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:35.912413 23070 ts_tablet_manager.cc:1434] T 13e53d9e00624dfcb87aca9b055fdc9a P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.001s
I20260709 13:07:35.913549 23070 tablet_bootstrap.cc:492] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:35.914618 23069 tablet_bootstrap.cc:654] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:35.917680 23140 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.924795 23140 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:35.927034 23069 tablet_bootstrap.cc:492] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:35.927536 23069 ts_tablet_manager.cc:1403] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.068s	user 0.019s	sys 0.007s
I20260709 13:07:35.927392 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:35.928110 23070 tablet_bootstrap.cc:654] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:35.928236 22954 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:35.929404 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:07:35.932761 23073 tablet_bootstrap.cc:654] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:35.934252 23075 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.934722 23075 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.935878 22878 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:35.937423 23075 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:35.939173 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:07:35.940557 22918 leader_election.cc:343] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:35.941013 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:35.941550 22879 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 0.
I20260709 13:07:35.942646 22920 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:35.943792 23075 raft_consensus.cc:2804] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:35.944175 23075 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:35.944530 23075 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:35.950016 22954 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:07:35.949015 22776 catalog_manager.cc:5697] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:35.951529 23075 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.954211 22992 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:35.955058 23140 raft_consensus.cc:2804] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:35.955477 23075 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:35.955530 23140 raft_consensus.cc:697] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:07:35.951913 22878 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:07:35.931299 23069 raft_consensus.cc:359] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.956940 23069 raft_consensus.cc:385] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:35.956645 23140 consensus_queue.cc:237] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.957242 23069 raft_consensus.cc:740] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:35.957183 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:07:35.958165 23069 consensus_queue.cc:260] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.959003 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:07:35.959522 22879 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:35.960197 23069 ts_tablet_manager.cc:1434] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.032s	user 0.001s	sys 0.003s
I20260709 13:07:35.961108 23069 tablet_bootstrap.cc:492] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:35.962462 22777 catalog_manager.cc:5697] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:35.965335 22918 leader_election.cc:343] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:35.967109 23069 tablet_bootstrap.cc:654] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:35.967837 22879 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 1.
I20260709 13:07:35.969348 22920 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:35.970841 23070 tablet_bootstrap.cc:492] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:35.971437 23070 ts_tablet_manager.cc:1403] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.058s	user 0.004s	sys 0.007s
I20260709 13:07:35.973274 23115 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:35.973877 23115 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.975056 23073 tablet_bootstrap.cc:492] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:35.975478 23073 ts_tablet_manager.cc:1403] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.068s	user 0.014s	sys 0.001s
I20260709 13:07:35.976032 23115 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:35.973172 23075 raft_consensus.cc:2804] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:35.977125 23075 raft_consensus.cc:697] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:07:35.977674 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:35.978264 23075 consensus_queue.cc:237] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:07:35.979712 22918 leader_election.cc:343] T 13e53d9e00624dfcb87aca9b055fdc9a P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:35.983515 23073 raft_consensus.cc:359] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:35.989521 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:35.984472 23073 raft_consensus.cc:385] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:35.992395 23073 raft_consensus.cc:740] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:35.993551 23073 consensus_queue.cc:260] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:07:35.998169 22920 leader_election.cc:343] T 13e53d9e00624dfcb87aca9b055fdc9a P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:35.998700 22920 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:35.999825 23075 raft_consensus.cc:2749] T 13e53d9e00624dfcb87aca9b055fdc9a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:35.991255 23070 raft_consensus.cc:359] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.000828 23070 raft_consensus.cc:385] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.001251 23070 raft_consensus.cc:740] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.002417 23070 consensus_queue.cc:260] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.005577 23070 ts_tablet_manager.cc:1434] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.034s	user 0.004s	sys 0.007s
I20260709 13:07:36.006896 23070 tablet_bootstrap.cc:492] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:35.996577 23073 ts_tablet_manager.cc:1434] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 13:07:36.010051 23069 tablet_bootstrap.cc:492] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:36.010721 23073 tablet_bootstrap.cc:492] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:36.011181 23069 ts_tablet_manager.cc:1403] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.050s	user 0.024s	sys 0.020s
I20260709 13:07:36.014456 22778 catalog_manager.cc:5697] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:36.016037 23070 tablet_bootstrap.cc:654] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.025031 23069 raft_consensus.cc:359] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.025799 23069 raft_consensus.cc:385] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.026120 23069 raft_consensus.cc:740] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.026859 23069 consensus_queue.cc:260] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.027063 22776 catalog_manager.cc:5697] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 1 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:36.029357 23069 ts_tablet_manager.cc:1434] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.008s
I20260709 13:07:36.030310 23069 tablet_bootstrap.cc:492] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:36.035830 23069 tablet_bootstrap.cc:654] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.036643 23073 tablet_bootstrap.cc:654] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.043607 23070 tablet_bootstrap.cc:492] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:36.044042 23070 ts_tablet_manager.cc:1403] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.037s	user 0.016s	sys 0.016s
I20260709 13:07:36.046597 23070 raft_consensus.cc:359] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.047112 23073 tablet_bootstrap.cc:492] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:36.047281 23069 tablet_bootstrap.cc:492] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:36.047242 23070 raft_consensus.cc:385] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.047559 23073 ts_tablet_manager.cc:1403] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.039s	user 0.013s	sys 0.000s
I20260709 13:07:36.047796 23069 ts_tablet_manager.cc:1403] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.018s	user 0.015s	sys 0.000s
I20260709 13:07:36.047701 23070 raft_consensus.cc:740] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.048627 23070 consensus_queue.cc:260] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.050249 23073 raft_consensus.cc:359] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.050640 23070 ts_tablet_manager.cc:1434] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:07:36.050917 23073 raft_consensus.cc:385] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.051216 23073 raft_consensus.cc:740] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.051548 23070 tablet_bootstrap.cc:492] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:36.052572 23069 raft_consensus.cc:359] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.053241 23069 raft_consensus.cc:385] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.053499 23069 raft_consensus.cc:740] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.054128 23069 consensus_queue.cc:260] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.056002 23069 ts_tablet_manager.cc:1434] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.000s
I20260709 13:07:36.056778 23069 tablet_bootstrap.cc:492] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:36.057348 23070 tablet_bootstrap.cc:654] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.056955 23073 consensus_queue.cc:260] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.062211 23069 tablet_bootstrap.cc:654] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.067052 23075 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.067788 23075 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.068711 23070 tablet_bootstrap.cc:492] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:36.068575 23073 ts_tablet_manager.cc:1434] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 13:07:36.069149 23070 ts_tablet_manager.cc:1403] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.018s	user 0.016s	sys 0.000s
I20260709 13:07:36.069753 23073 tablet_bootstrap.cc:492] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:36.071656 23070 raft_consensus.cc:359] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.072326 23070 raft_consensus.cc:385] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.072599 23070 raft_consensus.cc:740] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.074606 23070 consensus_queue.cc:260] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.075673 23069 tablet_bootstrap.cc:492] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:36.076114 23069 ts_tablet_manager.cc:1403] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.001s
I20260709 13:07:36.076884 23070 ts_tablet_manager.cc:1434] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:07:36.077734 23070 tablet_bootstrap.cc:492] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:36.078686 23069 raft_consensus.cc:359] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.079334 23069 raft_consensus.cc:385] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.079599 23069 raft_consensus.cc:740] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.080430 23069 consensus_queue.cc:260] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.082409 23069 ts_tablet_manager.cc:1434] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:07:36.083230 23069 tablet_bootstrap.cc:492] T ec334666b7964c969125af95d9c797ed P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:36.083746 23115 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.084308 23115 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.085666 23073 tablet_bootstrap.cc:654] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.086334 23115 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:36.086426 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:36.087122 22879 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 0.
I20260709 13:07:36.087984 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:36.088712 23028 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 0.
I20260709 13:07:36.088851 22920 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:07:36.090348 23069 tablet_bootstrap.cc:654] T ec334666b7964c969125af95d9c797ed P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.090332 22918 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:36.091614 23116 raft_consensus.cc:2804] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:36.092067 23116 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:36.092482 23116 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.092730 23140 consensus_queue.cc:1048] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:36.096994 23094 raft_consensus.cc:2804] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:36.097559 23094 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:36.097986 23094 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.101749 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:36.102612 22879 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 0.
I20260709 13:07:36.102612 23070 tablet_bootstrap.cc:654] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.102114 23116 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.111663 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:07:36.112417 22879 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.112720 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:07:36.113511 23028 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.123811 23140 consensus_queue.cc:1048] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:36.131533 23116 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:36.134649 23140 consensus_queue.cc:1048] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:36.141237 23139 consensus_queue.cc:1048] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:36.150945 23028 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 1.
I20260709 13:07:36.156909 22918 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:36.155304 23073 tablet_bootstrap.cc:492] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:36.157750 23073 ts_tablet_manager.cc:1403] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.088s	user 0.016s	sys 0.044s
I20260709 13:07:36.157863 23116 raft_consensus.cc:2804] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:36.159265 23116 raft_consensus.cc:697] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:07:36.160362 23116 consensus_queue.cc:237] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.163610 23075 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:36.165946 23069 tablet_bootstrap.cc:492] T ec334666b7964c969125af95d9c797ed P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:36.166534 23069 ts_tablet_manager.cc:1403] T ec334666b7964c969125af95d9c797ed P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.083s	user 0.035s	sys 0.042s
I20260709 13:07:36.165879 23073 raft_consensus.cc:359] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.169317 23073 raft_consensus.cc:385] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.169255 23069 raft_consensus.cc:359] T ec334666b7964c969125af95d9c797ed P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.170626 23069 raft_consensus.cc:385] T ec334666b7964c969125af95d9c797ed P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.170279 23073 raft_consensus.cc:740] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.170945 23069 raft_consensus.cc:740] T ec334666b7964c969125af95d9c797ed P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.171690 23069 consensus_queue.cc:260] T ec334666b7964c969125af95d9c797ed P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.171766 23073 consensus_queue.cc:260] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.174537 23069 ts_tablet_manager.cc:1434] T ec334666b7964c969125af95d9c797ed P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:07:36.175207 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:36.175683 23069 tablet_bootstrap.cc:492] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:36.178005 22879 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 1.
I20260709 13:07:36.176748 23028 raft_consensus.cc:2376] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 1. Re-sending same reply.
I20260709 13:07:36.173617 23094 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.183017 23094 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:36.183712 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:07:36.183876 23069 tablet_bootstrap.cc:654] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.184026 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:07:36.176589 23073 ts_tablet_manager.cc:1434] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.018s	user 0.003s	sys 0.004s
I20260709 13:07:36.184329 22879 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.187320 23094 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.187808 23094 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.191093 23073 tablet_bootstrap.cc:492] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:36.191093 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:36.191810 22880 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 0.
I20260709 13:07:36.185771 23028 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.194269 22920 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:07:36.205646 23075 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:36.206434 23075 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:36.206946 23075 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.211280 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:07:36.213133 22918 leader_election.cc:343] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:36.213762 23094 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:36.231423 23028 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 1.
I20260709 13:07:36.234253 22918 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:36.228329 23073 tablet_bootstrap.cc:654] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.236541 23094 consensus_queue.cc:1048] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:36.236233 23117 raft_consensus.cc:2804] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:36.242085 23117 raft_consensus.cc:697] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:07:36.269836 23070 tablet_bootstrap.cc:492] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:36.270349 23070 ts_tablet_manager.cc:1403] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.193s	user 0.006s	sys 0.047s
I20260709 13:07:36.283519 23093 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.284232 23093 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.286516 23093 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:36.287937 23070 raft_consensus.cc:359] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.288769 23070 raft_consensus.cc:385] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.289129 23070 raft_consensus.cc:740] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.290349 23070 consensus_queue.cc:260] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.289139 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:36.300384 23140 consensus_queue.cc:1048] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:07:36.305022 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:36.306890 22955 raft_consensus.cc:2393] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 1: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:07:36.308032 23070 ts_tablet_manager.cc:1434] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.002s
I20260709 13:07:36.254026 23117 consensus_queue.cc:237] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.332438 23073 tablet_bootstrap.cc:492] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:36.332981 23073 ts_tablet_manager.cc:1403] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.147s	user 0.020s	sys 0.005s
I20260709 13:07:36.335588 23073 raft_consensus.cc:359] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.336336 23073 raft_consensus.cc:385] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.330265 23028 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.343209 23073 raft_consensus.cc:740] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.344226 23073 consensus_queue.cc:260] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.322038 23075 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.336262 23129 consensus_queue.cc:1048] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.020s
I20260709 13:07:36.346866 22879 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 1.
I20260709 13:07:36.362883 23070 tablet_bootstrap.cc:492] T 392f5a2324d74a499e6baad745c941fd P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:36.338130 22843 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:36.398996 23115 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.399587 23115 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.401455 23094 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.458391 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:07:36.402141 22776 catalog_manager.cc:5697] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 1 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:36.457880 23073 ts_tablet_manager.cc:1434] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.124s	user 0.009s	sys 0.000s
I20260709 13:07:36.462150 23069 tablet_bootstrap.cc:492] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:36.462579 23069 ts_tablet_manager.cc:1403] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.287s	user 0.014s	sys 0.041s
I20260709 13:07:36.463369 23073 tablet_bootstrap.cc:492] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:36.407707 23093 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.402105 23094 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.461069 22879 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.402181 23075 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:07:36.457420 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 126606us
I20260709 13:07:36.408329 23183 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:36.461261 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 56464.1us
I20260709 13:07:36.767169 23186 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.773512 23185 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.773557 23186 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.591682 23125 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.774138 23185 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.591293 23117 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.511404 23094 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:36.467259 23140 consensus_queue.cc:1048] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.305s
I20260709 13:07:36.585213 23115 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:36.778092 23125 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.766191 23183 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.781404 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:07:36.779162 23117 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.800437 23193 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.801172 23193 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.804330 23193 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:36.805063 23193 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.805737 23193 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.794715 23027 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.817472 23069 raft_consensus.cc:359] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.853022 23070 tablet_bootstrap.cc:654] T 392f5a2324d74a499e6baad745c941fd P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.852417 23069 raft_consensus.cc:385] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.853775 23069 raft_consensus.cc:740] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.855053 23069 consensus_queue.cc:260] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.825919 23185 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:36.828543 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:36.833994 23117 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:36.862900 23193 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:36.858486 23191 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:36.864285 23191 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.859203 23093 raft_consensus.cc:2749] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:36.872323 23069 ts_tablet_manager.cc:1434] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.409s	user 0.005s	sys 0.008s
I20260709 13:07:36.867710 22877 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 0.
I20260709 13:07:36.873380 23069 tablet_bootstrap.cc:492] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:36.871538 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:36.876585 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:36.876737 23125 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:07:36.877574 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 388579us
I20260709 13:07:36.886710 23125 consensus_queue.cc:1048] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:07:36.889576 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:36.888970 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:36.890411 22953 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 0.
I20260709 13:07:36.895534 23140 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:36.943151 23197 log.cc:927] Time spent T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:07:36.929493 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:36.896371 23183 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:36.947718 23073 tablet_bootstrap.cc:654] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:36.891433 22880 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 0.
I20260709 13:07:36.959904 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:36.960543 23070 tablet_bootstrap.cc:492] T 392f5a2324d74a499e6baad745c941fd P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:36.961045 23070 ts_tablet_manager.cc:1403] T 392f5a2324d74a499e6baad745c941fd P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.598s	user 0.018s	sys 0.017s
I20260709 13:07:36.960894 22952 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 0.
I20260709 13:07:36.961239 22955 raft_consensus.cc:2393] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 1: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:07:36.944101 23140 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.971479 22879 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 1.
I20260709 13:07:36.973839 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:36.972147 23070 raft_consensus.cc:359] T 392f5a2324d74a499e6baad745c941fd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.976305 23070 raft_consensus.cc:385] T 392f5a2324d74a499e6baad745c941fd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:36.976524 23070 raft_consensus.cc:740] T 392f5a2324d74a499e6baad745c941fd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:36.976948 23028 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.977340 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:36.977221 23070 consensus_queue.cc:260] T 392f5a2324d74a499e6baad745c941fd P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:36.978330 22875 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 0.
I20260709 13:07:36.979353 23070 ts_tablet_manager.cc:1434] T 392f5a2324d74a499e6baad745c941fd P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 13:07:36.983750 22843 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:07:36.984571 23125 raft_consensus.cc:2804] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:36.984691 22843 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:07:36.984925 23140 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:36.985061 23125 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:36.986452 23201 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:36.986857 23201 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:36.987182 23201 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:36.975051 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:36.998308 23186 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:36.985589 23125 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.025846 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:37.026655 22877 raft_consensus.cc:2393] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 1: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:07:37.041296 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:37.019064 23196 consensus_queue.cc:1048] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:07:37.003587 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:37.074916 23070 tablet_bootstrap.cc:492] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:37.021574 23191 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:37.149753 22952 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 0.
I20260709 13:07:37.157610 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:37.168529 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:37.183597 23125 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.202350 22875 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.203825 23125 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:37.206588 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:37.210062 23201 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.213289 23069 tablet_bootstrap.cc:654] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:37.214078 23207 consensus_queue.cc:1048] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:37.216707 23027 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 1.
I20260709 13:07:37.221454 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:37.222354 22876 raft_consensus.cc:2393] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 1: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:07:37.223233 23070 tablet_bootstrap.cc:654] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:37.224717 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:37.226598 23073 tablet_bootstrap.cc:492] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:37.226914 23201 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:37.227710 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:37.229935 23116 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:37.230266 22826 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:37.230372 23116 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.230993 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:07:37.232126 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 207525us
I20260709 13:07:37.232431 23116 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:07:37.233067 22918 leader_election.cc:343] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:37.232987 23116 consensus_queue.cc:1048] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:07:37.233723 22918 leader_election.cc:343] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:07:37.234550 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 188399us
I20260709 13:07:37.235687 22920 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:37.236274 23184 consensus_queue.cc:1048] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:37.236514 23116 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:37.227052 23073 ts_tablet_manager.cc:1403] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.764s	user 0.019s	sys 0.056s
I20260709 13:07:37.237143 23117 consensus_queue.cc:1048] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:37.237275 22920 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:37.238391 23117 raft_consensus.cc:2804] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:37.238744 23117 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.238843 22920 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:07:37.239331 23117 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.240177 23075 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:37.240097 23073 raft_consensus.cc:359] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.240746 23075 raft_consensus.cc:697] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:07:37.241752 23075 consensus_queue.cc:237] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.236986 23116 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.245827 23116 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.248422 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:37.240855 23073 raft_consensus.cc:385] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:37.247319 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:37.261663 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:07:37.262370 22994 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:07:37.263218 23073 raft_consensus.cc:740] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:37.263953 23073 consensus_queue.cc:260] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.264858 23027 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 0.
I20260709 13:07:37.265031 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:07:37.274596 23069 tablet_bootstrap.cc:492] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:37.275116 23069 ts_tablet_manager.cc:1403] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.402s	user 0.021s	sys 0.003s
I20260709 13:07:37.279150 23186 raft_consensus.cc:2749] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:37.279541 23070 tablet_bootstrap.cc:492] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:37.280036 23070 ts_tablet_manager.cc:1403] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.205s	user 0.016s	sys 0.009s
I20260709 13:07:37.283092 23073 ts_tablet_manager.cc:1434] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.046s	user 0.005s	sys 0.000s
I20260709 13:07:37.284025 23073 tablet_bootstrap.cc:492] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:37.287147 23207 consensus_queue.cc:1048] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:37.288858 23070 raft_consensus.cc:359] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.289562 23070 raft_consensus.cc:385] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:37.289824 23070 raft_consensus.cc:740] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:37.291903 23207 consensus_queue.cc:1048] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:37.293012 23070 consensus_queue.cc:260] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.296677 22994 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:07:37.297029 23070 ts_tablet_manager.cc:1434] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.003s
I20260709 13:07:37.301903 23070 tablet_bootstrap.cc:492] T aa6540758938493d98d7852fdbe3ffab P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:37.302883 23116 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.326357 23116 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:37.306864 23069 raft_consensus.cc:359] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.307981 23207 raft_consensus.cc:2749] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:37.309247 22994 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:07:37.332202 23073 tablet_bootstrap.cc:654] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
W20260709 13:07:37.234545 22918 leader_election.cc:343] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:37.311558 23070 tablet_bootstrap.cc:654] T aa6540758938493d98d7852fdbe3ffab P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:37.332501 23069 raft_consensus.cc:385] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:37.332872 23069 raft_consensus.cc:740] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:37.334090 23207 raft_consensus.cc:2749] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:37.334684 22920 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
W20260709 13:07:37.335338 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 95602.2us
I20260709 13:07:37.335493 23207 consensus_queue.cc:1048] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:37.336241 23075 raft_consensus.cc:2804] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:37.336597 23075 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.336946 23075 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:07:37.338174 22843 leader_election.cc:343] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:37.339157 23069 consensus_queue.cc:260] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:07:37.340520 22920 leader_election.cc:343] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:07:37.340534 22843 leader_election.cc:343] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:37.344574 22843 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:07:37.345433 23201 raft_consensus.cc:2804] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:37.309453 23117 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.352104 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:37.344962 23075 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.356801 23201 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.357349 23201 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.357497 22843 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:37.358464 23075 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:37.358834 23125 raft_consensus.cc:2749] T ec334666b7964c969125af95d9c797ed P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:07:37.345657 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 111133us
I20260709 13:07:37.359015 23075 consensus_queue.cc:1048] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
W20260709 13:07:37.370752 22918 leader_election.cc:343] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:37.364034 23069 ts_tablet_manager.cc:1434] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.089s	user 0.008s	sys 0.000s
I20260709 13:07:37.367033 23070 tablet_bootstrap.cc:492] T aa6540758938493d98d7852fdbe3ffab P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:37.401233 23070 ts_tablet_manager.cc:1403] T aa6540758938493d98d7852fdbe3ffab P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.101s	user 0.011s	sys 0.017s
I20260709 13:07:37.401072 22918 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:37.402735 23189 raft_consensus.cc:2749] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:37.403339 23189 consensus_queue.cc:1048] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:37.403918 23070 raft_consensus.cc:359] T aa6540758938493d98d7852fdbe3ffab P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.371446 23184 consensus_queue.cc:1048] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:37.372321 23117 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:37.374119 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:07:37.405313 22876 raft_consensus.cc:2393] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 1: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:07:37.375631 23073 tablet_bootstrap.cc:492] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:37.406116 23069 tablet_bootstrap.cc:492] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:37.406260 23073 ts_tablet_manager.cc:1403] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.122s	user 0.020s	sys 0.007s
I20260709 13:07:37.375846 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:37.409134 22954 raft_consensus.cc:2393] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 1: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
W20260709 13:07:37.363808 22843 leader_election.cc:343] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:07:37.410331 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 46569.6us
I20260709 13:07:37.404603 23070 raft_consensus.cc:385] T aa6540758938493d98d7852fdbe3ffab P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:07:37.411157 22843 leader_election.cc:343] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:37.412700 23073 raft_consensus.cc:359] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.413427 23073 raft_consensus.cc:385] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:37.413740 23073 raft_consensus.cc:740] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:37.414299 23069 tablet_bootstrap.cc:654] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:37.415002 23073 consensus_queue.cc:260] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.417246 23073 ts_tablet_manager.cc:1434] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:07:37.418540 23073 tablet_bootstrap.cc:492] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:37.382728 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:07:37.419826 22877 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.383690 23191 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.426357 22776 catalog_manager.cc:5697] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 1 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:37.427551 23191 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.383827 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:07:37.411355 23070 raft_consensus.cc:740] T aa6540758938493d98d7852fdbe3ffab P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:37.429350 22874 raft_consensus.cc:2393] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 1: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:07:37.429239 23070 consensus_queue.cc:260] T aa6540758938493d98d7852fdbe3ffab P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.430492 23117 consensus_queue.cc:1048] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:37.431404 23070 ts_tablet_manager.cc:1434] T aa6540758938493d98d7852fdbe3ffab P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.030s	user 0.005s	sys 0.001s
I20260709 13:07:37.431852 23191 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:37.432319 23070 tablet_bootstrap.cc:492] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
W20260709 13:07:37.381767 22920 leader_election.cc:343] T 392f5a2324d74a499e6baad745c941fd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:07:37.434057 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 52331.9us
I20260709 13:07:37.435271 23184 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.435452 22877 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 1.
I20260709 13:07:37.435719 23184 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.437842 23184 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:37.438251 22843 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:37.439863 23191 raft_consensus.cc:2749] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:07:37.377094 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:37.443769 22953 raft_consensus.cc:2393] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 1: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:07:37.446497 23201 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.450049 22920 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:07:37.450845 23189 raft_consensus.cc:2804] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:37.451050 23201 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:37.451323 23189 raft_consensus.cc:697] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:07:37.452180 23189 consensus_queue.cc:237] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.453814 23069 tablet_bootstrap.cc:492] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:37.454272 23069 ts_tablet_manager.cc:1403] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.048s	user 0.011s	sys 0.002s
I20260709 13:07:37.454406 22843 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:37.455508 23070 tablet_bootstrap.cc:654] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:37.455711 23191 raft_consensus.cc:2749] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:07:37.457578 23117 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.458083 23117 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.460227 23117 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:37.457690 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:37.465997 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:37.473450 22776 catalog_manager.cc:5697] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 1 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:37.477334 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:07:37.478117 23025 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.481782 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:07:37.464071 23069 raft_consensus.cc:359] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.485602 23069 raft_consensus.cc:385] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:37.486689 23025 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 1.
I20260709 13:07:37.486678 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:37.485898 23069 raft_consensus.cc:740] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:37.488778 23069 consensus_queue.cc:260] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.489682 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:07:37.490741 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:37.491631 22953 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.492143 23069 ts_tablet_manager.cc:1434] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.038s	user 0.008s	sys 0.000s
I20260709 13:07:37.493062 23069 tablet_bootstrap.cc:492] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:37.497745 23070 tablet_bootstrap.cc:492] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:37.498157 23070 ts_tablet_manager.cc:1403] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.066s	user 0.016s	sys 0.001s
W20260709 13:07:37.499938 22843 leader_election.cc:343] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:37.499992 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:37.504098 23073 tablet_bootstrap.cc:654] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:37.503985 23070 raft_consensus.cc:359] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.504720 23070 raft_consensus.cc:385] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:37.505069 23070 raft_consensus.cc:740] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:37.506333 23070 consensus_queue.cc:260] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.513365 23189 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.513928 23189 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.521974 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:37.522289 23189 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:37.524961 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:37.527930 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:37.533816 23070 ts_tablet_manager.cc:1434] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.035s	user 0.009s	sys 0.000s
I20260709 13:07:37.538043 23070 tablet_bootstrap.cc:492] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
W20260709 13:07:37.544931 22918 leader_election.cc:343] T 392f5a2324d74a499e6baad745c941fd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:37.545457 22918 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:37.546231 23116 raft_consensus.cc:2749] T 392f5a2324d74a499e6baad745c941fd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:07:37.547006 22918 leader_election.cc:343] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:37.551721 23069 tablet_bootstrap.cc:654] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
W20260709 13:07:37.552281 22920 leader_election.cc:343] T 7f35ebcf93ad423ea8bd0b777bee23ee P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:37.564641 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:37.558511 22953 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 1.
I20260709 13:07:37.569221 22918 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
W20260709 13:07:37.571413 22918 leader_election.cc:343] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:37.571834 22918 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
W20260709 13:07:37.573029 22918 leader_election.cc:343] T 7f35ebcf93ad423ea8bd0b777bee23ee P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:37.573539 23183 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:37.573998 23116 raft_consensus.cc:2749] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:07:37.574041 23183 raft_consensus.cc:697] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:07:37.573464 22918 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:37.576879 23075 raft_consensus.cc:2749] T 7f35ebcf93ad423ea8bd0b777bee23ee P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:07:37.569591 22920 leader_election.cc:343] T aa6540758938493d98d7852fdbe3ffab P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:07:37.582645 22843 leader_election.cc:343] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:37.581256 22778 catalog_manager.cc:5697] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 1 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:37.583145 22843 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:37.587733 23201 raft_consensus.cc:2749] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:07:37.587806 22918 leader_election.cc:343] T aa6540758938493d98d7852fdbe3ffab P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:37.574987 23183 consensus_queue.cc:237] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.589195 22918 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:37.593149 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:07:37.593959 22918 leader_election.cc:343] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:37.585222 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:07:37.597218 23029 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.615384 22843 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:07:37.616240 23201 raft_consensus.cc:2804] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:37.617262 23201 raft_consensus.cc:697] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:07:37.618105 23201 consensus_queue.cc:237] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.600364 23117 raft_consensus.cc:2749] T aa6540758938493d98d7852fdbe3ffab P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:07:37.622503 22920 leader_election.cc:343] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:37.623024 22920 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:37.627277 23184 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:37.628741 23029 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 1.
I20260709 13:07:37.633531 23191 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.634100 23191 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.635210 23073 tablet_bootstrap.cc:492] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:37.635643 23073 ts_tablet_manager.cc:1403] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.217s	user 0.029s	sys 0.017s
I20260709 13:07:37.636379 23191 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:37.637537 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:37.638252 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:37.638368 22953 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 0.
I20260709 13:07:37.641954 23070 tablet_bootstrap.cc:654] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:37.641691 22776 catalog_manager.cc:5697] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 1 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:37.641664 22843 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:07:37.646221 23073 raft_consensus.cc:359] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.647677 23073 raft_consensus.cc:385] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:37.647748 23191 raft_consensus.cc:2804] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:07:37.647982 22843 leader_election.cc:343] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:37.648173 23191 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.648058 23073 raft_consensus.cc:740] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:37.648587 23191 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.651122 23073 consensus_queue.cc:260] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.657454 23069 tablet_bootstrap.cc:492] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:37.657934 23069 ts_tablet_manager.cc:1403] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.165s	user 0.007s	sys 0.014s
I20260709 13:07:37.659127 23073 ts_tablet_manager.cc:1434] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.023s	user 0.005s	sys 0.004s
I20260709 13:07:37.660017 23073 tablet_bootstrap.cc:492] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:37.678081 23191 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.680418 23069 raft_consensus.cc:359] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.682344 23069 raft_consensus.cc:385] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:37.682693 23069 raft_consensus.cc:740] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:37.683681 23069 consensus_queue.cc:260] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.687793 23069 ts_tablet_manager.cc:1434] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.001s
I20260709 13:07:37.688092 23073 tablet_bootstrap.cc:654] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:37.688611 23069 tablet_bootstrap.cc:492] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:37.689513 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:37.690116 22953 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.694633 22778 catalog_manager.cc:5697] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:37.699393 23070 tablet_bootstrap.cc:492] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:37.699805 23070 ts_tablet_manager.cc:1403] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.162s	user 0.027s	sys 0.008s
I20260709 13:07:37.700671 23191 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:37.701022 22953 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 1.
I20260709 13:07:37.702965 22843 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:07:37.704483 23069 tablet_bootstrap.cc:654] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:37.704825 23191 raft_consensus.cc:2804] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:37.704946 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:07:37.706233 22843 leader_election.cc:343] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:37.705593 23191 raft_consensus.cc:697] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:07:37.708117 23191 consensus_queue.cc:237] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.710047 23070 raft_consensus.cc:359] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.710713 23070 raft_consensus.cc:385] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:37.710978 23070 raft_consensus.cc:740] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:37.711681 23070 consensus_queue.cc:260] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.725428 23070 ts_tablet_manager.cc:1434] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.001s
I20260709 13:07:37.726392 23070 tablet_bootstrap.cc:492] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:37.728608 23069 tablet_bootstrap.cc:492] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:37.729146 23069 ts_tablet_manager.cc:1403] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.041s	user 0.006s	sys 0.005s
I20260709 13:07:37.729727 23073 tablet_bootstrap.cc:492] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:37.730161 23073 ts_tablet_manager.cc:1403] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.070s	user 0.012s	sys 0.000s
I20260709 13:07:37.732806 23069 raft_consensus.cc:359] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.733685 23069 raft_consensus.cc:385] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:37.733969 23069 raft_consensus.cc:740] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:37.737528 23073 raft_consensus.cc:359] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.738251 23073 raft_consensus.cc:385] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:37.738520 23073 raft_consensus.cc:740] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:37.738797 23184 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.739372 23070 tablet_bootstrap.cc:654] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:37.739249 23184 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.741493 23184 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:37.742054 23069 consensus_queue.cc:260] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.742326 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:37.742449 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:37.739259 23073 consensus_queue.cc:260] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:07:37.743711 22920 leader_election.cc:343] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:37.744370 23069 ts_tablet_manager.cc:1434] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
W20260709 13:07:37.744354 22918 leader_election.cc:343] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:37.744897 22918 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:37.745357 23069 tablet_bootstrap.cc:492] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:37.745980 23184 raft_consensus.cc:2749] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:37.748719 23073 ts_tablet_manager.cc:1434] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.002s
I20260709 13:07:37.753952 23073 tablet_bootstrap.cc:492] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:37.754520 23070 tablet_bootstrap.cc:492] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:37.754954 23070 ts_tablet_manager.cc:1403] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.029s	user 0.009s	sys 0.000s
I20260709 13:07:37.757531 23070 raft_consensus.cc:359] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.758215 23070 raft_consensus.cc:385] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:37.758482 23070 raft_consensus.cc:740] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:37.759461 23073 tablet_bootstrap.cc:654] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:37.759150 23070 consensus_queue.cc:260] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.751823 23069 tablet_bootstrap.cc:654] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:37.751574 23184 consensus_queue.cc:1048] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:37.752187 22776 catalog_manager.cc:5697] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:37.765658 23070 ts_tablet_manager.cc:1434] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 13:07:37.767203 23070 tablet_bootstrap.cc:492] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:37.776183 23070 tablet_bootstrap.cc:654] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:37.787417 23191 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.788017 23191 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.791015 23191 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:37.791054 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:07:37.793021 22843 leader_election.cc:343] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:37.792907 23183 consensus_queue.cc:1048] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:37.793802 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:07:37.795205 22843 leader_election.cc:343] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:37.795624 22843 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:37.796648 23191 raft_consensus.cc:2749] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:37.811957 23070 tablet_bootstrap.cc:492] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:37.812515 23070 ts_tablet_manager.cc:1403] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.046s	user 0.009s	sys 0.008s
I20260709 13:07:37.833319 23069 tablet_bootstrap.cc:492] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:37.833794 23069 ts_tablet_manager.cc:1403] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.089s	user 0.015s	sys 0.025s
I20260709 13:07:37.833851 23073 tablet_bootstrap.cc:492] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:37.834326 23073 ts_tablet_manager.cc:1403] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.081s	user 0.000s	sys 0.009s
I20260709 13:07:37.836448 23069 raft_consensus.cc:359] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.838441 23070 raft_consensus.cc:359] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.839093 23070 raft_consensus.cc:385] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:37.839325 23070 raft_consensus.cc:740] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:37.840320 23070 consensus_queue.cc:260] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.837124 23069 raft_consensus.cc:385] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:37.841477 23069 raft_consensus.cc:740] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:37.841531 23073 raft_consensus.cc:359] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.842264 23073 raft_consensus.cc:385] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:37.844339 23073 raft_consensus.cc:740] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:37.842311 23069 consensus_queue.cc:260] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.845152 23073 consensus_queue.cc:260] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.846369 23069 ts_tablet_manager.cc:1434] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:07:37.847180 23069 tablet_bootstrap.cc:492] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:37.847441 23073 ts_tablet_manager.cc:1434] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:07:37.848308 23073 tablet_bootstrap.cc:492] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:37.844067 23070 ts_tablet_manager.cc:1434] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.031s	user 0.005s	sys 0.007s
I20260709 13:07:37.850234 23070 tablet_bootstrap.cc:492] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:37.860381 23069 tablet_bootstrap.cc:654] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:37.864971 23073 tablet_bootstrap.cc:654] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:37.864971 23070 tablet_bootstrap.cc:654] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:37.871855 23069 tablet_bootstrap.cc:492] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:37.873081 23069 ts_tablet_manager.cc:1403] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.026s	user 0.010s	sys 0.007s
I20260709 13:07:37.874500 23073 tablet_bootstrap.cc:492] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:37.874843 23070 tablet_bootstrap.cc:492] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:37.874935 23073 ts_tablet_manager.cc:1403] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.027s	user 0.005s	sys 0.006s
I20260709 13:07:37.875358 23070 ts_tablet_manager.cc:1403] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.025s	user 0.003s	sys 0.007s
I20260709 13:07:37.875085 23069 raft_consensus.cc:359] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.875779 23069 raft_consensus.cc:385] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:37.876010 23069 raft_consensus.cc:740] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:37.876709 23069 consensus_queue.cc:260] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.877671 23073 raft_consensus.cc:359] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.877985 23070 raft_consensus.cc:359] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.878458 23073 raft_consensus.cc:385] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:37.878616 23070 raft_consensus.cc:385] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:37.878752 23073 raft_consensus.cc:740] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:37.879197 23070 raft_consensus.cc:740] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:37.879220 23069 ts_tablet_manager.cc:1434] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.001s
I20260709 13:07:37.879704 23073 consensus_queue.cc:260] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.880781 23069 tablet_bootstrap.cc:492] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:37.881870 23073 ts_tablet_manager.cc:1434] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:07:37.880048 23070 consensus_queue.cc:260] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.884001 23070 ts_tablet_manager.cc:1434] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 13:07:37.884927 23073 tablet_bootstrap.cc:492] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:37.887356 23069 tablet_bootstrap.cc:654] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:37.887633 23070 tablet_bootstrap.cc:492] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:37.888530 23183 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.889022 23183 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.891166 23183 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:37.891587 23183 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.892016 23183 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.893622 23070 tablet_bootstrap.cc:654] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:37.898095 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:37.900955 23073 tablet_bootstrap.cc:654] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:37.901962 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:37.902621 22879 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 0.
W20260709 13:07:37.906879 22918 leader_election.cc:343] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:37.910239 23073 tablet_bootstrap.cc:492] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:37.910703 23073 ts_tablet_manager.cc:1403] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.026s	user 0.011s	sys 0.000s
I20260709 13:07:37.912451 22920 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:37.913370 23073 raft_consensus.cc:359] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.914119 23117 raft_consensus.cc:2804] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:37.914453 23073 raft_consensus.cc:385] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:37.914537 23117 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.914798 23073 raft_consensus.cc:740] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:37.914958 23117 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.916533 23073 consensus_queue.cc:260] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.918535 23073 ts_tablet_manager.cc:1434] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:07:37.920770 23069 tablet_bootstrap.cc:492] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:37.919405 23073 tablet_bootstrap.cc:492] T 91c88873d54149c58a20f4cf6194e5fc P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:37.921799 23117 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.923405 23069 ts_tablet_manager.cc:1403] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.043s	user 0.023s	sys 0.011s
I20260709 13:07:37.925873 23117 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:37.926164 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:07:37.926741 23069 raft_consensus.cc:359] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.927520 23069 raft_consensus.cc:385] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:07:37.927623 22918 leader_election.cc:343] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:37.927891 23069 raft_consensus.cc:740] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:37.928656 23069 consensus_queue.cc:260] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.931882 23073 tablet_bootstrap.cc:654] T 91c88873d54149c58a20f4cf6194e5fc P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:37.934247 23069 ts_tablet_manager.cc:1434] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:07:37.935444 23069 tablet_bootstrap.cc:492] T 8217046fd4e14cc4a0194da4f6ad3887 P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:37.942413 23069 tablet_bootstrap.cc:654] T 8217046fd4e14cc4a0194da4f6ad3887 P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:37.942811 23073 tablet_bootstrap.cc:492] T 91c88873d54149c58a20f4cf6194e5fc P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:37.943393 23073 ts_tablet_manager.cc:1403] T 91c88873d54149c58a20f4cf6194e5fc P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.024s	user 0.010s	sys 0.007s
I20260709 13:07:37.945981 23073 raft_consensus.cc:359] T 91c88873d54149c58a20f4cf6194e5fc P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.946710 23073 raft_consensus.cc:385] T 91c88873d54149c58a20f4cf6194e5fc P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:37.947039 23073 raft_consensus.cc:740] T 91c88873d54149c58a20f4cf6194e5fc P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:37.947734 23073 consensus_queue.cc:260] T 91c88873d54149c58a20f4cf6194e5fc P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.932816 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:07:37.948865 22879 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.950241 23073 ts_tablet_manager.cc:1434] T 91c88873d54149c58a20f4cf6194e5fc P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:07:37.952013 23183 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:37.953410 23073 tablet_bootstrap.cc:492] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:37.953413 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:07:37.954913 22918 leader_election.cc:343] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:37.956434 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:37.957268 22878 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 0.
I20260709 13:07:37.958595 22920 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:37.959975 22879 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 1.
I20260709 13:07:37.960096 23183 raft_consensus.cc:2804] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:37.960779 23183 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.961282 23183 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.966496 22920 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:37.968873 23117 raft_consensus.cc:2804] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:37.969183 23070 tablet_bootstrap.cc:492] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:37.969695 23070 ts_tablet_manager.cc:1403] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.082s	user 0.019s	sys 0.021s
I20260709 13:07:37.969527 23117 raft_consensus.cc:697] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:07:37.970788 23117 consensus_queue.cc:237] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.972513 23070 raft_consensus.cc:359] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.973157 23070 raft_consensus.cc:385] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:37.973408 23070 raft_consensus.cc:740] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:37.973973 23070 consensus_queue.cc:260] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.983238 23191 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:37.983754 23191 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:37.984397 23070 ts_tablet_manager.cc:1434] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:07:37.985334 23070 tablet_bootstrap.cc:492] T cdebaf291e5d4ae9814a80ce81ee19fe P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:37.985739 23191 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:37.987439 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:37.987627 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:37.988189 23029 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 0.
I20260709 13:07:37.988353 22954 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 0.
I20260709 13:07:37.989571 23073 tablet_bootstrap.cc:654] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:37.989449 22843 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:37.990406 23201 raft_consensus.cc:2804] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:37.990753 23201 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:37.991076 23201 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:37.991999 23070 tablet_bootstrap.cc:654] T cdebaf291e5d4ae9814a80ce81ee19fe P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:37.994840 23075 consensus_queue.cc:1048] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:37.997341 23070 tablet_bootstrap.cc:492] T cdebaf291e5d4ae9814a80ce81ee19fe P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:37.997867 23070 ts_tablet_manager.cc:1403] T cdebaf291e5d4ae9814a80ce81ee19fe P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.013s	user 0.007s	sys 0.003s
I20260709 13:07:37.998337 23201 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.000849 23070 raft_consensus.cc:359] T cdebaf291e5d4ae9814a80ce81ee19fe P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.001533 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:38.002202 23070 raft_consensus.cc:385] T cdebaf291e5d4ae9814a80ce81ee19fe P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:38.002444 22954 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:38.002568 23070 raft_consensus.cc:740] T cdebaf291e5d4ae9814a80ce81ee19fe P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:38.003587 23070 consensus_queue.cc:260] T cdebaf291e5d4ae9814a80ce81ee19fe P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.006369 23070 ts_tablet_manager.cc:1434] T cdebaf291e5d4ae9814a80ce81ee19fe P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:07:38.007426 23070 tablet_bootstrap.cc:492] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:38.008623 22777 catalog_manager.cc:5697] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 1 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:38.013360 23070 tablet_bootstrap.cc:654] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:38.013906 23192 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.014420 23183 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.014461 23192 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.018162 23191 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.018652 23191 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.020169 22954 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 1.
I20260709 13:07:38.020026 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:07:38.020828 23029 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:38.021015 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:38.021926 22953 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 0.
I20260709 13:07:38.021109 23191 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:38.022886 23192 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:38.025553 22992 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:38.025921 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:38.026916 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:07:38.027590 22843 leader_election.cc:343] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:38.029278 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:38.030043 22953 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 0.
I20260709 13:07:38.035801 22843 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
W20260709 13:07:38.036064 22994 leader_election.cc:343] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:38.036847 23191 raft_consensus.cc:2804] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:38.037315 23191 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:38.037889 23191 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:38.037307 23192 raft_consensus.cc:2804] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:38.041447 23192 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:38.042098 23201 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:38.043342 23029 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 1.
I20260709 13:07:38.045527 22843 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:07:38.046660 23201 raft_consensus.cc:2804] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:38.041904 23192 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:38.047415 23201 raft_consensus.cc:697] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:07:38.048465 23201 consensus_queue.cc:237] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.055452 23070 tablet_bootstrap.cc:492] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:38.055783 23191 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:07:38.060572 22918 consensus_peers.cc:597] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Couldn't send request to peer f3c799b1249e40119d2a5864231c6c75. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:38.055995 23070 ts_tablet_manager.cc:1403] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.049s	user 0.017s	sys 0.028s
I20260709 13:07:38.061393 23189 consensus_queue.cc:1048] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:07:38.063431 23207 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.071548 23070 raft_consensus.cc:359] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.071411 23207 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.073812 23207 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:38.074455 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:38.074990 23191 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:38.075037 22878 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 0.
I20260709 13:07:38.075623 23191 consensus_queue.cc:1048] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:38.076436 22994 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:38.077275 23070 raft_consensus.cc:385] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:38.077617 23207 raft_consensus.cc:2804] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:38.078008 23207 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:38.077911 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:07:38.078311 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:38.077601 23070 raft_consensus.cc:740] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:38.078922 22951 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:38.077864 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:07:38.079475 22843 leader_election.cc:343] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:38.079704 22953 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 0.
I20260709 13:07:38.079425 23070 consensus_queue.cc:260] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.081763 23070 ts_tablet_manager.cc:1434] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.002s
I20260709 13:07:38.082476 23069 tablet_bootstrap.cc:492] T 8217046fd4e14cc4a0194da4f6ad3887 P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:38.082706 23070 tablet_bootstrap.cc:492] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:38.082958 23069 ts_tablet_manager.cc:1403] T 8217046fd4e14cc4a0194da4f6ad3887 P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.148s	user 0.035s	sys 0.036s
I20260709 13:07:38.085767 23069 raft_consensus.cc:359] T 8217046fd4e14cc4a0194da4f6ad3887 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.086721 23069 raft_consensus.cc:385] T 8217046fd4e14cc4a0194da4f6ad3887 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:38.087004 23069 raft_consensus.cc:740] T 8217046fd4e14cc4a0194da4f6ad3887 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:38.087764 23069 consensus_queue.cc:260] T 8217046fd4e14cc4a0194da4f6ad3887 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.089749 23070 tablet_bootstrap.cc:654] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:38.090372 23069 ts_tablet_manager.cc:1434] T 8217046fd4e14cc4a0194da4f6ad3887 P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:07:38.091346 23069 tablet_bootstrap.cc:492] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:38.078485 23207 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:38.096418 23189 consensus_queue.cc:1048] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:38.100104 23125 consensus_queue.cc:1048] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:38.136188 23192 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.138449 23190 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.140255 23069 tablet_bootstrap.cc:654] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:38.157533 23183 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:38.158123 23207 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.159132 23206 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.160238 23207 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:38.159772 23206 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.149796 23192 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:38.161656 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:07:38.162017 23206 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:38.147622 23189 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.143277 23190 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.163417 23189 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.164568 23190 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:38.165341 23189 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:38.166837 23070 tablet_bootstrap.cc:492] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:38.167008 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:07:38.167287 23070 ts_tablet_manager.cc:1403] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.085s	user 0.020s	sys 0.035s
I20260709 13:07:38.167534 23189 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.167590 22879 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:38.168004 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:07:38.168053 23189 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.174065 23070 raft_consensus.cc:359] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.174654 23070 raft_consensus.cc:385] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:38.174898 23070 raft_consensus.cc:740] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:38.174816 22778 catalog_manager.cc:5697] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:38.176357 23070 consensus_queue.cc:260] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.179484 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:07:38.179550 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:38.179925 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:38.180259 22950 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:38.180912 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:38.181046 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:38.181281 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:38.180140 22880 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:38.181855 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:07:38.187593 22920 leader_election.cc:343] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:38.193034 22949 raft_consensus.cc:2393] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 1: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:07:38.191838 22955 raft_consensus.cc:2393] T 392f5a2324d74a499e6baad745c941fd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 1: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:07:38.192016 22952 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:38.194154 23073 tablet_bootstrap.cc:492] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:38.194597 23073 ts_tablet_manager.cc:1403] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.241s	user 0.010s	sys 0.032s
I20260709 13:07:38.194808 23070 ts_tablet_manager.cc:1434] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.001s
W20260709 13:07:38.182551 22994 leader_election.cc:343] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:38.198159 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:07:38.198710 22920 leader_election.cc:343] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:07:38.199429 22918 leader_election.cc:343] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:07:38.200342 22918 leader_election.cc:343] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:38.200995 23070 tablet_bootstrap.cc:492] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:38.200995 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:38.177706 23189 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:38.178352 22951 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 1.
I20260709 13:07:38.214248 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:38.215081 22877 raft_consensus.cc:2393] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 1: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:07:38.214921 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:38.216389 22843 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:38.217684 23191 raft_consensus.cc:2804] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:38.218271 23191 raft_consensus.cc:697] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:07:38.207232 22918 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:38.219048 23191 consensus_queue.cc:237] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.220057 23117 raft_consensus.cc:2749] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:07:38.220439 22918 leader_election.cc:343] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:38.220908 22918 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:38.224151 23075 raft_consensus.cc:2749] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:38.224296 22952 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:07:38.209539 22879 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 1.
I20260709 13:07:38.230831 23073 raft_consensus.cc:359] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.235169 23073 raft_consensus.cc:385] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:38.235736 23073 raft_consensus.cc:740] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:38.236404 23073 consensus_queue.cc:260] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.239535 23073 ts_tablet_manager.cc:1434] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.045s	user 0.008s	sys 0.000s
I20260709 13:07:38.240471 23073 tablet_bootstrap.cc:492] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:38.240334 22880 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:07:38.240845 22920 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:38.241704 23183 raft_consensus.cc:2804] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:38.242327 23183 raft_consensus.cc:697] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:07:38.243327 23183 consensus_queue.cc:237] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.243198 22992 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:38.254858 23190 raft_consensus.cc:2804] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:38.255407 23190 raft_consensus.cc:697] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:07:38.256218 23190 consensus_queue.cc:237] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.257742 22994 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:07:38.315837 23192 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:38.326447 23207 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.327138 23207 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.365409 23207 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:38.373401 23189 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.365152 23117 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.374518 23069 tablet_bootstrap.cc:492] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:38.374974 23069 ts_tablet_manager.cc:1403] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.284s	user 0.032s	sys 0.006s
W20260709 13:07:38.316895 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 100608us
I20260709 13:07:38.380149 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:38.381017 22952 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 0.
I20260709 13:07:38.381145 23183 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.381690 23183 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:07:38.334045 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 84367.1us
I20260709 13:07:38.382344 22992 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:38.376116 23189 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.382222 23069 raft_consensus.cc:359] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.387183 23069 raft_consensus.cc:385] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:38.387880 23069 raft_consensus.cc:740] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:38.376415 23184 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.389925 23069 consensus_queue.cc:260] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.387192 23192 raft_consensus.cc:2749] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:38.377552 23117 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.392927 22994 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:07:38.389884 23184 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.398288 23117 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:38.398949 23184 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:38.400391 23189 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:38.390424 23207 raft_consensus.cc:2804] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:38.400830 23206 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:38.400954 23207 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:38.401310 23207 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:38.403091 23069 ts_tablet_manager.cc:1434] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.028s	user 0.003s	sys 0.005s
I20260709 13:07:38.405504 23069 tablet_bootstrap.cc:492] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:38.405778 22950 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:07:38.408234 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:38.408898 22880 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 0.
I20260709 13:07:38.409616 23070 tablet_bootstrap.cc:654] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:38.415176 22778 catalog_manager.cc:5697] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:38.423633 22992 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:07:38.427361 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:38.428781 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:07:38.429086 22826 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:38.436633 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:38.436637 23183 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:38.446087 23192 raft_consensus.cc:2804] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:38.455711 23073 tablet_bootstrap.cc:654] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:38.476423 23192 raft_consensus.cc:697] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:07:38.479024 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:38.503579 23070 tablet_bootstrap.cc:492] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:38.504034 23070 ts_tablet_manager.cc:1403] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.303s	user 0.011s	sys 0.011s
I20260709 13:07:38.501735 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:07:38.506749 23279 log.cc:927] Time spent T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
W20260709 13:07:38.500641 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 51724.4us
I20260709 13:07:38.504639 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:38.506004 23190 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.512917 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:38.506870 22875 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 0.
I20260709 13:07:38.514370 23274 consensus_queue.cc:1048] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:07:38.517791 22920 leader_election.cc:343] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:07:38.518118 22918 leader_election.cc:343] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:07:38.518680 22920 leader_election.cc:343] T cdebaf291e5d4ae9814a80ce81ee19fe P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:38.519367 22920 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
W20260709 13:07:38.519558 22918 leader_election.cc:343] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:38.520011 22918 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:38.520877 23075 raft_consensus.cc:2749] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:38.514999 23206 raft_consensus.cc:2749] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:38.520846 23094 raft_consensus.cc:2804] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:38.521871 23094 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:38.522215 23094 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:07:38.522640 22918 leader_election.cc:343] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:38.524895 23073 tablet_bootstrap.cc:492] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:38.516109 23190 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.527268 23073 ts_tablet_manager.cc:1403] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.287s	user 0.015s	sys 0.013s
I20260709 13:07:38.533002 23190 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:38.534441 23070 raft_consensus.cc:359] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.534724 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:38.535127 23070 raft_consensus.cc:385] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:38.534999 23094 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.535557 23070 raft_consensus.cc:740] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:38.536224 23070 consensus_queue.cc:260] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.537825 23094 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:38.537976 23278 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.538450 23278 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.539072 23070 ts_tablet_manager.cc:1434] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.035s	user 0.006s	sys 0.000s
I20260709 13:07:38.539847 23070 tablet_bootstrap.cc:492] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:38.537537 23192 consensus_queue.cc:237] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.542732 23278 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:38.541054 22777 catalog_manager.cc:5697] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:38.543803 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:07:38.546875 22920 leader_election.cc:343] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:38.547156 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:07:38.547322 22920 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:38.548092 23094 raft_consensus.cc:2749] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:38.551937 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:07:38.552835 22875 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:38.560303 22875 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 1.
I20260709 13:07:38.553493 23207 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.554530 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:38.559749 23073 raft_consensus.cc:359] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.563051 23073 raft_consensus.cc:385] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:38.563402 23073 raft_consensus.cc:740] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:38.565328 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:38.562673 23207 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:38.565883 22951 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:38.570086 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:38.570698 23274 consensus_queue.cc:1048] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:38.564081 23073 consensus_queue.cc:260] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.576474 23069 tablet_bootstrap.cc:654] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:38.579398 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:38.579698 23274 consensus_queue.cc:1048] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:07:38.580807 22918 leader_election.cc:343] T cdebaf291e5d4ae9814a80ce81ee19fe P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:38.581270 22918 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:38.582310 22951 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:07:38.582931 23183 raft_consensus.cc:2749] T cdebaf291e5d4ae9814a80ce81ee19fe P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:38.578647 22920 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:07:38.584161 22992 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:07:38.584764 22920 leader_election.cc:343] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:38.585026 23094 raft_consensus.cc:2804] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:38.585598 23190 raft_consensus.cc:2804] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:38.586083 23190 raft_consensus.cc:697] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:07:38.586918 23190 consensus_queue.cc:237] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:07:38.587803 22918 leader_election.cc:343] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:38.587818 22994 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:07:38.588946 23206 raft_consensus.cc:2749] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:38.589308 23070 tablet_bootstrap.cc:654] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:38.585618 23094 raft_consensus.cc:697] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:07:38.590751 23073 ts_tablet_manager.cc:1434] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.063s	user 0.004s	sys 0.004s
I20260709 13:07:38.590441 23094 consensus_queue.cc:237] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:07:38.591065 22918 leader_election.cc:343] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:38.591637 23073 tablet_bootstrap.cc:492] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:38.591526 22918 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:38.594962 23278 raft_consensus.cc:2749] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:38.584990 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:07:38.597086 22875 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:38.596619 22775 catalog_manager.cc:5697] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 1 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:38.612635 23274 consensus_queue.cc:1048] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:38.614051 23075 consensus_queue.cc:1048] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:07:38.618014 22918 consensus_peers.cc:597] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Couldn't send request to peer f3c799b1249e40119d2a5864231c6c75. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:38.683434 23029 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:38.684839 23073 tablet_bootstrap.cc:654] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:38.691885 22777 catalog_manager.cc:5697] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:38.695369 22777 catalog_manager.cc:5697] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:38.697228 23069 tablet_bootstrap.cc:492] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:38.697635 23069 ts_tablet_manager.cc:1403] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.292s	user 0.016s	sys 0.006s
I20260709 13:07:38.719529 22875 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:07:38.723481 23281 consensus_queue.cc:1048] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:07:38.727665 23291 log.cc:927] Time spent T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:07:38.729943 23274 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.730495 23274 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.735144 23274 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:38.736923 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:38.737617 22952 raft_consensus.cc:2393] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 1: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:07:38.741472 23069 raft_consensus.cc:359] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.742126 23069 raft_consensus.cc:385] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:38.742373 23069 raft_consensus.cc:740] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:38.743106 23069 consensus_queue.cc:260] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.745558 23069 ts_tablet_manager.cc:1434] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.048s	user 0.006s	sys 0.000s
I20260709 13:07:38.746385 23069 tablet_bootstrap.cc:492] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:38.752863 23073 tablet_bootstrap.cc:492] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:38.753417 23073 ts_tablet_manager.cc:1403] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.162s	user 0.014s	sys 0.004s
I20260709 13:07:38.752936 22775 catalog_manager.cc:5697] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 1 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:38.758122 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:38.760047 23070 tablet_bootstrap.cc:492] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:38.760440 23070 ts_tablet_manager.cc:1403] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.221s	user 0.016s	sys 0.008s
I20260709 13:07:38.764117 23070 raft_consensus.cc:359] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.764837 23070 raft_consensus.cc:385] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:38.765144 23070 raft_consensus.cc:740] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:38.765802 23070 consensus_queue.cc:260] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.774544 23201 consensus_queue.cc:1048] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:38.776384 22843 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:38.776859 23070 ts_tablet_manager.cc:1434] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.001s
I20260709 13:07:38.777184 23201 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:38.777762 23070 tablet_bootstrap.cc:492] T fab4388fa7ea49bda9fa070cf542577f P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:38.778784 23073 raft_consensus.cc:359] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.779438 23073 raft_consensus.cc:385] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:38.779731 23073 raft_consensus.cc:740] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:38.780479 23073 consensus_queue.cc:260] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.782238 23073 ts_tablet_manager.cc:1434] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.000s
I20260709 13:07:38.783104 23073 tablet_bootstrap.cc:492] T 579096ba88474cfbbe40e8bf769879ec P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:38.785310 23201 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:07:38.786545 22843 consensus_peers.cc:597] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f -> Peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Couldn't send request to peer f3c799b1249e40119d2a5864231c6c75. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:38.791996 23069 tablet_bootstrap.cc:654] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:38.805653 23073 tablet_bootstrap.cc:654] T 579096ba88474cfbbe40e8bf769879ec P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:38.811923 23192 consensus_queue.cc:1048] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:38.814958 23070 tablet_bootstrap.cc:654] T fab4388fa7ea49bda9fa070cf542577f P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:38.834960 23069 tablet_bootstrap.cc:492] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:38.839394 23069 ts_tablet_manager.cc:1403] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.093s	user 0.008s	sys 0.010s
I20260709 13:07:38.840350 23073 tablet_bootstrap.cc:492] T 579096ba88474cfbbe40e8bf769879ec P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:38.837358 23070 tablet_bootstrap.cc:492] T fab4388fa7ea49bda9fa070cf542577f P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:38.841320 23073 ts_tablet_manager.cc:1403] T 579096ba88474cfbbe40e8bf769879ec P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.058s	user 0.012s	sys 0.007s
I20260709 13:07:38.841413 23070 ts_tablet_manager.cc:1403] T fab4388fa7ea49bda9fa070cf542577f P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.064s	user 0.011s	sys 0.005s
I20260709 13:07:38.843874 23073 raft_consensus.cc:359] T 579096ba88474cfbbe40e8bf769879ec P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.844544 23073 raft_consensus.cc:385] T 579096ba88474cfbbe40e8bf769879ec P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:38.845011 23073 raft_consensus.cc:740] T 579096ba88474cfbbe40e8bf769879ec P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:38.845409 23069 raft_consensus.cc:359] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.846093 23069 raft_consensus.cc:385] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:38.845738 23073 consensus_queue.cc:260] T 579096ba88474cfbbe40e8bf769879ec P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.846764 23069 raft_consensus.cc:740] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:38.847564 23069 consensus_queue.cc:260] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.848503 23073 ts_tablet_manager.cc:1434] T 579096ba88474cfbbe40e8bf769879ec P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.002s
I20260709 13:07:38.844157 23070 raft_consensus.cc:359] T fab4388fa7ea49bda9fa070cf542577f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.849725 23070 raft_consensus.cc:385] T fab4388fa7ea49bda9fa070cf542577f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:38.849931 23069 ts_tablet_manager.cc:1434] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:07:38.850941 23069 tablet_bootstrap.cc:492] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:38.852882 23073 tablet_bootstrap.cc:492] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:38.850009 23070 raft_consensus.cc:740] T fab4388fa7ea49bda9fa070cf542577f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:38.853602 23070 consensus_queue.cc:260] T fab4388fa7ea49bda9fa070cf542577f P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.855619 23070 ts_tablet_manager.cc:1434] T fab4388fa7ea49bda9fa070cf542577f P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.001s
I20260709 13:07:38.856562 23070 tablet_bootstrap.cc:492] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:38.864279 23183 consensus_queue.cc:1048] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:38.866043 23069 tablet_bootstrap.cc:654] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:38.874698 23073 tablet_bootstrap.cc:654] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:38.876255 23021 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:38.880247 23070 tablet_bootstrap.cc:654] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:38.895412 23183 consensus_queue.cc:1048] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:38.896919 23070 tablet_bootstrap.cc:492] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:38.897559 23070 ts_tablet_manager.cc:1403] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.041s	user 0.008s	sys 0.006s
I20260709 13:07:38.902911 23070 raft_consensus.cc:359] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.903662 23070 raft_consensus.cc:385] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:38.904251 23281 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.904827 23281 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.904008 23070 raft_consensus.cc:740] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:38.906237 23070 consensus_queue.cc:260] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.908136 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:38.910513 23073 tablet_bootstrap.cc:492] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:38.910979 23073 ts_tablet_manager.cc:1403] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.058s	user 0.012s	sys 0.002s
I20260709 13:07:38.912490 23069 tablet_bootstrap.cc:492] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:38.912941 23069 ts_tablet_manager.cc:1403] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.062s	user 0.017s	sys 0.021s
I20260709 13:07:38.913916 23073 raft_consensus.cc:359] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.914583 23073 raft_consensus.cc:385] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:38.914844 23073 raft_consensus.cc:740] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:38.915474 23073 consensus_queue.cc:260] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.917362 23070 ts_tablet_manager.cc:1434] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.002s
I20260709 13:07:38.917516 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:38.918308 23070 tablet_bootstrap.cc:492] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:38.917666 23281 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:38.919607 23073 ts_tablet_manager.cc:1434] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:07:38.919548 22994 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:07:38.920881 23073 tablet_bootstrap.cc:492] T 1120807254f5444997e16f6a7feece4d P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:38.921027 23281 raft_consensus.cc:2749] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:38.916147 23069 raft_consensus.cc:359] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.921613 23069 raft_consensus.cc:385] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:38.921900 23069 raft_consensus.cc:740] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:38.922515 23069 consensus_queue.cc:260] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.924535 23069 ts_tablet_manager.cc:1434] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.011s	user 0.003s	sys 0.000s
I20260709 13:07:38.925868 23069 tablet_bootstrap.cc:492] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:38.930369 23070 tablet_bootstrap.cc:654] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:38.931591 23069 tablet_bootstrap.cc:654] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:38.935739 23073 tablet_bootstrap.cc:654] T 1120807254f5444997e16f6a7feece4d P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:38.941905 23070 tablet_bootstrap.cc:492] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:38.942353 23070 ts_tablet_manager.cc:1403] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.024s	user 0.009s	sys 0.000s
I20260709 13:07:38.944365 23069 tablet_bootstrap.cc:492] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:38.944815 23070 raft_consensus.cc:359] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.945574 23070 raft_consensus.cc:385] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:38.944818 23069 ts_tablet_manager.cc:1403] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.004s
I20260709 13:07:38.945848 23070 raft_consensus.cc:740] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:38.946527 23070 consensus_queue.cc:260] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.948122 23069 raft_consensus.cc:359] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.948531 23070 ts_tablet_manager.cc:1434] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:07:38.948778 23069 raft_consensus.cc:385] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:38.949079 23069 raft_consensus.cc:740] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:38.949643 23070 tablet_bootstrap.cc:492] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:38.949834 23069 consensus_queue.cc:260] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.952445 23073 tablet_bootstrap.cc:492] T 1120807254f5444997e16f6a7feece4d P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:38.952525 23069 ts_tablet_manager.cc:1434] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:07:38.952992 23073 ts_tablet_manager.cc:1403] T 1120807254f5444997e16f6a7feece4d P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.032s	user 0.006s	sys 0.006s
I20260709 13:07:38.954284 23069 tablet_bootstrap.cc:492] T 7ca8b9a08f4146c386bc06d03c497038 P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:38.955178 23073 raft_consensus.cc:359] T 1120807254f5444997e16f6a7feece4d P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.955806 23073 raft_consensus.cc:385] T 1120807254f5444997e16f6a7feece4d P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:38.956061 23073 raft_consensus.cc:740] T 1120807254f5444997e16f6a7feece4d P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:38.956712 23073 consensus_queue.cc:260] T 1120807254f5444997e16f6a7feece4d P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.958774 23073 ts_tablet_manager.cc:1434] T 1120807254f5444997e16f6a7feece4d P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:07:38.960021 23069 tablet_bootstrap.cc:654] T 7ca8b9a08f4146c386bc06d03c497038 P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:38.960697 23073 tablet_bootstrap.cc:492] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:38.963624 23070 tablet_bootstrap.cc:654] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:38.967196 23073 tablet_bootstrap.cc:654] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:38.974829 23070 tablet_bootstrap.cc:492] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:38.975288 23070 ts_tablet_manager.cc:1403] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.026s	user 0.007s	sys 0.008s
I20260709 13:07:38.977731 23070 raft_consensus.cc:359] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.979190 23070 raft_consensus.cc:385] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:38.979467 23070 raft_consensus.cc:740] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:38.980196 23070 consensus_queue.cc:260] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.983052 23070 ts_tablet_manager.cc:1434] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:07:38.983907 23070 tablet_bootstrap.cc:492] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:38.989642 23070 tablet_bootstrap.cc:654] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:38.993434 23073 tablet_bootstrap.cc:492] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:38.993878 23073 ts_tablet_manager.cc:1403] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.033s	user 0.010s	sys 0.010s
I20260709 13:07:38.995609 23191 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:38.996105 23191 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.996336 23073 raft_consensus.cc:359] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:38.997020 23073 raft_consensus.cc:385] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:38.997336 23073 raft_consensus.cc:740] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:38.998131 23191 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:38.998047 23073 consensus_queue.cc:260] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:39.000123 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:39.000788 22952 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 0.
I20260709 13:07:39.001994 22843 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:07:39.002344 23069 tablet_bootstrap.cc:492] T 7ca8b9a08f4146c386bc06d03c497038 P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:39.002345 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:39.002707 23201 raft_consensus.cc:2804] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:39.003036 23069 ts_tablet_manager.cc:1403] T 7ca8b9a08f4146c386bc06d03c497038 P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.049s	user 0.012s	sys 0.001s
I20260709 13:07:39.003100 23201 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:39.003516 23201 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.000862 23073 ts_tablet_manager.cc:1434] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:07:39.005906 23069 raft_consensus.cc:359] T 7ca8b9a08f4146c386bc06d03c497038 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:39.006464 23073 tablet_bootstrap.cc:492] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:39.006579 23069 raft_consensus.cc:385] T 7ca8b9a08f4146c386bc06d03c497038 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.006948 23069 raft_consensus.cc:740] T 7ca8b9a08f4146c386bc06d03c497038 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
W20260709 13:07:39.009155 22843 leader_election.cc:343] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.010748 23070 tablet_bootstrap.cc:492] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:39.011193 23070 ts_tablet_manager.cc:1403] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.027s	user 0.009s	sys 0.000s
I20260709 13:07:39.011794 23201 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:39.013329 23073 tablet_bootstrap.cc:654] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.013661 23070 raft_consensus.cc:359] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:39.014309 23070 raft_consensus.cc:385] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.014560 23070 raft_consensus.cc:740] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.015174 23070 consensus_queue.cc:260] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:39.016146 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:39.016824 22952 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.017237 23069 consensus_queue.cc:260] T 7ca8b9a08f4146c386bc06d03c497038 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:39.019233 23069 ts_tablet_manager.cc:1434] T 7ca8b9a08f4146c386bc06d03c497038 P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.000s
I20260709 13:07:39.020079 23069 tablet_bootstrap.cc:492] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:39.023367 23070 ts_tablet_manager.cc:1434] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:07:39.024312 23070 tablet_bootstrap.cc:492] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:39.028173 23073 tablet_bootstrap.cc:492] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:39.028728 23073 ts_tablet_manager.cc:1403] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.000s
I20260709 13:07:39.031559 23069 tablet_bootstrap.cc:654] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.031441 23073 raft_consensus.cc:359] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:39.032226 23073 raft_consensus.cc:385] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.032361 23278 consensus_queue.cc:1048] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:39.032613 23073 raft_consensus.cc:740] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.033571 23073 consensus_queue.cc:260] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:39.035660 23073 ts_tablet_manager.cc:1434] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:07:39.035804 23070 tablet_bootstrap.cc:654] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.036821 23073 tablet_bootstrap.cc:492] T a424231ef7b4470bbd575564909a365e P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:39.044572 23073 tablet_bootstrap.cc:654] T a424231ef7b4470bbd575564909a365e P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.050860 23281 consensus_queue.cc:1048] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:39.054901 23201 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:39.056406 23281 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.056813 23073 tablet_bootstrap.cc:492] T a424231ef7b4470bbd575564909a365e P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:39.034081 22952 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 1.
I20260709 13:07:39.056977 23281 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:39.057499 23073 ts_tablet_manager.cc:1403] T a424231ef7b4470bbd575564909a365e P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.021s	user 0.017s	sys 0.002s
I20260709 13:07:39.060319 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:39.060441 23281 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:39.060616 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:39.060979 22877 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 0.
I20260709 13:07:39.061296 22952 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 0.
I20260709 13:07:39.062387 22992 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:39.063516 23281 raft_consensus.cc:2804] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:39.063858 23281 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:39.064188 23281 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.064047 23073 raft_consensus.cc:359] T a424231ef7b4470bbd575564909a365e P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:39.065237 23073 raft_consensus.cc:385] T a424231ef7b4470bbd575564909a365e P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.065665 23073 raft_consensus.cc:740] T a424231ef7b4470bbd575564909a365e P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.066538 23073 consensus_queue.cc:260] T a424231ef7b4470bbd575564909a365e P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:39.067647 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:07:39.068922 22843 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:07:39.069397 23073 ts_tablet_manager.cc:1434] T a424231ef7b4470bbd575564909a365e P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:07:39.070582 23073 tablet_bootstrap.cc:492] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:39.077840 23073 tablet_bootstrap.cc:654] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
W20260709 13:07:39.080202 22843 leader_election.cc:343] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.081297 23192 consensus_queue.cc:1048] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:39.084817 23191 raft_consensus.cc:2804] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:39.085602 23191 raft_consensus.cc:697] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:07:39.072295 23281 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:39.086680 23191 consensus_queue.cc:237] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:39.089537 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:39.090114 22952 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.092860 23075 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.093308 23075 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:39.095887 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:39.096554 23075 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:07:39.097215 22920 leader_election.cc:343] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.098740 23021 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.101034 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:39.102034 23281 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:07:39.102233 22918 leader_election.cc:343] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.102650 22918 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:39.103969 23183 raft_consensus.cc:2749] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:39.106590 22952 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:07:39.107741 22992 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:39.108582 23281 raft_consensus.cc:2804] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:39.109051 23281 raft_consensus.cc:697] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:07:39.110374 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:07:39.110931 22875 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.114104 23075 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.114321 23274 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.114605 23075 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:39.114738 23274 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:39.117100 23274 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:39.119071 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:07:39.120481 22920 leader_election.cc:343] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.123027 23192 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.123502 23192 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:39.126106 23192 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:39.130173 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:39.130801 22951 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 0.
I20260709 13:07:39.133723 22992 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:39.135771 23190 raft_consensus.cc:2804] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:39.136128 23190 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:39.136469 23190 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.109900 23281 consensus_queue.cc:237] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:39.145507 22875 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:07:39.147938 23274 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.148098 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:39.148424 23274 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:39.150578 23274 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:07:39.154181 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 73999.7us
I20260709 13:07:39.156651 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:39.158952 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:39.159525 23075 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:39.159530 22952 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 0.
W20260709 13:07:39.165565 22843 leader_election.cc:343] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.166261 22843 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:07:39.158965 23190 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:39.171124 23125 raft_consensus.cc:2804] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:39.171509 23125 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:39.171861 23125 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.172114 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:39.173623 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:39.174182 23026 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 0.
W20260709 13:07:39.176733 22994 leader_election.cc:343] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.179433 23075 consensus_queue.cc:1048] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:39.179888 23125 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:39.183473 23183 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.183966 23183 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:39.184234 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:39.184983 23021 raft_consensus.cc:2393] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 1: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:07:39.185971 23183 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:39.188886 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:07:39.190200 22920 leader_election.cc:343] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.191850 23125 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:39.193917 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:39.194463 22952 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.196627 22777 catalog_manager.cc:5697] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:39.201997 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:39.202668 22951 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.209331 22951 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:07:39.210732 22992 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:39.211732 23190 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:39.212965 22776 catalog_manager.cc:5697] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:39.215945 22952 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 1.
I20260709 13:07:39.216411 23207 consensus_queue.cc:1048] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:39.217365 22843 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:07:39.218520 23125 raft_consensus.cc:2804] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:39.218997 23125 raft_consensus.cc:697] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:07:39.220335 23192 raft_consensus.cc:2804] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:39.221092 23192 raft_consensus.cc:697] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:07:39.223362 23192 consensus_queue.cc:237] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:39.222254 23190 consensus_queue.cc:1048] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:39.219872 23125 consensus_queue.cc:237] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:39.240401 23075 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.240965 23075 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:39.243659 23075 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:39.243866 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:39.245178 23281 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:39.245334 22920 leader_election.cc:343] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.245728 23281 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:39.247756 23281 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:39.248816 23191 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.249353 23191 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:07:39.250123 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 72499.6us
I20260709 13:07:39.251588 23191 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:39.252297 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:07:39.255918 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:39.256604 23191 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.257264 23075 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.257115 23191 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:39.257808 23075 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:39.259199 23191 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:39.261497 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:07:39.262825 22920 leader_election.cc:343] T fab4388fa7ea49bda9fa070cf542577f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.264575 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:39.263686 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:39.265791 22955 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 1.
I20260709 13:07:39.267439 22843 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:07:39.270678 23191 raft_consensus.cc:2804] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:39.271056 23191 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:39.271396 23191 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:39.271080 23281 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.273844 23281 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:39.275915 23281 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:39.278327 23075 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:39.282461 22950 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 0.
I20260709 13:07:39.283888 22843 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:07:39.285010 23201 raft_consensus.cc:2804] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:39.285440 23201 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:39.285851 23201 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:07:39.289888 22843 leader_election.cc:343] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.290460 22843 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:39.291349 23274 raft_consensus.cc:2749] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:39.293085 22918 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:39.296419 23075 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:39.300304 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:39.301534 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:39.304116 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:07:39.304411 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 50299us
I20260709 13:07:39.312156 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:39.314762 22955 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 0.
I20260709 13:07:39.326284 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:07:39.357371 22994 leader_election.cc:343] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.358897 22994 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:07:39.327270 23026 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 0.
I20260709 13:07:39.371417 22992 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:39.371937 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:39.363433 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:07:39.377578 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:07:39.365402 22918 leader_election.cc:343] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.392726 23075 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.389921 23070 tablet_bootstrap.cc:492] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:39.400306 23070 ts_tablet_manager.cc:1403] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.376s	user 0.046s	sys 0.107s
W20260709 13:07:39.389999 22994 leader_election.cc:343] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.399854 23075 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:39.380589 23024 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.389014 23281 raft_consensus.cc:2804] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:39.373876 23191 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:39.412087 23075 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:39.387212 23025 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.393368 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:39.418753 23069 tablet_bootstrap.cc:492] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:39.387960 23022 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 0.
I20260709 13:07:39.419387 23069 ts_tablet_manager.cc:1403] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.399s	user 0.055s	sys 0.089s
I20260709 13:07:39.415946 23281 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:39.424721 23073 tablet_bootstrap.cc:492] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:39.388108 22918 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:39.419859 23028 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 1.
I20260709 13:07:39.427790 23073 ts_tablet_manager.cc:1403] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.357s	user 0.086s	sys 0.131s
I20260709 13:07:39.430626 23070 raft_consensus.cc:359] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:39.431479 23070 raft_consensus.cc:385] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.431788 23070 raft_consensus.cc:740] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.432538 23070 consensus_queue.cc:260] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:39.428755 23281 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.449131 23070 ts_tablet_manager.cc:1434] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.048s	user 0.007s	sys 0.000s
I20260709 13:07:39.523430 23075 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.544847 23075 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:39.545493 23070 tablet_bootstrap.cc:492] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
W20260709 13:07:39.508261 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 106719us
I20260709 13:07:39.492063 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:39.509037 23330 raft_consensus.cc:2749] T 579096ba88474cfbbe40e8bf769879ec P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:07:39.445616 23183 raft_consensus.cc:2749] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:07:39.553570 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 189497us
W20260709 13:07:39.495275 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 90766.6us
I20260709 13:07:39.559108 23069 raft_consensus.cc:359] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:39.559886 23069 raft_consensus.cc:385] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.560185 23069 raft_consensus.cc:740] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.560967 23069 consensus_queue.cc:260] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:39.576951 23069 ts_tablet_manager.cc:1434] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.157s	user 0.012s	sys 0.006s
I20260709 13:07:39.577517 23070 tablet_bootstrap.cc:654] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.578500 23069 tablet_bootstrap.cc:492] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:39.581270 23075 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:39.583925 23073 raft_consensus.cc:359] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:39.584766 23073 raft_consensus.cc:385] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.585083 23073 raft_consensus.cc:740] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.585886 23073 consensus_queue.cc:260] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:39.523506 23201 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:07:39.601176 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 46625.3us
W20260709 13:07:39.696980 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 105560us
I20260709 13:07:39.701078 23073 ts_tablet_manager.cc:1434] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.272s	user 0.007s	sys 0.002s
W20260709 13:07:39.698849 22918 leader_election.cc:343] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.698668 23191 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:39.726900 23070 tablet_bootstrap.cc:492] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:39.727375 23070 ts_tablet_manager.cc:1403] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.182s	user 0.031s	sys 0.031s
I20260709 13:07:39.730019 23070 raft_consensus.cc:359] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:39.730681 23070 raft_consensus.cc:385] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:39.730789 23025 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 1.
I20260709 13:07:39.732445 23201 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:39.730964 23070 raft_consensus.cc:740] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:39.710078 22918 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:39.739418 23278 raft_consensus.cc:2749] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:39.740147 22918 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:07:39.735522 23070 consensus_queue.cc:260] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:39.741912 23075 raft_consensus.cc:2804] T fab4388fa7ea49bda9fa070cf542577f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:39.742413 23075 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:39.742758 23075 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.749122 23070 ts_tablet_manager.cc:1434] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.021s	user 0.004s	sys 0.002s
I20260709 13:07:39.735622 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:39.752131 23070 tablet_bootstrap.cc:492] T 4e689c91ad7343b5b6ba36fa648a06ac P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
W20260709 13:07:39.756623 22920 leader_election.cc:343] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.759135 23075 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:39.756124 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:39.743752 23339 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.766706 23339 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:39.775003 23075 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:39.761406 22951 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:39.805028 23069 tablet_bootstrap.cc:654] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:39.805056 23339 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:07:39.839816 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 76787.4us
I20260709 13:07:39.849090 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:39.853493 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:39.855007 23021 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:07:39.871848 22918 leader_election.cc:343] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.860514 23201 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:39.872812 22918 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:39.873613 23201 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:07:39.876441 22920 leader_election.cc:343] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.880911 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:39.885102 23201 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:39.887195 23075 raft_consensus.cc:2749] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:39.893491 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:07:39.896894 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:07:39.897954 23281 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:39.908067 23281 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:39.916328 23028 raft_consensus.cc:2393] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 1: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:07:39.938705 23073 tablet_bootstrap.cc:492] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
W20260709 13:07:39.942884 22918 leader_election.cc:343] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:39.943369 22918 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:39.945201 23278 raft_consensus.cc:2749] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:39.950835 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:07:39.951620 22826 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:39.953269 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:39.953995 22954 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.960798 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:39.966439 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:39.967410 22955 raft_consensus.cc:2393] T fab4388fa7ea49bda9fa070cf542577f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 1: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:07:39.971805 22954 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 1.
I20260709 13:07:39.975477 23207 consensus_queue.cc:1048] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:39.978539 23192 consensus_queue.cc:1048] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 13:07:39.981312 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:07:39.982062 23027 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:39.991133 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:39.993713 22955 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 0.
I20260709 13:07:39.993894 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:07:39.994597 23025 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:39.997835 23027 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
W20260709 13:07:40.001859 23359 log.cc:927] Time spent T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:07:40.006029 23070 tablet_bootstrap.cc:654] T 4e689c91ad7343b5b6ba36fa648a06ac P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:40.015148 23025 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 1.
I20260709 13:07:40.017953 23069 tablet_bootstrap.cc:492] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:40.018468 23069 ts_tablet_manager.cc:1403] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.440s	user 0.010s	sys 0.014s
I20260709 13:07:40.020961 23073 tablet_bootstrap.cc:654] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:40.022030 23191 consensus_queue.cc:1048] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:07:40.023489 23353 consensus_queue.cc:1048] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:40.024005 22994 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:07:40.025852 23281 raft_consensus.cc:2749] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:07:40.027632 22994 consensus_peers.cc:597] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:07:40.028687 22920 leader_election.cc:343] T fab4388fa7ea49bda9fa070cf542577f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:40.029129 22920 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:40.029863 23350 consensus_queue.cc:1048] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:40.031476 23094 raft_consensus.cc:2749] T fab4388fa7ea49bda9fa070cf542577f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:07:40.033806 23069 raft_consensus.cc:359] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.022152 22843 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:07:40.035880 23069 raft_consensus.cc:385] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:40.037770 22777 catalog_manager.cc:5697] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:40.044688 22994 leader_election.cc:343] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:40.045225 22994 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:07:40.046037 23281 raft_consensus.cc:2749] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:07:40.040587 23069 raft_consensus.cc:740] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:40.057046 23353 raft_consensus.cc:2804] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:40.057752 23353 raft_consensus.cc:697] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:07:40.036173 22951 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:07:40.062464 22843 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:07:40.063377 23191 raft_consensus.cc:2804] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:40.063782 23191 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:40.064172 23191 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:40.053689 23069 consensus_queue.cc:260] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.058966 23353 consensus_queue.cc:237] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.075938 23069 ts_tablet_manager.cc:1434] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.057s	user 0.008s	sys 0.000s
I20260709 13:07:40.081736 23070 tablet_bootstrap.cc:492] T 4e689c91ad7343b5b6ba36fa648a06ac P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:40.085807 23070 ts_tablet_manager.cc:1403] T 4e689c91ad7343b5b6ba36fa648a06ac P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.334s	user 0.024s	sys 0.005s
I20260709 13:07:40.092063 23069 tablet_bootstrap.cc:492] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:40.090368 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:40.097586 23183 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.098189 23183 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.107636 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:40.113620 23183 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:40.114321 23073 tablet_bootstrap.cc:492] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:40.114861 23073 ts_tablet_manager.cc:1403] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.177s	user 0.022s	sys 0.013s
I20260709 13:07:40.118758 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:07:40.116144 22920 leader_election.cc:343] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:40.122792 23191 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.123766 23070 raft_consensus.cc:359] T 4e689c91ad7343b5b6ba36fa648a06ac P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.093709 22843 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:40.125607 23073 raft_consensus.cc:359] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.125967 23191 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:40.126302 23073 raft_consensus.cc:385] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:40.126698 23073 raft_consensus.cc:740] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
W20260709 13:07:40.127159 22843 consensus_peers.cc:597] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f -> Peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Couldn't send request to peer f3c799b1249e40119d2a5864231c6c75. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:40.127185 23201 consensus_queue.cc:1048] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:40.127317 23073 consensus_queue.cc:260] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.128121 23191 raft_consensus.cc:2804] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:40.128633 23191 raft_consensus.cc:697] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:07:40.129907 23191 consensus_queue.cc:237] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.134099 23073 ts_tablet_manager.cc:1434] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.002s
I20260709 13:07:40.124473 23070 raft_consensus.cc:385] T 4e689c91ad7343b5b6ba36fa648a06ac P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:40.135936 23070 raft_consensus.cc:740] T 4e689c91ad7343b5b6ba36fa648a06ac P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:40.136307 23073 tablet_bootstrap.cc:492] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:40.137063 23070 consensus_queue.cc:260] T 4e689c91ad7343b5b6ba36fa648a06ac P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:07:40.138769 22843 leader_election.cc:343] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:40.135887 23069 tablet_bootstrap.cc:654] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
W20260709 13:07:40.141811 22918 leader_election.cc:343] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:40.142252 22918 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:40.143589 23094 raft_consensus.cc:2749] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:40.143805 23070 ts_tablet_manager.cc:1434] T 4e689c91ad7343b5b6ba36fa648a06ac P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.058s	user 0.007s	sys 0.001s
I20260709 13:07:40.144194 23183 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.145234 23070 tablet_bootstrap.cc:492] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:40.144712 23183 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.151449 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:07:40.150421 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:40.157497 22951 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:40.165918 23183 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:40.169370 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:40.169559 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:07:40.170656 22918 leader_election.cc:343] T 4e689c91ad7343b5b6ba36fa648a06ac P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:07:40.171208 22920 leader_election.cc:343] T 4e689c91ad7343b5b6ba36fa648a06ac P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:40.171595 22920 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:40.172169 23183 raft_consensus.cc:2749] T 4e689c91ad7343b5b6ba36fa648a06ac P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:07:40.175860 22843 leader_election.cc:343] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:40.186334 23070 tablet_bootstrap.cc:654] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:40.193207 23073 tablet_bootstrap.cc:654] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:40.218525 22951 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 1.
I20260709 13:07:40.231487 22843 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:40.239219 23073 tablet_bootstrap.cc:492] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:40.239723 23073 ts_tablet_manager.cc:1403] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.104s	user 0.019s	sys 0.021s
I20260709 13:07:40.240657 23353 raft_consensus.cc:2804] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:40.241166 23353 raft_consensus.cc:697] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:07:40.241664 23070 tablet_bootstrap.cc:492] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:40.242094 23070 ts_tablet_manager.cc:1403] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.097s	user 0.017s	sys 0.000s
I20260709 13:07:40.242007 23353 consensus_queue.cc:237] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.246932 23073 raft_consensus.cc:359] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.247620 23073 raft_consensus.cc:385] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:40.247910 23073 raft_consensus.cc:740] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:40.248581 23073 consensus_queue.cc:260] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.248721 23070 raft_consensus.cc:359] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.249470 23070 raft_consensus.cc:385] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:40.249778 23070 raft_consensus.cc:740] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:40.250510 23070 consensus_queue.cc:260] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.251791 23070 ts_tablet_manager.cc:1434] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.001s
I20260709 13:07:40.253244 23070 tablet_bootstrap.cc:492] T e3b39c18a21f448783cdd998acfb2f80 P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:40.255470 22776 catalog_manager.cc:5697] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:40.258574 23070 tablet_bootstrap.cc:654] T e3b39c18a21f448783cdd998acfb2f80 P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:40.258659 23069 tablet_bootstrap.cc:492] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:40.259153 23069 ts_tablet_manager.cc:1403] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.168s	user 0.015s	sys 0.029s
I20260709 13:07:40.265552 23070 tablet_bootstrap.cc:492] T e3b39c18a21f448783cdd998acfb2f80 P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:40.265966 23070 ts_tablet_manager.cc:1403] T e3b39c18a21f448783cdd998acfb2f80 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 13:07:40.266882 23337 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.267313 23337 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.269526 23337 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:40.269685 23070 raft_consensus.cc:359] T e3b39c18a21f448783cdd998acfb2f80 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.270332 23070 raft_consensus.cc:385] T e3b39c18a21f448783cdd998acfb2f80 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:40.270565 23070 raft_consensus.cc:740] T e3b39c18a21f448783cdd998acfb2f80 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:40.271198 23070 consensus_queue.cc:260] T e3b39c18a21f448783cdd998acfb2f80 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.272620 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:40.273288 23070 ts_tablet_manager.cc:1434] T e3b39c18a21f448783cdd998acfb2f80 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:07:40.273241 22951 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 0.
I20260709 13:07:40.273325 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:40.274708 23070 tablet_bootstrap.cc:492] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
W20260709 13:07:40.274875 22994 leader_election.cc:343] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:40.275555 22777 catalog_manager.cc:5697] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:40.276759 22992 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:07:40.277158 22777 catalog_manager.cc:5697] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:40.278103 23191 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.278460 23337 raft_consensus.cc:2804] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:40.278932 23337 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:40.279296 23337 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:40.279047 22777 catalog_manager.cc:5697] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 0 to 2, leader changed from <none> to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 2 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:40.278530 23191 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.282191 23191 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:40.283519 23073 ts_tablet_manager.cc:1434] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.044s	user 0.007s	sys 0.000s
I20260709 13:07:40.283766 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:40.284430 22952 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 0.
I20260709 13:07:40.284607 23073 tablet_bootstrap.cc:492] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:40.285848 23070 tablet_bootstrap.cc:654] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:40.287295 22843 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:07:40.288118 23353 raft_consensus.cc:2804] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:40.288882 23353 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:40.289292 23353 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:40.291864 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:07:40.296586 22843 leader_election.cc:343] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:40.297678 23070 tablet_bootstrap.cc:492] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:40.297640 23281 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.298399 23070 ts_tablet_manager.cc:1403] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.006s
I20260709 13:07:40.298360 23281 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.299508 23337 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.303892 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:40.307436 23073 tablet_bootstrap.cc:654] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:40.299542 23353 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.309479 23353 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:40.310647 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:40.312417 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:07:40.333879 22952 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:40.335536 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:40.336362 22950 raft_consensus.cc:2393] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 1: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
W20260709 13:07:40.337293 22843 leader_election.cc:343] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:40.340596 22992 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:07:40.324743 23070 raft_consensus.cc:359] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.329771 23337 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:40.331307 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:07:40.343112 23070 raft_consensus.cc:385] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:40.343430 23070 raft_consensus.cc:740] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:40.332109 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:40.343920 22951 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:40.344110 23070 consensus_queue.cc:260] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.297705 23069 raft_consensus.cc:359] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.345407 23069 raft_consensus.cc:385] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:40.345691 23069 raft_consensus.cc:740] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:40.342190 22952 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 1.
I20260709 13:07:40.346341 23070 ts_tablet_manager.cc:1434] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.024s	user 0.002s	sys 0.004s
W20260709 13:07:40.344498 22994 leader_election.cc:343] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:40.347397 23070 tablet_bootstrap.cc:492] T 6bdc53dc56134e28bcc43e68e76055fc P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:40.333403 23281 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:40.349758 23069 consensus_queue.cc:260] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.351640 23337 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:40.355572 22951 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:07:40.349411 22843 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:40.357200 23191 raft_consensus.cc:2804] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:40.357736 23191 raft_consensus.cc:697] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:07:40.358247 23069 ts_tablet_manager.cc:1434] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.099s	user 0.006s	sys 0.002s
I20260709 13:07:40.359361 23069 tablet_bootstrap.cc:492] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:40.359501 22992 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:07:40.360402 23281 raft_consensus.cc:2804] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:40.360919 23281 raft_consensus.cc:697] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:07:40.360682 23191 consensus_queue.cc:237] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.361927 23281 consensus_queue.cc:237] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.363920 23070 tablet_bootstrap.cc:654] T 6bdc53dc56134e28bcc43e68e76055fc P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:40.365015 23073 tablet_bootstrap.cc:492] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:40.365911 23073 ts_tablet_manager.cc:1403] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.082s	user 0.013s	sys 0.028s
I20260709 13:07:40.364542 23337 raft_consensus.cc:2749] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:40.370875 23073 raft_consensus.cc:359] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.372298 23073 raft_consensus.cc:385] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:40.372632 23073 raft_consensus.cc:740] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:40.378037 23073 consensus_queue.cc:260] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.379755 23073 ts_tablet_manager.cc:1434] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:07:40.381454 23073 tablet_bootstrap.cc:492] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:40.383122 23070 tablet_bootstrap.cc:492] T 6bdc53dc56134e28bcc43e68e76055fc P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:40.383569 23070 ts_tablet_manager.cc:1403] T 6bdc53dc56134e28bcc43e68e76055fc P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.036s	user 0.005s	sys 0.014s
I20260709 13:07:40.385865 23183 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.386343 23183 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.388280 23183 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:40.391489 23357 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.391654 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:40.393896 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:40.396224 22776 catalog_manager.cc:5697] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:40.389819 23353 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.397755 23353 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.398748 23069 tablet_bootstrap.cc:654] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:40.399596 23353 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:07:40.402340 22920 leader_election.cc:343] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:40.402465 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:40.403267 22952 raft_consensus.cc:2468] T cdebaf291e5d4ae9814a80ce81ee19fe P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 0.
I20260709 13:07:40.406507 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:07:40.407711 22918 leader_election.cc:343] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:40.408170 22918 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
W20260709 13:07:40.408824 22843 leader_election.cc:343] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:40.409407 23183 raft_consensus.cc:2749] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:40.409505 22843 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:40.409775 22777 catalog_manager.cc:5697] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:40.410844 23125 raft_consensus.cc:2804] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:40.411240 23125 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:40.411621 23125 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:40.395503 23357 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.416772 23073 tablet_bootstrap.cc:654] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:40.416484 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:40.417460 23357 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:40.419466 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:07:40.419989 22920 leader_election.cc:343] T e3b39c18a21f448783cdd998acfb2f80 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:07:40.421293 22918 leader_election.cc:343] T e3b39c18a21f448783cdd998acfb2f80 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:40.421727 22918 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:40.423940 23183 raft_consensus.cc:2749] T e3b39c18a21f448783cdd998acfb2f80 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:40.425364 23070 raft_consensus.cc:359] T 6bdc53dc56134e28bcc43e68e76055fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.426287 23070 raft_consensus.cc:385] T 6bdc53dc56134e28bcc43e68e76055fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:40.426614 23070 raft_consensus.cc:740] T 6bdc53dc56134e28bcc43e68e76055fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:40.432377 23070 consensus_queue.cc:260] T 6bdc53dc56134e28bcc43e68e76055fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.435036 23125 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.436755 23070 ts_tablet_manager.cc:1434] T 6bdc53dc56134e28bcc43e68e76055fc P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.053s	user 0.007s	sys 0.000s
I20260709 13:07:40.438059 23070 tablet_bootstrap.cc:492] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:40.439759 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:40.440392 22952 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:40.443914 23070 tablet_bootstrap.cc:654] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:40.448005 22952 raft_consensus.cc:2468] T cdebaf291e5d4ae9814a80ce81ee19fe P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 1.
I20260709 13:07:40.452404 22843 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:07:40.453073 23191 raft_consensus.cc:2804] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:40.453516 23191 raft_consensus.cc:697] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:07:40.454118 23125 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:40.454306 23191 consensus_queue.cc:237] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.457188 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:07:40.457739 23070 tablet_bootstrap.cc:492] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
W20260709 13:07:40.459052 22843 leader_election.cc:343] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:40.458228 23070 ts_tablet_manager.cc:1403] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.000s
I20260709 13:07:40.463380 23070 raft_consensus.cc:359] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.464527 23070 raft_consensus.cc:385] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:40.464895 23070 raft_consensus.cc:740] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:40.465765 23070 consensus_queue.cc:260] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.468554 23070 ts_tablet_manager.cc:1434] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260709 13:07:40.469666 23070 tablet_bootstrap.cc:492] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:40.477345 23070 tablet_bootstrap.cc:654] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:40.488283 23069 tablet_bootstrap.cc:492] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:40.488775 23069 ts_tablet_manager.cc:1403] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.130s	user 0.015s	sys 0.005s
I20260709 13:07:40.491135 23069 raft_consensus.cc:359] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.491861 23069 raft_consensus.cc:385] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:40.492075 23069 raft_consensus.cc:740] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:40.492384 23070 tablet_bootstrap.cc:492] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:40.493043 23070 ts_tablet_manager.cc:1403] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.024s	user 0.020s	sys 0.000s
I20260709 13:07:40.495693 23073 tablet_bootstrap.cc:492] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:40.496145 23073 ts_tablet_manager.cc:1403] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.115s	user 0.025s	sys 0.016s
I20260709 13:07:40.496917 23070 raft_consensus.cc:359] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.497864 23070 raft_consensus.cc:385] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:40.498199 23070 raft_consensus.cc:740] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:40.492614 23069 consensus_queue.cc:260] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.506464 23069 ts_tablet_manager.cc:1434] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.000s
I20260709 13:07:40.508451 23069 tablet_bootstrap.cc:492] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:40.498690 23073 raft_consensus.cc:359] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.513025 23073 raft_consensus.cc:385] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:40.513325 23073 raft_consensus.cc:740] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:40.513986 23073 consensus_queue.cc:260] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.516134 23073 ts_tablet_manager.cc:1434] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.020s	user 0.001s	sys 0.005s
I20260709 13:07:40.512503 23070 consensus_queue.cc:260] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.518365 23070 ts_tablet_manager.cc:1434] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 13:07:40.520316 23073 tablet_bootstrap.cc:492] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:40.527094 23069 tablet_bootstrap.cc:654] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:40.528390 23070 tablet_bootstrap.cc:492] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:40.532377 23073 tablet_bootstrap.cc:654] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:40.546697 23070 tablet_bootstrap.cc:654] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:40.547577 22776 catalog_manager.cc:5697] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:40.559142 23069 tablet_bootstrap.cc:492] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:40.559679 23069 ts_tablet_manager.cc:1403] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.052s	user 0.014s	sys 0.002s
I20260709 13:07:40.561245 23070 tablet_bootstrap.cc:492] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:40.561714 23070 ts_tablet_manager.cc:1403] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.042s	user 0.013s	sys 0.002s
I20260709 13:07:40.562664 23069 raft_consensus.cc:359] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.563380 23069 raft_consensus.cc:385] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:40.563755 23069 raft_consensus.cc:740] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:40.564081 23070 raft_consensus.cc:359] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.564775 23070 raft_consensus.cc:385] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:40.564366 23069 consensus_queue.cc:260] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.565054 23070 raft_consensus.cc:740] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:40.566090 23070 consensus_queue.cc:260] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.566687 23069 ts_tablet_manager.cc:1434] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:07:40.567744 23069 tablet_bootstrap.cc:492] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:40.567744 23070 ts_tablet_manager.cc:1434] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.002s
I20260709 13:07:40.570252 23094 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.570772 23094 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.573150 23094 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:40.574708 23070 tablet_bootstrap.cc:492] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:40.575845 23069 tablet_bootstrap.cc:654] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:40.576201 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:40.578642 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:07:40.578848 22920 leader_election.cc:343] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:07:40.584195 22918 leader_election.cc:343] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:40.584941 22918 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:40.586004 23094 raft_consensus.cc:2749] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:40.589128 23069 tablet_bootstrap.cc:492] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:40.589617 23069 ts_tablet_manager.cc:1403] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.022s	user 0.015s	sys 0.002s
I20260709 13:07:40.595170 23073 tablet_bootstrap.cc:492] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:40.595963 23073 ts_tablet_manager.cc:1403] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.076s	user 0.011s	sys 0.006s
I20260709 13:07:40.596038 23069 raft_consensus.cc:359] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.596853 23069 raft_consensus.cc:385] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:40.597167 23069 raft_consensus.cc:740] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:40.600454 23070 tablet_bootstrap.cc:654] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:40.600114 23073 raft_consensus.cc:359] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.600986 23073 raft_consensus.cc:385] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:40.600741 23069 consensus_queue.cc:260] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.601275 23073 raft_consensus.cc:740] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:40.601964 23073 consensus_queue.cc:260] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.603627 23069 ts_tablet_manager.cc:1434] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:07:40.604107 23073 ts_tablet_manager.cc:1434] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.001s
I20260709 13:07:40.604915 23069 tablet_bootstrap.cc:492] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:40.605496 23073 tablet_bootstrap.cc:492] T b65d9dbf257d434d86d4656dabfd1b70 P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:40.613554 23069 tablet_bootstrap.cc:654] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:40.616964 23073 tablet_bootstrap.cc:654] T b65d9dbf257d434d86d4656dabfd1b70 P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:40.618768 23094 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.619467 23070 tablet_bootstrap.cc:492] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:40.619333 23094 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.620481 23070 ts_tablet_manager.cc:1403] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.046s	user 0.007s	sys 0.016s
I20260709 13:07:40.621999 23094 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:40.622474 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:40.623118 23070 raft_consensus.cc:359] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.623919 23070 raft_consensus.cc:385] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:07:40.624056 22920 leader_election.cc:343] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:40.624234 23070 raft_consensus.cc:740] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:40.625957 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:40.627414 23069 tablet_bootstrap.cc:492] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:40.627885 23069 ts_tablet_manager.cc:1403] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.023s	user 0.010s	sys 0.003s
I20260709 13:07:40.630777 23069 raft_consensus.cc:359] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.631409 23069 raft_consensus.cc:385] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:40.631642 23069 raft_consensus.cc:740] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:40.633071 23070 consensus_queue.cc:260] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:07:40.634984 22918 leader_election.cc:343] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:40.635154 23070 ts_tablet_manager.cc:1434] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.000s
I20260709 13:07:40.635402 22918 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:40.636107 23070 tablet_bootstrap.cc:492] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:40.636603 23183 raft_consensus.cc:2749] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:40.632234 23069 consensus_queue.cc:260] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.641498 23070 tablet_bootstrap.cc:654] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:40.642642 23069 ts_tablet_manager.cc:1434] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:07:40.647056 23069 tablet_bootstrap.cc:492] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:40.647957 23353 consensus_queue.cc:1048] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:40.655486 23069 tablet_bootstrap.cc:654] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:40.662264 23337 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.662811 23337 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.664999 23337 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:40.667400 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:40.671041 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:40.672585 22994 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:07:40.673863 23337 raft_consensus.cc:2749] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:40.678231 23073 tablet_bootstrap.cc:492] T b65d9dbf257d434d86d4656dabfd1b70 P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:40.678735 23073 ts_tablet_manager.cc:1403] T b65d9dbf257d434d86d4656dabfd1b70 P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.073s	user 0.006s	sys 0.040s
I20260709 13:07:40.678953 23201 consensus_queue.cc:1048] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:40.681690 23073 raft_consensus.cc:359] T b65d9dbf257d434d86d4656dabfd1b70 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.682425 23073 raft_consensus.cc:385] T b65d9dbf257d434d86d4656dabfd1b70 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:40.682723 23073 raft_consensus.cc:740] T b65d9dbf257d434d86d4656dabfd1b70 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:40.683393 23073 consensus_queue.cc:260] T b65d9dbf257d434d86d4656dabfd1b70 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.685091 23073 ts_tablet_manager.cc:1434] T b65d9dbf257d434d86d4656dabfd1b70 P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:07:40.686035 23073 tablet_bootstrap.cc:492] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:40.693207 23073 tablet_bootstrap.cc:654] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:40.695233 23094 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.695876 23094 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.698393 23094 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:40.702258 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:40.700150 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:07:40.714015 22918 leader_election.cc:343] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:40.715001 23073 tablet_bootstrap.cc:492] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:40.715507 23073 ts_tablet_manager.cc:1403] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.030s	user 0.006s	sys 0.011s
I20260709 13:07:40.717995 23339 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.718405 23073 raft_consensus.cc:359] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.718490 23339 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.719249 23073 raft_consensus.cc:385] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:40.719560 23073 raft_consensus.cc:740] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:40.720978 23339 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:40.722065 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:40.722922 22951 raft_consensus.cc:2393] T cdebaf291e5d4ae9814a80ce81ee19fe P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 1: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
W20260709 13:07:40.723520 22920 leader_election.cc:343] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:40.723856 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:40.724092 22920 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:40.725028 23094 raft_consensus.cc:2749] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:40.720367 23073 consensus_queue.cc:260] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.726306 22994 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:07:40.727002 23281 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:40.727870 23191 consensus_queue.cc:1048] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:40.734504 23281 raft_consensus.cc:2749] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:40.736320 23073 ts_tablet_manager.cc:1434] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.021s	user 0.006s	sys 0.002s
I20260709 13:07:40.737478 23073 tablet_bootstrap.cc:492] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:40.743538 23353 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.744443 23073 tablet_bootstrap.cc:654] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:40.744041 23353 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.748520 23353 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:40.751631 23069 tablet_bootstrap.cc:492] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:40.757409 23073 tablet_bootstrap.cc:492] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:40.760228 23073 ts_tablet_manager.cc:1403] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.023s	user 0.018s	sys 0.000s
I20260709 13:07:40.761379 23021 raft_consensus.cc:1275] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:07:40.763104 23073 raft_consensus.cc:359] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.759559 23281 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.764052 23281 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.766506 23281 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:40.766571 22951 raft_consensus.cc:1275] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:07:40.767128 23073 raft_consensus.cc:385] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:40.763295 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:40.767432 23073 raft_consensus.cc:740] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:40.767899 23021 raft_consensus.cc:2393] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 1: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:07:40.768365 23073 consensus_queue.cc:260] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.769609 23339 consensus_queue.cc:1048] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:40.770262 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:40.771018 23073 ts_tablet_manager.cc:1434] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:07:40.770921 22951 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 0.
I20260709 13:07:40.772145 23073 tablet_bootstrap.cc:492] T 698093275c264d4d99fb8c1a49e3b28f P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:40.774233 22992 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:40.770826 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:40.775534 23339 raft_consensus.cc:2804] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:40.776171 22950 raft_consensus.cc:2393] T fab4388fa7ea49bda9fa070cf542577f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 1: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:07:40.776124 23339 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:40.776819 23339 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:40.777269 23274 consensus_queue.cc:1048] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:40.780009 23073 tablet_bootstrap.cc:654] T 698093275c264d4d99fb8c1a49e3b28f P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:40.783651 23094 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.784267 23094 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.788005 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:40.788825 23027 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 0.
I20260709 13:07:40.790310 22918 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:40.791527 23183 raft_consensus.cc:2804] T 6bdc53dc56134e28bcc43e68e76055fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:40.792364 23274 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.792521 23183 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:40.792986 23183 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:40.792958 23274 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.786684 23094 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:40.798060 23274 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:40.798661 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:40.759649 23069 ts_tablet_manager.cc:1403] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.113s	user 0.019s	sys 0.006s
I20260709 13:07:40.791478 23337 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.800568 23337 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.802050 23069 raft_consensus.cc:359] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.802677 23069 raft_consensus.cc:385] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:40.802939 23069 raft_consensus.cc:740] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:40.804064 23274 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.804538 23274 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.806668 23274 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:40.806548 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:40.807612 22950 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 0.
I20260709 13:07:40.807669 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:40.808374 23025 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 0.
I20260709 13:07:40.809103 22992 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:40.809667 22843 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:40.810014 23281 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:40.810413 23281 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:40.810434 23274 raft_consensus.cc:2804] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:40.810906 23281 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:40.810933 23274 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:40.811479 23274 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:40.811426 23339 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.813053 23027 raft_consensus.cc:2393] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 1: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:07:40.819178 23281 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.819784 23274 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.823083 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:40.824131 22950 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:40.824856 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:07:40.803483 23069 consensus_queue.cc:260] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.825477 23024 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:40.834359 23339 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:40.835377 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:40.836089 22951 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:40.861402 23357 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.861956 23357 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.864177 23357 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:40.865370 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:40.838958 23274 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:40.838444 23024 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 1.
I20260709 13:07:40.866225 23027 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 0.
I20260709 13:07:40.867636 22843 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:40.867597 22918 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:40.868698 23274 raft_consensus.cc:2804] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:40.869307 23274 raft_consensus.cc:697] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:07:40.836627 23069 ts_tablet_manager.cc:1434] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.037s	user 0.005s	sys 0.001s
I20260709 13:07:40.870149 23278 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:40.871426 23278 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:40.871871 23278 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:40.872339 23069 tablet_bootstrap.cc:492] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:40.870335 23274 consensus_queue.cc:237] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.836766 23337 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:40.867043 22951 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:07:40.869894 23357 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.879550 22992 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:40.879952 23357 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.883203 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:40.883489 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:40.884059 22955 raft_consensus.cc:2393] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 1: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:07:40.884306 23353 consensus_queue.cc:1048] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:40.884969 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:40.887473 23357 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:40.887892 23353 consensus_queue.cc:1048] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:40.890077 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:40.890776 23281 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:07:40.891516 22918 leader_election.cc:343] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:40.891700 23281 raft_consensus.cc:2804] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:40.902031 23281 raft_consensus.cc:697] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:07:40.903178 23281 consensus_queue.cc:237] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.906991 23278 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.911803 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:07:40.912518 23027 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:40.835870 23183 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.917853 22949 raft_consensus.cc:2393] T 6bdc53dc56134e28bcc43e68e76055fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 1: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:07:40.919852 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:07:40.920595 23024 raft_consensus.cc:2393] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 1: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:07:40.921972 23027 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 1.
I20260709 13:07:40.923441 22918 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:40.924813 23357 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:40.930102 22843 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:40.932356 23191 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:40.933197 22950 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:07:40.935089 23201 consensus_queue.cc:1048] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:40.935521 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:40.937189 23357 raft_consensus.cc:697] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:07:40.937594 22992 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:40.938263 23357 consensus_queue.cc:237] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.938730 23337 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:40.939693 23337 raft_consensus.cc:697] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:07:40.941432 23337 consensus_queue.cc:237] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.947654 23191 raft_consensus.cc:2749] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:40.931986 23278 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:40.951190 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:40.961491 23191 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:40.961973 23191 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:40.965041 23191 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:07:40.965988 22843 consensus_peers.cc:597] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f -> Peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Couldn't send request to peer f3c799b1249e40119d2a5864231c6c75. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:40.966714 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:40.981287 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:40.981586 23027 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:40.982142 22875 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 0.
I20260709 13:07:40.988061 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:40.988801 22875 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 0.
I20260709 13:07:40.989993 23183 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:40.991737 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:07:40.992398 22874 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:41.000138 22843 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:41.003415 23404 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:41.015050 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:41.015904 23274 consensus_queue.cc:1048] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:41.021968 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:07:41.023068 23404 raft_consensus.cc:2749] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:41.015516 22874 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:07:41.025305 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:07:41.039315 22920 leader_election.cc:343] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:07:41.043179 22920 leader_election.cc:343] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:41.043625 22920 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:41.044414 23415 raft_consensus.cc:2749] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:40.989718 23069 tablet_bootstrap.cc:654] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
W20260709 13:07:41.053539 22920 leader_election.cc:343] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:41.054023 23073 tablet_bootstrap.cc:492] T 698093275c264d4d99fb8c1a49e3b28f P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:41.054726 23073 ts_tablet_manager.cc:1403] T 698093275c264d4d99fb8c1a49e3b28f P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.283s	user 0.051s	sys 0.115s
I20260709 13:07:41.055444 22920 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:41.056326 23357 raft_consensus.cc:2749] T 6bdc53dc56134e28bcc43e68e76055fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:07:41.065400 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:07:41.070266 23073 raft_consensus.cc:359] T 698093275c264d4d99fb8c1a49e3b28f P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.073212 23073 raft_consensus.cc:385] T 698093275c264d4d99fb8c1a49e3b28f P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:41.073556 23073 raft_consensus.cc:740] T 698093275c264d4d99fb8c1a49e3b28f P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:41.074682 23073 consensus_queue.cc:260] T 698093275c264d4d99fb8c1a49e3b28f P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.075074 22875 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:41.076467 23073 ts_tablet_manager.cc:1434] T 698093275c264d4d99fb8c1a49e3b28f P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.021s	user 0.004s	sys 0.002s
I20260709 13:07:41.077404 23073 tablet_bootstrap.cc:492] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:41.077448 22778 catalog_manager.cc:5697] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 1 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:41.083279 22878 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:41.084285 23073 tablet_bootstrap.cc:654] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:41.095947 22874 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:07:41.098981 23069 tablet_bootstrap.cc:492] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:41.099531 23069 ts_tablet_manager.cc:1403] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.228s	user 0.010s	sys 0.001s
I20260709 13:07:41.071980 23070 tablet_bootstrap.cc:492] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:41.105091 23070 ts_tablet_manager.cc:1403] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.469s	user 0.068s	sys 0.088s
I20260709 13:07:41.119691 23073 tablet_bootstrap.cc:492] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:41.120325 23073 ts_tablet_manager.cc:1403] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.043s	user 0.009s	sys 0.031s
I20260709 13:07:41.123646 23073 raft_consensus.cc:359] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.124260 23073 raft_consensus.cc:385] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:41.124503 23073 raft_consensus.cc:740] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:41.126233 23073 consensus_queue.cc:260] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.128147 23073 ts_tablet_manager.cc:1434] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:07:41.129299 23073 tablet_bootstrap.cc:492] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:41.137681 23073 tablet_bootstrap.cc:654] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:41.157238 23073 tablet_bootstrap.cc:492] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:41.157842 23073 ts_tablet_manager.cc:1403] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.029s	user 0.014s	sys 0.006s
I20260709 13:07:41.161015 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:41.168622 23069 raft_consensus.cc:359] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.169621 23069 raft_consensus.cc:385] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:41.172482 23069 raft_consensus.cc:740] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:41.178371 22843 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:41.180106 23191 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:41.178485 23073 raft_consensus.cc:359] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.178586 23069 consensus_queue.cc:260] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.183238 23069 ts_tablet_manager.cc:1434] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.083s	user 0.008s	sys 0.000s
I20260709 13:07:41.183090 23404 consensus_queue.cc:1048] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:41.180896 23073 raft_consensus.cc:385] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:41.183790 23073 raft_consensus.cc:740] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:41.184834 23073 consensus_queue.cc:260] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.187495 23073 ts_tablet_manager.cc:1434] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.029s	user 0.010s	sys 0.000s
I20260709 13:07:41.191707 23073 tablet_bootstrap.cc:492] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:41.188706 23069 tablet_bootstrap.cc:492] T 3047c357228240a2aa4d84e8fbf437fc P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:41.107877 23070 raft_consensus.cc:359] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.195955 23070 raft_consensus.cc:385] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:41.196187 23070 raft_consensus.cc:740] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:41.198707 23070 consensus_queue.cc:260] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.229351 23070 ts_tablet_manager.cc:1434] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.124s	user 0.005s	sys 0.003s
I20260709 13:07:41.241657 23070 tablet_bootstrap.cc:492] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:41.246503 23418 consensus_queue.cc:1048] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:07:41.256053 23073 tablet_bootstrap.cc:654] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:41.272289 23069 tablet_bootstrap.cc:654] T 3047c357228240a2aa4d84e8fbf437fc P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:41.286897 22776 catalog_manager.cc:5697] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:41.288900 22776 catalog_manager.cc:5697] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:41.284250 23274 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.291096 23357 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.302192 23357 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:07:41.305652 22826 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:41.307327 23274 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.311817 23070 tablet_bootstrap.cc:654] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:41.312268 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:41.313891 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:07:41.314627 22920 leader_election.cc:343] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:41.314745 22953 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 0.
I20260709 13:07:41.316898 23274 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:41.319291 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:41.318521 23357 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:41.331740 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:41.337438 23357 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.338003 23070 tablet_bootstrap.cc:492] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:41.338610 23070 ts_tablet_manager.cc:1403] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.097s	user 0.011s	sys 0.004s
W20260709 13:07:41.339206 22843 leader_election.cc:343] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:07:41.341135 22918 leader_election.cc:343] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:41.341569 22918 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:41.342588 23415 raft_consensus.cc:2749] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:41.343448 23357 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.347036 23073 tablet_bootstrap.cc:492] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:41.347458 23073 ts_tablet_manager.cc:1403] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.156s	user 0.023s	sys 0.014s
I20260709 13:07:41.350484 22843 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:41.358129 23357 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:41.359419 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:41.360206 23191 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:41.360567 23191 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:41.360881 22920 leader_election.cc:343] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:41.360903 23191 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:41.361896 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:41.362553 23021 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 0.
I20260709 13:07:41.365850 23070 raft_consensus.cc:359] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.366510 23070 raft_consensus.cc:385] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:41.366771 23070 raft_consensus.cc:740] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:41.367487 23070 consensus_queue.cc:260] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.368558 22918 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:07:41.369505 23070 ts_tablet_manager.cc:1434] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.000s
I20260709 13:07:41.369987 23274 consensus_queue.cc:1048] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:41.369505 23415 raft_consensus.cc:2804] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:41.371182 23415 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:41.371511 23415 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:41.372251 23070 tablet_bootstrap.cc:492] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:41.374833 22778 catalog_manager.cc:5697] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:41.379704 23073 raft_consensus.cc:359] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.385038 23191 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.386490 23073 raft_consensus.cc:385] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:41.386839 23073 raft_consensus.cc:740] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:41.389143 23073 consensus_queue.cc:260] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.391336 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:07:41.391582 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:41.391927 23024 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:41.392211 22951 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:41.391547 23073 ts_tablet_manager.cc:1434] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.044s	user 0.003s	sys 0.003s
I20260709 13:07:41.394301 23191 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:41.395121 23073 tablet_bootstrap.cc:492] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:41.395442 23069 tablet_bootstrap.cc:492] T 3047c357228240a2aa4d84e8fbf437fc P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:41.400646 23069 ts_tablet_manager.cc:1403] T 3047c357228240a2aa4d84e8fbf437fc P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.212s	user 0.013s	sys 0.015s
I20260709 13:07:41.409188 23357 consensus_queue.cc:1048] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:41.412485 22951 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 1.
I20260709 13:07:41.417244 23415 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.421272 22843 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:07:41.423038 23191 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:41.423594 23191 raft_consensus.cc:697] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:07:41.424341 23191 consensus_queue.cc:237] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.428822 23415 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:41.430315 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:07:41.432912 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:07:41.433468 23021 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:41.434834 23069 raft_consensus.cc:359] T 3047c357228240a2aa4d84e8fbf437fc P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.435458 23069 raft_consensus.cc:385] T 3047c357228240a2aa4d84e8fbf437fc P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:41.435708 23069 raft_consensus.cc:740] T 3047c357228240a2aa4d84e8fbf437fc P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:41.436343 23069 consensus_queue.cc:260] T 3047c357228240a2aa4d84e8fbf437fc P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.438628 22874 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:41.447389 23021 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 1.
W20260709 13:07:41.451656 22920 consensus_peers.cc:597] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:41.453197 23405 consensus_queue.cc:1048] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:41.456737 22918 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:41.459443 23024 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 1.
I20260709 13:07:41.459444 23415 raft_consensus.cc:2804] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:41.453490 22874 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 1.
I20260709 13:07:41.462420 23415 raft_consensus.cc:697] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:07:41.463662 23415 consensus_queue.cc:237] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.478278 23418 consensus_queue.cc:1048] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:41.487515 23070 tablet_bootstrap.cc:654] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:41.492141 23073 tablet_bootstrap.cc:654] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:41.482537 23418 consensus_queue.cc:1048] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 13:07:41.495498 23069 ts_tablet_manager.cc:1434] T 3047c357228240a2aa4d84e8fbf437fc P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.094s	user 0.009s	sys 0.000s
I20260709 13:07:41.495530 23339 consensus_queue.cc:1048] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:41.497462 23418 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.497959 23418 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.496711 22778 catalog_manager.cc:5697] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:41.524942 23069 tablet_bootstrap.cc:492] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:41.526754 23417 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.527371 23417 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.558205 23418 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:41.561734 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:41.564113 23418 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.573843 23418 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.575206 23419 consensus_queue.cc:1048] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:41.556239 23024 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:41.588430 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:41.591519 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:41.592769 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:41.593755 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:07:41.596498 22920 leader_election.cc:343] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:41.577792 23417 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:41.576588 23418 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:07:41.615305 22918 leader_election.cc:343] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:41.617055 22918 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:41.620450 23073 tablet_bootstrap.cc:492] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
W20260709 13:07:41.616336 22994 leader_election.cc:343] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:41.620914 23073 ts_tablet_manager.cc:1403] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.226s	user 0.016s	sys 0.017s
W20260709 13:07:41.616142 23441 log.cc:927] Time spent T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 13:07:41.621878 22992 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:07:41.619946 23183 raft_consensus.cc:2749] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:41.629904 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:41.630448 22949 raft_consensus.cc:2468] T e3b39c18a21f448783cdd998acfb2f80 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 0.
I20260709 13:07:41.633868 23073 raft_consensus.cc:359] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.634567 23073 raft_consensus.cc:385] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:41.634842 23073 raft_consensus.cc:740] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:41.635447 23073 consensus_queue.cc:260] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.637354 22992 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:07:41.639866 23419 raft_consensus.cc:2804] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:41.640244 23419 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:41.640569 23419 raft_consensus.cc:3060] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:41.647625 22778 catalog_manager.cc:5697] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 1 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:41.650478 23419 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.655519 23069 tablet_bootstrap.cc:654] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:41.658241 23073 ts_tablet_manager.cc:1434] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.037s	user 0.006s	sys 0.000s
I20260709 13:07:41.658419 23070 tablet_bootstrap.cc:492] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:41.658877 23070 ts_tablet_manager.cc:1403] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.287s	user 0.018s	sys 0.002s
I20260709 13:07:41.659256 23073 tablet_bootstrap.cc:492] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:41.659821 23281 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.660414 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:41.661002 22951 raft_consensus.cc:3060] T e3b39c18a21f448783cdd998acfb2f80 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:41.661453 23070 raft_consensus.cc:359] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.662097 23070 raft_consensus.cc:385] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:41.662350 23070 raft_consensus.cc:740] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:41.663139 23070 consensus_queue.cc:260] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.660240 23281 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.664153 23419 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:41.667389 23070 ts_tablet_manager.cc:1434] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
I20260709 13:07:41.668439 23070 tablet_bootstrap.cc:492] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:41.669560 23281 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:41.669741 23073 tablet_bootstrap.cc:654] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:41.671482 22951 raft_consensus.cc:2468] T e3b39c18a21f448783cdd998acfb2f80 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:07:41.674293 23070 tablet_bootstrap.cc:654] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:41.676331 22992 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:41.677131 23281 raft_consensus.cc:2804] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:41.677738 23281 raft_consensus.cc:697] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:07:41.678339 23069 tablet_bootstrap.cc:492] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:41.678764 23069 ts_tablet_manager.cc:1403] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.162s	user 0.010s	sys 0.002s
I20260709 13:07:41.678781 23281 consensus_queue.cc:237] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.683027 23070 tablet_bootstrap.cc:492] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:41.683470 23070 ts_tablet_manager.cc:1403] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.015s	user 0.005s	sys 0.008s
I20260709 13:07:41.685242 23069 raft_consensus.cc:359] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.685987 23069 raft_consensus.cc:385] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:41.686223 23070 raft_consensus.cc:359] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.687541 23070 raft_consensus.cc:385] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:41.687894 23070 raft_consensus.cc:740] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:41.688297 23073 tablet_bootstrap.cc:492] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:41.688825 23073 ts_tablet_manager.cc:1403] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.030s	user 0.011s	sys 0.000s
I20260709 13:07:41.688710 23070 consensus_queue.cc:260] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.690945 23070 ts_tablet_manager.cc:1434] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:07:41.691808 23073 raft_consensus.cc:359] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.686287 23069 raft_consensus.cc:740] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:41.692358 23070 tablet_bootstrap.cc:492] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:41.692291 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:41.692740 23073 raft_consensus.cc:385] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:41.692971 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:07:41.693153 23073 raft_consensus.cc:740] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:41.693236 22953 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 0.
I20260709 13:07:41.694813 22992 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:41.697109 23419 raft_consensus.cc:2804] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:41.697510 23419 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:41.697736 23339 raft_consensus.cc:2749] T 698093275c264d4d99fb8c1a49e3b28f P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:41.697834 23419 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:41.701784 23069 consensus_queue.cc:260] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.704448 23069 ts_tablet_manager.cc:1434] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 13:07:41.705484 23069 tablet_bootstrap.cc:492] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:41.705924 22874 raft_consensus.cc:3060] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:41.714995 23419 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.717283 23069 tablet_bootstrap.cc:654] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:41.726099 22776 catalog_manager.cc:5697] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:41.730525 23069 tablet_bootstrap.cc:492] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:41.730698 22874 raft_consensus.cc:2468] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:07:41.731104 23069 ts_tablet_manager.cc:1403] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.026s	user 0.009s	sys 0.008s
I20260709 13:07:41.693981 23073 consensus_queue.cc:260] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.734129 23069 raft_consensus.cc:359] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.734866 23069 raft_consensus.cc:385] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:41.735176 23069 raft_consensus.cc:740] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:41.735941 23069 consensus_queue.cc:260] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.737336 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:41.738485 23069 ts_tablet_manager.cc:1434] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:07:41.710654 23070 tablet_bootstrap.cc:654] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:41.739890 23073 ts_tablet_manager.cc:1434] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.051s	user 0.006s	sys 0.000s
I20260709 13:07:41.740999 23073 tablet_bootstrap.cc:492] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:41.759963 23069 tablet_bootstrap.cc:492] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:41.760767 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:41.761585 22951 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:41.767654 23069 tablet_bootstrap.cc:654] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:41.775381 22951 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:07:41.776854 22992 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:41.777920 23281 raft_consensus.cc:2804] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:41.778620 23281 raft_consensus.cc:697] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:07:41.779554 23281 consensus_queue.cc:237] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.789119 23069 tablet_bootstrap.cc:492] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:41.789670 23069 ts_tablet_manager.cc:1403] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.030s	user 0.009s	sys 0.006s
I20260709 13:07:41.792551 23069 raft_consensus.cc:359] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.793037 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:07:41.793900 23069 raft_consensus.cc:385] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:41.794528 23069 raft_consensus.cc:740] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:41.794020 23419 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:41.796109 23069 consensus_queue.cc:260] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.799599 23069 ts_tablet_manager.cc:1434] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:07:41.803071 23069 tablet_bootstrap.cc:492] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:41.805878 23073 tablet_bootstrap.cc:654] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:41.809144 23069 tablet_bootstrap.cc:654] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
W20260709 13:07:41.827653 22994 leader_election.cc:343] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:07:41.829892 22994 leader_election.cc:343] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:41.834069 22776 catalog_manager.cc:5697] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:41.859997 23125 consensus_queue.cc:1048] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:41.869850 23337 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.870361 23337 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.872349 23070 tablet_bootstrap.cc:492] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:41.872838 23070 ts_tablet_manager.cc:1403] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.181s	user 0.041s	sys 0.038s
I20260709 13:07:41.874441 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:41.874938 23191 consensus_queue.cc:1048] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:41.875097 22953 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 0.
I20260709 13:07:41.875382 23070 raft_consensus.cc:359] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.876032 23070 raft_consensus.cc:385] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:41.876307 23070 raft_consensus.cc:740] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:41.876310 22992 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:41.876919 23070 consensus_queue.cc:260] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.878188 23339 raft_consensus.cc:2804] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:41.878528 23339 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:41.878911 23339 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:41.879909 23070 ts_tablet_manager.cc:1434] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:07:41.880909 23070 tablet_bootstrap.cc:492] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:41.872471 23337 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:41.893296 23183 consensus_queue.cc:1048] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:41.897330 23070 tablet_bootstrap.cc:654] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:41.915148 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:41.908847 23183 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.916406 23183 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.917495 23125 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.917943 23125 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.919423 23339 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.920120 23183 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:07:41.925747 22994 leader_election.cc:343] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:41.920537 23125 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:41.929122 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:41.958031 23339 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:41.958539 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:41.958987 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:41.960646 23073 tablet_bootstrap.cc:492] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:41.961054 23417 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:41.961400 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:41.961710 23417 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.962159 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:07:41.963550 22920 leader_election.cc:343] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:41.964046 23417 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:41.965526 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:07:41.966991 22920 leader_election.cc:343] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:41.969882 23339 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:41.970147 22918 leader_election.cc:343] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:41.970577 22918 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:41.970366 23339 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:41.971525 23183 raft_consensus.cc:2749] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:41.972434 23339 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:41.961108 23073 ts_tablet_manager.cc:1403] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.220s	user 0.014s	sys 0.018s
W20260709 13:07:41.980011 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 48378.8us
I20260709 13:07:41.980405 23069 tablet_bootstrap.cc:492] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:41.980926 23069 ts_tablet_manager.cc:1403] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.178s	user 0.029s	sys 0.006s
I20260709 13:07:41.981877 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:41.982653 22949 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:41.985162 22843 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:41.986183 23125 raft_consensus.cc:2749] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:41.991588 22949 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:07:41.995659 22992 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:41.996886 23339 raft_consensus.cc:2804] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:41.997198 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:41.997440 23339 raft_consensus.cc:697] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:07:41.997907 22875 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:07:41.998361 23339 consensus_queue.cc:237] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.003619 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:42.005522 23069 raft_consensus.cc:359] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.011490 23069 raft_consensus.cc:385] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:42.011759 23069 raft_consensus.cc:740] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:42.012387 23069 consensus_queue.cc:260] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.014719 23069 ts_tablet_manager.cc:1434] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.034s	user 0.006s	sys 0.000s
I20260709 13:07:42.015664 23069 tablet_bootstrap.cc:492] T abd5e7d435ca4afd9727e37b2ab31589 P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:42.021210 23070 tablet_bootstrap.cc:492] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:42.021665 23070 ts_tablet_manager.cc:1403] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.141s	user 0.016s	sys 0.019s
I20260709 13:07:42.021546 22949 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:07:42.010577 23073 raft_consensus.cc:359] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.022797 23073 raft_consensus.cc:385] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:42.023298 23404 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.024384 22992 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:42.024130 23073 raft_consensus.cc:740] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:42.024799 22874 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:42.024509 23404 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.025111 23337 raft_consensus.cc:2804] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:42.023702 23069 tablet_bootstrap.cc:654] T abd5e7d435ca4afd9727e37b2ab31589 P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:42.025494 23337 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:42.025882 23337 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:42.025894 23073 consensus_queue.cc:260] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.024228 23070 raft_consensus.cc:359] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.027518 23070 raft_consensus.cc:385] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:42.026707 23404 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:42.031839 23073 ts_tablet_manager.cc:1434] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.055s	user 0.006s	sys 0.000s
I20260709 13:07:42.032325 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:42.035355 23337 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.036043 22951 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 0.
I20260709 13:07:42.027804 23070 raft_consensus.cc:740] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:42.037194 23070 consensus_queue.cc:260] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.037569 22843 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:07:42.038278 23125 raft_consensus.cc:2804] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:42.038460 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:42.038657 23125 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:42.039016 23125 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:42.039542 23337 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:42.040023 23070 ts_tablet_manager.cc:1434] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.002s
I20260709 13:07:42.039047 22951 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:42.050024 22951 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 2.
I20260709 13:07:42.051450 23070 tablet_bootstrap.cc:492] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:42.055301 23125 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.041085 23073 tablet_bootstrap.cc:492] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:42.056142 22992 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:42.057158 23337 raft_consensus.cc:2804] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:42.057878 23337 raft_consensus.cc:697] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 2 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:07:42.057219 22778 catalog_manager.cc:5697] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:42.058748 23337 consensus_queue.cc:237] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.064993 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:42.065932 23069 tablet_bootstrap.cc:492] T abd5e7d435ca4afd9727e37b2ab31589 P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:42.066433 23069 ts_tablet_manager.cc:1403] T abd5e7d435ca4afd9727e37b2ab31589 P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.051s	user 0.008s	sys 0.002s
I20260709 13:07:42.068998 23069 raft_consensus.cc:359] T abd5e7d435ca4afd9727e37b2ab31589 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.069670 23069 raft_consensus.cc:385] T abd5e7d435ca4afd9727e37b2ab31589 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:42.069972 23069 raft_consensus.cc:740] T abd5e7d435ca4afd9727e37b2ab31589 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:42.070581 23069 consensus_queue.cc:260] T abd5e7d435ca4afd9727e37b2ab31589 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.073200 23069 ts_tablet_manager.cc:1434] T abd5e7d435ca4afd9727e37b2ab31589 P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:07:42.073853 23073 tablet_bootstrap.cc:654] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
W20260709 13:07:42.074492 22994 leader_election.cc:343] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:42.077365 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:42.077903 22953 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:42.081743 23070 tablet_bootstrap.cc:654] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:42.083499 23125 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:42.086002 23069 tablet_bootstrap.cc:492] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:42.087636 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:42.094975 23069 tablet_bootstrap.cc:654] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:42.095384 23073 tablet_bootstrap.cc:492] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:42.095858 23073 ts_tablet_manager.cc:1403] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.055s	user 0.008s	sys 0.007s
I20260709 13:07:42.098763 23073 raft_consensus.cc:359] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.099614 23073 raft_consensus.cc:385] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:42.099954 23073 raft_consensus.cc:740] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:42.100648 23073 consensus_queue.cc:260] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.102599 23073 ts_tablet_manager.cc:1434] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:07:42.103646 23073 tablet_bootstrap.cc:492] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:42.105883 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:07:42.106464 22874 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:42.107796 23069 tablet_bootstrap.cc:492] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:42.109473 23069 ts_tablet_manager.cc:1403] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.024s	user 0.007s	sys 0.005s
I20260709 13:07:42.110113 23073 tablet_bootstrap.cc:654] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:42.113034 23069 raft_consensus.cc:359] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.113845 23069 raft_consensus.cc:385] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:42.115586 23070 tablet_bootstrap.cc:492] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:42.116092 23070 ts_tablet_manager.cc:1403] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.065s	user 0.012s	sys 0.004s
I20260709 13:07:42.118338 22874 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 2.
W20260709 13:07:42.118924 22843 leader_election.cc:343] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:42.118974 23070 raft_consensus.cc:359] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.119844 23070 raft_consensus.cc:385] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:42.120239 23070 raft_consensus.cc:740] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:42.121129 22953 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 1.
I20260709 13:07:42.121582 23069 raft_consensus.cc:740] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:42.122717 23069 consensus_queue.cc:260] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.121054 23070 consensus_queue.cc:260] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.124878 23069 ts_tablet_manager.cc:1434] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:07:42.116092 23073 tablet_bootstrap.cc:492] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:42.122978 22843 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:07:42.125572 23070 ts_tablet_manager.cc:1434] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.004s
I20260709 13:07:42.126494 23070 tablet_bootstrap.cc:492] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7: Bootstrap starting.
I20260709 13:07:42.128984 23404 raft_consensus.cc:2804] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:42.129927 23417 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.130359 23417 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:07:42.131611 22918 leader_election.cc:343] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:42.132022 22918 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:42.132236 23417 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:42.132853 23404 raft_consensus.cc:697] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:07:42.133875 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:42.134071 23357 raft_consensus.cc:2749] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:07:42.135226 22920 leader_election.cc:343] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:42.133673 23404 consensus_queue.cc:237] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.136510 23070 tablet_bootstrap.cc:654] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:42.137458 22776 catalog_manager.cc:5697] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 0 to 2, leader changed from <none> to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 2 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:42.145900 23417 consensus_queue.cc:1048] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:42.149539 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:07:42.152251 23073 ts_tablet_manager.cc:1403] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.049s	user 0.010s	sys 0.001s
W20260709 13:07:42.156895 22843 leader_election.cc:343] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:42.165517 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:42.175902 23069 tablet_bootstrap.cc:492] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:42.179385 23070 tablet_bootstrap.cc:492] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7: No bootstrap required, opened a new log
I20260709 13:07:42.179870 23070 ts_tablet_manager.cc:1403] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent bootstrapping tablet: real 0.054s	user 0.021s	sys 0.022s
I20260709 13:07:42.183349 23073 raft_consensus.cc:359] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.184051 23073 raft_consensus.cc:385] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:42.184336 23073 raft_consensus.cc:740] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:42.184983 23073 consensus_queue.cc:260] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.189797 23339 consensus_queue.cc:1048] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:42.191849 23070 raft_consensus.cc:359] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.192492 23070 raft_consensus.cc:385] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:42.192790 23070 raft_consensus.cc:740] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Initialized, Role: FOLLOWER
I20260709 13:07:42.193514 23070 consensus_queue.cc:260] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.194566 23069 tablet_bootstrap.cc:654] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:42.194736 23419 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.196214 23419 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.200184 23073 ts_tablet_manager.cc:1434] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.047s	user 0.007s	sys 0.000s
I20260709 13:07:42.200325 23419 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:42.201382 23073 tablet_bootstrap.cc:492] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:42.205304 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:42.205984 22953 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 0.
I20260709 13:07:42.206466 23070 ts_tablet_manager.cc:1434] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent starting tablet: real 0.026s	user 0.004s	sys 0.004s
I20260709 13:07:42.207178 22992 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:42.208120 23337 raft_consensus.cc:2804] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:42.208604 23337 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:42.209103 23337 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:42.213227 23073 tablet_bootstrap.cc:654] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:42.214560 23339 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.216841 23339 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.218680 23339 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:42.223094 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:42.223932 22953 raft_consensus.cc:2393] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 1: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:07:42.223991 23069 tablet_bootstrap.cc:492] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:42.224442 23069 ts_tablet_manager.cc:1403] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.049s	user 0.011s	sys 0.001s
I20260709 13:07:42.229027 23069 raft_consensus.cc:359] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.229869 23069 raft_consensus.cc:385] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:42.230201 23069 raft_consensus.cc:740] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:42.231021 23069 consensus_queue.cc:260] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:07:42.243661 22918 leader_election.cc:343] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:42.244163 22918 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:42.246726 23357 raft_consensus.cc:2749] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:42.249289 23069 ts_tablet_manager.cc:1434] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 13:07:42.257021 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:42.258275 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:42.260934 23069 tablet_bootstrap.cc:492] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:42.271776 23069 tablet_bootstrap.cc:654] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:42.275517 23073 tablet_bootstrap.cc:492] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:42.276067 23073 ts_tablet_manager.cc:1403] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.075s	user 0.026s	sys 0.010s
I20260709 13:07:42.282990 23073 raft_consensus.cc:359] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.283674 23073 raft_consensus.cc:385] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:42.283399 23337 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.283936 23073 raft_consensus.cc:740] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:42.284353 23069 tablet_bootstrap.cc:492] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:42.284792 23069 ts_tablet_manager.cc:1403] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.005s
I20260709 13:07:42.284554 23073 consensus_queue.cc:260] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.284006 22778 catalog_manager.cc:5697] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:42.287117 23073 ts_tablet_manager.cc:1434] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 13:07:42.287245 23069 raft_consensus.cc:359] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.287937 23069 raft_consensus.cc:385] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:42.288089 23073 tablet_bootstrap.cc:492] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:42.288194 23069 raft_consensus.cc:740] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:42.288921 23069 consensus_queue.cc:260] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.292972 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:42.293606 22949 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:42.294564 23069 ts_tablet_manager.cc:1434] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:07:42.295954 23069 tablet_bootstrap.cc:492] T 3f1c9e3f9b244b0695775e91f28d67d7 P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:42.297212 23073 tablet_bootstrap.cc:654] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:42.300611 22949 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:07:42.302008 23069 tablet_bootstrap.cc:654] T 3f1c9e3f9b244b0695775e91f28d67d7 P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:42.302604 22992 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:42.303615 23419 raft_consensus.cc:2804] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:42.304207 23419 raft_consensus.cc:697] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:07:42.305276 23419 consensus_queue.cc:237] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.313926 23404 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.314482 23404 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.320650 23337 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:42.326241 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:07:42.332626 22994 leader_election.cc:343] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:42.335444 22994 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:07:42.336078 23191 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.336350 23419 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:42.336582 23191 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.338692 23191 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:42.346263 23337 consensus_queue.cc:1048] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:42.347028 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:42.348109 22950 raft_consensus.cc:2393] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 1: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:07:42.349205 23191 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:42.349624 22994 leader_election.cc:343] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:42.349839 23191 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.352135 23191 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:42.354104 23404 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:42.356909 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:42.357499 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:42.368201 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:42.368769 22950 raft_consensus.cc:2468] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 0.
I20260709 13:07:42.369016 22949 raft_consensus.cc:2393] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 1: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:07:42.358425 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:42.354149 23419 raft_consensus.cc:2749] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:42.359120 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:07:42.370958 22826 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:42.361961 23073 tablet_bootstrap.cc:492] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:42.373037 23073 ts_tablet_manager.cc:1403] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.085s	user 0.032s	sys 0.035s
I20260709 13:07:42.375490 23073 raft_consensus.cc:359] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.376471 23073 raft_consensus.cc:385] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:42.376749 23073 raft_consensus.cc:740] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:42.377352 22843 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:07:42.378211 23404 raft_consensus.cc:2804] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:42.378607 23404 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:42.379170 23404 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:07:42.378639 22843 leader_election.cc:343] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:42.384729 22843 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:42.385504 23125 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:42.389483 23404 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.377345 23073 consensus_queue.cc:260] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.392128 23404 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:42.394483 23073 ts_tablet_manager.cc:1434] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.000s
I20260709 13:07:42.395509 23073 tablet_bootstrap.cc:492] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:42.401587 23073 tablet_bootstrap.cc:654] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:42.404088 22843 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:42.417315 23417 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.425155 23125 raft_consensus.cc:2749] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:42.425036 23417 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.423391 23339 consensus_queue.cc:1048] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:42.425698 23404 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:42.426510 23069 tablet_bootstrap.cc:492] T 3f1c9e3f9b244b0695775e91f28d67d7 P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:42.427052 23417 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:42.428615 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:07:42.430038 22920 leader_election.cc:343] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:42.427065 23069 ts_tablet_manager.cc:1403] T 3f1c9e3f9b244b0695775e91f28d67d7 P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.132s	user 0.043s	sys 0.037s
I20260709 13:07:42.432713 23125 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.433125 23125 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.433465 23069 raft_consensus.cc:359] T 3f1c9e3f9b244b0695775e91f28d67d7 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.434309 23069 raft_consensus.cc:385] T 3f1c9e3f9b244b0695775e91f28d67d7 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:42.434609 23069 raft_consensus.cc:740] T 3f1c9e3f9b244b0695775e91f28d67d7 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:42.435464 23125 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:42.435253 23069 consensus_queue.cc:260] T 3f1c9e3f9b244b0695775e91f28d67d7 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.433483 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:42.436406 22950 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:42.439815 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:42.440378 22951 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 0.
I20260709 13:07:42.442034 22843 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:07:42.442708 23191 raft_consensus.cc:2804] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:42.443041 23191 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:42.443331 23191 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:42.444557 23073 tablet_bootstrap.cc:492] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:42.445065 23073 ts_tablet_manager.cc:1403] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.050s	user 0.020s	sys 0.018s
I20260709 13:07:42.445211 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:07:42.445936 23022 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:42.448365 23073 raft_consensus.cc:359] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.448345 22776 catalog_manager.cc:5697] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:42.449352 23073 raft_consensus.cc:385] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:42.449689 23073 raft_consensus.cc:740] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:42.450434 23073 consensus_queue.cc:260] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.452419 23073 ts_tablet_manager.cc:1434] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:07:42.453325 23073 tablet_bootstrap.cc:492] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
W20260709 13:07:42.453908 22906 tablet.cc:2406] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:07:42.454962 23022 raft_consensus.cc:2468] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 1.
I20260709 13:07:42.457224 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:42.455853 23191 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.459180 23073 tablet_bootstrap.cc:654] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:42.459287 23191 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:42.460034 23339 consensus_queue.cc:1048] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:42.460525 23069 ts_tablet_manager.cc:1434] T 3f1c9e3f9b244b0695775e91f28d67d7 P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.029s	user 0.005s	sys 0.000s
I20260709 13:07:42.460480 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:42.461103 22951 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:42.461784 23069 tablet_bootstrap.cc:492] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:42.466820 22953 raft_consensus.cc:1275] T fab4388fa7ea49bda9fa070cf542577f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:07:42.470127 23419 consensus_queue.cc:1048] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:42.481312 23069 tablet_bootstrap.cc:654] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:42.482818 22951 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 1.
I20260709 13:07:42.484043 22843 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:07:42.484849 23191 raft_consensus.cc:2804] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:42.485265 23191 raft_consensus.cc:697] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:07:42.485975 23191 consensus_queue.cc:237] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.490747 22843 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:42.491882 23125 raft_consensus.cc:2804] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:42.492514 23125 raft_consensus.cc:697] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:07:42.493158 23404 raft_consensus.cc:2749] T abd5e7d435ca4afd9727e37b2ab31589 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:42.493651 23190 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.494343 23190 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.493697 23125 consensus_queue.cc:237] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:07:42.495127 22918 leader_election.cc:343] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:42.495582 22918 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:42.497426 23183 raft_consensus.cc:2749] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:42.499096 23069 tablet_bootstrap.cc:492] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:42.499580 23069 ts_tablet_manager.cc:1403] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.038s	user 0.009s	sys 0.002s
I20260709 13:07:42.503878 23190 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:42.507928 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:07:42.513077 22843 leader_election.cc:343] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:42.514532 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:42.515311 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:42.515910 22874 raft_consensus.cc:1275] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:07:42.517820 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:42.518553 22951 raft_consensus.cc:2468] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 0.
I20260709 13:07:42.519572 22992 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:42.521075 22950 raft_consensus.cc:2468] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 1.
I20260709 13:07:42.530189 23281 consensus_queue.cc:1048] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:07:42.537187 22843 leader_election.cc:343] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:42.537830 23069 raft_consensus.cc:359] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.527478 23418 raft_consensus.cc:2804] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:42.538615 23069 raft_consensus.cc:385] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:42.544658 23183 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.544895 23069 raft_consensus.cc:740] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:42.545121 23183 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.545599 23069 consensus_queue.cc:260] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.539815 23418 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:42.551357 22878 raft_consensus.cc:2468] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 0.
I20260709 13:07:42.551826 23073 tablet_bootstrap.cc:492] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:42.552119 23183 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:42.552253 23073 ts_tablet_manager.cc:1403] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.099s	user 0.020s	sys 0.021s
I20260709 13:07:42.550952 23418 raft_consensus.cc:3060] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:42.554087 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:07:42.555431 22920 leader_election.cc:343] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:42.554903 23073 raft_consensus.cc:359] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.556015 23073 raft_consensus.cc:385] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:42.556340 23073 raft_consensus.cc:740] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:42.557379 23073 consensus_queue.cc:260] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.558777 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:42.559417 23024 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 0.
I20260709 13:07:42.560634 22918 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:07:42.562047 23281 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.562695 23417 raft_consensus.cc:2804] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:42.562608 23281 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.563249 23417 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:42.563788 23417 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:42.569132 23281 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:42.598660 23183 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.613119 23073 ts_tablet_manager.cc:1434] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.061s	user 0.007s	sys 0.000s
I20260709 13:07:42.606312 22776 catalog_manager.cc:5697] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:42.666033 23069 ts_tablet_manager.cc:1434] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.166s	user 0.006s	sys 0.003s
I20260709 13:07:42.660513 23183 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.643596 23419 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.685915 23281 consensus_queue.cc:1048] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:42.573184 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:42.645417 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:42.688987 23183 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:42.690002 23069 tablet_bootstrap.cc:492] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:42.656391 23418 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.705175 23417 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.706044 23357 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.710661 23417 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:42.711890 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:07:42.712540 23026 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:42.689497 23419 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.699723 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:42.706568 23357 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.730015 23419 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:42.720772 22950 raft_consensus.cc:2393] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 1: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
W20260709 13:07:42.744756 22918 leader_election.cc:343] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:42.766345 23357 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:42.792320 23418 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:07:42.801487 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 121446us
W20260709 13:07:42.823025 22994 leader_election.cc:343] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:42.805013 23405 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:07:42.847105 22994 consensus_peers.cc:597] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:42.828367 23405 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.837145 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:42.837337 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:42.837347 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:42.862339 23026 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 1.
I20260709 13:07:42.868842 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:07:42.871932 22877 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 0.
I20260709 13:07:42.874003 22949 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 0.
I20260709 13:07:42.877225 23405 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:07:42.853384 23494 log.cc:927] Time spent T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 13:07:42.883527 23069 tablet_bootstrap.cc:654] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:42.885954 22918 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:42.888103 22994 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:42.867659 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:42.891824 23499 raft_consensus.cc:2804] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:42.892292 23499 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:42.892702 23499 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:42.896432 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:07:42.896860 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:42.897229 22878 raft_consensus.cc:3060] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:42.897631 22954 raft_consensus.cc:3060] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:42.904749 22954 raft_consensus.cc:2468] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
W20260709 13:07:42.909222 22918 leader_election.cc:343] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:42.911640 23073 tablet_bootstrap.cc:492] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:42.909332 23500 raft_consensus.cc:2804] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:42.913295 23500 raft_consensus.cc:697] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:07:42.914119 23500 consensus_queue.cc:237] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:07:42.921329 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 80552.9us
I20260709 13:07:42.928049 22992 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:07:42.910930 23506 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:42.934415 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:42.935088 22879 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 0.
I20260709 13:07:42.934803 23506 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:42.941270 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:42.942097 22951 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 0.
I20260709 13:07:42.958134 23496 raft_consensus.cc:2749] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:07:42.961556 22920 leader_election.cc:343] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:42.962160 23073 tablet_bootstrap.cc:654] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
W20260709 13:07:42.963409 22920 leader_election.cc:343] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:42.967386 22843 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:07:42.972236 23191 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:42.973393 23191 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:42.973817 23191 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:42.970352 22920 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
W20260709 13:07:42.980877 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 48378.2us
I20260709 13:07:42.983575 23504 consensus_queue.cc:1048] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:42.986698 22992 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:42.987565 23504 raft_consensus.cc:2804] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:42.988140 23504 raft_consensus.cc:697] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:07:43.001866 23506 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:43.005803 23417 raft_consensus.cc:2749] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:43.018445 23496 consensus_queue.cc:1048] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:07:43.033591 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 70223.5us
I20260709 13:07:43.033681 23191 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.006917 23499 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.038249 22920 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:43.042335 23191 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:43.044040 23501 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:43.045083 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:43.044457 23501 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:43.054705 23496 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.055233 23496 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.056964 23496 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:43.045719 22955 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 0.
I20260709 13:07:43.049084 23501 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:43.085628 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:43.093605 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:43.093976 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:43.094734 23021 raft_consensus.cc:2393] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 1: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:07:43.098726 23069 tablet_bootstrap.cc:492] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:43.099210 23069 ts_tablet_manager.cc:1403] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.416s	user 0.018s	sys 0.018s
I20260709 13:07:43.102171 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:07:43.102743 22875 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:43.088889 23499 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:43.112030 22875 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:07:43.112630 22878 raft_consensus.cc:2468] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
W20260709 13:07:43.114717 22843 leader_election.cc:343] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:43.067731 23504 consensus_queue.cc:237] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.135577 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:43.134445 22843 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:07:43.152364 23404 raft_consensus.cc:2764] T 3f1c9e3f9b244b0695775e91f28d67d7 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:07:43.126490 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:07:43.154999 23125 consensus_queue.cc:1048] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 13:07:43.166051 23069 raft_consensus.cc:359] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.166828 23069 raft_consensus.cc:385] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:43.167119 23069 raft_consensus.cc:740] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:43.167778 23069 consensus_queue.cc:260] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.160363 23501 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.180883 23511 consensus_queue.cc:1048] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:43.182199 23069 ts_tablet_manager.cc:1434] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.083s	user 0.005s	sys 0.003s
I20260709 13:07:43.183419 23073 tablet_bootstrap.cc:492] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:43.183869 23073 ts_tablet_manager.cc:1403] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.272s	user 0.016s	sys 0.010s
I20260709 13:07:43.185410 23069 tablet_bootstrap.cc:492] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:43.184171 23507 consensus_queue.cc:1048] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 13:07:43.188221 22946 raft_consensus.cc:2393] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 1: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:07:43.192562 23501 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:43.193965 23069 tablet_bootstrap.cc:654] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:43.195225 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:43.197841 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:07:43.198629 22880 raft_consensus.cc:2393] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 1: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:07:43.199146 23073 raft_consensus.cc:359] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.201051 23073 raft_consensus.cc:385] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:43.201357 23073 raft_consensus.cc:740] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:43.202431 23073 consensus_queue.cc:260] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.204476 23073 ts_tablet_manager.cc:1434] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.020s	user 0.003s	sys 0.003s
I20260709 13:07:43.207147 23073 tablet_bootstrap.cc:492] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:43.219676 23073 tablet_bootstrap.cc:654] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:43.222160 23417 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.222442 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:43.222867 23417 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.223297 22954 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:43.225512 23417 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:43.228396 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:07:43.230595 22920 leader_election.cc:343] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:43.255128 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:07:43.258299 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:43.258643 22778 catalog_manager.cc:5697] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:43.265089 23026 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:43.273694 23073 tablet_bootstrap.cc:492] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:43.274292 23073 ts_tablet_manager.cc:1403] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.067s	user 0.016s	sys 0.011s
I20260709 13:07:43.277284 23073 raft_consensus.cc:359] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.278127 23073 raft_consensus.cc:385] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:43.278469 23073 raft_consensus.cc:740] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
W20260709 13:07:43.278604 22918 leader_election.cc:343] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:43.279062 22918 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:43.279279 23073 consensus_queue.cc:260] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.280140 23498 raft_consensus.cc:2749] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:07:43.282101 22918 leader_election.cc:343] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:43.282615 23073 ts_tablet_manager.cc:1434] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:07:43.282552 22918 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:43.283794 23073 tablet_bootstrap.cc:492] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:43.284065 23498 raft_consensus.cc:2749] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:43.284013 23191 consensus_queue.cc:1048] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:43.288766 22994 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:43.289480 23337 raft_consensus.cc:2804] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:43.290129 23337 raft_consensus.cc:697] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:07:43.290993 23337 consensus_queue.cc:237] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.293135 23073 tablet_bootstrap.cc:654] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:43.304947 22994 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:07:43.306553 23499 raft_consensus.cc:2749] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:07:43.289505 22843 leader_election.cc:343] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:43.310132 23191 consensus_queue.cc:1048] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:43.321882 22872 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:43.323096 22954 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:07:43.337383 23191 consensus_queue.cc:1048] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:43.369417 22843 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:43.370608 23125 raft_consensus.cc:2749] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:07:43.439733 22777 catalog_manager.cc:5697] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 1 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:43.437523 22776 catalog_manager.cc:5697] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:43.465761 23073 tablet_bootstrap.cc:492] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:43.466257 23073 ts_tablet_manager.cc:1403] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.183s	user 0.068s	sys 0.092s
I20260709 13:07:43.466421 23069 tablet_bootstrap.cc:492] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:43.466912 23069 ts_tablet_manager.cc:1403] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.282s	user 0.078s	sys 0.139s
I20260709 13:07:43.469388 23073 raft_consensus.cc:359] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.470125 23073 raft_consensus.cc:385] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:43.470402 23073 raft_consensus.cc:740] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:43.471014 23073 consensus_queue.cc:260] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.472525 23073 ts_tablet_manager.cc:1434] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:07:43.473515 23073 tablet_bootstrap.cc:492] T 4c0bea78fb2a40ef90cff914c5c6e250 P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:43.479984 23073 tablet_bootstrap.cc:654] T 4c0bea78fb2a40ef90cff914c5c6e250 P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:43.484733 23069 raft_consensus.cc:359] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.485492 23069 raft_consensus.cc:385] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:43.485738 23069 raft_consensus.cc:740] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:43.503012 23069 consensus_queue.cc:260] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.504087 23500 consensus_queue.cc:1048] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:43.510370 23069 ts_tablet_manager.cc:1434] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.043s	user 0.007s	sys 0.001s
I20260709 13:07:43.511830 23337 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.512868 23073 tablet_bootstrap.cc:492] T 4c0bea78fb2a40ef90cff914c5c6e250 P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:43.513341 23073 ts_tablet_manager.cc:1403] T 4c0bea78fb2a40ef90cff914c5c6e250 P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.040s	user 0.023s	sys 0.015s
I20260709 13:07:43.514446 23337 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.515880 23073 raft_consensus.cc:359] T 4c0bea78fb2a40ef90cff914c5c6e250 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.516558 23073 raft_consensus.cc:385] T 4c0bea78fb2a40ef90cff914c5c6e250 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:43.516863 23073 raft_consensus.cc:740] T 4c0bea78fb2a40ef90cff914c5c6e250 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:43.517454 23073 consensus_queue.cc:260] T 4c0bea78fb2a40ef90cff914c5c6e250 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.515002 23069 tablet_bootstrap.cc:492] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:43.520422 23073 ts_tablet_manager.cc:1434] T 4c0bea78fb2a40ef90cff914c5c6e250 P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:07:43.521344 23073 tablet_bootstrap.cc:492] T 0fa29d11409a4d209411116a7ef8af04 P f3c799b1249e40119d2a5864231c6c75: Bootstrap starting.
I20260709 13:07:43.527189 23069 tablet_bootstrap.cc:654] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:43.538539 23073 tablet_bootstrap.cc:654] T 0fa29d11409a4d209411116a7ef8af04 P f3c799b1249e40119d2a5864231c6c75: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:43.554970 23337 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:43.557809 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:43.558465 22872 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 0.
I20260709 13:07:43.563995 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:43.564851 22946 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 0.
I20260709 13:07:43.563076 22994 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:43.566917 23516 raft_consensus.cc:2804] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:43.567435 23516 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:43.567934 23516 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:07:43.592847 22920 consensus_peers.cc:597] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:43.592617 23496 consensus_queue.cc:1048] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:43.597298 23073 tablet_bootstrap.cc:492] T 0fa29d11409a4d209411116a7ef8af04 P f3c799b1249e40119d2a5864231c6c75: No bootstrap required, opened a new log
I20260709 13:07:43.597918 23073 ts_tablet_manager.cc:1403] T 0fa29d11409a4d209411116a7ef8af04 P f3c799b1249e40119d2a5864231c6c75: Time spent bootstrapping tablet: real 0.077s	user 0.038s	sys 0.021s
I20260709 13:07:43.600322 23073 raft_consensus.cc:359] T 0fa29d11409a4d209411116a7ef8af04 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.601018 23073 raft_consensus.cc:385] T 0fa29d11409a4d209411116a7ef8af04 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:43.601286 23073 raft_consensus.cc:740] T 0fa29d11409a4d209411116a7ef8af04 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Initialized, Role: FOLLOWER
I20260709 13:07:43.601989 23073 consensus_queue.cc:260] T 0fa29d11409a4d209411116a7ef8af04 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.602169 23516 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.604552 23073 ts_tablet_manager.cc:1434] T 0fa29d11409a4d209411116a7ef8af04 P f3c799b1249e40119d2a5864231c6c75: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 13:07:43.605563 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:43.606173 22954 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:43.608985 23069 tablet_bootstrap.cc:492] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:43.609437 23069 ts_tablet_manager.cc:1403] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.095s	user 0.020s	sys 0.009s
I20260709 13:07:43.611816 23069 raft_consensus.cc:359] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.612424 23069 raft_consensus.cc:385] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:43.614441 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:07:43.615005 22872 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:43.617421 23069 raft_consensus.cc:740] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:43.618098 23069 consensus_queue.cc:260] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.619132 23496 consensus_queue.cc:1048] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:43.620517 23069 ts_tablet_manager.cc:1434] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:07:43.615406 23516 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:43.614770 22954 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:07:43.620644 22872 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:07:43.623418 23069 tablet_bootstrap.cc:492] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:43.648322 23069 tablet_bootstrap.cc:654] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:43.638258 22992 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:43.652103 23504 raft_consensus.cc:2804] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:43.652585 23504 raft_consensus.cc:697] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:07:43.654547 23504 consensus_queue.cc:237] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.637482 22776 catalog_manager.cc:5697] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:43.670604 23069 tablet_bootstrap.cc:492] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:43.671103 23069 ts_tablet_manager.cc:1403] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.048s	user 0.008s	sys 0.000s
W20260709 13:07:43.671201 22826 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:43.677170 23069 raft_consensus.cc:359] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.677969 23069 raft_consensus.cc:385] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:43.678275 23069 raft_consensus.cc:740] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:43.679098 23069 consensus_queue.cc:260] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.681569 23069 ts_tablet_manager.cc:1434] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:07:43.684031 23069 tablet_bootstrap.cc:492] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:43.689936 23069 tablet_bootstrap.cc:654] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:43.706218 23504 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.706724 23504 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.708724 23504 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:43.709522 23069 tablet_bootstrap.cc:492] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:43.709991 23069 ts_tablet_manager.cc:1403] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.026s	user 0.012s	sys 0.010s
I20260709 13:07:43.709847 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:43.712502 23069 raft_consensus.cc:359] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.713160 23069 raft_consensus.cc:385] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:43.713430 23069 raft_consensus.cc:740] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:43.714040 23069 consensus_queue.cc:260] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.714123 22778 catalog_manager.cc:5697] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:43.714515 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:43.715132 22947 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 0.
I20260709 13:07:43.716115 23069 ts_tablet_manager.cc:1434] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:07:43.716220 22992 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:43.716950 23502 raft_consensus.cc:2804] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:43.717347 23502 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:07:43.717587 22994 leader_election.cc:343] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:43.717072 23069 tablet_bootstrap.cc:492] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f: Bootstrap starting.
I20260709 13:07:43.717687 23502 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:43.727025 23069 tablet_bootstrap.cc:654] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f: Neither blocks nor log segments found. Creating new log.
I20260709 13:07:43.728438 23502 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.729475 23507 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.729959 23507 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.731010 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:07:43.731241 23502 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:43.732141 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:07:43.732292 22994 leader_election.cc:343] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:07:43.732754 22947 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:43.733304 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:43.734048 23507 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:43.735683 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:43.737169 22843 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:43.738401 23507 raft_consensus.cc:2749] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:43.740077 22947 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:07:43.741205 22992 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:07:43.742975 23502 raft_consensus.cc:2804] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:43.746268 23069 tablet_bootstrap.cc:492] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f: No bootstrap required, opened a new log
I20260709 13:07:43.746721 23069 ts_tablet_manager.cc:1403] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f: Time spent bootstrapping tablet: real 0.030s	user 0.015s	sys 0.007s
I20260709 13:07:43.748607 23504 consensus_queue.cc:1048] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:43.749133 23069 raft_consensus.cc:359] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.749852 23069 raft_consensus.cc:385] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:07:43.750097 23069 raft_consensus.cc:740] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Initialized, Role: FOLLOWER
I20260709 13:07:43.750661 23069 consensus_queue.cc:260] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.752544 23069 ts_tablet_manager.cc:1434] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:07:43.752730 23502 raft_consensus.cc:697] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:07:43.753621 23502 consensus_queue.cc:237] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.762300 23504 consensus_queue.cc:1048] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:43.775672 22778 catalog_manager.cc:5697] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:43.831720 23507 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.832101 23507 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.833822 23507 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:43.834345 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:43.834597 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:43.835004 22947 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 0.
I20260709 13:07:43.835325 23020 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 0.
I20260709 13:07:43.836015 22843 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:07:43.836726 23507 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:43.837072 23507 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:43.837427 23507 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:43.843719 23507 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.845192 23507 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:43.845999 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:43.846328 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:07:43.846465 22947 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:43.846868 23020 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:43.851056 22947 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 1.
I20260709 13:07:43.851127 23020 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 1.
I20260709 13:07:43.852195 22843 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:43.852854 23507 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:43.853250 23507 raft_consensus.cc:697] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:07:43.854110 23507 consensus_queue.cc:237] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.865811 22778 catalog_manager.cc:5697] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:43.875998 23502 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.876785 23502 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.878852 23502 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:43.879186 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:43.879848 22873 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 0.
I20260709 13:07:43.879385 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:43.880709 22947 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 0.
I20260709 13:07:43.880919 22994 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:43.881675 23502 raft_consensus.cc:2804] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:07:43.882081 23502 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:43.882391 23502 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:43.888047 23502 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.890058 23502 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:43.890677 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:07:43.891228 22873 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:43.891273 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:43.891875 22947 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:43.895712 22873 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:07:43.896356 22947 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:07:43.896842 22994 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:43.897547 23502 raft_consensus.cc:2804] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:07:43.897955 23502 raft_consensus.cc:697] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:07:43.898658 23502 consensus_queue.cc:237] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.908492 22778 catalog_manager.cc:5697] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:07:43.958483 23507 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.959075 23507 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.960739 23507 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:43.960839 23502 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:43.961359 23502 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:43.962770 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:43.963651 23502 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:43.964617 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:07:43.965432 22873 raft_consensus.cc:2393] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 1: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:07:43.968438 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:43.968868 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:43.969242 22946 raft_consensus.cc:2393] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 1: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:07:43.969601 22947 raft_consensus.cc:2393] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 1: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:07:43.970175 22992 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:07:43.970466 22843 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:07:43.970801 23502 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P f3c799b1249e40119d2a5864231c6c75 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:43.971108 23507 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:43.975577 23507 raft_consensus.cc:2749] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:43.975579 23502 raft_consensus.cc:2749] T 4c0bea78fb2a40ef90cff914c5c6e250 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:07:44.016340 22872 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:07:44.115649 23504 consensus_queue.cc:1048] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:44.127315 23504 consensus_queue.cc:1048] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:44.177444 23516 consensus_queue.cc:1048] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:44.199433 23504 consensus_queue.cc:1048] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:07:44.252367 23191 consensus_queue.cc:1048] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:44.268142 23191 consensus_queue.cc:1048] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:44.320303 23504 consensus_queue.cc:1048] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:44.332589 23504 consensus_queue.cc:1048] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:07:44.706664 22826 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:07:45.194621 23191 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:07:45.195122 23191 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:45.196978 23191 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:45.197669 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:07:45.198029 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:07:45.198369 22954 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 1.
I20260709 13:07:45.198655 23020 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 1.
I20260709 13:07:45.199767 22843 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:07:45.200430 23191 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:07:45.200758 23191 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:07:45.201125 23191 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:45.205399 23191 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:45.206749 23191 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:45.207521 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:07:45.208091 22954 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:45.208060 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:07:45.208819 23020 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:07:45.213284 22954 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:07:45.215445 22843 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:07:45.216408 23506 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:07:45.216917 23506 raft_consensus.cc:697] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:07:45.217074 23020 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:07:45.217810 23506 consensus_queue.cc:237] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:07:45.231729 22777 catalog_manager.cc:5697] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 0 to 2, leader changed from <none> to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 2 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:45.651212 23025 raft_consensus.cc:1275] T 4c0bea78fb2a40ef90cff914c5c6e250 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:07:45.652446 23506 consensus_queue.cc:1048] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:45.665421 22947 raft_consensus.cc:1275] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:07:45.674275 23191 consensus_queue.cc:1048] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:07:45.798611 21756 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:07:45.838896 23579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:45.839133 23580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:07:45.839792 23582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:07:45.841112 21756 server_base.cc:1061] running on GCE node
I20260709 13:07:45.842512 21756 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:07:45.842764 21756 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:07:45.842993 21756 hybrid_clock.cc:648] HybridClock initialized: now 1783602465842975 us; error 0 us; skew 500 ppm
I20260709 13:07:45.843520 21756 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:07:45.846549 21756 webserver.cc:533] Webserver started at http://127.21.63.4:39705/ using document root <none> and password file <none>
I20260709 13:07:45.847044 21756 fs_manager.cc:362] Metadata directory not provided
I20260709 13:07:45.847273 21756 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:07:45.847693 21756 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:07:45.849308 21756 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-3-root/instance:
uuid: "5b346767a89745fa82928a5dac9cc0b2"
format_stamp: "Formatted at 2026-07-09 13:07:45 on dist-test-slave-c6mt"
I20260709 13:07:45.854290 21756 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:07:45.857935 23587 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:45.858883 21756 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:07:45.859239 21756 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-3-root
uuid: "5b346767a89745fa82928a5dac9cc0b2"
format_stamp: "Formatted at 2026-07-09 13:07:45 on dist-test-slave-c6mt"
I20260709 13:07:45.859603 21756 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:07:45.894377 21756 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:07:45.895879 21756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:07:45.897495 21756 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:07:45.899982 21756 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:07:45.900178 21756 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:45.900383 21756 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:07:45.900542 21756 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:07:45.954596 21756 rpc_server.cc:307] RPC server started. Bound to: 127.21.63.4:44043
I20260709 13:07:45.954725 23649 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.63.4:44043 every 8 connection(s)
I20260709 13:07:45.959592 21756 auto_leader_rebalancer-test.cc:384] add a tserver: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:07:45.979104 23650 heartbeater.cc:344] Connected to a master server at 127.21.63.62:33399
I20260709 13:07:45.979605 23650 heartbeater.cc:461] Registering TS with master...
I20260709 13:07:45.980417 23650 heartbeater.cc:507] Master 127.21.63.62:33399 requested a full tablet report, sending...
I20260709 13:07:45.982352 22776 ts_manager.cc:194] Registered new tserver with Master: 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:07:45.983992 22776 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:48906
I20260709 13:07:46.856539 22879 consensus_queue.cc:237] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:46.865948 22947 raft_consensus.cc:1275] T cdebaf291e5d4ae9814a80ce81ee19fe P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:07:46.867856 23654 consensus_queue.cc:1048] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:46.868780 23020 raft_consensus.cc:1275] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:07:46.870752 23654 consensus_queue.cc:1048] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:07:46.883492 22844 consensus_peers.cc:597] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f -> Peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Couldn't send request to peer 5b346767a89745fa82928a5dac9cc0b2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cdebaf291e5d4ae9814a80ce81ee19fe. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:46.886685 23654 raft_consensus.cc:2955] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:46.895128 22947 raft_consensus.cc:2955] T cdebaf291e5d4ae9814a80ce81ee19fe P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:46.906945 22879 consensus_queue.cc:237] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:46.906638 23025 raft_consensus.cc:2955] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:46.913470 22776 catalog_manager.cc:5697] T cdebaf291e5d4ae9814a80ce81ee19fe P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
W20260709 13:07:46.943324 22844 consensus_peers.cc:597] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f -> Peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Couldn't send request to peer 5b346767a89745fa82928a5dac9cc0b2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1120807254f5444997e16f6a7feece4d. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:46.945015 22947 raft_consensus.cc:1275] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:07:46.948050 23662 consensus_queue.cc:1048] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:07:46.978276 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 47171.8us
I20260709 13:07:46.978813 23654 raft_consensus.cc:2955] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:46.980578 22946 raft_consensus.cc:2955] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:46.982340 23020 raft_consensus.cc:1275] T 1120807254f5444997e16f6a7feece4d P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:07:46.984570 23507 consensus_queue.cc:1048] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:46.986783 23650 heartbeater.cc:499] Master 127.21.63.62:33399 was elected leader, sending a full tablet report...
I20260709 13:07:47.000155 23020 raft_consensus.cc:2955] T 1120807254f5444997e16f6a7feece4d P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:47.014159 22778 catalog_manager.cc:5697] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:47.029438 23020 consensus_queue.cc:237] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:47.038691 22872 raft_consensus.cc:1275] T 3f1c9e3f9b244b0695775e91f28d67d7 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:07:47.039232 22946 raft_consensus.cc:1275] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:07:47.040613 23666 consensus_queue.cc:1048] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:47.041546 23667 consensus_queue.cc:1048] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:47.054106 23667 raft_consensus.cc:2955] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:47.055431 22946 raft_consensus.cc:2955] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:47.056957 22879 raft_consensus.cc:2955] T 3f1c9e3f9b244b0695775e91f28d67d7 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
W20260709 13:07:47.058434 22993 consensus_peers.cc:597] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 -> Peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Couldn't send request to peer 5b346767a89745fa82928a5dac9cc0b2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3f1c9e3f9b244b0695775e91f28d67d7. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:47.075817 22776 catalog_manager.cc:5697] T 3f1c9e3f9b244b0695775e91f28d67d7 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:47.115984 22946 consensus_queue.cc:237] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:47.124469 23020 raft_consensus.cc:1275] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:07:47.126003 22879 raft_consensus.cc:1275] T 0c4ecd16880043f4a75006a3da4717bf P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:07:47.126639 23567 consensus_queue.cc:1048] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:47.127887 23498 consensus_queue.cc:1048] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:47.148058 23498 raft_consensus.cc:2955] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:47.152343 23025 raft_consensus.cc:2955] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
W20260709 13:07:47.153683 22919 consensus_peers.cc:597] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Couldn't send request to peer 5b346767a89745fa82928a5dac9cc0b2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0c4ecd16880043f4a75006a3da4717bf. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:47.164110 22879 raft_consensus.cc:2955] T 0c4ecd16880043f4a75006a3da4717bf P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:47.180814 22775 catalog_manager.cc:5697] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New cstate: current_term: 1 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:47.354368 23685 ts_tablet_manager.cc:933] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2: Initiating tablet copy from peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929)
I20260709 13:07:47.356714 23685 tablet_copy_client.cc:323] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.63.1:33929
I20260709 13:07:47.377112 22890 tablet_copy_service.cc:140] P 0a407504ed1741bf89a276558adf4c0f: Received BeginTabletCopySession request for tablet cdebaf291e5d4ae9814a80ce81ee19fe from peer 5b346767a89745fa82928a5dac9cc0b2 ({username='slave'} at 127.0.0.1:53724)
I20260709 13:07:47.377715 22890 tablet_copy_service.cc:161] P 0a407504ed1741bf89a276558adf4c0f: Beginning new tablet copy session on tablet cdebaf291e5d4ae9814a80ce81ee19fe from peer 5b346767a89745fa82928a5dac9cc0b2 at {username='slave'} at 127.0.0.1:53724: session id = 5b346767a89745fa82928a5dac9cc0b2-cdebaf291e5d4ae9814a80ce81ee19fe
I20260709 13:07:47.386386 22890 tablet_copy_source_session.cc:215] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:07:47.390506 23685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdebaf291e5d4ae9814a80ce81ee19fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:47.406991 23685 tablet_copy_client.cc:806] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 0 data blocks...
I20260709 13:07:47.407783 23685 tablet_copy_client.cc:670] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:07:47.413118 23685 tablet_copy_client.cc:538] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:07:47.423913 23685 tablet_bootstrap.cc:492] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap starting.
I20260709 13:07:47.447041 23685 tablet_bootstrap.cc:492] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:47.448033 23685 tablet_bootstrap.cc:492] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap complete.
I20260709 13:07:47.448705 23685 ts_tablet_manager.cc:1403] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2: Time spent bootstrapping tablet: real 0.025s	user 0.025s	sys 0.000s
I20260709 13:07:47.451586 23685 raft_consensus.cc:359] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:47.452422 23685 raft_consensus.cc:740] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Initialized, Role: LEARNER
I20260709 13:07:47.453063 23685 consensus_queue.cc:260] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:47.456009 23685 ts_tablet_manager.cc:1434] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:07:47.458617 22890 tablet_copy_service.cc:342] P 0a407504ed1741bf89a276558adf4c0f: Request end of tablet copy session 5b346767a89745fa82928a5dac9cc0b2-cdebaf291e5d4ae9814a80ce81ee19fe received from {username='slave'} at 127.0.0.1:53724
I20260709 13:07:47.459074 22890 tablet_copy_service.cc:434] P 0a407504ed1741bf89a276558adf4c0f: ending tablet copy session 5b346767a89745fa82928a5dac9cc0b2-cdebaf291e5d4ae9814a80ce81ee19fe on tablet cdebaf291e5d4ae9814a80ce81ee19fe with peer 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:07:47.505927 23685 ts_tablet_manager.cc:933] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2: Initiating tablet copy from peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929)
I20260709 13:07:47.508313 23685 tablet_copy_client.cc:323] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.63.1:33929
I20260709 13:07:47.509955 22890 tablet_copy_service.cc:140] P 0a407504ed1741bf89a276558adf4c0f: Received BeginTabletCopySession request for tablet 1120807254f5444997e16f6a7feece4d from peer 5b346767a89745fa82928a5dac9cc0b2 ({username='slave'} at 127.0.0.1:53724)
I20260709 13:07:47.510497 22890 tablet_copy_service.cc:161] P 0a407504ed1741bf89a276558adf4c0f: Beginning new tablet copy session on tablet 1120807254f5444997e16f6a7feece4d from peer 5b346767a89745fa82928a5dac9cc0b2 at {username='slave'} at 127.0.0.1:53724: session id = 5b346767a89745fa82928a5dac9cc0b2-1120807254f5444997e16f6a7feece4d
I20260709 13:07:47.515375 22890 tablet_copy_source_session.cc:215] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:07:47.518100 23685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1120807254f5444997e16f6a7feece4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:47.530726 23685 tablet_copy_client.cc:806] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 0 data blocks...
I20260709 13:07:47.531287 23685 tablet_copy_client.cc:670] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:07:47.534404 23685 tablet_copy_client.cc:538] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:07:47.541405 23685 tablet_bootstrap.cc:492] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap starting.
I20260709 13:07:47.560451 23685 tablet_bootstrap.cc:492] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:47.561314 23685 tablet_bootstrap.cc:492] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap complete.
I20260709 13:07:47.561949 23685 ts_tablet_manager.cc:1403] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2: Time spent bootstrapping tablet: real 0.021s	user 0.019s	sys 0.000s
I20260709 13:07:47.564175 23685 raft_consensus.cc:359] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:47.564944 23685 raft_consensus.cc:740] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Initialized, Role: LEARNER
I20260709 13:07:47.565546 23685 consensus_queue.cc:260] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:47.567633 23685 ts_tablet_manager.cc:1434] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:07:47.570303 22890 tablet_copy_service.cc:342] P 0a407504ed1741bf89a276558adf4c0f: Request end of tablet copy session 5b346767a89745fa82928a5dac9cc0b2-1120807254f5444997e16f6a7feece4d received from {username='slave'} at 127.0.0.1:53724
I20260709 13:07:47.570703 22890 tablet_copy_service.cc:434] P 0a407504ed1741bf89a276558adf4c0f: ending tablet copy session 5b346767a89745fa82928a5dac9cc0b2-1120807254f5444997e16f6a7feece4d on tablet 1120807254f5444997e16f6a7feece4d with peer 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:07:47.573714 23685 ts_tablet_manager.cc:933] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2: Initiating tablet copy from peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:47.590154 23685 tablet_copy_client.cc:323] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.63.3:42499
I20260709 13:07:47.601454 23039 tablet_copy_service.cc:140] P f3c799b1249e40119d2a5864231c6c75: Received BeginTabletCopySession request for tablet 3f1c9e3f9b244b0695775e91f28d67d7 from peer 5b346767a89745fa82928a5dac9cc0b2 ({username='slave'} at 127.0.0.1:32794)
I20260709 13:07:47.601985 23039 tablet_copy_service.cc:161] P f3c799b1249e40119d2a5864231c6c75: Beginning new tablet copy session on tablet 3f1c9e3f9b244b0695775e91f28d67d7 from peer 5b346767a89745fa82928a5dac9cc0b2 at {username='slave'} at 127.0.0.1:32794: session id = 5b346767a89745fa82928a5dac9cc0b2-3f1c9e3f9b244b0695775e91f28d67d7
I20260709 13:07:47.607106 23039 tablet_copy_source_session.cc:215] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:07:47.610069 23685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f1c9e3f9b244b0695775e91f28d67d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:47.635959 23685 tablet_copy_client.cc:806] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 0 data blocks...
I20260709 13:07:47.636505 23685 tablet_copy_client.cc:670] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:07:47.640249 23685 tablet_copy_client.cc:538] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:07:47.647990 23685 tablet_bootstrap.cc:492] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap starting.
I20260709 13:07:47.663839 23685 tablet_bootstrap.cc:492] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:47.664467 23685 tablet_bootstrap.cc:492] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap complete.
I20260709 13:07:47.664955 23685 ts_tablet_manager.cc:1403] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2: Time spent bootstrapping tablet: real 0.017s	user 0.010s	sys 0.004s
I20260709 13:07:47.666618 23685 raft_consensus.cc:359] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:47.667122 23685 raft_consensus.cc:740] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Initialized, Role: LEARNER
I20260709 13:07:47.667568 23685 consensus_queue.cc:260] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:47.669323 23685 ts_tablet_manager.cc:1434] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:07:47.670964 23039 tablet_copy_service.cc:342] P f3c799b1249e40119d2a5864231c6c75: Request end of tablet copy session 5b346767a89745fa82928a5dac9cc0b2-3f1c9e3f9b244b0695775e91f28d67d7 received from {username='slave'} at 127.0.0.1:32794
I20260709 13:07:47.671391 23039 tablet_copy_service.cc:434] P f3c799b1249e40119d2a5864231c6c75: ending tablet copy session 5b346767a89745fa82928a5dac9cc0b2-3f1c9e3f9b244b0695775e91f28d67d7 on tablet 3f1c9e3f9b244b0695775e91f28d67d7 with peer 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:07:47.715656 23685 ts_tablet_manager.cc:933] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2: Initiating tablet copy from peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:47.717307 23685 tablet_copy_client.cc:323] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.63.2:33309
I20260709 13:07:47.728827 22965 tablet_copy_service.cc:140] P aed3f089ec8b49af9c8cf5cdee76e8e7: Received BeginTabletCopySession request for tablet 0c4ecd16880043f4a75006a3da4717bf from peer 5b346767a89745fa82928a5dac9cc0b2 ({username='slave'} at 127.0.0.1:52222)
I20260709 13:07:47.729478 22965 tablet_copy_service.cc:161] P aed3f089ec8b49af9c8cf5cdee76e8e7: Beginning new tablet copy session on tablet 0c4ecd16880043f4a75006a3da4717bf from peer 5b346767a89745fa82928a5dac9cc0b2 at {username='slave'} at 127.0.0.1:52222: session id = 5b346767a89745fa82928a5dac9cc0b2-0c4ecd16880043f4a75006a3da4717bf
I20260709 13:07:47.735013 22965 tablet_copy_source_session.cc:215] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:07:47.738683 23685 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0c4ecd16880043f4a75006a3da4717bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:47.749728 23685 tablet_copy_client.cc:806] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 0 data blocks...
I20260709 13:07:47.750185 23685 tablet_copy_client.cc:670] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:07:47.754056 23685 tablet_copy_client.cc:538] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:07:47.764797 23685 tablet_bootstrap.cc:492] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap starting.
I20260709 13:07:47.787662 23685 tablet_bootstrap.cc:492] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:47.788564 23685 tablet_bootstrap.cc:492] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap complete.
I20260709 13:07:47.789170 23685 ts_tablet_manager.cc:1403] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2: Time spent bootstrapping tablet: real 0.025s	user 0.024s	sys 0.003s
I20260709 13:07:47.791486 23685 raft_consensus.cc:359] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:47.792255 23685 raft_consensus.cc:740] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Initialized, Role: LEARNER
I20260709 13:07:47.793257 23685 consensus_queue.cc:260] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:47.795101 23685 ts_tablet_manager.cc:1434] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2: Time spent starting tablet: real 0.006s	user 0.000s	sys 0.004s
I20260709 13:07:47.796609 22965 tablet_copy_service.cc:342] P aed3f089ec8b49af9c8cf5cdee76e8e7: Request end of tablet copy session 5b346767a89745fa82928a5dac9cc0b2-0c4ecd16880043f4a75006a3da4717bf received from {username='slave'} at 127.0.0.1:52222
I20260709 13:07:47.797128 22965 tablet_copy_service.cc:434] P aed3f089ec8b49af9c8cf5cdee76e8e7: ending tablet copy session 5b346767a89745fa82928a5dac9cc0b2-0c4ecd16880043f4a75006a3da4717bf on tablet 0c4ecd16880043f4a75006a3da4717bf with peer 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:07:47.880843 23625 raft_consensus.cc:1217] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:07:48.063938 23625 raft_consensus.cc:1217] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:07:48.103556 23625 raft_consensus.cc:1217] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:07:48.165755 23625 raft_consensus.cc:1217] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:07:48.331386 23691 raft_consensus.cc:1064] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
I20260709 13:07:48.333464 23691 consensus_queue.cc:237] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:07:48.339263 23625 raft_consensus.cc:1275] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:48.339411 23020 raft_consensus.cc:1275] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:48.340498 23691 consensus_queue.cc:1048] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:48.340852 22947 raft_consensus.cc:1275] T cdebaf291e5d4ae9814a80ce81ee19fe P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:48.347426 23691 consensus_queue.cc:1048] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:48.349025 23692 consensus_queue.cc:1048] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:48.363610 23692 raft_consensus.cc:2955] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:48.365633 22947 raft_consensus.cc:2955] T cdebaf291e5d4ae9814a80ce81ee19fe P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:48.368060 23020 raft_consensus.cc:2955] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:48.378540 23625 raft_consensus.cc:2955] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:48.384043 22776 catalog_manager.cc:5697] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 reported cstate change: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:48.427896 22872 consensus_queue.cc:237] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:07:48.433877 23625 raft_consensus.cc:1275] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:48.436122 23692 consensus_queue.cc:1048] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:48.443495 23654 raft_consensus.cc:2955] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:48.445200 23625 raft_consensus.cc:2955] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:48.458698 22777 catalog_manager.cc:5697] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: config changed from index 3 to 4, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:48.465006 22764 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet cdebaf291e5d4ae9814a80ce81ee19fe with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:07:48.469523 22943 tablet_service.cc:1558] Processing DeleteTablet for tablet cdebaf291e5d4ae9814a80ce81ee19fe with delete_type TABLET_DATA_TOMBSTONED (TS aed3f089ec8b49af9c8cf5cdee76e8e7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58302
I20260709 13:07:48.472100 23711 tablet_replica.cc:333] T cdebaf291e5d4ae9814a80ce81ee19fe P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:07:48.473135 23711 raft_consensus.cc:2243] T cdebaf291e5d4ae9814a80ce81ee19fe P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:48.473263 23020 raft_consensus.cc:1275] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:48.473922 23711 raft_consensus.cc:2272] T cdebaf291e5d4ae9814a80ce81ee19fe P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:48.479843 23711 ts_tablet_manager.cc:1916] T cdebaf291e5d4ae9814a80ce81ee19fe P aed3f089ec8b49af9c8cf5cdee76e8e7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:07:48.480266 23691 consensus_queue.cc:1048] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:48.497635 23711 ts_tablet_manager.cc:1929] T cdebaf291e5d4ae9814a80ce81ee19fe P aed3f089ec8b49af9c8cf5cdee76e8e7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:07:48.498380 23711 log.cc:1199] T cdebaf291e5d4ae9814a80ce81ee19fe P aed3f089ec8b49af9c8cf5cdee76e8e7: Deleting WAL directory at /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-1-root/wals/cdebaf291e5d4ae9814a80ce81ee19fe
I20260709 13:07:48.500243 22762 catalog_manager.cc:5028] TS aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): tablet cdebaf291e5d4ae9814a80ce81ee19fe (table test-workload [id=a42f2ccfbe484baca3019323a490c64a]) successfully deleted
I20260709 13:07:48.503391 23025 raft_consensus.cc:2955] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:48.514740 23691 raft_consensus.cc:1064] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
I20260709 13:07:48.517376 23691 consensus_queue.cc:237] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:07:48.539908 23625 raft_consensus.cc:1275] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:48.542032 23692 consensus_queue.cc:1048] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:48.542451 22954 raft_consensus.cc:1275] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:48.543301 23025 raft_consensus.cc:1275] T 1120807254f5444997e16f6a7feece4d P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:48.545508 23654 consensus_queue.cc:1048] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:48.549535 23691 consensus_queue.cc:1048] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:48.569597 23692 raft_consensus.cc:2955] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:48.569787 23625 raft_consensus.cc:2955] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:48.585026 22777 catalog_manager.cc:5697] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:48.585466 22947 raft_consensus.cc:2955] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:48.594712 23667 raft_consensus.cc:1064] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
I20260709 13:07:48.600766 23667 consensus_queue.cc:237] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:07:48.604326 22872 consensus_queue.cc:237] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:07:48.623131 23702 raft_consensus.cc:1064] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
I20260709 13:07:48.625792 23702 consensus_queue.cc:237] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:07:48.627110 22954 raft_consensus.cc:1275] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:48.639309 23624 raft_consensus.cc:1275] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:48.642194 22872 raft_consensus.cc:1275] T 3f1c9e3f9b244b0695775e91f28d67d7 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:48.643362 23624 raft_consensus.cc:1275] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:48.643877 23337 consensus_queue.cc:1048] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:48.647547 23654 consensus_queue.cc:1048] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:48.650810 23623 raft_consensus.cc:1275] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:48.652465 23567 consensus_queue.cc:1048] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:48.660195 22872 raft_consensus.cc:1275] T 0c4ecd16880043f4a75006a3da4717bf P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:48.662148 23691 raft_consensus.cc:2955] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:48.663167 23567 consensus_queue.cc:1048] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:48.664395 23623 raft_consensus.cc:2955] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:48.674475 23026 raft_consensus.cc:1275] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:48.679579 22764 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 1120807254f5444997e16f6a7feece4d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:07:48.688001 22775 catalog_manager.cc:5697] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: config changed from index 3 to 4, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:48.692055 23684 consensus_queue.cc:1048] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:48.702024 23716 consensus_queue.cc:1048] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 13:07:48.710031 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 113072us
I20260709 13:07:48.717288 22954 raft_consensus.cc:2955] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:48.726044 23019 tablet_service.cc:1558] Processing DeleteTablet for tablet 1120807254f5444997e16f6a7feece4d with delete_type TABLET_DATA_TOMBSTONED (TS f3c799b1249e40119d2a5864231c6c75 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33406
I20260709 13:07:48.688760 23723 consensus_queue.cc:1048] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 13:07:48.731251 23729 tablet_replica.cc:333] T 1120807254f5444997e16f6a7feece4d P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:07:48.732197 23729 raft_consensus.cc:2243] T 1120807254f5444997e16f6a7feece4d P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:48.732766 23729 pending_rounds.cc:70] T 1120807254f5444997e16f6a7feece4d P f3c799b1249e40119d2a5864231c6c75: Trying to abort 1 pending ops.
I20260709 13:07:48.737562 22947 raft_consensus.cc:1275] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:48.739094 23684 raft_consensus.cc:2955] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:48.740309 22872 raft_consensus.cc:2955] T 0c4ecd16880043f4a75006a3da4717bf P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:48.733083 23729 pending_rounds.cc:77] T 1120807254f5444997e16f6a7feece4d P f3c799b1249e40119d2a5864231c6c75: Aborting op as it isn't in flight: id { term: 1 index: 3 } timestamp: 7305635711040630784 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "1120807254f5444997e16f6a7feece4d" old_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } } new_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } } }
I20260709 13:07:48.747051 23729 raft_consensus.cc:2917] T 1120807254f5444997e16f6a7feece4d P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Aborting config change with OpId 1.3: Aborted: Op aborted
I20260709 13:07:48.747587 23729 raft_consensus.cc:2889] T 1120807254f5444997e16f6a7feece4d P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 13:07:48.748041 23729 raft_consensus.cc:2272] T 1120807254f5444997e16f6a7feece4d P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:48.752271 23624 raft_consensus.cc:2955] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:48.764573 23666 consensus_queue.cc:1048] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:48.772296 22777 catalog_manager.cc:5697] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:48.776383 23729 ts_tablet_manager.cc:1916] T 1120807254f5444997e16f6a7feece4d P f3c799b1249e40119d2a5864231c6c75: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:07:48.794373 23729 ts_tablet_manager.cc:1929] T 1120807254f5444997e16f6a7feece4d P f3c799b1249e40119d2a5864231c6c75: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:07:48.794797 23729 log.cc:1199] T 1120807254f5444997e16f6a7feece4d P f3c799b1249e40119d2a5864231c6c75: Deleting WAL directory at /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-2-root/wals/1120807254f5444997e16f6a7feece4d
I20260709 13:07:48.799582 23026 raft_consensus.cc:2955] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:48.808974 22762 catalog_manager.cc:5028] TS f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): tablet 1120807254f5444997e16f6a7feece4d (table test-workload [id=a42f2ccfbe484baca3019323a490c64a]) successfully deleted
I20260709 13:07:48.818596 22954 consensus_queue.cc:237] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:07:48.819430 23666 raft_consensus.cc:2955] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:48.831144 23624 raft_consensus.cc:2955] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:48.849478 22775 catalog_manager.cc:5697] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:48.850474 22873 raft_consensus.cc:2955] T 3f1c9e3f9b244b0695775e91f28d67d7 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:48.869122 23624 raft_consensus.cc:1275] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:48.870575 23684 consensus_queue.cc:1048] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:48.877454 23026 raft_consensus.cc:1275] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:48.883106 23567 raft_consensus.cc:2955] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:48.890159 22946 raft_consensus.cc:2955] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:48.890565 23623 raft_consensus.cc:2955] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:48.893781 23727 consensus_queue.cc:1048] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:07:48.905773 22762 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0c4ecd16880043f4a75006a3da4717bf with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:07:48.914362 23026 raft_consensus.cc:2955] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:48.916232 23020 consensus_queue.cc:237] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:07:48.925158 22777 catalog_manager.cc:5697] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: config changed from index 3 to 4, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New cstate: current_term: 1 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:48.938308 22954 raft_consensus.cc:1275] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:48.939043 23623 raft_consensus.cc:1275] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:48.941035 23725 consensus_queue.cc:1048] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:07:48.942548 23723 consensus_queue.cc:1048] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:48.952139 23666 raft_consensus.cc:2955] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:48.957125 22867 tablet_service.cc:1558] Processing DeleteTablet for tablet 0c4ecd16880043f4a75006a3da4717bf with delete_type TABLET_DATA_TOMBSTONED (TS 0a407504ed1741bf89a276558adf4c0f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:35344
I20260709 13:07:48.961269 23624 raft_consensus.cc:2955] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:48.973940 23733 tablet_replica.cc:333] T 0c4ecd16880043f4a75006a3da4717bf P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:07:48.963587 22946 raft_consensus.cc:2955] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:48.966954 22762 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3f1c9e3f9b244b0695775e91f28d67d7 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:07:48.980306 23733 raft_consensus.cc:2243] T 0c4ecd16880043f4a75006a3da4717bf P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:48.981171 23733 raft_consensus.cc:2272] T 0c4ecd16880043f4a75006a3da4717bf P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:48.984809 23733 ts_tablet_manager.cc:1916] T 0c4ecd16880043f4a75006a3da4717bf P 0a407504ed1741bf89a276558adf4c0f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:07:48.994664 22777 catalog_manager.cc:5697] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: config changed from index 3 to 4, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:49.002065 23733 ts_tablet_manager.cc:1929] T 0c4ecd16880043f4a75006a3da4717bf P 0a407504ed1741bf89a276558adf4c0f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:07:49.002516 23733 log.cc:1199] T 0c4ecd16880043f4a75006a3da4717bf P 0a407504ed1741bf89a276558adf4c0f: Deleting WAL directory at /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-0-root/wals/0c4ecd16880043f4a75006a3da4717bf
I20260709 13:07:49.004351 22764 catalog_manager.cc:5028] TS 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): tablet 0c4ecd16880043f4a75006a3da4717bf (table test-workload [id=a42f2ccfbe484baca3019323a490c64a]) successfully deleted
I20260709 13:07:49.007409 22867 tablet_service.cc:1558] Processing DeleteTablet for tablet 3f1c9e3f9b244b0695775e91f28d67d7 with delete_type TABLET_DATA_TOMBSTONED (TS 0a407504ed1741bf89a276558adf4c0f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:35344
I20260709 13:07:49.023018 23733 tablet_replica.cc:333] T 3f1c9e3f9b244b0695775e91f28d67d7 P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:07:49.027087 23733 raft_consensus.cc:2243] T 3f1c9e3f9b244b0695775e91f28d67d7 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:49.027688 23733 raft_consensus.cc:2272] T 3f1c9e3f9b244b0695775e91f28d67d7 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:49.030002 23733 ts_tablet_manager.cc:1916] T 3f1c9e3f9b244b0695775e91f28d67d7 P 0a407504ed1741bf89a276558adf4c0f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:07:49.041126 23733 ts_tablet_manager.cc:1929] T 3f1c9e3f9b244b0695775e91f28d67d7 P 0a407504ed1741bf89a276558adf4c0f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:07:49.041543 23733 log.cc:1199] T 3f1c9e3f9b244b0695775e91f28d67d7 P 0a407504ed1741bf89a276558adf4c0f: Deleting WAL directory at /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-0-root/wals/3f1c9e3f9b244b0695775e91f28d67d7
I20260709 13:07:49.043219 22764 catalog_manager.cc:5028] TS 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): tablet 3f1c9e3f9b244b0695775e91f28d67d7 (table test-workload [id=a42f2ccfbe484baca3019323a490c64a]) successfully deleted
I20260709 13:07:50.142357 22872 consensus_queue.cc:237] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:50.151298 22953 raft_consensus.cc:1275] T 392f5a2324d74a499e6baad745c941fd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:07:50.151297 23023 raft_consensus.cc:1275] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:07:50.152923 22844 consensus_peers.cc:597] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f -> Peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Couldn't send request to peer 5b346767a89745fa82928a5dac9cc0b2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 392f5a2324d74a499e6baad745c941fd. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:50.153870 23737 consensus_queue.cc:1048] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:50.155509 23738 consensus_queue.cc:1048] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:50.164183 23738 raft_consensus.cc:2955] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:50.168547 22953 raft_consensus.cc:2955] T 392f5a2324d74a499e6baad745c941fd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:50.176155 23023 raft_consensus.cc:2955] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:50.188738 22876 consensus_queue.cc:237] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:50.193099 22777 catalog_manager.cc:5697] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f reported cstate change: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:50.204481 22844 consensus_peers.cc:597] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f -> Peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Couldn't send request to peer 5b346767a89745fa82928a5dac9cc0b2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a424231ef7b4470bbd575564909a365e. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:50.205514 23023 raft_consensus.cc:1275] T a424231ef7b4470bbd575564909a365e P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:07:50.208634 22953 raft_consensus.cc:1275] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:07:50.209017 23716 consensus_queue.cc:1048] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:50.212360 23716 consensus_queue.cc:1048] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:50.221186 23691 raft_consensus.cc:2955] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:50.222992 23026 raft_consensus.cc:2955] T a424231ef7b4470bbd575564909a365e P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:50.236547 22953 raft_consensus.cc:2955] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:50.238379 22872 consensus_queue.cc:237] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:50.252751 22775 catalog_manager.cc:5697] T a424231ef7b4470bbd575564909a365e P f3c799b1249e40119d2a5864231c6c75 reported cstate change: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:50.253718 23026 raft_consensus.cc:1275] T 0fa29d11409a4d209411116a7ef8af04 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:07:50.254840 22844 consensus_peers.cc:597] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f -> Peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Couldn't send request to peer 5b346767a89745fa82928a5dac9cc0b2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0fa29d11409a4d209411116a7ef8af04. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:50.255452 23738 consensus_queue.cc:1048] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:50.258390 22953 raft_consensus.cc:1275] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:07:50.260500 23738 consensus_queue.cc:1048] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:50.281450 23738 raft_consensus.cc:2955] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:50.292253 23023 raft_consensus.cc:2955] T 0fa29d11409a4d209411116a7ef8af04 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:50.298643 22954 consensus_queue.cc:237] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:50.302340 22953 raft_consensus.cc:2955] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:50.307449 22778 catalog_manager.cc:5697] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:50.309885 22919 consensus_peers.cc:597] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Couldn't send request to peer 5b346767a89745fa82928a5dac9cc0b2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a0b6723ad2db4fccb7a1c219ff04e99a. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:50.310798 23023 raft_consensus.cc:1275] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:07:50.312359 22876 raft_consensus.cc:1275] T a0b6723ad2db4fccb7a1c219ff04e99a P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:07:50.312919 23567 consensus_queue.cc:1048] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:50.314996 23740 consensus_queue.cc:1048] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:50.327114 23567 raft_consensus.cc:2955] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:50.333359 23023 raft_consensus.cc:2955] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:50.351485 22777 catalog_manager.cc:5697] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 reported cstate change: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New cstate: current_term: 1 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:50.373965 22876 raft_consensus.cc:2955] T a0b6723ad2db4fccb7a1c219ff04e99a P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:50.652370 23759 ts_tablet_manager.cc:933] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2: Initiating tablet copy from peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929)
W20260709 13:07:50.720032 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 57226.3us
W20260709 13:07:50.721524 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 59480.8us
I20260709 13:07:50.751819 23759 tablet_copy_client.cc:323] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.63.1:33929
I20260709 13:07:50.756083 22890 tablet_copy_service.cc:140] P 0a407504ed1741bf89a276558adf4c0f: Received BeginTabletCopySession request for tablet 0fa29d11409a4d209411116a7ef8af04 from peer 5b346767a89745fa82928a5dac9cc0b2 ({username='slave'} at 127.0.0.1:53724)
I20260709 13:07:50.756762 22890 tablet_copy_service.cc:161] P 0a407504ed1741bf89a276558adf4c0f: Beginning new tablet copy session on tablet 0fa29d11409a4d209411116a7ef8af04 from peer 5b346767a89745fa82928a5dac9cc0b2 at {username='slave'} at 127.0.0.1:53724: session id = 5b346767a89745fa82928a5dac9cc0b2-0fa29d11409a4d209411116a7ef8af04
I20260709 13:07:50.767738 22890 tablet_copy_source_session.cc:215] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:07:50.781523 23761 ts_tablet_manager.cc:933] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2: Initiating tablet copy from peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929)
I20260709 13:07:50.782577 23759 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0fa29d11409a4d209411116a7ef8af04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:50.788507 23762 ts_tablet_manager.cc:933] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2: Initiating tablet copy from peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929)
I20260709 13:07:50.801805 23761 tablet_copy_client.cc:323] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.63.1:33929
I20260709 13:07:50.817299 23759 tablet_copy_client.cc:806] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 0 data blocks...
I20260709 13:07:50.806260 23762 tablet_copy_client.cc:323] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.63.1:33929
I20260709 13:07:50.805795 22890 tablet_copy_service.cc:140] P 0a407504ed1741bf89a276558adf4c0f: Received BeginTabletCopySession request for tablet a424231ef7b4470bbd575564909a365e from peer 5b346767a89745fa82928a5dac9cc0b2 ({username='slave'} at 127.0.0.1:53724)
I20260709 13:07:50.804805 23765 ts_tablet_manager.cc:933] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2: Initiating tablet copy from peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:50.826278 23759 tablet_copy_client.cc:670] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:07:50.826284 22890 tablet_copy_service.cc:161] P 0a407504ed1741bf89a276558adf4c0f: Beginning new tablet copy session on tablet a424231ef7b4470bbd575564909a365e from peer 5b346767a89745fa82928a5dac9cc0b2 at {username='slave'} at 127.0.0.1:53724: session id = 5b346767a89745fa82928a5dac9cc0b2-a424231ef7b4470bbd575564909a365e
I20260709 13:07:50.827665 22889 tablet_copy_service.cc:140] P 0a407504ed1741bf89a276558adf4c0f: Received BeginTabletCopySession request for tablet 392f5a2324d74a499e6baad745c941fd from peer 5b346767a89745fa82928a5dac9cc0b2 ({username='slave'} at 127.0.0.1:53724)
I20260709 13:07:50.828241 22889 tablet_copy_service.cc:161] P 0a407504ed1741bf89a276558adf4c0f: Beginning new tablet copy session on tablet 392f5a2324d74a499e6baad745c941fd from peer 5b346767a89745fa82928a5dac9cc0b2 at {username='slave'} at 127.0.0.1:53724: session id = 5b346767a89745fa82928a5dac9cc0b2-392f5a2324d74a499e6baad745c941fd
I20260709 13:07:50.838760 23759 tablet_copy_client.cc:538] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:07:50.929657 23759 tablet_bootstrap.cc:492] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap starting.
W20260709 13:07:50.941668 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 86697.5us
I20260709 13:07:50.941856 22889 tablet_copy_source_session.cc:215] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:07:50.994925 23762 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 392f5a2324d74a499e6baad745c941fd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:51.004438 22890 tablet_copy_source_session.cc:215] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:07:51.008152 23761 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a424231ef7b4470bbd575564909a365e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:51.012157 23765 tablet_copy_client.cc:323] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.63.2:33309
I20260709 13:07:51.052390 22965 tablet_copy_service.cc:140] P aed3f089ec8b49af9c8cf5cdee76e8e7: Received BeginTabletCopySession request for tablet a0b6723ad2db4fccb7a1c219ff04e99a from peer 5b346767a89745fa82928a5dac9cc0b2 ({username='slave'} at 127.0.0.1:52222)
I20260709 13:07:51.053427 22965 tablet_copy_service.cc:161] P aed3f089ec8b49af9c8cf5cdee76e8e7: Beginning new tablet copy session on tablet a0b6723ad2db4fccb7a1c219ff04e99a from peer 5b346767a89745fa82928a5dac9cc0b2 at {username='slave'} at 127.0.0.1:52222: session id = 5b346767a89745fa82928a5dac9cc0b2-a0b6723ad2db4fccb7a1c219ff04e99a
I20260709 13:07:51.064440 22965 tablet_copy_source_session.cc:215] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 13:07:51.070016 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 61918.6us
I20260709 13:07:51.073271 23765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a0b6723ad2db4fccb7a1c219ff04e99a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:51.150251 23762 tablet_copy_client.cc:806] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 0 data blocks...
I20260709 13:07:51.150894 23762 tablet_copy_client.cc:670] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:07:51.151379 23759 tablet_bootstrap.cc:492] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:51.152233 23765 tablet_copy_client.cc:806] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 0 data blocks...
I20260709 13:07:51.153000 23765 tablet_copy_client.cc:670] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:07:51.159983 23762 tablet_copy_client.cc:538] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:07:51.163187 23765 tablet_copy_client.cc:538] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:07:51.167043 23761 tablet_copy_client.cc:806] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 0 data blocks...
I20260709 13:07:51.167629 23761 tablet_copy_client.cc:670] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:07:51.168407 23759 tablet_bootstrap.cc:492] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap complete.
I20260709 13:07:51.169155 23759 ts_tablet_manager.cc:1403] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2: Time spent bootstrapping tablet: real 0.240s	user 0.018s	sys 0.007s
I20260709 13:07:51.172423 23762 tablet_bootstrap.cc:492] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap starting.
I20260709 13:07:51.177251 23761 tablet_copy_client.cc:538] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:07:51.184211 23765 tablet_bootstrap.cc:492] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap starting.
I20260709 13:07:51.191730 23761 tablet_bootstrap.cc:492] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap starting.
I20260709 13:07:51.209776 23762 tablet_bootstrap.cc:492] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:51.212791 23762 tablet_bootstrap.cc:492] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap complete.
I20260709 13:07:51.214871 23762 ts_tablet_manager.cc:1403] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2: Time spent bootstrapping tablet: real 0.044s	user 0.019s	sys 0.012s
I20260709 13:07:51.176479 23759 raft_consensus.cc:359] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:51.218034 23759 raft_consensus.cc:740] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Initialized, Role: LEARNER
I20260709 13:07:51.219820 23759 consensus_queue.cc:260] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:51.224401 23762 raft_consensus.cc:359] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:51.225298 23762 raft_consensus.cc:740] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Initialized, Role: LEARNER
I20260709 13:07:51.226178 23762 consensus_queue.cc:260] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:51.232471 23759 ts_tablet_manager.cc:1434] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2: Time spent starting tablet: real 0.061s	user 0.006s	sys 0.000s
I20260709 13:07:51.240608 22890 tablet_copy_service.cc:342] P 0a407504ed1741bf89a276558adf4c0f: Request end of tablet copy session 5b346767a89745fa82928a5dac9cc0b2-0fa29d11409a4d209411116a7ef8af04 received from {username='slave'} at 127.0.0.1:53724
I20260709 13:07:51.241133 22890 tablet_copy_service.cc:434] P 0a407504ed1741bf89a276558adf4c0f: ending tablet copy session 5b346767a89745fa82928a5dac9cc0b2-0fa29d11409a4d209411116a7ef8af04 on tablet 0fa29d11409a4d209411116a7ef8af04 with peer 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:07:51.243248 23762 ts_tablet_manager.cc:1434] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2: Time spent starting tablet: real 0.028s	user 0.009s	sys 0.002s
I20260709 13:07:51.247982 22890 tablet_copy_service.cc:342] P 0a407504ed1741bf89a276558adf4c0f: Request end of tablet copy session 5b346767a89745fa82928a5dac9cc0b2-392f5a2324d74a499e6baad745c941fd received from {username='slave'} at 127.0.0.1:53724
I20260709 13:07:51.248416 22890 tablet_copy_service.cc:434] P 0a407504ed1741bf89a276558adf4c0f: ending tablet copy session 5b346767a89745fa82928a5dac9cc0b2-392f5a2324d74a499e6baad745c941fd on tablet 392f5a2324d74a499e6baad745c941fd with peer 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:07:51.272219 23761 tablet_bootstrap.cc:492] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:51.273062 23761 tablet_bootstrap.cc:492] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap complete.
I20260709 13:07:51.273727 23761 ts_tablet_manager.cc:1403] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2: Time spent bootstrapping tablet: real 0.084s	user 0.023s	sys 0.007s
I20260709 13:07:51.276703 23761 raft_consensus.cc:359] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:51.277535 23761 raft_consensus.cc:740] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Initialized, Role: LEARNER
I20260709 13:07:51.278154 23761 consensus_queue.cc:260] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:51.281019 23761 ts_tablet_manager.cc:1434] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 13:07:51.285560 22890 tablet_copy_service.cc:342] P 0a407504ed1741bf89a276558adf4c0f: Request end of tablet copy session 5b346767a89745fa82928a5dac9cc0b2-a424231ef7b4470bbd575564909a365e received from {username='slave'} at 127.0.0.1:53724
I20260709 13:07:51.286029 22890 tablet_copy_service.cc:434] P 0a407504ed1741bf89a276558adf4c0f: ending tablet copy session 5b346767a89745fa82928a5dac9cc0b2-a424231ef7b4470bbd575564909a365e on tablet a424231ef7b4470bbd575564909a365e with peer 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:07:51.292411 23765 tablet_bootstrap.cc:492] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:51.293464 23765 tablet_bootstrap.cc:492] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap complete.
I20260709 13:07:51.294184 23765 ts_tablet_manager.cc:1403] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2: Time spent bootstrapping tablet: real 0.110s	user 0.021s	sys 0.012s
I20260709 13:07:51.300359 23765 raft_consensus.cc:359] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:51.301345 23765 raft_consensus.cc:740] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Initialized, Role: LEARNER
I20260709 13:07:51.302320 23765 consensus_queue.cc:260] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:51.306391 23765 ts_tablet_manager.cc:1434] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.002s
I20260709 13:07:51.308280 22965 tablet_copy_service.cc:342] P aed3f089ec8b49af9c8cf5cdee76e8e7: Request end of tablet copy session 5b346767a89745fa82928a5dac9cc0b2-a0b6723ad2db4fccb7a1c219ff04e99a received from {username='slave'} at 127.0.0.1:52222
I20260709 13:07:51.308832 22965 tablet_copy_service.cc:434] P aed3f089ec8b49af9c8cf5cdee76e8e7: ending tablet copy session 5b346767a89745fa82928a5dac9cc0b2-a0b6723ad2db4fccb7a1c219ff04e99a on tablet a0b6723ad2db4fccb7a1c219ff04e99a with peer 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:07:51.382613 23623 raft_consensus.cc:1217] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:07:51.581207 23623 raft_consensus.cc:1217] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:07:51.657487 23623 raft_consensus.cc:1217] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:07:51.668366 23623 raft_consensus.cc:1217] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:07:51.988171 23738 raft_consensus.cc:1064] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
I20260709 13:07:51.990667 23738 consensus_queue.cc:237] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:07:52.005789 23623 raft_consensus.cc:1275] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:52.006302 22954 raft_consensus.cc:1275] T 392f5a2324d74a499e6baad745c941fd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:52.007659 23716 consensus_queue.cc:1048] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:52.010403 23025 raft_consensus.cc:1275] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:52.015875 23716 consensus_queue.cc:1048] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:52.018553 23774 consensus_queue.cc:1048] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:52.039708 23781 raft_consensus.cc:2955] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:52.043684 23623 raft_consensus.cc:2955] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:52.047741 22954 raft_consensus.cc:2955] T 392f5a2324d74a499e6baad745c941fd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:52.062160 23023 raft_consensus.cc:2955] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:52.068974 22778 catalog_manager.cc:5697] T 392f5a2324d74a499e6baad745c941fd P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:52.088073 23716 raft_consensus.cc:1064] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
I20260709 13:07:52.090606 23716 consensus_queue.cc:237] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:07:52.098915 23623 raft_consensus.cc:1275] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:52.099433 23023 raft_consensus.cc:1275] T 0fa29d11409a4d209411116a7ef8af04 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:52.100899 23774 consensus_queue.cc:1048] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:52.110813 22880 consensus_queue.cc:237] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:07:52.123848 23781 consensus_queue.cc:1048] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:52.141804 23623 raft_consensus.cc:1275] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:52.143813 23738 consensus_queue.cc:1048] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:07:52.151031 23025 raft_consensus.cc:1275] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:52.162122 23774 consensus_queue.cc:1048] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:07:52.170575 23716 raft_consensus.cc:2955] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:52.173439 22954 raft_consensus.cc:1275] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:52.182623 23624 raft_consensus.cc:2955] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:52.191835 23781 consensus_queue.cc:1048] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:52.198108 22764 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 392f5a2324d74a499e6baad745c941fd with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:07:52.211383 22775 catalog_manager.cc:5697] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f reported cstate change: config changed from index 3 to 4, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:52.250918 23770 raft_consensus.cc:1064] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
I20260709 13:07:52.251001 22943 tablet_service.cc:1558] Processing DeleteTablet for tablet 392f5a2324d74a499e6baad745c941fd with delete_type TABLET_DATA_TOMBSTONED (TS aed3f089ec8b49af9c8cf5cdee76e8e7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58302
I20260709 13:07:52.254487 23770 consensus_queue.cc:237] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:07:52.275238 23023 raft_consensus.cc:1275] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:52.275240 23624 raft_consensus.cc:1275] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:52.281580 23797 consensus_queue.cc:1048] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:52.282601 23757 consensus_queue.cc:1048] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:52.291188 23716 raft_consensus.cc:1064] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
I20260709 13:07:52.293761 23716 consensus_queue.cc:237] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:07:52.300323 22880 raft_consensus.cc:1275] T a0b6723ad2db4fccb7a1c219ff04e99a P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:52.303908 23796 consensus_queue.cc:1048] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:52.322036 23623 raft_consensus.cc:1275] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:07:52.364442 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 159599us
W20260709 13:07:52.405807 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 80278.6us
I20260709 13:07:52.413331 22953 raft_consensus.cc:1275] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:52.421799 23781 raft_consensus.cc:2955] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:52.434072 23793 tablet_replica.cc:333] T 392f5a2324d74a499e6baad745c941fd P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:07:52.435207 23793 raft_consensus.cc:2243] T 392f5a2324d74a499e6baad745c941fd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:52.434062 23757 raft_consensus.cc:2955] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:52.435938 23793 raft_consensus.cc:2272] T 392f5a2324d74a499e6baad745c941fd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:52.462241 23020 raft_consensus.cc:1275] T a424231ef7b4470bbd575564909a365e P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:52.464866 23023 raft_consensus.cc:2955] T 0fa29d11409a4d209411116a7ef8af04 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:52.465801 23020 raft_consensus.cc:2955] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:52.421530 23774 consensus_queue.cc:1048] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 13:07:52.477697 23026 raft_consensus.cc:2955] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:52.483616 23793 ts_tablet_manager.cc:1916] T 392f5a2324d74a499e6baad745c941fd P aed3f089ec8b49af9c8cf5cdee76e8e7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:07:52.491731 23716 consensus_queue.cc:1048] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:52.519026 22776 catalog_manager.cc:5697] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:07:52.524757 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 57640.9us
I20260709 13:07:52.528697 23623 raft_consensus.cc:2955] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:52.537328 23625 raft_consensus.cc:2955] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:52.545724 23792 consensus_queue.cc:1048] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:07:52.566978 22777 catalog_manager.cc:5697] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:52.565061 22954 raft_consensus.cc:2955] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:52.617558 23793 ts_tablet_manager.cc:1929] T 392f5a2324d74a499e6baad745c941fd P aed3f089ec8b49af9c8cf5cdee76e8e7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:07:52.618074 23793 log.cc:1199] T 392f5a2324d74a499e6baad745c941fd P aed3f089ec8b49af9c8cf5cdee76e8e7: Deleting WAL directory at /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-1-root/wals/392f5a2324d74a499e6baad745c941fd
I20260709 13:07:52.655067 22762 catalog_manager.cc:5028] TS aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): tablet 392f5a2324d74a499e6baad745c941fd (table test-workload [id=a42f2ccfbe484baca3019323a490c64a]) successfully deleted
I20260709 13:07:52.633448 22878 raft_consensus.cc:2955] T a0b6723ad2db4fccb7a1c219ff04e99a P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:52.688493 23774 raft_consensus.cc:2955] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:52.695463 23624 raft_consensus.cc:2955] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:52.723240 22876 consensus_queue.cc:237] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:07:52.747008 22955 raft_consensus.cc:1275] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:52.747578 23624 raft_consensus.cc:1275] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:52.748813 23774 consensus_queue.cc:1048] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:07:52.749871 22954 raft_consensus.cc:2955] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:52.750761 23809 consensus_queue.cc:1048] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:52.751289 22955 consensus_queue.cc:237] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:07:52.757498 22775 catalog_manager.cc:5697] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:52.759979 23805 raft_consensus.cc:2955] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:52.762159 23623 raft_consensus.cc:1275] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:52.761595 23625 raft_consensus.cc:2955] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:52.764264 23808 consensus_queue.cc:1048] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:07:52.786545 23806 raft_consensus.cc:2955] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:52.789862 23623 raft_consensus.cc:2955] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:52.792982 23029 raft_consensus.cc:1275] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:52.796010 23023 raft_consensus.cc:2955] T a424231ef7b4470bbd575564909a365e P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:52.810732 22762 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a0b6723ad2db4fccb7a1c219ff04e99a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:07:52.797794 22764 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 0fa29d11409a4d209411116a7ef8af04 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:07:52.811379 23806 consensus_queue.cc:1048] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:52.798174 22777 catalog_manager.cc:5697] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: config changed from index 3 to 4, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:52.796034 22953 raft_consensus.cc:2955] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:52.817006 23029 raft_consensus.cc:2955] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:52.835403 22876 consensus_queue.cc:237] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:07:52.850175 23623 raft_consensus.cc:1275] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:52.850370 22953 raft_consensus.cc:1275] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:52.850543 22775 catalog_manager.cc:5697] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: config changed from index 3 to 4, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New cstate: current_term: 1 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:52.851886 23774 consensus_queue.cc:1048] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:52.854221 23805 consensus_queue.cc:1048] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:07:52.859521 23774 raft_consensus.cc:2955] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:52.863222 23019 tablet_service.cc:1558] Processing DeleteTablet for tablet 0fa29d11409a4d209411116a7ef8af04 with delete_type TABLET_DATA_TOMBSTONED (TS f3c799b1249e40119d2a5864231c6c75 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33406
I20260709 13:07:52.868568 22954 raft_consensus.cc:2955] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:52.869305 23623 raft_consensus.cc:2955] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:52.879374 22867 tablet_service.cc:1558] Processing DeleteTablet for tablet a0b6723ad2db4fccb7a1c219ff04e99a with delete_type TABLET_DATA_TOMBSTONED (TS 0a407504ed1741bf89a276558adf4c0f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:35344
I20260709 13:07:52.882196 23816 tablet_replica.cc:333] T 0fa29d11409a4d209411116a7ef8af04 P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:07:52.888149 22775 catalog_manager.cc:5697] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: config changed from index 3 to 4, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:52.899211 22764 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a424231ef7b4470bbd575564909a365e with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:07:52.905405 23817 tablet_replica.cc:333] T a0b6723ad2db4fccb7a1c219ff04e99a P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:07:52.906159 23816 raft_consensus.cc:2243] T 0fa29d11409a4d209411116a7ef8af04 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:52.906632 23817 raft_consensus.cc:2243] T a0b6723ad2db4fccb7a1c219ff04e99a P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:52.907035 23816 raft_consensus.cc:2272] T 0fa29d11409a4d209411116a7ef8af04 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:52.907346 23817 raft_consensus.cc:2272] T a0b6723ad2db4fccb7a1c219ff04e99a P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:52.910473 23817 ts_tablet_manager.cc:1916] T a0b6723ad2db4fccb7a1c219ff04e99a P 0a407504ed1741bf89a276558adf4c0f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:07:52.911665 23019 tablet_service.cc:1558] Processing DeleteTablet for tablet a424231ef7b4470bbd575564909a365e with delete_type TABLET_DATA_TOMBSTONED (TS f3c799b1249e40119d2a5864231c6c75 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33406
I20260709 13:07:52.915621 23816 ts_tablet_manager.cc:1916] T 0fa29d11409a4d209411116a7ef8af04 P f3c799b1249e40119d2a5864231c6c75: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:07:52.930874 23817 ts_tablet_manager.cc:1929] T a0b6723ad2db4fccb7a1c219ff04e99a P 0a407504ed1741bf89a276558adf4c0f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:07:52.931298 23817 log.cc:1199] T a0b6723ad2db4fccb7a1c219ff04e99a P 0a407504ed1741bf89a276558adf4c0f: Deleting WAL directory at /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-0-root/wals/a0b6723ad2db4fccb7a1c219ff04e99a
I20260709 13:07:52.931681 23816 ts_tablet_manager.cc:1929] T 0fa29d11409a4d209411116a7ef8af04 P f3c799b1249e40119d2a5864231c6c75: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:07:52.932068 23816 log.cc:1199] T 0fa29d11409a4d209411116a7ef8af04 P f3c799b1249e40119d2a5864231c6c75: Deleting WAL directory at /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-2-root/wals/0fa29d11409a4d209411116a7ef8af04
I20260709 13:07:52.932888 22764 catalog_manager.cc:5028] TS 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): tablet a0b6723ad2db4fccb7a1c219ff04e99a (table test-workload [id=a42f2ccfbe484baca3019323a490c64a]) successfully deleted
I20260709 13:07:52.933555 22762 catalog_manager.cc:5028] TS f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): tablet 0fa29d11409a4d209411116a7ef8af04 (table test-workload [id=a42f2ccfbe484baca3019323a490c64a]) successfully deleted
I20260709 13:07:52.934238 23816 tablet_replica.cc:333] T a424231ef7b4470bbd575564909a365e P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:07:52.935209 23816 raft_consensus.cc:2243] T a424231ef7b4470bbd575564909a365e P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:52.936002 23816 raft_consensus.cc:2272] T a424231ef7b4470bbd575564909a365e P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:52.939096 23816 ts_tablet_manager.cc:1916] T a424231ef7b4470bbd575564909a365e P f3c799b1249e40119d2a5864231c6c75: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:07:52.951208 23816 ts_tablet_manager.cc:1929] T a424231ef7b4470bbd575564909a365e P f3c799b1249e40119d2a5864231c6c75: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:07:52.951591 23816 log.cc:1199] T a424231ef7b4470bbd575564909a365e P f3c799b1249e40119d2a5864231c6c75: Deleting WAL directory at /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-2-root/wals/a424231ef7b4470bbd575564909a365e
I20260709 13:07:52.952976 22762 catalog_manager.cc:5028] TS f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): tablet a424231ef7b4470bbd575564909a365e (table test-workload [id=a42f2ccfbe484baca3019323a490c64a]) successfully deleted
I20260709 13:07:53.988481 23029 consensus_queue.cc:237] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:53.995028 22954 raft_consensus.cc:1275] T e3b39c18a21f448783cdd998acfb2f80 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:07:53.995154 22993 consensus_peers.cc:597] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 -> Peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Couldn't send request to peer 5b346767a89745fa82928a5dac9cc0b2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e3b39c18a21f448783cdd998acfb2f80. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:53.995931 22880 raft_consensus.cc:1275] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:07:53.996454 23819 consensus_queue.cc:1048] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:53.997812 23735 consensus_queue.cc:1048] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:54.008736 23735 raft_consensus.cc:2955] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:54.011338 22880 raft_consensus.cc:2955] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:54.011266 22954 raft_consensus.cc:2955] T e3b39c18a21f448783cdd998acfb2f80 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:54.022873 22953 consensus_queue.cc:237] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:54.030362 22776 catalog_manager.cc:5697] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:07:54.038874 22919 consensus_peers.cc:597] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Couldn't send request to peer 5b346767a89745fa82928a5dac9cc0b2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cbf52b91ec054ac6a2f8503462b1b7a6. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:54.038854 22880 raft_consensus.cc:1275] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:07:54.041018 23818 consensus_queue.cc:1048] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:54.051662 23023 raft_consensus.cc:1275] T cbf52b91ec054ac6a2f8503462b1b7a6 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:07:54.067083 23806 consensus_queue.cc:1048] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:54.078689 23806 raft_consensus.cc:2955] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:54.080606 22880 raft_consensus.cc:2955] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:54.103428 23020 raft_consensus.cc:2955] T cbf52b91ec054ac6a2f8503462b1b7a6 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:54.112001 22954 consensus_queue.cc:237] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
W20260709 13:07:54.121593 22919 consensus_peers.cc:597] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Couldn't send request to peer 5b346767a89745fa82928a5dac9cc0b2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3047c357228240a2aa4d84e8fbf437fc. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:54.125883 22880 raft_consensus.cc:1275] T 3047c357228240a2aa4d84e8fbf437fc P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:07:54.130427 23806 consensus_queue.cc:1048] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:54.132269 23020 raft_consensus.cc:1275] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:07:54.133879 23806 consensus_queue.cc:1048] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:54.150668 22774 catalog_manager.cc:5697] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New cstate: current_term: 1 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:54.154443 23818 raft_consensus.cc:2955] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:54.157060 22880 raft_consensus.cc:2955] T 3047c357228240a2aa4d84e8fbf437fc P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:54.172730 23023 raft_consensus.cc:2955] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:54.173440 23020 consensus_queue.cc:237] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:54.187914 22776 catalog_manager.cc:5697] T 3047c357228240a2aa4d84e8fbf437fc P 0a407504ed1741bf89a276558adf4c0f reported cstate change: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New cstate: current_term: 1 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:54.209484 22953 raft_consensus.cc:1275] T dd740f20d2d74669b0fd10f1ec5b44f0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:07:54.210402 22993 consensus_peers.cc:597] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 -> Peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Couldn't send request to peer 5b346767a89745fa82928a5dac9cc0b2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dd740f20d2d74669b0fd10f1ec5b44f0. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:54.211638 23735 consensus_queue.cc:1048] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:54.213393 22873 raft_consensus.cc:1275] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:07:54.215458 23803 consensus_queue.cc:1048] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:54.241003 23735 raft_consensus.cc:2955] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:54.247471 22953 raft_consensus.cc:2955] T dd740f20d2d74669b0fd10f1ec5b44f0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:54.249300 22873 raft_consensus.cc:2955] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:54.258849 22774 catalog_manager.cc:5697] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:54.495102 23837 ts_tablet_manager.cc:933] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2: Initiating tablet copy from peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:54.497066 23837 tablet_copy_client.cc:323] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.63.3:42499
I20260709 13:07:54.499032 23039 tablet_copy_service.cc:140] P f3c799b1249e40119d2a5864231c6c75: Received BeginTabletCopySession request for tablet e3b39c18a21f448783cdd998acfb2f80 from peer 5b346767a89745fa82928a5dac9cc0b2 ({username='slave'} at 127.0.0.1:32794)
I20260709 13:07:54.499616 23039 tablet_copy_service.cc:161] P f3c799b1249e40119d2a5864231c6c75: Beginning new tablet copy session on tablet e3b39c18a21f448783cdd998acfb2f80 from peer 5b346767a89745fa82928a5dac9cc0b2 at {username='slave'} at 127.0.0.1:32794: session id = 5b346767a89745fa82928a5dac9cc0b2-e3b39c18a21f448783cdd998acfb2f80
I20260709 13:07:54.511406 23039 tablet_copy_source_session.cc:215] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:07:54.514605 23837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e3b39c18a21f448783cdd998acfb2f80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.530927 23837 tablet_copy_client.cc:806] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 0 data blocks...
I20260709 13:07:54.531514 23837 tablet_copy_client.cc:670] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:07:54.536695 23837 tablet_copy_client.cc:538] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:07:54.546249 23837 tablet_bootstrap.cc:492] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap starting.
I20260709 13:07:54.568336 23837 tablet_bootstrap.cc:492] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:54.569356 23837 tablet_bootstrap.cc:492] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap complete.
I20260709 13:07:54.570030 23837 ts_tablet_manager.cc:1403] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2: Time spent bootstrapping tablet: real 0.024s	user 0.022s	sys 0.000s
I20260709 13:07:54.572808 23837 raft_consensus.cc:359] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:54.573426 23837 raft_consensus.cc:740] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Initialized, Role: LEARNER
I20260709 13:07:54.573943 23837 consensus_queue.cc:260] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:54.576972 23837 ts_tablet_manager.cc:1434] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:07:54.578845 23039 tablet_copy_service.cc:342] P f3c799b1249e40119d2a5864231c6c75: Request end of tablet copy session 5b346767a89745fa82928a5dac9cc0b2-e3b39c18a21f448783cdd998acfb2f80 received from {username='slave'} at 127.0.0.1:32794
I20260709 13:07:54.579244 23039 tablet_copy_service.cc:434] P f3c799b1249e40119d2a5864231c6c75: ending tablet copy session 5b346767a89745fa82928a5dac9cc0b2-e3b39c18a21f448783cdd998acfb2f80 on tablet e3b39c18a21f448783cdd998acfb2f80 with peer 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:07:54.604096 23837 ts_tablet_manager.cc:933] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2: Initiating tablet copy from peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:54.613698 23837 tablet_copy_client.cc:323] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.63.3:42499
I20260709 13:07:54.619802 23039 tablet_copy_service.cc:140] P f3c799b1249e40119d2a5864231c6c75: Received BeginTabletCopySession request for tablet dd740f20d2d74669b0fd10f1ec5b44f0 from peer 5b346767a89745fa82928a5dac9cc0b2 ({username='slave'} at 127.0.0.1:32794)
I20260709 13:07:54.620428 23039 tablet_copy_service.cc:161] P f3c799b1249e40119d2a5864231c6c75: Beginning new tablet copy session on tablet dd740f20d2d74669b0fd10f1ec5b44f0 from peer 5b346767a89745fa82928a5dac9cc0b2 at {username='slave'} at 127.0.0.1:32794: session id = 5b346767a89745fa82928a5dac9cc0b2-dd740f20d2d74669b0fd10f1ec5b44f0
I20260709 13:07:54.629607 23039 tablet_copy_source_session.cc:215] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:07:54.631235 23842 ts_tablet_manager.cc:933] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2: Initiating tablet copy from peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:54.633970 23843 ts_tablet_manager.cc:933] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2: Initiating tablet copy from peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:54.634361 23837 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd740f20d2d74669b0fd10f1ec5b44f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.640152 23843 tablet_copy_client.cc:323] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.63.2:33309
I20260709 13:07:54.642524 22965 tablet_copy_service.cc:140] P aed3f089ec8b49af9c8cf5cdee76e8e7: Received BeginTabletCopySession request for tablet cbf52b91ec054ac6a2f8503462b1b7a6 from peer 5b346767a89745fa82928a5dac9cc0b2 ({username='slave'} at 127.0.0.1:52222)
I20260709 13:07:54.643077 22965 tablet_copy_service.cc:161] P aed3f089ec8b49af9c8cf5cdee76e8e7: Beginning new tablet copy session on tablet cbf52b91ec054ac6a2f8503462b1b7a6 from peer 5b346767a89745fa82928a5dac9cc0b2 at {username='slave'} at 127.0.0.1:52222: session id = 5b346767a89745fa82928a5dac9cc0b2-cbf52b91ec054ac6a2f8503462b1b7a6
I20260709 13:07:54.648484 22965 tablet_copy_source_session.cc:215] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:07:54.652828 23842 tablet_copy_client.cc:323] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.63.2:33309
I20260709 13:07:54.654486 23843 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cbf52b91ec054ac6a2f8503462b1b7a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.654438 22965 tablet_copy_service.cc:140] P aed3f089ec8b49af9c8cf5cdee76e8e7: Received BeginTabletCopySession request for tablet 3047c357228240a2aa4d84e8fbf437fc from peer 5b346767a89745fa82928a5dac9cc0b2 ({username='slave'} at 127.0.0.1:52222)
I20260709 13:07:54.655088 22965 tablet_copy_service.cc:161] P aed3f089ec8b49af9c8cf5cdee76e8e7: Beginning new tablet copy session on tablet 3047c357228240a2aa4d84e8fbf437fc from peer 5b346767a89745fa82928a5dac9cc0b2 at {username='slave'} at 127.0.0.1:52222: session id = 5b346767a89745fa82928a5dac9cc0b2-3047c357228240a2aa4d84e8fbf437fc
I20260709 13:07:54.662863 22965 tablet_copy_source_session.cc:215] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:07:54.666170 23842 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3047c357228240a2aa4d84e8fbf437fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:54.669561 23837 tablet_copy_client.cc:806] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 0 data blocks...
I20260709 13:07:54.670145 23837 tablet_copy_client.cc:670] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:07:54.674026 23837 tablet_copy_client.cc:538] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:07:54.685315 23843 tablet_copy_client.cc:806] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 0 data blocks...
I20260709 13:07:54.686010 23843 tablet_copy_client.cc:670] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:07:54.686563 23837 tablet_bootstrap.cc:492] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap starting.
I20260709 13:07:54.690892 23842 tablet_copy_client.cc:806] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 0 data blocks...
I20260709 13:07:54.693406 23842 tablet_copy_client.cc:670] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:07:54.698583 23842 tablet_copy_client.cc:538] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:07:54.709956 23842 tablet_bootstrap.cc:492] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap starting.
I20260709 13:07:54.710371 23843 tablet_copy_client.cc:538] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:07:54.723126 23837 tablet_bootstrap.cc:492] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:54.724082 23837 tablet_bootstrap.cc:492] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap complete.
I20260709 13:07:54.724785 23837 ts_tablet_manager.cc:1403] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2: Time spent bootstrapping tablet: real 0.038s	user 0.019s	sys 0.004s
I20260709 13:07:54.726851 23843 tablet_bootstrap.cc:492] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap starting.
I20260709 13:07:54.731693 23837 raft_consensus.cc:359] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:54.732841 23837 raft_consensus.cc:740] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Initialized, Role: LEARNER
I20260709 13:07:54.733553 23837 consensus_queue.cc:260] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:54.736101 23837 ts_tablet_manager.cc:1434] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.004s
I20260709 13:07:54.739336 23039 tablet_copy_service.cc:342] P f3c799b1249e40119d2a5864231c6c75: Request end of tablet copy session 5b346767a89745fa82928a5dac9cc0b2-dd740f20d2d74669b0fd10f1ec5b44f0 received from {username='slave'} at 127.0.0.1:32794
I20260709 13:07:54.739738 23039 tablet_copy_service.cc:434] P f3c799b1249e40119d2a5864231c6c75: ending tablet copy session 5b346767a89745fa82928a5dac9cc0b2-dd740f20d2d74669b0fd10f1ec5b44f0 on tablet dd740f20d2d74669b0fd10f1ec5b44f0 with peer 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:07:54.762084 23842 tablet_bootstrap.cc:492] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:54.766119 23842 tablet_bootstrap.cc:492] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap complete.
I20260709 13:07:54.766883 23842 ts_tablet_manager.cc:1403] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2: Time spent bootstrapping tablet: real 0.057s	user 0.015s	sys 0.011s
I20260709 13:07:54.770784 23842 raft_consensus.cc:359] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:54.772210 23842 raft_consensus.cc:740] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Initialized, Role: LEARNER
I20260709 13:07:54.773908 23842 consensus_queue.cc:260] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:54.782680 23843 tablet_bootstrap.cc:492] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:54.784929 23842 ts_tablet_manager.cc:1434] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2: Time spent starting tablet: real 0.018s	user 0.003s	sys 0.005s
I20260709 13:07:54.786448 23843 tablet_bootstrap.cc:492] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap complete.
I20260709 13:07:54.787227 23843 ts_tablet_manager.cc:1403] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2: Time spent bootstrapping tablet: real 0.061s	user 0.017s	sys 0.010s
I20260709 13:07:54.788134 22965 tablet_copy_service.cc:342] P aed3f089ec8b49af9c8cf5cdee76e8e7: Request end of tablet copy session 5b346767a89745fa82928a5dac9cc0b2-3047c357228240a2aa4d84e8fbf437fc received from {username='slave'} at 127.0.0.1:52222
I20260709 13:07:54.788594 22965 tablet_copy_service.cc:434] P aed3f089ec8b49af9c8cf5cdee76e8e7: ending tablet copy session 5b346767a89745fa82928a5dac9cc0b2-3047c357228240a2aa4d84e8fbf437fc on tablet 3047c357228240a2aa4d84e8fbf437fc with peer 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:07:54.790606 23843 raft_consensus.cc:359] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:54.791450 23843 raft_consensus.cc:740] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Initialized, Role: LEARNER
I20260709 13:07:54.792076 23843 consensus_queue.cc:260] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:54.794644 23843 ts_tablet_manager.cc:1434] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:07:54.796214 22965 tablet_copy_service.cc:342] P aed3f089ec8b49af9c8cf5cdee76e8e7: Request end of tablet copy session 5b346767a89745fa82928a5dac9cc0b2-cbf52b91ec054ac6a2f8503462b1b7a6 received from {username='slave'} at 127.0.0.1:52222
I20260709 13:07:54.796603 22965 tablet_copy_service.cc:434] P aed3f089ec8b49af9c8cf5cdee76e8e7: ending tablet copy session 5b346767a89745fa82928a5dac9cc0b2-cbf52b91ec054ac6a2f8503462b1b7a6 on tablet cbf52b91ec054ac6a2f8503462b1b7a6 with peer 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:07:54.990794 23623 raft_consensus.cc:1217] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:07:55.114292 23623 raft_consensus.cc:1217] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:07:55.143245 23623 raft_consensus.cc:1217] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:07:55.229158 23623 raft_consensus.cc:1217] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:07:55.465873 23735 raft_consensus.cc:1064] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
I20260709 13:07:55.468075 23735 consensus_queue.cc:237] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:07:55.475066 23623 raft_consensus.cc:1275] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:55.476300 23819 consensus_queue.cc:1048] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:55.481124 22954 raft_consensus.cc:1275] T e3b39c18a21f448783cdd998acfb2f80 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:55.482895 23803 consensus_queue.cc:1048] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:07:55.495822 23735 raft_consensus.cc:2955] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:55.498265 23623 raft_consensus.cc:2955] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:55.507025 22954 raft_consensus.cc:2955] T e3b39c18a21f448783cdd998acfb2f80 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:55.515000 22774 catalog_manager.cc:5697] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:55.525537 22880 raft_consensus.cc:1275] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:55.536814 23819 consensus_queue.cc:1048] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:55.559101 22876 raft_consensus.cc:2955] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:55.646615 23806 raft_consensus.cc:1064] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
I20260709 13:07:55.649194 23806 consensus_queue.cc:237] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:07:55.653911 23819 raft_consensus.cc:1064] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
I20260709 13:07:55.657923 23623 raft_consensus.cc:1275] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:55.658437 22876 raft_consensus.cc:1275] T 3047c357228240a2aa4d84e8fbf437fc P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:55.656549 23819 consensus_queue.cc:237] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:07:55.659973 23806 consensus_queue.cc:1048] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:55.660935 23851 consensus_queue.cc:1048] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:55.662215 23023 raft_consensus.cc:1275] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:55.664065 23848 consensus_queue.cc:1048] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:55.668305 23623 raft_consensus.cc:1275] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:55.669927 23735 consensus_queue.cc:1048] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:55.685364 22954 raft_consensus.cc:1275] T dd740f20d2d74669b0fd10f1ec5b44f0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:55.686981 23803 consensus_queue.cc:1048] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:55.698056 22880 raft_consensus.cc:1275] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:55.703960 23848 raft_consensus.cc:2955] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:55.706103 23803 raft_consensus.cc:2955] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:55.713783 22954 raft_consensus.cc:2955] T dd740f20d2d74669b0fd10f1ec5b44f0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:55.719908 23854 consensus_queue.cc:1048] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:55.720793 23023 raft_consensus.cc:2955] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:55.730576 22876 raft_consensus.cc:2955] T 3047c357228240a2aa4d84e8fbf437fc P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:55.737982 23625 raft_consensus.cc:2955] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:55.743934 22777 catalog_manager.cc:5697] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 reported cstate change: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:55.751579 23623 raft_consensus.cc:2955] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:55.770426 22778 catalog_manager.cc:5697] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
W20260709 13:07:55.786999 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 67499.5us
I20260709 13:07:55.813349 22953 consensus_queue.cc:237] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:07:55.834200 23851 raft_consensus.cc:1064] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
W20260709 13:07:55.839351 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 48175.2us
I20260709 13:07:55.847723 23020 raft_consensus.cc:1275] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:55.850122 23818 consensus_queue.cc:1048] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:55.849591 22880 raft_consensus.cc:2955] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:55.837165 23851 consensus_queue.cc:237] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:07:55.866374 23818 raft_consensus.cc:2955] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:55.869665 23625 raft_consensus.cc:1275] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:55.876925 23020 raft_consensus.cc:2955] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:55.878244 23868 consensus_queue.cc:1048] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:07:55.882601 22762 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3047c357228240a2aa4d84e8fbf437fc with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:07:55.886157 23625 raft_consensus.cc:2955] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:55.902894 22776 catalog_manager.cc:5697] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 reported cstate change: config changed from index 3 to 4, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New cstate: current_term: 1 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:55.912912 23020 raft_consensus.cc:1275] T cbf52b91ec054ac6a2f8503462b1b7a6 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:55.913385 22876 raft_consensus.cc:1275] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:55.914764 23868 consensus_queue.cc:1048] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:55.915669 23848 consensus_queue.cc:1048] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:55.916658 23625 raft_consensus.cc:1275] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:55.918164 23818 consensus_queue.cc:1048] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:55.922421 22867 tablet_service.cc:1558] Processing DeleteTablet for tablet 3047c357228240a2aa4d84e8fbf437fc with delete_type TABLET_DATA_TOMBSTONED (TS 0a407504ed1741bf89a276558adf4c0f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:35344
I20260709 13:07:55.939155 23873 tablet_replica.cc:333] T 3047c357228240a2aa4d84e8fbf437fc P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:07:55.940388 23023 consensus_queue.cc:237] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:07:55.942212 23873 raft_consensus.cc:2243] T 3047c357228240a2aa4d84e8fbf437fc P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:55.942984 23873 raft_consensus.cc:2272] T 3047c357228240a2aa4d84e8fbf437fc P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:55.949749 23623 raft_consensus.cc:1275] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:55.952168 23819 consensus_queue.cc:1048] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:55.954056 23873 ts_tablet_manager.cc:1916] T 3047c357228240a2aa4d84e8fbf437fc P 0a407504ed1741bf89a276558adf4c0f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:07:55.959443 22876 raft_consensus.cc:1275] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:55.961277 23803 raft_consensus.cc:2955] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:55.961885 23854 consensus_queue.cc:1048] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:55.966748 23623 raft_consensus.cc:2955] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:55.970533 22876 raft_consensus.cc:2955] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:55.995880 23848 raft_consensus.cc:2955] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:55.998482 23625 raft_consensus.cc:2955] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:56.003029 23873 ts_tablet_manager.cc:1929] T 3047c357228240a2aa4d84e8fbf437fc P 0a407504ed1741bf89a276558adf4c0f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:07:56.003562 23873 log.cc:1199] T 3047c357228240a2aa4d84e8fbf437fc P 0a407504ed1741bf89a276558adf4c0f: Deleting WAL directory at /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-0-root/wals/3047c357228240a2aa4d84e8fbf437fc
I20260709 13:07:56.006135 22764 catalog_manager.cc:5028] TS 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): tablet 3047c357228240a2aa4d84e8fbf437fc (table test-workload [id=a42f2ccfbe484baca3019323a490c64a]) successfully deleted
I20260709 13:07:55.995194 22776 catalog_manager.cc:5697] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: config changed from index 3 to 4, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:56.011070 22762 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e3b39c18a21f448783cdd998acfb2f80 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:07:56.018002 22778 catalog_manager.cc:5697] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:56.020157 22943 tablet_service.cc:1558] Processing DeleteTablet for tablet e3b39c18a21f448783cdd998acfb2f80 with delete_type TABLET_DATA_TOMBSTONED (TS aed3f089ec8b49af9c8cf5cdee76e8e7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58302
I20260709 13:07:56.021940 23876 tablet_replica.cc:333] T e3b39c18a21f448783cdd998acfb2f80 P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:07:56.022984 23876 raft_consensus.cc:2243] T e3b39c18a21f448783cdd998acfb2f80 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:56.023865 23876 raft_consensus.cc:2272] T e3b39c18a21f448783cdd998acfb2f80 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:56.028654 23876 ts_tablet_manager.cc:1916] T e3b39c18a21f448783cdd998acfb2f80 P aed3f089ec8b49af9c8cf5cdee76e8e7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:07:56.029844 22880 raft_consensus.cc:2955] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:56.060227 23020 raft_consensus.cc:2955] T cbf52b91ec054ac6a2f8503462b1b7a6 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:56.064178 23876 ts_tablet_manager.cc:1929] T e3b39c18a21f448783cdd998acfb2f80 P aed3f089ec8b49af9c8cf5cdee76e8e7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:07:56.064740 23876 log.cc:1199] T e3b39c18a21f448783cdd998acfb2f80 P aed3f089ec8b49af9c8cf5cdee76e8e7: Deleting WAL directory at /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-1-root/wals/e3b39c18a21f448783cdd998acfb2f80
I20260709 13:07:56.066524 22762 catalog_manager.cc:5028] TS aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): tablet e3b39c18a21f448783cdd998acfb2f80 (table test-workload [id=a42f2ccfbe484baca3019323a490c64a]) successfully deleted
I20260709 13:07:56.070087 22954 consensus_queue.cc:237] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:07:56.097549 22880 raft_consensus.cc:1275] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:56.098954 23851 consensus_queue.cc:1048] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:56.099288 23623 raft_consensus.cc:1275] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:56.101455 23851 consensus_queue.cc:1048] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:56.105120 23848 raft_consensus.cc:2955] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:56.106727 22880 raft_consensus.cc:2955] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:56.110921 23625 raft_consensus.cc:2955] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:56.121605 22762 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet cbf52b91ec054ac6a2f8503462b1b7a6 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:07:56.125859 22774 catalog_manager.cc:5697] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: config changed from index 3 to 4, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New cstate: current_term: 1 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:56.143529 23019 tablet_service.cc:1558] Processing DeleteTablet for tablet cbf52b91ec054ac6a2f8503462b1b7a6 with delete_type TABLET_DATA_TOMBSTONED (TS f3c799b1249e40119d2a5864231c6c75 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33406
I20260709 13:07:56.149434 23878 tablet_replica.cc:333] T cbf52b91ec054ac6a2f8503462b1b7a6 P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:07:56.155349 23878 raft_consensus.cc:2243] T cbf52b91ec054ac6a2f8503462b1b7a6 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:56.156167 23878 raft_consensus.cc:2272] T cbf52b91ec054ac6a2f8503462b1b7a6 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:56.159418 23878 ts_tablet_manager.cc:1916] T cbf52b91ec054ac6a2f8503462b1b7a6 P f3c799b1249e40119d2a5864231c6c75: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:07:56.174923 23878 ts_tablet_manager.cc:1929] T cbf52b91ec054ac6a2f8503462b1b7a6 P f3c799b1249e40119d2a5864231c6c75: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:07:56.175321 23878 log.cc:1199] T cbf52b91ec054ac6a2f8503462b1b7a6 P f3c799b1249e40119d2a5864231c6c75: Deleting WAL directory at /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-2-root/wals/cbf52b91ec054ac6a2f8503462b1b7a6
I20260709 13:07:56.177019 22762 catalog_manager.cc:5028] TS f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): tablet cbf52b91ec054ac6a2f8503462b1b7a6 (table test-workload [id=a42f2ccfbe484baca3019323a490c64a]) successfully deleted
I20260709 13:07:56.227283 23023 consensus_queue.cc:237] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:07:56.233433 22880 raft_consensus.cc:1275] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:56.233464 23625 raft_consensus.cc:1275] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:56.235356 23819 consensus_queue.cc:1048] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:56.236253 23735 consensus_queue.cc:1048] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:56.246174 23625 raft_consensus.cc:2955] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:56.249840 22876 raft_consensus.cc:2955] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:56.244107 23735 raft_consensus.cc:2955] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:56.266003 22762 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet dd740f20d2d74669b0fd10f1ec5b44f0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:07:56.274087 22778 catalog_manager.cc:5697] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: config changed from index 3 to 4, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:07:56.291219 22943 tablet_service.cc:1558] Processing DeleteTablet for tablet dd740f20d2d74669b0fd10f1ec5b44f0 with delete_type TABLET_DATA_TOMBSTONED (TS aed3f089ec8b49af9c8cf5cdee76e8e7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58302
I20260709 13:07:56.292083 23876 tablet_replica.cc:333] T dd740f20d2d74669b0fd10f1ec5b44f0 P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:07:56.293042 23876 raft_consensus.cc:2243] T dd740f20d2d74669b0fd10f1ec5b44f0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:56.293794 23876 raft_consensus.cc:2272] T dd740f20d2d74669b0fd10f1ec5b44f0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:56.303216 23876 ts_tablet_manager.cc:1916] T dd740f20d2d74669b0fd10f1ec5b44f0 P aed3f089ec8b49af9c8cf5cdee76e8e7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:07:56.316768 23876 ts_tablet_manager.cc:1929] T dd740f20d2d74669b0fd10f1ec5b44f0 P aed3f089ec8b49af9c8cf5cdee76e8e7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:07:56.317209 23876 log.cc:1199] T dd740f20d2d74669b0fd10f1ec5b44f0 P aed3f089ec8b49af9c8cf5cdee76e8e7: Deleting WAL directory at /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-1-root/wals/dd740f20d2d74669b0fd10f1ec5b44f0
I20260709 13:07:56.318667 22762 catalog_manager.cc:5028] TS aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): tablet dd740f20d2d74669b0fd10f1ec5b44f0 (table test-workload [id=a42f2ccfbe484baca3019323a490c64a]) successfully deleted
I20260709 13:07:57.359010 23020 consensus_queue.cc:237] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
W20260709 13:07:57.365456 22993 consensus_peers.cc:597] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 -> Peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Couldn't send request to peer 5b346767a89745fa82928a5dac9cc0b2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7ca8b9a08f4146c386bc06d03c497038. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:57.365602 22953 raft_consensus.cc:1275] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:07:57.366712 22876 raft_consensus.cc:1275] T 7ca8b9a08f4146c386bc06d03c497038 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:07:57.366848 23881 consensus_queue.cc:1048] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:57.368064 23735 consensus_queue.cc:1048] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:57.376200 23735 raft_consensus.cc:2955] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:57.378005 22953 raft_consensus.cc:2955] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:57.378893 22876 raft_consensus.cc:2955] T 7ca8b9a08f4146c386bc06d03c497038 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:57.401952 22774 catalog_manager.cc:5697] T 7ca8b9a08f4146c386bc06d03c497038 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:57.402449 22954 consensus_queue.cc:237] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
W20260709 13:07:57.448433 22919 consensus_peers.cc:597] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Couldn't send request to peer 5b346767a89745fa82928a5dac9cc0b2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 579096ba88474cfbbe40e8bf769879ec. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:57.449581 23020 raft_consensus.cc:1275] T 579096ba88474cfbbe40e8bf769879ec P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:07:57.450579 22876 raft_consensus.cc:1275] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:07:57.451217 23851 consensus_queue.cc:1048] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:57.452091 23868 consensus_queue.cc:1048] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:57.461560 23851 raft_consensus.cc:2955] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:57.483503 23020 raft_consensus.cc:2955] T 579096ba88474cfbbe40e8bf769879ec P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:57.485368 22880 consensus_queue.cc:237] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:57.483106 22876 raft_consensus.cc:2955] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:57.502166 23023 raft_consensus.cc:1275] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:07:57.503809 23805 consensus_queue.cc:1048] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:07:57.507086 22844 consensus_peers.cc:597] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f -> Peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Couldn't send request to peer 5b346767a89745fa82928a5dac9cc0b2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 13e53d9e00624dfcb87aca9b055fdc9a. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:57.513525 22947 raft_consensus.cc:1275] T 13e53d9e00624dfcb87aca9b055fdc9a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:07:57.512830 23805 raft_consensus.cc:2955] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:57.520394 23890 consensus_queue.cc:1048] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:07:57.526774 22778 catalog_manager.cc:5697] T 579096ba88474cfbbe40e8bf769879ec P f3c799b1249e40119d2a5864231c6c75 reported cstate change: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New cstate: current_term: 1 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:57.531185 23020 raft_consensus.cc:2955] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:57.534067 22947 raft_consensus.cc:2955] T 13e53d9e00624dfcb87aca9b055fdc9a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:57.539286 22880 consensus_queue.cc:237] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
W20260709 13:07:57.551894 22844 consensus_peers.cc:597] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f -> Peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Couldn't send request to peer 5b346767a89745fa82928a5dac9cc0b2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4e689c91ad7343b5b6ba36fa648a06ac. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:07:57.552837 23029 raft_consensus.cc:1275] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:07:57.553220 22954 raft_consensus.cc:1275] T 4e689c91ad7343b5b6ba36fa648a06ac P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:07:57.553932 23890 consensus_queue.cc:1048] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:57.555200 23849 consensus_queue.cc:1048] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:07:57.564529 22777 catalog_manager.cc:5697] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f reported cstate change: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:57.581951 23890 raft_consensus.cc:2955] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:57.593708 22954 raft_consensus.cc:2955] T 4e689c91ad7343b5b6ba36fa648a06ac P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:57.605316 23029 raft_consensus.cc:2955] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:07:57.607688 22778 catalog_manager.cc:5697] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f reported cstate change: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:07:57.865061 23904 ts_tablet_manager.cc:933] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2: Initiating tablet copy from peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:07:57.867362 23904 tablet_copy_client.cc:323] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.63.3:42499
I20260709 13:07:57.869059 23039 tablet_copy_service.cc:140] P f3c799b1249e40119d2a5864231c6c75: Received BeginTabletCopySession request for tablet 7ca8b9a08f4146c386bc06d03c497038 from peer 5b346767a89745fa82928a5dac9cc0b2 ({username='slave'} at 127.0.0.1:32794)
I20260709 13:07:57.869585 23039 tablet_copy_service.cc:161] P f3c799b1249e40119d2a5864231c6c75: Beginning new tablet copy session on tablet 7ca8b9a08f4146c386bc06d03c497038 from peer 5b346767a89745fa82928a5dac9cc0b2 at {username='slave'} at 127.0.0.1:32794: session id = 5b346767a89745fa82928a5dac9cc0b2-7ca8b9a08f4146c386bc06d03c497038
I20260709 13:07:57.881150 23039 tablet_copy_source_session.cc:215] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:07:57.884970 23904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ca8b9a08f4146c386bc06d03c497038. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:57.901861 23904 tablet_copy_client.cc:806] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 0 data blocks...
I20260709 13:07:57.902550 23904 tablet_copy_client.cc:670] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:07:57.906842 23904 tablet_copy_client.cc:538] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:07:57.913103 23904 tablet_bootstrap.cc:492] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap starting.
I20260709 13:07:57.933306 23904 tablet_bootstrap.cc:492] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:57.934224 23904 tablet_bootstrap.cc:492] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap complete.
I20260709 13:07:57.934908 23904 ts_tablet_manager.cc:1403] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.004s
I20260709 13:07:57.937726 23904 raft_consensus.cc:359] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:57.938519 23904 raft_consensus.cc:740] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Initialized, Role: LEARNER
I20260709 13:07:57.939173 23904 consensus_queue.cc:260] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:57.942118 23904 ts_tablet_manager.cc:1434] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.006s
I20260709 13:07:57.943709 23039 tablet_copy_service.cc:342] P f3c799b1249e40119d2a5864231c6c75: Request end of tablet copy session 5b346767a89745fa82928a5dac9cc0b2-7ca8b9a08f4146c386bc06d03c497038 received from {username='slave'} at 127.0.0.1:32794
I20260709 13:07:57.944175 23039 tablet_copy_service.cc:434] P f3c799b1249e40119d2a5864231c6c75: ending tablet copy session 5b346767a89745fa82928a5dac9cc0b2-7ca8b9a08f4146c386bc06d03c497038 on tablet 7ca8b9a08f4146c386bc06d03c497038 with peer 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:07:57.957039 23904 ts_tablet_manager.cc:933] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2: Initiating tablet copy from peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929)
I20260709 13:07:57.959384 23904 tablet_copy_client.cc:323] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.63.1:33929
I20260709 13:07:57.961119 22890 tablet_copy_service.cc:140] P 0a407504ed1741bf89a276558adf4c0f: Received BeginTabletCopySession request for tablet 4e689c91ad7343b5b6ba36fa648a06ac from peer 5b346767a89745fa82928a5dac9cc0b2 ({username='slave'} at 127.0.0.1:53724)
I20260709 13:07:57.961659 22890 tablet_copy_service.cc:161] P 0a407504ed1741bf89a276558adf4c0f: Beginning new tablet copy session on tablet 4e689c91ad7343b5b6ba36fa648a06ac from peer 5b346767a89745fa82928a5dac9cc0b2 at {username='slave'} at 127.0.0.1:53724: session id = 5b346767a89745fa82928a5dac9cc0b2-4e689c91ad7343b5b6ba36fa648a06ac
I20260709 13:07:57.974381 23909 ts_tablet_manager.cc:933] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2: Initiating tablet copy from peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:07:57.976492 23909 tablet_copy_client.cc:323] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.63.2:33309
I20260709 13:07:57.978081 22965 tablet_copy_service.cc:140] P aed3f089ec8b49af9c8cf5cdee76e8e7: Received BeginTabletCopySession request for tablet 579096ba88474cfbbe40e8bf769879ec from peer 5b346767a89745fa82928a5dac9cc0b2 ({username='slave'} at 127.0.0.1:52222)
I20260709 13:07:57.978581 22965 tablet_copy_service.cc:161] P aed3f089ec8b49af9c8cf5cdee76e8e7: Beginning new tablet copy session on tablet 579096ba88474cfbbe40e8bf769879ec from peer 5b346767a89745fa82928a5dac9cc0b2 at {username='slave'} at 127.0.0.1:52222: session id = 5b346767a89745fa82928a5dac9cc0b2-579096ba88474cfbbe40e8bf769879ec
I20260709 13:07:57.984342 22890 tablet_copy_source_session.cc:215] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:07:57.985306 22965 tablet_copy_source_session.cc:215] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:07:57.987574 23904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4e689c91ad7343b5b6ba36fa648a06ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:57.988396 23909 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 579096ba88474cfbbe40e8bf769879ec. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:58.005249 23904 tablet_copy_client.cc:806] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 0 data blocks...
I20260709 13:07:58.006958 23904 tablet_copy_client.cc:670] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:07:58.018446 23904 tablet_copy_client.cc:538] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:07:58.027016 23909 tablet_copy_client.cc:806] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 0 data blocks...
I20260709 13:07:58.027791 23909 tablet_copy_client.cc:670] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:07:58.035163 23904 tablet_bootstrap.cc:492] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap starting.
I20260709 13:07:58.036146 23909 tablet_copy_client.cc:538] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:07:58.059747 23909 tablet_bootstrap.cc:492] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap starting.
I20260709 13:07:58.060341 23904 tablet_bootstrap.cc:492] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:58.061254 23904 tablet_bootstrap.cc:492] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap complete.
I20260709 13:07:58.062081 23904 ts_tablet_manager.cc:1403] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2: Time spent bootstrapping tablet: real 0.034s	user 0.017s	sys 0.007s
I20260709 13:07:58.065474 23904 raft_consensus.cc:359] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:58.066411 23904 raft_consensus.cc:740] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Initialized, Role: LEARNER
I20260709 13:07:58.067101 23904 consensus_queue.cc:260] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:58.069949 23904 ts_tablet_manager.cc:1434] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:07:58.078027 22890 tablet_copy_service.cc:342] P 0a407504ed1741bf89a276558adf4c0f: Request end of tablet copy session 5b346767a89745fa82928a5dac9cc0b2-4e689c91ad7343b5b6ba36fa648a06ac received from {username='slave'} at 127.0.0.1:53724
I20260709 13:07:58.081463 22890 tablet_copy_service.cc:434] P 0a407504ed1741bf89a276558adf4c0f: ending tablet copy session 5b346767a89745fa82928a5dac9cc0b2-4e689c91ad7343b5b6ba36fa648a06ac on tablet 4e689c91ad7343b5b6ba36fa648a06ac with peer 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:07:58.087632 23904 ts_tablet_manager.cc:933] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2: Initiating tablet copy from peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929)
I20260709 13:07:58.099670 23904 tablet_copy_client.cc:323] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.63.1:33929
I20260709 13:07:58.101464 22890 tablet_copy_service.cc:140] P 0a407504ed1741bf89a276558adf4c0f: Received BeginTabletCopySession request for tablet 13e53d9e00624dfcb87aca9b055fdc9a from peer 5b346767a89745fa82928a5dac9cc0b2 ({username='slave'} at 127.0.0.1:53724)
I20260709 13:07:58.102011 22890 tablet_copy_service.cc:161] P 0a407504ed1741bf89a276558adf4c0f: Beginning new tablet copy session on tablet 13e53d9e00624dfcb87aca9b055fdc9a from peer 5b346767a89745fa82928a5dac9cc0b2 at {username='slave'} at 127.0.0.1:53724: session id = 5b346767a89745fa82928a5dac9cc0b2-13e53d9e00624dfcb87aca9b055fdc9a
I20260709 13:07:58.102392 23909 tablet_bootstrap.cc:492] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:58.103128 23909 tablet_bootstrap.cc:492] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap complete.
I20260709 13:07:58.103688 23909 ts_tablet_manager.cc:1403] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2: Time spent bootstrapping tablet: real 0.044s	user 0.011s	sys 0.010s
I20260709 13:07:58.107009 23909 raft_consensus.cc:359] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:58.107805 23909 raft_consensus.cc:740] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Initialized, Role: LEARNER
I20260709 13:07:58.108393 23909 consensus_queue.cc:260] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:58.110675 23909 ts_tablet_manager.cc:1434] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.003s
I20260709 13:07:58.112654 22965 tablet_copy_service.cc:342] P aed3f089ec8b49af9c8cf5cdee76e8e7: Request end of tablet copy session 5b346767a89745fa82928a5dac9cc0b2-579096ba88474cfbbe40e8bf769879ec received from {username='slave'} at 127.0.0.1:52222
I20260709 13:07:58.113178 22965 tablet_copy_service.cc:434] P aed3f089ec8b49af9c8cf5cdee76e8e7: ending tablet copy session 5b346767a89745fa82928a5dac9cc0b2-579096ba88474cfbbe40e8bf769879ec on tablet 579096ba88474cfbbe40e8bf769879ec with peer 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:07:58.113269 22890 tablet_copy_source_session.cc:215] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:07:58.117305 23904 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 13e53d9e00624dfcb87aca9b055fdc9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:07:58.138756 23904 tablet_copy_client.cc:806] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 0 data blocks...
I20260709 13:07:58.139369 23904 tablet_copy_client.cc:670] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:07:58.142737 23904 tablet_copy_client.cc:538] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:07:58.150682 23904 tablet_bootstrap.cc:492] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap starting.
I20260709 13:07:58.166870 23904 tablet_bootstrap.cc:492] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:07:58.167667 23904 tablet_bootstrap.cc:492] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap complete.
I20260709 13:07:58.168284 23904 ts_tablet_manager.cc:1403] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2: Time spent bootstrapping tablet: real 0.018s	user 0.015s	sys 0.000s
I20260709 13:07:58.170563 23904 raft_consensus.cc:359] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:58.171147 23904 raft_consensus.cc:740] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Initialized, Role: LEARNER
I20260709 13:07:58.171701 23904 consensus_queue.cc:260] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:07:58.173775 23904 ts_tablet_manager.cc:1434] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 13:07:58.175709 22890 tablet_copy_service.cc:342] P 0a407504ed1741bf89a276558adf4c0f: Request end of tablet copy session 5b346767a89745fa82928a5dac9cc0b2-13e53d9e00624dfcb87aca9b055fdc9a received from {username='slave'} at 127.0.0.1:53724
I20260709 13:07:58.176105 22890 tablet_copy_service.cc:434] P 0a407504ed1741bf89a276558adf4c0f: ending tablet copy session 5b346767a89745fa82928a5dac9cc0b2-13e53d9e00624dfcb87aca9b055fdc9a on tablet 13e53d9e00624dfcb87aca9b055fdc9a with peer 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:07:58.252887 23623 raft_consensus.cc:1217] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:07:58.379719 23623 raft_consensus.cc:1217] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:07:58.445770 23623 raft_consensus.cc:1217] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:07:58.501834 23623 raft_consensus.cc:1217] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:07:58.699781 23884 raft_consensus.cc:1064] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
I20260709 13:07:58.701854 23884 consensus_queue.cc:237] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:07:58.708941 23623 raft_consensus.cc:1275] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:58.711174 22954 raft_consensus.cc:1275] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:58.712044 23884 consensus_queue.cc:1048] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:58.712981 23911 consensus_queue.cc:1048] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:58.713724 22880 raft_consensus.cc:1275] T 7ca8b9a08f4146c386bc06d03c497038 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:58.716594 23911 consensus_queue.cc:1048] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:58.740969 23735 raft_consensus.cc:2955] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:58.745354 22872 raft_consensus.cc:2955] T 7ca8b9a08f4146c386bc06d03c497038 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:58.751884 23623 raft_consensus.cc:2955] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:58.768415 22777 catalog_manager.cc:5697] T 7ca8b9a08f4146c386bc06d03c497038 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:58.773340 22955 raft_consensus.cc:2955] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:58.849300 23029 consensus_queue.cc:237] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:07:58.854106 23623 raft_consensus.cc:1275] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:58.854102 22955 raft_consensus.cc:1275] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:58.855784 23923 consensus_queue.cc:1048] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:07:58.856643 23735 consensus_queue.cc:1048] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:58.863446 23922 raft_consensus.cc:2955] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:58.865285 22955 raft_consensus.cc:2955] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:58.870874 23623 raft_consensus.cc:2955] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:58.888106 22762 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7ca8b9a08f4146c386bc06d03c497038 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:07:58.892629 22776 catalog_manager.cc:5697] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: config changed from index 3 to 4, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:58.901762 23896 raft_consensus.cc:1064] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
I20260709 13:07:58.905118 22867 tablet_service.cc:1558] Processing DeleteTablet for tablet 7ca8b9a08f4146c386bc06d03c497038 with delete_type TABLET_DATA_TOMBSTONED (TS 0a407504ed1741bf89a276558adf4c0f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:35344
I20260709 13:07:58.906574 23896 consensus_queue.cc:237] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:07:58.928719 23029 raft_consensus.cc:1275] T 579096ba88474cfbbe40e8bf769879ec P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:58.930069 23623 raft_consensus.cc:1275] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:58.930938 23894 consensus_queue.cc:1048] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:58.930928 23928 tablet_replica.cc:333] T 7ca8b9a08f4146c386bc06d03c497038 P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:07:58.934984 22880 raft_consensus.cc:1275] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:58.936977 23894 consensus_queue.cc:1048] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:58.938654 23894 consensus_queue.cc:1048] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:58.946519 23928 raft_consensus.cc:2243] T 7ca8b9a08f4146c386bc06d03c497038 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:58.947278 23928 raft_consensus.cc:2272] T 7ca8b9a08f4146c386bc06d03c497038 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:58.955214 23849 raft_consensus.cc:1064] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
I20260709 13:07:58.957011 23849 consensus_queue.cc:237] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:07:58.961408 23928 ts_tablet_manager.cc:1916] T 7ca8b9a08f4146c386bc06d03c497038 P 0a407504ed1741bf89a276558adf4c0f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:07:58.960392 23851 raft_consensus.cc:2955] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:58.974529 23623 raft_consensus.cc:1275] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:58.976114 23625 raft_consensus.cc:2955] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:58.978997 23029 raft_consensus.cc:2955] T 579096ba88474cfbbe40e8bf769879ec P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:58.984736 23917 consensus_queue.cc:1048] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:58.995270 22872 raft_consensus.cc:2955] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:59.003804 22776 catalog_manager.cc:5697] T 579096ba88474cfbbe40e8bf769879ec P f3c799b1249e40119d2a5864231c6c75 reported cstate change: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:59.007433 22953 raft_consensus.cc:1275] T 4e689c91ad7343b5b6ba36fa648a06ac P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:59.008826 23029 raft_consensus.cc:1275] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:59.010049 23890 consensus_queue.cc:1048] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:59.013767 23928 ts_tablet_manager.cc:1929] T 7ca8b9a08f4146c386bc06d03c497038 P 0a407504ed1741bf89a276558adf4c0f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:07:59.012579 23890 consensus_queue.cc:1048] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:07:59.014236 23928 log.cc:1199] T 7ca8b9a08f4146c386bc06d03c497038 P 0a407504ed1741bf89a276558adf4c0f: Deleting WAL directory at /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-0-root/wals/7ca8b9a08f4146c386bc06d03c497038
I20260709 13:07:59.016348 22764 catalog_manager.cc:5028] TS 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): tablet 7ca8b9a08f4146c386bc06d03c497038 (table test-workload [id=a42f2ccfbe484baca3019323a490c64a]) successfully deleted
I20260709 13:07:59.021293 23917 raft_consensus.cc:1064] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
I20260709 13:07:59.024920 23917 consensus_queue.cc:237] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:07:59.049062 22953 raft_consensus.cc:2955] T 4e689c91ad7343b5b6ba36fa648a06ac P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:59.048609 23890 raft_consensus.cc:2955] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:59.071354 22955 consensus_queue.cc:237] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:07:59.077505 23623 raft_consensus.cc:1275] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:59.080386 23894 consensus_queue.cc:1048] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:59.086298 22872 raft_consensus.cc:1275] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:59.091346 23894 consensus_queue.cc:1048] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:59.096716 23623 raft_consensus.cc:1275] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:59.101903 23625 raft_consensus.cc:2955] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:59.103762 23941 consensus_queue.cc:1048] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:59.106623 23894 raft_consensus.cc:2955] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:59.124256 22872 raft_consensus.cc:2955] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
W20260709 13:07:59.129350 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 53039.6us
I20260709 13:07:59.133997 23029 raft_consensus.cc:1275] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:59.134248 23624 raft_consensus.cc:2955] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:59.139070 22955 raft_consensus.cc:1275] T 13e53d9e00624dfcb87aca9b055fdc9a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:07:59.125697 22776 catalog_manager.cc:5697] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: config changed from index 3 to 4, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New cstate: current_term: 1 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:59.143705 23917 consensus_queue.cc:1048] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:59.151440 23023 raft_consensus.cc:2955] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:59.156260 22762 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 579096ba88474cfbbe40e8bf769879ec with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:07:59.167263 23941 consensus_queue.cc:1048] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:07:59.171552 22774 catalog_manager.cc:5697] T 4e689c91ad7343b5b6ba36fa648a06ac P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:59.176890 23019 tablet_service.cc:1558] Processing DeleteTablet for tablet 579096ba88474cfbbe40e8bf769879ec with delete_type TABLET_DATA_TOMBSTONED (TS f3c799b1249e40119d2a5864231c6c75 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33406
I20260709 13:07:59.184801 23936 raft_consensus.cc:2955] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:59.190158 23948 tablet_replica.cc:333] T 579096ba88474cfbbe40e8bf769879ec P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:07:59.191316 23948 raft_consensus.cc:2243] T 579096ba88474cfbbe40e8bf769879ec P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:59.192019 23948 raft_consensus.cc:2272] T 579096ba88474cfbbe40e8bf769879ec P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:59.194411 22955 raft_consensus.cc:2955] T 13e53d9e00624dfcb87aca9b055fdc9a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:59.196101 23948 ts_tablet_manager.cc:1916] T 579096ba88474cfbbe40e8bf769879ec P f3c799b1249e40119d2a5864231c6c75: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:07:59.196013 23029 raft_consensus.cc:2955] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:59.218884 22872 consensus_queue.cc:237] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:07:59.224917 23624 raft_consensus.cc:2955] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:59.230477 22778 catalog_manager.cc:5697] T 13e53d9e00624dfcb87aca9b055fdc9a P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:59.234390 23948 ts_tablet_manager.cc:1929] T 579096ba88474cfbbe40e8bf769879ec P f3c799b1249e40119d2a5864231c6c75: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:07:59.234833 23948 log.cc:1199] T 579096ba88474cfbbe40e8bf769879ec P f3c799b1249e40119d2a5864231c6c75: Deleting WAL directory at /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-2-root/wals/579096ba88474cfbbe40e8bf769879ec
I20260709 13:07:59.236404 22762 catalog_manager.cc:5028] TS f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): tablet 579096ba88474cfbbe40e8bf769879ec (table test-workload [id=a42f2ccfbe484baca3019323a490c64a]) successfully deleted
I20260709 13:07:59.239379 23625 raft_consensus.cc:1275] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:59.241447 23917 consensus_queue.cc:1048] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:07:59.247663 23020 raft_consensus.cc:1275] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:59.261612 23937 consensus_queue.cc:1048] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:59.264788 23941 raft_consensus.cc:2955] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:59.266717 23625 raft_consensus.cc:2955] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:59.262254 22872 consensus_queue.cc:237] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:07:59.274436 23023 raft_consensus.cc:2955] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:59.287021 23625 raft_consensus.cc:1275] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:59.290040 23937 consensus_queue.cc:1048] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:59.295486 23020 raft_consensus.cc:1275] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:07:59.294973 22778 catalog_manager.cc:5697] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: config changed from index 3 to 4, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:59.296828 23937 consensus_queue.cc:1048] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:07:59.305368 22764 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4e689c91ad7343b5b6ba36fa648a06ac with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:07:59.308272 22943 tablet_service.cc:1558] Processing DeleteTablet for tablet 4e689c91ad7343b5b6ba36fa648a06ac with delete_type TABLET_DATA_TOMBSTONED (TS aed3f089ec8b49af9c8cf5cdee76e8e7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58302
I20260709 13:07:59.314704 23624 raft_consensus.cc:2955] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:59.317308 23917 raft_consensus.cc:2955] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:59.318594 23949 tablet_replica.cc:333] T 4e689c91ad7343b5b6ba36fa648a06ac P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:07:59.319782 23949 raft_consensus.cc:2243] T 4e689c91ad7343b5b6ba36fa648a06ac P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:59.320518 23949 raft_consensus.cc:2272] T 4e689c91ad7343b5b6ba36fa648a06ac P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:59.323815 23949 ts_tablet_manager.cc:1916] T 4e689c91ad7343b5b6ba36fa648a06ac P aed3f089ec8b49af9c8cf5cdee76e8e7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:07:59.337996 22778 catalog_manager.cc:5697] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: config changed from index 3 to 4, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:59.338863 23029 raft_consensus.cc:2955] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:07:59.339902 22764 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 13e53d9e00624dfcb87aca9b055fdc9a with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:07:59.350000 22943 tablet_service.cc:1558] Processing DeleteTablet for tablet 13e53d9e00624dfcb87aca9b055fdc9a with delete_type TABLET_DATA_TOMBSTONED (TS aed3f089ec8b49af9c8cf5cdee76e8e7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58302
I20260709 13:07:59.356350 23949 ts_tablet_manager.cc:1929] T 4e689c91ad7343b5b6ba36fa648a06ac P aed3f089ec8b49af9c8cf5cdee76e8e7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:07:59.356853 23949 log.cc:1199] T 4e689c91ad7343b5b6ba36fa648a06ac P aed3f089ec8b49af9c8cf5cdee76e8e7: Deleting WAL directory at /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-1-root/wals/4e689c91ad7343b5b6ba36fa648a06ac
I20260709 13:07:59.358407 22762 catalog_manager.cc:5028] TS aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): tablet 4e689c91ad7343b5b6ba36fa648a06ac (table test-workload [id=a42f2ccfbe484baca3019323a490c64a]) successfully deleted
I20260709 13:07:59.359279 23949 tablet_replica.cc:333] T 13e53d9e00624dfcb87aca9b055fdc9a P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:07:59.372359 23949 raft_consensus.cc:2243] T 13e53d9e00624dfcb87aca9b055fdc9a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:07:59.373168 23949 raft_consensus.cc:2272] T 13e53d9e00624dfcb87aca9b055fdc9a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:07:59.376296 23949 ts_tablet_manager.cc:1916] T 13e53d9e00624dfcb87aca9b055fdc9a P aed3f089ec8b49af9c8cf5cdee76e8e7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:07:59.388630 23949 ts_tablet_manager.cc:1929] T 13e53d9e00624dfcb87aca9b055fdc9a P aed3f089ec8b49af9c8cf5cdee76e8e7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:07:59.388967 23949 log.cc:1199] T 13e53d9e00624dfcb87aca9b055fdc9a P aed3f089ec8b49af9c8cf5cdee76e8e7: Deleting WAL directory at /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-1-root/wals/13e53d9e00624dfcb87aca9b055fdc9a
I20260709 13:07:59.390049 22762 catalog_manager.cc:5028] TS aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): tablet 13e53d9e00624dfcb87aca9b055fdc9a (table test-workload [id=a42f2ccfbe484baca3019323a490c64a]) successfully deleted
I20260709 13:08:00.427101 22876 consensus_queue.cc:237] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
W20260709 13:08:00.438457 22844 consensus_peers.cc:597] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f -> Peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Couldn't send request to peer 5b346767a89745fa82928a5dac9cc0b2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 91c88873d54149c58a20f4cf6194e5fc. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:00.439025 22954 raft_consensus.cc:1275] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:08:00.438452 23029 raft_consensus.cc:1275] T 91c88873d54149c58a20f4cf6194e5fc P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:08:00.440630 23953 consensus_queue.cc:1048] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:00.441735 23936 consensus_queue.cc:1048] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:00.453861 23936 raft_consensus.cc:2955] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:00.455430 22954 raft_consensus.cc:2955] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:00.456030 23029 raft_consensus.cc:2955] T 91c88873d54149c58a20f4cf6194e5fc P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:00.476564 22778 catalog_manager.cc:5697] T 91c88873d54149c58a20f4cf6194e5fc P f3c799b1249e40119d2a5864231c6c75 reported cstate change: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:00.478518 23023 consensus_queue.cc:237] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
W20260709 13:08:00.498991 22993 consensus_peers.cc:597] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 -> Peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Couldn't send request to peer 5b346767a89745fa82928a5dac9cc0b2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6eba8b8b84ca4a9aa5d9eaca94670686. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:00.499047 22876 raft_consensus.cc:1275] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:08:00.501435 23960 consensus_queue.cc:1048] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:00.503674 22954 raft_consensus.cc:1275] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:08:00.505182 23960 consensus_queue.cc:1048] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:00.525646 23922 raft_consensus.cc:2955] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:00.528074 22955 raft_consensus.cc:2955] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:00.534343 22880 raft_consensus.cc:2955] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:00.553115 22880 consensus_queue.cc:237] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:00.575124 22777 catalog_manager.cc:5697] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:00.581281 22844 consensus_peers.cc:597] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f -> Peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Couldn't send request to peer 5b346767a89745fa82928a5dac9cc0b2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 698093275c264d4d99fb8c1a49e3b28f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:00.581954 23029 raft_consensus.cc:1275] T 698093275c264d4d99fb8c1a49e3b28f P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:08:00.583345 23956 consensus_queue.cc:1048] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:00.592192 22955 raft_consensus.cc:1275] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:08:00.593570 23936 consensus_queue.cc:1048] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:00.599700 23956 raft_consensus.cc:2955] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:00.601758 23029 raft_consensus.cc:2955] T 698093275c264d4d99fb8c1a49e3b28f P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:00.604341 22955 raft_consensus.cc:2955] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:00.614424 22954 consensus_queue.cc:237] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:00.619223 22777 catalog_manager.cc:5697] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f reported cstate change: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:00.633858 22919 consensus_peers.cc:597] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Couldn't send request to peer 5b346767a89745fa82928a5dac9cc0b2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 67e7fdd613434ffaacb8a030d4693490. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:00.637454 23020 raft_consensus.cc:1275] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:08:00.636353 22880 raft_consensus.cc:1275] T 67e7fdd613434ffaacb8a030d4693490 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:08:00.639586 23957 consensus_queue.cc:1048] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:00.643455 23957 consensus_queue.cc:1048] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:00.655043 23020 raft_consensus.cc:2955] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:00.655404 23851 raft_consensus.cc:2955] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:00.661517 22880 raft_consensus.cc:2955] T 67e7fdd613434ffaacb8a030d4693490 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:00.676231 22776 catalog_manager.cc:5697] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New cstate: current_term: 1 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:00.893226 23973 ts_tablet_manager.cc:933] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2: Initiating tablet copy from peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:00.900305 23973 tablet_copy_client.cc:323] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.63.3:42499
I20260709 13:08:00.902902 23039 tablet_copy_service.cc:140] P f3c799b1249e40119d2a5864231c6c75: Received BeginTabletCopySession request for tablet 6eba8b8b84ca4a9aa5d9eaca94670686 from peer 5b346767a89745fa82928a5dac9cc0b2 ({username='slave'} at 127.0.0.1:32794)
I20260709 13:08:00.905531 23039 tablet_copy_service.cc:161] P f3c799b1249e40119d2a5864231c6c75: Beginning new tablet copy session on tablet 6eba8b8b84ca4a9aa5d9eaca94670686 from peer 5b346767a89745fa82928a5dac9cc0b2 at {username='slave'} at 127.0.0.1:32794: session id = 5b346767a89745fa82928a5dac9cc0b2-6eba8b8b84ca4a9aa5d9eaca94670686
I20260709 13:08:00.912860 23039 tablet_copy_source_session.cc:215] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:08:00.915860 23973 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6eba8b8b84ca4a9aa5d9eaca94670686. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:00.932210 23973 tablet_copy_client.cc:806] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 0 data blocks...
I20260709 13:08:00.932895 23973 tablet_copy_client.cc:670] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:08:00.936746 23973 tablet_copy_client.cc:538] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:08:00.950412 23973 tablet_bootstrap.cc:492] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap starting.
I20260709 13:08:00.986667 23976 ts_tablet_manager.cc:933] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2: Initiating tablet copy from peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929)
I20260709 13:08:00.988799 23976 tablet_copy_client.cc:323] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.63.1:33929
I20260709 13:08:00.991595 22890 tablet_copy_service.cc:140] P 0a407504ed1741bf89a276558adf4c0f: Received BeginTabletCopySession request for tablet 91c88873d54149c58a20f4cf6194e5fc from peer 5b346767a89745fa82928a5dac9cc0b2 ({username='slave'} at 127.0.0.1:53724)
I20260709 13:08:00.992364 22890 tablet_copy_service.cc:161] P 0a407504ed1741bf89a276558adf4c0f: Beginning new tablet copy session on tablet 91c88873d54149c58a20f4cf6194e5fc from peer 5b346767a89745fa82928a5dac9cc0b2 at {username='slave'} at 127.0.0.1:53724: session id = 5b346767a89745fa82928a5dac9cc0b2-91c88873d54149c58a20f4cf6194e5fc
I20260709 13:08:00.994891 23973 tablet_bootstrap.cc:492] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:00.995913 23973 tablet_bootstrap.cc:492] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap complete.
I20260709 13:08:00.997215 23973 ts_tablet_manager.cc:1403] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2: Time spent bootstrapping tablet: real 0.047s	user 0.015s	sys 0.023s
I20260709 13:08:01.000293 23973 raft_consensus.cc:359] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:01.001168 23973 raft_consensus.cc:740] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Initialized, Role: LEARNER
I20260709 13:08:01.001930 23973 consensus_queue.cc:260] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:01.004014 22890 tablet_copy_source_session.cc:215] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:08:01.009768 23976 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 91c88873d54149c58a20f4cf6194e5fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:01.014937 23973 ts_tablet_manager.cc:1434] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2: Time spent starting tablet: real 0.017s	user 0.004s	sys 0.012s
I20260709 13:08:01.018647 23039 tablet_copy_service.cc:342] P f3c799b1249e40119d2a5864231c6c75: Request end of tablet copy session 5b346767a89745fa82928a5dac9cc0b2-6eba8b8b84ca4a9aa5d9eaca94670686 received from {username='slave'} at 127.0.0.1:32794
I20260709 13:08:01.019132 23039 tablet_copy_service.cc:434] P f3c799b1249e40119d2a5864231c6c75: ending tablet copy session 5b346767a89745fa82928a5dac9cc0b2-6eba8b8b84ca4a9aa5d9eaca94670686 on tablet 6eba8b8b84ca4a9aa5d9eaca94670686 with peer 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:01.034416 23976 tablet_copy_client.cc:806] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 0 data blocks...
I20260709 13:08:01.034965 23976 tablet_copy_client.cc:670] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:08:01.038698 23976 tablet_copy_client.cc:538] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:08:01.047017 23976 tablet_bootstrap.cc:492] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap starting.
I20260709 13:08:01.065546 23976 tablet_bootstrap.cc:492] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:01.066413 23976 tablet_bootstrap.cc:492] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap complete.
I20260709 13:08:01.067056 23976 ts_tablet_manager.cc:1403] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2: Time spent bootstrapping tablet: real 0.020s	user 0.015s	sys 0.004s
I20260709 13:08:01.069932 23976 raft_consensus.cc:359] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:01.070750 23976 raft_consensus.cc:740] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Initialized, Role: LEARNER
I20260709 13:08:01.071377 23976 consensus_queue.cc:260] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:01.073792 23976 ts_tablet_manager.cc:1434] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:08:01.075516 22890 tablet_copy_service.cc:342] P 0a407504ed1741bf89a276558adf4c0f: Request end of tablet copy session 5b346767a89745fa82928a5dac9cc0b2-91c88873d54149c58a20f4cf6194e5fc received from {username='slave'} at 127.0.0.1:53724
I20260709 13:08:01.075914 22890 tablet_copy_service.cc:434] P 0a407504ed1741bf89a276558adf4c0f: ending tablet copy session 5b346767a89745fa82928a5dac9cc0b2-91c88873d54149c58a20f4cf6194e5fc on tablet 91c88873d54149c58a20f4cf6194e5fc with peer 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:01.078622 23973 ts_tablet_manager.cc:933] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2: Initiating tablet copy from peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929)
I20260709 13:08:01.081797 23973 tablet_copy_client.cc:323] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.63.1:33929
I20260709 13:08:01.083263 22890 tablet_copy_service.cc:140] P 0a407504ed1741bf89a276558adf4c0f: Received BeginTabletCopySession request for tablet 698093275c264d4d99fb8c1a49e3b28f from peer 5b346767a89745fa82928a5dac9cc0b2 ({username='slave'} at 127.0.0.1:53724)
I20260709 13:08:01.083760 22890 tablet_copy_service.cc:161] P 0a407504ed1741bf89a276558adf4c0f: Beginning new tablet copy session on tablet 698093275c264d4d99fb8c1a49e3b28f from peer 5b346767a89745fa82928a5dac9cc0b2 at {username='slave'} at 127.0.0.1:53724: session id = 5b346767a89745fa82928a5dac9cc0b2-698093275c264d4d99fb8c1a49e3b28f
I20260709 13:08:01.091377 22890 tablet_copy_source_session.cc:215] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:08:01.102937 23973 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 698093275c264d4d99fb8c1a49e3b28f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:01.118822 23973 tablet_copy_client.cc:806] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 0 data blocks...
I20260709 13:08:01.119372 23973 tablet_copy_client.cc:670] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:08:01.125471 23973 tablet_copy_client.cc:538] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:08:01.134826 23973 tablet_bootstrap.cc:492] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap starting.
I20260709 13:08:01.143661 23976 ts_tablet_manager.cc:933] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2: Initiating tablet copy from peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:01.157486 23976 tablet_copy_client.cc:323] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.63.2:33309
I20260709 13:08:01.159428 22965 tablet_copy_service.cc:140] P aed3f089ec8b49af9c8cf5cdee76e8e7: Received BeginTabletCopySession request for tablet 67e7fdd613434ffaacb8a030d4693490 from peer 5b346767a89745fa82928a5dac9cc0b2 ({username='slave'} at 127.0.0.1:52222)
I20260709 13:08:01.159965 22965 tablet_copy_service.cc:161] P aed3f089ec8b49af9c8cf5cdee76e8e7: Beginning new tablet copy session on tablet 67e7fdd613434ffaacb8a030d4693490 from peer 5b346767a89745fa82928a5dac9cc0b2 at {username='slave'} at 127.0.0.1:52222: session id = 5b346767a89745fa82928a5dac9cc0b2-67e7fdd613434ffaacb8a030d4693490
I20260709 13:08:01.170524 22965 tablet_copy_source_session.cc:215] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:08:01.177516 23976 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 67e7fdd613434ffaacb8a030d4693490. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:01.201618 23976 tablet_copy_client.cc:806] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 0 data blocks...
I20260709 13:08:01.202710 23976 tablet_copy_client.cc:670] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:08:01.206122 23973 tablet_bootstrap.cc:492] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:01.206787 23976 tablet_copy_client.cc:538] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:08:01.207063 23973 tablet_bootstrap.cc:492] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap complete.
I20260709 13:08:01.207844 23973 ts_tablet_manager.cc:1403] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2: Time spent bootstrapping tablet: real 0.073s	user 0.029s	sys 0.037s
I20260709 13:08:01.210920 23973 raft_consensus.cc:359] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:01.211745 23973 raft_consensus.cc:740] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Initialized, Role: LEARNER
I20260709 13:08:01.212394 23973 consensus_queue.cc:260] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:01.215971 23976 tablet_bootstrap.cc:492] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap starting.
I20260709 13:08:01.216706 23973 ts_tablet_manager.cc:1434] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.001s
I20260709 13:08:01.218603 22890 tablet_copy_service.cc:342] P 0a407504ed1741bf89a276558adf4c0f: Request end of tablet copy session 5b346767a89745fa82928a5dac9cc0b2-698093275c264d4d99fb8c1a49e3b28f received from {username='slave'} at 127.0.0.1:53724
I20260709 13:08:01.219038 22890 tablet_copy_service.cc:434] P 0a407504ed1741bf89a276558adf4c0f: ending tablet copy session 5b346767a89745fa82928a5dac9cc0b2-698093275c264d4d99fb8c1a49e3b28f on tablet 698093275c264d4d99fb8c1a49e3b28f with peer 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:01.253661 23976 tablet_bootstrap.cc:492] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:01.254673 23976 tablet_bootstrap.cc:492] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap complete.
I20260709 13:08:01.255424 23976 ts_tablet_manager.cc:1403] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2: Time spent bootstrapping tablet: real 0.040s	user 0.019s	sys 0.015s
I20260709 13:08:01.258531 23976 raft_consensus.cc:359] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:01.259419 23976 raft_consensus.cc:740] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Initialized, Role: LEARNER
I20260709 13:08:01.260088 23976 consensus_queue.cc:260] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:01.262872 23976 ts_tablet_manager.cc:1434] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2: Time spent starting tablet: real 0.007s	user 0.009s	sys 0.000s
I20260709 13:08:01.264705 22965 tablet_copy_service.cc:342] P aed3f089ec8b49af9c8cf5cdee76e8e7: Request end of tablet copy session 5b346767a89745fa82928a5dac9cc0b2-67e7fdd613434ffaacb8a030d4693490 received from {username='slave'} at 127.0.0.1:52222
I20260709 13:08:01.265157 22965 tablet_copy_service.cc:434] P aed3f089ec8b49af9c8cf5cdee76e8e7: ending tablet copy session 5b346767a89745fa82928a5dac9cc0b2-67e7fdd613434ffaacb8a030d4693490 on tablet 67e7fdd613434ffaacb8a030d4693490 with peer 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:01.348057 23624 raft_consensus.cc:1217] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:08:01.519919 23624 raft_consensus.cc:1217] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:08:01.533691 23624 raft_consensus.cc:1217] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:08:01.677042 23623 raft_consensus.cc:1217] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:08:01.908351 23922 raft_consensus.cc:1064] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
I20260709 13:08:01.911718 23922 consensus_queue.cc:237] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:01.922873 23623 raft_consensus.cc:1275] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:08:01.925792 23951 consensus_queue.cc:1048] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:01.927906 22955 raft_consensus.cc:1275] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:08:01.931599 22880 raft_consensus.cc:1275] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:08:01.933000 23984 consensus_queue.cc:1048] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:01.943274 23984 consensus_queue.cc:1048] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:01.946244 23956 raft_consensus.cc:1064] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
I20260709 13:08:01.952988 23956 consensus_queue.cc:237] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:01.993616 22953 raft_consensus.cc:1275] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:08:01.993655 23623 raft_consensus.cc:1275] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 13:08:01.996402 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 72298.1us
I20260709 13:08:01.998315 23945 consensus_queue.cc:1048] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:01.997530 23988 raft_consensus.cc:2955] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:02.004812 23956 consensus_queue.cc:1048] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:02.012364 22953 raft_consensus.cc:2955] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:02.016268 23623 raft_consensus.cc:2955] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:02.030720 22777 catalog_manager.cc:5697] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:02.043077 22880 raft_consensus.cc:2955] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:02.057189 23023 raft_consensus.cc:1275] T 91c88873d54149c58a20f4cf6194e5fc P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:08:02.066299 23956 raft_consensus.cc:2955] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:02.069418 23945 consensus_queue.cc:1048] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:02.067286 22955 raft_consensus.cc:2955] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:02.083926 23623 raft_consensus.cc:2955] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:02.105619 23029 raft_consensus.cc:2955] T 91c88873d54149c58a20f4cf6194e5fc P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:02.128253 22778 catalog_manager.cc:5697] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:02.132781 23985 raft_consensus.cc:1064] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
I20260709 13:08:02.135476 23985 consensus_queue.cc:237] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:02.147166 22880 consensus_queue.cc:237] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:02.148203 23029 raft_consensus.cc:1275] T 698093275c264d4d99fb8c1a49e3b28f P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:08:02.149966 23945 consensus_queue.cc:1048] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:02.150213 23623 raft_consensus.cc:1275] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:08:02.151631 23936 consensus_queue.cc:1048] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:02.154695 22955 raft_consensus.cc:1275] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:08:02.155948 23936 consensus_queue.cc:1048] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:02.161371 23624 raft_consensus.cc:1275] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:08:02.162904 23936 consensus_queue.cc:1048] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:02.166769 22954 raft_consensus.cc:1275] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:08:02.169782 23945 consensus_queue.cc:1048] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:02.172248 23945 raft_consensus.cc:2955] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:02.173286 23624 raft_consensus.cc:2955] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:02.186535 22764 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 91c88873d54149c58a20f4cf6194e5fc with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:08:02.184907 23956 raft_consensus.cc:2955] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:02.188724 22955 raft_consensus.cc:2955] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:02.189221 23023 raft_consensus.cc:2955] T 698093275c264d4d99fb8c1a49e3b28f P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:02.201790 23623 raft_consensus.cc:2955] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:02.200532 22777 catalog_manager.cc:5697] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f reported cstate change: config changed from index 3 to 4, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:02.218914 22954 raft_consensus.cc:2955] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:02.230042 22776 catalog_manager.cc:5697] T 698093275c264d4d99fb8c1a49e3b28f P f3c799b1249e40119d2a5864231c6c75 reported cstate change: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:02.233352 23019 tablet_service.cc:1558] Processing DeleteTablet for tablet 91c88873d54149c58a20f4cf6194e5fc with delete_type TABLET_DATA_TOMBSTONED (TS f3c799b1249e40119d2a5864231c6c75 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33406
I20260709 13:08:02.249861 24005 tablet_replica.cc:333] T 91c88873d54149c58a20f4cf6194e5fc P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:08:02.251170 24005 raft_consensus.cc:2243] T 91c88873d54149c58a20f4cf6194e5fc P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:02.258466 24005 raft_consensus.cc:2272] T 91c88873d54149c58a20f4cf6194e5fc P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:02.258616 23894 raft_consensus.cc:1064] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
I20260709 13:08:02.261145 23894 consensus_queue.cc:237] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:02.262661 24005 ts_tablet_manager.cc:1916] T 91c88873d54149c58a20f4cf6194e5fc P f3c799b1249e40119d2a5864231c6c75: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:08:02.272467 22880 consensus_queue.cc:237] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:02.280067 23623 raft_consensus.cc:1275] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:08:02.280930 23023 raft_consensus.cc:1275] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:08:02.282646 23623 raft_consensus.cc:1275] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:08:02.282729 22955 raft_consensus.cc:1275] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:08:02.284296 23936 consensus_queue.cc:1048] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:02.286649 23972 consensus_queue.cc:1048] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:02.287464 24005 ts_tablet_manager.cc:1929] T 91c88873d54149c58a20f4cf6194e5fc P f3c799b1249e40119d2a5864231c6c75: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:08:02.288005 24005 log.cc:1199] T 91c88873d54149c58a20f4cf6194e5fc P f3c799b1249e40119d2a5864231c6c75: Deleting WAL directory at /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-2-root/wals/91c88873d54149c58a20f4cf6194e5fc
I20260709 13:08:02.290302 22762 catalog_manager.cc:5028] TS f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): tablet 91c88873d54149c58a20f4cf6194e5fc (table test-workload [id=a42f2ccfbe484baca3019323a490c64a]) successfully deleted
I20260709 13:08:02.290746 23936 consensus_queue.cc:1048] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:02.295902 23957 consensus_queue.cc:1048] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:02.295404 23956 raft_consensus.cc:2955] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:02.308833 23624 raft_consensus.cc:2955] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
W20260709 13:08:02.340416 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 47615.3us
I20260709 13:08:02.340917 22776 catalog_manager.cc:5697] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: config changed from index 3 to 4, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:02.349439 22764 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 698093275c264d4d99fb8c1a49e3b28f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:08:02.363255 22876 raft_consensus.cc:1275] T 67e7fdd613434ffaacb8a030d4693490 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:08:02.368017 23957 consensus_queue.cc:1048] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:02.369184 23019 tablet_service.cc:1558] Processing DeleteTablet for tablet 698093275c264d4d99fb8c1a49e3b28f with delete_type TABLET_DATA_TOMBSTONED (TS f3c799b1249e40119d2a5864231c6c75 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33406
I20260709 13:08:02.366379 22955 raft_consensus.cc:2955] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:02.370221 24005 tablet_replica.cc:333] T 698093275c264d4d99fb8c1a49e3b28f P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:08:02.391934 24005 raft_consensus.cc:2243] T 698093275c264d4d99fb8c1a49e3b28f P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:02.392720 24005 raft_consensus.cc:2272] T 698093275c264d4d99fb8c1a49e3b28f P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:02.398612 24005 ts_tablet_manager.cc:1916] T 698093275c264d4d99fb8c1a49e3b28f P f3c799b1249e40119d2a5864231c6c75: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:08:02.397398 24009 raft_consensus.cc:2955] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:02.399852 22880 raft_consensus.cc:2955] T 67e7fdd613434ffaacb8a030d4693490 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:02.407754 23023 raft_consensus.cc:2955] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:02.412163 24005 ts_tablet_manager.cc:1929] T 698093275c264d4d99fb8c1a49e3b28f P f3c799b1249e40119d2a5864231c6c75: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:08:02.412595 24005 log.cc:1199] T 698093275c264d4d99fb8c1a49e3b28f P f3c799b1249e40119d2a5864231c6c75: Deleting WAL directory at /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-2-root/wals/698093275c264d4d99fb8c1a49e3b28f
I20260709 13:08:02.414836 22762 catalog_manager.cc:5028] TS f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): tablet 698093275c264d4d99fb8c1a49e3b28f (table test-workload [id=a42f2ccfbe484baca3019323a490c64a]) successfully deleted
I20260709 13:08:02.420583 23624 raft_consensus.cc:2955] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:02.424214 22776 catalog_manager.cc:5697] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:02.458278 22955 consensus_queue.cc:237] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:02.465065 23624 raft_consensus.cc:1275] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:08:02.467073 23023 raft_consensus.cc:1275] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:08:02.467303 24009 consensus_queue.cc:1048] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:02.469094 23894 consensus_queue.cc:1048] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:02.475292 23957 raft_consensus.cc:2955] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:02.481017 23624 raft_consensus.cc:2955] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:02.485400 23023 raft_consensus.cc:2955] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:02.488703 22762 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 67e7fdd613434ffaacb8a030d4693490 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:08:02.498289 22778 catalog_manager.cc:5697] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: config changed from index 3 to 4, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New cstate: current_term: 1 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:02.510404 22867 tablet_service.cc:1558] Processing DeleteTablet for tablet 67e7fdd613434ffaacb8a030d4693490 with delete_type TABLET_DATA_TOMBSTONED (TS 0a407504ed1741bf89a276558adf4c0f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:35344
I20260709 13:08:02.520936 24012 tablet_replica.cc:333] T 67e7fdd613434ffaacb8a030d4693490 P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:08:02.522037 24012 raft_consensus.cc:2243] T 67e7fdd613434ffaacb8a030d4693490 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:02.522678 24012 raft_consensus.cc:2272] T 67e7fdd613434ffaacb8a030d4693490 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:02.525879 24012 ts_tablet_manager.cc:1916] T 67e7fdd613434ffaacb8a030d4693490 P 0a407504ed1741bf89a276558adf4c0f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:08:02.538861 24012 ts_tablet_manager.cc:1929] T 67e7fdd613434ffaacb8a030d4693490 P 0a407504ed1741bf89a276558adf4c0f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:08:02.539228 24012 log.cc:1199] T 67e7fdd613434ffaacb8a030d4693490 P 0a407504ed1741bf89a276558adf4c0f: Deleting WAL directory at /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-0-root/wals/67e7fdd613434ffaacb8a030d4693490
I20260709 13:08:02.540714 22764 catalog_manager.cc:5028] TS 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): tablet 67e7fdd613434ffaacb8a030d4693490 (table test-workload [id=a42f2ccfbe484baca3019323a490c64a]) successfully deleted
I20260709 13:08:02.624963 23023 consensus_queue.cc:237] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:02.630023 23624 raft_consensus.cc:1275] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:08:02.630595 22954 raft_consensus.cc:1275] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:08:02.631618 23984 consensus_queue.cc:1048] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:02.632453 23988 consensus_queue.cc:1048] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:02.638578 23988 raft_consensus.cc:2955] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:02.640499 22954 raft_consensus.cc:2955] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:02.640383 23624 raft_consensus.cc:2955] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:02.651242 22762 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6eba8b8b84ca4a9aa5d9eaca94670686 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:08:02.661854 22776 catalog_manager.cc:5697] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: config changed from index 3 to 4, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:02.685402 22867 tablet_service.cc:1558] Processing DeleteTablet for tablet 6eba8b8b84ca4a9aa5d9eaca94670686 with delete_type TABLET_DATA_TOMBSTONED (TS 0a407504ed1741bf89a276558adf4c0f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:35344
I20260709 13:08:02.686388 24012 tablet_replica.cc:333] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:08:02.687340 24012 raft_consensus.cc:2243] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:02.687943 24012 raft_consensus.cc:2272] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:02.691015 24012 ts_tablet_manager.cc:1916] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 0a407504ed1741bf89a276558adf4c0f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:08:02.703356 24012 ts_tablet_manager.cc:1929] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 0a407504ed1741bf89a276558adf4c0f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:08:02.703724 24012 log.cc:1199] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 0a407504ed1741bf89a276558adf4c0f: Deleting WAL directory at /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-0-root/wals/6eba8b8b84ca4a9aa5d9eaca94670686
I20260709 13:08:02.705127 22764 catalog_manager.cc:5028] TS 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): tablet 6eba8b8b84ca4a9aa5d9eaca94670686 (table test-workload [id=a42f2ccfbe484baca3019323a490c64a]) successfully deleted
I20260709 13:08:03.759590 23023 consensus_queue.cc:237] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
W20260709 13:08:03.773888 22993 consensus_peers.cc:597] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 -> Peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Couldn't send request to peer 5b346767a89745fa82928a5dac9cc0b2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: fab4388fa7ea49bda9fa070cf542577f. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:03.773888 22953 raft_consensus.cc:1275] T fab4388fa7ea49bda9fa070cf542577f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:03.774180 22880 raft_consensus.cc:1275] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:03.775262 23988 consensus_queue.cc:1048] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:03.776093 24014 consensus_queue.cc:1048] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:03.785230 23987 raft_consensus.cc:2955] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:03.786808 22953 raft_consensus.cc:2955] T fab4388fa7ea49bda9fa070cf542577f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:03.787374 22880 raft_consensus.cc:2955] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:03.806416 22776 catalog_manager.cc:5697] T fab4388fa7ea49bda9fa070cf542577f P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New cstate: current_term: 2 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:03.808496 22953 consensus_queue.cc:237] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:03.816406 23023 raft_consensus.cc:1275] T b65d9dbf257d434d86d4656dabfd1b70 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:08:03.817315 22880 raft_consensus.cc:1275] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:08:03.817529 22919 consensus_peers.cc:597] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Couldn't send request to peer 5b346767a89745fa82928a5dac9cc0b2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b65d9dbf257d434d86d4656dabfd1b70. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:03.817778 24009 consensus_queue.cc:1048] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:03.818745 23957 consensus_queue.cc:1048] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:03.829308 24009 raft_consensus.cc:2955] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:03.830642 22872 raft_consensus.cc:2955] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:03.845845 23029 raft_consensus.cc:2955] T b65d9dbf257d434d86d4656dabfd1b70 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:03.846429 23020 consensus_queue.cc:237] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:03.874979 22955 raft_consensus.cc:1275] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:08:03.876515 22993 consensus_peers.cc:597] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 -> Peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Couldn't send request to peer 5b346767a89745fa82928a5dac9cc0b2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: abd5e7d435ca4afd9727e37b2ab31589. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:03.877671 22876 raft_consensus.cc:1275] T abd5e7d435ca4afd9727e37b2ab31589 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:08:03.878098 23987 consensus_queue.cc:1048] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:03.872809 22778 catalog_manager.cc:5697] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New cstate: current_term: 1 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:03.880610 23988 consensus_queue.cc:1048] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:03.887966 23987 raft_consensus.cc:2955] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:03.895493 22955 raft_consensus.cc:2955] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:03.901014 22872 raft_consensus.cc:2955] T abd5e7d435ca4afd9727e37b2ab31589 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:03.908844 22876 consensus_queue.cc:237] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:03.934820 22774 catalog_manager.cc:5697] T abd5e7d435ca4afd9727e37b2ab31589 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
W20260709 13:08:03.943554 22844 consensus_peers.cc:597] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f -> Peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Couldn't send request to peer 5b346767a89745fa82928a5dac9cc0b2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6bdc53dc56134e28bcc43e68e76055fc. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:03.948506 23029 raft_consensus.cc:1275] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:08:03.949844 22955 raft_consensus.cc:1275] T 6bdc53dc56134e28bcc43e68e76055fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:08:03.951073 23985 consensus_queue.cc:1048] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:03.952317 23956 consensus_queue.cc:1048] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:03.980217 24013 raft_consensus.cc:2955] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:03.984572 22955 raft_consensus.cc:2955] T 6bdc53dc56134e28bcc43e68e76055fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:03.989208 23029 raft_consensus.cc:2955] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:03.999109 22778 catalog_manager.cc:5697] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f reported cstate change: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:04.377068 24036 ts_tablet_manager.cc:933] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2: Initiating tablet copy from peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:04.377822 24035 ts_tablet_manager.cc:933] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2: Initiating tablet copy from peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:04.381645 24036 tablet_copy_client.cc:323] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.63.2:33309
I20260709 13:08:04.383920 24035 tablet_copy_client.cc:323] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.63.3:42499
I20260709 13:08:04.385370 22965 tablet_copy_service.cc:140] P aed3f089ec8b49af9c8cf5cdee76e8e7: Received BeginTabletCopySession request for tablet b65d9dbf257d434d86d4656dabfd1b70 from peer 5b346767a89745fa82928a5dac9cc0b2 ({username='slave'} at 127.0.0.1:52222)
I20260709 13:08:04.385722 23039 tablet_copy_service.cc:140] P f3c799b1249e40119d2a5864231c6c75: Received BeginTabletCopySession request for tablet abd5e7d435ca4afd9727e37b2ab31589 from peer 5b346767a89745fa82928a5dac9cc0b2 ({username='slave'} at 127.0.0.1:32794)
I20260709 13:08:04.385932 22965 tablet_copy_service.cc:161] P aed3f089ec8b49af9c8cf5cdee76e8e7: Beginning new tablet copy session on tablet b65d9dbf257d434d86d4656dabfd1b70 from peer 5b346767a89745fa82928a5dac9cc0b2 at {username='slave'} at 127.0.0.1:52222: session id = 5b346767a89745fa82928a5dac9cc0b2-b65d9dbf257d434d86d4656dabfd1b70
I20260709 13:08:04.386349 23039 tablet_copy_service.cc:161] P f3c799b1249e40119d2a5864231c6c75: Beginning new tablet copy session on tablet abd5e7d435ca4afd9727e37b2ab31589 from peer 5b346767a89745fa82928a5dac9cc0b2 at {username='slave'} at 127.0.0.1:32794: session id = 5b346767a89745fa82928a5dac9cc0b2-abd5e7d435ca4afd9727e37b2ab31589
I20260709 13:08:04.394805 23039 tablet_copy_source_session.cc:215] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:08:04.395529 22965 tablet_copy_source_session.cc:215] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:08:04.397082 24039 ts_tablet_manager.cc:933] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2: Initiating tablet copy from peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:04.399080 24036 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b65d9dbf257d434d86d4656dabfd1b70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.402983 24035 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet abd5e7d435ca4afd9727e37b2ab31589. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.407658 24039 tablet_copy_client.cc:323] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.63.3:42499
I20260709 13:08:04.409449 23039 tablet_copy_service.cc:140] P f3c799b1249e40119d2a5864231c6c75: Received BeginTabletCopySession request for tablet fab4388fa7ea49bda9fa070cf542577f from peer 5b346767a89745fa82928a5dac9cc0b2 ({username='slave'} at 127.0.0.1:32794)
I20260709 13:08:04.409961 23039 tablet_copy_service.cc:161] P f3c799b1249e40119d2a5864231c6c75: Beginning new tablet copy session on tablet fab4388fa7ea49bda9fa070cf542577f from peer 5b346767a89745fa82928a5dac9cc0b2 at {username='slave'} at 127.0.0.1:32794: session id = 5b346767a89745fa82928a5dac9cc0b2-fab4388fa7ea49bda9fa070cf542577f
I20260709 13:08:04.417316 24036 tablet_copy_client.cc:806] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 0 data blocks...
I20260709 13:08:04.417464 23039 tablet_copy_source_session.cc:215] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:08:04.417963 24036 tablet_copy_client.cc:670] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:08:04.421257 24039 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fab4388fa7ea49bda9fa070cf542577f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.421926 24036 tablet_copy_client.cc:538] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:08:04.426061 24035 tablet_copy_client.cc:806] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 0 data blocks...
I20260709 13:08:04.426697 24035 tablet_copy_client.cc:670] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:08:04.430735 24035 tablet_copy_client.cc:538] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:08:04.432330 24036 tablet_bootstrap.cc:492] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap starting.
I20260709 13:08:04.438020 24039 tablet_copy_client.cc:806] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 0 data blocks...
I20260709 13:08:04.441073 24039 tablet_copy_client.cc:670] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:08:04.440448 24035 tablet_bootstrap.cc:492] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap starting.
I20260709 13:08:04.445703 24039 tablet_copy_client.cc:538] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:08:04.457403 24039 tablet_bootstrap.cc:492] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap starting.
I20260709 13:08:04.477679 24041 ts_tablet_manager.cc:933] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2: Initiating tablet copy from peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929)
I20260709 13:08:04.488704 24041 tablet_copy_client.cc:323] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.63.1:33929
I20260709 13:08:04.490726 22890 tablet_copy_service.cc:140] P 0a407504ed1741bf89a276558adf4c0f: Received BeginTabletCopySession request for tablet 6bdc53dc56134e28bcc43e68e76055fc from peer 5b346767a89745fa82928a5dac9cc0b2 ({username='slave'} at 127.0.0.1:53724)
I20260709 13:08:04.491284 22890 tablet_copy_service.cc:161] P 0a407504ed1741bf89a276558adf4c0f: Beginning new tablet copy session on tablet 6bdc53dc56134e28bcc43e68e76055fc from peer 5b346767a89745fa82928a5dac9cc0b2 at {username='slave'} at 127.0.0.1:53724: session id = 5b346767a89745fa82928a5dac9cc0b2-6bdc53dc56134e28bcc43e68e76055fc
I20260709 13:08:04.500799 22890 tablet_copy_source_session.cc:215] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:08:04.505015 24041 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6bdc53dc56134e28bcc43e68e76055fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:04.513753 24039 tablet_bootstrap.cc:492] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:04.515017 24039 tablet_bootstrap.cc:492] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap complete.
I20260709 13:08:04.515827 24039 ts_tablet_manager.cc:1403] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2: Time spent bootstrapping tablet: real 0.059s	user 0.023s	sys 0.009s
I20260709 13:08:04.517215 24036 tablet_bootstrap.cc:492] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:04.518249 24036 tablet_bootstrap.cc:492] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap complete.
I20260709 13:08:04.519378 24036 ts_tablet_manager.cc:1403] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2: Time spent bootstrapping tablet: real 0.087s	user 0.024s	sys 0.013s
I20260709 13:08:04.519090 24039 raft_consensus.cc:359] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:04.521816 24039 raft_consensus.cc:740] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Initialized, Role: LEARNER
I20260709 13:08:04.522478 24039 consensus_queue.cc:260] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:04.522838 24036 raft_consensus.cc:359] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:04.523844 24036 raft_consensus.cc:740] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Initialized, Role: LEARNER
I20260709 13:08:04.524587 24036 consensus_queue.cc:260] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:04.534005 24036 ts_tablet_manager.cc:1434] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.004s
I20260709 13:08:04.536643 22965 tablet_copy_service.cc:342] P aed3f089ec8b49af9c8cf5cdee76e8e7: Request end of tablet copy session 5b346767a89745fa82928a5dac9cc0b2-b65d9dbf257d434d86d4656dabfd1b70 received from {username='slave'} at 127.0.0.1:52222
I20260709 13:08:04.537081 22965 tablet_copy_service.cc:434] P aed3f089ec8b49af9c8cf5cdee76e8e7: ending tablet copy session 5b346767a89745fa82928a5dac9cc0b2-b65d9dbf257d434d86d4656dabfd1b70 on tablet b65d9dbf257d434d86d4656dabfd1b70 with peer 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:04.541936 24035 tablet_bootstrap.cc:492] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:04.542871 24035 tablet_bootstrap.cc:492] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap complete.
I20260709 13:08:04.543411 24041 tablet_copy_client.cc:806] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 0 data blocks...
I20260709 13:08:04.544610 24041 tablet_copy_client.cc:670] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:08:04.544921 24039 ts_tablet_manager.cc:1434] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.002s
I20260709 13:08:04.546712 24035 ts_tablet_manager.cc:1403] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2: Time spent bootstrapping tablet: real 0.106s	user 0.023s	sys 0.004s
I20260709 13:08:04.550318 24035 raft_consensus.cc:359] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:04.551236 24035 raft_consensus.cc:740] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Initialized, Role: LEARNER
I20260709 13:08:04.551822 23039 tablet_copy_service.cc:342] P f3c799b1249e40119d2a5864231c6c75: Request end of tablet copy session 5b346767a89745fa82928a5dac9cc0b2-fab4388fa7ea49bda9fa070cf542577f received from {username='slave'} at 127.0.0.1:32794
I20260709 13:08:04.552353 23039 tablet_copy_service.cc:434] P f3c799b1249e40119d2a5864231c6c75: ending tablet copy session 5b346767a89745fa82928a5dac9cc0b2-fab4388fa7ea49bda9fa070cf542577f on tablet fab4388fa7ea49bda9fa070cf542577f with peer 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:04.553756 24035 consensus_queue.cc:260] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:04.555783 24041 tablet_copy_client.cc:538] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:08:04.556731 24035 ts_tablet_manager.cc:1434] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 13:08:04.558586 23039 tablet_copy_service.cc:342] P f3c799b1249e40119d2a5864231c6c75: Request end of tablet copy session 5b346767a89745fa82928a5dac9cc0b2-abd5e7d435ca4afd9727e37b2ab31589 received from {username='slave'} at 127.0.0.1:32794
I20260709 13:08:04.559145 23039 tablet_copy_service.cc:434] P f3c799b1249e40119d2a5864231c6c75: ending tablet copy session 5b346767a89745fa82928a5dac9cc0b2-abd5e7d435ca4afd9727e37b2ab31589 on tablet abd5e7d435ca4afd9727e37b2ab31589 with peer 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:04.574774 24041 tablet_bootstrap.cc:492] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap starting.
I20260709 13:08:04.599359 24041 tablet_bootstrap.cc:492] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:04.600230 24041 tablet_bootstrap.cc:492] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap complete.
I20260709 13:08:04.600926 24041 ts_tablet_manager.cc:1403] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2: Time spent bootstrapping tablet: real 0.026s	user 0.016s	sys 0.008s
I20260709 13:08:04.603802 24041 raft_consensus.cc:359] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:04.604600 24041 raft_consensus.cc:740] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Initialized, Role: LEARNER
I20260709 13:08:04.605288 24041 consensus_queue.cc:260] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:04.607807 24041 ts_tablet_manager.cc:1434] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:08:04.609684 22890 tablet_copy_service.cc:342] P 0a407504ed1741bf89a276558adf4c0f: Request end of tablet copy session 5b346767a89745fa82928a5dac9cc0b2-6bdc53dc56134e28bcc43e68e76055fc received from {username='slave'} at 127.0.0.1:53724
I20260709 13:08:04.610097 22890 tablet_copy_service.cc:434] P 0a407504ed1741bf89a276558adf4c0f: ending tablet copy session 5b346767a89745fa82928a5dac9cc0b2-6bdc53dc56134e28bcc43e68e76055fc on tablet 6bdc53dc56134e28bcc43e68e76055fc with peer 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:04.771761 23623 raft_consensus.cc:1217] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:08:04.878852 23623 raft_consensus.cc:1217] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:08:04.966686 23623 raft_consensus.cc:1217] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:08:05.002713 23623 raft_consensus.cc:1217] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2]   Dedup: 2.2->[]
I20260709 13:08:05.311731 24048 raft_consensus.cc:1064] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
I20260709 13:08:05.314257 24048 consensus_queue.cc:237] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:05.323748 23623 raft_consensus.cc:1275] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:08:05.325208 23987 consensus_queue.cc:1048] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:05.326422 22953 raft_consensus.cc:1275] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:08:05.326604 22872 raft_consensus.cc:1275] T abd5e7d435ca4afd9727e37b2ab31589 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:08:05.328027 23987 consensus_queue.cc:1048] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:05.329895 24029 consensus_queue.cc:1048] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 13:08:05.352802 22953 raft_consensus.cc:2955] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:05.372007 22774 catalog_manager.cc:5697] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:05.383320 24013 raft_consensus.cc:1064] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
I20260709 13:08:05.349520 24029 raft_consensus.cc:2955] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:05.409654 24013 consensus_queue.cc:237] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:05.423882 22872 raft_consensus.cc:2955] T abd5e7d435ca4afd9727e37b2ab31589 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:05.425050 23624 raft_consensus.cc:2955] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:05.427227 23623 raft_consensus.cc:1275] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:08:05.428745 24013 consensus_queue.cc:1048] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:05.443621 23029 raft_consensus.cc:1275] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:08:05.447677 22953 raft_consensus.cc:1275] T 6bdc53dc56134e28bcc43e68e76055fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:08:05.451251 24013 consensus_queue.cc:1048] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:05.457836 24057 consensus_queue.cc:1048] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:05.468323 23985 raft_consensus.cc:2955] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:05.470579 23029 raft_consensus.cc:2955] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:05.484174 24031 raft_consensus.cc:1064] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
I20260709 13:08:05.489446 23624 raft_consensus.cc:2955] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:05.503511 24031 consensus_queue.cc:237] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:05.505239 22953 raft_consensus.cc:2955] T 6bdc53dc56134e28bcc43e68e76055fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:05.520267 23624 raft_consensus.cc:1275] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:08:05.521111 23029 raft_consensus.cc:1275] T b65d9dbf257d434d86d4656dabfd1b70 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:08:05.521359 22880 raft_consensus.cc:1275] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:08:05.521760 24050 consensus_queue.cc:1048] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:05.523008 24031 consensus_queue.cc:1048] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:05.529663 24030 consensus_queue.cc:1048] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:05.539573 23020 consensus_queue.cc:237] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:05.540627 22777 catalog_manager.cc:5697] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:05.550992 23624 raft_consensus.cc:1275] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:08:05.554005 22953 raft_consensus.cc:1275] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:08:05.557896 24055 consensus_queue.cc:1048] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:05.560343 24055 consensus_queue.cc:1048] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:05.564750 24029 raft_consensus.cc:2955] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:05.566596 24030 raft_consensus.cc:2955] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:05.574760 22873 consensus_queue.cc:237] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:05.576361 22953 raft_consensus.cc:2955] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:05.579255 22762 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet abd5e7d435ca4afd9727e37b2ab31589 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:08:05.594696 23625 raft_consensus.cc:2955] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:05.596061 23623 raft_consensus.cc:2955] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:05.605304 23020 raft_consensus.cc:1275] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:08:05.606737 23624 raft_consensus.cc:1275] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:08:05.607052 24054 raft_consensus.cc:1064] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
I20260709 13:08:05.608724 23956 consensus_queue.cc:1048] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:05.610339 24054 consensus_queue.cc:237] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:05.603665 22880 raft_consensus.cc:2955] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:05.618083 23956 consensus_queue.cc:1048] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:05.612056 22774 catalog_manager.cc:5697] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: config changed from index 3 to 4, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:05.621549 23985 raft_consensus.cc:2955] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:05.624940 23029 raft_consensus.cc:2955] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:05.628135 23622 raft_consensus.cc:2955] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
W20260709 13:08:05.638927 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 51566.6us
I20260709 13:08:05.646596 23020 raft_consensus.cc:2955] T b65d9dbf257d434d86d4656dabfd1b70 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:05.655481 22872 raft_consensus.cc:1275] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:08:05.658595 22955 raft_consensus.cc:1275] T fab4388fa7ea49bda9fa070cf542577f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:08:05.659780 24029 consensus_queue.cc:1048] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:05.665531 24048 consensus_queue.cc:1048] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:05.667881 23623 raft_consensus.cc:1275] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEARNER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:08:05.674669 24055 consensus_queue.cc:1048] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:05.693886 22764 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6bdc53dc56134e28bcc43e68e76055fc with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:08:05.695096 22777 catalog_manager.cc:5697] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:05.710589 24054 raft_consensus.cc:2955] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:05.713213 22880 raft_consensus.cc:2955] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:05.722540 22867 tablet_service.cc:1558] Processing DeleteTablet for tablet abd5e7d435ca4afd9727e37b2ab31589 with delete_type TABLET_DATA_TOMBSTONED (TS 0a407504ed1741bf89a276558adf4c0f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:35344
I20260709 13:08:05.738693 23623 raft_consensus.cc:2955] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:05.749580 22953 raft_consensus.cc:2955] T fab4388fa7ea49bda9fa070cf542577f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:05.754979 22776 catalog_manager.cc:5697] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: config changed from index 3 to 4, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:05.771826 24080 tablet_replica.cc:333] T abd5e7d435ca4afd9727e37b2ab31589 P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:08:05.773089 22778 catalog_manager.cc:5697] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 reported cstate change: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:05.774194 22943 tablet_service.cc:1558] Processing DeleteTablet for tablet 6bdc53dc56134e28bcc43e68e76055fc with delete_type TABLET_DATA_TOMBSTONED (TS aed3f089ec8b49af9c8cf5cdee76e8e7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58302
I20260709 13:08:05.776402 24081 tablet_replica.cc:333] T 6bdc53dc56134e28bcc43e68e76055fc P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:08:05.780434 24081 raft_consensus.cc:2243] T 6bdc53dc56134e28bcc43e68e76055fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:05.782296 24081 raft_consensus.cc:2272] T 6bdc53dc56134e28bcc43e68e76055fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:05.825766 24080 raft_consensus.cc:2243] T abd5e7d435ca4afd9727e37b2ab31589 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:05.835407 22953 consensus_queue.cc:237] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:05.848111 24080 raft_consensus.cc:2272] T abd5e7d435ca4afd9727e37b2ab31589 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:05.851820 23623 raft_consensus.cc:1275] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:08:05.854023 24075 consensus_queue.cc:1048] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:08:05.859171 24080 ts_tablet_manager.cc:1916] T abd5e7d435ca4afd9727e37b2ab31589 P 0a407504ed1741bf89a276558adf4c0f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:08:05.863463 24050 raft_consensus.cc:2955] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:05.865271 23622 raft_consensus.cc:2955] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:05.873229 23023 consensus_queue.cc:237] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:05.879283 23623 raft_consensus.cc:1275] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:08:05.880836 24029 consensus_queue.cc:1048] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:05.891757 22880 raft_consensus.cc:1275] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:08:05.893172 24077 raft_consensus.cc:2955] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:05.902072 24066 consensus_queue.cc:1048] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:08:05.909422 22762 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet fab4388fa7ea49bda9fa070cf542577f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:08:05.914489 24081 ts_tablet_manager.cc:1916] T 6bdc53dc56134e28bcc43e68e76055fc P aed3f089ec8b49af9c8cf5cdee76e8e7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:08:05.916235 23622 raft_consensus.cc:2955] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:05.920941 22762 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b65d9dbf257d434d86d4656dabfd1b70 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:08:05.933699 22776 catalog_manager.cc:5697] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: config changed from index 3 to 4, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New cstate: current_term: 1 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:05.934012 22873 raft_consensus.cc:1275] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:08:05.940832 23987 consensus_queue.cc:1048] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:05.935715 22778 catalog_manager.cc:5697] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 reported cstate change: config changed from index 3 to 4, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New cstate: current_term: 2 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:05.939877 22872 raft_consensus.cc:2955] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:05.955143 22873 raft_consensus.cc:2955] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:05.949352 24080 ts_tablet_manager.cc:1929] T abd5e7d435ca4afd9727e37b2ab31589 P 0a407504ed1741bf89a276558adf4c0f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:08:05.959255 24080 log.cc:1199] T abd5e7d435ca4afd9727e37b2ab31589 P 0a407504ed1741bf89a276558adf4c0f: Deleting WAL directory at /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-0-root/wals/abd5e7d435ca4afd9727e37b2ab31589
I20260709 13:08:05.961043 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:05.967621 22764 catalog_manager.cc:5028] TS 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): tablet abd5e7d435ca4afd9727e37b2ab31589 (table test-workload [id=a42f2ccfbe484baca3019323a490c64a]) successfully deleted
I20260709 13:08:05.968807 23019 tablet_service.cc:1558] Processing DeleteTablet for tablet b65d9dbf257d434d86d4656dabfd1b70 with delete_type TABLET_DATA_TOMBSTONED (TS f3c799b1249e40119d2a5864231c6c75 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33406
I20260709 13:08:05.984912 22943 tablet_service.cc:1558] Processing DeleteTablet for tablet fab4388fa7ea49bda9fa070cf542577f with delete_type TABLET_DATA_TOMBSTONED (TS aed3f089ec8b49af9c8cf5cdee76e8e7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58302
I20260709 13:08:05.992220 24082 tablet_replica.cc:333] T b65d9dbf257d434d86d4656dabfd1b70 P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:08:05.993351 24082 raft_consensus.cc:2243] T b65d9dbf257d434d86d4656dabfd1b70 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:05.994068 24082 raft_consensus.cc:2272] T b65d9dbf257d434d86d4656dabfd1b70 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:05.998466 24082 ts_tablet_manager.cc:1916] T b65d9dbf257d434d86d4656dabfd1b70 P f3c799b1249e40119d2a5864231c6c75: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:08:06.003204 24081 ts_tablet_manager.cc:1929] T 6bdc53dc56134e28bcc43e68e76055fc P aed3f089ec8b49af9c8cf5cdee76e8e7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:08:06.003839 24081 log.cc:1199] T 6bdc53dc56134e28bcc43e68e76055fc P aed3f089ec8b49af9c8cf5cdee76e8e7: Deleting WAL directory at /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-1-root/wals/6bdc53dc56134e28bcc43e68e76055fc
I20260709 13:08:06.006122 22762 catalog_manager.cc:5028] TS aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): tablet 6bdc53dc56134e28bcc43e68e76055fc (table test-workload [id=a42f2ccfbe484baca3019323a490c64a]) successfully deleted
I20260709 13:08:06.006323 24081 tablet_replica.cc:333] T fab4388fa7ea49bda9fa070cf542577f P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:08:06.008162 24081 raft_consensus.cc:2243] T fab4388fa7ea49bda9fa070cf542577f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:06.008870 24081 raft_consensus.cc:2272] T fab4388fa7ea49bda9fa070cf542577f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:06.013705 24081 ts_tablet_manager.cc:1916] T fab4388fa7ea49bda9fa070cf542577f P aed3f089ec8b49af9c8cf5cdee76e8e7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:08:06.031667 24082 ts_tablet_manager.cc:1929] T b65d9dbf257d434d86d4656dabfd1b70 P f3c799b1249e40119d2a5864231c6c75: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:08:06.031972 24081 ts_tablet_manager.cc:1929] T fab4388fa7ea49bda9fa070cf542577f P aed3f089ec8b49af9c8cf5cdee76e8e7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20260709 13:08:06.032239 24082 log.cc:1199] T b65d9dbf257d434d86d4656dabfd1b70 P f3c799b1249e40119d2a5864231c6c75: Deleting WAL directory at /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-2-root/wals/b65d9dbf257d434d86d4656dabfd1b70
I20260709 13:08:06.032426 24081 log.cc:1199] T fab4388fa7ea49bda9fa070cf542577f P aed3f089ec8b49af9c8cf5cdee76e8e7: Deleting WAL directory at /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-1-root/wals/fab4388fa7ea49bda9fa070cf542577f
I20260709 13:08:06.034538 22762 catalog_manager.cc:5028] TS f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): tablet b65d9dbf257d434d86d4656dabfd1b70 (table test-workload [id=a42f2ccfbe484baca3019323a490c64a]) successfully deleted
I20260709 13:08:06.035210 22762 catalog_manager.cc:5028] TS aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): tablet fab4388fa7ea49bda9fa070cf542577f (table test-workload [id=a42f2ccfbe484baca3019323a490c64a]) successfully deleted
I20260709 13:08:06.057348 22873 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
mode: GRACEFUL
new_leader_uuid: "5b346767a89745fa82928a5dac9cc0b2"
 from {username='slave'} at 127.0.0.1:51434
I20260709 13:08:06.058063 22873 raft_consensus.cc:606] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Received request to transfer leadership to TS 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:06.060571 22873 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
mode: GRACEFUL
new_leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:51434
I20260709 13:08:06.061090 22873 raft_consensus.cc:606] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Received request to transfer leadership to TS aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:06.073526 22947 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:36566
I20260709 13:08:06.074079 22947 raft_consensus.cc:606] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:08:06.076891 22873 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
mode: GRACEFUL
new_leader_uuid: "5b346767a89745fa82928a5dac9cc0b2"
 from {username='slave'} at 127.0.0.1:51434
I20260709 13:08:06.077486 22873 raft_consensus.cc:606] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Received request to transfer leadership to TS 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:06.079972 22873 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
mode: GRACEFUL
new_leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:51434
I20260709 13:08:06.080507 22873 raft_consensus.cc:606] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Received request to transfer leadership to TS aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:06.092867 23023 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:08:06.093361 23023 raft_consensus.cc:606] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:08:06.094410 21756 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:08:06.095377 21756 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:06.332336 24061 raft_consensus.cc:993] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f: : Instructing follower aed3f089ec8b49af9c8cf5cdee76e8e7 to start an election
I20260709 13:08:06.332825 23985 raft_consensus.cc:1081] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Signalling peer aed3f089ec8b49af9c8cf5cdee76e8e7 to start an election
I20260709 13:08:06.334126 22947 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:58310
I20260709 13:08:06.334695 22947 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:06.335050 22947 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:06.340430 22947 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:06.342252 22947 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:06.342933 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:06.343303 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:06.343753 22873 raft_consensus.cc:3055] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:06.344067 22873 raft_consensus.cc:740] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:06.344097 23026 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:06.344753 22873 consensus_queue.cc:260] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:06.345852 22873 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:06.349181 23026 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:06.349797 22873 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:06.350292 22918 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:06.351230 24075 raft_consensus.cc:2804] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:06.351773 24075 raft_consensus.cc:697] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:06.352610 24075 consensus_queue.cc:237] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:06.362761 22776 catalog_manager.cc:5697] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 1 to 2, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 2 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:06.423105 24030 raft_consensus.cc:993] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7: : Instructing follower 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:08:06.423601 24075 raft_consensus.cc:1081] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Signalling peer 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:08:06.425100 22873 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:35360
I20260709 13:08:06.425624 22873 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:06.425901 22873 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:06.430346 22873 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:06.431840 22873 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:06.432536 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:06.433163 22947 raft_consensus.cc:3055] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:06.433486 22947 raft_consensus.cc:740] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:06.433719 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:06.434338 23026 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:06.434423 22947 consensus_queue.cc:260] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:06.435451 23987 raft_consensus.cc:993] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75: : Instructing follower 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:08:06.435724 22947 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:06.435899 24029 raft_consensus.cc:1081] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Signalling peer 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:08:06.437531 22873 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:35370
I20260709 13:08:06.437978 22873 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:06.438283 22873 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:06.440517 23026 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:08:06.441957 22947 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:08:06.444942 22843 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:06.445137 22873 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:06.446172 23985 raft_consensus.cc:2804] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:06.447712 22873 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:06.449719 23985 raft_consensus.cc:697] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:06.451377 23985 consensus_queue.cc:237] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:06.450397 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:06.453145 22947 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:06.453809 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:06.454321 23026 raft_consensus.cc:3055] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:06.454576 23026 raft_consensus.cc:740] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:06.455258 23026 consensus_queue.cc:260] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:06.456341 23026 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:06.459503 22947 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:08:06.460780 22843 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:06.461508 23026 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:08:06.463510 23985 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:06.464025 23985 raft_consensus.cc:697] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:06.464854 23985 consensus_queue.cc:237] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:06.470520 22776 catalog_manager.cc:5697] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 1 to 2, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 2 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:06.476339 24061 raft_consensus.cc:993] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f: : Instructing follower 5b346767a89745fa82928a5dac9cc0b2 to start an election
I20260709 13:08:06.476934 23985 raft_consensus.cc:1081] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Signalling peer 5b346767a89745fa82928a5dac9cc0b2 to start an election
I20260709 13:08:06.478384 23622 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a"
dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
 from {username='slave'} at 127.0.0.1:60026
I20260709 13:08:06.478935 23622 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:06.479265 23622 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:06.484722 23622 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:06.488967 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:06.489813 22873 raft_consensus.cc:3055] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:06.490113 22873 raft_consensus.cc:740] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:06.490881 22873 consensus_queue.cc:260] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:06.492013 22873 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:06.492482 24061 raft_consensus.cc:993] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f: : Instructing follower aed3f089ec8b49af9c8cf5cdee76e8e7 to start an election
I20260709 13:08:06.493017 24061 raft_consensus.cc:1081] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Signalling peer aed3f089ec8b49af9c8cf5cdee76e8e7 to start an election
I20260709 13:08:06.494158 22776 catalog_manager.cc:5697] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 1 to 2, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 2 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:06.495462 22947 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:58310
I20260709 13:08:06.495961 22947 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:06.496274 22947 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:06.497663 23622 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:06.499596 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:06.499890 22873 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 2.
I20260709 13:08:06.500236 23026 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:06.501410 23590 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:06.502719 22947 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:06.505592 22947 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:06.506974 23026 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 2.
I20260709 13:08:06.508214 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:06.508776 23026 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:06.509423 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:06.510663 22873 raft_consensus.cc:3055] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:06.511432 22873 raft_consensus.cc:740] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:06.513088 24090 raft_consensus.cc:2804] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:06.512790 22873 consensus_queue.cc:260] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:06.515434 23026 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:06.516641 22873 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:06.517180 22918 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:06.517969 24075 raft_consensus.cc:2804] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:06.518399 24075 raft_consensus.cc:697] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:06.519194 24075 consensus_queue.cc:237] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:06.520452 24090 raft_consensus.cc:697] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:06.521426 24090 consensus_queue.cc:237] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:06.530069 22873 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:06.532373 22776 catalog_manager.cc:5697] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 1 to 2, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 2 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:06.537220 22777 catalog_manager.cc:5697] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 1 to 2, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 2 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:06.590317 24061 raft_consensus.cc:993] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f: : Instructing follower 5b346767a89745fa82928a5dac9cc0b2 to start an election
I20260709 13:08:06.590802 23985 raft_consensus.cc:1081] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Signalling peer 5b346767a89745fa82928a5dac9cc0b2 to start an election
I20260709 13:08:06.592152 23622 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc"
dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
 from {username='slave'} at 127.0.0.1:60026
I20260709 13:08:06.592818 23622 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:06.593393 23622 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:06.600355 23622 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:06.602599 23622 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:06.603506 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:06.603873 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:06.604082 22947 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:06.604382 22873 raft_consensus.cc:3055] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:06.604734 22873 raft_consensus.cc:740] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:06.605454 22873 consensus_queue.cc:260] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:06.606571 22873 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:06.610787 22947 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 2.
I20260709 13:08:06.612085 23588 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:06.612865 24090 raft_consensus.cc:2804] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:06.612964 22873 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 2.
I20260709 13:08:06.613400 24090 raft_consensus.cc:697] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:06.614440 24090 consensus_queue.cc:237] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:06.623997 22776 catalog_manager.cc:5697] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 1 to 2, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 2 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:06.893263 23020 raft_consensus.cc:1275] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:06.894835 24030 consensus_queue.cc:1048] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:06.912916 22873 raft_consensus.cc:1275] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:06.918319 24097 consensus_queue.cc:1048] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:06.935405 23020 raft_consensus.cc:1275] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:08:06.936816 24090 consensus_queue.cc:1048] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:06.949782 22873 raft_consensus.cc:1275] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:08:06.951915 24091 consensus_queue.cc:1048] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:06.971356 23020 raft_consensus.cc:1275] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:06.972595 24097 consensus_queue.cc:1048] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:06.987012 23023 raft_consensus.cc:1275] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:06.990103 24061 consensus_queue.cc:1048] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:06.990936 23020 raft_consensus.cc:1275] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:06.992384 24061 consensus_queue.cc:1048] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:06.996966 22873 raft_consensus.cc:1275] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:06.998217 24066 consensus_queue.cc:1048] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:07.000463 22953 raft_consensus.cc:1275] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:07.004844 23985 consensus_queue.cc:1048] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:07.060316 22947 raft_consensus.cc:1275] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:07.063459 24111 consensus_queue.cc:1048] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:07.096175 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:07.107138 22947 consensus_queue.cc:237] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
W20260709 13:08:07.113690 22919 consensus_peers.cc:597] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Couldn't send request to peer 5b346767a89745fa82928a5dac9cc0b2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c26347a647b240d7a3bc5cd449b7993b. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:07.114076 22872 raft_consensus.cc:1275] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:08:07.115262 24066 consensus_queue.cc:1048] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:07.116489 23023 raft_consensus.cc:1275] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:08:07.118029 24030 consensus_queue.cc:1048] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:07.124770 24030 raft_consensus.cc:2955] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:07.128353 22947 raft_consensus.cc:1275] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:08:07.126468 22872 raft_consensus.cc:2955] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:07.129884 24091 consensus_queue.cc:1048] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:07.133486 23023 raft_consensus.cc:2955] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:07.161114 22873 raft_consensus.cc:1275] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:08:07.163048 22774 catalog_manager.cc:5697] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 reported cstate change: config changed from index -1 to 3, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New cstate: current_term: 2 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:07.163141 24103 consensus_queue.cc:1048] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:08:07.166232 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:07.171048 22872 consensus_queue.cc:237] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
W20260709 13:08:07.183660 22844 consensus_peers.cc:597] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f -> Peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Couldn't send request to peer 5b346767a89745fa82928a5dac9cc0b2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4c0bea78fb2a40ef90cff914c5c6e250. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:07.187057 23026 raft_consensus.cc:1275] T 4c0bea78fb2a40ef90cff914c5c6e250 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:07.188724 24112 consensus_queue.cc:1048] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:07.191560 22953 raft_consensus.cc:1275] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:07.193030 23985 consensus_queue.cc:1048] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:07.211834 23985 raft_consensus.cc:2955] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:07.214772 22955 raft_consensus.cc:2955] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:07.228343 22872 consensus_queue.cc:237] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:07.236428 22777 catalog_manager.cc:5697] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New cstate: current_term: 2 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:07.240002 23026 raft_consensus.cc:2955] T 4c0bea78fb2a40ef90cff914c5c6e250 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Committing config change with OpId 2.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
W20260709 13:08:07.249568 22844 consensus_peers.cc:597] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f -> Peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Couldn't send request to peer 5b346767a89745fa82928a5dac9cc0b2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: aa6540758938493d98d7852fdbe3ffab. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:07.249620 23020 raft_consensus.cc:1275] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:08:07.250294 22955 raft_consensus.cc:1275] T aa6540758938493d98d7852fdbe3ffab P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:08:07.255474 24113 consensus_queue.cc:1048] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:07.256471 24112 consensus_queue.cc:1048] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:07.258807 22955 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:36566
I20260709 13:08:07.259378 22955 raft_consensus.cc:606] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:08:07.268499 22954 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
mode: GRACEFUL
new_leader_uuid: "5b346767a89745fa82928a5dac9cc0b2"
 from {username='slave'} at 127.0.0.1:36566
I20260709 13:08:07.269071 22954 raft_consensus.cc:606] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Received request to transfer leadership to TS 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:07.273422 23020 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "5b346767a89745fa82928a5dac9cc0b2"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:08:07.274061 23020 raft_consensus.cc:606] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Received request to transfer leadership to TS 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:07.278721 22872 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
mode: GRACEFUL
new_leader_uuid: "5b346767a89745fa82928a5dac9cc0b2"
 from {username='slave'} at 127.0.0.1:51434
I20260709 13:08:07.279421 22872 raft_consensus.cc:606] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Received request to transfer leadership to TS 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:07.283849 22876 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
mode: GRACEFUL
new_leader_uuid: "5b346767a89745fa82928a5dac9cc0b2"
 from {username='slave'} at 127.0.0.1:51434
I20260709 13:08:07.284420 22876 raft_consensus.cc:606] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Received request to transfer leadership to TS 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:07.284829 23985 raft_consensus.cc:2955] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:07.285959 21756 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:08:07.287298 21756 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:07.289863 22955 raft_consensus.cc:2955] T aa6540758938493d98d7852fdbe3ffab P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:07.297258 23026 raft_consensus.cc:2955] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:07.299753 23020 consensus_queue.cc:237] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:07.303395 22774 catalog_manager.cc:5697] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f reported cstate change: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:07.314080 22993 consensus_peers.cc:597] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 -> Peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Couldn't send request to peer 5b346767a89745fa82928a5dac9cc0b2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2959101199a44a4583a4ee456ecf044d. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:07.315333 22955 raft_consensus.cc:1275] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:08:07.315572 22876 raft_consensus.cc:1275] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:08:07.317106 23987 consensus_queue.cc:1048] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:07.327433 24029 consensus_queue.cc:1048] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:07.349157 24029 raft_consensus.cc:2955] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:07.350662 22953 raft_consensus.cc:2955] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:07.352104 22876 raft_consensus.cc:2955] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:07.371340 22777 catalog_manager.cc:5697] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:07.390755 24066 raft_consensus.cc:993] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7: : Instructing follower 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:08:07.391417 24097 raft_consensus.cc:1081] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Signalling peer 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:08:07.392958 22876 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:35360
I20260709 13:08:07.393786 22876 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:07.394158 22876 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:07.401556 22876 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:07.403688 22876 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:07.404484 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:07.404810 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:07.405083 22953 raft_consensus.cc:3055] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:07.405400 23020 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:07.405403 22953 raft_consensus.cc:740] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:07.406353 22953 consensus_queue.cc:260] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:07.407325 22953 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:07.412328 23020 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:08:07.413486 22843 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:07.413690 22953 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:08:07.414274 24113 raft_consensus.cc:2804] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:07.414868 24113 raft_consensus.cc:697] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:07.415907 24113 consensus_queue.cc:237] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:07.438416 22776 catalog_manager.cc:5697] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 1 to 2, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 2 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:07.509615 24097 raft_consensus.cc:993] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7: : Instructing follower 5b346767a89745fa82928a5dac9cc0b2 to start an election
I20260709 13:08:07.510238 24066 raft_consensus.cc:1081] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Signalling peer 5b346767a89745fa82928a5dac9cc0b2 to start an election
I20260709 13:08:07.511724 23623 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc"
dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:08:07.512231 23623 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:07.512538 23623 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:07.519387 23623 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:07.520918 23623 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:07.521893 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:07.522506 23020 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:07.524495 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:07.525105 22953 raft_consensus.cc:3055] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:07.525445 22953 raft_consensus.cc:740] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:07.526193 22953 consensus_queue.cc:260] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:07.527376 22953 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:07.533046 23020 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 2.
I20260709 13:08:07.533738 22953 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 2.
I20260709 13:08:07.534240 23588 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:07.534996 24091 raft_consensus.cc:2804] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:07.535446 24091 raft_consensus.cc:697] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:07.536262 24091 consensus_queue.cc:237] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:07.547176 22776 catalog_manager.cc:5697] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 1 to 2, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 2 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:07.566421 24131 ts_tablet_manager.cc:933] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2: Initiating tablet copy from peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:07.571015 24131 tablet_copy_client.cc:323] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.63.2:33309
I20260709 13:08:07.572723 22965 tablet_copy_service.cc:140] P aed3f089ec8b49af9c8cf5cdee76e8e7: Received BeginTabletCopySession request for tablet c26347a647b240d7a3bc5cd449b7993b from peer 5b346767a89745fa82928a5dac9cc0b2 ({username='slave'} at 127.0.0.1:52222)
I20260709 13:08:07.573241 22965 tablet_copy_service.cc:161] P aed3f089ec8b49af9c8cf5cdee76e8e7: Beginning new tablet copy session on tablet c26347a647b240d7a3bc5cd449b7993b from peer 5b346767a89745fa82928a5dac9cc0b2 at {username='slave'} at 127.0.0.1:52222: session id = 5b346767a89745fa82928a5dac9cc0b2-c26347a647b240d7a3bc5cd449b7993b
I20260709 13:08:07.580912 22965 tablet_copy_source_session.cc:215] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:08:07.584472 24131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c26347a647b240d7a3bc5cd449b7993b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:07.601930 24131 tablet_copy_client.cc:806] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 0 data blocks...
I20260709 13:08:07.602577 24131 tablet_copy_client.cc:670] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:08:07.607620 24131 tablet_copy_client.cc:538] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:08:07.611795 24111 raft_consensus.cc:993] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f: : Instructing follower 5b346767a89745fa82928a5dac9cc0b2 to start an election
I20260709 13:08:07.612571 24112 raft_consensus.cc:1081] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Signalling peer 5b346767a89745fa82928a5dac9cc0b2 to start an election
I20260709 13:08:07.614141 23623 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe"
dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
 from {username='slave'} at 127.0.0.1:60026
I20260709 13:08:07.614697 23623 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:07.615072 23623 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:07.618310 24131 tablet_bootstrap.cc:492] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap starting.
I20260709 13:08:07.622641 23623 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:07.625412 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:07.625919 23623 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:07.625964 22876 raft_consensus.cc:3055] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:07.627565 22876 raft_consensus.cc:740] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:07.628296 22876 consensus_queue.cc:260] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:07.629570 22876 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:07.634943 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:07.635598 23020 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:07.649798 23020 raft_consensus.cc:2468] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 2.
I20260709 13:08:07.651371 23588 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:07.652603 24131 tablet_bootstrap.cc:492] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:07.651073 22876 raft_consensus.cc:2468] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 2.
I20260709 13:08:07.652807 24090 raft_consensus.cc:2804] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:07.653383 24090 raft_consensus.cc:697] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:07.653990 24131 tablet_bootstrap.cc:492] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap complete.
I20260709 13:08:07.654258 24090 consensus_queue.cc:237] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:07.654902 24131 ts_tablet_manager.cc:1403] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2: Time spent bootstrapping tablet: real 0.037s	user 0.024s	sys 0.009s
I20260709 13:08:07.658722 24131 raft_consensus.cc:359] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:07.659781 24131 raft_consensus.cc:740] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Initialized, Role: LEARNER
I20260709 13:08:07.660650 24131 consensus_queue.cc:260] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:07.664199 24131 ts_tablet_manager.cc:1434] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:08:07.667555 22965 tablet_copy_service.cc:342] P aed3f089ec8b49af9c8cf5cdee76e8e7: Request end of tablet copy session 5b346767a89745fa82928a5dac9cc0b2-c26347a647b240d7a3bc5cd449b7993b received from {username='slave'} at 127.0.0.1:52222
I20260709 13:08:07.668172 22965 tablet_copy_service.cc:434] P aed3f089ec8b49af9c8cf5cdee76e8e7: ending tablet copy session 5b346767a89745fa82928a5dac9cc0b2-c26347a647b240d7a3bc5cd449b7993b on tablet c26347a647b240d7a3bc5cd449b7993b with peer 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:07.670230 22776 catalog_manager.cc:5697] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 1 to 2, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 2 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:07.696965 24131 ts_tablet_manager.cc:933] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2: Initiating tablet copy from peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929)
I20260709 13:08:07.698661 24131 tablet_copy_client.cc:323] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.63.1:33929
I20260709 13:08:07.700373 22890 tablet_copy_service.cc:140] P 0a407504ed1741bf89a276558adf4c0f: Received BeginTabletCopySession request for tablet aa6540758938493d98d7852fdbe3ffab from peer 5b346767a89745fa82928a5dac9cc0b2 ({username='slave'} at 127.0.0.1:53724)
I20260709 13:08:07.700908 22890 tablet_copy_service.cc:161] P 0a407504ed1741bf89a276558adf4c0f: Beginning new tablet copy session on tablet aa6540758938493d98d7852fdbe3ffab from peer 5b346767a89745fa82928a5dac9cc0b2 at {username='slave'} at 127.0.0.1:53724: session id = 5b346767a89745fa82928a5dac9cc0b2-aa6540758938493d98d7852fdbe3ffab
I20260709 13:08:07.706805 22890 tablet_copy_source_session.cc:215] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:08:07.709379 24131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aa6540758938493d98d7852fdbe3ffab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:07.722767 24112 raft_consensus.cc:993] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f: : Instructing follower 5b346767a89745fa82928a5dac9cc0b2 to start an election
I20260709 13:08:07.723287 24111 raft_consensus.cc:1081] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Signalling peer 5b346767a89745fa82928a5dac9cc0b2 to start an election
I20260709 13:08:07.724835 23622 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc"
dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
 from {username='slave'} at 127.0.0.1:60026
I20260709 13:08:07.725378 23622 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:07.725730 23622 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:07.726029 24131 tablet_copy_client.cc:806] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 0 data blocks...
I20260709 13:08:07.726575 24131 tablet_copy_client.cc:670] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:08:07.734395 24131 tablet_copy_client.cc:538] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:08:07.742731 23622 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:07.744058 24131 tablet_bootstrap.cc:492] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap starting.
I20260709 13:08:07.745703 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:07.746263 22876 raft_consensus.cc:3055] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:07.746541 22876 raft_consensus.cc:740] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:07.747308 22876 consensus_queue.cc:260] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:07.750607 22876 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:07.751107 24136 ts_tablet_manager.cc:933] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2: Initiating tablet copy from peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:07.759435 24136 tablet_copy_client.cc:323] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.63.3:42499
I20260709 13:08:07.765722 23622 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:07.765846 24138 ts_tablet_manager.cc:933] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2: Initiating tablet copy from peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929)
I20260709 13:08:07.767264 23039 tablet_copy_service.cc:140] P f3c799b1249e40119d2a5864231c6c75: Received BeginTabletCopySession request for tablet 2959101199a44a4583a4ee456ecf044d from peer 5b346767a89745fa82928a5dac9cc0b2 ({username='slave'} at 127.0.0.1:32794)
I20260709 13:08:07.767750 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:07.767903 23039 tablet_copy_service.cc:161] P f3c799b1249e40119d2a5864231c6c75: Beginning new tablet copy session on tablet 2959101199a44a4583a4ee456ecf044d from peer 5b346767a89745fa82928a5dac9cc0b2 at {username='slave'} at 127.0.0.1:32794: session id = 5b346767a89745fa82928a5dac9cc0b2-2959101199a44a4583a4ee456ecf044d
I20260709 13:08:07.768275 23026 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:07.764739 22876 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 2.
I20260709 13:08:07.774819 23590 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:07.775187 23039 tablet_copy_source_session.cc:215] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:08:07.776273 24077 raft_consensus.cc:993] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75: : Instructing follower 5b346767a89745fa82928a5dac9cc0b2 to start an election
I20260709 13:08:07.776787 24077 raft_consensus.cc:1081] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Signalling peer 5b346767a89745fa82928a5dac9cc0b2 to start an election
I20260709 13:08:07.777300 24138 tablet_copy_client.cc:323] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.63.1:33929
I20260709 13:08:07.778297 24090 raft_consensus.cc:2804] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:07.778791 24090 raft_consensus.cc:697] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:07.779938 24136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2959101199a44a4583a4ee456ecf044d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:07.779620 24090 consensus_queue.cc:237] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:07.783438 22890 tablet_copy_service.cc:140] P 0a407504ed1741bf89a276558adf4c0f: Received BeginTabletCopySession request for tablet 4c0bea78fb2a40ef90cff914c5c6e250 from peer 5b346767a89745fa82928a5dac9cc0b2 ({username='slave'} at 127.0.0.1:53724)
I20260709 13:08:07.783967 22890 tablet_copy_service.cc:161] P 0a407504ed1741bf89a276558adf4c0f: Beginning new tablet copy session on tablet 4c0bea78fb2a40ef90cff914c5c6e250 from peer 5b346767a89745fa82928a5dac9cc0b2 at {username='slave'} at 127.0.0.1:53724: session id = 5b346767a89745fa82928a5dac9cc0b2-4c0bea78fb2a40ef90cff914c5c6e250
I20260709 13:08:07.785132 23625 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7"
dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
 from {username='slave'} at 127.0.0.1:60032
I20260709 13:08:07.785634 23625 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:07.785952 23625 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:07.789641 23026 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 2.
I20260709 13:08:07.795078 22890 tablet_copy_source_session.cc:215] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:08:07.806125 23026 raft_consensus.cc:1275] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:07.816653 24111 consensus_queue.cc:1048] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:07.787161 24131 tablet_bootstrap.cc:492] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:07.807840 24138 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c0bea78fb2a40ef90cff914c5c6e250. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:07.829221 24131 tablet_bootstrap.cc:492] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap complete.
I20260709 13:08:07.830108 24131 ts_tablet_manager.cc:1403] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2: Time spent bootstrapping tablet: real 0.086s	user 0.027s	sys 0.003s
I20260709 13:08:07.809208 23625 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:07.822523 22776 catalog_manager.cc:5697] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 1 to 2, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 2 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:07.832727 23625 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:07.834856 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:07.835506 22953 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:07.842669 24131 raft_consensus.cc:359] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:07.843526 24131 raft_consensus.cc:740] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Initialized, Role: LEARNER
I20260709 13:08:07.844183 24131 consensus_queue.cc:260] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:07.849592 24138 tablet_copy_client.cc:806] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 0 data blocks...
I20260709 13:08:07.850160 24138 tablet_copy_client.cc:670] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:08:07.855680 24131 ts_tablet_manager.cc:1434] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 13:08:07.860983 22889 tablet_copy_service.cc:342] P 0a407504ed1741bf89a276558adf4c0f: Request end of tablet copy session 5b346767a89745fa82928a5dac9cc0b2-aa6540758938493d98d7852fdbe3ffab received from {username='slave'} at 127.0.0.1:53724
I20260709 13:08:07.861207 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:07.869310 23026 raft_consensus.cc:3055] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:07.869606 23026 raft_consensus.cc:740] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:07.870316 23026 consensus_queue.cc:260] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:07.871493 23026 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:07.873204 24136 tablet_copy_client.cc:806] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 0 data blocks...
I20260709 13:08:07.861438 22889 tablet_copy_service.cc:434] P 0a407504ed1741bf89a276558adf4c0f: ending tablet copy session 5b346767a89745fa82928a5dac9cc0b2-aa6540758938493d98d7852fdbe3ffab on tablet aa6540758938493d98d7852fdbe3ffab with peer 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:07.875502 22953 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 2.
I20260709 13:08:07.877476 23588 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:07.878335 24090 raft_consensus.cc:2804] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:07.878808 24090 raft_consensus.cc:697] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:07.879648 24090 consensus_queue.cc:237] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:07.892915 23026 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 2.
I20260709 13:08:07.899097 24138 tablet_copy_client.cc:538] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260709 13:08:07.910331 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 54278.5us
I20260709 13:08:07.921154 22953 raft_consensus.cc:1275] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:07.923305 24061 consensus_queue.cc:1048] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:07.933463 22776 catalog_manager.cc:5697] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 1 to 2, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 2 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:07.936273 24138 tablet_bootstrap.cc:492] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap starting.
I20260709 13:08:07.873808 24136 tablet_copy_client.cc:670] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:08:07.949348 24136 tablet_copy_client.cc:538] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:08:07.974053 24138 tablet_bootstrap.cc:492] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:07.976303 24138 tablet_bootstrap.cc:492] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap complete.
I20260709 13:08:07.977044 24138 ts_tablet_manager.cc:1403] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2: Time spent bootstrapping tablet: real 0.059s	user 0.021s	sys 0.005s
I20260709 13:08:07.977103 24136 tablet_bootstrap.cc:492] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap starting.
I20260709 13:08:07.980381 24138 raft_consensus.cc:359] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:07.981254 24138 raft_consensus.cc:740] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Initialized, Role: LEARNER
I20260709 13:08:07.981941 24138 consensus_queue.cc:260] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:07.990283 24138 ts_tablet_manager.cc:1434] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 13:08:07.991915 22890 tablet_copy_service.cc:342] P 0a407504ed1741bf89a276558adf4c0f: Request end of tablet copy session 5b346767a89745fa82928a5dac9cc0b2-4c0bea78fb2a40ef90cff914c5c6e250 received from {username='slave'} at 127.0.0.1:53724
I20260709 13:08:07.992319 22890 tablet_copy_service.cc:434] P 0a407504ed1741bf89a276558adf4c0f: ending tablet copy session 5b346767a89745fa82928a5dac9cc0b2-4c0bea78fb2a40ef90cff914c5c6e250 on tablet 4c0bea78fb2a40ef90cff914c5c6e250 with peer 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:08.010752 24136 tablet_bootstrap.cc:492] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:08.011951 24136 tablet_bootstrap.cc:492] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap complete.
I20260709 13:08:08.012881 24136 ts_tablet_manager.cc:1403] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2: Time spent bootstrapping tablet: real 0.036s	user 0.013s	sys 0.012s
I20260709 13:08:08.015326 24136 raft_consensus.cc:359] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:08.015883 24136 raft_consensus.cc:740] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Initialized, Role: LEARNER
I20260709 13:08:08.016414 24136 consensus_queue.cc:260] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:08.018275 23020 raft_consensus.cc:1275] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:08:08.019685 24136 ts_tablet_manager.cc:1434] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:08:08.019860 24090 consensus_queue.cc:1048] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:08.021536 23039 tablet_copy_service.cc:342] P f3c799b1249e40119d2a5864231c6c75: Request end of tablet copy session 5b346767a89745fa82928a5dac9cc0b2-2959101199a44a4583a4ee456ecf044d received from {username='slave'} at 127.0.0.1:32794
I20260709 13:08:08.022032 23039 tablet_copy_service.cc:434] P f3c799b1249e40119d2a5864231c6c75: ending tablet copy session 5b346767a89745fa82928a5dac9cc0b2-2959101199a44a4583a4ee456ecf044d on tablet 2959101199a44a4583a4ee456ecf044d with peer 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:08.033836 22947 raft_consensus.cc:1275] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:08:08.037117 24091 consensus_queue.cc:1048] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:08.077593 23622 raft_consensus.cc:1217] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:08:08.104802 23622 raft_consensus.cc:1217] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:08:08.182089 22873 raft_consensus.cc:1275] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:08:08.183326 24090 consensus_queue.cc:1048] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:08.194779 23020 raft_consensus.cc:1275] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:08:08.196306 24103 consensus_queue.cc:1048] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:08.205780 23026 raft_consensus.cc:1275] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:08:08.209945 24155 consensus_queue.cc:1048] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:08:08.237061 22873 raft_consensus.cc:1275] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:08:08.238708 24090 consensus_queue.cc:1048] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:08.250185 23625 raft_consensus.cc:1217] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:08:08.288110 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:08.333199 23622 raft_consensus.cc:1217] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.1->[2.2-2.2]   Dedup: 2.2->[]
I20260709 13:08:08.339097 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:08.419960 23020 raft_consensus.cc:1275] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:08:08.421761 23020 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:08:08.422269 23020 raft_consensus.cc:606] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Received request to transfer leadership to TS aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:08.425073 24090 consensus_queue.cc:1048] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:08:08.430052 22873 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
mode: GRACEFUL
new_leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:51434
I20260709 13:08:08.430687 22873 raft_consensus.cc:606] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Received request to transfer leadership to TS aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:08.440404 23026 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:08:08.440992 23026 raft_consensus.cc:606] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:08:08.446911 23020 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:08:08.447510 23020 raft_consensus.cc:606] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:08:08.450699 23020 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:08:08.451375 23020 raft_consensus.cc:606] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:08:08.450738 22953 raft_consensus.cc:1275] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:08:08.453125 21756 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:08:08.453455 24090 consensus_queue.cc:1048] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:08.462924 21756 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:08.521447 24111 raft_consensus.cc:993] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f: : Instructing follower aed3f089ec8b49af9c8cf5cdee76e8e7 to start an election
I20260709 13:08:08.521981 24112 raft_consensus.cc:1081] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Signalling peer aed3f089ec8b49af9c8cf5cdee76e8e7 to start an election
I20260709 13:08:08.523536 22947 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:58310
I20260709 13:08:08.524159 22947 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:08.525249 22947 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:08.532610 22947 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:08.534929 22947 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:08.535305 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:08.535676 24099 raft_consensus.cc:993] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75: : Instructing follower 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:08:08.535933 22873 raft_consensus.cc:3055] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:08.536109 24029 raft_consensus.cc:1081] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Signalling peer 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:08:08.536237 22873 raft_consensus.cc:740] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:08.537047 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:08.537541 23020 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:08.537606 22876 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:35370
I20260709 13:08:08.538147 22876 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:08.538465 22876 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:08.541785 23020 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:08.542687 22873 consensus_queue.cc:260] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:08.543345 22918 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:08.543927 22873 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:08.544943 24097 raft_consensus.cc:2804] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:08.545445 24097 raft_consensus.cc:697] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:08.546247 24097 consensus_queue.cc:237] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:08.552304 22873 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:08.553210 22876 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:08.555773 22876 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:08.553519 24112 raft_consensus.cc:1064] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
I20260709 13:08:08.557009 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:08.557797 23026 raft_consensus.cc:3055] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:08.559800 23026 raft_consensus.cc:740] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:08.557822 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:08.560545 23026 consensus_queue.cc:260] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:08.561259 22947 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:08.561800 23026 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:08.558352 24112 consensus_queue.cc:237] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:08.568063 23026 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:08:08.573112 22843 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:08.573850 24113 raft_consensus.cc:2804] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:08.574286 24099 raft_consensus.cc:993] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75: : Instructing follower aed3f089ec8b49af9c8cf5cdee76e8e7 to start an election
I20260709 13:08:08.574309 24113 raft_consensus.cc:697] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:08.574859 24099 raft_consensus.cc:1081] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Signalling peer aed3f089ec8b49af9c8cf5cdee76e8e7 to start an election
I20260709 13:08:08.575466 24113 consensus_queue.cc:237] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:08.577426 23622 raft_consensus.cc:1275] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:08:08.579751 22953 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:58320
I20260709 13:08:08.580355 22953 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:08.580787 22953 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:08.581735 24111 consensus_queue.cc:1048] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:08.588055 22774 catalog_manager.cc:5697] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 2 to 3, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 3 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:08.590526 23026 raft_consensus.cc:1275] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:08:08.592025 24077 raft_consensus.cc:993] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75: : Instructing follower 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:08:08.605324 24077 raft_consensus.cc:1081] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Signalling peer 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:08:08.609869 22776 catalog_manager.cc:5697] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 1 to 2, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 2 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:08.612848 22876 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:35370
I20260709 13:08:08.613945 22876 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:08.614352 22876 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:08.592950 22947 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:08:08.614737 24111 consensus_queue.cc:1048] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:08.592778 22953 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:08.620492 22953 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:08.620728 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:08.621326 22873 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:08.622846 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:08.623582 23020 raft_consensus.cc:3055] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:08.623868 23020 raft_consensus.cc:740] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:08.624840 23020 consensus_queue.cc:260] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:08.632565 24075 raft_consensus.cc:1064] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
I20260709 13:08:08.635641 23020 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:08.636646 22873 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:08.638731 22920 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:08.639871 24066 raft_consensus.cc:2804] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:08.640302 24066 raft_consensus.cc:697] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:08.641112 24066 consensus_queue.cc:237] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:08.642501 23020 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:08.646450 22876 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:08.650499 22876 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:08.651525 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:08.652406 23023 raft_consensus.cc:3055] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:08.652812 23023 raft_consensus.cc:740] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:08.653900 23023 consensus_queue.cc:260] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:08.654858 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:08.655372 22953 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:08.655699 24075 consensus_queue.cc:237] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:08.655037 23023 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:08.663280 22953 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:08:08.665174 22843 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:08.667493 24111 raft_consensus.cc:2804] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:08.668114 24111 raft_consensus.cc:697] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:08.669075 24111 consensus_queue.cc:237] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:08.686615 23023 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:08:08.727115 24113 raft_consensus.cc:2955] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:08.760144 22872 raft_consensus.cc:1275] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:08:08.767350 23622 raft_consensus.cc:1275] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEARNER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:08:08.770614 24169 consensus_queue.cc:1048] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:08:08.775058 24173 log.cc:927] Time spent T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
I20260709 13:08:08.778126 24097 consensus_queue.cc:1048] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:08.795907 23023 raft_consensus.cc:1275] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:08:08.806452 24097 consensus_queue.cc:1048] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:08:08.819326 24097 raft_consensus.cc:2955] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:08.829737 22876 raft_consensus.cc:2955] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
W20260709 13:08:08.831226 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 109866us
I20260709 13:08:08.834796 23625 raft_consensus.cc:2955] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:08.846241 23623 raft_consensus.cc:2955] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:08.859354 24163 raft_consensus.cc:1064] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
I20260709 13:08:08.859236 22774 catalog_manager.cc:5697] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 1 to 2, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 2 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:08.862335 24163 consensus_queue.cc:237] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 0, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:08.862471 22774 catalog_manager.cc:5697] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: config changed from index 3 to 4, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:08.866706 22953 raft_consensus.cc:1275] T aa6540758938493d98d7852fdbe3ffab P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:08:08.875587 24077 raft_consensus.cc:993] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75: : Instructing follower 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:08:08.875579 22953 raft_consensus.cc:1275] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:08:08.876087 24077 raft_consensus.cc:1081] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Signalling peer 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:08:08.877717 24113 consensus_queue.cc:1048] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:08.882203 23623 raft_consensus.cc:1275] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEARNER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:08:08.883069 22880 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:35370
I20260709 13:08:08.883556 22880 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:08.883852 22880 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:08.879509 24113 consensus_queue.cc:1048] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:08.886386 23026 raft_consensus.cc:2955] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:08.887154 24112 consensus_queue.cc:1048] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:08.893682 22880 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:08.898736 23026 raft_consensus.cc:1275] T 4c0bea78fb2a40ef90cff914c5c6e250 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:08:08.901909 22880 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:08.903113 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:08.905040 22954 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:08.910940 23026 raft_consensus.cc:2955] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:08.917050 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:08.917675 23020 raft_consensus.cc:3055] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:08.917992 23020 raft_consensus.cc:740] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:08.912997 22954 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:08:08.918773 23020 consensus_queue.cc:260] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:08.919950 23020 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:08.926191 23020 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:08:08.927769 22776 catalog_manager.cc:5697] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 1 to 2, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 2 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:08.928895 22948 consensus_queue.cc:237] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:08.931018 22776 catalog_manager.cc:5697] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f reported cstate change: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:08.933465 23023 raft_consensus.cc:1275] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:08.935312 22948 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:58302
I20260709 13:08:08.936117 22948 raft_consensus.cc:606] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 LEADER]: Received request to transfer leadership
I20260709 13:08:08.936851 24075 consensus_queue.cc:1048] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:08.937438 23623 raft_consensus.cc:1275] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:08:08.938961 24075 consensus_queue.cc:1048] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:08.943429 24177 consensus_queue.cc:1048] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:08:08.945554 22953 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:58302
I20260709 13:08:08.946125 22953 raft_consensus.cc:606] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 LEADER]: Received request to transfer leadership
W20260709 13:08:08.947209 22762 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leadership transfer for 2959101199a44a4583a4ee456ecf044d already in progress [suppressed 2 similar messages]
I20260709 13:08:08.955199 23023 raft_consensus.cc:1275] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:08:08.954389 24097 raft_consensus.cc:2955] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:08.960130 22843 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:08.961890 24066 consensus_queue.cc:1048] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:08:08.963691 24112 raft_consensus.cc:2804] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:08.964222 24112 raft_consensus.cc:697] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:08.963971 23623 raft_consensus.cc:2955] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:08.965127 24112 consensus_queue.cc:237] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:08.977661 22872 consensus_queue.cc:237] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:08.985621 23625 raft_consensus.cc:1275] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:08:08.987986 24113 consensus_queue.cc:1048] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:08.995920 24163 raft_consensus.cc:2955] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:09.002982 22777 catalog_manager.cc:5697] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: config changed from index 4 to 5, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New cstate: current_term: 2 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:09.010663 22764 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet aa6540758938493d98d7852fdbe3ffab with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:08:09.005963 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 45910.1us
I20260709 13:08:09.005352 23625 raft_consensus.cc:2955] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
W20260709 13:08:09.112737 22762 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 165521us
I20260709 13:08:09.124189 22947 raft_consensus.cc:2955] T aa6540758938493d98d7852fdbe3ffab P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:09.134992 23026 raft_consensus.cc:2955] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:09.141060 23029 raft_consensus.cc:1275] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:08:09.147107 22762 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet c26347a647b240d7a3bc5cd449b7993b with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:08:09.157536 24163 consensus_queue.cc:1048] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:09.160465 22867 tablet_service.cc:1558] Processing DeleteTablet for tablet c26347a647b240d7a3bc5cd449b7993b with delete_type TABLET_DATA_TOMBSTONED (TS 0a407504ed1741bf89a276558adf4c0f not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:35344
I20260709 13:08:09.162387 23029 raft_consensus.cc:2955] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:09.172228 22954 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:58302
I20260709 13:08:09.172853 22954 raft_consensus.cc:606] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 LEADER]: Received request to transfer leadership
I20260709 13:08:09.178457 22777 catalog_manager.cc:5697] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: config changed from index 3 to 4, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New cstate: current_term: 1 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:09.189968 23625 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Advancing to term 2
I20260709 13:08:09.190290 22953 raft_consensus.cc:1275] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:09.193645 22955 raft_consensus.cc:1275] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:09.198449 24166 raft_consensus.cc:2955] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:09.205343 23625 raft_consensus.cc:1275] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEARNER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:08:09.206696 24075 consensus_queue.cc:1048] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:09.210376 24111 consensus_queue.cc:1048] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:09.214766 24111 consensus_queue.cc:1048] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:09.218699 22948 raft_consensus.cc:2955] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:09.224428 23029 raft_consensus.cc:1275] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:09.229408 24112 consensus_queue.cc:1048] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260709 13:08:09.241684 22943 tablet_service.cc:1558] Processing DeleteTablet for tablet aa6540758938493d98d7852fdbe3ffab with delete_type TABLET_DATA_TOMBSTONED (TS aed3f089ec8b49af9c8cf5cdee76e8e7 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:58302
I20260709 13:08:09.246939 23026 raft_consensus.cc:2955] T 4c0bea78fb2a40ef90cff914c5c6e250 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:09.275879 22880 raft_consensus.cc:1275] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:08:09.276635 24200 tablet_replica.cc:333] T aa6540758938493d98d7852fdbe3ffab P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:08:09.278458 22873 raft_consensus.cc:1275] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:09.281270 24200 raft_consensus.cc:2243] T aa6540758938493d98d7852fdbe3ffab P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:09.282166 24200 raft_consensus.cc:2272] T aa6540758938493d98d7852fdbe3ffab P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:09.281275 23625 raft_consensus.cc:2955] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:09.282756 24075 consensus_queue.cc:1048] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:09.286023 24200 ts_tablet_manager.cc:1916] T aa6540758938493d98d7852fdbe3ffab P aed3f089ec8b49af9c8cf5cdee76e8e7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:08:09.286057 24075 consensus_queue.cc:1048] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:09.294135 23020 raft_consensus.cc:1275] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:09.313321 22777 catalog_manager.cc:5697] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:09.317970 24097 raft_consensus.cc:1064] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
W20260709 13:08:09.318434 24097 raft_consensus.cc:1068] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7: Unable to promote non-voter 5b346767a89745fa82928a5dac9cc0b2: Service unavailable: leader transfer in progress
I20260709 13:08:09.335896 24200 ts_tablet_manager.cc:1929] T aa6540758938493d98d7852fdbe3ffab P aed3f089ec8b49af9c8cf5cdee76e8e7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:08:09.336426 24200 log.cc:1199] T aa6540758938493d98d7852fdbe3ffab P aed3f089ec8b49af9c8cf5cdee76e8e7: Deleting WAL directory at /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-1-root/wals/aa6540758938493d98d7852fdbe3ffab
I20260709 13:08:09.343534 24195 tablet_replica.cc:333] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
W20260709 13:08:09.344798 22826 auto_rebalancer.cc:893] Removing replace marker failed: Service unavailable: leader transfer in progress
W20260709 13:08:09.345197 22826 auto_rebalancer.cc:901] Could not move replica: Remote error: Service unavailable: leadership transfer for 2959101199a44a4583a4ee456ecf044d already in progress
W20260709 13:08:09.345474 22826 auto_rebalancer.cc:330] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 2959101199a44a4583a4ee456ecf044d already in progress
I20260709 13:08:09.259614 22774 catalog_manager.cc:5697] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 1 to 2, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 2 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:09.353817 24113 consensus_queue.cc:1048] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:09.357098 22762 catalog_manager.cc:5028] TS aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): tablet aa6540758938493d98d7852fdbe3ffab (table test-workload [id=a42f2ccfbe484baca3019323a490c64a]) successfully deleted
I20260709 13:08:09.366523 23020 raft_consensus.cc:1275] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:08:09.344527 24195 raft_consensus.cc:2243] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:09.386973 24195 raft_consensus.cc:2272] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:09.389616 24181 consensus_queue.cc:1048] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:09.400007 24075 raft_consensus.cc:993] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7: : Instructing follower 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:08:09.400486 24169 raft_consensus.cc:1081] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 LEADER]: Signalling peer 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:08:09.404255 24195 ts_tablet_manager.cc:1916] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:08:09.415565 22876 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:35360
I20260709 13:08:09.416128 22876 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:09.416433 22876 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:09.417230 24196 raft_consensus.cc:1064] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
W20260709 13:08:09.417665 24196 raft_consensus.cc:1068] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7: Unable to promote non-voter 5b346767a89745fa82928a5dac9cc0b2: Service unavailable: leader transfer in progress
I20260709 13:08:09.423448 22880 consensus_queue.cc:237] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:09.426568 23023 raft_consensus.cc:1275] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:09.427940 24166 consensus_queue.cc:1048] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:09.435079 22947 raft_consensus.cc:1275] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:08:09.435895 23622 raft_consensus.cc:1275] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:08:09.436334 24166 consensus_queue.cc:1048] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:09.438285 22876 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:09.440950 22876 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:09.443464 24113 consensus_queue.cc:1048] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:09.447970 22873 raft_consensus.cc:1240] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:09.446166 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:09.451144 23020 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:09.457801 24195 ts_tablet_manager.cc:1929] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:08:09.458292 24195 log.cc:1199] T c26347a647b240d7a3bc5cd449b7993b P 0a407504ed1741bf89a276558adf4c0f: Deleting WAL directory at /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-0-root/wals/c26347a647b240d7a3bc5cd449b7993b
I20260709 13:08:09.460130 22764 catalog_manager.cc:5028] TS 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): tablet c26347a647b240d7a3bc5cd449b7993b (table test-workload [id=a42f2ccfbe484baca3019323a490c64a]) successfully deleted
I20260709 13:08:09.461323 23020 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 3.
I20260709 13:08:09.462409 24182 raft_consensus.cc:2955] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:09.483366 22843 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:09.484638 24111 raft_consensus.cc:2804] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:09.485174 24111 raft_consensus.cc:697] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 3 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:09.482010 22947 raft_consensus.cc:2955] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:09.464860 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:09.486307 24111 consensus_queue.cc:237] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:09.465543 23625 raft_consensus.cc:2955] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:09.470755 24075 consensus_queue.cc:1059] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:09.471992 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:09.474840 22955 raft_consensus.cc:1275] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:09.493569 22954 raft_consensus.cc:3055] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:09.493913 22954 raft_consensus.cc:740] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:09.494617 22954 consensus_queue.cc:260] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:09.496004 22954 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:09.498440 24111 consensus_queue.cc:1048] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 13:08:09.502947 22954 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 3.
I20260709 13:08:09.513267 22774 catalog_manager.cc:5697] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: config changed from index 3 to 4, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New cstate: current_term: 2 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:09.525517 22764 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4c0bea78fb2a40ef90cff914c5c6e250 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:08:09.534914 22777 catalog_manager.cc:5697] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 2 to 3, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 3 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:09.546370 23019 tablet_service.cc:1558] Processing DeleteTablet for tablet 4c0bea78fb2a40ef90cff914c5c6e250 with delete_type TABLET_DATA_TOMBSTONED (TS f3c799b1249e40119d2a5864231c6c75 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33406
I20260709 13:08:09.573702 24213 tablet_replica.cc:333] T 4c0bea78fb2a40ef90cff914c5c6e250 P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:08:09.574877 24213 raft_consensus.cc:2243] T 4c0bea78fb2a40ef90cff914c5c6e250 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:09.575659 24213 raft_consensus.cc:2272] T 4c0bea78fb2a40ef90cff914c5c6e250 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:09.586194 24213 ts_tablet_manager.cc:1916] T 4c0bea78fb2a40ef90cff914c5c6e250 P f3c799b1249e40119d2a5864231c6c75: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:08:09.595667 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:09.600598 24213 ts_tablet_manager.cc:1929] T 4c0bea78fb2a40ef90cff914c5c6e250 P f3c799b1249e40119d2a5864231c6c75: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.3
I20260709 13:08:09.601089 24213 log.cc:1199] T 4c0bea78fb2a40ef90cff914c5c6e250 P f3c799b1249e40119d2a5864231c6c75: Deleting WAL directory at /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-2-root/wals/4c0bea78fb2a40ef90cff914c5c6e250
I20260709 13:08:09.602559 22762 catalog_manager.cc:5028] TS f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): tablet 4c0bea78fb2a40ef90cff914c5c6e250 (table test-workload [id=a42f2ccfbe484baca3019323a490c64a]) successfully deleted
I20260709 13:08:09.636399 22955 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
mode: GRACEFUL
new_leader_uuid: "5b346767a89745fa82928a5dac9cc0b2"
 from {username='slave'} at 127.0.0.1:36566
I20260709 13:08:09.636987 22955 raft_consensus.cc:606] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Received request to transfer leadership to TS 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:09.639384 22873 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
mode: GRACEFUL
new_leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:51434
I20260709 13:08:09.639851 22873 raft_consensus.cc:606] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 3 LEADER]: Received request to transfer leadership to TS aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:09.642382 22873 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
mode: GRACEFUL
new_leader_uuid: "5b346767a89745fa82928a5dac9cc0b2"
 from {username='slave'} at 127.0.0.1:51434
I20260709 13:08:09.642900 22873 raft_consensus.cc:606] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Received request to transfer leadership to TS 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:09.646049 22873 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
mode: GRACEFUL
new_leader_uuid: "5b346767a89745fa82928a5dac9cc0b2"
 from {username='slave'} at 127.0.0.1:51434
I20260709 13:08:09.646610 22873 raft_consensus.cc:606] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Received request to transfer leadership to TS 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:09.648953 23020 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:08:09.649415 23020 raft_consensus.cc:606] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:08:09.651654 22873 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
mode: GRACEFUL
new_leader_uuid: "5b346767a89745fa82928a5dac9cc0b2"
 from {username='slave'} at 127.0.0.1:51434
I20260709 13:08:09.652168 22873 raft_consensus.cc:606] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Received request to transfer leadership to TS 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:09.653158 21756 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:08:09.654389 21756 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:09.760413 24113 raft_consensus.cc:993] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f: : Instructing follower 5b346767a89745fa82928a5dac9cc0b2 to start an election
I20260709 13:08:09.761085 24182 raft_consensus.cc:1081] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Signalling peer 5b346767a89745fa82928a5dac9cc0b2 to start an election
I20260709 13:08:09.762892 23622 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd"
dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
 from {username='slave'} at 127.0.0.1:60026
I20260709 13:08:09.763441 23622 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:09.763798 23622 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:09.769867 23622 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:09.771291 23622 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:09.772040 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:09.772483 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:09.772768 22873 raft_consensus.cc:3055] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:09.773058 23020 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:09.773111 22873 raft_consensus.cc:740] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:09.773816 22873 consensus_queue.cc:260] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:09.774696 22873 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:09.778620 23020 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 2.
I20260709 13:08:09.779713 23588 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:09.780004 22873 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 2.
I20260709 13:08:09.780527 24103 raft_consensus.cc:2804] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:09.781128 24103 raft_consensus.cc:697] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:09.782426 24103 consensus_queue.cc:237] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:09.792047 22777 catalog_manager.cc:5697] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 1 to 2, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 2 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:09.857307 24178 raft_consensus.cc:993] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75: : Instructing follower 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:08:09.857865 24077 raft_consensus.cc:1081] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Signalling peer 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:08:09.859244 22880 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:35370
I20260709 13:08:09.859722 22880 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:09.860155 22880 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:09.865012 22880 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:09.866516 22880 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:09.867286 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:09.867673 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:09.868220 22947 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:09.868268 23020 raft_consensus.cc:3055] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:09.868827 23020 raft_consensus.cc:740] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:09.869659 23020 consensus_queue.cc:260] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:09.870776 23020 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:09.872364 24197 raft_consensus.cc:993] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7: : Instructing follower 5b346767a89745fa82928a5dac9cc0b2 to start an election
I20260709 13:08:09.872852 24197 raft_consensus.cc:1081] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Signalling peer 5b346767a89745fa82928a5dac9cc0b2 to start an election
I20260709 13:08:09.875100 23622 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf"
dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:08:09.875589 23622 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:09.875895 23622 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:09.875125 22947 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:08:09.877301 23029 raft_consensus.cc:1275] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:08:09.878237 22843 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:09.879004 24182 raft_consensus.cc:2804] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:09.879483 24113 consensus_queue.cc:1048] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:09.879813 24182 raft_consensus.cc:697] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:09.880122 23020 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:08:09.881320 24182 consensus_queue.cc:237] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:09.882361 23622 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:09.884188 23622 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:09.885500 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:09.899260 22947 raft_consensus.cc:3055] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:09.899938 22947 raft_consensus.cc:740] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:09.900813 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:09.901382 23029 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:09.907006 23625 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEARNER]: Advancing to term 3
I20260709 13:08:09.908458 23029 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 2.
I20260709 13:08:09.909691 22954 raft_consensus.cc:1275] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:08:09.909462 22947 consensus_queue.cc:260] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:09.910742 22947 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:09.912072 23588 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:09.912905 24103 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:09.913039 24166 consensus_queue.cc:1048] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:08:09.913417 24103 raft_consensus.cc:697] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:09.914273 24103 consensus_queue.cc:237] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:09.922734 23625 raft_consensus.cc:1275] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 3 LEARNER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:08:09.924268 24162 consensus_queue.cc:1048] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:09.931030 22776 catalog_manager.cc:5697] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 1 to 2, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 2 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:09.933630 24111 raft_consensus.cc:993] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f: : Instructing follower aed3f089ec8b49af9c8cf5cdee76e8e7 to start an election
I20260709 13:08:09.938104 24111 raft_consensus.cc:1081] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 3 LEADER]: Signalling peer aed3f089ec8b49af9c8cf5cdee76e8e7 to start an election
I20260709 13:08:09.948216 22954 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:58310
I20260709 13:08:09.949018 22954 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:09.949488 22954 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:09.960059 24182 raft_consensus.cc:993] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f: : Instructing follower 5b346767a89745fa82928a5dac9cc0b2 to start an election
I20260709 13:08:09.961941 24111 raft_consensus.cc:1081] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Signalling peer 5b346767a89745fa82928a5dac9cc0b2 to start an election
I20260709 13:08:09.960175 22954 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:09.960063 22947 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 2.
I20260709 13:08:09.964365 23622 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac"
dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
 from {username='slave'} at 127.0.0.1:60026
I20260709 13:08:09.964931 23622 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:09.965730 22954 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:09.965968 23622 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:09.966894 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: true dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:09.967465 22880 raft_consensus.cc:3055] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:09.967782 22880 raft_consensus.cc:740] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:09.968554 22880 consensus_queue.cc:260] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:09.969964 22777 catalog_manager.cc:5697] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 1 to 2, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 2 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:09.971037 22880 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:09.970961 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:09.972522 23023 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:09.972138 23622 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:09.976930 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:09.977466 22876 raft_consensus.cc:3055] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:09.977754 22876 raft_consensus.cc:740] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:09.978404 22876 consensus_queue.cc:260] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:09.979384 22876 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:09.984035 22880 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
I20260709 13:08:09.985987 22920 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:09.986958 24196 raft_consensus.cc:2804] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:09.986023 23023 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
I20260709 13:08:09.993474 24196 raft_consensus.cc:697] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:09.994423 24196 consensus_queue.cc:237] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:10.001142 22876 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 2.
I20260709 13:08:10.002118 23590 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:10.002918 24103 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:10.003392 24103 raft_consensus.cc:697] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:10.004338 24103 consensus_queue.cc:237] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:10.013208 23622 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:10.013432 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:10.016359 23020 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:10.024384 22777 catalog_manager.cc:5697] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 1 to 2, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 2 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:10.026350 23020 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 2.
I20260709 13:08:10.027791 22776 catalog_manager.cc:5697] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 3 to 4, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 4 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:10.099900 24166 raft_consensus.cc:993] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f: : Instructing follower 5b346767a89745fa82928a5dac9cc0b2 to start an election
I20260709 13:08:10.101159 24166 raft_consensus.cc:1081] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Signalling peer 5b346767a89745fa82928a5dac9cc0b2 to start an election
I20260709 13:08:10.104761 23625 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab"
dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
 from {username='slave'} at 127.0.0.1:60026
I20260709 13:08:10.105662 23625 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:10.106150 23625 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:10.136474 23625 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:10.148945 23625 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:10.150056 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:10.150816 22880 raft_consensus.cc:3055] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:10.151222 22880 raft_consensus.cc:740] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:10.152576 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:10.153648 23023 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:10.153033 22880 consensus_queue.cc:260] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:10.154758 22880 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:10.171974 22880 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 2.
I20260709 13:08:10.173799 23590 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:10.174855 24103 raft_consensus.cc:2804] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:10.175318 24103 raft_consensus.cc:697] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:10.176258 24103 consensus_queue.cc:237] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:10.186794 23023 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 2.
I20260709 13:08:10.189396 22777 catalog_manager.cc:5697] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 1 to 2, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 2 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:10.215981 23029 raft_consensus.cc:1275] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:08:10.217335 24103 consensus_queue.cc:1048] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:08:10.244966 22880 raft_consensus.cc:1275] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:08:10.247910 24091 consensus_queue.cc:1048] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:08:10.390661 23023 raft_consensus.cc:1275] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:10.394670 23020 raft_consensus.cc:1275] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:08:10.398545 24111 consensus_queue.cc:1048] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:10.400681 24103 consensus_queue.cc:1048] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:10.407821 22954 raft_consensus.cc:1275] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:08:10.409855 24103 consensus_queue.cc:1048] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:08:10.448637 23020 raft_consensus.cc:1275] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:08:10.450464 24075 consensus_queue.cc:1048] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:10.469986 23625 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 3 LEARNER]: Advancing to term 4
I20260709 13:08:10.470868 22873 raft_consensus.cc:1275] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:08:10.472496 24220 consensus_queue.cc:1048] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:08:10.474650 23020 raft_consensus.cc:1275] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:08:10.480752 23625 raft_consensus.cc:1275] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 4 LEARNER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:08:10.487010 24103 consensus_queue.cc:1048] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:10.488076 22876 raft_consensus.cc:1275] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:08:10.489025 24169 consensus_queue.cc:1048] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:08:10.493335 24223 consensus_queue.cc:1048] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:08:10.507540 24197 raft_consensus.cc:1064] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
I20260709 13:08:10.511863 24197 consensus_queue.cc:237] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 4, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:10.527910 23622 raft_consensus.cc:1275] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 4 LEARNER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:08:10.530954 22947 raft_consensus.cc:1275] T 7f35ebcf93ad423ea8bd0b777bee23ee P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:10.532428 24197 consensus_queue.cc:1048] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:08:10.537371 24113 consensus_queue.cc:1048] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:10.540388 22876 raft_consensus.cc:1275] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:08:10.545010 24220 consensus_queue.cc:1048] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:08:10.553495 23029 raft_consensus.cc:1275] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:08:10.555619 24169 consensus_queue.cc:1048] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:08:10.579082 24233 raft_consensus.cc:2955] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 LEADER]: Committing config change with OpId 4.6: config changed from index 2 to 6, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:10.582473 22876 raft_consensus.cc:2955] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 2 to 6, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:10.591737 23622 raft_consensus.cc:2955] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 2 to 6, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:10.596462 23029 raft_consensus.cc:2955] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 2 to 6, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:10.608083 23023 raft_consensus.cc:1275] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:08:10.609696 24228 consensus_queue.cc:1048] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:08:10.622525 22775 catalog_manager.cc:5697] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: config changed from index 2 to 6, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:10.636941 22873 raft_consensus.cc:1275] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:08:10.649351 24091 consensus_queue.cc:1048] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:10.655576 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:10.693040 23020 consensus_queue.cc:237] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
W20260709 13:08:10.716289 22993 consensus_peers.cc:597] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 -> Peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Couldn't send request to peer 5b346767a89745fa82928a5dac9cc0b2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8217046fd4e14cc4a0194da4f6ad3887. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:10.717698 22873 raft_consensus.cc:1275] T 8217046fd4e14cc4a0194da4f6ad3887 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:08:10.720793 22955 raft_consensus.cc:1275] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:08:10.720996 24077 consensus_queue.cc:1048] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:10.725751 24099 consensus_queue.cc:1048] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:10.738992 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:10.745751 24225 raft_consensus.cc:2955] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:10.747545 22880 raft_consensus.cc:2955] T 8217046fd4e14cc4a0194da4f6ad3887 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:10.752581 22947 raft_consensus.cc:2955] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:10.799047 22774 catalog_manager.cc:5697] T 8217046fd4e14cc4a0194da4f6ad3887 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:10.810093 23029 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:08:10.810806 23029 raft_consensus.cc:606] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:08:10.821825 22880 consensus_queue.cc:237] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:10.823097 22955 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
mode: GRACEFUL
new_leader_uuid: "f3c799b1249e40119d2a5864231c6c75"
 from {username='slave'} at 127.0.0.1:36566
I20260709 13:08:10.824276 22955 raft_consensus.cc:606] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Received request to transfer leadership to TS f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:10.830361 22844 consensus_peers.cc:597] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f -> Peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Couldn't send request to peer 5b346767a89745fa82928a5dac9cc0b2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7f35ebcf93ad423ea8bd0b777bee23ee. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:10.830767 23029 raft_consensus.cc:1275] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:08:10.833375 22873 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
mode: GRACEFUL
new_leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:51434
I20260709 13:08:10.833926 22873 raft_consensus.cc:606] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Received request to transfer leadership to TS aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:10.834905 24111 consensus_queue.cc:1048] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:10.837757 23020 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:08:10.838296 23020 raft_consensus.cc:606] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Received request to transfer leadership to TS aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:10.839557 21756 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:08:10.840148 22955 raft_consensus.cc:1275] T 7f35ebcf93ad423ea8bd0b777bee23ee P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:08:10.841403 21756 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:10.846258 24113 consensus_queue.cc:1048] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:10.851085 24182 raft_consensus.cc:2955] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:10.856374 22955 raft_consensus.cc:2955] T 7f35ebcf93ad423ea8bd0b777bee23ee P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:10.858597 23029 raft_consensus.cc:2955] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:10.880592 22774 catalog_manager.cc:5697] T 7f35ebcf93ad423ea8bd0b777bee23ee P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: config changed from index -1 to 3, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New cstate: current_term: 2 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:10.885061 22955 consensus_queue.cc:237] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:10.896230 23029 raft_consensus.cc:1275] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 13:08:10.900992 22919 consensus_peers.cc:597] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Couldn't send request to peer 5b346767a89745fa82928a5dac9cc0b2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5ab4700268fc4c67b9930087c5da71c8. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:10.903374 22880 raft_consensus.cc:1275] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:08:10.903398 24220 consensus_queue.cc:1048] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:10.905102 24197 consensus_queue.cc:1048] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:10.917402 24233 raft_consensus.cc:2955] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:10.924685 23029 raft_consensus.cc:2955] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:10.926990 22880 raft_consensus.cc:2955] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:10.930958 24111 raft_consensus.cc:993] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f: : Instructing follower aed3f089ec8b49af9c8cf5cdee76e8e7 to start an election
I20260709 13:08:10.931447 24111 raft_consensus.cc:1081] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Signalling peer aed3f089ec8b49af9c8cf5cdee76e8e7 to start an election
I20260709 13:08:10.933667 22947 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:58310
I20260709 13:08:10.934175 22947 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:10.934504 22947 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:10.936975 23023 consensus_queue.cc:237] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:10.942075 22947 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:10.944967 22947 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:10.945108 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:10.945746 22873 raft_consensus.cc:3055] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:10.946043 22873 raft_consensus.cc:740] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:10.947000 22873 consensus_queue.cc:260] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:10.948403 22873 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:10.945976 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:10.955274 23622 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:08:10.964100 22993 consensus_peers.cc:597] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 -> Peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Couldn't send request to peer 5b346767a89745fa82928a5dac9cc0b2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ec334666b7964c969125af95d9c797ed. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:08:10.965855 22955 raft_consensus.cc:1275] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:08:10.966177 22876 raft_consensus.cc:1275] T ec334666b7964c969125af95d9c797ed P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 13:08:10.951015 22777 catalog_manager.cc:5697] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New cstate: current_term: 1 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:10.967837 22873 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:10.968226 24225 consensus_queue.cc:1048] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:10.969041 22920 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:10.969177 24077 consensus_queue.cc:1048] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:10.970211 24075 raft_consensus.cc:2804] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:10.970654 24075 raft_consensus.cc:697] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:10.971129 23622 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:10.971571 24075 consensus_queue.cc:237] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:11.009630 24099 raft_consensus.cc:2955] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:11.014221 22873 raft_consensus.cc:2955] T ec334666b7964c969125af95d9c797ed P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:11.021351 22947 raft_consensus.cc:2955] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:08:11.032814 22777 catalog_manager.cc:5697] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 1 to 2, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 2 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:11.038590 22775 catalog_manager.cc:5697] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 reported cstate change: config changed from index -1 to 2, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:11.056080 24225 raft_consensus.cc:993] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75: : Instructing follower 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:08:11.056903 24225 raft_consensus.cc:1081] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Signalling peer 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:08:11.058509 22873 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:35370
I20260709 13:08:11.059032 22873 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:11.059356 22873 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:11.065078 22873 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:11.066599 22873 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:11.067795 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:11.068373 22947 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:11.068513 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:11.069113 23029 raft_consensus.cc:3055] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:11.069398 23029 raft_consensus.cc:740] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:11.070417 23029 consensus_queue.cc:260] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:11.071434 23029 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:11.075241 22947 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:08:11.077087 22843 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:11.078924 24111 raft_consensus.cc:2804] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:11.079383 24111 raft_consensus.cc:697] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:11.080232 24111 consensus_queue.cc:237] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:11.089275 23029 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:08:11.097781 22777 catalog_manager.cc:5697] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 1 to 2, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 2 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:11.203492 24075 raft_consensus.cc:993] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7: : Instructing follower f3c799b1249e40119d2a5864231c6c75 to start an election
I20260709 13:08:11.204097 24233 raft_consensus.cc:1081] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Signalling peer f3c799b1249e40119d2a5864231c6c75 to start an election
I20260709 13:08:11.206056 23029 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
 from {username='slave'} at 127.0.0.1:33420
I20260709 13:08:11.206573 23029 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:11.206907 23029 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:11.213603 23029 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:11.215766 23029 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:11.216077 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:11.216264 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:11.216656 22955 raft_consensus.cc:3055] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:11.217000 22955 raft_consensus.cc:740] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:11.217173 23622 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:11.217816 22955 consensus_queue.cc:260] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:11.219156 22955 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:11.226235 23622 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 2.
I20260709 13:08:11.227608 22993 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:11.228390 24077 raft_consensus.cc:2804] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:11.228835 24077 raft_consensus.cc:697] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 2 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:11.229890 24248 ts_tablet_manager.cc:933] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2: Initiating tablet copy from peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:11.230360 24077 consensus_queue.cc:237] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:11.244305 22955 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 2.
I20260709 13:08:11.246668 24248 tablet_copy_client.cc:323] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.63.3:42499
I20260709 13:08:11.248934 23039 tablet_copy_service.cc:140] P f3c799b1249e40119d2a5864231c6c75: Received BeginTabletCopySession request for tablet 8217046fd4e14cc4a0194da4f6ad3887 from peer 5b346767a89745fa82928a5dac9cc0b2 ({username='slave'} at 127.0.0.1:32794)
I20260709 13:08:11.249440 23039 tablet_copy_service.cc:161] P f3c799b1249e40119d2a5864231c6c75: Beginning new tablet copy session on tablet 8217046fd4e14cc4a0194da4f6ad3887 from peer 5b346767a89745fa82928a5dac9cc0b2 at {username='slave'} at 127.0.0.1:32794: session id = 5b346767a89745fa82928a5dac9cc0b2-8217046fd4e14cc4a0194da4f6ad3887
I20260709 13:08:11.255951 23039 tablet_copy_source_session.cc:215] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:08:11.261345 22777 catalog_manager.cc:5697] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 1 to 2, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 2 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:11.271891 24248 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8217046fd4e14cc4a0194da4f6ad3887. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:11.283432 24251 ts_tablet_manager.cc:933] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2: Initiating tablet copy from peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929)
I20260709 13:08:11.289084 24251 tablet_copy_client.cc:323] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.63.1:33929
I20260709 13:08:11.290947 22890 tablet_copy_service.cc:140] P 0a407504ed1741bf89a276558adf4c0f: Received BeginTabletCopySession request for tablet 7f35ebcf93ad423ea8bd0b777bee23ee from peer 5b346767a89745fa82928a5dac9cc0b2 ({username='slave'} at 127.0.0.1:53724)
I20260709 13:08:11.291469 22890 tablet_copy_service.cc:161] P 0a407504ed1741bf89a276558adf4c0f: Beginning new tablet copy session on tablet 7f35ebcf93ad423ea8bd0b777bee23ee from peer 5b346767a89745fa82928a5dac9cc0b2 at {username='slave'} at 127.0.0.1:53724: session id = 5b346767a89745fa82928a5dac9cc0b2-7f35ebcf93ad423ea8bd0b777bee23ee
I20260709 13:08:11.299320 22890 tablet_copy_source_session.cc:215] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:08:11.304140 24248 tablet_copy_client.cc:806] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 0 data blocks...
I20260709 13:08:11.304813 24248 tablet_copy_client.cc:670] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:08:11.308233 24251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f35ebcf93ad423ea8bd0b777bee23ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:11.311527 24248 tablet_copy_client.cc:538] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:08:11.321635 24248 tablet_bootstrap.cc:492] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap starting.
I20260709 13:08:11.329907 24251 tablet_copy_client.cc:806] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 0 data blocks...
I20260709 13:08:11.330509 24251 tablet_copy_client.cc:670] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:08:11.338966 24251 tablet_copy_client.cc:538] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:08:11.348433 24251 tablet_bootstrap.cc:492] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap starting.
I20260709 13:08:11.390377 24225 raft_consensus.cc:993] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75: : Instructing follower aed3f089ec8b49af9c8cf5cdee76e8e7 to start an election
I20260709 13:08:11.391008 24225 raft_consensus.cc:1081] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Signalling peer aed3f089ec8b49af9c8cf5cdee76e8e7 to start an election
I20260709 13:08:11.396109 22955 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:58320
I20260709 13:08:11.396840 22955 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:11.397291 22955 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:11.399127 24254 ts_tablet_manager.cc:933] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2: Initiating tablet copy from peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:11.406672 24251 tablet_bootstrap.cc:492] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:11.407615 24251 tablet_bootstrap.cc:492] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap complete.
I20260709 13:08:11.408305 24251 ts_tablet_manager.cc:1403] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2: Time spent bootstrapping tablet: real 0.060s	user 0.019s	sys 0.010s
I20260709 13:08:11.411363 24251 raft_consensus.cc:359] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:11.412181 24251 raft_consensus.cc:740] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Initialized, Role: LEARNER
I20260709 13:08:11.411924 22955 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:11.412909 24251 consensus_queue.cc:260] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:11.416002 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:11.416615 23029 raft_consensus.cc:3055] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:11.417048 23029 raft_consensus.cc:740] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:11.417960 23029 consensus_queue.cc:260] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:11.419077 23029 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:11.420295 24248 tablet_bootstrap.cc:492] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:11.421258 24248 tablet_bootstrap.cc:492] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap complete.
I20260709 13:08:11.421928 24248 ts_tablet_manager.cc:1403] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2: Time spent bootstrapping tablet: real 0.101s	user 0.029s	sys 0.035s
I20260709 13:08:11.425586 24251 ts_tablet_manager.cc:1434] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2: Time spent starting tablet: real 0.017s	user 0.010s	sys 0.000s
I20260709 13:08:11.428918 24248 raft_consensus.cc:359] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:11.429790 24248 raft_consensus.cc:740] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Initialized, Role: LEARNER
I20260709 13:08:11.430480 24248 consensus_queue.cc:260] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:11.435904 24248 ts_tablet_manager.cc:1434] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:08:11.439824 23039 tablet_copy_service.cc:342] P f3c799b1249e40119d2a5864231c6c75: Request end of tablet copy session 5b346767a89745fa82928a5dac9cc0b2-8217046fd4e14cc4a0194da4f6ad3887 received from {username='slave'} at 127.0.0.1:32794
I20260709 13:08:11.440423 23039 tablet_copy_service.cc:434] P f3c799b1249e40119d2a5864231c6c75: ending tablet copy session 5b346767a89745fa82928a5dac9cc0b2-8217046fd4e14cc4a0194da4f6ad3887 on tablet 8217046fd4e14cc4a0194da4f6ad3887 with peer 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:11.450018 22890 tablet_copy_service.cc:342] P 0a407504ed1741bf89a276558adf4c0f: Request end of tablet copy session 5b346767a89745fa82928a5dac9cc0b2-7f35ebcf93ad423ea8bd0b777bee23ee received from {username='slave'} at 127.0.0.1:53724
I20260709 13:08:11.450693 22890 tablet_copy_service.cc:434] P 0a407504ed1741bf89a276558adf4c0f: ending tablet copy session 5b346767a89745fa82928a5dac9cc0b2-7f35ebcf93ad423ea8bd0b777bee23ee on tablet 7f35ebcf93ad423ea8bd0b777bee23ee with peer 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:11.454741 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:11.455104 22955 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:11.455293 23625 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:11.457094 24254 tablet_copy_client.cc:323] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.63.2:33309
I20260709 13:08:11.459048 22965 tablet_copy_service.cc:140] P aed3f089ec8b49af9c8cf5cdee76e8e7: Received BeginTabletCopySession request for tablet 5ab4700268fc4c67b9930087c5da71c8 from peer 5b346767a89745fa82928a5dac9cc0b2 ({username='slave'} at 127.0.0.1:52222)
I20260709 13:08:11.459643 22965 tablet_copy_service.cc:161] P aed3f089ec8b49af9c8cf5cdee76e8e7: Beginning new tablet copy session on tablet 5ab4700268fc4c67b9930087c5da71c8 from peer 5b346767a89745fa82928a5dac9cc0b2 at {username='slave'} at 127.0.0.1:52222: session id = 5b346767a89745fa82928a5dac9cc0b2-5ab4700268fc4c67b9930087c5da71c8
I20260709 13:08:11.460542 23029 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:11.462046 22918 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:11.463486 24075 raft_consensus.cc:2804] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:11.464021 24075 raft_consensus.cc:697] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:11.464882 24075 consensus_queue.cc:237] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:11.468926 23625 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:11.478585 22965 tablet_copy_source_session.cc:215] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:08:11.482383 24254 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ab4700268fc4c67b9930087c5da71c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:11.481854 22775 catalog_manager.cc:5697] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 1 to 2, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 2 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:11.502780 24254 tablet_copy_client.cc:806] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 0 data blocks...
I20260709 13:08:11.503429 24254 tablet_copy_client.cc:670] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:08:11.505203 24251 ts_tablet_manager.cc:933] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2: Initiating tablet copy from peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:11.509654 24254 tablet_copy_client.cc:538] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:08:11.512880 22955 raft_consensus.cc:1275] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:11.514379 24111 consensus_queue.cc:1048] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:11.517062 24251 tablet_copy_client.cc:323] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.63.3:42499
I20260709 13:08:11.517231 24254 tablet_bootstrap.cc:492] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap starting.
I20260709 13:08:11.519209 23039 tablet_copy_service.cc:140] P f3c799b1249e40119d2a5864231c6c75: Received BeginTabletCopySession request for tablet ec334666b7964c969125af95d9c797ed from peer 5b346767a89745fa82928a5dac9cc0b2 ({username='slave'} at 127.0.0.1:32794)
I20260709 13:08:11.519856 23039 tablet_copy_service.cc:161] P f3c799b1249e40119d2a5864231c6c75: Beginning new tablet copy session on tablet ec334666b7964c969125af95d9c797ed from peer 5b346767a89745fa82928a5dac9cc0b2 at {username='slave'} at 127.0.0.1:32794: session id = 5b346767a89745fa82928a5dac9cc0b2-ec334666b7964c969125af95d9c797ed
I20260709 13:08:11.530736 23625 raft_consensus.cc:1275] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:08:11.532481 24220 consensus_queue.cc:1048] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:08:11.562733 23039 tablet_copy_source_session.cc:215] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:08:11.569483 24251 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec334666b7964c969125af95d9c797ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:08:11.586664 22873 raft_consensus.cc:1275] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:08:11.596336 24220 consensus_queue.cc:1048] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:11.597766 24251 tablet_copy_client.cc:806] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 0 data blocks...
I20260709 13:08:11.598372 24251 tablet_copy_client.cc:670] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:08:11.623472 23020 raft_consensus.cc:1275] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:11.626226 24111 consensus_queue.cc:1048] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:11.659291 22955 raft_consensus.cc:1275] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:08:11.668952 24077 consensus_queue.cc:1048] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:11.672185 23622 raft_consensus.cc:1217] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:08:11.689077 24251 tablet_copy_client.cc:538] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:08:11.689589 24254 tablet_bootstrap.cc:492] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:11.690500 24254 tablet_bootstrap.cc:492] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap complete.
I20260709 13:08:11.691131 24254 ts_tablet_manager.cc:1403] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2: Time spent bootstrapping tablet: real 0.174s	user 0.056s	sys 0.039s
I20260709 13:08:11.710259 24254 raft_consensus.cc:359] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:11.711115 24254 raft_consensus.cc:740] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Initialized, Role: LEARNER
I20260709 13:08:11.711867 24254 consensus_queue.cc:260] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:11.713685 24254 ts_tablet_manager.cc:1434] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2: Time spent starting tablet: real 0.022s	user 0.004s	sys 0.004s
I20260709 13:08:11.718698 22965 tablet_copy_service.cc:342] P aed3f089ec8b49af9c8cf5cdee76e8e7: Request end of tablet copy session 5b346767a89745fa82928a5dac9cc0b2-5ab4700268fc4c67b9930087c5da71c8 received from {username='slave'} at 127.0.0.1:52222
I20260709 13:08:11.719133 22965 tablet_copy_service.cc:434] P aed3f089ec8b49af9c8cf5cdee76e8e7: ending tablet copy session 5b346767a89745fa82928a5dac9cc0b2-5ab4700268fc4c67b9930087c5da71c8 on tablet 5ab4700268fc4c67b9930087c5da71c8 with peer 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:11.732244 23624 raft_consensus.cc:1275] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:08:11.735821 24251 tablet_bootstrap.cc:492] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap starting.
I20260709 13:08:11.736562 24077 consensus_queue.cc:1048] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:11.813817 24251 tablet_bootstrap.cc:492] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:08:11.814822 24251 tablet_bootstrap.cc:492] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap complete.
I20260709 13:08:11.815539 24251 ts_tablet_manager.cc:1403] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2: Time spent bootstrapping tablet: real 0.080s	user 0.026s	sys 0.011s
I20260709 13:08:11.829687 23624 raft_consensus.cc:1217] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 13:08:11.834753 24251 raft_consensus.cc:359] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:11.838354 24251 raft_consensus.cc:740] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Initialized, Role: LEARNER
I20260709 13:08:11.839524 24251 consensus_queue.cc:260] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:11.842692 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:11.844822 24251 ts_tablet_manager.cc:1434] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.000s
I20260709 13:08:11.859408 23039 tablet_copy_service.cc:342] P f3c799b1249e40119d2a5864231c6c75: Request end of tablet copy session 5b346767a89745fa82928a5dac9cc0b2-ec334666b7964c969125af95d9c797ed received from {username='slave'} at 127.0.0.1:32794
I20260709 13:08:11.859840 23039 tablet_copy_service.cc:434] P f3c799b1249e40119d2a5864231c6c75: ending tablet copy session 5b346767a89745fa82928a5dac9cc0b2-ec334666b7964c969125af95d9c797ed on tablet ec334666b7964c969125af95d9c797ed with peer 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:11.898671 23624 raft_consensus.cc:1217] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:08:11.914230 23020 raft_consensus.cc:1275] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:08:11.915364 24220 consensus_queue.cc:1048] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:11.918298 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:11.966620 23624 raft_consensus.cc:1275] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:08:11.968948 23624 raft_consensus.cc:1217] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 13:08:11.981305 24075 consensus_queue.cc:1048] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:11.991976 22947 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:36566
I20260709 13:08:11.992497 22947 raft_consensus.cc:606] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:08:12.026456 22955 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:36566
I20260709 13:08:12.027248 22955 raft_consensus.cc:606] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:08:12.036620 23029 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:08:12.037245 23029 raft_consensus.cc:606] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Received request to transfer leadership to TS aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:12.041479 22876 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
mode: GRACEFUL
new_leader_uuid: "f3c799b1249e40119d2a5864231c6c75"
 from {username='slave'} at 127.0.0.1:51434
I20260709 13:08:12.042047 22876 raft_consensus.cc:606] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Received request to transfer leadership to TS f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:12.043283 21756 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:08:12.045086 21756 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:12.094192 24099 raft_consensus.cc:1064] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
I20260709 13:08:12.096395 24099 consensus_queue.cc:237] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:12.105669 22955 raft_consensus.cc:1275] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:08:12.106065 22876 raft_consensus.cc:1275] T 8217046fd4e14cc4a0194da4f6ad3887 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:08:12.106812 24077 consensus_queue.cc:1048] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:12.108156 24099 consensus_queue.cc:1048] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:12.110009 23624 raft_consensus.cc:1275] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:08:12.116108 24225 consensus_queue.cc:1048] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260709 13:08:12.146881 24099 raft_consensus.cc:2955] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:12.148113 22876 raft_consensus.cc:2955] T 8217046fd4e14cc4a0194da4f6ad3887 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:12.149787 23622 raft_consensus.cc:2955] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:12.171309 22776 catalog_manager.cc:5697] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:12.179030 22955 raft_consensus.cc:2955] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:12.210180 23029 consensus_queue.cc:237] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:12.214941 23622 raft_consensus.cc:1275] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:08:12.215181 22955 raft_consensus.cc:1275] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:08:12.216156 24282 consensus_queue.cc:1048] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:12.217128 24099 consensus_queue.cc:1048] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:12.223727 24099 raft_consensus.cc:2955] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:12.225709 22955 raft_consensus.cc:2955] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:12.230175 23622 raft_consensus.cc:2955] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:12.245061 22762 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8217046fd4e14cc4a0194da4f6ad3887 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:08:12.246018 22774 catalog_manager.cc:5697] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: config changed from index 3 to 4, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:12.258332 22867 tablet_service.cc:1558] Processing DeleteTablet for tablet 8217046fd4e14cc4a0194da4f6ad3887 with delete_type TABLET_DATA_TOMBSTONED (TS 0a407504ed1741bf89a276558adf4c0f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:35344
I20260709 13:08:12.261328 24220 raft_consensus.cc:993] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7: : Instructing follower 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:08:12.261837 24075 raft_consensus.cc:1081] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Signalling peer 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:08:12.263782 22876 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:35360
I20260709 13:08:12.264287 22876 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:12.264600 22876 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:12.272214 24287 tablet_replica.cc:333] T 8217046fd4e14cc4a0194da4f6ad3887 P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:08:12.274356 24287 raft_consensus.cc:2243] T 8217046fd4e14cc4a0194da4f6ad3887 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:12.275036 24287 raft_consensus.cc:2272] T 8217046fd4e14cc4a0194da4f6ad3887 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:12.275539 22876 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:12.277994 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:12.278528 22955 raft_consensus.cc:3055] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:12.278616 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:12.279322 23029 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:12.281499 22876 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:12.278807 22955 raft_consensus.cc:740] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:12.284569 22955 consensus_queue.cc:260] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:12.285701 22955 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:12.286409 23029 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:08:12.292559 22843 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:12.293512 24111 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:12.293991 24111 raft_consensus.cc:697] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:12.294801 24111 consensus_queue.cc:237] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:12.298135 22955 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:08:12.298401 24287 ts_tablet_manager.cc:1916] T 8217046fd4e14cc4a0194da4f6ad3887 P 0a407504ed1741bf89a276558adf4c0f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:08:12.317660 24287 ts_tablet_manager.cc:1929] T 8217046fd4e14cc4a0194da4f6ad3887 P 0a407504ed1741bf89a276558adf4c0f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:08:12.318218 24287 log.cc:1199] T 8217046fd4e14cc4a0194da4f6ad3887 P 0a407504ed1741bf89a276558adf4c0f: Deleting WAL directory at /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-0-root/wals/8217046fd4e14cc4a0194da4f6ad3887
I20260709 13:08:12.321167 22764 catalog_manager.cc:5028] TS 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): tablet 8217046fd4e14cc4a0194da4f6ad3887 (table test-workload [id=a42f2ccfbe484baca3019323a490c64a]) successfully deleted
I20260709 13:08:12.330912 22777 catalog_manager.cc:5697] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 1 to 2, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 2 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:12.341853 24162 raft_consensus.cc:993] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f: : Instructing follower f3c799b1249e40119d2a5864231c6c75 to start an election
I20260709 13:08:12.343302 24162 raft_consensus.cc:1081] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Signalling peer f3c799b1249e40119d2a5864231c6c75 to start an election
I20260709 13:08:12.346194 23029 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
 from {username='slave'} at 127.0.0.1:33418
I20260709 13:08:12.346740 23029 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:12.347214 23029 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:12.354666 23029 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:12.356941 23029 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:12.357921 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:12.358223 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:12.358526 22876 raft_consensus.cc:3055] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:12.358852 22876 raft_consensus.cc:740] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:12.358937 22955 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:12.359676 22876 consensus_queue.cc:260] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:12.360826 22876 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:12.368264 22955 raft_consensus.cc:2468] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 2.
I20260709 13:08:12.369598 22992 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:12.370484 22876 raft_consensus.cc:2468] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 2.
I20260709 13:08:12.371510 24225 raft_consensus.cc:2804] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:12.376963 24225 raft_consensus.cc:697] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 2 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:12.377876 24225 consensus_queue.cc:237] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:12.402936 22777 catalog_manager.cc:5697] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 1 to 2, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 2 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:12.406304 24077 raft_consensus.cc:1064] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
I20260709 13:08:12.408058 24077 consensus_queue.cc:237] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:12.413229 24111 raft_consensus.cc:1064] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
I20260709 13:08:12.415565 24111 consensus_queue.cc:237] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:12.417330 24220 raft_consensus.cc:993] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7: : Instructing follower 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:08:12.417912 24075 raft_consensus.cc:1081] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Signalling peer 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:08:12.420058 22873 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:35360
I20260709 13:08:12.420626 22873 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:12.420956 22873 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:12.427613 23622 raft_consensus.cc:1275] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEARNER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:08:12.428402 23029 raft_consensus.cc:1275] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:08:12.429299 24111 consensus_queue.cc:1048] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:12.430096 22955 raft_consensus.cc:1275] T 7f35ebcf93ad423ea8bd0b777bee23ee P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 13:08:12.431264 22873 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:12.431489 24267 consensus_queue.cc:1048] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:12.434087 24162 consensus_queue.cc:1048] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:12.444952 22873 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:12.445005 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:12.446353 22954 raft_consensus.cc:3055] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:12.446715 22954 raft_consensus.cc:740] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:12.446213 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:12.447472 23029 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:12.447528 22954 consensus_queue.cc:260] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:12.449180 22954 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:12.448736 24266 raft_consensus.cc:2955] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:12.454644 23029 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:08:12.462236 22843 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:12.465184 23624 raft_consensus.cc:2955] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:12.467020 24162 raft_consensus.cc:2804] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:12.473114 24162 raft_consensus.cc:697] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:12.475667 24162 consensus_queue.cc:237] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:12.483234 23020 raft_consensus.cc:2955] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:12.487380 22777 catalog_manager.cc:5697] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: config changed from index 3 to 4, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:12.510897 22954 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:08:12.528729 23624 raft_consensus.cc:1275] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:08:12.529877 22947 raft_consensus.cc:1275] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:08:12.531517 22880 raft_consensus.cc:1275] T ec334666b7964c969125af95d9c797ed P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:08:12.531361 24099 consensus_queue.cc:1048] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:12.533380 24282 raft_consensus.cc:993] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75: : Instructing follower aed3f089ec8b49af9c8cf5cdee76e8e7 to start an election
I20260709 13:08:12.533874 24282 raft_consensus.cc:1081] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Signalling peer aed3f089ec8b49af9c8cf5cdee76e8e7 to start an election
I20260709 13:08:12.535808 24225 consensus_queue.cc:1048] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:12.536585 22954 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:58320
I20260709 13:08:12.542367 22954 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:12.542872 22954 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:12.549613 22953 raft_consensus.cc:2955] T 7f35ebcf93ad423ea8bd0b777bee23ee P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:12.552767 22954 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:12.540071 22777 catalog_manager.cc:5697] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 1 to 2, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 2 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:12.558444 24099 consensus_queue.cc:1048] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:12.559095 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:12.559736 23029 raft_consensus.cc:3055] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:12.560031 23029 raft_consensus.cc:740] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:12.560822 23029 consensus_queue.cc:260] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:12.561856 23029 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:12.563403 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:12.564424 22876 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:12.587134 22876 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:12.589728 22920 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:12.590986 24233 raft_consensus.cc:2804] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:12.591501 24233 raft_consensus.cc:697] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:12.592461 24233 consensus_queue.cc:237] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:12.594065 22876 consensus_queue.cc:237] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:12.593012 24220 raft_consensus.cc:1064] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
I20260709 13:08:12.604238 22954 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:12.604519 23625 raft_consensus.cc:1275] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:08:12.604759 23023 raft_consensus.cc:1275] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:08:12.609544 24295 consensus_queue.cc:1048] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:12.610495 24162 consensus_queue.cc:1048] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:12.606449 23029 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:12.599687 24220 consensus_queue.cc:237] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:12.618935 24295 raft_consensus.cc:2955] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:12.620805 23625 raft_consensus.cc:2955] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:12.642954 23624 raft_consensus.cc:1275] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 LEARNER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:08:12.643612 24077 raft_consensus.cc:2955] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:12.645267 23020 raft_consensus.cc:2955] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:12.646035 24220 consensus_queue.cc:1048] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:12.650076 22954 raft_consensus.cc:2955] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:12.665764 22876 raft_consensus.cc:1275] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:08:12.676788 24233 consensus_queue.cc:1048] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:12.682106 22775 catalog_manager.cc:5697] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 reported cstate change: config changed from index 4 to 5, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New cstate: current_term: 2 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:12.689836 23622 raft_consensus.cc:2955] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:12.697649 22943 tablet_service.cc:1558] Processing DeleteTablet for tablet 7f35ebcf93ad423ea8bd0b777bee23ee with delete_type TABLET_DATA_TOMBSTONED (TS aed3f089ec8b49af9c8cf5cdee76e8e7 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:58302
I20260709 13:08:12.710392 22775 catalog_manager.cc:5697] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 reported cstate change: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:12.730989 22777 catalog_manager.cc:5697] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 1 to 2, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 2 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:12.747165 23023 raft_consensus.cc:1275] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 13:08:12.756822 24220 consensus_queue.cc:1048] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:12.760361 22764 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7f35ebcf93ad423ea8bd0b777bee23ee with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:08:12.759596 24305 tablet_replica.cc:333] T 7f35ebcf93ad423ea8bd0b777bee23ee P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:08:12.768512 24305 raft_consensus.cc:2243] T 7f35ebcf93ad423ea8bd0b777bee23ee P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:12.769404 24305 raft_consensus.cc:2272] T 7f35ebcf93ad423ea8bd0b777bee23ee P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:12.770597 22873 raft_consensus.cc:2955] T ec334666b7964c969125af95d9c797ed P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:12.775332 24305 ts_tablet_manager.cc:1916] T 7f35ebcf93ad423ea8bd0b777bee23ee P aed3f089ec8b49af9c8cf5cdee76e8e7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:08:12.776563 23023 raft_consensus.cc:1275] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:12.789485 24295 consensus_queue.cc:1048] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:13.980839 24317 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:13.981581 24317 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:13.984935 24317 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:13.986017 24317 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:13.986477 24317 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:13.988330 24317 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:14.021467 23591 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.719s	user 0.000s	sys 0.012s
W20260709 13:08:14.021872 23591 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.755s	user 0.000s	sys 0.012s
W20260709 13:08:14.022521 21761 debug-util.cc:398] Leaking SignalData structure 0x7b080000f800 after lost signal to thread 22920
W20260709 13:08:14.023190 21761 kernel_stack_watchdog.cc:198] Thread 22920 stuck at /apache/dev/git/kudu/src/kudu/rpc/outbound_call.cc:307 for 230ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
I20260709 13:08:14.023267 24313 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:14.023983 24313 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:14.024619 24305 ts_tablet_manager.cc:1929] T 7f35ebcf93ad423ea8bd0b777bee23ee P aed3f089ec8b49af9c8cf5cdee76e8e7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 13:08:14.025110 24305 log.cc:1199] T 7f35ebcf93ad423ea8bd0b777bee23ee P aed3f089ec8b49af9c8cf5cdee76e8e7: Deleting WAL directory at /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-1-root/wals/7f35ebcf93ad423ea8bd0b777bee23ee
I20260709 13:08:14.026077 24313 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:14.027604 24312 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:14.028127 24305 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:58302 (request call id 79) took 1331 ms. Trace:
I20260709 13:08:14.028103 24312 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:14.029356 24305 rpcz_store.cc:276] 0709 13:08:12.696958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:12.697284 (+   326us) service_pool.cc:224] Handling call
0709 13:08:12.759176 (+ 61892us) ts_tablet_manager.cc:1151] Deleting tablet 7f35ebcf93ad423ea8bd0b777bee23ee
0709 13:08:12.759230 (+    54us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:08:12.805320 (+ 46090us) tablet_metadata.cc:665] Metadata flushed
0709 13:08:14.024460 (+1219140us) tablet_metadata.cc:665] Metadata flushed
0709 13:08:14.028014 (+  3554us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":60779,"thread_start_us":62020,"threads_started":1}
I20260709 13:08:14.033051 24312 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:14.028762 24313 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:14.034157 24313 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:14.036054 24313 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:14.046126 23023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 1206) took 1248 ms. Trace:
I20260709 13:08:14.046576 23023 rpcz_store.cc:276] 0709 13:08:12.797466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:12.797622 (+   156us) service_pool.cc:224] Handling call
0709 13:08:12.798093 (+   471us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:14.040089 (+1241996us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:14.040127 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:14.040576 (+   449us) log.cc:844] Serialized 26 byte log entry
0709 13:08:14.041725 (+  1149us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:08:14.041757 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:14.041828 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:14.045881 (+  4053us) raft_consensus.cc:1685] finished
0709 13:08:14.045898 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:14.046027 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":790,"threads_started":1,"wal-append.queue_time_us":1154}
I20260709 13:08:14.048180 22954 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 1572) took 1249 ms. Trace:
I20260709 13:08:14.048614 22954 rpcz_store.cc:276] 0709 13:08:12.798784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:14.047410 (+1248626us) service_pool.cc:224] Handling call
0709 13:08:14.047763 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:14.047917 (+   154us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:14.047940 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:14.047976 (+    36us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:08:14.048003 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:14.048047 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:14.048135 (+    88us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:14.053610 24320 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
W20260709 13:08:14.054203 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 1.26304e+06us
I20260709 13:08:14.056499 24321 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.057132 24321 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:14.054195 24320 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:14.065011 24320 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:14.069375 23588 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.823s	user 0.006s	sys 0.030s
W20260709 13:08:14.069737 23588 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.823s	user 0.006s	sys 0.030s
I20260709 13:08:14.069314 24321 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:14.081648 24313 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:14.082237 24313 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:14.084244 24313 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:14.087456 24309 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:14.087985 24309 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:14.089807 24309 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:14.091769 24309 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:14.092048 24322 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:14.092288 24309 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:14.092578 24322 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:14.094295 24309 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:14.094467 24322 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:14.056375 22762 catalog_manager.cc:5028] TS aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): tablet 7f35ebcf93ad423ea8bd0b777bee23ee (table test-workload [id=a42f2ccfbe484baca3019323a490c64a]) successfully deleted
I20260709 13:08:14.095570 24321 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:13.146565 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:14.096027 24321 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:14.096732 24313 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:14.097282 24075 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:14.098007 24321 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:14.098529 24321 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:14.097783 24075 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:14.098995 24321 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:14.100803 24322 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:14.100972 24317 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:14.101308 24322 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:14.101431 24317 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:14.103207 24322 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:14.103212 24317 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:14.097267 24313 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:14.104296 24321 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:14.106186 24313 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:14.107579 24075 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:14.109614 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:14.110323 22948 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 1.
I20260709 13:08:14.111761 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:14.112401 24315 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:14.112892 24315 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:14.113286 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:14.114300 24323 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:14.114389 24075 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:14.114889 24075 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:14.114858 24323 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:14.116815 24075 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:14.117166 24323 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:14.118911 22880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 1132) took 1435 ms. Trace:
I20260709 13:08:14.119418 23588 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:14.120311 24313 raft_consensus.cc:2804] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:14.120693 24313 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:14.121071 24313 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:14.122002 22954 raft_consensus.cc:1275] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:14.124326 24320 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:14.124956 24320 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:14.125780 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:14.127431 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:14.128144 22954 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 1.
I20260709 13:08:14.129405 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:14.130187 22954 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 1.
I20260709 13:08:14.134845 22948 raft_consensus.cc:1275] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:14.142347 24075 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:14.142932 24075 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:14.114866 24315 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:14.145615 22955 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 1573) took 1346 ms. Trace:
I20260709 13:08:14.146097 22955 rpcz_store.cc:276] 0709 13:08:12.799143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:14.144778 (+1345635us) service_pool.cc:224] Handling call
0709 13:08:14.145053 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:14.145236 (+   183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:14.145271 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:14.145324 (+    53us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:08:14.145367 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:14.145438 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:14.145554 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:14.148712 24075 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:14.149470 24320 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:14.153463 24327 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:14.119297 22880 rpcz_store.cc:276] 0709 13:08:12.683715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:12.683826 (+   111us) service_pool.cc:224] Handling call
0709 13:08:12.684286 (+   460us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:12.685358 (+  1072us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:12.685383 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:12.686195 (+   812us) log.cc:844] Serialized 548 byte log entry
0709 13:08:14.118341 (+1432146us) spinlock_profiling.cc:243] Waited 27.1 ms on lock 0x7b5400335520. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4f090 00007f6135d5c6b8 00007f6135de592b 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c
0709 13:08:14.118477 (+   136us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:08:14.118532 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:14.118599 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:14.118638 (+    39us) raft_consensus.cc:1685] finished
0709 13:08:14.118649 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:14.118835 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":62435072,"thread_start_us":62402,"threads_started":1,"wal-append.queue_time_us":58636,"wal-append.run_cpu_time_us":1653,"wal-append.run_wall_time_us":1046756}
I20260709 13:08:14.154369 24327 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:14.155956 24282 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
W20260709 13:08:14.156793 23589 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.890s	user 0.010s	sys 0.008s
I20260709 13:08:14.156477 24282 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:14.157099 23589 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.891s	user 0.010s	sys 0.009s
I20260709 13:08:14.157311 24327 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:14.157886 24327 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:14.158463 24282 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:14.158331 24327 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:14.160267 24327 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:14.153098 24313 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:14.162943 24316 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:14.163615 22775 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33488 (request call id 258) took 1460 ms. Trace:
I20260709 13:08:14.179847 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:14.164017 22775 rpcz_store.cc:276] 0709 13:08:12.703670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:12.703798 (+   128us) service_pool.cc:224] Handling call
0709 13:08:12.761665 (+ 57867us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:12.761706 (+    41us) write_op.cc:454] Released schema lock
0709 13:08:14.163529 (+1401823us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:12.737831 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:12.738333 (+   502us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:12.738362 (+    29us) write_op.cc:435] Acquired schema lock
0709 13:08:12.738381 (+    19us) tablet.cc:662] Decoding operations
0709 13:08:12.738692 (+   311us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:12.738757 (+    65us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:12.738795 (+    38us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:08:12.739025 (+   230us) tablet.cc:701] Row locks acquired
0709 13:08:12.739049 (+    24us) write_op.cc:260] PREPARE: finished
0709 13:08:12.739193 (+   144us) write_op.cc:270] Start()
0709 13:08:12.739335 (+   142us) write_op.cc:276] Timestamp: P: 1783602492739145 usec, L: 0
0709 13:08:12.739363 (+    28us) op_driver.cc:348] REPLICATION: starting
0709 13:08:12.740273 (+   910us) log.cc:844] Serialized 591 byte log entry
0709 13:08:12.742676 (+  2403us) op_driver.cc:464] REPLICATION: finished
0709 13:08:12.743145 (+   469us) write_op.cc:301] APPLY: starting
0709 13:08:12.743243 (+    98us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:12.743371 (+   128us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:12.743501 (+   130us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:12.743765 (+   264us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:12.743965 (+   200us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:08:12.744028 (+    63us) write_op.cc:312] APPLY: finished
0709 13:08:12.744243 (+   215us) log.cc:844] Serialized 23 byte log entry
0709 13:08:12.744395 (+   152us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:12.744476 (+    81us) write_op.cc:454] Released schema lock
0709 13:08:12.744526 (+    50us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":9342,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":257,"apply.run_cpu_time_us":1700,"apply.run_wall_time_us":1699,"num_ops":1,"prepare.queue_time_us":12790,"prepare.run_cpu_time_us":2743,"prepare.run_wall_time_us":2743,"replication_time_us":3172}]]}
I20260709 13:08:14.163722 24316 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:14.196637 24077 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:14.197558 24316 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:14.197352 24077 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:14.198061 24316 consensus_queue.cc:1048] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:14.176385 24312 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:14.198948 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:14.199563 24318 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:14.199704 24312 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:14.200062 24318 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:14.205112 24312 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:14.185371 24309 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:14.210151 24075 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:14.210382 24320 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:14.210671 24075 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:14.191354 24091 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:14.213203 24077 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:14.213987 24077 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:14.213430 24309 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:14.214793 24077 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:14.216243 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 53097.1us
I20260709 13:08:14.202549 24225 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:14.205822 24318 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:14.222018 24075 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:14.223996 22843 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:14.210855 24320 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:14.226454 24075 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:14.227128 24325 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:14.227191 24075 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:14.227650 24325 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:14.228183 24320 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:14.207464 24282 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:14.229856 24075 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:14.226790 24308 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:14.229700 24282 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:14.230640 24308 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:14.232282 24282 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:14.229905 24325 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:14.210193 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:14.233624 24077 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:14.237340 24325 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:14.238025 24320 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:14.238473 24075 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:14.239086 22843 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:14.213094 24323 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:14.218555 24313 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:14.241074 24075 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:14.215831 24312 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:14.222767 24322 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:14.248597 24077 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:14.238543 24317 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:14.251299 24317 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:14.251334 24323 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:14.253239 24317 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:14.253983 24323 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:14.254558 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:14.255498 24333 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:14.256058 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:14.249248 24077 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:14.242590 24318 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:14.261981 24075 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:14.256124 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:14.260437 24077 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:14.220803 24091 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:14.256004 24333 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:14.257205 24322 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:14.268034 24311 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:14.268432 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:14.234234 24225 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:14.268049 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:14.274152 22947 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 2.
I20260709 13:08:14.267436 24077 consensus_queue.cc:1048] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 13:08:14.269589 23588 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:14.275125 24321 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:14.275359 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:14.268769 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:08:14.275746 23590 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.190s	user 0.005s	sys 0.004s
I20260709 13:08:14.268781 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:14.273226 24330 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:14.263310 24308 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:14.263582 24318 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:14.243680 24320 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:14.247133 24325 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:14.276961 23590 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.191s	user 0.006s	sys 0.004s
I20260709 13:08:14.271409 24332 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:14.278874 24077 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:14.249325 24312 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:14.282050 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:14.269750 23028 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 2.
I20260709 13:08:14.275727 24322 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:14.280654 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:14.284556 23026 consensus_queue.cc:237] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:14.279130 24313 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:14.293188 24314 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:14.275692 24321 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:14.293790 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:14.712846 24336 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:14.723728 24225 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:14.279793 24311 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:14.738003 24311 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:14.740547 24333 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:14.742292 24333 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:14.742928 24333 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:14.745503 24309 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:14.718392 24336 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:14.749570 24336 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:14.686590 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:14.690336 24338 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.761201 24338 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:14.417912 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:14.275156 24334 raft_consensus.cc:2804] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:14.763741 24334 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:14.765221 24334 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:14.692101 24075 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:14.769389 24075 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:14.772991 24075 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:14.291934 24330 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:14.787022 24333 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:14.617436 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 347884us
I20260709 13:08:14.802886 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:14.803787 22947 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 1.
W20260709 13:08:14.702962 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 456154us
I20260709 13:08:14.807287 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:14.807909 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:14.807991 24342 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:14.808914 24342 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:14.809393 24091 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:14.810333 22949 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:08:14.810922 24342 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:14.811268 24321 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:14.811432 24342 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:14.811895 24342 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:14.296883 24332 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:14.814848 24332 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:14.764410 24338 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:14.305796 24077 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:14.278157 22948 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:08:14.284835 24318 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:14.858369 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:14.806452 24330 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:14.305641 24314 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:14.827708 22809 debug-util.cc:398] Leaking SignalData structure 0x7b080008aec0 after lost signal to thread 21759
W20260709 13:08:14.857650 23591 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.005s	sys 0.015s
I20260709 13:08:14.869598 24077 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:14.881871 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:08:14.916838 23590 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.002s	sys 0.011s
I20260709 13:08:14.917650 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:08:14.917325 23590 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.002s	sys 0.011s
I20260709 13:08:14.918273 22951 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 1.
W20260709 13:08:14.920593 22809 debug-util.cc:398] Leaking SignalData structure 0x7b08001a84e0 after lost signal to thread 22812
I20260709 13:08:14.897675 23588 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:08:14.856305 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.290426 24315 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:14.537300 24275 raft_consensus.cc:2749] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:08:14.924484 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:14.935947 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:14.537575 24322 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:14.937819 24322 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:14.940016 24342 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:14.663244 24328 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:14.940768 24339 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:14.663605 24335 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
W20260709 13:08:14.914314 23591 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.006s	sys 0.016s
I20260709 13:08:14.921756 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:14.945091 23029 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 1180) took 2168 ms. Trace:
I20260709 13:08:14.952391 23029 rpcz_store.cc:276] 0709 13:08:12.769337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:12.769696 (+   359us) service_pool.cc:224] Handling call
0709 13:08:12.770487 (+   791us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:12.771036 (+   549us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:12.771059 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:12.772183 (+  1124us) log.cc:844] Serialized 548 byte log entry
0709 13:08:12.784269 (+ 12086us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:08:12.784304 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:12.784366 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:14.937017 (+2152651us) raft_consensus.cc:1685] finished
0709 13:08:14.937048 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:14.937237 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":11769,"threads_started":1,"wal-append.queue_time_us":988811,"wal-append.run_cpu_time_us":1979,"wal-append.run_wall_time_us":1020093}
I20260709 13:08:14.900621 24333 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:14.977744 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:14.940549 24342 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:14.859251 24313 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:14.978397 23026 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 4.
W20260709 13:08:14.979880 22993 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.717s	user 0.008s	sys 0.030s
I20260709 13:08:14.941146 24328 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:15.002218 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:15.007215 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:15.007766 22947 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
W20260709 13:08:15.001149 22993 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.738s	user 0.009s	sys 0.030s
W20260709 13:08:15.017946 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 120353us
I20260709 13:08:14.941545 24339 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:15.023865 21761 debug-util.cc:398] Leaking SignalData structure 0x7b080000fc20 after lost signal to thread 22920
I20260709 13:08:15.024037 24339 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:14.979108 24325 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:14.950482 24315 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:14.987458 24312 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:14.979964 24335 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:15.036914 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:14.986250 24340 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:15.029184 24311 raft_consensus.cc:2804] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:15.029348 24322 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:15.006271 24342 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:15.013989 24344 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:14.979736 24333 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:15.051817 23588 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:15.052179 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:15.074710 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:15.064077 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:08:15.161302 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 109541us
I20260709 13:08:15.163475 24343 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:15.164245 24315 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:15.162537 23588 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:15.089324 24335 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:15.049335 24330 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:15.164137 24343 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:15.052052 24311 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:15.188001 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:15.058630 24338 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:15.089771 24347 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:15.191037 22946 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 2.
I20260709 13:08:15.193751 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:15.189142 24330 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:15.197921 24313 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:15.189175 24346 log.cc:927] Time spent T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 13:08:15.089974 24341 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:15.089582 24348 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:15.189766 24332 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:15.191610 24311 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:15.203102 24345 raft_consensus.cc:2804] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:15.050025 24340 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:15.186273 24344 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:15.217284 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:15.202546 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:15.203549 24345 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:15.202960 24341 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:15.218827 22992 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:15.211397 24348 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:15.223326 22952 raft_consensus.cc:1275] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:08:15.223735 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:15.195561 24338 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:15.230507 22992 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:15.209940 24328 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:15.235074 22843 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:15.209594 24332 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:15.225518 24345 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:15.232740 24336 raft_consensus.cc:2804] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:15.243333 24312 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:15.243798 24313 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:15.244686 24348 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:15.239639 24344 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:15.249620 24352 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:15.255254 24312 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:15.255738 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:15.255970 24313 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:15.256687 24353 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:15.240226 24328 raft_consensus.cc:2804] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:15.238019 24314 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:15.246618 24336 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:15.201973 24347 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:15.268128 24336 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:15.274173 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:15.264511 24355 raft_consensus.cc:2804] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:15.276414 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:15.264755 24328 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:15.267922 24352 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:15.267745 24314 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:15.269562 24353 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:15.285272 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:15.281432 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:15.287933 22992 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:15.269976 24344 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:15.282493 24313 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:15.288641 22774 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48906 (request call id 190) took 2565 ms. Trace:
I20260709 13:08:15.318531 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:15.320565 23025 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 1.
I20260709 13:08:15.319705 24352 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:15.278545 24342 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:15.283214 24333 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:15.320310 22774 rpcz_store.cc:276] 0709 13:08:12.719642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:12.719990 (+   348us) service_pool.cc:224] Handling call
0709 13:08:15.284753 (+2564763us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1330211,"reported_tablets":1}
I20260709 13:08:15.281526 24358 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:15.344455 24360 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:15.345890 24360 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:15.346052 24358 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:15.278141 24355 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:15.349210 24355 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:15.349306 24360 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:15.350198 24360 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:15.350961 24360 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:15.282336 24338 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:15.353330 24338 raft_consensus.cc:2804] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:15.353781 24360 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:15.353910 24338 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:15.354753 24338 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:15.289345 24312 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:15.289131 24353 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:15.317476 24343 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:15.366479 24338 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:15.367139 24358 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:15.321830 24332 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:15.290165 24351 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:15.368129 24350 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.317659 24314 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:15.368542 24357 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:15.323342 24348 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:15.267027 24359 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:15.281759 24315 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:15.288833 24328 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:15.325350 24342 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.299475 24345 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:15.381345 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 146332us
I20260709 13:08:15.282954 24318 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:15.385004 23625 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 1.
I20260709 13:08:15.323553 24344 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:15.386665 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 64024.8us
I20260709 13:08:15.390888 23622 raft_consensus.cc:2393] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 2: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:15.385322 24315 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:15.382666 24351 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:15.394235 22876 raft_consensus.cc:1240] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:15.395357 22954 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 1579) took 1149 ms. Trace:
I20260709 13:08:15.395718 22954 rpcz_store.cc:276] 0709 13:08:14.246171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:14.247374 (+  1203us) service_pool.cc:224] Handling call
0709 13:08:14.247740 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:14.292397 (+ 44657us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:14.292426 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:14.292464 (+    38us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:08:14.292495 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:15.395150 (+1102655us) spinlock_profiling.cc:243] Waited 44.5 ms on lock 0x7b54000a96d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:15.395189 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:15.395286 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":102302464}
I20260709 13:08:15.396237 24351 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:15.400842 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:15.401028 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.401706 22874 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 2.
I20260709 13:08:15.403345 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.404806 22955 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 1578) took 1159 ms. Trace:
I20260709 13:08:15.404892 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:15.405296 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.405263 22955 rpcz_store.cc:276] 0709 13:08:14.245117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:14.245324 (+   207us) service_pool.cc:224] Handling call
0709 13:08:14.245676 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:14.294647 (+ 48971us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:14.294688 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:14.294740 (+    52us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:14.294781 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:15.404586 (+1109805us) spinlock_profiling.cc:243] Waited 48.8 ms on lock 0x7b5400411754. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:15.404624 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:15.404752 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":112180480}
I20260709 13:08:15.410784 22843 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:15.411660 24351 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:15.413846 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:15.416181 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:15.416368 24315 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:15.385565 24357 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:15.412842 22843 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:15.382402 24348 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:15.429221 22843 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:15.434583 24344 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:15.434587 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.435379 24341 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:15.436319 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.430991 24338 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:15.436964 22878 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:08:15.437638 24314 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:15.387130 24332 consensus_queue.cc:1048] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 13:08:15.438722 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:15.385571 24342 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:15.428467 24334 consensus_meta.cc:287] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.662s	user 0.012s	sys 0.000s
I20260709 13:08:15.428336 24315 raft_consensus.cc:2749] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:15.447383 24347 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:15.447515 24318 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:15.383596 24359 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:15.432103 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:15.444993 24356 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:15.446902 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:15.417584 24325 raft_consensus.cc:2955] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:15.447182 24357 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:15.455129 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:15.383664 24350 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:15.461866 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:15.459272 22843 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:15.461760 24356 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:15.464025 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:15.459328 24318 raft_consensus.cc:2749] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:15.464931 24357 raft_consensus.cc:2749] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:15.445593 24334 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:15.468912 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:15.463565 24342 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:15.466924 24356 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:15.483186 22872 raft_consensus.cc:1240] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:15.483834 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:15.461241 24336 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:15.485370 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:15.487746 24345 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:15.487735 24357 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:15.489605 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:15.490764 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:15.474642 24348 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:15.490149 24357 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:15.493335 22879 raft_consensus.cc:1275] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:15.493448 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:15.486894 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:15.493448 24315 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:15.494473 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:15.497833 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:15.479475 22949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58320 (request call id 1582) took 1201 ms. Trace:
I20260709 13:08:15.492504 24359 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:15.490087 24353 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:15.501855 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:15.502199 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:15.502298 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:15.502918 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:15.505811 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:15.503285 24318 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:15.500806 24315 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:15.506932 24318 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:15.508460 24318 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:15.508728 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.505896 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.510135 22874 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:08:15.499670 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:15.513454 24357 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:15.508414 24315 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:15.492324 23618 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 1.
I20260709 13:08:15.515573 24369 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:15.499879 22949 rpcz_store.cc:276] 0709 13:08:14.274238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:14.278237 (+  3999us) service_pool.cc:224] Handling call
0709 13:08:15.467740 (+1189503us) spinlock_profiling.cc:243] Waited 520 ms on lock 0x7b54000eced4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:15.472107 (+  4367us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1195287296}
I20260709 13:08:15.517720 22992 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:15.519227 24347 raft_consensus.cc:2804] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:15.519587 24347 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:15.519922 24347 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:15.520231 23619 raft_consensus.cc:2393] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 2: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:15.503679 23617 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:15.523942 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:15.524364 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:15.525871 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.526151 24330 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:15.528654 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:15.521256 24359 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:15.525022 24334 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:15.535835 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.532545 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:15.540485 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:15.541337 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:15.542699 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:15.542840 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.543447 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:15.544010 23618 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 1.
I20260709 13:08:15.543912 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:15.544646 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:15.544751 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:15.546708 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:15.546914 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:15.533434 22874 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:15.547320 23618 raft_consensus.cc:2468] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 1.
I20260709 13:08:15.536751 24359 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:15.548825 22875 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 2.
I20260709 13:08:15.549316 23021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 140) took 1294 ms. Trace:
I20260709 13:08:15.553869 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:15.554210 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:15.556391 23617 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
I20260709 13:08:15.556830 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.521413 24369 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:15.554883 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:15.533656 24355 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:15.553628 23021 rpcz_store.cc:276] 0709 13:08:14.254892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:14.870305 (+615413us) service_pool.cc:224] Handling call
0709 13:08:14.919289 (+ 48984us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:15.079604 (+160315us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:15.090890 (+ 11286us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:15.122751 (+ 31861us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:15.132650 (+  9899us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:15.549077 (+416427us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b5400359154. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:15.549119 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:15.549236 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":319578880}
I20260709 13:08:15.562198 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:15.565801 22953 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:15.566717 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.567392 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:15.570153 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:15.573421 23616 raft_consensus.cc:1240] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:15.574860 22953 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:08:15.568382 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:15.577272 24336 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:15.540778 24311 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:15.579288 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.537058 24350 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:15.579936 22880 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 2.
I20260709 13:08:15.582216 23020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 1214) took 1345 ms. Trace:
I20260709 13:08:15.562229 23022 raft_consensus.cc:1275] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:15.582145 23023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 129) took 1361 ms. Trace:
I20260709 13:08:15.584933 23621 raft_consensus.cc:1240] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:15.585050 23023 rpcz_store.cc:276] 0709 13:08:14.221225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:14.222268 (+  1043us) service_pool.cc:224] Handling call
0709 13:08:14.222609 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:14.777060 (+554451us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:14.777101 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:14.777167 (+    66us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:14.777210 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:15.581918 (+804708us) spinlock_profiling.cc:243] Waited 554 ms on lock 0x7b54001c84d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:15.581961 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:15.582080 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1274534784}
I20260709 13:08:15.583328 23020 rpcz_store.cc:276] 0709 13:08:14.237126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:14.237313 (+   187us) service_pool.cc:224] Handling call
0709 13:08:14.237589 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:14.285387 (+ 47798us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:14.285439 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:14.285500 (+    61us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:14.285547 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:15.582004 (+1296457us) spinlock_profiling.cc:243] Waited 47.6 ms on lock 0x7b54002c9554. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:15.582054 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:15.582167 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":109365504}
I20260709 13:08:15.558604 22877 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:08:15.587303 22949 raft_consensus.cc:1275] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:15.587918 24355 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:15.580626 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.590231 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.589761 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:15.590551 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:15.590917 22872 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:08:15.588919 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:08:15.592324 22809 debug-util.cc:398] Leaking SignalData structure 0x7b080027efc0 after lost signal to thread 22904
W20260709 13:08:15.592831 22809 debug-util.cc:398] Leaking SignalData structure 0x7b0800230760 after lost signal to thread 22905
I20260709 13:08:15.579632 23618 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 1.
W20260709 13:08:15.593834 22809 debug-util.cc:398] Leaking SignalData structure 0x7b080024b660 after lost signal to thread 22979
I20260709 13:08:15.594288 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:15.594234 24328 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:15.592291 24359 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:15.597563 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:15.585220 24353 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:15.598717 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:15.599277 23028 raft_consensus.cc:1240] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:15.599838 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:15.600181 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:15.600421 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:15.601059 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:15.609074 22955 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:08:15.594847 22809 debug-util.cc:398] Leaking SignalData structure 0x7b08003179a0 after lost signal to thread 23053
I20260709 13:08:15.594901 23618 raft_consensus.cc:2468] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 1.
I20260709 13:08:15.601683 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:15.629750 22776 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33466 (request call id 278) took 1626 ms. Trace:
I20260709 13:08:15.603710 24328 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:15.607131 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:15.601187 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:15.611194 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:15.706426 23622 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 1.
I20260709 13:08:15.611378 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:15.582340 24311 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:15.613154 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:15.611431 24340 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:15.620304 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:15.630306 22776 rpcz_store.cc:276] 0709 13:08:14.003650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:14.003781 (+   131us) service_pool.cc:224] Handling call
0709 13:08:15.629666 (+1625885us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":158563,"reported_tablets":1,"spinlock_wait_cycles":1792}
W20260709 13:08:15.642606 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:15.711740 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:08:15.665583 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 53376.6us
I20260709 13:08:15.712447 23618 raft_consensus.cc:2468] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 1.
I20260709 13:08:15.713196 24077 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:15.713663 24077 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:15.713980 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.610478 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.715575 24077 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:15.590996 23616 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
W20260709 13:08:15.727995 22809 debug-util.cc:398] Leaking SignalData structure 0x7b080034c160 after lost signal to thread 23649
W20260709 13:08:15.728482 22809 debug-util.cc:398] Leaking SignalData structure 0x7b080034c300 after lost signal to thread 24220
I20260709 13:08:15.730587 24336 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:15.731104 24336 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:15.724805 24306 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:08:15.732627 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:15.757676 24077 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:15.770694 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:15.770622 24077 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:15.777168 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.777501 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:15.734576 24308 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:15.730023 24347 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:15.781234 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:15.781960 23020 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 1.
I20260709 13:08:15.733124 24336 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:08:15.783584 22920 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.63.1:33929 timed out after 1.624s (SENT)
I20260709 13:08:15.711800 24369 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:15.783128 24308 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:15.789346 24308 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:15.787227 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.793756 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:15.736269 24306 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:15.803285 24077 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:15.804606 24328 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:15.805394 24328 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:15.805720 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:15.807572 24328 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:15.809123 24336 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:15.809873 24336 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:15.812191 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:15.813810 24332 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:15.815198 24332 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:15.816102 22955 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:15.821172 24332 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:15.830652 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.812644 24336 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:15.816056 24328 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:15.833716 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:15.833459 24328 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:15.835865 24328 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:15.821110 24306 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:15.836843 22948 raft_consensus.cc:1240] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:15.837366 24347 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:15.837672 24320 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:15.841643 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.842073 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.843209 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.843811 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.844723 22879 raft_consensus.cc:2393] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 2: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:15.845677 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:15.845082 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.846195 22871 raft_consensus.cc:1275] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:15.845292 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.849221 23617 raft_consensus.cc:1240] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:15.849905 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.851377 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.853991 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.855005 23617 raft_consensus.cc:1240] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:15.855373 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.860415 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:15.861253 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.862703 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:08:15.863792 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 80258.4us
I20260709 13:08:15.864240 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.865661 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.866528 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:15.867357 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.867906 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:15.869092 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.869900 22877 raft_consensus.cc:2393] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 3: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:15.871613 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.871886 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.874145 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:15.873656 22871 raft_consensus.cc:2955] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:15.876104 23617 raft_consensus.cc:1275] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:08:15.876379 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.879195 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.880537 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:15.881358 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.886370 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:15.887948 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.888173 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:15.889600 24347 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:15.889639 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.890336 24347 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:15.892462 24347 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:15.894013 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.895591 22874 raft_consensus.cc:1240] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:15.896732 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.897075 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:15.898406 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.899436 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:15.900013 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.900734 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:15.901659 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.903541 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.903813 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.904593 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:15.906935 24363 consensus_queue.cc:1059] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:15.908062 24363 raft_consensus.cc:3055] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:15.908414 24363 raft_consensus.cc:740] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:15.909369 24363 consensus_queue.cc:260] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:15.910717 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:15.911019 24363 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:15.911623 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:15.912047 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:15.912849 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:15.913939 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:15.914288 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:15.915397 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:15.915938 22955 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 2.
I20260709 13:08:15.916754 22993 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:15.917683 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:15.917852 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:15.918174 22993 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:15.918932 24336 raft_consensus.cc:2749] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:15.918929 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:15.919638 24336 raft_consensus.cc:2749] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:15.919989 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:15.920379 24336 consensus_queue.cc:1048] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:15.921105 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:15.921290 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:15.923164 22993 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:15.923919 24328 raft_consensus.cc:2749] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:15.924842 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:15.925180 22993 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:15.925552 23029 raft_consensus.cc:2393] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 3: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:15.925958 24328 raft_consensus.cc:2749] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:15.926892 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:15.927439 22947 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 2.
I20260709 13:08:15.929574 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:15.930869 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:15.935050 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:15.936055 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:15.936932 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:15.945735 22992 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:15.946044 24328 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:15.937534 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:15.931128 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:15.946540 24328 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:15.938236 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:15.947762 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:15.947943 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.938301 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:15.937489 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.938740 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:15.939208 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:15.948635 24328 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:15.948791 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.951783 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:15.941438 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.949533 24347 raft_consensus.cc:2955] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:15.941731 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.942440 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:15.942761 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.946555 24336 raft_consensus.cc:2749] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:15.943187 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:15.943351 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:15.943888 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.944437 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:15.944710 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:15.937983 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:15.964061 23622 raft_consensus.cc:2393] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 2: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:15.939270 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:08:15.966514 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 103311us
I20260709 13:08:15.966760 23590 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:15.953262 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:08:15.956820 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 92129.1us
I20260709 13:08:15.972141 22919 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:08:15.957779 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 49449.1us
I20260709 13:08:15.959467 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:15.968092 23590 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:15.968746 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:15.969345 24091 raft_consensus.cc:2749] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:15.949002 22992 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:15.977432 24311 raft_consensus.cc:2749] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:15.978320 22992 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:15.978626 22918 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:15.978399 24363 raft_consensus.cc:2804] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:15.979888 22992 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:15.980069 22918 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:15.980882 24077 raft_consensus.cc:2749] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:15.980387 24364 consensus_queue.cc:1059] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.009s
I20260709 13:08:15.980804 24363 raft_consensus.cc:697] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:15.989490 24332 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:15.989920 24332 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:15.990293 24332 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:15.991694 23590 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:15.990725 24308 raft_consensus.cc:2749] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:15.993606 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:15.993746 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:15.990245 24363 consensus_queue.cc:237] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:15.995687 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:16.000766 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:16.003086 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:16.005704 24364 raft_consensus.cc:3055] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:16.006145 24364 raft_consensus.cc:740] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:16.006256 23590 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:16.007403 24314 raft_consensus.cc:2749] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:16.007081 24364 consensus_queue.cc:260] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:16.007973 24311 raft_consensus.cc:2749] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:16.006441 24323 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:16.008883 24323 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:16.009300 24323 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:15.991694 22843 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:16.013532 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:16.017391 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:16.018536 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:16.025424 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:16.028175 23590 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:16.032994 23590 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:16.008428 24364 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:16.035015 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:16.036594 23590 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:16.040388 23590 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:16.038426 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:16.039565 23023 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:16.042312 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:16.043133 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:08:16.047593 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 63264.5us
I20260709 13:08:16.048956 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:16.049105 23029 raft_consensus.cc:1240] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:16.048419 22920 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:15.997184 24322 raft_consensus.cc:2749] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:16.049813 23026 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
W20260709 13:08:16.049043 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 76953.5us
I20260709 13:08:16.048595 24314 raft_consensus.cc:2749] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:16.055346 24347 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:33406 (request call id 76) took 1838 ms. Trace:
I20260709 13:08:16.048631 24311 consensus_queue.cc:1059] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:08:16.056540 24311 raft_consensus.cc:3055] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:16.056960 24311 raft_consensus.cc:740] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:16.056818 23026 raft_consensus.cc:2955] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:16.057991 24311 consensus_queue.cc:260] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:16.055877 24347 rpcz_store.cc:276] 0709 13:08:14.217315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:14.217465 (+   150us) service_pool.cc:224] Handling call
0709 13:08:14.338524 (+121059us) log.cc:844] Serialized 487 byte log entry
0709 13:08:16.055255 (+1716731us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":764,"raft.queue_time_us":1251,"raft.run_cpu_time_us":1282,"raft.run_wall_time_us":26325,"thread_start_us":493206,"threads_started":1,"wal-append.queue_time_us":675716}
I20260709 13:08:16.059350 24311 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:16.060957 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:16.051789 24340 raft_consensus.cc:2749] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:16.060456 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:16.064697 22762 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ec334666b7964c969125af95d9c797ed with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:08:16.061030 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:16.067349 23023 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:16.069356 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:16.069629 22920 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:16.070681 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 87726.2us
I20260709 13:08:16.033934 24365 raft_consensus.cc:2804] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:16.037779 24358 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:16.074203 22919 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:16.078584 22920 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:16.078876 22844 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:16.076767 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:16.085218 22994 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:08:16.086033 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 44172.3us
I20260709 13:08:16.086517 24328 raft_consensus.cc:2804] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:15.994796 22952 raft_consensus.cc:2955] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:16.087244 24328 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:16.087916 24328 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:16.088826 24365 raft_consensus.cc:697] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:16.090075 24365 consensus_queue.cc:237] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:16.094973 22844 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:16.084862 24317 consensus_queue.cc:1048] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:08:16.042541 24336 consensus_queue.cc:1059] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.038s
I20260709 13:08:16.093730 22994 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:16.097859 22920 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:16.098636 24320 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:16.099743 22920 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:16.100715 24324 raft_consensus.cc:2749] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:16.101477 24381 raft_consensus.cc:2749] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:16.072829 24322 raft_consensus.cc:2749] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:16.103062 22994 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:16.077523 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:16.106400 22918 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:16.106753 24321 consensus_queue.cc:1048] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:16.081909 24333 raft_consensus.cc:2749] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:16.115547 22775 catalog_manager.cc:5697] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 2 to 3, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 3 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:16.110162 22919 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:16.115909 24077 raft_consensus.cc:2749] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:16.116158 22994 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:16.082091 24306 raft_consensus.cc:2749] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:16.120950 22920 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:16.121536 22844 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:16.129180 24077 raft_consensus.cc:2804] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:16.129573 24077 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:16.129900 24077 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:16.102417 24315 raft_consensus.cc:2749] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:16.128291 24330 raft_consensus.cc:2804] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:16.137342 24330 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:16.137151 24322 consensus_queue.cc:1048] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:16.137730 24330 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:16.135429 22994 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:16.103051 24360 raft_consensus.cc:2749] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:16.141750 22994 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:16.113270 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:16.113925 24381 raft_consensus.cc:2804] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:16.113101 22874 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 2.
I20260709 13:08:16.148994 22844 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:16.149307 22920 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:16.149861 22844 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:16.146291 22843 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:16.152482 24317 raft_consensus.cc:2749] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:16.166841 22920 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:16.145008 24308 raft_consensus.cc:2749] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:16.144131 24306 raft_consensus.cc:2804] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:16.147440 24363 raft_consensus.cc:2804] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:16.141868 24379 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:16.120026 24323 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:16.152758 24342 raft_consensus.cc:2804] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:16.131757 24359 consensus_queue.cc:1059] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:08:16.147135 24381 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:16.184801 24355 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:16.172075 24324 raft_consensus.cc:2749] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:16.171068 24306 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:16.186614 24306 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:16.167560 24380 raft_consensus.cc:2749] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:16.171254 24077 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:16.170591 22920 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:16.170794 24317 raft_consensus.cc:2804] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:16.175151 24371 raft_consensus.cc:3055] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:16.175709 24315 consensus_queue.cc:1059] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:16.170207 22844 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:16.178476 22843 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:16.152787 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:16.161752 24336 raft_consensus.cc:2804] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:16.181917 24342 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:16.171484 24363 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:16.210000 24363 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:16.211689 24330 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:16.212952 24359 raft_consensus.cc:2749] T ec334666b7964c969125af95d9c797ed P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:16.213289 22871 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
mode: GRACEFUL
new_leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:51434
I20260709 13:08:16.213675 24359 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:16.213883 22871 raft_consensus.cc:606] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Received request to transfer leadership to TS aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:16.214080 24359 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:16.214520 24359 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:08:16.198257 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 50423.6us
I20260709 13:08:16.204254 24336 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:16.187740 24381 raft_consensus.cc:3060] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:16.198272 24077 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:16.195819 24317 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:08:16.189793 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 79710.1us
I20260709 13:08:16.231280 22919 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:16.206413 22843 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:16.206606 22844 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
W20260709 13:08:16.233734 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 76957.2us
I20260709 13:08:16.208953 24332 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:16.218966 23020 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:08:16.199069 22920 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:16.234856 22843 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:16.196029 24371 raft_consensus.cc:740] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:16.228027 22994 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:08:16.237051 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 117476us
I20260709 13:08:16.237558 22994 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:16.240111 24077 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:16.204617 24321 raft_consensus.cc:2749] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:16.240471 24077 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:16.240993 24077 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:16.242103 22843 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:16.243173 22843 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:16.215633 24342 raft_consensus.cc:3060] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:16.245281 24315 raft_consensus.cc:2749] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:16.246191 22919 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:16.228485 24336 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:16.235193 24317 raft_consensus.cc:3060] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:16.243825 24316 raft_consensus.cc:2804] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:16.235737 24320 raft_consensus.cc:2749] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:16.259599 23588 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:16.272169 23588 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:16.242389 24350 raft_consensus.cc:2749] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:16.252996 24321 raft_consensus.cc:2749] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:16.265671 24371 consensus_queue.cc:260] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:16.266798 23619 raft_consensus.cc:2955] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:16.245800 24369 raft_consensus.cc:2749] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:16.235783 24387 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:16.266041 24385 raft_consensus.cc:2749] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:16.274848 24314 raft_consensus.cc:2749] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:16.275451 24358 raft_consensus.cc:2749] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:16.261791 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:16.276281 24350 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:16.241945 24365 consensus_queue.cc:1059] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 13:08:16.286901 23588 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:16.283375 24077 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:16.271291 24383 log.cc:927] Time spent T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
I20260709 13:08:16.325205 24077 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:16.278940 24389 raft_consensus.cc:2804] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:16.297276 24318 raft_consensus.cc:3055] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:16.326646 24318 raft_consensus.cc:740] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:16.315492 24306 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:16.269454 24316 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:16.270474 23024 raft_consensus.cc:1240] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:16.304741 24371 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:16.246452 24325 consensus_queue.cc:1059] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:08:16.273124 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:16.326704 24389 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:16.285634 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:16.329317 23588 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:16.338912 24318 consensus_queue.cc:260] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:16.339555 24306 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:16.326681 24323 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:16.337389 24316 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:16.321127 24314 raft_consensus.cc:2749] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:16.370151 23588 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:16.367074 24389 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:16.378213 24317 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:16.380681 23588 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:16.376360 24275 raft_consensus.cc:2804] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:16.376598 24330 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:16.383592 23588 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:16.384141 24333 raft_consensus.cc:2749] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:16.376109 24336 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:16.392920 23588 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:16.392484 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 154987us
I20260709 13:08:16.385795 24318 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:08:16.397461 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:16.399472 22994 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:16.401579 24330 raft_consensus.cc:2749] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:16.402916 22871 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a424231ef7b4470bbd575564909a365e"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
mode: GRACEFUL
new_leader_uuid: "5b346767a89745fa82928a5dac9cc0b2"
 from {username='slave'} at 127.0.0.1:51434
I20260709 13:08:16.404289 24330 consensus_queue.cc:1048] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:16.386369 24275 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:16.395521 24091 raft_consensus.cc:2749] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:16.410893 24311 raft_consensus.cc:2749] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:16.412351 24363 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:16.414675 24363 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:16.414374 24359 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:16.390754 24332 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:16.410955 24275 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:08:16.430857 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 187741us
I20260709 13:08:16.389919 24328 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:16.408074 22994 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:16.440119 23020 raft_consensus.cc:1240] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:16.443094 22994 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:16.444257 22994 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:16.446074 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:08:16.446077 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 47502.7us
I20260709 13:08:16.447225 23588 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:16.448096 24077 raft_consensus.cc:2749] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:16.432782 24360 raft_consensus.cc:3055] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:16.448506 22994 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:16.436621 24317 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:16.451284 23588 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:16.443533 24379 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:16.455133 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:16.453967 22994 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:16.444278 24332 raft_consensus.cc:2749] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:08:16.456409 22994 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:16.446717 24363 raft_consensus.cc:2749] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:08:16.456997 24077 raft_consensus.cc:2749] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:16.458887 22994 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:16.449080 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:16.453090 24333 raft_consensus.cc:2749] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:16.462056 22994 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:16.450028 24360 raft_consensus.cc:740] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:16.368376 23617 raft_consensus.cc:2955] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:16.456343 24391 raft_consensus.cc:2749] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:16.474187 22994 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:16.474817 24363 raft_consensus.cc:2749] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:16.479812 24363 raft_consensus.cc:2749] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:16.476528 24330 raft_consensus.cc:2749] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:16.477133 24328 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:16.481315 22994 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:16.482383 22994 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:16.487087 24330 raft_consensus.cc:2749] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:16.489684 24355 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:16.490057 24328 raft_consensus.cc:2749] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:16.490219 24355 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:16.491156 22994 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:16.491835 24328 raft_consensus.cc:2804] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:16.492177 24328 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:16.492471 24328 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:16.498605 22843 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:16.494155 24360 consensus_queue.cc:260] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:16.474825 24332 raft_consensus.cc:2749] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:16.499630 24360 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:16.499845 24315 raft_consensus.cc:2749] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:16.502060 22843 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:16.503480 22918 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:16.508554 23026 raft_consensus.cc:1240] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:16.508991 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:16.511466 23020 raft_consensus.cc:1240] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:16.511617 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:16.513458 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:16.513607 22843 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:16.515090 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:16.515419 22843 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:16.516449 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:16.517982 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:16.518687 22918 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:16.486155 24359 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:16.522915 22918 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:16.525161 24396 raft_consensus.cc:2749] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:16.525501 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:16.526072 23025 raft_consensus.cc:1240] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:16.489254 24336 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:16.528276 24306 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:16.528563 23021 raft_consensus.cc:1240] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:16.529472 24379 raft_consensus.cc:2804] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:16.529942 24379 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:16.530352 24379 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:16.533023 24306 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:16.533478 24306 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:16.533615 22776 catalog_manager.cc:5697] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 1 to 2, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 2 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:16.535297 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:16.536798 24397 raft_consensus.cc:2749] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:08:16.494971 24387 raft_consensus.cc:2749] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:16.521169 23023 raft_consensus.cc:1240] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:16.533528 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:16.538414 22843 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:16.544958 24366 raft_consensus.cc:2804] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:16.545423 24366 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:16.545782 24366 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:16.543046 22918 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:16.525403 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:16.531163 24381 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:16.544517 22843 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:16.519791 24334 raft_consensus.cc:2749] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:16.544472 22994 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:16.571835 22843 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:16.572872 24359 raft_consensus.cc:2749] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:16.550369 24350 raft_consensus.cc:2749] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:16.547207 24315 raft_consensus.cc:2749] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:16.575424 22843 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:16.521351 24317 consensus_queue.cc:1059] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.050s
I20260709 13:08:16.529793 24389 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:16.564455 24320 raft_consensus.cc:2749] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:16.575032 24332 raft_consensus.cc:2749] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:08:16.595896 22918 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:16.605568 24323 consensus_queue.cc:1059] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:08:16.605768 24316 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:16.584414 24395 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:08:16.596184 22952 raft_consensus.cc:1240] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:16.593930 24334 raft_consensus.cc:2749] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:16.609442 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:16.609179 24325 raft_consensus.cc:3055] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:16.611420 24320 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:16.626354 24322 raft_consensus.cc:3055] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:16.626489 24328 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:16.627319 24342 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:16.628589 24325 raft_consensus.cc:740] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:16.631533 24322 raft_consensus.cc:740] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:16.610594 24387 consensus_queue.cc:1059] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
W20260709 13:08:16.636574 24401 log.cc:927] Time spent T 00000000000000000000000000000000 P 8d175b4ae72f4fc48971c2f31668971c: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
I20260709 13:08:16.633108 24379 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:16.638823 24389 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:16.642421 22774 catalog_manager.cc:5697] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: config changed from index 3 to 4, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New cstate: current_term: 1 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:16.657203 23020 raft_consensus.cc:1240] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:08:16.624959 24275 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:16.628073 24306 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:16.722898 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 271655us
I20260709 13:08:16.659021 24381 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:16.723687 23588 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:16.724488 24091 raft_consensus.cc:2749] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:16.722254 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:16.663111 24317 consensus_queue.cc:1243] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer f3c799b1249e40119d2a5864231c6c75 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
W20260709 13:08:16.746973 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:16.727453 24322 consensus_queue.cc:260] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:16.765071 24325 consensus_queue.cc:260] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:16.751473 24355 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:16.941946 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 366599us
I20260709 13:08:16.955467 22843 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:16.956295 24306 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:16.956600 24342 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:16.957571 24342 raft_consensus.cc:2749] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:16.957996 24360 consensus_queue.cc:1059] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:16.959290 24360 raft_consensus.cc:3055] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:16.959677 24360 raft_consensus.cc:740] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:16.960487 24360 consensus_queue.cc:260] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:16.961695 24360 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:16.961901 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:16.967926 24364 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:16.968518 24364 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:16.970662 24364 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:16.970839 22843 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:16.971832 24365 raft_consensus.cc:2749] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:08:16.971921 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 772874us
I20260709 13:08:16.972728 22843 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:16.973063 22920 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:16.943236 24379 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:16.943454 24316 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:16.944772 24318 consensus_meta.cc:287] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.527s	user 0.015s	sys 0.003s
I20260709 13:08:16.975111 22871 raft_consensus.cc:606] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Received request to transfer leadership to TS 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:16.975684 22871 raft_consensus.cc:614] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:16.975953 24369 raft_consensus.cc:3055] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:16.976300 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:16.947141 24355 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:16.950327 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:16.980211 24320 raft_consensus.cc:3055] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:16.941968 24275 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:16.980640 24320 raft_consensus.cc:740] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:16.981442 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:16.981770 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:16.981547 24320 consensus_queue.cc:260] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:16.982702 22875 raft_consensus.cc:1240] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:16.982956 24320 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:16.983134 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:16.984227 22874 raft_consensus.cc:1240] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:16.984896 23616 raft_consensus.cc:1240] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:16.987149 22877 raft_consensus.cc:3055] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:16.987579 22877 raft_consensus.cc:740] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:16.988503 22877 consensus_queue.cc:260] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:16.989949 22877 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:16.947947 24366 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:16.976384 24369 raft_consensus.cc:740] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:16.977447 24328 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:16.748762 24333 consensus_queue.cc:1059] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:08:17.103701 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:16.996958 23020 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b7633b7b4c334b039db57d77774df345"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:60044
W20260709 13:08:17.058287 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 54171.9us
I20260709 13:08:17.107179 23020 raft_consensus.cc:606] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:08:16.991689 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:17.010094 23023 raft_consensus.cc:1240] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:17.025499 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:17.056452 22920 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:17.076098 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:17.084637 24355 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:17.014201 23028 raft_consensus.cc:1240] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:17.023489 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:17.111663 24355 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:16.990497 22994 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:08:17.092444 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 51310us
W20260709 13:08:17.077904 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 91391.9us
W20260709 13:08:17.113232 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 122807us
I20260709 13:08:16.993595 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:16.990803 24308 raft_consensus.cc:2749] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:17.114405 24362 raft_consensus.cc:2749] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:08:17.114717 24091 raft_consensus.cc:3055] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:17.114770 24342 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:17.115363 24091 raft_consensus.cc:740] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:17.051637 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:16.994694 22879 raft_consensus.cc:1240] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:17.116520 24392 consensus_queue.cc:1059] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 13:08:17.023617 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 49939.3us
I20260709 13:08:17.116585 24091 consensus_queue.cc:260] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:17.117822 24091 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:17.091445 24314 raft_consensus.cc:2749] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:17.096930 24412 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:17.041076 24320 consensus_queue.cc:1059] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.075s
I20260709 13:08:17.011155 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:17.119372 24412 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:17.120224 22874 raft_consensus.cc:2393] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 3: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:17.023711 24350 consensus_queue.cc:1059] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.097s
I20260709 13:08:17.115402 24362 consensus_queue.cc:1059] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:08:17.117938 24402 raft_consensus.cc:3055] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:17.120911 24402 raft_consensus.cc:740] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:17.121405 24318 raft_consensus.cc:3055] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:17.121932 24412 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:17.121812 24318 raft_consensus.cc:740] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:17.121881 24402 consensus_queue.cc:260] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:17.123067 24318 consensus_queue.cc:260] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:17.123924 24389 raft_consensus.cc:3055] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:17.124058 24318 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:17.124368 24389 raft_consensus.cc:740] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:17.124991 24342 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:17.122669 22955 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:08:17.125289 24389 consensus_queue.cc:260] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:17.126511 24389 raft_consensus.cc:3060] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:17.127033 24342 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:17.128914 24355 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:17.134016 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:17.058997 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 46694.7us
I20260709 13:08:17.001248 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:17.135653 22872 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:17.043596 24334 raft_consensus.cc:2749] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:17.000476 23590 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:17.136354 24077 consensus_queue.cc:1059] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:08:17.134754 22877 raft_consensus.cc:1275] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
W20260709 13:08:17.138182 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 137792us
I20260709 13:08:17.013325 24275 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:16.992141 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:17.139478 24077 raft_consensus.cc:3055] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:17.140009 24077 raft_consensus.cc:740] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:08:17.140573 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 84239.8us
I20260709 13:08:17.141314 22920 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:17.141378 24077 consensus_queue.cc:260] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:17.142879 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:17.107522 24369 consensus_queue.cc:260] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:17.143937 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:17.079533 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:17.145071 24385 raft_consensus.cc:2749] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:17.148306 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:17.145426 24320 raft_consensus.cc:3055] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:17.139061 24314 raft_consensus.cc:2749] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:08:17.149354 24320 raft_consensus.cc:740] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:17.149659 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:17.051541 24322 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:17.149930 24402 raft_consensus.cc:3060] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:17.150274 24320 consensus_queue.cc:260] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:17.151417 24320 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:17.059091 24325 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:17.152554 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:17.144006 22844 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:17.154086 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:17.145121 24308 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:17.147526 24365 consensus_queue.cc:1059] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:08:17.148342 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:17.145303 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:17.145649 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:17.139290 22877 pending_rounds.cc:85] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 1
I20260709 13:08:17.161805 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:17.155308 24360 raft_consensus.cc:2749] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:08:17.163866 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:17.169798 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:17.177327 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:17.177374 23616 raft_consensus.cc:1240] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:17.181461 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:17.182525 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:17.182969 24077 raft_consensus.cc:3060] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:17.183473 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:17.183669 22872 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
W20260709 13:08:17.184305 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:17.187904 24308 raft_consensus.cc:2749] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:17.164001 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:17.156222 24275 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:17.197422 23624 raft_consensus.cc:2393] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 2: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:17.198053 24275 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:17.198825 22875 raft_consensus.cc:1240] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:08:17.190251 24366 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:17.200510 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:17.201125 22955 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 2.
I20260709 13:08:17.202302 23588 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:17.203372 24275 raft_consensus.cc:2804] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:17.203902 24275 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:17.204322 24275 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:17.081063 24400 consensus_queue.cc:1059] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.091s
I20260709 13:08:17.207566 24369 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:17.210532 22955 raft_consensus.cc:1275] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:08:17.213878 24400 raft_consensus.cc:3055] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:17.214350 24400 raft_consensus.cc:740] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:17.214015 24314 consensus_queue.cc:1059] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:08:17.215207 24400 consensus_queue.cc:260] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:17.216034 24314 raft_consensus.cc:3055] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:17.219460 23588 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:17.216768 24314 raft_consensus.cc:740] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:17.162561 22877 pending_rounds.cc:107] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:08:17.221113 24314 consensus_queue.cc:260] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:17.221690 22992 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:17.221732 22877 raft_consensus.cc:2889] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:08:17.225880 24359 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 13:08:17.226668 24275 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:17.229825 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:17.230517 24397 raft_consensus.cc:2749] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:17.231578 24314 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:17.231658 24402 raft_consensus.cc:2804] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:17.233124 22919 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:17.235903 24402 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
W20260709 13:08:17.239673 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 71010.6us
I20260709 13:08:17.240396 22994 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:08:17.240694 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:17.241853 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:17.226783 23618 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:17.243923 22993 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:17.234393 24400 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:17.235985 24320 raft_consensus.cc:2749] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:08:17.237740 24402 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:17.247371 22947 raft_consensus.cc:1240] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:17.249475 22955 raft_consensus.cc:1240] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:17.250474 24077 raft_consensus.cc:2749] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:08:17.251220 24077 consensus_queue.cc:1059] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:17.252874 22949 raft_consensus.cc:1240] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:17.225080 24333 consensus_queue.cc:1059] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:08:17.265954 24402 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:17.269233 24413 raft_consensus.cc:2749] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:17.236567 24392 consensus_queue.cc:1048] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.014s
I20260709 13:08:17.273052 22776 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33466 (request call id 281) took 1031 ms. Trace:
I20260709 13:08:17.275525 22775 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33488 (request call id 260) took 1202 ms. Trace:
I20260709 13:08:17.250481 22994 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:17.282716 24328 raft_consensus.cc:3055] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:17.282994 24412 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:17.283171 24328 raft_consensus.cc:740] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:17.283659 24412 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:17.286053 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:17.286615 22947 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:17.284080 24328 consensus_queue.cc:260] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:17.287426 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:17.288082 23024 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 2.
I20260709 13:08:17.288144 24328 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:17.288582 24412 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:17.295359 22867 tablet_service.cc:1558] Processing DeleteTablet for tablet ec334666b7964c969125af95d9c797ed with delete_type TABLET_DATA_TOMBSTONED (TS 0a407504ed1741bf89a276558adf4c0f not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:35344
W20260709 13:08:17.296559 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 46144.7us
I20260709 13:08:17.297402 22994 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:17.298745 22947 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 2.
I20260709 13:08:17.300285 22992 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:17.300638 22993 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:17.303869 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:17.307484 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:17.276084 22775 rpcz_store.cc:276] 0709 13:08:16.072778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:16.073243 (+   465us) service_pool.cc:224] Handling call
0709 13:08:17.275450 (+1202207us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:16.191791 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:16.276957 (+ 85166us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:16.277004 (+    47us) write_op.cc:435] Acquired schema lock
0709 13:08:16.277026 (+    22us) tablet.cc:662] Decoding operations
0709 13:08:16.277590 (+   564us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:16.277662 (+    72us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:16.277693 (+    31us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:08:16.277894 (+   201us) tablet.cc:701] Row locks acquired
0709 13:08:16.277912 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:08:16.278043 (+   131us) write_op.cc:270] Start()
0709 13:08:16.278157 (+   114us) write_op.cc:276] Timestamp: P: 1783602496278003 usec, L: 0
0709 13:08:16.278181 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:08:16.327864 (+ 49683us) log.cc:844] Serialized 535 byte log entry
0709 13:08:16.948781 (+620917us) op_driver.cc:464] REPLICATION: finished
0709 13:08:17.249710 (+300929us) write_op.cc:301] APPLY: starting
0709 13:08:17.258914 (+  9204us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:17.259055 (+   141us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:17.259221 (+   166us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:17.259487 (+   266us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:17.259692 (+   205us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:08:17.259767 (+    75us) write_op.cc:312] APPLY: finished
0709 13:08:17.260010 (+   243us) log.cc:844] Serialized 23 byte log entry
0709 13:08:17.260250 (+   240us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:17.260348 (+    98us) write_op.cc:454] Released schema lock
0709 13:08:17.260397 (+    49us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:17.260856 (+   459us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:17.260899 (+    43us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":290243,"apply.run_cpu_time_us":2043,"apply.run_wall_time_us":14844,"num_ops":1,"prepare.queue_time_us":49819,"prepare.run_cpu_time_us":22284,"prepare.run_wall_time_us":276654,"raft.queue_time_us":852763,"raft.run_cpu_time_us":54480,"raft.run_wall_time_us":329301,"replication_time_us":670448,"spinlock_wait_cycles":4352,"thread_start_us":510556,"threads_started":4,"wal-append.queue_time_us":108258}]]}
I20260709 13:08:17.310608 23020 raft_consensus.cc:1240] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:08:17.315508 23023 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 3.
I20260709 13:08:17.316404 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:17.317076 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:17.318169 24420 tablet_replica.cc:333] T ec334666b7964c969125af95d9c797ed P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:08:17.318316 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:17.319208 24420 raft_consensus.cc:2243] T ec334666b7964c969125af95d9c797ed P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:17.319626 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:17.319970 24420 raft_consensus.cc:2272] T ec334666b7964c969125af95d9c797ed P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:17.322638 24420 ts_tablet_manager.cc:1916] T ec334666b7964c969125af95d9c797ed P 0a407504ed1741bf89a276558adf4c0f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:08:17.324779 24414 raft_consensus.cc:2804] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:17.310832 24364 raft_consensus.cc:2749] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:17.326565 24416 raft_consensus.cc:993] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75: : Instructing follower 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:08:17.327687 23027 raft_consensus.cc:1275] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:17.328285 22955 raft_consensus.cc:1240] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:17.329192 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:17.329979 23025 raft_consensus.cc:2393] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 2: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:17.330251 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:17.331416 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:17.331918 22953 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:17.333636 22955 raft_consensus.cc:1240] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:17.273559 22776 rpcz_store.cc:276] 0709 13:08:16.241806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:16.241930 (+   124us) service_pool.cc:224] Handling call
0709 13:08:17.272976 (+1031046us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:16.536897 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:16.545260 (+  8363us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:16.545292 (+    32us) write_op.cc:435] Acquired schema lock
0709 13:08:16.545312 (+    20us) tablet.cc:662] Decoding operations
0709 13:08:16.545656 (+   344us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:16.545696 (+    40us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:16.545727 (+    31us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:08:16.545893 (+   166us) tablet.cc:701] Row locks acquired
0709 13:08:16.545909 (+    16us) write_op.cc:260] PREPARE: finished
0709 13:08:16.546060 (+   151us) write_op.cc:270] Start()
0709 13:08:16.546197 (+   137us) write_op.cc:276] Timestamp: P: 1783602496546003 usec, L: 0
0709 13:08:16.546227 (+    30us) op_driver.cc:348] REPLICATION: starting
0709 13:08:16.582830 (+ 36603us) log.cc:844] Serialized 535 byte log entry
0709 13:08:17.225166 (+642336us) op_driver.cc:464] REPLICATION: finished
0709 13:08:17.261326 (+ 36160us) write_op.cc:301] APPLY: starting
0709 13:08:17.261425 (+    99us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:17.261558 (+   133us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:17.261692 (+   134us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:17.261943 (+   251us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:17.262138 (+   195us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:08:17.262205 (+    67us) write_op.cc:312] APPLY: finished
0709 13:08:17.262407 (+   202us) log.cc:844] Serialized 23 byte log entry
0709 13:08:17.262598 (+   191us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:17.262689 (+    91us) write_op.cc:454] Released schema lock
0709 13:08:17.262736 (+    47us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":3265,"reported_tablets":2,"child_traces":[["op",{"apply.queue_time_us":35896,"apply.run_cpu_time_us":1753,"apply.run_wall_time_us":7181,"num_ops":1,"prepare.queue_time_us":355,"prepare.run_cpu_time_us":6346,"prepare.run_wall_time_us":67391,"replication_time_us":629758,"thread_start_us":54747,"threads_started":1}]]}
I20260709 13:08:17.335287 23020 raft_consensus.cc:2393] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 3: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:17.335697 22955 raft_consensus.cc:1240] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:17.338707 23021 raft_consensus.cc:1240] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:17.338990 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:17.340912 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:17.276882 24402 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:17.342515 24275 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:17.344039 24359 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:17.344070 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:17.307518 23028 raft_consensus.cc:1240] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:17.344605 24359 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:17.345469 23023 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:17.346979 24407 raft_consensus.cc:3055] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:17.347472 24407 raft_consensus.cc:740] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:17.346736 22953 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 3.
W20260709 13:08:17.349117 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 48590.5us
I20260709 13:08:17.349881 22993 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:17.350267 24416 raft_consensus.cc:1081] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Signalling peer 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:08:17.351213 24416 raft_consensus.cc:2749] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:17.338754 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:17.351859 24416 consensus_queue.cc:1059] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:08:17.329257 22777 catalog_manager.cc:5697] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: config changed from index 2 to 3, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:17.352555 24414 raft_consensus.cc:697] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:17.352985 24415 raft_consensus.cc:2749] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:17.368743 24417 raft_consensus.cc:2804] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:17.353749 22993 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:08:17.353801 23588 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:17.370656 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:08:17.360461 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:17.346925 24359 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:17.379984 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:17.338150 24328 consensus_queue.cc:1048] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
W20260709 13:08:17.383123 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:17.376619 24091 raft_consensus.cc:2804] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:17.374878 24392 raft_consensus.cc:2804] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:17.378484 24417 raft_consensus.cc:697] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 2 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:17.397027 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:17.396713 24417 consensus_queue.cc:237] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:17.397924 22953 raft_consensus.cc:2410] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:08:17.401278 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:17.385917 23020 raft_consensus.cc:1240] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:17.386193 24402 raft_consensus.cc:3055] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:17.388378 23022 raft_consensus.cc:2468] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:08:17.403591 24402 raft_consensus.cc:740] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:17.403985 22950 raft_consensus.cc:1240] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:08:17.404413 24402 consensus_queue.cc:260] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:17.405838 22871 raft_consensus.cc:1240] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:17.406482 24402 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:17.374717 24407 consensus_queue.cc:260] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:17.394963 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:17.407930 24407 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:17.395522 22843 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:17.409375 22952 raft_consensus.cc:1275] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:17.411355 22946 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
mode: GRACEFUL
new_leader_uuid: "f3c799b1249e40119d2a5864231c6c75"
 from {username='slave'} at 127.0.0.1:36566
I20260709 13:08:17.411760 22920 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:17.411877 22946 raft_consensus.cc:606] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 LEADER]: Received request to transfer leadership to TS f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:17.368161 24414 consensus_queue.cc:237] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:17.415910 24325 raft_consensus.cc:2749] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:08:17.415936 24359 raft_consensus.cc:2749] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:17.417115 22920 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:17.428505 22920 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:17.431285 22920 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:17.434705 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:17.435529 24412 raft_consensus.cc:2749] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:17.408905 24091 raft_consensus.cc:697] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:17.412987 24392 raft_consensus.cc:697] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:17.450428 22920 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:17.441455 24412 raft_consensus.cc:2749] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:17.440403 24334 consensus_queue.cc:1059] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 13:08:17.440784 24396 raft_consensus.cc:2749] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:17.453197 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:17.453980 23020 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:17.455783 22918 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:17.455968 24091 consensus_queue.cc:237] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:17.457134 24381 raft_consensus.cc:2749] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:17.469256 24325 consensus_queue.cc:1059] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:17.469553 22918 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:17.470288 21756 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:08:17.470780 24325 raft_consensus.cc:2764] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won
I20260709 13:08:17.471382 22918 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:17.471407 24325 consensus_queue.cc:1059] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:17.472151 21756 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:17.470273 24387 raft_consensus.cc:3055] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:17.476550 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:17.476851 22843 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:17.472766 24325 raft_consensus.cc:3055] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:17.473583 24412 consensus_queue.cc:1059] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 13:08:17.484906 22843 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:17.480000 24359 raft_consensus.cc:3055] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:17.488059 24359 raft_consensus.cc:740] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:17.480566 24350 consensus_queue.cc:1059] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:17.480844 24342 raft_consensus.cc:2749] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:08:17.474915 24306 raft_consensus.cc:2749] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:17.489493 24342 raft_consensus.cc:3055] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:17.481760 22877 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b7633b7b4c334b039db57d77774df345"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:35370
I20260709 13:08:17.474531 24381 raft_consensus.cc:2749] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:17.485008 24365 consensus_queue.cc:1059] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:17.487090 24323 raft_consensus.cc:993] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7: : Instructing follower f3c799b1249e40119d2a5864231c6c75 to start an election
I20260709 13:08:17.486789 24355 consensus_queue.cc:1059] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:17.487512 24360 raft_consensus.cc:2749] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:08:17.491163 22918 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:08:17.493427 22843 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:17.489876 24342 raft_consensus.cc:740] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:17.474824 24387 raft_consensus.cc:740] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:17.502254 22877 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:17.485450 24325 raft_consensus.cc:740] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:17.517103 24379 raft_consensus.cc:1081] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 LEADER]: Signalling peer f3c799b1249e40119d2a5864231c6c75 to start an election
I20260709 13:08:17.520241 24316 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:17.520321 24396 raft_consensus.cc:2804] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:17.533007 22947 raft_consensus.cc:1240] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:17.529181 24396 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:17.534376 24396 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:17.522462 22877 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:17.534451 24325 consensus_queue.cc:260] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:17.529965 24406 raft_consensus.cc:3055] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Stepping down as leader of term 2
W20260709 13:08:17.559859 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:17.527302 24355 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:17.564319 22949 consensus_queue.cc:237] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:17.567350 24387 consensus_queue.cc:260] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:17.568135 23022 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
 from {username='slave'} at 127.0.0.1:33420
I20260709 13:08:17.568804 23022 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:17.569176 23022 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:17.503512 24359 consensus_queue.cc:260] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:17.541831 24334 consensus_queue.cc:1048] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
W20260709 13:08:17.585657 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 54573.5us
I20260709 13:08:17.586668 22843 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:17.559791 24406 raft_consensus.cc:740] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:17.584542 24325 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:17.504012 24392 consensus_queue.cc:237] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:17.608870 24387 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:17.612039 24406 consensus_queue.cc:260] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:17.613566 24406 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:17.572993 24355 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:17.597004 24317 raft_consensus.cc:2804] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:17.621517 24317 raft_consensus.cc:697] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 3 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:17.622651 24317 consensus_queue.cc:237] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:17.623548 23022 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:17.571897 24342 consensus_queue.cc:260] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:17.628547 24342 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:17.629669 23022 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:17.630429 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:17.631759 22950 raft_consensus.cc:3055] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:17.620986 24355 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:17.635944 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:17.637502 23624 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:08:17.638777 22844 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:17.627017 24420 ts_tablet_manager.cc:1929] T ec334666b7964c969125af95d9c797ed P 0a407504ed1741bf89a276558adf4c0f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 13:08:17.632848 22950 raft_consensus.cc:740] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:17.639546 24275 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:17.639570 24420 log.cc:1199] T ec334666b7964c969125af95d9c797ed P 0a407504ed1741bf89a276558adf4c0f: Deleting WAL directory at /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-0-root/wals/ec334666b7964c969125af95d9c797ed
I20260709 13:08:17.635807 24328 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:17.640337 22950 consensus_queue.cc:260] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:17.640156 24275 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:17.640874 24328 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:17.641752 22950 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:17.641759 22877 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:17.643041 22872 raft_consensus.cc:1240] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:17.643245 24431 raft_consensus.cc:2804] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:17.643688 24431 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:17.642997 24275 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:17.644222 24431 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:17.646320 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:17.646945 22947 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 2.
I20260709 13:08:17.649187 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:17.649714 22880 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 2.
I20260709 13:08:17.653884 22877 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:17.648407 22992 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:17.654699 24417 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:17.655097 24417 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:17.660403 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:17.661146 22875 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 3.
I20260709 13:08:17.657717 24417 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:17.662289 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:17.663393 22947 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 3.
I20260709 13:08:17.664897 23588 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:17.643059 24328 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:17.667055 24091 raft_consensus.cc:2804] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:17.662328 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:17.673318 23618 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:08:17.677968 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 91371.5us
I20260709 13:08:17.667537 24091 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:17.679371 24091 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:17.679620 22877 raft_consensus.cc:1240] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:17.684325 22843 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:17.689319 24369 consensus_queue.cc:1048] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 13:08:17.688418 24323 consensus_queue.cc:1059] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 13:08:17.692217 23617 raft_consensus.cc:1275] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 13:08:17.692435 24306 raft_consensus.cc:3055] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:17.692884 24306 raft_consensus.cc:740] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:17.693763 24306 consensus_queue.cc:260] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:17.694792 24306 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:08:17.704000 24426 log.cc:927] Time spent T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
I20260709 13:08:17.689687 24355 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:17.707185 24318 raft_consensus.cc:3055] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:17.707576 24318 raft_consensus.cc:740] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:17.700817 22843 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:17.677975 24396 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:17.718062 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:17.712373 24396 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:17.724270 24091 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:17.725260 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 72075.2us
I20260709 13:08:17.727088 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:17.727702 23020 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:17.728185 22843 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:17.727429 22777 catalog_manager.cc:5697] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 1 to 3, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 3 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:17.730526 24091 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:17.731508 24342 raft_consensus.cc:2749] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:17.729990 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:17.733134 22949 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:17.733517 24369 raft_consensus.cc:2804] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:17.734004 24369 raft_consensus.cc:697] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:17.735176 24359 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:17.734865 24369 consensus_queue.cc:237] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:17.739162 23020 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:17.742415 24318 consensus_queue.cc:260] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:17.743024 22950 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:17.744258 24318 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:17.744580 22992 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:17.745365 24364 raft_consensus.cc:2804] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:17.745831 24364 raft_consensus.cc:697] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:17.746618 24364 consensus_queue.cc:237] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:17.729745 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:17.748955 24320 raft_consensus.cc:3055] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:17.749518 24320 raft_consensus.cc:740] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:17.749205 22843 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:17.750592 24320 consensus_queue.cc:260] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:17.750998 24359 raft_consensus.cc:2749] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:17.752632 24431 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:17.754129 23618 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:17.755064 24431 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:17.756518 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:17.757115 23619 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:17.761281 24417 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:17.764225 24355 raft_consensus.cc:2749] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:17.764528 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:17.765151 22950 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:17.765964 23619 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 3.
I20260709 13:08:17.766930 22918 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:08:17.767215 24417 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:17.767616 22844 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:17.768229 22764 catalog_manager.cc:5028] TS 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): tablet ec334666b7964c969125af95d9c797ed (table test-workload [id=a42f2ccfbe484baca3019323a490c64a]) successfully deleted
I20260709 13:08:17.768791 24360 raft_consensus.cc:2804] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:17.769399 24360 raft_consensus.cc:697] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 3 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:17.770653 24306 raft_consensus.cc:2804] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:17.770336 24360 consensus_queue.cc:237] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:17.771193 24306 raft_consensus.cc:697] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:17.773188 24306 consensus_queue.cc:237] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:17.774482 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:17.774681 24328 consensus_queue.cc:1059] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:17.775233 22872 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:17.775998 24392 raft_consensus.cc:3055] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:17.776408 24392 raft_consensus.cc:740] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:17.777518 24392 consensus_queue.cc:260] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:17.779314 24392 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:17.783700 22950 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:17.798656 22992 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:17.799856 24328 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:17.749017 22875 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:17.800499 24328 raft_consensus.cc:697] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:17.801725 24328 consensus_queue.cc:237] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:17.804503 24091 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:17.804945 24320 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:17.805441 24091 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:17.809250 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:17.819649 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:17.820317 22947 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:08:17.822785 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:17.823496 22947 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:17.780092 22776 catalog_manager.cc:5697] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 3 to 4, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 4 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:17.826586 22776 catalog_manager.cc:5697] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 1 to 2, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 2 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:17.831877 22947 raft_consensus.cc:2468] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:08:17.828696 22776 catalog_manager.cc:5697] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 2 to 3, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 3 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:17.836515 24091 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:17.837028 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:17.871840 23020 raft_consensus.cc:2393] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 3: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:17.845067 22950 raft_consensus.cc:1240] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 LEADER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:08:17.873970 23588 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:17.846675 23024 raft_consensus.cc:1240] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:08:17.847029 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:17.876171 24091 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:17.883960 23022 raft_consensus.cc:2468] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:08:17.888885 23022 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:17.851330 22843 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:17.893299 24360 raft_consensus.cc:2804] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:17.893862 24360 raft_consensus.cc:697] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:17.894815 24360 consensus_queue.cc:237] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:17.898720 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 47467.6us
I20260709 13:08:17.900403 24369 consensus_queue.cc:1059] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:17.901186 24369 consensus_queue.cc:1243] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer aed3f089ec8b49af9c8cf5cdee76e8e7 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 13:08:17.901623 24431 raft_consensus.cc:3055] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:17.901976 24431 raft_consensus.cc:740] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:17.902815 24431 consensus_queue.cc:260] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:17.907701 24091 raft_consensus.cc:2749] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:17.869194 22949 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 4.
I20260709 13:08:17.921265 23588 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:17.908874 24431 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:17.911103 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:17.922308 22950 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:17.934324 23022 raft_consensus.cc:1275] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:08:17.936878 22950 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 3.
I20260709 13:08:17.860528 22777 catalog_manager.cc:5697] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 2 to 3, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 3 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:17.859813 22872 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:17.943444 22774 catalog_manager.cc:5697] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 2 to 3, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 3 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:17.945045 22774 catalog_manager.cc:5697] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:17.953933 22774 catalog_manager.cc:5697] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 1 to 2, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 2 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:17.955837 22774 catalog_manager.cc:5697] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 1 to 2, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 2 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:17.916402 22875 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 4.
I20260709 13:08:17.960067 24091 raft_consensus.cc:2804] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:17.969704 22950 raft_consensus.cc:1275] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:08:17.971477 24328 consensus_queue.cc:1048] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:17.976090 24091 raft_consensus.cc:697] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 4 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:17.978991 24091 consensus_queue.cc:237] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:17.991739 22875 raft_consensus.cc:1275] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:17.993512 24328 consensus_queue.cc:1048] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:17.994964 24392 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:17.995499 24392 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:17.996938 22949 raft_consensus.cc:1275] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:17.997583 24392 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:17.998971 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:17.999770 22872 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:18.001266 22994 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:18.002539 24392 raft_consensus.cc:2804] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:18.003053 24392 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:18.004734 24392 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:18.005539 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:18.003973 24328 consensus_queue.cc:1048] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:18.008402 22872 raft_consensus.cc:1275] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:08:18.014832 22948 raft_consensus.cc:1275] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:08:18.016364 24407 consensus_queue.cc:1048] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:18.021967 24328 consensus_queue.cc:1048] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:18.056604 24392 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:18.059861 24407 consensus_queue.cc:1048] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:08:18.062258 24416 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:18.062999 24416 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:18.064707 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:18.065580 22871 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:18.065904 24416 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:18.070195 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:18.070971 23619 raft_consensus.cc:2393] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 3: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:18.071847 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:18.072448 22950 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:18.076351 22992 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:18.087431 22777 catalog_manager.cc:5697] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 2 to 3, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 3 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:18.089315 22777 catalog_manager.cc:5697] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:18.118543 22774 catalog_manager.cc:5697] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:18.125195 24392 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:08:18.127074 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 50756.9us
I20260709 13:08:18.096397 24431 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:18.190949 23618 raft_consensus.cc:1275] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:08:18.192632 24318 consensus_queue.cc:1048] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:18.131750 24364 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:18.137214 24392 raft_consensus.cc:2804] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:18.182560 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:18.140198 24431 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:18.209668 24392 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:18.228305 23618 raft_consensus.cc:3060] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:18.231405 23023 raft_consensus.cc:1275] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:08:18.222589 24306 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:18.210086 24364 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:18.230930 24392 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:18.242939 23022 raft_consensus.cc:1275] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:08:18.266455 24306 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:18.282680 24364 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:18.275318 22877 raft_consensus.cc:1275] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:08:18.285123 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:18.285414 22877 pending_rounds.cc:85] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 1
I20260709 13:08:18.274222 24412 consensus_queue.cc:1048] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 13:08:18.285758 23624 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:18.285741 22877 pending_rounds.cc:107] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:08:18.293533 22877 raft_consensus.cc:2889] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:18.294202 22919 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:18.295851 24412 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:18.296308 24412 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:18.296738 24412 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:18.305087 22949 raft_consensus.cc:1275] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:18.305485 24417 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:18.306126 24417 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:18.309904 24453 log.cc:927] Time spent T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:08:18.312443 23624 raft_consensus.cc:1275] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:08:18.312585 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:18.313948 24404 consensus_queue.cc:1048] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:18.324865 22871 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:08:18.330749 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:18.331610 22872 raft_consensus.cc:2468] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 2.
I20260709 13:08:18.332288 24404 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:18.333285 24404 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:18.334000 24431 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:18.335876 24404 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:18.336553 24364 consensus_queue.cc:1048] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:08:18.338529 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:18.339356 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:18.344779 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:18.345635 23024 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:08:18.347266 22994 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:18.349758 24414 raft_consensus.cc:2804] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:18.350396 24414 raft_consensus.cc:697] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:18.351508 24414 consensus_queue.cc:237] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:18.353547 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:18.354364 23022 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:08:18.282761 24306 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:18.361208 23624 raft_consensus.cc:1275] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:08:18.362929 24308 consensus_queue.cc:1048] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:18.367983 22994 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:18.370056 22843 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:18.373466 22843 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:18.374084 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:18.374873 22872 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:18.383549 24402 raft_consensus.cc:2804] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:18.384253 24402 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:18.384725 24402 raft_consensus.cc:3060] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:18.359153 24417 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:18.387923 24318 raft_consensus.cc:2804] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:18.370007 24404 consensus_queue.cc:1048] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260709 13:08:18.415408 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:18.424691 24318 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:18.547946 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:18.442528 24360 raft_consensus.cc:2804] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:08:18.549818 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 176305us
I20260709 13:08:18.572993 24091 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:18.573607 24091 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:18.539958 24318 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:18.581368 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:18.589958 24091 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:18.445190 24431 consensus_queue.cc:1048] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.132s
I20260709 13:08:18.554337 24360 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:18.573036 24369 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:18.618479 22950 raft_consensus.cc:1275] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:08:18.603663 24360 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:18.598502 24402 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:18.620033 23618 raft_consensus.cc:1275] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:08:18.607013 24369 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:18.644523 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 89793.7us
I20260709 13:08:18.627427 24407 consensus_queue.cc:1048] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
W20260709 13:08:18.679076 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 305654us
I20260709 13:08:18.655107 24464 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:18.691159 23624 raft_consensus.cc:1275] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:08:18.694968 24456 log.cc:927] Time spent T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:08:18.695290 24308 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:18.695755 24308 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:18.697165 24369 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:18.697729 24308 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:18.698571 24369 consensus_queue.cc:1048] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:18.699301 24467 consensus_queue.cc:1048] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:08:18.693383 24464 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:18.706666 24412 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:18.707708 24392 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:18.713521 24402 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:18.692943 24275 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:18.715327 24275 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:18.718568 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:18.719199 23026 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:18.719470 23023 raft_consensus.cc:1275] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 13:08:18.717981 24412 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:18.718775 24464 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:18.735100 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:18.727571 24475 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:18.729951 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:18.730867 24466 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:18.730146 24407 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:18.745527 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:18.746126 22872 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:18.739810 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:18.742379 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:18.749392 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:18.766552 24478 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:18.776870 23622 raft_consensus.cc:2393] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 3: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:18.766784 24318 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:18.745970 24407 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:18.748057 24392 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:18.774884 24275 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:18.746150 22949 raft_consensus.cc:1275] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:08:18.765617 24369 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:18.744719 24475 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:18.821847 24478 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:18.833457 24369 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:18.834761 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:18.835451 23020 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 3.
I20260709 13:08:18.836302 24318 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:18.829376 24392 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:18.834857 24478 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:18.837131 24392 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:18.837970 24407 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:18.838918 23618 raft_consensus.cc:3060] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:08:18.839201 24392 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:18.835466 24369 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:18.837221 24475 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:18.745810 24466 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:18.850754 22777 catalog_manager.cc:5697] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 2 to 4, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 4 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:18.851358 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:18.852098 23623 raft_consensus.cc:2393] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 3: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:18.856096 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:18.850101 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:18.857486 23622 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:18.857931 22880 raft_consensus.cc:2393] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 2: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:18.793275 22871 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 1.
I20260709 13:08:18.790710 22872 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:18.872061 23622 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:18.857374 24417 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:18.822763 23621 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 3.
I20260709 13:08:18.883867 23618 raft_consensus.cc:2468] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:18.887684 22949 raft_consensus.cc:1275] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:08:18.890516 24369 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:18.891062 24369 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:18.888283 24464 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:18.893564 24464 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:18.894449 24369 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:18.895025 22877 raft_consensus.cc:1275] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:08:18.896148 24464 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:18.888485 24417 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:18.899396 24417 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:18.900173 22871 raft_consensus.cc:1275] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:08:18.904071 24478 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:18.904580 24478 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:18.845177 24360 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:18.908871 24478 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:18.908998 24320 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:18.888862 24392 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:18.909924 24392 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:18.909827 24320 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:18.910720 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:18.911288 22875 raft_consensus.cc:3060] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:18.906543 24306 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:18.913290 24306 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:18.914448 24392 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:18.914644 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:18.915410 23620 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:08:18.915654 24306 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:18.917080 24458 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:18.917140 24474 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:18.917743 24458 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:18.917843 24474 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:18.918267 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:18.918699 22878 raft_consensus.cc:1275] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:08:18.918845 22874 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:08:18.919786 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:18.919963 24474 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:18.920444 22879 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:18.921802 22875 raft_consensus.cc:2468] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:18.922506 24392 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:18.922874 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:18.923004 24392 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:18.923699 22874 raft_consensus.cc:2393] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 2: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:18.926185 24392 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:18.926383 24360 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:18.927435 22871 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:18.919939 24458 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:18.962793 22994 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:18.969753 22994 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:18.972437 22994 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:18.938321 24320 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:18.967481 24464 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:18.961426 24392 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:18.973434 24467 raft_consensus.cc:2804] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:18.986591 24467 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:18.960858 24407 consensus_queue.cc:1048] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 13:08:18.987063 24467 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:18.967953 24417 consensus_queue.cc:1048] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 13:08:18.990447 24466 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:18.977136 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:18.991832 22871 raft_consensus.cc:1275] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:08:18.992048 22874 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:18.978019 24362 raft_consensus.cc:2804] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:18.993149 24362 raft_consensus.cc:697] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:18.994242 24362 consensus_queue.cc:237] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:18.995229 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:18.995788 23624 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:18.971266 24415 consensus_queue.cc:1048] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 13:08:18.996886 22993 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:18.997146 24364 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:18.973294 24471 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:18.997716 24328 raft_consensus.cc:2804] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:18.997596 24364 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:18.998034 24471 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:18.998672 24471 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:18.999718 24364 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:18.998107 24328 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:19.001869 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:18.984519 22920 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:19.003106 23624 raft_consensus.cc:2468] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:08:19.004870 22993 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:19.009505 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:19.010157 22878 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:19.011451 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:19.012218 22873 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:19.004133 22920 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:19.015234 22994 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:19.011528 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:19.017629 22920 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:19.024241 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:19.024957 22950 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 2.
W20260709 13:08:19.025064 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 292539us
I20260709 13:08:19.028388 22950 raft_consensus.cc:1275] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:08:19.019578 24364 raft_consensus.cc:2804] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:19.042106 24364 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:19.042574 24364 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:18.985512 24392 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:18.981282 24464 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:19.047140 22871 raft_consensus.cc:2368] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 2. Current term is 3.
I20260709 13:08:19.048331 24464 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:19.016991 24414 raft_consensus.cc:2804] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:19.016943 24417 consensus_queue.cc:1048] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:08:19.013445 24328 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:19.013214 24466 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:19.059805 24466 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:19.063126 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:19.063467 24466 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:19.064410 24392 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:19.053956 24414 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:19.065640 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:19.065898 24414 raft_consensus.cc:3060] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:19.066303 23620 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:19.053997 24458 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:19.068881 24458 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:19.069342 24458 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:19.014695 24308 raft_consensus.cc:2804] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:19.073050 24308 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:19.073508 24308 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:19.020184 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:19.019385 24320 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:19.048494 23588 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:08:19.078727 22920 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:19.086112 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:19.088393 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:08:19.109596 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 61198.3us
I20260709 13:08:19.113843 22874 raft_consensus.cc:2368] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 2. Current term is 3.
I20260709 13:08:19.108319 24320 raft_consensus.cc:697] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:19.117936 24490 raft_consensus.cc:2804] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:19.126421 24091 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:19.126231 22920 leader_election.cc:400] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 2. Current term is 3.
W20260709 13:08:19.124861 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 402304us
I20260709 13:08:19.127123 24091 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:19.127822 22920 leader_election.cc:403] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:19.124218 24325 raft_consensus.cc:2804] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:19.118144 24414 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:19.136933 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:19.137873 22993 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:08:19.139250 22993 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:19.140627 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:19.121716 24489 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:19.124228 24472 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:19.137773 22844 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:19.146343 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:19.149349 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:19.125932 24490 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:19.147117 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:19.116483 24308 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:19.150069 24091 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:19.150120 23023 raft_consensus.cc:2393] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 3: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:19.154430 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:19.155015 22952 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:08:19.153321 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:19.156471 22920 leader_election.cc:400] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 2. Current term is 3.
I20260709 13:08:19.156962 22920 leader_election.cc:403] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:19.162175 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:19.162909 22955 raft_consensus.cc:2393] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 3: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:19.164346 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:19.165210 23624 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 3.
I20260709 13:08:19.166841 23020 raft_consensus.cc:2393] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 3: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:19.168296 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:19.171075 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:19.171729 23624 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:08:19.175025 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:19.176201 23029 raft_consensus.cc:2393] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 2: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:19.135453 24325 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:19.177094 22844 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:19.134951 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:19.193559 22950 raft_consensus.cc:2368] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 2. Current term is 3.
I20260709 13:08:19.148979 24402 raft_consensus.cc:2804] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:19.194417 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:19.147778 24416 raft_consensus.cc:2749] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:19.185864 22952 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 3.
I20260709 13:08:19.144491 24474 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:08:19.190394 22844 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:19.187911 22947 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:19.154096 24407 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:19.153644 24490 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:19.205793 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:19.151407 24491 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:19.193169 24404 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:19.206714 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:19.214191 24463 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:19.214641 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:19.214573 24320 consensus_queue.cc:237] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:19.174899 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:19.200791 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:19.216825 23025 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:19.174885 24360 raft_consensus.cc:2804] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:19.218572 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:19.199039 24402 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:19.227005 23023 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:19.257062 22955 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:08:19.202020 24306 raft_consensus.cc:2749] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 2. Current term is 3.
I20260709 13:08:19.191184 24325 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:19.264151 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:19.248315 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:19.264943 22955 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 3.
I20260709 13:08:19.254715 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:19.266845 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:19.266809 23020 raft_consensus.cc:2393] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 3: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:19.162366 24472 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:19.247788 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:19.211931 23024 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 3.
I20260709 13:08:19.244292 24404 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:19.330216 24404 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:19.249020 24463 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:19.332580 24463 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:19.258493 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:19.154676 24489 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:19.289489 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:19.290271 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:19.293051 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:19.294564 24306 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:19.342362 24306 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:19.343976 22948 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 2.
I20260709 13:08:19.260733 24402 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:19.219082 24407 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:19.296526 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:19.268651 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:19.357827 23020 raft_consensus.cc:2393] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 2: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:19.256857 24360 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:19.358909 24360 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:19.243515 24491 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:19.364431 24407 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:19.366302 24306 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:19.225131 24468 consensus_queue.cc:1048] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.084s
W20260709 13:08:19.325783 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 74942.2us
I20260709 13:08:19.368039 22949 raft_consensus.cc:2410] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:08:19.368393 23025 raft_consensus.cc:1275] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:19.258425 23028 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:19.196115 24464 consensus_queue.cc:1048] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 13:08:19.289172 23022 raft_consensus.cc:2368] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 2. Current term is 3.
I20260709 13:08:19.366437 24491 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:19.367805 22953 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:08:19.267769 22955 raft_consensus.cc:2468] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 2.
I20260709 13:08:19.559446 22954 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 2.
I20260709 13:08:19.372310 24417 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:19.495633 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 333406us
I20260709 13:08:19.567631 23023 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 3.
I20260709 13:08:19.561205 24477 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:19.534838 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 354733us
W20260709 13:08:19.573004 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 312675us
W20260709 13:08:19.564395 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 195294us
I20260709 13:08:19.626909 23588 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:19.629549 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:19.631070 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:19.631712 23624 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:19.637125 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:19.637933 23624 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 1.
I20260709 13:08:19.627754 22843 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:19.640765 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:19.641268 22950 raft_consensus.cc:2393] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 2: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
W20260709 13:08:19.572588 24364 consensus_meta.cc:287] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.530s	user 0.017s	sys 0.011s
I20260709 13:08:19.647734 22954 raft_consensus.cc:2393] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 3: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
W20260709 13:08:19.573624 24471 consensus_meta.cc:287] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.574s	user 0.016s	sys 0.018s
W20260709 13:08:19.575551 24467 consensus_meta.cc:287] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.588s	user 0.007s	sys 0.012s
W20260709 13:08:19.605190 24458 consensus_meta.cc:287] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.535s	user 0.015s	sys 0.000s
I20260709 13:08:19.655052 24091 raft_consensus.cc:2749] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:19.654927 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:19.654907 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:08:19.647686 24496 log.cc:927] Time spent T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 13:08:19.664688 22874 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
mode: GRACEFUL
new_leader_uuid: "f3c799b1249e40119d2a5864231c6c75"
 from {username='slave'} at 127.0.0.1:51434
I20260709 13:08:19.665591 22874 raft_consensus.cc:606] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Received request to transfer leadership to TS f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:19.667088 24489 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:08:19.605406 24328 consensus_meta.cc:287] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.549s	user 0.009s	sys 0.009s
I20260709 13:08:19.656985 22953 raft_consensus.cc:1275] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:08:19.665252 24404 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:19.572606 24481 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:19.690948 24481 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:19.668083 24414 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:19.607268 24490 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:19.700908 22874 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
mode: GRACEFUL
new_leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:51434
I20260709 13:08:19.701552 24472 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:19.674384 24500 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:19.700434 24481 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:19.703059 24497 log.cc:927] Time spent T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 13:08:19.668200 24328 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:19.644935 24477 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:19.704164 24491 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:19.707896 24360 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:19.653352 24501 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:19.644639 24417 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:19.669404 24467 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:19.660511 24364 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:19.671597 24458 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:19.696135 24416 consensus_queue.cc:1048] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:08:19.661597 24471 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:19.747746 23026 raft_consensus.cc:2368] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 3. Current term is 4.
I20260709 13:08:19.732896 22874 raft_consensus.cc:606] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Received request to transfer leadership to TS aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:19.735929 24469 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:19.709694 24463 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:19.749588 24474 consensus_meta.cc:287] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.539s	user 0.011s	sys 0.002s
I20260709 13:08:19.757403 23029 raft_consensus.cc:2368] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 2. Current term is 3.
I20260709 13:08:19.761068 22947 raft_consensus.cc:1813] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Handling vote request from an unknown peer f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:19.761569 22947 raft_consensus.cc:2368] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 2. Current term is 3.
I20260709 13:08:19.747885 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:19.769039 24467 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:19.754005 22874 raft_consensus.cc:614] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:19.742276 24491 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:19.763370 24474 raft_consensus.cc:2749] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 2. Current term is 3.
W20260709 13:08:19.802758 22920 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.003s	sys 0.025s
W20260709 13:08:19.803267 22920 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.003s	sys 0.025s
I20260709 13:08:19.780154 24360 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:19.808020 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 181481us
I20260709 13:08:19.818879 24490 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:19.818681 22992 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:08:19.826043 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:19.826886 22946 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 2.
I20260709 13:08:19.835727 24474 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:19.836490 24474 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:19.784497 24402 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:19.804771 22871 raft_consensus.cc:2368] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 3. Current term is 4.
I20260709 13:08:19.745627 24501 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:19.743319 24500 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:19.818122 24362 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.740s	user 0.015s	sys 0.061s
I20260709 13:08:19.760157 24469 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:19.863346 23020 raft_consensus.cc:1240] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:08:19.841049 24467 raft_consensus.cc:2804] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:19.842344 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
W20260709 13:08:19.860646 24362 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.783s	user 0.016s	sys 0.062s
I20260709 13:08:19.875566 24469 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:19.861964 24325 consensus_meta.cc:287] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.590s	user 0.013s	sys 0.000s
I20260709 13:08:19.881618 22951 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 3.
I20260709 13:08:19.862640 24404 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:19.871476 24474 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:19.897053 24504 log.cc:927] Time spent T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
I20260709 13:08:19.864784 24308 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:19.951537 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:19.870273 24467 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:19.953606 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:19.930397 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:19.918641 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:19.869527 23619 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 1.
I20260709 13:08:19.958038 22880 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:19.956805 24467 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:19.871515 24362 consensus_queue.cc:1048] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.093s
I20260709 13:08:19.964198 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:19.981969 22878 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:19.979545 23620 raft_consensus.cc:2368] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 3. Current term is 4.
I20260709 13:08:19.950838 24275 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:19.954324 24328 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:20.003861 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 376148us
I20260709 13:08:20.005101 23624 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 3.
I20260709 13:08:19.968528 24364 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:20.012094 22880 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:19.967852 22877 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 1.
I20260709 13:08:19.897927 24325 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:19.982167 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:19.969708 24417 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:19.984759 24477 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:20.018229 24487 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:20.018298 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:19.984400 24458 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:20.013741 24471 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:20.023324 24463 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:20.025851 22874 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:08:20.012550 24275 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:20.040381 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 386003us
I20260709 13:08:20.047914 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:20.048513 23621 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:08:20.041261 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 373043us
I20260709 13:08:20.049268 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:20.049850 23622 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:20.050665 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:20.052055 23588 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:20.053807 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:20.033527 23620 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 1.
I20260709 13:08:20.058467 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:20.066342 23624 raft_consensus.cc:3060] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:08:20.040913 24464 raft_consensus.cc:2749] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:20.030077 24487 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:20.067834 22993 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:20.052130 24402 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:20.066134 24502 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:20.075320 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:20.109853 22879 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 2.
I20260709 13:08:20.098803 24275 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:20.107121 23028 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 3.
I20260709 13:08:20.055399 24320 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
W20260709 13:08:20.116730 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 276488us
I20260709 13:08:20.146828 22992 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:20.147169 24320 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:20.150378 24518 raft_consensus.cc:2804] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:20.144979 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:20.151777 23623 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:08:20.142422 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:20.153337 23624 raft_consensus.cc:2468] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:20.109586 24502 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:20.154816 22992 leader_election.cc:400] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 pre-election: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 2. Current term is 3.
I20260709 13:08:20.156143 22992 leader_election.cc:403] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:20.132296 24417 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:20.164199 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:20.164791 22953 raft_consensus.cc:3060] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:20.165124 24515 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:08:20.151019 24518 raft_consensus.cc:697] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:20.166867 23588 leader_election.cc:400] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 2. Current term is 3.
I20260709 13:08:20.170539 23588 leader_election.cc:403] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:20.165357 24325 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:20.107466 24467 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:20.171438 22953 raft_consensus.cc:2468] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:20.171586 24490 raft_consensus.cc:2749] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 2. Current term is 3.
I20260709 13:08:20.172331 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:20.171521 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:20.157704 24502 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:20.174014 22946 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:08:20.174928 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:20.176215 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:20.173236 22948 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:20.178716 22955 raft_consensus.cc:2393] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 3: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:20.158859 24417 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:20.156270 24525 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:20.178892 22953 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:20.191897 23621 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:08:20.193480 24487 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:20.179211 24491 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:20.193917 24455 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:20.163887 24415 consensus_queue.cc:1048] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 13:08:20.181689 24517 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:20.180439 24417 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:20.184887 24525 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:20.199558 23623 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:20.205366 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:20.199839 24475 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:20.224354 22953 raft_consensus.cc:2410] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:08:20.203482 24517 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:20.225675 23618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 457) took 1053 ms. Trace:
I20260709 13:08:20.233407 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:20.233981 23026 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:20.218468 24455 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:20.236572 24511 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
W20260709 13:08:20.238025 24520 log.cc:927] Time spent T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:08:20.224813 24475 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:20.238164 24511 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:20.242063 23026 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 4.
I20260709 13:08:20.244828 24517 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:20.213770 24518 consensus_queue.cc:237] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:20.246631 24467 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:20.256824 24475 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:20.258785 24502 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:20.262033 24455 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:20.263664 24511 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:20.264938 24515 raft_consensus.cc:2749] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 2. Current term is 3.
I20260709 13:08:20.265213 24525 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:20.267177 24502 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:20.325861 24325 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:20.325861 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:20.330150 23026 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 1.
I20260709 13:08:20.232931 23618 rpcz_store.cc:276] 0709 13:08:19.162784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:19.175155 (+ 12371us) service_pool.cc:224] Handling call
0709 13:08:19.217894 (+ 42739us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:19.270739 (+ 52845us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:19.355195 (+ 84456us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:19.653754 (+298559us) log.cc:844] Serialized 26 byte log entry
0709 13:08:20.051431 (+397677us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:08:20.058118 (+  6687us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:20.067973 (+  9855us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:20.204900 (+136927us) raft_consensus.cc:1685] finished
0709 13:08:20.204920 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:20.216498 (+ 11578us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3248,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":120,"thread_start_us":347121,"threads_started":1,"wal-append.queue_time_us":483535}
I20260709 13:08:20.327430 24325 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:20.332759 23029 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33418 (request call id 1304) took 1153 ms. Trace:
I20260709 13:08:20.333868 24325 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:20.333469 23029 rpcz_store.cc:276] 0709 13:08:19.178928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:19.211448 (+ 32520us) service_pool.cc:224] Handling call
0709 13:08:20.332184 (+1120736us) spinlock_profiling.cc:243] Waited 196 ms on lock 0x7b5400249354. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:20.332476 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":451641600}
I20260709 13:08:20.358814 22918 leader_election.cc:400] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 3. Current term is 4.
I20260709 13:08:20.387822 22918 leader_election.cc:403] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:20.364212 22948 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:20.374575 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:20.375475 22919 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:20.375890 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:20.376597 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:20.377898 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:08:20.378180 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 372451us
I20260709 13:08:20.378685 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:08:20.381911 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 444113us
I20260709 13:08:20.387209 24515 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:20.389937 23588 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:20.393973 22993 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:20.394985 24501 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:20.397522 22874 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:20.408337 22871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53724 (request call id 160) took 1249 ms. Trace:
I20260709 13:08:20.412253 24481 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:20.421885 22955 raft_consensus.cc:2410] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:08:20.422545 22950 raft_consensus.cc:2393] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 2: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:20.422271 24481 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:20.423019 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:20.423368 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:20.423801 22955 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:08:20.426972 22953 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 3.
I20260709 13:08:20.420861 22871 rpcz_store.cc:276] 0709 13:08:19.158799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:19.162838 (+  4039us) service_pool.cc:224] Handling call
0709 13:08:20.407680 (+1244842us) spinlock_profiling.cc:243] Waited 517 ms on lock 0x7b54002d7154. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:20.408031 (+   351us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1189944704}
W20260709 13:08:20.431229 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 54637.3us
I20260709 13:08:20.433109 24500 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:20.434098 22843 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:20.434967 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:20.435415 24404 consensus_queue.cc:1048] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:20.435918 22872 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:08:20.436349 24463 raft_consensus.cc:2804] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:20.436442 24476 consensus_queue.cc:1059] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:08:20.437220 24463 raft_consensus.cc:697] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 3 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:20.437904 24476 raft_consensus.cc:3055] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:20.438434 24476 raft_consensus.cc:740] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:20.438812 24463 consensus_queue.cc:237] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:20.439489 24476 consensus_queue.cc:260] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:20.440150 24501 raft_consensus.cc:2804] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:20.440802 24501 raft_consensus.cc:697] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 4 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:20.441746 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:20.441937 24501 consensus_queue.cc:237] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:20.386384 22946 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:08:20.421638 22947 raft_consensus.cc:2393] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 3: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
W20260709 13:08:20.445173 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 51585.8us
I20260709 13:08:20.446254 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:20.459826 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:20.460685 23029 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:20.461017 22878 raft_consensus.cc:2393] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 2: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:20.462307 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:20.462823 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:20.463562 22879 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:20.464689 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:20.467363 24476 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:20.468537 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:20.460300 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:20.470482 23025 raft_consensus.cc:2393] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 3: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:20.470690 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:20.419323 22949 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 3.
I20260709 13:08:20.471932 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:20.472762 23027 raft_consensus.cc:2468] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:20.443596 24515 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:20.460170 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:20.473728 24412 raft_consensus.cc:2749] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 3. Current term is 4.
I20260709 13:08:20.473824 23023 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:20.474610 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:20.475762 24320 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:20.444226 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:08:20.482534 22951 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58310 (request call id 1353) took 1300 ms (client timeout 1559 ms). Trace:
I20260709 13:08:20.482412 23027 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:08:20.483289 22993 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:20.483417 23027 raft_consensus.cc:606] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:08:20.485149 22872 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:20.486552 22843 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:20.488451 23029 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:20.445248 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:20.445712 24392 raft_consensus.cc:2804] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:08:20.483107 22951 rpcz_store.cc:269] 0709 13:08:19.181678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:19.192442 (+ 10764us) service_pool.cc:224] Handling call
0709 13:08:20.482130 (+1289688us) spinlock_profiling.cc:243] Waited 602 ms on lock 0x7b54000ed8d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:20.482362 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1384306432}
I20260709 13:08:20.490267 22880 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:20.491286 22955 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 4.
I20260709 13:08:20.462234 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:20.445089 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:20.493196 22879 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:08:20.499703 24417 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:20.510212 24417 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:20.502615 24362 raft_consensus.cc:2804] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:20.517832 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:20.506760 24481 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:20.518481 24362 raft_consensus.cc:697] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:20.518576 23620 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 1.
I20260709 13:08:20.502280 22871 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:20.519997 23590 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:20.519887 24362 consensus_queue.cc:237] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:20.515884 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:20.523752 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:20.523996 22879 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:20.524468 22876 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:20.517477 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:20.507162 22877 raft_consensus.cc:2410] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 13:08:20.516422 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:20.526017 22878 raft_consensus.cc:2393] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 4: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:20.526858 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:20.528983 24511 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:20.529686 24511 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:20.492295 24500 raft_consensus.cc:2764] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:08:20.532176 24511 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:20.525141 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:20.542374 24490 raft_consensus.cc:2804] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:20.542851 24490 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:20.543277 24490 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:20.549081 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:20.556038 23619 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:08:20.546346 24275 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:20.553167 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:20.557312 23023 raft_consensus.cc:2393] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 4: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:20.557027 24275 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:20.554253 23590 leader_election.cc:400] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 3. Current term is 4.
I20260709 13:08:20.536394 24515 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:20.560791 24481 raft_consensus.cc:2804] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:20.560884 23590 leader_election.cc:403] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:08:20.561252 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 125481us
I20260709 13:08:20.561556 24481 raft_consensus.cc:697] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 3 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:20.562347 22843 leader_election.cc:400] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 2. Current term is 3.
I20260709 13:08:20.562678 22843 leader_election.cc:403] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:20.555130 22876 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:20.562811 24481 consensus_queue.cc:237] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:20.567225 22875 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:20.560256 22871 raft_consensus.cc:1275] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:08:20.569872 24476 raft_consensus.cc:2749] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 2. Current term is 3.
W20260709 13:08:20.574695 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 84835.3us
I20260709 13:08:20.579548 22880 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 4.
I20260709 13:08:20.570515 24275 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:20.587895 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:20.588946 23617 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:20.589706 22880 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
mode: GRACEFUL
new_leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:51434
I20260709 13:08:20.590284 22880 raft_consensus.cc:606] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Received request to transfer leadership to TS aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:20.590656 22880 raft_consensus.cc:614] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:08:20.591691 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 173407us
I20260709 13:08:20.593122 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:20.595103 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:20.597082 22871 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:20.595665 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:20.581960 24392 raft_consensus.cc:697] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:20.602355 21756 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:08:20.613467 22918 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:20.631392 24392 consensus_queue.cc:237] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:20.636873 21756 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:20.639416 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:20.640070 22874 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:20.641626 24320 raft_consensus.cc:2749] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:08:20.641273 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 89196.2us
I20260709 13:08:20.665776 22918 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:20.669639 22874 raft_consensus.cc:1275] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:08:20.670936 24518 raft_consensus.cc:993] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75: : Instructing follower 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:08:20.676702 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:20.677390 24518 raft_consensus.cc:1081] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Signalling peer 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:08:20.681951 22844 leader_election.cc:400] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 election: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 3. Current term is 4.
W20260709 13:08:20.670789 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 65878.7us
I20260709 13:08:20.685503 22994 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:20.668617 23617 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:08:20.687552 24515 raft_consensus.cc:2804] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:20.688817 24515 raft_consensus.cc:697] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:20.690388 24515 consensus_queue.cc:237] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:20.675513 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:20.674958 22871 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:08:20.686332 22844 leader_election.cc:403] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
W20260709 13:08:20.691601 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 48754.3us
I20260709 13:08:20.687572 22994 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:20.689373 22775 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33466 (request call id 286) took 1754 ms. Trace:
I20260709 13:08:20.691846 24320 raft_consensus.cc:2804] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:20.700628 24320 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:20.701120 24320 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:20.701279 22844 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:20.702544 24487 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:20.702823 22844 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:20.703110 24487 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:20.703728 24473 raft_consensus.cc:2749] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 3. Current term is 4.
I20260709 13:08:20.683909 22777 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33488 (request call id 264) took 2007 ms. Trace:
I20260709 13:08:20.697302 22993 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:08:20.697365 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:20.708093 22774 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48906 (request call id 196) took 2258 ms. Trace:
I20260709 13:08:20.709511 22774 rpcz_store.cc:276] 0709 13:08:18.441353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:18.441491 (+   138us) service_pool.cc:224] Handling call
0709 13:08:18.891363 (+449872us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b54002b6e54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4f090 00007f6135dedf3a 00007f613bba41cc 00007f613bca237e 00007f613a1632a4 00007f613a163232 00007f613a163161 00007f613a1630dc 00007f613a162392 00007f613412048c 00007f613411f8c6
0709 13:08:20.699813 (+1808450us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11473,"reported_tablets":2,"spinlock_wait_cycles":241348992}
I20260709 13:08:20.707372 24404 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:20.704406 24473 raft_consensus.cc:2764] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
I20260709 13:08:20.705384 22775 rpcz_store.cc:276] 0709 13:08:18.934860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:18.935028 (+   168us) service_pool.cc:224] Handling call
0709 13:08:19.098571 (+163543us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b5400002210. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc10b2 00007f613bc3aa89 00007f613bbd2da6 00007f613bba4088 00007f613bca237e 00007f613a1632a4 00007f613a163232 00007f613a163161 00007f613a1630dc
0709 13:08:20.689130 (+1590559us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11624,"reported_tablets":1,"spinlock_wait_cycles":335750016}
I20260709 13:08:20.713336 24414 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:20.711710 22918 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:20.713459 24431 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:20.712215 24404 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:20.718230 22844 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:20.723170 22843 leader_election.cc:400] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 2. Current term is 3.
I20260709 13:08:20.723667 22843 leader_election.cc:403] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:20.723106 24328 raft_consensus.cc:2804] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:20.724466 24463 raft_consensus.cc:2749] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 2. Current term is 3.
I20260709 13:08:20.721786 24404 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:20.725157 22920 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:20.728909 24328 raft_consensus.cc:697] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:08:20.729377 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 122820us
I20260709 13:08:20.729024 24308 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:20.697217 24490 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:20.730484 24308 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:20.732429 24308 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:20.730580 22919 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:20.729784 24431 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:20.734335 22776 catalog_manager.cc:5697] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 1 to 3, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 3 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:20.738678 22918 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:20.743602 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:20.728827 24414 raft_consensus.cc:697] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:20.744247 23625 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:08:20.749931 24525 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:20.751691 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:20.752316 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:20.752336 23622 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 1.
I20260709 13:08:20.736491 24431 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:20.793081 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 210989us
I20260709 13:08:20.786509 22918 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:20.795042 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:20.753101 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:20.730185 24328 consensus_queue.cc:237] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:20.799558 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 77501.9us
W20260709 13:08:20.808893 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 75136.2us
I20260709 13:08:20.819497 23620 raft_consensus.cc:1275] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:08:20.822546 24458 raft_consensus.cc:2749] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:20.823136 24404 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:20.825806 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:20.825999 22919 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:08:20.827369 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 76884us
I20260709 13:08:20.789386 24525 raft_consensus.cc:697] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:20.796383 24476 raft_consensus.cc:2804] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:20.801038 24412 raft_consensus.cc:2804] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:20.799609 24275 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
W20260709 13:08:20.809779 24544 log.cc:927] Time spent T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 13:08:20.796625 24500 raft_consensus.cc:2804] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:20.813081 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:20.810904 24490 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:20.836350 22920 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:20.838193 22994 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:20.842540 24306 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:20.811195 24487 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:20.844533 24275 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:20.871785 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:20.872028 22920 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:20.871670 24517 raft_consensus.cc:2764] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:08:20.872639 22946 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 2.
I20260709 13:08:20.873446 24503 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:20.873732 24517 raft_consensus.cc:2749] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:20.873898 24503 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:20.874464 24503 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:20.874701 24306 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:20.876943 24306 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:20.877575 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:20.707039 22777 rpcz_store.cc:276] 0709 13:08:18.676094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:18.676237 (+   143us) service_pool.cc:224] Handling call
0709 13:08:20.683660 (+2007423us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:19.264219 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:19.297211 (+ 32992us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:19.297245 (+    34us) write_op.cc:435] Acquired schema lock
0709 13:08:19.297264 (+    19us) tablet.cc:662] Decoding operations
0709 13:08:19.329694 (+ 32430us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:19.559431 (+229737us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:19.573653 (+ 14222us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:08:19.634741 (+ 61088us) tablet.cc:701] Row locks acquired
0709 13:08:19.640389 (+  5648us) write_op.cc:260] PREPARE: finished
0709 13:08:19.659658 (+ 19269us) write_op.cc:270] Start()
0709 13:08:19.671641 (+ 11983us) write_op.cc:276] Timestamp: P: 1783602499653569 usec, L: 0
0709 13:08:19.674112 (+  2471us) op_driver.cc:348] REPLICATION: starting
0709 13:08:19.739147 (+ 65035us) log.cc:844] Serialized 535 byte log entry
0709 13:08:20.480895 (+741748us) op_driver.cc:464] REPLICATION: finished
0709 13:08:20.630573 (+149678us) write_op.cc:301] APPLY: starting
0709 13:08:20.633329 (+  2756us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:20.633546 (+   217us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:20.633769 (+   223us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:20.634198 (+   429us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:20.634749 (+   551us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:08:20.634997 (+   248us) write_op.cc:312] APPLY: finished
0709 13:08:20.636423 (+  1426us) log.cc:844] Serialized 23 byte log entry
0709 13:08:20.637196 (+   773us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:20.637655 (+   459us) write_op.cc:454] Released schema lock
0709 13:08:20.637910 (+   255us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:20.684496 (+ 46586us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:20.684564 (+    68us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":5,"child_traces":[["op",{"apply.queue_time_us":126702,"apply.run_cpu_time_us":3146,"apply.run_wall_time_us":53930,"num_ops":1,"prepare.queue_time_us":359800,"prepare.run_cpu_time_us":54061,"prepare.run_wall_time_us":797363,"raft.queue_time_us":142199,"raft.run_cpu_time_us":16862,"raft.run_wall_time_us":187829,"replication_time_us":715643,"thread_start_us":712497,"threads_started":4,"wal-append.queue_time_us":448490}]]}
I20260709 13:08:20.834261 24476 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:20.814065 23619 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
W20260709 13:08:20.846586 24542 log.cc:927] Time spent T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
W20260709 13:08:20.847290 24543 log.cc:927] Time spent T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
W20260709 13:08:20.904402 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 78546.9us
I20260709 13:08:20.833623 24458 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:20.856448 24325 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:08:20.864388 24549 log.cc:927] Time spent T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:08:20.866771 24392 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:20.868567 22994 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:20.868942 24415 raft_consensus.cc:2749] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:08:20.869498 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:20.834332 24412 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:20.891548 24500 raft_consensus.cc:697] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:20.892366 24525 consensus_queue.cc:237] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:20.901240 24476 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:20.919107 24502 raft_consensus.cc:2749] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 3. Current term is 4.
I20260709 13:08:20.894371 24404 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:20.855078 23588 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:20.905057 22919 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:20.924130 24515 raft_consensus.cc:2749] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:08:20.924494 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 69530.5us
I20260709 13:08:20.872974 24275 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:20.926818 23588 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:08:20.927634 24328 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:20.927932 24091 raft_consensus.cc:2804] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:20.928278 24091 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:20.928246 24328 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:20.930984 24487 raft_consensus.cc:2749] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:20.932292 22919 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:20.933452 24517 consensus_queue.cc:1048] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:20.934365 23588 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:20.917928 24412 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:20.939873 24528 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:20.939604 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:20.940430 24490 raft_consensus.cc:2804] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:20.941006 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:20.928761 24091 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:20.930460 24328 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:08:20.946434 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 77934us
I20260709 13:08:20.937031 24325 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:20.862696 24320 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:20.970611 22844 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:20.972111 22844 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:20.972281 23588 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:20.968538 24306 raft_consensus.cc:2749] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:20.918449 24503 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:20.991106 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:20.955438 24490 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:20.950353 24392 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:20.993011 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 121067us
I20260709 13:08:20.992444 24528 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:21.020757 24464 raft_consensus.cc:2804] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:21.021178 24464 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:21.021564 24464 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:21.021625 24528 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:20.989703 22994 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:20.978040 22871 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 4.
I20260709 13:08:20.914885 24414 consensus_queue.cc:237] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:21.025344 22952 raft_consensus.cc:2368] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 2. Current term is 3.
I20260709 13:08:20.942891 24500 consensus_queue.cc:237] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:21.029352 24091 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:20.978492 22775 catalog_manager.cc:5697] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:21.006814 23023 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:21.034309 24490 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:20.950191 24458 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:21.047956 24091 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:21.022972 24481 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:21.052937 24458 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:21.016505 24552 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:21.055778 24552 raft_consensus.cc:697] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 4 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:21.056864 24320 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:21.056728 24552 consensus_queue.cc:237] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:21.056944 24503 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:21.055771 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 83535.5us
I20260709 13:08:21.059002 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:21.059268 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:21.016835 24407 raft_consensus.cc:2804] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:21.066926 24407 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:21.067310 24407 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:21.067364 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:21.068112 23618 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:08:21.079310 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:21.080052 22954 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:21.070366 22775 catalog_manager.cc:5697] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:21.082099 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:21.084569 23588 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:21.085788 24091 raft_consensus.cc:2749] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:21.060674 24528 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:21.076174 24481 raft_consensus.cc:697] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 4 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
W20260709 13:08:21.093061 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 107896us
I20260709 13:08:21.093818 22994 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:21.093775 24481 consensus_queue.cc:237] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:21.094812 24518 raft_consensus.cc:2804] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:21.095083 22994 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:21.095541 24518 raft_consensus.cc:697] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:21.097266 22994 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:21.098228 24416 consensus_queue.cc:1048] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:08:21.098541 24467 raft_consensus.cc:2749] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:21.096942 24328 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:21.099869 24328 raft_consensus.cc:697] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:21.101284 24328 consensus_queue.cc:237] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:21.104585 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:21.104918 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:21.105549 22877 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:35370
W20260709 13:08:21.105786 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 61405.9us
I20260709 13:08:21.106248 22877 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:21.099993 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:21.106739 22920 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:21.110319 24517 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:21.110908 24517 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:21.074631 24417 raft_consensus.cc:2749] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:21.096938 24518 consensus_queue.cc:237] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:21.122460 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:21.106763 22877 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:21.130367 22918 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:21.132047 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:21.133831 23025 raft_consensus.cc:2393] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 4: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:21.081804 24308 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:21.138441 23029 raft_consensus.cc:1275] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:08:21.147662 22953 raft_consensus.cc:1275] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:08:21.149155 22992 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:21.153661 23590 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:21.132894 24525 raft_consensus.cc:2749] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:21.156311 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:21.157358 23021 raft_consensus.cc:2368] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 3. Current term is 4.
I20260709 13:08:21.157202 24364 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
W20260709 13:08:21.160285 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 53592.9us
I20260709 13:08:21.137761 24515 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:21.142459 24503 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:21.163719 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:21.163859 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:21.151623 23620 raft_consensus.cc:1240] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:21.141955 24487 raft_consensus.cc:2749] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:21.175136 23025 raft_consensus.cc:1275] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:08:21.163450 24275 raft_consensus.cc:2804] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:21.177052 24275 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:21.177517 24275 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:21.180706 24392 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:21.169661 24517 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:21.182153 24325 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:21.182802 24503 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:21.185420 22950 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 2.
I20260709 13:08:21.188010 24515 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:21.190619 24515 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:21.161953 24476 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:21.157998 24467 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:21.196336 22954 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 4.
I20260709 13:08:21.211522 23023 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 4.
I20260709 13:08:21.158222 24528 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:21.244234 22872 raft_consensus.cc:2368] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 2. Current term is 3.
I20260709 13:08:21.201185 24467 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:21.222940 24412 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:21.276218 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 113715us
I20260709 13:08:21.208271 24404 consensus_queue.cc:1059] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 13:08:21.280391 22955 raft_consensus.cc:2368] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 2. Current term is 3.
I20260709 13:08:21.201365 24364 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:21.277976 23588 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:08:21.283305 22920 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:08:21.279682 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 126121us
I20260709 13:08:21.277065 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:21.297180 22879 raft_consensus.cc:1275] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:08:21.279309 24467 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:21.301513 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:21.313403 23021 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:21.313975 24490 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:21.314447 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:21.315692 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:21.301406 24517 raft_consensus.cc:2749] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:21.298738 24500 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:21.334559 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:21.334826 24500 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:21.321839 22992 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:21.336963 24500 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:21.337517 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:21.339082 24362 raft_consensus.cc:2804] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:21.339462 24362 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:21.339830 24362 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:21.343122 24275 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:21.321153 24412 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:21.322392 24565 raft_consensus.cc:2804] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:21.319494 24308 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:21.367868 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 67053.5us
I20260709 13:08:21.321776 24528 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:21.372411 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:21.371407 22873 raft_consensus.cc:1275] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 13:08:21.373019 22950 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:21.348791 23618 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 3.
I20260709 13:08:21.375314 24503 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:21.375754 24565 raft_consensus.cc:697] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 4 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:21.316632 24490 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:21.376850 24565 consensus_queue.cc:237] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:21.376859 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:21.378079 24476 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:21.382226 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:21.382591 22877 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:21.329135 24560 raft_consensus.cc:3055] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:21.383495 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:21.330345 22953 raft_consensus.cc:1275] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:08:21.384193 23620 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:21.322516 24553 consensus_queue.cc:1048] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:08:21.332883 24407 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:21.322969 24559 log.cc:927] Time spent T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.158s	user 0.001s	sys 0.000s
I20260709 13:08:21.322600 23625 raft_consensus.cc:1240] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:21.386108 24407 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:21.348281 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:21.332824 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:21.386775 23619 raft_consensus.cc:1240] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:21.387161 22952 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 2.
I20260709 13:08:21.361227 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:21.388978 22994 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:21.297075 24552 consensus_queue.cc:1048] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 13:08:21.391578 24328 raft_consensus.cc:2749] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:21.394148 23025 raft_consensus.cc:1275] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:08:21.387161 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:21.395619 22946 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 2.
I20260709 13:08:21.397079 24502 consensus_queue.cc:1048] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:08:21.397502 22877 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:21.400645 22950 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 4.
I20260709 13:08:21.362263 22918 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:08:21.409189 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 47001.5us
I20260709 13:08:21.409791 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:21.410856 22918 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:21.411911 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:21.412179 24308 raft_consensus.cc:2804] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:21.412708 24308 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:21.413188 24308 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:21.413635 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:21.415789 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:21.417507 24487 raft_consensus.cc:2749] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:21.369904 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:21.418380 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:21.371380 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:21.421197 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:21.383610 24560 raft_consensus.cc:740] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:21.422425 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:21.422920 24560 consensus_queue.cc:260] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:21.424165 24560 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:21.323222 24417 consensus_queue.cc:1048] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:08:21.385987 22843 leader_election.cc:400] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 3. Current term is 4.
I20260709 13:08:21.387745 23622 raft_consensus.cc:1275] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:08:21.401059 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:21.401202 24275 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:21.402118 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:21.422058 24364 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:21.412715 22879 raft_consensus.cc:2393] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 3: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:21.365823 23588 leader_election.cc:400] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 2. Current term is 3.
I20260709 13:08:21.420395 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:21.491170 23025 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:21.493436 23588 leader_election.cc:403] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:21.499450 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:21.512655 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:21.408017 24362 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:21.512655 24464 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:21.424823 24476 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:21.527922 22875 raft_consensus.cc:2393] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 3: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:21.533227 24568 consensus_queue.cc:1048] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 13:08:21.518641 24407 raft_consensus.cc:2749] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 2. Current term is 3.
W20260709 13:08:21.537918 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 178998us
I20260709 13:08:21.545954 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:21.546607 22950 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 2.
W20260709 13:08:21.543087 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 157218us
I20260709 13:08:21.538767 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:21.542088 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:21.547399 22876 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:21.552963 23588 leader_election.cc:400] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 election: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 2. Current term is 3.
I20260709 13:08:21.553313 23588 leader_election.cc:403] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 13:08:21.559520 24362 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:21.560501 22879 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 4.
I20260709 13:08:21.561415 22918 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:21.562142 24525 raft_consensus.cc:2804] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:21.562528 24525 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:21.562878 24525 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:21.549849 24328 consensus_queue.cc:1059] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
I20260709 13:08:21.565740 24568 raft_consensus.cc:3055] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:21.566105 24568 raft_consensus.cc:740] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:21.566926 24568 consensus_queue.cc:260] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:21.568246 24568 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:21.569972 24476 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:21.575914 24464 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:21.580745 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:21.581631 23623 raft_consensus.cc:2410] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:08:21.581341 24481 consensus_queue.cc:1048] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:21.583168 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:21.584033 22950 raft_consensus.cc:2393] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 3: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:21.584218 23622 raft_consensus.cc:1275] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:08:21.588515 23590 leader_election.cc:400] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 election: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 2. Current term is 3.
I20260709 13:08:21.595556 22993 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:21.605091 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:21.605466 24467 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:21.606532 24364 raft_consensus.cc:2749] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:21.609146 23029 raft_consensus.cc:1240] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:21.609405 23588 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:21.610491 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:21.615916 24476 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:21.620193 23590 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:21.621209 22953 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:21.675354 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:21.676918 22952 raft_consensus.cc:2393] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 3: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
W20260709 13:08:21.710337 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 90342.7us
I20260709 13:08:21.617048 24362 consensus_queue.cc:1048] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.096s
I20260709 13:08:21.713989 24407 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:21.592219 24525 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:21.714542 24407 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:21.716822 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 96715.1us
I20260709 13:08:21.718894 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:21.719130 23020 raft_consensus.cc:1275] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:08:21.712680 22948 raft_consensus.cc:1275] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 13:08:21.721630 22872 raft_consensus.cc:1275] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:08:21.753159 24467 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:21.755731 24475 raft_consensus.cc:2804] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:21.756106 24407 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:21.779142 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 199073us
I20260709 13:08:21.606575 23620 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:21.787382 22876 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
I20260709 13:08:21.718041 24502 raft_consensus.cc:2804] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:21.592727 24517 consensus_queue.cc:1059] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.168s
I20260709 13:08:21.760113 22948 pending_rounds.cc:85] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting all ops after (but not including) 1
I20260709 13:08:21.822075 22950 raft_consensus.cc:1240] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:08:21.608150 24455 consensus_queue.cc:1048] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.241s
I20260709 13:08:21.839521 24502 raft_consensus.cc:697] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 4 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:21.840720 24502 consensus_queue.cc:237] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:21.841739 24525 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:21.809744 22948 pending_rounds.cc:107] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:08:21.842495 22948 raft_consensus.cc:2889] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:08:21.846185 24575 log.cc:927] Time spent T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.238s	user 0.002s	sys 0.000s
I20260709 13:08:21.847215 24561 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:21.848074 23623 raft_consensus.cc:1240] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 4 LEADER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:21.848750 24364 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:21.849428 24578 log.cc:927] Time spent T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:08:21.849498 24364 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:21.853191 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:21.854560 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:21.856362 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:21.856355 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:21.857178 22919 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:21.857976 24487 raft_consensus.cc:2804] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:21.858448 24487 raft_consensus.cc:697] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:21.859345 24487 consensus_queue.cc:237] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:21.860436 24364 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:21.847842 24561 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:21.866047 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:21.867556 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:21.794806 24501 raft_consensus.cc:2749] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 2. Current term is 3.
I20260709 13:08:21.885010 22953 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
I20260709 13:08:21.901190 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:21.795322 24475 raft_consensus.cc:697] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:21.944952 22953 raft_consensus.cc:1240] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:21.965404 22951 raft_consensus.cc:1240] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 2. Current term is 3. Ops: []
W20260709 13:08:21.966555 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:21.957005 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:21.967562 22844 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:21.969221 22947 raft_consensus.cc:1275] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:08:21.968539 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:21.971719 24404 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:21.972218 24404 raft_consensus.cc:697] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 3 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:21.973145 24404 consensus_queue.cc:237] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:21.972208 23026 raft_consensus.cc:2393] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 4: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:21.807121 24476 consensus_queue.cc:1048] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.137s
I20260709 13:08:21.980967 23623 raft_consensus.cc:1275] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:08:21.984338 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:08:21.969329 24308 consensus_meta.cc:287] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.556s	user 0.007s	sys 0.002s
I20260709 13:08:21.985709 23029 raft_consensus.cc:1275] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:08:21.986330 22951 raft_consensus.cc:1275] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:08:21.990667 23621 raft_consensus.cc:2393] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 3: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:21.983186 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:21.991900 24561 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:21.992568 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:21.993189 22947 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:21.988814 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:08:21.995463 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:22.074787 24575 log.cc:927] Time spent T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:08:22.135159 22955 raft_consensus.cc:1275] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:08:22.070482 24553 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.075789 23617 raft_consensus.cc:1275] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 13:08:22.128914 22843 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.63.2:33309 timed out after 1.505s (SENT) [suppressed 10 similar messages]
I20260709 13:08:21.995023 24308 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:22.138381 24475 consensus_queue.cc:237] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:22.156458 22947 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:08:22.168583 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:22.174705 22954 raft_consensus.cc:1240] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 2. Current term is 3. Ops: []
W20260709 13:08:22.178272 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 196444us
W20260709 13:08:22.179701 24580 log.cc:927] Time spent T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:08:22.179827 23588 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:22.181867 24565 raft_consensus.cc:2749] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:22.142760 24553 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:22.186271 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:22.189744 24308 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:22.193675 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:22.198541 22880 raft_consensus.cc:1240] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:22.202739 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:08:22.210026 24584 log.cc:927] Time spent T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
W20260709 13:08:22.212932 24576 log.cc:927] Time spent T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.607s	user 0.010s	sys 0.017s
I20260709 13:08:22.185667 24561 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:22.220508 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:08:22.243721 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 63960us
I20260709 13:08:22.244961 22876 raft_consensus.cc:1275] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:08:22.246196 22876 raft_consensus.cc:1240] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:22.249362 24560 consensus_queue.cc:1048] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:08:22.253351 24591 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.254019 24591 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:22.256582 24591 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:22.257850 23588 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:22.239925 24517 consensus_queue.cc:1048] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 13:08:22.261322 24591 consensus_queue.cc:1048] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:08:22.269541 22920 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:22.275777 22920 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:22.230881 24561 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:22.277114 24582 log.cc:927] Time spent T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 13:08:22.277218 24416 consensus_queue.cc:1048] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:22.278858 24553 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:22.281533 22952 raft_consensus.cc:1275] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:08:22.283335 24517 raft_consensus.cc:2749] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:22.259737 24475 raft_consensus.cc:2749] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:08:22.292604 22920 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:22.293423 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 164527us
I20260709 13:08:22.297406 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:22.306763 24476 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.310714 24476 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:22.314776 22920 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:22.315686 22920 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:22.318634 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:22.319244 22946 raft_consensus.cc:2393] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 3: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:22.323573 24308 raft_consensus.cc:2749] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:22.325897 24464 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:22.325906 24561 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:22.331159 22920 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:22.296900 24412 raft_consensus.cc:2749] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:22.338171 22920 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:22.338894 22951 raft_consensus.cc:1275] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:08:22.339651 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:22.326747 24464 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:22.344918 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:22.345481 22947 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 4.
I20260709 13:08:22.316550 24517 raft_consensus.cc:2749] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:22.348433 23588 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:22.327795 24553 raft_consensus.cc:2749] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:22.333025 24308 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.355840 22952 raft_consensus.cc:1275] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:08:22.337719 24556 raft_consensus.cc:2749] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:22.332954 24591 consensus_queue.cc:1048] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 13:08:22.360925 22951 raft_consensus.cc:1240] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:08:22.366590 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:22.356189 24407 raft_consensus.cc:2804] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:22.355598 24412 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:22.350926 24475 consensus_queue.cc:1048] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 13:08:22.345423 24455 consensus_queue.cc:1048] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.020s
I20260709 13:08:22.370918 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:22.377216 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:22.350992 24561 consensus_queue.cc:1059] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 13:08:22.362287 24476 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:22.355396 24571 consensus_queue.cc:1059] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 13:08:22.359472 24308 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:22.390702 24416 raft_consensus.cc:3055] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:22.391112 24416 raft_consensus.cc:740] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:22.380383 24412 raft_consensus.cc:697] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:22.392086 24308 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:22.379675 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:22.382656 24556 consensus_queue.cc:1059] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:08:22.383613 24561 raft_consensus.cc:3055] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:22.393024 24561 raft_consensus.cc:740] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:22.393304 24556 raft_consensus.cc:3055] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:22.393687 24556 raft_consensus.cc:740] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:22.393832 24561 consensus_queue.cc:260] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:22.394485 24556 consensus_queue.cc:260] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:22.395131 24561 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:22.395673 24556 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:22.383742 24517 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:22.372128 24407 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:22.397190 24407 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:22.380023 24487 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:22.400923 24487 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:22.392064 24416 consensus_queue.cc:260] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:22.402981 24487 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:22.403796 24416 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:22.411085 22953 raft_consensus.cc:1275] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:08:22.412930 24320 raft_consensus.cc:3055] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:22.413316 24320 raft_consensus.cc:740] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:22.406252 24517 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:22.414095 24320 consensus_queue.cc:260] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:22.415372 24320 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:22.416426 24517 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:22.417357 24407 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:22.417815 22871 raft_consensus.cc:1275] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:08:22.419468 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:22.420310 22880 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:22.409018 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:22.389205 23028 raft_consensus.cc:1240] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:22.429991 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:22.431532 22953 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:08:22.432011 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 78295.2us
I20260709 13:08:22.401955 24464 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:22.444624 22953 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 5.
I20260709 13:08:22.460027 23588 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:22.471320 24407 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:22.468348 24591 raft_consensus.cc:2804] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:08:22.473562 24591 raft_consensus.cc:697] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:22.474524 24591 consensus_queue.cc:237] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:22.469336 24412 consensus_queue.cc:237] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:22.477495 24556 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.478106 24556 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:22.482116 24464 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.482630 24464 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:22.484511 24464 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:22.484889 24556 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:22.487991 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 107108us
I20260709 13:08:22.439378 24475 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.431650 24517 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.509557 22875 raft_consensus.cc:1275] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:08:22.486948 24464 consensus_queue.cc:1059] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 13:08:22.546470 22951 raft_consensus.cc:1240] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 LEADER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:22.548475 23625 raft_consensus.cc:1240] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 LEADER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:08:22.503614 24475 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:22.532557 24597 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:22.521947 24517 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:22.553357 24599 consensus_queue.cc:1048] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.191s
W20260709 13:08:22.744102 24594 log.cc:927] Time spent T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.274s	user 0.000s	sys 0.001s
W20260709 13:08:22.747779 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:22.746673 24416 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:22.824956 24416 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:22.827643 24416 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:22.829124 23621 raft_consensus.cc:1275] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:08:22.757953 24571 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:22.833005 24571 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:22.835680 24571 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:08:22.902191 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:22.840601 24475 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:22.937397 24587 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:22.747042 24597 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:23.003146 24597 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:23.000871 24571 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
W20260709 13:08:23.005980 24320 consensus_meta.cc:287] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.567s	user 0.009s	sys 0.002s
I20260709 13:08:23.122695 24320 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:23.023669 24328 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:23.138754 24475 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:23.145780 22950 raft_consensus.cc:1275] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:08:23.139834 22953 raft_consensus.cc:1275] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:08:23.146163 24475 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:23.147236 24517 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:23.148902 24475 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:23.148820 24517 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:23.145447 24407 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:23.154278 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:23.154740 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:23.070923 24597 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:08:23.161156 22918 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.003s	sys 0.025s
W20260709 13:08:23.161613 22918 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.657s	user 0.003s	sys 0.025s
I20260709 13:08:23.004647 24587 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:23.123405 24320 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:23.125048 24571 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:23.151368 23022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 216) took 1262 ms. Trace:
I20260709 13:08:23.139021 24416 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:23.156399 24407 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:23.195185 23022 rpcz_store.cc:276] 0709 13:08:21.875363 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:21.883658 (+  8295us) service_pool.cc:224] Handling call
0709 13:08:21.989446 (+105788us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:22.143083 (+153637us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:22.145042 (+  1959us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:22.214969 (+ 69927us) log.cc:844] Serialized 26 byte log entry
0709 13:08:22.342344 (+127375us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:22.343950 (+  1606us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:22.353156 (+  9206us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:23.037607 (+684451us) raft_consensus.cc:1685] finished
0709 13:08:23.078672 (+ 41065us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:23.133563 (+ 54891us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13146,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":2891,"thread_start_us":71720,"threads_started":1,"wal-append.queue_time_us":134340}
I20260709 13:08:23.207120 24407 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:23.207857 24407 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:23.163865 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:23.209820 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:23.138748 24328 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:23.246594 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:23.157748 24501 consensus_queue.cc:1048] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:08:23.157943 24475 consensus_queue.cc:1048] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:08:23.200338 24552 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:23.160945 24517 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:23.201427 24412 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:23.164117 24597 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:23.254701 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:23.249370 23619 raft_consensus.cc:1275] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:08:23.208470 24407 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:23.216611 24416 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:23.263646 24587 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:23.264281 24320 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:23.266005 24328 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:23.251833 24552 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:23.271086 24552 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:23.266201 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:23.266330 24517 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:23.266739 24407 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:23.267341 24571 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:23.267377 24597 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:23.278458 22952 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.749s	user 0.005s	sys 0.001s
W20260709 13:08:23.278904 22952 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.750s	user 0.005s	sys 0.001s
I20260709 13:08:23.282598 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:23.283103 24328 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:23.283324 22947 raft_consensus.cc:2393] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 3: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:23.283663 24328 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:23.284348 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:23.284940 22946 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:08:23.285193 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:23.285657 24455 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:23.285890 24328 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:23.286187 24455 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:23.287135 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:23.288331 24455 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:23.290642 24364 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:23.291208 24364 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:23.293248 24364 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:23.295750 22992 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:08:23.296561 24571 consensus_queue.cc:1048] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:08:23.299047 24571 raft_consensus.cc:2804] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:23.299456 24571 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:23.299891 24571 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:23.304173 22992 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:08:23.305250 24455 raft_consensus.cc:2804] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:23.307178 24455 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:23.307729 24455 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:23.259692 24412 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:23.326587 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 44538.9us
W20260709 13:08:23.327394 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 70557.3us
I20260709 13:08:23.330113 24416 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:23.331118 24412 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:23.349869 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:23.345988 24455 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:23.358319 24597 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:23.358999 24597 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:23.361055 24614 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:23.361624 24614 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:23.369673 24597 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:23.370388 24597 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:23.371119 24597 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:23.375263 24597 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:23.375793 24597 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:23.376291 24597 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:23.378513 24597 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:23.443292 24597 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:23.443871 24597 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:23.446571 24597 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:23.593225 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 289334us
I20260709 13:08:23.594580 22992 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:08:23.692205 24621 raft_consensus.cc:2804] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:08:23.694321 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 98230.2us
I20260709 13:08:23.808566 24621 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:23.809056 24621 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:23.810125 24328 consensus_queue.cc:1048] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:23.813279 24328 consensus_queue.cc:1048] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:23.812954 22992 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:23.815260 24328 raft_consensus.cc:2749] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:23.817982 24328 consensus_queue.cc:1048] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:23.820838 24621 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:23.822278 24328 consensus_queue.cc:1048] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:23.823258 24621 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:23.821395 22992 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:23.824652 22992 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:23.825451 24621 raft_consensus.cc:2749] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:23.825829 22992 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:23.826104 24621 raft_consensus.cc:2749] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:23.826785 24621 raft_consensus.cc:2804] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:23.827154 24621 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:23.827471 24621 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:23.836602 24621 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:23.839206 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:23.839494 24621 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:23.840474 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:24.004582 24325 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:24.005688 24325 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:24.010141 24325 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:24.021979 23026 raft_consensus.cc:3055] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:08:24.022938 23026 raft_consensus.cc:740] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:24.026412 23026 consensus_queue.cc:260] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:24.554749 23026 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:08:24.612033 21761 debug-util.cc:398] Leaking SignalData structure 0x7b080000fd80 after lost signal to thread 23622
W20260709 13:08:24.612725 21761 kernel_stack_watchdog.cc:198] Thread 23622 stuck at /apache/dev/git/kudu/src/kudu/util/thread.cc:642 for 274ms:
Kernel stack:
(could not read kernel stack)
User stack:
<Timed out: thread did not respond: maybe it is blocking signals>
I20260709 13:08:24.717476 24621 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:24.718253 24621 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:24.729890 23026 raft_consensus.cc:1275] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:08:24.775635 23026 pending_rounds.cc:85] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75: Aborting all ops after (but not including) 5
I20260709 13:08:24.775894 23026 pending_rounds.cc:107] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:08:24.776115 23026 raft_consensus.cc:2889] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:08:24.744874 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:24.748267 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:08:24.768168 22901 debug-util.cc:398] Leaking SignalData structure 0x7b080011e800 after lost signal to thread 21759
I20260709 13:08:24.779459 24591 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:24.780145 24591 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:24.782400 24591 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:24.782918 24591 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:24.783396 24591 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:24.769078 24626 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:24.785472 24591 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:08:24.774329 22920 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.009s	sys 0.005s
I20260709 13:08:24.786237 24591 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:08:24.786387 22920 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.630s	user 0.010s	sys 0.005s
W20260709 13:08:24.786912 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 631194us
I20260709 13:08:24.786885 24591 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:24.788213 23020 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 5.
I20260709 13:08:24.785202 24626 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:24.789492 24591 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:24.790154 24626 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:24.791528 24621 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:24.792876 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:24.798950 24320 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:24.788216 23024 raft_consensus.cc:1240] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:24.799721 24320 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:24.801111 23028 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 3.
I20260709 13:08:24.802492 24320 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:24.804749 24626 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:24.805524 24626 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:24.808481 24626 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:24.811062 24626 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:24.811574 23618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 354) took 2309 ms. Trace:
I20260709 13:08:24.811847 24591 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:24.812494 24591 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:24.812183 23618 rpcz_store.cc:276] 0709 13:08:22.502224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.503493 (+  1269us) service_pool.cc:224] Handling call
0709 13:08:24.810726 (+2307233us) spinlock_profiling.cc:243] Waited 4.3 ms on lock 0x7b58001c4d98. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0a22 00007f613bbd9c09 00007f613ad33ee0 00007f613adfd0ab 00007f613adfd131 00007f613adbefc0 00007f613ada9929 00007f613ada93d2 00007f6134320bd4
0709 13:08:24.810995 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:24.811182 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:24.811212 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:24.811256 (+    44us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:24.811293 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:24.811347 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:24.811479 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9887488}
I20260709 13:08:24.814929 24591 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:24.811609 24626 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:24.811944 24629 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:24.817514 24629 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:24.818257 24630 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:24.820286 24629 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:24.818819 24630 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:24.821014 24629 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:24.821861 22954 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 1369) took 2950 ms. Trace:
I20260709 13:08:24.821677 24629 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:24.822587 24630 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:24.822283 22954 rpcz_store.cc:276] 0709 13:08:21.871414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:21.880465 (+  9051us) service_pool.cc:224] Handling call
0709 13:08:21.969884 (+ 89419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:22.175278 (+205394us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:24.821613 (+2646335us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b5400074ed4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:24.821780 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":470426880}
I20260709 13:08:24.823163 24630 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:24.823642 24630 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:24.824263 24629 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:24.824968 24629 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:24.825498 24629 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:24.827106 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:24.827620 24632 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:24.828119 24632 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:24.828439 24629 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:24.829191 24629 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:24.829802 24629 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:24.832384 24629 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:24.833045 24629 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:24.833624 24630 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:24.833559 24629 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:24.830240 24632 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:24.837833 24629 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:24.838653 24476 raft_consensus.cc:3055] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:24.839036 24476 raft_consensus.cc:740] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:24.838994 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:24.839917 24476 consensus_queue.cc:260] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:24.839968 24626 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:24.841133 24476 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:24.841382 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:08:24.845794 23622 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.508s	user 0.012s	sys 0.030s
W20260709 13:08:24.846259 23622 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.509s	user 0.012s	sys 0.030s
I20260709 13:08:24.846954 23622 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60044 (request call id 402) took 1598 ms. Trace:
I20260709 13:08:24.847293 23622 rpcz_store.cc:276] 0709 13:08:23.249571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:23.249715 (+   144us) service_pool.cc:224] Handling call
0709 13:08:23.254656 (+  4941us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:23.332283 (+ 77627us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:23.332316 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:23.337144 (+  4828us) log.cc:844] Serialized 513 byte log entry
0709 13:08:24.846631 (+1509487us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:08:24.846664 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:24.846724 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:24.846756 (+    32us) raft_consensus.cc:1685] finished
0709 13:08:24.846766 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:24.846885 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14477,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":121,"thread_start_us":1584055,"threads_started":2,"wal-append.queue_time_us":51579,"wal-append.run_cpu_time_us":2481,"wal-append.run_wall_time_us":1004596}
I20260709 13:08:24.849000 22953 raft_consensus.cc:1240] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:08:24.851362 24614 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:24.851547 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:24.852133 22948 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:08:24.852747 22918 connection.cc:256] RPC call timeout handler was delayed by -1.0733995990000267s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15410000000000001s for a response
W20260709 13:08:24.853170 22918 connection.cc:256] RPC call timeout handler was delayed by -1.06650455099998s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1534s for a response
I20260709 13:08:24.858305 24629 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:24.858811 24629 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:24.859141 22948 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
I20260709 13:08:24.861574 24629 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:24.862202 24557 consensus_queue.cc:1059] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:24.863497 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:24.866370 24476 raft_consensus.cc:3055] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:24.866763 24476 raft_consensus.cc:740] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:24.870793 22953 raft_consensus.cc:2393] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 3: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:24.872956 22992 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:24.867547 24476 consensus_queue.cc:260] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:24.875452 24416 raft_consensus.cc:2804] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:08:24.876035 24416 raft_consensus.cc:697] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 5 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:24.878482 24416 consensus_queue.cc:237] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:24.887426 22776 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33478 (request call id 274) took 4393 ms. Trace:
I20260709 13:08:24.769421 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:24.887866 22776 rpcz_store.cc:276] 0709 13:08:20.494492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:20.494791 (+   299us) service_pool.cc:224] Handling call
0709 13:08:20.639394 (+144603us) spinlock_profiling.cc:243] Waited 80.7 ms on lock 0x7b54002b6e54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4f090 00007f6135dedf3a 00007f613bba41cc 00007f613bca237e 00007f613a1632a4 00007f613a163232 00007f613a163161 00007f613a1630dc 00007f613a162392 00007f613412048c 00007f613411f8c6
0709 13:08:24.887316 (+4247922us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:21.028905 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:21.029610 (+   705us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:21.029667 (+    57us) write_op.cc:435] Acquired schema lock
0709 13:08:21.029685 (+    18us) tablet.cc:662] Decoding operations
0709 13:08:21.030157 (+   472us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:21.030291 (+   134us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:21.030336 (+    45us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:08:21.030684 (+   348us) tablet.cc:701] Row locks acquired
0709 13:08:21.030741 (+    57us) write_op.cc:260] PREPARE: finished
0709 13:08:21.030935 (+   194us) write_op.cc:270] Start()
0709 13:08:21.072780 (+ 41845us) write_op.cc:276] Timestamp: P: 1783602501030858 usec, L: 0
0709 13:08:21.074311 (+  1531us) op_driver.cc:348] REPLICATION: starting
0709 13:08:21.164285 (+ 89974us) log.cc:844] Serialized 526 byte log entry
0709 13:08:22.309607 (+1145322us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b54002b6ca8. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613c1a19e1 00007f613c1a064c 00007f613981c3bb 00007f613981e71a 00007f613981e6a8 00007f613981e628 00007f613981e5e3 00007f613981d7fc 00007f613bc50626 00007f613bc4add8 00007f6135df1310
0709 13:08:22.311097 (+  1490us) op_driver.cc:464] REPLICATION: finished
0709 13:08:22.539305 (+228208us) write_op.cc:301] APPLY: starting
0709 13:08:22.541083 (+  1778us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:22.551003 (+  9920us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:22.551034 (+    31us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:22.551293 (+   259us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:22.716054 (+164761us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 13:08:22.744829 (+ 28775us) write_op.cc:312] APPLY: finished
0709 13:08:22.745094 (+   265us) log.cc:844] Serialized 23 byte log entry
0709 13:08:22.745311 (+   217us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:22.745407 (+    96us) write_op.cc:454] Released schema lock
0709 13:08:22.745471 (+    64us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:22.745931 (+   460us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:22.745970 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":17804,"reported_tablets":2,"spinlock_wait_cycles":185559936,"child_traces":[["op",{"apply.queue_time_us":212252,"apply.run_cpu_time_us":2059,"apply.run_wall_time_us":213693,"num_ops":1,"prepare.queue_time_us":143753,"prepare.run_cpu_time_us":3984,"prepare.run_wall_time_us":202353,"replication_time_us":717583,"spinlock_wait_cycles":25001344,"thread_start_us":129585,"threads_started":2}]]}
I20260709 13:08:24.903976 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:24.907811 23622 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 1.
I20260709 13:08:24.908466 23022 raft_consensus.cc:1275] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:08:24.909108 23022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 227) took 1081 ms. Trace:
I20260709 13:08:24.909426 23022 rpcz_store.cc:276] 0709 13:08:23.827339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:24.907813 (+1080474us) service_pool.cc:224] Handling call
0709 13:08:24.908146 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:24.908869 (+   723us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:24.909045 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:08:24.911580 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 76265us
W20260709 13:08:24.912037 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:24.913291 22843 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:08:24.920145 22919 connection.cc:256] RPC call timeout handler was delayed by -1.1131257639999603s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15410000000000001s for a response
I20260709 13:08:24.920830 24632 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:24.921391 24630 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:24.923645 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:24.924484 23622 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:24.918099 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:24.927538 22843 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:24.918603 24629 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:24.919584 24560 consensus_queue.cc:1048] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 13:08:24.935250 24633 raft_consensus.cc:2749] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:24.938428 24629 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:24.927649 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:24.940928 23617 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:24.928928 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:24.933965 24630 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:24.945114 24630 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:24.945672 24630 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:24.945286 24633 consensus_queue.cc:1048] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:08:24.946889 24630 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:24.927579 24557 raft_consensus.cc:2749] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:24.950528 24630 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:24.951399 24630 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:24.952198 24629 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:24.930055 24556 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:24.952325 24630 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:24.952926 24556 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:24.926396 24632 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:24.955545 24556 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:24.956539 24632 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:24.963722 22952 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 1755) took 1682 ms. Trace:
I20260709 13:08:24.964130 22952 rpcz_store.cc:276] 0709 13:08:23.281291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:24.962986 (+1681695us) service_pool.cc:224] Handling call
0709 13:08:24.963283 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:24.963439 (+   156us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:24.963463 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:24.963501 (+    38us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:08:24.963531 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:24.963579 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:24.963669 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:24.968973 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:24.956701 24630 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:24.974397 22919 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:24.975004 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:24.975725 23621 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 352) took 2474 ms. Trace:
I20260709 13:08:24.976459 24557 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:24.976778 24632 raft_consensus.cc:2749] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:24.977924 24632 consensus_queue.cc:1048] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:24.977015 24557 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:24.981700 24557 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:24.976142 23621 rpcz_store.cc:276] 0709 13:08:22.501295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.523023 (+ 21728us) service_pool.cc:224] Handling call
0709 13:08:22.747409 (+224386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:22.829644 (+ 82235us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:24.975451 (+2145807us) spinlock_profiling.cc:243] Waited 14.6 ms on lock 0x7b54000cadd4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:24.975651 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":33564544}
I20260709 13:08:24.981712 24632 raft_consensus.cc:2955] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:24.983426 22874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 1687) took 2762 ms. Trace:
I20260709 13:08:24.983784 22874 rpcz_store.cc:276] 0709 13:08:22.220954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.223855 (+  2901us) service_pool.cc:224] Handling call
0709 13:08:24.982755 (+2758900us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b5800000c98. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0a22 00007f613bbd9c09 00007f613ad33ee0 00007f613adfd0ab 00007f613adfd131 00007f613adbefc0 00007f613ada9929 00007f613ada93d2 00007f6134320bd4
0709 13:08:24.982948 (+   193us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:24.983122 (+   174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:24.983146 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:24.983183 (+    37us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:08:24.983209 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:24.983256 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:24.983347 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29782272}
I20260709 13:08:24.986479 22951 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 1751) took 2614 ms. Trace:
I20260709 13:08:24.986805 24417 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:24.986852 22951 rpcz_store.cc:276] 0709 13:08:22.372236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.372372 (+   136us) service_pool.cc:224] Handling call
0709 13:08:22.380569 (+  8197us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:22.547080 (+166511us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:24.986285 (+2439205us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b54000ec9d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:24.986428 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":380861824}
I20260709 13:08:24.987411 24417 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:24.988996 22919 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:24.989712 24487 raft_consensus.cc:2804] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:24.990057 24487 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:24.990404 24487 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:24.990886 24455 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:24.991416 24455 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:24.991957 24328 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:24.991840 24455 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:24.994068 24467 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:24.994557 24467 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:24.997816 22919 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:24.999337 24487 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:25.000010 23027 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 226) took 1172 ms. Trace:
I20260709 13:08:25.000417 23027 rpcz_store.cc:276] 0709 13:08:23.827105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:24.999101 (+1171996us) service_pool.cc:224] Handling call
0709 13:08:24.999473 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:24.999674 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:24.999701 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:24.999745 (+    44us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:24.999781 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:24.999838 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:24.999952 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:25.001372 24487 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:24.992398 24328 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:25.002116 24629 raft_consensus.cc:2804] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:25.002509 24629 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:25.002849 24629 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:25.003293 24328 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:25.012192 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:25.012938 22948 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 3.
I20260709 13:08:25.013015 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:25.014743 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:25.015460 23029 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32794 (request call id 230) took 1187 ms. Trace:
I20260709 13:08:25.015740 23029 rpcz_store.cc:276] 0709 13:08:23.828102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:25.014496 (+1186394us) service_pool.cc:224] Handling call
0709 13:08:25.015407 (+   911us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:08:25.016278 24571 consensus_meta.cc:287] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 1.716s	user 0.009s	sys 0.016s
I20260709 13:08:25.017086 24502 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:25.017603 24502 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:25.018513 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:25.019153 23619 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 1.
I20260709 13:08:25.019634 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:25.019785 24502 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:25.020259 24502 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:25.020375 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:25.020849 24475 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:25.021018 22947 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:25.021350 24475 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:25.021859 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:25.022394 22949 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:25.023298 24475 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:24.996552 24467 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:25.020183 22953 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:25.026662 22955 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 150) took 2689 ms. Trace:
W20260709 13:08:25.028478 23588 connection.cc:256] RPC call timeout handler was delayed by -1.1936032129999603s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1505s for a response
I20260709 13:08:25.028254 22955 rpcz_store.cc:276] 0709 13:08:22.337227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.337357 (+   130us) service_pool.cc:224] Handling call
0709 13:08:22.348036 (+ 10679us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:22.391001 (+ 42965us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:22.391031 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:22.391447 (+   416us) log.cc:844] Serialized 26 byte log entry
0709 13:08:22.491767 (+100320us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:22.491815 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:25.026341 (+2534526us) spinlock_profiling.cc:243] Waited 42.7 ms on lock 0x7b54003796d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:25.026384 (+    43us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:25.026424 (+    40us) raft_consensus.cc:1685] finished
0709 13:08:25.026434 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:25.026613 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1645,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":119,"spinlock_wait_cycles":98117376,"thread_start_us":99973,"threads_started":1,"wal-append.queue_time_us":154130,"wal-append.run_cpu_time_us":2019,"wal-append.run_wall_time_us":1002073}
I20260709 13:08:25.032954 24560 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:25.033562 24560 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:25.035084 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:25.036175 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:25.038277 24560 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:25.039433 22946 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 3.
I20260709 13:08:25.040163 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:25.040716 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:25.041914 22877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 1258) took 2845 ms. Trace:
I20260709 13:08:25.089511 22877 rpcz_store.cc:276] 0709 13:08:22.196879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.198835 (+  1956us) service_pool.cc:224] Handling call
0709 13:08:22.257857 (+ 59022us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:22.334334 (+ 76477us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:22.336282 (+  1948us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:22.342435 (+  6153us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:08:22.345161 (+  2726us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:25.041669 (+2696508us) spinlock_profiling.cc:243] Waited 62.9 ms on lock 0x7b54003a0a54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:25.041720 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:25.041853 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":144728960}
I20260709 13:08:25.091188 22954 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 3.
I20260709 13:08:25.092327 24621 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:25.092499 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:25.092952 24621 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:25.093751 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:25.095680 24621 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:25.096284 24621 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:25.096787 24621 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:25.101992 24364 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:25.102741 24364 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:25.105655 24364 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:25.107739 24560 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:25.108304 24560 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:25.057727 22918 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:25.109942 24630 raft_consensus.cc:2749] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:08:25.109989 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 52340.1us
I20260709 13:08:25.110867 22918 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:25.111874 24630 raft_consensus.cc:2749] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 13:08:25.114768 22994 connection.cc:256] RPC call timeout handler was delayed by -1.4248756199999661s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1574s for a response
I20260709 13:08:25.116015 24364 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:25.116946 24364 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:25.120656 24364 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:25.044435 23588 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:25.044520 22951 raft_consensus.cc:1240] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 13:08:25.046531 24557 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:25.048923 24476 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:25.123080 24632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:58302 (request call id 80) took 7624 ms. Trace:
I20260709 13:08:25.123631 24632 rpcz_store.cc:276] 0709 13:08:17.499031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:17.502507 (+  3476us) service_pool.cc:224] Handling call
0709 13:08:17.569779 (+ 67272us) log.cc:844] Serialized 487 byte log entry
0709 13:08:25.122971 (+7553192us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":326,"mutex_wait_us":4869,"raft.queue_time_us":9634,"raft.run_cpu_time_us":1617,"raft.run_wall_time_us":17205,"thread_start_us":63823,"threads_started":1,"wal-append.queue_time_us":90196,"wal-append.run_cpu_time_us":1494,"wal-append.run_wall_time_us":1002719}
I20260709 13:08:25.127104 24560 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:25.127805 24560 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:25.130550 23021 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:08:25.130944 24476 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:08:25.131400 23021 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33418 (request call id 1320) took 2765 ms (client timeout 1809 ms). Trace:
I20260709 13:08:25.131492 24476 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:25.043448 22947 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:25.049412 24571 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:25.050526 24417 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:25.023366 22949 raft_consensus.cc:1240] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:25.053655 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:25.133324 24557 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:25.133908 22955 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 4.
I20260709 13:08:25.057430 24328 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:25.134435 24571 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:25.135244 24557 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:25.134887 24328 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:25.136865 24475 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:25.137089 24328 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:25.137266 24475 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:25.068957 24467 consensus_queue.cc:1059] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:25.137840 24467 consensus_queue.cc:1243] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer aed3f089ec8b49af9c8cf5cdee76e8e7 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260709 13:08:25.068871 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 56419.4us
W20260709 13:08:25.085917 22901 debug-util.cc:398] Leaking SignalData structure 0x7b080011eee0 after lost signal to thread 22812
I20260709 13:08:25.120728 22953 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:25.049072 24455 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:25.058768 24502 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:25.141944 24467 raft_consensus.cc:3055] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:25.142408 24467 raft_consensus.cc:740] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:25.143334 24502 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:25.143229 24467 consensus_queue.cc:260] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:25.143767 22901 debug-util.cc:398] Leaking SignalData structure 0x7b080038d1a0 after lost signal to thread 22904
W20260709 13:08:25.144207 22901 debug-util.cc:398] Leaking SignalData structure 0x7b080038d020 after lost signal to thread 22905
I20260709 13:08:25.144539 24467 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:08:25.146013 22901 debug-util.cc:398] Leaking SignalData structure 0x7b0800226580 after lost signal to thread 22979
I20260709 13:08:25.146759 22762 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 5ab4700268fc4c67b9930087c5da71c8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 13:08:25.147287 22901 debug-util.cc:398] Leaking SignalData structure 0x7b08002b87a0 after lost signal to thread 23053
W20260709 13:08:25.148901 22901 debug-util.cc:398] Leaking SignalData structure 0x7b080036f320 after lost signal to thread 23649
I20260709 13:08:25.155757 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:25.156653 22947 raft_consensus.cc:2393] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 4: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
W20260709 13:08:25.157939 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 113592us
I20260709 13:08:25.158496 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:25.101948 24629 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:25.165267 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:25.156438 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:25.166092 24621 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:25.166433 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:25.167151 23027 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 1.
I20260709 13:08:25.149430 24475 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:25.169100 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:08:25.172000 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 56486.2us
I20260709 13:08:25.169879 23029 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 2.
I20260709 13:08:25.160990 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:25.176018 22952 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:08:25.176823 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 237923us
W20260709 13:08:25.149328 22901 debug-util.cc:398] Leaking SignalData structure 0x7b080036f380 after lost signal to thread 23650
I20260709 13:08:25.178828 22919 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:25.184787 23588 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:25.187754 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:25.188740 24091 raft_consensus.cc:2804] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:25.189116 24091 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:25.189316 23026 raft_consensus.cc:2410] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:08:25.189422 24091 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:25.197467 22992 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:25.128364 24560 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:25.199931 24091 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:25.202467 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:25.205044 22919 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:25.206197 23588 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:08:25.132073 23021 rpcz_store.cc:269] 0709 13:08:22.366258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:22.367749 (+  1491us) service_pool.cc:224] Handling call
0709 13:08:24.968794 (+2601045us) spinlock_profiling.cc:243] Waited 7 ms on lock 0x7b58001ca198. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0e9c 00007f6132cc0a5b 00007f613bbd9c09 00007f613ad33ee0 00007f613adfd0ab 00007f613adfd131 00007f613adab0a0 00007f613adaa984 00007f61343256d4 00007f6134325662
0709 13:08:25.131280 (+162486us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18036352}
I20260709 13:08:25.199535 24557 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:25.254166 24647 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:25.278749 24647 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:25.279655 24476 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:25.280189 24476 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:25.199509 24632 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:25.281176 24632 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:25.281509 24632 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:25.281912 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:08:25.283030 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 81566.6us
I20260709 13:08:25.284132 24487 raft_consensus.cc:2804] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:08:25.255098 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 50559.3us
I20260709 13:08:25.270912 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:25.286244 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:25.286623 23029 raft_consensus.cc:2468] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 2.
I20260709 13:08:25.286653 22992 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:25.287621 22992 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:25.271833 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:08:25.289311 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:25.289476 22874 raft_consensus.cc:2468] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:25.289029 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:08:25.283092 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 78120.4us
I20260709 13:08:25.288568 24515 raft_consensus.cc:2804] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:25.288754 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:25.255720 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:25.283087 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:25.289979 22875 raft_consensus.cc:1275] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:08:25.292896 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 86776.1us
I20260709 13:08:25.294081 22873 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:25.296514 24091 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:08:25.290014 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 355953us
I20260709 13:08:25.298751 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:25.298838 22843 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:25.299847 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:25.276829 22948 raft_consensus.cc:2393] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 3: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:25.288311 24416 consensus_queue.cc:1059] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 13:08:25.199244 24621 consensus_queue.cc:1048] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.097s
I20260709 13:08:25.271142 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:25.284519 24487 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:25.288848 24362 raft_consensus.cc:2804] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:25.276561 24417 raft_consensus.cc:2804] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:25.305227 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:25.271389 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:25.296095 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:25.326385 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:25.328233 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:25.278075 24629 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:25.277333 22871 raft_consensus.cc:1275] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:08:25.303769 24091 consensus_queue.cc:1059] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:08:25.271031 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:25.277582 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:25.303902 23588 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:25.314941 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:25.320981 24328 consensus_queue.cc:1048] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:08:25.298156 24626 raft_consensus.cc:2804] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:25.328644 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:25.256203 24642 consensus_queue.cc:1048] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.068s
I20260709 13:08:25.330600 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:25.331362 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:25.277995 24557 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:25.334052 24644 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:25.333906 24475 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:25.301280 24647 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:25.324466 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:25.329322 22873 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:25.280619 24476 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:25.342344 23023 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:25.373440 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:25.320628 24487 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:25.307284 24646 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:25.322378 24417 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:25.347656 24091 raft_consensus.cc:3055] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:08:25.350060 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:25.359505 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:08:25.382382 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 83719.2us
I20260709 13:08:25.383024 22875 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 1.
I20260709 13:08:25.359339 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:08:25.368639 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 64807.2us
I20260709 13:08:25.387249 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:25.373095 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:25.374298 23624 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:25.342088 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:25.375557 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:25.368024 24515 raft_consensus.cc:697] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:25.379750 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:25.373133 24501 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:25.360406 23622 raft_consensus.cc:2955] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:25.392611 22874 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:25.346266 24626 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:25.330659 24652 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:25.382520 24091 raft_consensus.cc:740] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:25.384059 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:25.372536 22774 catalog_manager.cc:5697] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:25.410485 24091 consensus_queue.cc:260] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:25.367022 22775 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48906 (request call id 197) took 4498 ms. Trace:
I20260709 13:08:25.385962 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:25.374846 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:25.412515 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:25.413157 23623 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:25.369210 22877 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 1.
I20260709 13:08:25.368993 23620 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:25.394320 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:08:25.398648 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 100805us
I20260709 13:08:25.381561 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:25.387125 23022 raft_consensus.cc:2410] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:08:25.373402 24644 raft_consensus.cc:697] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:25.408914 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:25.400264 23588 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:25.412317 24091 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:25.413341 23026 raft_consensus.cc:2468] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:25.384228 24417 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:25.382083 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:25.430526 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:25.412428 22774 catalog_manager.cc:5697] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:25.417420 23023 raft_consensus.cc:2393] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 3: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:25.417508 22843 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:25.419272 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:25.425474 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:25.413703 24626 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:25.414549 24652 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:25.390072 24662 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:25.426947 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:25.442935 22874 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:25.415515 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:25.435467 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:25.401630 24501 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:25.444401 22878 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 1.
I20260709 13:08:25.447276 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:25.412708 22775 rpcz_store.cc:276] 0709 13:08:20.869035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:20.869605 (+   570us) service_pool.cc:224] Handling call
0709 13:08:20.894742 (+ 25137us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b5400002210. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc10b2 00007f613bc3aa89 00007f613bbd2da6 00007f613bba4088 00007f613bca237e 00007f613a1632a4 00007f613a163232 00007f613a163161 00007f613a1630dc
0709 13:08:25.366926 (+4472184us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:21.191499 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:21.192083 (+   584us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:21.192111 (+    28us) write_op.cc:435] Acquired schema lock
0709 13:08:21.192129 (+    18us) tablet.cc:662] Decoding operations
0709 13:08:21.192520 (+   391us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:21.192565 (+    45us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:21.192602 (+    37us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:08:21.192935 (+   333us) tablet.cc:701] Row locks acquired
0709 13:08:21.192953 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:08:21.276142 (+ 83189us) write_op.cc:270] Start()
0709 13:08:21.276264 (+   122us) write_op.cc:276] Timestamp: P: 1783602501276084 usec, L: 0
0709 13:08:21.276283 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:08:21.276889 (+   606us) log.cc:844] Serialized 907 byte log entry
0709 13:08:25.010661 (+3733772us) op_driver.cc:464] REPLICATION: finished
0709 13:08:25.090370 (+ 79709us) write_op.cc:301] APPLY: starting
0709 13:08:25.090458 (+    88us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:25.090595 (+   137us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:25.090758 (+   163us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:25.091235 (+   477us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:25.091409 (+   174us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:08:25.091458 (+    49us) write_op.cc:312] APPLY: finished
0709 13:08:25.091659 (+   201us) log.cc:844] Serialized 29 byte log entry
0709 13:08:25.296068 (+204409us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:25.304423 (+  8355us) write_op.cc:454] Released schema lock
0709 13:08:25.312726 (+  8303us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:25.346007 (+ 33281us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:25.356029 (+ 10022us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":9,"spinlock_wait_cycles":23551744,"child_traces":[["op",{"apply.queue_time_us":79479,"apply.run_cpu_time_us":12754,"apply.run_wall_time_us":249572,"num_ops":2,"prepare.queue_time_us":67184,"prepare.run_cpu_time_us":3524,"prepare.run_wall_time_us":89471,"replication_time_us":3733893,"thread_start_us":280076,"threads_started":2,"wal-append.queue_time_us":188418}]]}
I20260709 13:08:25.430464 24587 raft_consensus.cc:2749] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:25.454682 22875 raft_consensus.cc:1275] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:08:25.457181 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:25.430897 24560 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:25.426282 23028 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 1.
I20260709 13:08:25.433786 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:25.450994 24501 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:25.472018 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:25.392838 24646 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:25.472579 23027 raft_consensus.cc:2393] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 3: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:25.482811 23625 raft_consensus.cc:2410] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:08:25.476336 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:25.486297 23023 raft_consensus.cc:2410] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:08:25.486994 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:25.484863 24665 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:25.482280 24560 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:25.489507 24665 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:25.490307 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:25.491798 24652 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:25.467200 24659 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:25.492750 24659 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:25.491743 24665 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:25.511289 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:25.519590 22876 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 2.
I20260709 13:08:25.519656 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:25.520329 22875 raft_consensus.cc:1240] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 3. Current term is 4. Ops: []
W20260709 13:08:25.521852 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 94850.7us
I20260709 13:08:25.525588 23588 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:25.528149 24362 raft_consensus.cc:697] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:25.477034 24661 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:25.535354 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:25.557929 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:25.478015 24647 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:25.466672 24552 consensus_queue.cc:1048] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:08:25.558782 22879 raft_consensus.cc:2468] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:25.572609 22952 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:25.559872 23588 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:08:25.626884 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 67134.9us
I20260709 13:08:25.628247 23588 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:25.629074 24571 raft_consensus.cc:3055] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:25.629485 24571 raft_consensus.cc:740] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:25.630299 24571 consensus_queue.cc:260] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:25.631505 24571 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:25.628893 24490 raft_consensus.cc:2804] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:25.632956 24490 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
W20260709 13:08:25.636703 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 219223us
I20260709 13:08:25.518453 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:25.451140 24662 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:25.625445 22871 raft_consensus.cc:2410] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 5 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:08:25.640631 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:08:25.646234 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 159671us
I20260709 13:08:25.648303 22843 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:25.573889 24557 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:25.470757 24515 consensus_queue.cc:237] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:25.653237 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:25.637529 24552 raft_consensus.cc:2804] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:25.636075 24475 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:25.659178 22950 raft_consensus.cc:1275] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:08:25.640537 24490 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:25.641608 24476 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:25.575841 24647 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:25.693053 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:25.694741 24647 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:25.696107 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:25.696954 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:25.697281 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:25.697798 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:25.698122 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:25.698275 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:25.698587 22873 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 2.
I20260709 13:08:25.699343 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:25.699540 22954 raft_consensus.cc:1240] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:25.700349 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:25.699777 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:25.701363 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:25.701505 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:25.701977 22946 raft_consensus.cc:2376] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4. Re-sending same reply.
I20260709 13:08:25.702361 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:25.696970 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:25.703122 22947 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 3.
I20260709 13:08:25.703395 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:25.676162 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:25.704442 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:25.686887 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:25.688514 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:25.657907 24652 raft_consensus.cc:2749] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:25.698745 22871 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
W20260709 13:08:25.660955 24657 log.cc:927] Time spent T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.277s	user 0.001s	sys 0.000s
I20260709 13:08:25.705976 23026 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 3
I20260709 13:08:25.706794 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:25.561795 24644 consensus_queue.cc:237] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:25.717172 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:25.717805 22947 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:08:25.719233 22947 raft_consensus.cc:1240] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:25.709025 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:25.709642 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:25.720718 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:25.721959 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:25.722462 24662 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:25.723047 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:25.673631 22992 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:08:25.725186 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 51715.8us
I20260709 13:08:25.702005 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:25.636989 24362 consensus_queue.cc:237] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:25.558200 24673 consensus_queue.cc:1048] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.138s
I20260709 13:08:25.711078 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:25.729743 22875 raft_consensus.cc:2410] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 4.
I20260709 13:08:25.712513 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:25.663147 24552 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:25.713976 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:25.564445 24661 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:25.707744 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:08:25.738487 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 76154.2us
I20260709 13:08:25.726517 22873 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
W20260709 13:08:25.715073 24467 consensus_meta.cc:287] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.570s	user 0.006s	sys 0.005s
I20260709 13:08:25.733630 22952 raft_consensus.cc:1240] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:25.686435 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:25.710574 24476 consensus_queue.cc:1059] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 13:08:25.748603 24647 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:25.748755 24476 raft_consensus.cc:3055] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:25.749121 24476 raft_consensus.cc:740] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:25.749176 24647 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:25.749970 24476 consensus_queue.cc:260] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:25.751120 24476 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:25.724392 22948 raft_consensus.cc:1240] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:25.757727 23026 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:25.761138 23025 raft_consensus.cc:1275] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:08:25.766573 23025 pending_rounds.cc:85] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75: Aborting all ops after (but not including) 2
I20260709 13:08:25.766923 23025 pending_rounds.cc:107] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:08:25.767213 23025 raft_consensus.cc:2889] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:08:25.816248 21761 debug-util.cc:398] Leaking SignalData structure 0x7b080000f6c0 after lost signal to thread 23622
I20260709 13:08:25.737303 24621 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:25.751289 24647 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:25.738708 24552 raft_consensus.cc:3060] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:25.847957 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:25.890686 22993 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:25.742010 24659 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:25.730111 24662 raft_consensus.cc:2804] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:25.739418 24664 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:25.738111 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:25.880652 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:25.881369 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:25.997610 22880 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
I20260709 13:08:25.751748 24557 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:25.876020 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:25.955896 23623 consensus_meta.cc:287] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.538s	user 0.009s	sys 0.004s
W20260709 13:08:25.957118 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 202982us
I20260709 13:08:25.993422 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:26.019219 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:26.022069 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:26.023859 22919 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:26.028484 22876 raft_consensus.cc:2468] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:26.028712 23623 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:26.031795 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:25.846190 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:26.040027 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:26.036022 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:26.043366 22951 raft_consensus.cc:2410] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
W20260709 13:08:26.045702 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 155103us
I20260709 13:08:26.047214 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:26.048072 22876 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:26.059608 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:08:26.034580 24632 consensus_meta.cc:287] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.753s	user 0.004s	sys 0.010s
I20260709 13:08:26.022795 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:25.847911 24677 consensus_queue.cc:1048] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.318s
I20260709 13:08:25.897881 24621 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:26.024070 24664 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:26.133941 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 144156us
I20260709 13:08:26.166668 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:08:26.403820 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 379977us
W20260709 13:08:26.527854 22843 connection.cc:256] RPC call timeout handler was delayed by -2.3353735949999646s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1513s for a response
W20260709 13:08:26.528265 22843 connection.cc:256] RPC call timeout handler was delayed by -2.265901347999943s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1809s for a response
W20260709 13:08:26.528616 22843 connection.cc:256] RPC call timeout handler was delayed by -1.9275013479999643s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21850000000000003s for a response
I20260709 13:08:26.529599 22843 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:26.522293 24678 log.cc:927] Time spent T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.625s	user 0.001s	sys 0.000s
W20260709 13:08:26.532732 22994 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:26.533993 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:26.534561 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:26.535334 22876 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
I20260709 13:08:26.406654 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:26.407054 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:26.539261 23625 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:08:26.031481 24407 raft_consensus.cc:2764] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:08:26.539573 22879 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
W20260709 13:08:26.418695 23624 consensus_meta.cc:287] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 1.028s	user 0.008s	sys 0.014s
I20260709 13:08:26.257691 23025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 1309) took 1054 ms. Trace:
I20260709 13:08:26.556720 23624 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:26.057093 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:08:26.532568 24683 log.cc:927] Time spent T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
I20260709 13:08:26.571188 23023 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 3.
I20260709 13:08:26.586441 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:26.587323 23026 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:08:26.581362 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:26.589208 23021 raft_consensus.cc:2393] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 3: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:26.556357 23025 rpcz_store.cc:276] 0709 13:08:25.203802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:25.204772 (+   970us) service_pool.cc:224] Handling call
0709 13:08:25.205119 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:25.767737 (+562618us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:26.257400 (+489663us) spinlock_profiling.cc:243] Waited 555 ms on lock 0x7b54002c9054. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:26.257625 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1277576576}
I20260709 13:08:26.582373 23624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60044 (request call id 418) took 1209 ms. Trace:
I20260709 13:08:26.144098 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:08:26.532822 24655 log.cc:927] Time spent T 00000000000000000000000000000000 P 8d175b4ae72f4fc48971c2f31668971c: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:08:26.520440 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:08:26.375407 24668 log.cc:927] Time spent T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.735s	user 0.001s	sys 0.000s
I20260709 13:08:26.601948 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:26.603361 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:26.604123 23025 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 3.
I20260709 13:08:26.591249 23624 rpcz_store.cc:276] 0709 13:08:25.347772 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:25.347908 (+   136us) service_pool.cc:224] Handling call
0709 13:08:26.557483 (+1209575us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:26.591938 23021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33420 (request call id 1323) took 1235 ms. Trace:
I20260709 13:08:26.533272 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:26.135452 24646 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:26.290580 24630 raft_consensus.cc:2804] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:26.605623 23021 rpcz_store.cc:276] 0709 13:08:25.356622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:25.359087 (+  2465us) service_pool.cc:224] Handling call
0709 13:08:26.541851 (+1182764us) spinlock_profiling.cc:243] Waited 6.51 ms on lock 0x7b58001ca198. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0a22 00007f613bbd9c09 00007f613ad33ee0 00007f613adfd0ab 00007f613adfd131 00007f613adab0a0 00007f613adaa984 00007f61343256d4 00007f6134325662
0709 13:08:26.591878 (+ 50027us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14968704}
I20260709 13:08:26.606736 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:26.396853 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:26.541687 24642 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
W20260709 13:08:26.614789 22918 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.024s	sys 0.018s
W20260709 13:08:26.618405 22918 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.025s	sys 0.018s
I20260709 13:08:26.592622 24671 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:08:26.613308 24487 consensus_meta.cc:287] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 1.226s	user 0.013s	sys 0.010s
I20260709 13:08:26.614070 23623 raft_consensus.cc:2368] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 2. Current term is 3.
I20260709 13:08:26.546532 24659 raft_consensus.cc:2749] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:26.038928 24557 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:26.615993 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:26.631034 23625 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 1.
W20260709 13:08:26.611763 24684 log.cc:927] Time spent T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 13:08:26.618785 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 590717us
I20260709 13:08:26.569641 24455 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:26.541397 24632 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:26.639292 22918 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:26.639765 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:26.641418 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:26.643795 23625 raft_consensus.cc:1240] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:26.644491 23625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 482) took 1007 ms. Trace:
I20260709 13:08:26.645619 23624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 365) took 1153 ms. Trace:
I20260709 13:08:26.645969 24664 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:08:26.646667 23591 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.010s	sys 0.010s
W20260709 13:08:26.647023 23591 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.010s	sys 0.011s
I20260709 13:08:26.645987 23625 rpcz_store.cc:276] 0709 13:08:25.637495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:26.643338 (+1005843us) service_pool.cc:224] Handling call
0709 13:08:26.643613 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:26.644280 (+   667us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:26.644443 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:26.628756 22948 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:08:26.649335 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:08:26.630115 24662 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.007s	sys 0.020s
I20260709 13:08:26.650589 22918 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:26.648591 23624 rpcz_store.cc:276] 0709 13:08:25.492200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:26.644856 (+1152656us) service_pool.cc:224] Handling call
0709 13:08:26.645147 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:26.645325 (+   178us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:26.645356 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:26.645397 (+    41us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:26.645428 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:26.645479 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:26.645570 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:26.650713 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:26.651566 22879 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
W20260709 13:08:26.651827 24687 log.cc:927] Time spent T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 13:08:26.653461 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:26.653019 24455 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:26.652333 22918 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:26.650740 24662 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.008s	sys 0.021s
I20260709 13:08:26.661856 24662 raft_consensus.cc:697] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:26.663055 24662 consensus_queue.cc:237] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:26.664988 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:26.666910 24502 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:26.667438 24502 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:26.615330 24630 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:26.668870 24630 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:08:26.670964 24571 consensus_meta.cc:287] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 1.039s	user 0.010s	sys 0.000s
W20260709 13:08:26.678843 24501 consensus_meta.cc:287] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 1.193s	user 0.005s	sys 0.003s
W20260709 13:08:26.679124 22920 leader_election.cc:336] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Timed out: RequestConsensusVote RPC to 127.21.63.1:33929 timed out after 1.512s (SENT)
I20260709 13:08:26.679302 24501 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:26.680971 24679 log.cc:927] Time spent T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.424s	user 0.001s	sys 0.000s
W20260709 13:08:26.681041 22920 leader_election.cc:336] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Timed out: RequestConsensusVote RPC to 127.21.63.1:33929 timed out after 1.534s (SENT)
I20260709 13:08:26.682384 23621 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60044 (request call id 416) took 1344 ms. Trace:
I20260709 13:08:26.682827 24557 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:26.629383 24642 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:26.682896 23621 rpcz_store.cc:276] 0709 13:08:25.337871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:25.347482 (+  9611us) service_pool.cc:224] Handling call
0709 13:08:25.408086 (+ 60604us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:25.522831 (+114745us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:25.522880 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:25.564652 (+ 41772us) log.cc:844] Serialized 26 byte log entry
0709 13:08:25.707035 (+142383us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:08:25.713017 (+  5982us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:25.725572 (+ 12555us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:26.682049 (+956477us) raft_consensus.cc:1685] finished
0709 13:08:26.682081 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:26.682295 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":48784,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":7726,"thread_start_us":132177,"threads_started":2,"wal-append.queue_time_us":361412}
W20260709 13:08:26.685879 24417 consensus_meta.cc:287] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 1.246s	user 0.007s	sys 0.011s
I20260709 13:08:26.686275 24417 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:26.687549 22920 leader_election.cc:336] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Timed out: RequestConsensusVote RPC to 127.21.63.1:33929 timed out after 1.604s (SENT)
I20260709 13:08:26.627426 24661 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:26.625680 24663 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:26.689188 24501 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:26.689365 24663 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:26.690485 22920 leader_election.cc:336] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Timed out: RequestConsensusVote RPC to 127.21.63.1:33929 timed out after 1.631s (SENT)
I20260709 13:08:26.632733 24671 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:26.695317 24671 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:26.625960 23617 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 1.
I20260709 13:08:26.697232 22878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 1695) took 1189 ms. Trace:
I20260709 13:08:26.697678 22878 rpcz_store.cc:276] 0709 13:08:25.507634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:25.507759 (+   125us) service_pool.cc:224] Handling call
0709 13:08:25.532655 (+ 24896us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:25.625171 (+ 92516us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:25.625940 (+   769us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:25.647070 (+ 21130us) log.cc:844] Serialized 26 byte log entry
0709 13:08:25.882831 (+235761us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:08:25.882901 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:25.883000 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:26.696936 (+813936us) raft_consensus.cc:1685] finished
0709 13:08:26.696964 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:26.697137 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37955,"raft.queue_time_us":38138,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":65389,"thread_start_us":191645,"threads_started":1,"wal-append.queue_time_us":366467}
I20260709 13:08:26.687543 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:26.636749 22919 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:08:26.703418 22873 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.857s	user 0.012s	sys 0.021s
I20260709 13:08:26.692359 24663 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:26.656308 24455 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:26.696409 23617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 484) took 1046 ms. Trace:
W20260709 13:08:26.730302 22920 leader_election.cc:336] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Timed out: RequestConsensusVote RPC to 127.21.63.1:33929 timed out after 1.675s (SENT)
I20260709 13:08:26.730952 22920 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:26.703795 22873 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.857s	user 0.012s	sys 0.021s
I20260709 13:08:26.735863 22873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 1724) took 1007 ms. Trace:
W20260709 13:08:26.720260 22995 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.226s	user 0.006s	sys 0.041s
W20260709 13:08:26.739375 22995 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.245s	user 0.007s	sys 0.041s
I20260709 13:08:26.742389 22955 raft_consensus.cc:1240] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:26.747521 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:26.739282 22873 rpcz_store.cc:276] 0709 13:08:25.728421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:25.739903 (+ 11482us) service_pool.cc:224] Handling call
0709 13:08:25.740209 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:25.740414 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:08:25.846455 (+106041us) log.cc:844] Serialized 14 byte log entry
0709 13:08:26.735336 (+888881us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:26.735425 (+    89us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:08:26.735489 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:26.735587 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:26.735786 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":107525,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":44,"thread_start_us":888408,"threads_started":1}
W20260709 13:08:26.705987 24626 consensus_meta.cc:287] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 1.258s	user 0.010s	sys 0.000s
W20260709 13:08:26.716096 24552 consensus_meta.cc:287] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.697s	user 0.005s	sys 0.001s
I20260709 13:08:26.634155 24487 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:26.751850 22877 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.004s	sys 0.042s
I20260709 13:08:26.706504 24417 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:08:26.763214 22875 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.027s	sys 0.029s
W20260709 13:08:26.707882 24490 consensus_meta.cc:287] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 1.004s	user 0.006s	sys 0.002s
I20260709 13:08:26.759819 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:08:26.763525 22875 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.027s	sys 0.029s
I20260709 13:08:26.752744 24691 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:26.722457 23617 rpcz_store.cc:276] 0709 13:08:25.649825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:26.451608 (+801783us) service_pool.cc:224] Handling call
0709 13:08:26.696330 (+244722us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:26.768553 22950 raft_consensus.cc:1240] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:26.766599 22873 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
I20260709 13:08:26.770735 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:08:26.789439 23020 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.744s	user 0.013s	sys 0.011s
W20260709 13:08:26.856956 23020 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.811s	user 0.014s	sys 0.011s
I20260709 13:08:26.757683 24630 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:26.770265 24476 consensus_meta.cc:287] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.923s	user 0.008s	sys 0.002s
I20260709 13:08:26.762605 24552 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:26.765503 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:26.770021 24691 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:26.765791 24690 raft_consensus.cc:2749] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:26.857758 23020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 246) took 1193 ms. Trace:
W20260709 13:08:26.858570 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 206236us
W20260709 13:08:26.859694 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 129435us
I20260709 13:08:26.859961 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:26.882795 22955 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 1384) took 1186 ms. Trace:
I20260709 13:08:26.958074 22955 rpcz_store.cc:276] 0709 13:08:25.695964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:25.700769 (+  4805us) service_pool.cc:224] Handling call
0709 13:08:25.701027 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:26.745898 (+1044871us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:26.882391 (+136493us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b540037b4d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:26.882675 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2394295040}
I20260709 13:08:26.890626 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:26.959182 22918 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:26.903187 23023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33418 (request call id 1341) took 1165 ms. Trace:
W20260709 13:08:26.915766 23622 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.149s	user 0.020s	sys 0.022s
W20260709 13:08:26.960647 23622 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.194s	user 0.021s	sys 0.022s
I20260709 13:08:26.888127 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:26.959860 23023 rpcz_store.cc:276] 0709 13:08:25.737675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:26.031551 (+293876us) service_pool.cc:224] Handling call
0709 13:08:26.902945 (+871394us) spinlock_profiling.cc:243] Waited 539 ms on lock 0x7b5400358754. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:26.903101 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1238987136}
W20260709 13:08:26.964550 22920 leader_election.cc:336] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Timed out: RequestConsensusVote RPC to 127.21.63.1:33929 timed out after 1.708s (SENT)
I20260709 13:08:26.767331 24490 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:26.941035 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:08:26.946612 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:26.964120 22946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58310 (request call id 1399) took 1230 ms. Trace:
I20260709 13:08:26.965065 22920 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:26.961884 22873 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:08:26.967376 22948 raft_consensus.cc:2368] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 3. Current term is 4.
I20260709 13:08:26.760965 24626 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:26.968750 24416 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:26.905340 24660 raft_consensus.cc:2749] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:26.969060 23028 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33418 (request call id 1340) took 1231 ms. Trace:
I20260709 13:08:26.970455 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:26.969007 24690 raft_consensus.cc:2749] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:26.957351 23020 rpcz_store.cc:276] 0709 13:08:25.664538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:25.664648 (+   110us) service_pool.cc:224] Handling call
0709 13:08:25.750267 (+ 85619us) spinlock_profiling.cc:243] Waited 8.8 ms on lock 0x7b58001ca198. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0a22 00007f613bbd9c09 00007f613ad33ee0 00007f613adfd0ab 00007f613adfd131 00007f613adbefc0 00007f613ada9929 00007f613ada93d2 00007f6134320bd4
0709 13:08:25.750641 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:25.750865 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:08:25.751200 (+   335us) log.cc:844] Serialized 14 byte log entry
0709 13:08:26.857384 (+1106184us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:26.857448 (+    64us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:26.857495 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:26.857569 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:26.857703 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":444,"prepare.queue_time_us":9573,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":103967,"raft.run_cpu_time_us":51,"raft.run_wall_time_us":51,"spinlock_wait_cycles":20241664,"thread_start_us":1093453,"threads_started":2,"wal-append.queue_time_us":751976}
W20260709 13:08:26.999305 22920 leader_election.cc:336] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Timed out: RequestConsensusVote RPC to 127.21.63.1:33929 timed out after 1.732s (SENT)
I20260709 13:08:26.859737 23616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 358) took 1479 ms. Trace:
I20260709 13:08:27.004848 22920 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:26.999557 22946 rpcz_store.cc:276] 0709 13:08:25.733655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:26.060557 (+326902us) service_pool.cc:224] Handling call
0709 13:08:26.890485 (+829928us) spinlock_profiling.cc:243] Waited 2.93 ms on lock 0x7b5800043598. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0eee 0000000000375939 00007f613ad341e0 00007f613adfd0c4 00007f613adfd131 00007f613adab0a0 00007f613adaa984 00007f61343256d4 00007f6134325662
0709 13:08:26.964045 (+ 73560us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6749952}
W20260709 13:08:27.003911 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 367234us
W20260709 13:08:27.019098 23590 leader_election.cc:336] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Timed out: RequestConsensusVote RPC to 127.21.63.1:33929 timed out after 1.585s (SENT)
I20260709 13:08:27.019714 23590 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:27.021020 24710 raft_consensus.cc:2749] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:27.022651 24632 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:27.023168 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:27.039480 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:27.044102 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:27.045182 22879 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 1.
I20260709 13:08:27.040347 22954 raft_consensus.cc:2468] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 3.
W20260709 13:08:27.048391 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 46747.4us
I20260709 13:08:27.048888 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:27.049515 22954 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 5.
I20260709 13:08:27.051671 22994 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:08:27.052812 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 490802us
W20260709 13:08:27.052657 22843 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.016s	sys 0.023s
I20260709 13:08:26.959722 24691 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:08:27.055546 22843 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.017s	sys 0.023s
W20260709 13:08:27.056965 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 511594us
I20260709 13:08:27.048127 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:26.966678 24642 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:26.972764 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:08:27.107657 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 148565us
W20260709 13:08:27.105559 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 47463.1us
I20260709 13:08:27.057368 24659 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:27.171648 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 63226.4us
I20260709 13:08:27.173519 23622 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60044 (request call id 417) took 1830 ms. Trace:
I20260709 13:08:27.180959 23622 rpcz_store.cc:276] 0709 13:08:25.343539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:25.343703 (+   164us) service_pool.cc:224] Handling call
0709 13:08:25.360111 (+ 16408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:25.360302 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:08:25.766022 (+405720us) log.cc:844] Serialized 14 byte log entry
0709 13:08:27.170402 (+1404380us) log.cc:844] Serialized 14 byte log entry
0709 13:08:27.173193 (+  2791us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:27.173243 (+    50us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:27.173284 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:27.173351 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:27.173466 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2499,"raft.queue_time_us":98,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":154,"thread_start_us":1393949,"threads_started":2,"wal-append.queue_time_us":1001675}
W20260709 13:08:27.169468 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 170174us
I20260709 13:08:27.108816 23620 raft_consensus.cc:2393] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 3: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:27.103799 24467 raft_consensus.cc:2804] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:08:27.114828 24091 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.391s	user 0.014s	sys 0.038s
W20260709 13:08:27.179844 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 65846.3us
I20260709 13:08:27.003732 24416 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:27.203213 22918 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:27.167652 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 116713us
I20260709 13:08:27.171818 23619 raft_consensus.cc:2393] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 3: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
W20260709 13:08:27.170192 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 52507.9us
W20260709 13:08:27.203922 24091 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.480s	user 0.018s	sys 0.052s
I20260709 13:08:27.116209 23617 raft_consensus.cc:1240] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 3. Current term is 4. Ops: []
W20260709 13:08:27.205757 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:27.212996 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:08:27.214692 24685 log.cc:927] Time spent T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.371s	user 0.000s	sys 0.001s
I20260709 13:08:27.215041 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:27.209013 22994 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:27.230521 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:27.107636 24661 consensus_queue.cc:1059] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.095s
I20260709 13:08:27.177605 24690 raft_consensus.cc:2749] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:27.240516 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:08:27.242457 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 60927.7us
I20260709 13:08:27.175048 24587 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:27.174703 24475 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:27.200460 24571 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:27.200189 23618 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 360) took 1765 ms (client timeout 1883 ms). Trace:
I20260709 13:08:27.246119 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:27.003777 23028 rpcz_store.cc:276] 0709 13:08:25.737407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:26.396442 (+659035us) service_pool.cc:224] Handling call
0709 13:08:26.887900 (+491458us) spinlock_profiling.cc:243] Waited 8.88 ms on lock 0x7b58001ca198. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0a22 00007f613bbd9c09 00007f613ad33ee0 00007f613adfd0ab 00007f613adfd131 00007f613adab0a0 00007f613adaa984 00007f61343256d4 00007f6134325662
0709 13:08:26.968975 (+ 81075us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20432640}
I20260709 13:08:27.199193 24632 raft_consensus.cc:2749] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:08:27.245721 22993 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.63.4:44043 timed out after 1.760s (SENT) [suppressed 46 similar messages]
I20260709 13:08:27.205972 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:27.202656 24467 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:27.296208 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:27.183358 24502 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:27.274756 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:27.242384 24487 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:27.283389 23624 raft_consensus.cc:1275] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:08:27.286857 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:27.287712 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:08:27.293061 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 47360.8us
I20260709 13:08:27.301328 22954 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 2.
I20260709 13:08:27.294169 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:27.294889 22843 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:27.302227 22951 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 2.
I20260709 13:08:27.008059 23616 rpcz_store.cc:276] 0709 13:08:25.380605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:25.382227 (+  1622us) service_pool.cc:224] Handling call
0709 13:08:26.858778 (+1476551us) spinlock_profiling.cc:243] Waited 9 ms on lock 0x7b58001c4d98. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0a22 00007f613bbd9c09 00007f613ad33ee0 00007f613adfd0ab 00007f613adfd131 00007f613adbefc0 00007f613ada9929 00007f613ada93d2 00007f6134320bd4
0709 13:08:26.859047 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:26.859241 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:26.859278 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:26.859321 (+    43us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:26.859366 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:26.859430 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:26.859545 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20710272}
I20260709 13:08:27.273794 23027 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 1332) took 1553 ms. Trace:
I20260709 13:08:27.342006 23625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 371) took 1605 ms. Trace:
I20260709 13:08:27.353370 24642 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:27.298933 24467 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:27.344710 22951 raft_consensus.cc:1275] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:27.351320 22955 raft_consensus.cc:1275] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:27.294004 24707 raft_consensus.cc:2804] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:27.352849 24705 raft_consensus.cc:2749] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:27.361845 24700 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:27.366644 24632 raft_consensus.cc:2749] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:27.446133 24707 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:27.446656 24707 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:27.368566 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:27.447621 22879 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:27.369061 24708 raft_consensus.cc:2804] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:27.369691 23620 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 363) took 1878 ms. Trace:
W20260709 13:08:27.376065 24701 log.cc:927] Time spent T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.181s	user 0.000s	sys 0.000s
I20260709 13:08:27.450256 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:27.452114 24708 raft_consensus.cc:697] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:27.288578 24475 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:27.449174 23620 rpcz_store.cc:276] 0709 13:08:25.491415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:25.492615 (+  1200us) service_pool.cc:224] Handling call
0709 13:08:26.687383 (+1194768us) spinlock_profiling.cc:243] Waited 1.78 ms on lock 0x7b58001c4d98. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0a22 00007f613bbd9c09 00007f613ad33ee0 00007f613adfd0ab 00007f613adfd131 00007f613adab0a0 00007f613adaa984 00007f61343256d4 00007f6134325662
0709 13:08:27.369432 (+682049us) spinlock_profiling.cc:243] Waited 298 ms on lock 0x7b54003d90d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:27.369626 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":688402944}
I20260709 13:08:27.453424 24708 consensus_queue.cc:237] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:27.454491 24475 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:27.378669 24718 log.cc:927] Time spent T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
W20260709 13:08:27.404155 24697 log.cc:927] Time spent T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.271s	user 0.000s	sys 0.000s
I20260709 13:08:27.289428 24587 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:27.290501 24571 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:27.456552 22877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 1697) took 1948 ms. Trace:
I20260709 13:08:27.268800 24663 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:27.424366 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:27.460139 22876 raft_consensus.cc:2468] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:27.428004 24630 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:27.342118 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:27.458667 24490 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:27.425379 23625 rpcz_store.cc:276] 0709 13:08:25.736893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:26.647767 (+910874us) service_pool.cc:224] Handling call
0709 13:08:26.648058 (+   291us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:26.648413 (+   355us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:26.648442 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:26.650181 (+  1739us) log.cc:844] Serialized 26 byte log entry
0709 13:08:26.937117 (+286936us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:26.937179 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:26.937290 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:27.341781 (+404491us) raft_consensus.cc:1685] finished
0709 13:08:27.341807 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:27.341948 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":35924,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":99,"thread_start_us":286430,"threads_started":1,"wal-append.queue_time_us":420588}
W20260709 13:08:27.434499 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 231353us
I20260709 13:08:27.464175 22918 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:27.466171 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:27.466807 22873 raft_consensus.cc:2468] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 3.
I20260709 13:08:27.467628 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:27.463177 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:27.469736 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:27.470342 22871 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 1.
I20260709 13:08:27.471264 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:08:27.289340 23618 rpcz_store.cc:269] 0709 13:08:25.423852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:25.426459 (+  2607us) service_pool.cc:224] Handling call
0709 13:08:26.950359 (+1523900us) spinlock_profiling.cc:243] Waited 7.34 ms on lock 0x7b58001c4d98. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0a22 00007f613bbd9c09 00007f613ad33ee0 00007f613adfd0ab 00007f613adfd131 00007f613adab0a0 00007f613adaa984 00007f61343256d4 00007f6134325662
0709 13:08:27.189600 (+239241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16879488}
W20260709 13:08:27.441295 24698 log.cc:927] Time spent T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.234s	user 0.000s	sys 0.001s
I20260709 13:08:27.441767 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:27.475493 22918 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:27.444806 24700 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:27.356112 24552 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:27.451119 24699 log.cc:927] Time spent T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.321s	user 0.000s	sys 0.001s
I20260709 13:08:27.452216 22879 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 1.
I20260709 13:08:27.200552 24659 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:27.475449 24632 raft_consensus.cc:2749] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:27.459444 24571 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:27.460978 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:27.425977 24626 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:27.473893 22873 raft_consensus.cc:1275] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:08:27.473971 24696 log.cc:927] Time spent T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.342s	user 0.000s	sys 0.001s
I20260709 13:08:27.482252 22872 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
I20260709 13:08:27.443673 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:27.427138 23027 rpcz_store.cc:276] 0709 13:08:25.712417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:26.598678 (+886261us) service_pool.cc:224] Handling call
0709 13:08:26.599010 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:26.599267 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:08:26.793384 (+194117us) log.cc:844] Serialized 14 byte log entry
0709 13:08:27.179580 (+386196us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:27.202867 (+ 23287us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:27.208624 (+  5757us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:27.236570 (+ 27946us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:27.266181 (+ 29611us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53134,"prepare.queue_time_us":245925,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":77506,"spinlock_wait_cycles":24553728,"thread_start_us":334117,"threads_started":2,"wal-append.queue_time_us":331197}
I20260709 13:08:27.485431 22843 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:27.437143 24642 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:27.487233 24722 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:27.489840 22918 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:27.456940 22877 rpcz_store.cc:276] 0709 13:08:25.508303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:25.513055 (+  4752us) service_pool.cc:224] Handling call
0709 13:08:25.639687 (+126632us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:25.691266 (+ 51579us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:25.704293 (+ 13027us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:25.898567 (+194274us) log.cc:844] Serialized 26 byte log entry
0709 13:08:26.766404 (+867837us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:08:26.768225 (+  1821us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:26.773115 (+  4890us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:27.456325 (+683210us) raft_consensus.cc:1685] finished
0709 13:08:27.456350 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:27.456491 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14980,"raft.run_cpu_time_us":980,"raft.run_wall_time_us":26231,"thread_start_us":600837,"threads_started":1,"wal-append.queue_time_us":903822}
I20260709 13:08:27.521564 22843 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:27.503322 24467 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:27.505141 23623 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 368) took 1845 ms (client timeout 1760 ms). Trace:
I20260709 13:08:27.506001 24659 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:27.525697 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:08:27.526257 24713 log.cc:927] Time spent T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.133s	user 0.000s	sys 0.000s
I20260709 13:08:27.527146 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:27.528096 22880 raft_consensus.cc:2393] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 3: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
W20260709 13:08:27.506449 23619 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 366) took 2013 ms (client timeout 2205 ms). Trace:
W20260709 13:08:27.508496 24681 log.cc:927] Time spent T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:08:27.508781 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:27.540051 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:27.540452 24642 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:27.540745 22871 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 3.
I20260709 13:08:27.541050 24416 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:27.542309 22877 raft_consensus.cc:1275] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:08:27.508586 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:27.510843 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:27.488907 24690 raft_consensus.cc:2749] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:27.518815 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:27.546228 22876 raft_consensus.cc:2393] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 3: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:27.546773 22873 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 3.
I20260709 13:08:27.521409 22918 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:27.524024 22950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 1804) took 1792 ms. Trace:
I20260709 13:08:27.525509 24658 raft_consensus.cc:2749] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:08:27.523484 23623 rpcz_store.cc:269] 0709 13:08:25.659444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:26.058774 (+399330us) service_pool.cc:224] Handling call
0709 13:08:27.504918 (+1446144us) spinlock_profiling.cc:243] Waited 552 ms on lock 0x7b54005aebd4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:27.505044 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1269458816}
I20260709 13:08:27.538388 22843 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:27.519773 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:27.549469 22918 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:27.553120 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:27.553933 22879 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 1.
I20260709 13:08:27.555352 24709 consensus_queue.cc:1059] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 13:08:27.548012 22950 rpcz_store.cc:276] 0709 13:08:25.731939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:25.732095 (+   156us) service_pool.cc:224] Handling call
0709 13:08:25.732387 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:26.768997 (+1036610us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:27.523824 (+754827us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54000ed3d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:27.523974 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2382497536}
I20260709 13:08:27.560020 22875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 1725) took 1822 ms. Trace:
I20260709 13:08:27.523983 24652 raft_consensus.cc:2749] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:27.524489 24632 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:27.561317 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:27.561228 22875 rpcz_store.cc:276] 0709 13:08:25.737631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:25.897085 (+159454us) service_pool.cc:224] Handling call
0709 13:08:25.990227 (+ 93142us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:25.996769 (+  6542us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:25.996813 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:26.044572 (+ 47759us) log.cc:844] Serialized 26 byte log entry
0709 13:08:26.768554 (+723982us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:26.768596 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:26.768684 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:27.559806 (+791122us) raft_consensus.cc:1685] finished
0709 13:08:27.559828 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:27.559967 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6100,"raft.queue_time_us":6517,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":15331,"thread_start_us":508429,"threads_started":1,"wal-append.queue_time_us":863086}
W20260709 13:08:27.538002 23619 rpcz_store.cc:269] 0709 13:08:25.492653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:26.143816 (+651163us) service_pool.cc:224] Handling call
0709 13:08:27.506254 (+1362438us) spinlock_profiling.cc:243] Waited 461 ms on lock 0x7b54006383d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:27.506387 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1060786560}
I20260709 13:08:27.585492 24467 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:27.546087 24722 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:27.559262 24659 raft_consensus.cc:2749] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:27.560179 24629 raft_consensus.cc:2749] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:27.554186 24642 raft_consensus.cc:2749] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:27.588616 24722 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:27.621482 24659 consensus_queue.cc:1059] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:27.624841 24700 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:27.629830 24587 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:27.630983 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:27.630648 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:27.633366 22875 raft_consensus.cc:2468] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 3.
I20260709 13:08:27.645103 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:27.661942 24621 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:27.630607 24707 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:27.652211 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:27.685513 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:27.686555 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:27.693658 22948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52222 (request call id 179) took 1122 ms. Trace:
I20260709 13:08:27.711759 22880 raft_consensus.cc:2393] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 3: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
W20260709 13:08:27.702021 23029 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33420 (request call id 1319) took 2376 ms (client timeout 1512 ms). Trace:
W20260709 13:08:27.712958 23029 rpcz_store.cc:269] 0709 13:08:25.325790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:25.326017 (+   227us) service_pool.cc:224] Handling call
0709 13:08:27.701749 (+2375732us) spinlock_profiling.cc:243] Waited 1.74 s on lock 0x7b540018b354. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:27.701932 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4007483648}
I20260709 13:08:27.712052 22948 rpcz_store.cc:276] 0709 13:08:26.570817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:26.649072 (+ 78255us) service_pool.cc:224] Handling call
0709 13:08:27.693411 (+1044339us) spinlock_profiling.cc:243] Waited 317 ms on lock 0x7b5400148f54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:27.693572 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":728948608}
W20260709 13:08:27.718441 24706 log.cc:927] Time spent T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.248s	user 0.001s	sys 0.000s
W20260709 13:08:27.722489 24691 consensus_meta.cc:287] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.615s	user 0.006s	sys 0.006s
I20260709 13:08:27.722914 24691 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:27.728618 22880 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
mode: GRACEFUL
new_leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:51434
I20260709 13:08:27.729550 22880 raft_consensus.cc:606] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Received request to transfer leadership to TS aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:27.730253 24707 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:27.733999 22774 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33466 (request call id 287) took 6837 ms. Trace:
I20260709 13:08:27.770920 24722 raft_consensus.cc:3055] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:27.771431 24722 raft_consensus.cc:740] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:27.772296 24722 consensus_queue.cc:260] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:27.734408 22774 rpcz_store.cc:276] 0709 13:08:20.896420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:20.896595 (+   175us) service_pool.cc:224] Handling call
0709 13:08:27.733941 (+6837346us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:25.524769 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:25.525435 (+   666us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:25.525466 (+    31us) write_op.cc:435] Acquired schema lock
0709 13:08:25.525490 (+    24us) tablet.cc:662] Decoding operations
0709 13:08:25.525958 (+   468us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:25.526041 (+    83us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:25.526080 (+    39us) tablet.cc:685] Acquiring locks for 2 operations
0709 13:08:25.526452 (+   372us) tablet.cc:701] Row locks acquired
0709 13:08:25.526471 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:08:25.526608 (+   137us) write_op.cc:270] Start()
0709 13:08:25.526711 (+   103us) write_op.cc:276] Timestamp: P: 1783602505526568 usec, L: 0
0709 13:08:25.526732 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:08:25.527480 (+   748us) log.cc:844] Serialized 917 byte log entry
0709 13:08:25.650232 (+122752us) op_driver.cc:464] REPLICATION: finished
0709 13:08:25.659198 (+  8966us) write_op.cc:301] APPLY: starting
0709 13:08:25.661464 (+  2266us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:25.688163 (+ 26699us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:25.710355 (+ 22192us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:25.897641 (+187286us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:26.045938 (+148297us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 13:08:26.060983 (+ 15045us) write_op.cc:312] APPLY: finished
0709 13:08:26.375634 (+314651us) log.cc:844] Serialized 29 byte log entry
0709 13:08:26.421920 (+ 46286us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:26.449712 (+ 27792us) write_op.cc:454] Released schema lock
0709 13:08:26.451537 (+  1825us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:26.558724 (+107187us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:26.569918 (+ 11194us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":228,"reported_tablets":12,"child_traces":[["op",{"apply.queue_time_us":6661,"apply.run_cpu_time_us":3066,"apply.run_wall_time_us":864451,"num_ops":2,"prepare.queue_time_us":59191,"prepare.run_cpu_time_us":2999,"prepare.run_wall_time_us":3000,"replication_time_us":48845,"thread_start_us":58853,"threads_started":1}]]}
I20260709 13:08:27.783874 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:27.775134 22776 catalog_manager.cc:5697] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 2 to 5, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 5 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:27.787314 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:27.796197 22776 catalog_manager.cc:5697] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 2 to 4, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 4 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
W20260709 13:08:27.798270 22918 connection.cc:256] RPC call timeout handler was delayed by -1.5808836549998659s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1512s for a response
I20260709 13:08:27.804593 24725 consensus_queue.cc:1048] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:27.807765 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:27.809090 23621 raft_consensus.cc:2393] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 3: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
W20260709 13:08:27.810290 22920 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.024s	sys 0.015s
W20260709 13:08:27.810647 22920 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.024s	sys 0.015s
I20260709 13:08:27.804963 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:08:27.811175 22920 leader_election.cc:336] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Timed out: RequestConsensusVote RPC to 127.21.63.1:33929 timed out after 1.772s (SENT)
I20260709 13:08:27.800177 23619 raft_consensus.cc:2368] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 2. Current term is 3.
W20260709 13:08:27.815022 22994 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.011s	sys 0.041s
W20260709 13:08:27.815524 22994 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.011s	sys 0.041s
W20260709 13:08:27.819119 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 610150us
I20260709 13:08:27.798537 22776 catalog_manager.cc:5697] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 2 to 4, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 4 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:27.821467 22994 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:08:27.812475 24091 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.048s	sys 0.071s
I20260709 13:08:27.823246 22994 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:27.823191 22776 catalog_manager.cc:5697] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:27.800477 24362 consensus_queue.cc:1048] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 13:08:27.812458 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:27.827256 22994 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:08:27.828639 24091 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.048s	sys 0.072s
I20260709 13:08:27.828020 22993 leader_election.cc:400] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 2. Current term is 3.
I20260709 13:08:27.803656 24515 consensus_queue.cc:1059] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:08:27.831889 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:27.833441 22993 leader_election.cc:403] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:08:27.833200 23020 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.015s	sys 0.041s
W20260709 13:08:27.835484 23020 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.015s	sys 0.041s
I20260709 13:08:27.834398 22994 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:27.836016 22776 catalog_manager.cc:5697] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 1 to 3, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 3 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:27.813704 24691 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:27.820420 24476 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.005s	sys 0.050s
I20260709 13:08:27.816483 23618 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 2.
I20260709 13:08:27.826751 24417 raft_consensus.cc:3060] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:08:27.845975 23588 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.000s	sys 0.028s
W20260709 13:08:27.846464 23588 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.555s	user 0.000s	sys 0.028s
W20260709 13:08:27.846908 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 557620us
I20260709 13:08:27.847040 22994 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:27.829757 24660 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:27.848978 22994 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:27.830060 24362 consensus_queue.cc:1243] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer 5b346767a89745fa82928a5dac9cc0b2 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 13:08:27.843398 24476 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.558s	user 0.005s	sys 0.050s
W20260709 13:08:27.858332 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:27.856062 23590 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:27.839931 22776 catalog_manager.cc:5697] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 2 to 3, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 3 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:27.834190 24648 raft_consensus.cc:2804] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:27.855070 22993 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:08:27.837669 24661 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.000s	sys 0.015s
I20260709 13:08:27.843736 24674 raft_consensus.cc:2749] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:27.843878 24621 raft_consensus.cc:2749] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 2. Current term is 3.
I20260709 13:08:27.860436 22994 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:08:27.859560 22952 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.021s	sys 0.008s
I20260709 13:08:27.811549 22920 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:27.864367 22878 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.868s	user 0.009s	sys 0.017s
W20260709 13:08:27.864399 22992 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.000s	sys 0.016s
I20260709 13:08:27.844918 24672 consensus_queue.cc:1048] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
W20260709 13:08:27.874250 22952 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.022s	sys 0.009s
W20260709 13:08:27.875020 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 63868.7us
I20260709 13:08:27.854363 24467 raft_consensus.cc:2804] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 13:08:27.869144 24661 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.000s	sys 0.016s
I20260709 13:08:27.868512 24515 raft_consensus.cc:2804] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:08:27.876502 22992 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.000s	sys 0.017s
I20260709 13:08:27.876715 24691 raft_consensus.cc:2749] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:27.870147 22776 catalog_manager.cc:5697] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:27.878527 24587 raft_consensus.cc:2804] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:27.868162 24648 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:27.896165 22844 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:27.889791 23590 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:08:27.876273 22878 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.938s	user 0.010s	sys 0.017s
W20260709 13:08:27.890421 22920 leader_election.cc:336] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Timed out: RequestConsensusVote RPC to 127.21.63.1:33929 timed out after 1.780s (SENT)
I20260709 13:08:27.894853 22776 catalog_manager.cc:5697] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 1 to 3, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 3 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:27.927166 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:27.982339 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:27.983019 22946 raft_consensus.cc:2368] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 3. Current term is 4.
I20260709 13:08:27.899538 22844 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:27.981439 22920 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:27.930721 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:27.937068 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:08:27.976236 22919 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.772s	user 0.007s	sys 0.031s
I20260709 13:08:27.985049 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:08:27.985705 22919 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.781s	user 0.008s	sys 0.031s
I20260709 13:08:27.899655 23590 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:27.932952 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:27.986457 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:08:27.897575 22874 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.004s	sys 0.016s
I20260709 13:08:27.987392 23025 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:08:27.988116 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 781539us
I20260709 13:08:27.930924 22953 raft_consensus.cc:1275] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:27.928445 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:27.989532 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:27.956717 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:27.990300 23023 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
W20260709 13:08:27.987600 22874 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.006s	sys 0.017s
I20260709 13:08:27.984071 22951 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 2.
I20260709 13:08:27.993548 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:27.900004 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:08:27.994995 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 104593us
W20260709 13:08:27.984401 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 84961.1us
I20260709 13:08:27.994311 22955 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 2.
I20260709 13:08:27.889515 24467 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:27.999102 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:27.937794 22918 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:28.003968 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:28.004283 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:27.992025 22874 raft_consensus.cc:2955] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:28.008316 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:28.014048 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:27.892535 24515 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:28.003844 24630 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:27.893931 24407 consensus_queue.cc:1048] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.134s
I20260709 13:08:27.932119 24693 raft_consensus.cc:2804] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:27.900156 24648 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:28.026672 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:28.026614 23623 raft_consensus.cc:1240] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:28.027581 23619 raft_consensus.cc:1240] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:27.900038 24694 raft_consensus.cc:2804] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:28.027966 23029 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:27.930466 24661 raft_consensus.cc:2804] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:27.897513 24587 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:27.897387 24674 raft_consensus.cc:2749] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:28.004000 24467 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:28.029162 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:28.030102 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:28.048792 24515 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:08:28.030614 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 131059us
I20260709 13:08:28.031217 24490 raft_consensus.cc:2749] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:08:28.031271 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:28.031901 24700 raft_consensus.cc:3055] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:28.084975 24700 raft_consensus.cc:740] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:28.032493 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:28.053088 24693 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:28.035562 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:28.089269 22880 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 2.
I20260709 13:08:28.090829 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:28.035632 22879 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:28.092360 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:28.036494 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:28.039937 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:08:28.041002 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 51581.9us
I20260709 13:08:28.040366 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:28.094919 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:08:28.043972 23591 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.826s	user 0.009s	sys 0.021s
W20260709 13:08:28.095791 23591 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.878s	user 0.009s	sys 0.021s
I20260709 13:08:27.897325 24730 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:28.048460 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 486032us
I20260709 13:08:28.099110 22843 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:28.051738 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:28.048866 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:28.100328 22875 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
W20260709 13:08:28.052469 23024 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.081s	user 0.013s	sys 0.032s
W20260709 13:08:28.047047 24644 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.409s	user 0.011s	sys 0.030s
I20260709 13:08:28.030306 23025 raft_consensus.cc:2468] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
W20260709 13:08:28.063517 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 125819us
I20260709 13:08:28.068365 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:28.063983 22994 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:28.081056 23590 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:28.066429 24587 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:28.033555 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:28.086386 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:27.997723 22844 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:28.118606 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:28.064378 24694 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:28.121759 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:28.064229 24724 raft_consensus.cc:2749] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:28.123366 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:28.092895 24693 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:28.125065 22875 raft_consensus.cc:3055] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:28.126111 24417 raft_consensus.cc:2749] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:28.065378 22877 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:28.136969 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:28.137989 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:28.092970 22879 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 1.
I20260709 13:08:28.160904 23588 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:28.065914 24661 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
W20260709 13:08:28.162758 23024 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.198s	user 0.016s	sys 0.039s
I20260709 13:08:28.200071 23588 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:28.165930 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 102009us
I20260709 13:08:28.176476 24672 raft_consensus.cc:2804] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:08:28.173491 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 175842us
I20260709 13:08:28.174176 23025 raft_consensus.cc:1240] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 3. Current term is 4. Ops: []
W20260709 13:08:28.173960 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 93050.1us
I20260709 13:08:28.208573 23590 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:28.212265 22993 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:28.167356 23023 raft_consensus.cc:1240] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:28.175215 24695 raft_consensus.cc:2749] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:28.177096 22918 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:28.180152 22875 raft_consensus.cc:740] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
W20260709 13:08:28.164785 24644 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.533s	user 0.011s	sys 0.031s
I20260709 13:08:28.229259 24644 raft_consensus.cc:2749] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:28.229882 24644 raft_consensus.cc:2804] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:28.230217 24644 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:28.230542 24644 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:08:28.231348 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 54363.8us
I20260709 13:08:28.232065 22918 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:28.164604 24709 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.010s	sys 0.020s
I20260709 13:08:28.175518 24694 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:08:28.232833 24709 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.010s	sys 0.021s
I20260709 13:08:28.232954 24738 raft_consensus.cc:2804] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:28.194427 24630 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:28.233343 24738 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:28.195125 24467 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:28.233634 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:28.233523 24630 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:28.234236 24733 raft_consensus.cc:2749] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:28.235036 23020 raft_consensus.cc:2393] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 3: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:28.234956 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:28.235409 24467 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:28.235664 22877 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 3.
I20260709 13:08:28.196007 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:28.138988 24730 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:28.217877 24672 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:28.201272 24661 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:28.203931 24729 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:28.218169 24736 raft_consensus.cc:2804] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:28.225454 24742 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:28.218351 23024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 247) took 2537 ms. Trace:
I20260709 13:08:28.237043 22875 consensus_queue.cc:260] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:28.240311 22875 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:28.199056 24700 consensus_queue.cc:260] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:28.243646 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 43673.1us
I20260709 13:08:28.244432 22918 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:28.246323 22918 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:28.241845 24417 raft_consensus.cc:2804] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:28.247475 24417 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:28.247471 24733 raft_consensus.cc:2749] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:28.247975 24417 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:28.249382 22876 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:08:28.250123 24730 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:28.250886 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:28.251659 23620 raft_consensus.cc:2393] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 4: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:28.224350 22880 raft_consensus.cc:2468] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:28.252871 24515 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:28.240971 24476 consensus_queue.cc:1059] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 13:08:28.252029 24648 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:28.253381 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:28.217567 24723 raft_consensus.cc:2749] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:28.260713 23027 raft_consensus.cc:1240] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:28.235908 24630 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:28.261572 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:28.262360 23624 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:28.254470 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:28.163280 23029 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:28.239828 23024 rpcz_store.cc:276] 0709 13:08:25.664812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:25.664916 (+   104us) service_pool.cc:224] Handling call
0709 13:08:25.665183 (+   267us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:25.665470 (+   287us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:25.665494 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:25.666709 (+  1215us) log.cc:844] Serialized 26 byte log entry
0709 13:08:26.884249 (+1217540us) spinlock_profiling.cc:243] Waited 4.16 ms on lock 0x7b18003acc60. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f613598d4bc 00007f6132cdb48b 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf7c02 00007f613597f20b 00007f6135984d23 00007f61359850d1
0709 13:08:26.884466 (+   217us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:27.409767 (+525301us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b18003acc60. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cf83f3 00007f6132cfa7c1 00007f6132cfa779 00007f6132cfa709 00007f6132cfa6d1 00007f6132cf99cd 000000000038f354 000000000038f199 00007f6132cdb83a
0709 13:08:28.201579 (+791812us) log.cc:844] Serialized 14 byte log entry
0709 13:08:28.201855 (+   276us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:28.202108 (+   253us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:28.202157 (+    49us) raft_consensus.cc:1685] finished
0709 13:08:28.202175 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:28.202339 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":339,"prepare.queue_time_us":112011,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":47,"raft.queue_time_us":8742,"raft.run_cpu_time_us":257,"raft.run_wall_time_us":24137,"spinlock_wait_cycles":35149568,"thread_start_us":1275917,"threads_started":2,"wal-append.queue_time_us":33700}
I20260709 13:08:28.237937 24672 raft_consensus.cc:3060] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:28.256404 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:28.258666 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:28.265303 23026 raft_consensus.cc:1240] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:28.266271 22951 raft_consensus.cc:2393] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 2: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:28.267554 23590 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:28.257805 23588 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:28.269116 22918 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:28.238669 24736 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:28.236582 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:28.242542 24660 raft_consensus.cc:2749] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:28.233749 24738 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:28.254384 24490 raft_consensus.cc:2804] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:28.270908 24091 raft_consensus.cc:2749] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:08:28.318578 23624 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
I20260709 13:08:28.267390 24733 raft_consensus.cc:2804] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:28.322702 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:28.301640 24724 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:28.340546 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:28.340826 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:28.342330 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:28.329327 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:28.265699 24700 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:28.258419 24742 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:28.254223 24743 consensus_queue.cc:1059] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.075s
I20260709 13:08:28.346695 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:28.351181 23624 raft_consensus.cc:2393] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 5: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:28.303459 24723 raft_consensus.cc:2804] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:28.311187 24722 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:28.349742 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:28.319272 24490 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:28.313361 24736 raft_consensus.cc:3060] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:28.242122 24729 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:28.361784 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:08:28.373628 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 377691us
I20260709 13:08:28.374384 22875 raft_consensus.cc:1275] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:28.377305 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:28.356895 23022 raft_consensus.cc:1240] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 2. Current term is 3. Ops: []
W20260709 13:08:28.374827 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 275809us
I20260709 13:08:28.349932 24733 raft_consensus.cc:697] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:28.381273 22878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 1300) took 1617 ms. Trace:
I20260709 13:08:28.382025 23024 raft_consensus.cc:1240] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:28.382181 22843 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:28.386662 22994 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:08:28.388911 22920 leader_election.cc:336] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Timed out: RequestConsensusVote RPC to 127.21.63.1:33929 timed out after 2.235s (SENT)
I20260709 13:08:28.389366 22920 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:28.360702 24723 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:28.370868 24490 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:28.357055 24724 raft_consensus.cc:697] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:28.397347 22843 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:08:28.398182 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:28.398840 23020 raft_consensus.cc:2468] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 3.
I20260709 13:08:28.407182 22992 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:28.382781 24709 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:28.408556 22874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 1308) took 1199 ms. Trace:
I20260709 13:08:28.408967 22874 rpcz_store.cc:276] 0709 13:08:27.209314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:27.209708 (+   394us) service_pool.cc:224] Handling call
0709 13:08:27.231282 (+ 21574us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:27.238573 (+  7291us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:27.238627 (+    54us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:27.285943 (+ 47316us) log.cc:844] Serialized 513 byte log entry
0709 13:08:27.991901 (+705958us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:28.184584 (+192683us) log.cc:844] Serialized 14 byte log entry
0709 13:08:28.400699 (+216115us) log.cc:844] Serialized 14 byte log entry
0709 13:08:28.408225 (+  7526us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:28.408312 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:28.408354 (+    42us) raft_consensus.cc:1685] finished
0709 13:08:28.408371 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:28.408500 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7274,"prepare.queue_time_us":18270,"raft.queue_time_us":94619,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":161,"thread_start_us":836573,"threads_started":2,"wal-append.queue_time_us":776661}
I20260709 13:08:28.410552 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:28.410923 22874 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:28.390740 24708 raft_consensus.cc:2749] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:08:28.414757 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:28.416813 24455 raft_consensus.cc:2749] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:28.421459 24487 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:28.421980 24487 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:28.422276 24742 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:28.412081 24652 raft_consensus.cc:2749] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:28.426011 24417 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:28.428120 24417 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:28.433265 24724 consensus_queue.cc:237] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:28.435635 24587 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:28.401979 24515 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:28.397586 24743 raft_consensus.cc:2749] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:28.437696 24648 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:28.438309 24743 raft_consensus.cc:2749] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:28.440325 22919 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:28.441262 22992 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:28.415659 22994 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:28.444919 23616 raft_consensus.cc:1240] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:28.446410 24733 consensus_queue.cc:237] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:28.450989 22874 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
I20260709 13:08:28.452104 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:28.399485 24723 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:28.456995 22992 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:08:28.409090 22843 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:08:28.454316 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 176767us
I20260709 13:08:28.438664 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:28.464512 23590 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:08:28.462414 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 46873.3us
I20260709 13:08:28.465884 22994 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:28.442463 24662 raft_consensus.cc:2749] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:08:28.450081 24708 raft_consensus.cc:2804] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:28.469378 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:08:28.471025 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 61996us
I20260709 13:08:28.455214 23620 raft_consensus.cc:1240] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:28.473206 22874 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:28.443264 24487 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:28.461622 24626 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:28.461834 24455 raft_consensus.cc:2804] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:28.440323 24709 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:28.468596 24695 raft_consensus.cc:2804] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:28.492970 22994 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:28.511439 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
W20260709 13:08:28.520128 22843 connection.cc:256] RPC call timeout handler was delayed by -1.1013001230001009s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1916s for a response
W20260709 13:08:28.522042 22843 connection.cc:256] RPC call timeout handler was delayed by -1.0689001230000486s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1952s for a response
I20260709 13:08:28.484463 24362 raft_consensus.cc:2804] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:28.512475 23625 raft_consensus.cc:2376] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2. Re-sending same reply.
I20260709 13:08:28.486260 24571 consensus_queue.cc:1048] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:08:28.489310 24759 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:28.468899 24644 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:28.495857 24648 raft_consensus.cc:2804] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:28.486299 24455 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:28.497434 24695 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:28.484400 24518 consensus_queue.cc:1048] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 13:08:28.524727 24646 raft_consensus.cc:3055] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:28.529368 22992 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:28.526185 24362 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:28.533178 24759 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:28.528136 24417 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:28.548344 22880 raft_consensus.cc:2393] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 2: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:28.486088 24626 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:28.550899 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 55816.1us
I20260709 13:08:28.520682 24467 consensus_queue.cc:1059] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:08:28.538736 24695 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:08:28.579795 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 115348us
I20260709 13:08:28.541868 24455 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:28.539371 24648 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:28.543861 24743 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:28.584571 24416 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:28.546049 24646 raft_consensus.cc:740] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:28.551819 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:28.549614 24362 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:28.538076 24756 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:08:28.589144 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 65263.7us
I20260709 13:08:28.532011 24758 consensus_queue.cc:1059] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260709 13:08:28.549594 24417 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:28.549461 24759 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:28.581465 23590 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:28.587520 24648 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:08:28.600834 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 135677us
I20260709 13:08:28.605566 22994 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:28.593154 23621 raft_consensus.cc:1240] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:28.610430 22844 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:28.611169 22994 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:28.622432 22874 raft_consensus.cc:2468] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
I20260709 13:08:28.623294 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:28.623843 23024 raft_consensus.cc:1240] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:28.623826 22874 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:28.611474 22919 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:28.595106 24587 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:28.592855 24416 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:28.628698 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:28.629760 24662 consensus_queue.cc:1048] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:28.630048 24758 raft_consensus.cc:2749] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:28.630952 23028 raft_consensus.cc:1240] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:28.605769 22920 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:28.643918 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:28.644801 23028 raft_consensus.cc:2410] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:08:28.592916 24743 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:28.644929 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:28.645711 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:28.646271 23028 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 3.
I20260709 13:08:28.647090 23028 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "5b346767a89745fa82928a5dac9cc0b2"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:08:28.639128 24416 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:28.639537 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:28.608530 22877 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:28.617811 24476 raft_consensus.cc:3055] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:28.640421 24693 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:28.641652 22872 raft_consensus.cc:1275] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:28.642594 24720 raft_consensus.cc:2804] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:28.608780 24417 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:28.626075 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:28.603070 24764 raft_consensus.cc:2749] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:28.654263 22843 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:28.658592 24646 consensus_queue.cc:260] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:28.627734 24518 raft_consensus.cc:2804] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:28.602942 24708 raft_consensus.cc:697] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:28.678294 24557 consensus_queue.cc:1059] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:08:28.670463 23028 raft_consensus.cc:606] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Received request to transfer leadership to TS 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:28.679275 24708 consensus_queue.cc:237] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:28.680764 24757 raft_consensus.cc:2749] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:28.684415 24693 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:28.626618 24694 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:28.660699 24476 raft_consensus.cc:740] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:28.685809 23028 raft_consensus.cc:1240] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:28.686538 24694 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:28.686399 24476 consensus_queue.cc:260] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:28.687415 24476 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:28.684417 24416 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:28.687695 23616 raft_consensus.cc:1240] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:28.688550 23025 raft_consensus.cc:1240] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:28.690171 24416 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:28.632103 24571 raft_consensus.cc:2764] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
I20260709 13:08:28.672560 22843 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:28.599306 24724 consensus_queue.cc:1048] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 13:08:28.691764 24758 raft_consensus.cc:2749] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:28.691948 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:28.693650 22843 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:28.694425 24758 raft_consensus.cc:2749] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:28.674929 24518 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:28.608790 24756 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:28.677424 24646 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:28.677488 24626 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:28.678237 24644 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:28.679738 24557 raft_consensus.cc:3055] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:28.661800 24722 consensus_queue.cc:1059] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.029s
I20260709 13:08:28.627089 24760 raft_consensus.cc:2749] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:28.669555 24730 raft_consensus.cc:3055] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:28.696651 22843 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:28.696725 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:28.694375 24695 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:28.705933 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:28.650810 24691 consensus_queue.cc:1048] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.057s
W20260709 13:08:28.740509 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:28.767165 22874 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 4.
I20260709 13:08:28.715893 22994 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:28.707785 24729 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:28.749595 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 53137.4us
I20260709 13:08:28.750401 24733 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:28.696622 24518 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:08:28.755131 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 225784us
I20260709 13:08:28.708797 24767 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:28.673545 24743 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:28.747973 23625 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:28.768488 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:28.747260 24720 raft_consensus.cc:697] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
W20260709 13:08:28.732625 24752 log.cc:927] Time spent T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
I20260709 13:08:28.692620 22871 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 4.
I20260709 13:08:28.775897 22843 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:28.778134 22992 leader_election.cc:400] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 3. Current term is 4.
I20260709 13:08:28.787815 22992 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:08:28.755795 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 103016us
I20260709 13:08:28.789840 22920 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:28.781922 22874 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:08:28.769097 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 53271.1us
I20260709 13:08:28.732259 24557 raft_consensus.cc:740] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:28.609886 24762 consensus_queue.cc:1048] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.149s
I20260709 13:08:28.700206 24648 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:28.794211 22843 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:28.749080 24730 raft_consensus.cc:740] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:28.777719 24416 raft_consensus.cc:2804] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:28.688584 24455 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:28.700635 24417 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:28.801152 23588 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:28.696517 24759 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:28.802278 22920 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:28.802539 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 152312us
I20260709 13:08:28.799809 22843 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:28.821704 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:28.757860 24490 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:28.824034 22843 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:28.821570 23588 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:28.798514 24660 raft_consensus.cc:2749] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:28.814502 24768 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:28.825098 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:28.825021 24557 consensus_queue.cc:260] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:28.817307 24708 raft_consensus.cc:2749] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:28.815872 24416 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:28.828073 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:28.826133 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:28.830510 22920 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:28.797451 24743 raft_consensus.cc:2749] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:28.834920 22919 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:28.832361 24776 raft_consensus.cc:2749] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:28.779744 24733 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:28.826280 24644 consensus_queue.cc:1059] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:28.795971 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:28.838431 24652 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:28.775525 24766 consensus_queue.cc:1059] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.088s
I20260709 13:08:28.817500 24722 raft_consensus.cc:2749] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:28.862291 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:28.863005 22949 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:08:28.847317 24416 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:28.774048 24738 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:28.844976 23623 raft_consensus.cc:1240] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 5. Current term is 6. Ops: [3.6-5.7]
I20260709 13:08:28.844889 22994 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:28.838718 24768 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:28.849303 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:28.830823 24764 raft_consensus.cc:2749] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:28.833838 24659 raft_consensus.cc:2749] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:28.850682 22920 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:28.874059 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:28.852133 24557 raft_consensus.cc:3060] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:28.854544 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:28.855245 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:28.878732 22920 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:28.856907 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:28.841392 24407 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:28.839709 23588 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:28.857316 24770 raft_consensus.cc:2749] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:28.859205 24691 raft_consensus.cc:2749] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:08:28.869342 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:28.797101 24729 consensus_queue.cc:1048] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 13:08:28.983729 22873 raft_consensus.cc:3055] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:28.987093 22873 raft_consensus.cc:740] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:28.987385 22952 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 1816) took 1702 ms. Trace:
I20260709 13:08:28.862794 22843 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:28.844028 24587 consensus_queue.cc:1059] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
W20260709 13:08:28.991478 24777 log.cc:927] Time spent T 00000000000000000000000000000000 P 8d175b4ae72f4fc48971c2f31668971c: Append to log took a long time: real 0.129s	user 0.004s	sys 0.005s
I20260709 13:08:28.987780 22952 rpcz_store.cc:276] 0709 13:08:27.285165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:27.285898 (+   733us) service_pool.cc:224] Handling call
0709 13:08:27.286296 (+   398us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:27.878649 (+592353us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:27.878681 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:27.897823 (+ 19142us) log.cc:844] Serialized 52 byte log entry
0709 13:08:28.362413 (+464590us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:28.362452 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:28.362529 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:28.987179 (+624650us) raft_consensus.cc:1685] finished
0709 13:08:28.987202 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:28.987334 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6006,"raft.queue_time_us":76470,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":74,"thread_start_us":943910,"threads_started":2,"wal-append.queue_time_us":521749}
I20260709 13:08:28.859869 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:28.864056 24417 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:28.873414 24743 consensus_queue.cc:1059] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.109s
I20260709 13:08:28.871317 24776 raft_consensus.cc:3055] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:28.980497 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:28.873320 24467 raft_consensus.cc:2749] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:28.880116 24771 raft_consensus.cc:2804] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:28.817306 24730 consensus_queue.cc:260] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:28.983898 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 144254us
I20260709 13:08:28.988749 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:28.981041 24766 raft_consensus.cc:2749] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:08:28.983289 22874 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:08:28.992799 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:29.006810 22952 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
I20260709 13:08:28.989144 24763 raft_consensus.cc:2804] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:08:28.993757 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 131035us
W20260709 13:08:28.986251 24672 consensus_meta.cc:287] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.667s	user 0.011s	sys 0.010s
I20260709 13:08:28.985910 24762 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:29.007782 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:08:28.967526 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 89653.8us
I20260709 13:08:29.010145 22994 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:29.010401 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:29.021795 22992 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:29.022912 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:29.023564 22949 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:08:28.990798 23027 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 1347) took 1004 ms. Trace:
I20260709 13:08:29.013206 22843 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:28.992174 24587 raft_consensus.cc:3055] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:29.001432 24776 raft_consensus.cc:740] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:29.001168 24771 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:29.013677 24743 consensus_queue.cc:1243] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer f3c799b1249e40119d2a5864231c6c75 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:08:29.003331 24695 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:28.870870 24720 consensus_queue.cc:237] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:29.062075 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:29.030998 22951 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:36566
I20260709 13:08:29.095564 22951 raft_consensus.cc:606] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:08:29.096057 22951 raft_consensus.cc:614] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:29.035063 24763 raft_consensus.cc:697] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:29.061283 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:29.066495 24733 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:29.017169 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:29.030402 22843 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:29.097321 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:08:29.099685 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:29.030234 24730 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:29.029901 24694 raft_consensus.cc:2804] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:29.067019 24771 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:29.027791 24587 raft_consensus.cc:740] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:29.060525 24659 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:29.061146 24758 raft_consensus.cc:2749] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:08:29.107824 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:29.118610 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:29.111573 23625 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:29.112409 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:08:29.105456 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 75134.1us
I20260709 13:08:29.120585 22879 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:29.116814 24746 raft_consensus.cc:2804] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:29.121577 22843 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:29.122479 22843 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:29.101711 24455 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:29.107573 24490 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:29.098747 24756 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:29.023960 24672 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:29.118098 24723 consensus_meta.cc:287] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.650s	user 0.010s	sys 0.000s
I20260709 13:08:29.108209 24759 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:29.156118 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:29.157318 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:29.156543 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:29.159025 22953 raft_consensus.cc:2393] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 4: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:29.161183 22879 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:08:29.107134 24416 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:29.128820 24694 raft_consensus.cc:697] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:29.125970 24722 raft_consensus.cc:2749] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:29.025574 24762 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:29.175400 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:29.113083 24743 raft_consensus.cc:2749] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:29.166141 21756 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260709 13:08:29.175958 23624 raft_consensus.cc:1240] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:29.177593 21756 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:08:29.151876 24700 consensus_meta.cc:287] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.781s	user 0.013s	sys 0.007s
I20260709 13:08:29.180202 24762 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:29.180998 24416 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:29.181061 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:29.167028 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:29.184707 22953 raft_consensus.cc:1240] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 LEADER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:29.167048 22994 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:29.168337 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:29.186486 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:29.167285 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:29.170465 24694 consensus_queue.cc:237] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:29.173617 24782 raft_consensus.cc:2749] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:29.174023 24780 raft_consensus.cc:2749] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:29.151209 24746 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:29.178795 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:29.167100 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:29.025656 23027 rpcz_store.cc:276] 0709 13:08:27.984999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:27.992453 (+  7454us) service_pool.cc:224] Handling call
0709 13:08:28.028290 (+ 35837us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:28.261150 (+232860us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:28.956882 (+695732us) spinlock_profiling.cc:243] Waited 226 ms on lock 0x7b54002ca954. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:28.989387 (+ 32505us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":519176960}
I20260709 13:08:29.164577 24587 consensus_queue.cc:260] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:29.190119 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:29.191443 23023 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 4.
I20260709 13:08:29.204178 22876 raft_consensus.cc:2368] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 3. Current term is 4.
I20260709 13:08:29.192863 24700 raft_consensus.cc:2804] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:29.197154 23029 raft_consensus.cc:2393] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 4: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:29.206949 22992 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:29.207896 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:29.208557 22876 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:29.163699 24723 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:29.203652 24768 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:29.117009 24776 consensus_queue.cc:260] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:29.198344 24736 consensus_meta.cc:287] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.817s	user 0.016s	sys 0.000s
I20260709 13:08:29.193782 24407 raft_consensus.cc:2749] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 13:08:29.204912 24661 consensus_meta.cc:287] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.960s	user 0.011s	sys 0.005s
I20260709 13:08:29.198098 24767 raft_consensus.cc:2749] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:29.196797 24738 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:29.184929 24763 consensus_queue.cc:237] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:29.226958 24587 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:29.227327 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:29.195811 24746 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:29.233490 22954 raft_consensus.cc:2393] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 4: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:29.233300 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:29.220086 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:29.234274 22876 raft_consensus.cc:2393] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 4: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:29.234706 22953 raft_consensus.cc:1240] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:29.214690 24662 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:29.238005 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:29.203783 24722 consensus_queue.cc:1059] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
W20260709 13:08:29.242865 24772 log.cc:927] Time spent T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.237s	user 0.000s	sys 0.001s
I20260709 13:08:29.217739 24700 raft_consensus.cc:697] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:28.383742 22878 rpcz_store.cc:276] 0709 13:08:26.764554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:26.764706 (+   152us) service_pool.cc:224] Handling call
0709 13:08:26.772151 (+  7445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:26.772389 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:08:27.898793 (+1126404us) log.cc:844] Serialized 14 byte log entry
0709 13:08:28.380945 (+482152us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:28.381003 (+    58us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:28.381047 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:28.381109 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:28.381219 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1089745,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":52,"raft.queue_time_us":56569,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":75,"thread_start_us":1441967,"threads_started":3,"wal-append.queue_time_us":484323}
I20260709 13:08:29.249015 24672 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:29.248958 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:29.250732 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:29.252197 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:29.247387 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:29.253794 22876 raft_consensus.cc:2376] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4. Re-sending same reply.
I20260709 13:08:29.244068 24407 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:29.228899 24661 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:29.262990 24723 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:29.219141 24736 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:29.247834 24722 consensus_queue.cc:1243] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer aed3f089ec8b49af9c8cf5cdee76e8e7 log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
W20260709 13:08:29.265568 24784 log.cc:927] Time spent T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
W20260709 13:08:29.273762 24754 log.cc:927] Time spent T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.842s	user 0.004s	sys 0.000s
I20260709 13:08:29.274205 24652 raft_consensus.cc:2749] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:29.266489 24700 consensus_queue.cc:237] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:29.269133 24787 log.cc:927] Time spent T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.104s	user 0.000s	sys 0.001s
I20260709 13:08:29.263623 24407 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:29.298563 23617 raft_consensus.cc:1240] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:29.252897 23027 raft_consensus.cc:1240] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:29.275161 23021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 1352) took 1288 ms. Trace:
I20260709 13:08:29.255484 24662 raft_consensus.cc:697] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:29.300266 24407 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:29.280071 24646 consensus_meta.cc:287] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.564s	user 0.007s	sys 0.004s
I20260709 13:08:29.283200 22874 raft_consensus.cc:2393] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 2: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:29.296027 23026 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 1348) took 1300 ms. Trace:
I20260709 13:08:29.299691 23021 rpcz_store.cc:276] 0709 13:08:27.986341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:27.986440 (+    99us) service_pool.cc:224] Handling call
0709 13:08:28.028454 (+ 42014us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:28.039060 (+ 10606us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:28.039088 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:28.039486 (+   398us) log.cc:844] Serialized 26 byte log entry
0709 13:08:28.347258 (+307772us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:08:28.347295 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:28.347363 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:29.270599 (+923236us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:08:29.274875 (+  4276us) raft_consensus.cc:1685] finished
0709 13:08:29.274897 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:29.275108 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10252,"raft.queue_time_us":11097,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":5163,"spinlock_wait_cycles":23876992,"thread_start_us":307502,"threads_started":1,"wal-append.queue_time_us":373074}
I20260709 13:08:29.302117 23026 rpcz_store.cc:276] 0709 13:08:27.995669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:28.123528 (+127859us) service_pool.cc:224] Handling call
0709 13:08:28.265121 (+141593us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:28.266253 (+  1132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:29.295735 (+1029482us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b54000bc7d0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:29.295938 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":324979968}
I20260709 13:08:29.305125 24662 consensus_queue.cc:237] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:29.310196 22872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 1733) took 1375 ms. Trace:
I20260709 13:08:29.311328 22954 raft_consensus.cc:1240] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:29.314642 24661 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:29.310607 22872 rpcz_store.cc:276] 0709 13:08:27.934402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:28.095385 (+160983us) service_pool.cc:224] Handling call
0709 13:08:28.095652 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:28.661060 (+565408us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:29.309936 (+648876us) spinlock_profiling.cc:243] Waited 546 ms on lock 0x7b54002a1cd4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:29.310134 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1254684928}
I20260709 13:08:29.324074 22843 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:29.325709 22843 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:29.327095 22843 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:29.332154 23026 raft_consensus.cc:1275] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:08:29.333513 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:29.334333 23026 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:29.334717 22843 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:29.335678 24652 raft_consensus.cc:2749] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:29.337112 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:29.337347 24646 raft_consensus.cc:2749] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:08:29.338923 23625 raft_consensus.cc:1275] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:08:29.328255 24362 consensus_meta.cc:287] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.711s	user 0.010s	sys 0.001s
I20260709 13:08:29.344945 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:29.330916 24661 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:29.329053 24758 raft_consensus.cc:2749] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:29.351477 23026 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
I20260709 13:08:29.337883 24769 consensus_queue.cc:1048] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:29.352404 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:29.329058 22871 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:29.359134 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:29.359858 22871 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:29.366297 22994 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:29.339792 24487 consensus_queue.cc:1059] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 13:08:29.338461 24630 consensus_queue.cc:1059] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 13:08:29.343492 24646 consensus_queue.cc:1059] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 13:08:29.347877 24652 consensus_queue.cc:1059] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.021s
I20260709 13:08:29.380447 24743 raft_consensus.cc:3055] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:29.366840 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:29.404066 24743 raft_consensus.cc:740] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:29.407408 22994 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:29.409034 24736 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:29.409215 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:29.376425 23621 raft_consensus.cc:1275] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:08:29.409674 24660 raft_consensus.cc:2749] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:08:29.409825 22877 raft_consensus.cc:3060] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:29.410844 24742 raft_consensus.cc:3055] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:29.411211 24742 raft_consensus.cc:740] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:29.411983 24742 consensus_queue.cc:260] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:29.413136 24742 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:29.384119 22871 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
W20260709 13:08:29.417164 24518 consensus_meta.cc:287] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.633s	user 0.011s	sys 0.000s
I20260709 13:08:29.417225 23022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 1349) took 1417 ms. Trace:
I20260709 13:08:29.417716 24518 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:29.418238 23025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 262) took 1045 ms. Trace:
I20260709 13:08:29.418720 23025 rpcz_store.cc:276] 0709 13:08:28.372650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:28.372968 (+   318us) service_pool.cc:224] Handling call
0709 13:08:28.373341 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:28.688970 (+315629us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:29.418045 (+729075us) spinlock_profiling.cc:243] Waited 315 ms on lock 0x7b5400248e54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:29.418188 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":724456192}
I20260709 13:08:29.419799 23023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32794 (request call id 259) took 1095 ms. Trace:
I20260709 13:08:29.419970 24518 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:29.420280 23023 rpcz_store.cc:276] 0709 13:08:28.323963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:28.329003 (+  5040us) service_pool.cc:224] Handling call
0709 13:08:29.419655 (+1090652us) spinlock_profiling.cc:243] Waited 836 ms on lock 0x7b54002da5d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:29.419744 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1923699968}
I20260709 13:08:29.420715 23020 raft_consensus.cc:1240] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:29.388619 23624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 390) took 1122 ms. Trace:
I20260709 13:08:29.421096 22877 raft_consensus.cc:2468] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 4.
I20260709 13:08:29.421332 23624 rpcz_store.cc:276] 0709 13:08:28.266297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:28.361161 (+ 94864us) service_pool.cc:224] Handling call
0709 13:08:28.361404 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:29.176323 (+814919us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:29.388362 (+212039us) spinlock_profiling.cc:243] Waited 814 ms on lock 0x7b54003d9ad4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:29.388558 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1872845056}
I20260709 13:08:29.398077 22880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53724 (request call id 222) took 1191 ms. Trace:
I20260709 13:08:29.400861 24790 raft_consensus.cc:2764] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
I20260709 13:08:29.349068 24362 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:29.433461 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:29.434037 22875 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:29.434527 24362 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:29.405706 23623 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 386) took 1147 ms. Trace:
I20260709 13:08:29.435257 23623 rpcz_store.cc:276] 0709 13:08:28.258642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:28.260792 (+  2150us) service_pool.cc:224] Handling call
0709 13:08:28.261150 (+   358us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:28.867463 (+606313us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:29.405331 (+537868us) spinlock_profiling.cc:243] Waited 583 ms on lock 0x7b540049a2d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:29.405635 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1342003456}
I20260709 13:08:29.436781 23024 raft_consensus.cc:1240] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 2. Current term is 4. Ops: []
I20260709 13:08:29.370219 23027 raft_consensus.cc:1240] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:29.419018 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:29.384996 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:29.426046 22994 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:29.422536 22880 rpcz_store.cc:276] 0709 13:08:28.206290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:28.342059 (+135769us) service_pool.cc:224] Handling call
0709 13:08:29.397893 (+1055834us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b5400301e54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:29.398019 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":471619840}
I20260709 13:08:29.442977 22880 raft_consensus.cc:1240] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:08:29.419799 23622 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60044 (request call id 430) took 1136 ms. Trace:
I20260709 13:08:29.444105 23622 rpcz_store.cc:276] 0709 13:08:28.283472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:28.419502 (+136030us) service_pool.cc:224] Handling call
0709 13:08:28.693199 (+273697us) spinlock_profiling.cc:243] Waited 2.61 ms on lock 0x7b58001c4d98. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0a22 00007f613bbd9c09 00007f613ad33ee0 00007f613adfd0ab 00007f613adfd131 00007f613adbefc0 00007f613ada9929 00007f613ada93d2 00007f6134320bd4
0709 13:08:28.693368 (+   169us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:28.769594 (+ 76226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:28.769637 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:28.769674 (+    37us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:28.769707 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:29.419600 (+649893us) spinlock_profiling.cc:243] Waited 76 ms on lock 0x7b54004943d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:29.419646 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:29.419744 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":180868736}
I20260709 13:08:29.428884 24727 raft_consensus.cc:3055] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:29.431474 22878 raft_consensus.cc:2410] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:08:29.452057 22875 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 2.
I20260709 13:08:29.452987 22872 raft_consensus.cc:1240] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:08:29.454267 22875 raft_consensus.cc:1240] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:08:29.454567 22872 raft_consensus.cc:1240] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:29.456089 22872 raft_consensus.cc:1240] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:29.460767 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:29.461052 22994 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:29.461344 22872 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:29.464110 23590 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:29.465986 23590 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:29.462796 24727 raft_consensus.cc:740] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:29.417749 23022 rpcz_store.cc:276] 0709 13:08:27.999789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:28.129360 (+129571us) service_pool.cc:224] Handling call
0709 13:08:28.129670 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:28.378042 (+248372us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:29.416997 (+1038955us) spinlock_profiling.cc:243] Waited 210 ms on lock 0x7b54002c9554. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:29.417169 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":482764160}
I20260709 13:08:29.467901 23590 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:29.471351 24490 raft_consensus.cc:2804] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:29.473062 22872 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:08:29.458628 22871 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:29.475054 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:29.475732 23623 raft_consensus.cc:1240] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 3. Current term is 4. Ops: []
W20260709 13:08:29.476723 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 136245us
I20260709 13:08:29.477393 23624 raft_consensus.cc:1240] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:29.481364 24743 consensus_queue.cc:260] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:29.464248 24416 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:08:29.503166 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 169637us
I20260709 13:08:29.467275 24518 raft_consensus.cc:2804] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:29.463641 24362 consensus_queue.cc:1059] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.020s
I20260709 13:08:29.482686 24736 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:29.479135 24490 raft_consensus.cc:697] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 4 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:29.515372 24518 raft_consensus.cc:697] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 2 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:29.518178 24518 consensus_queue.cc:237] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:29.520789 24795 log.cc:927] Time spent T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 13:08:29.521342 24761 raft_consensus.cc:2804] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:29.521047 24490 consensus_queue.cc:237] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:29.521843 24761 raft_consensus.cc:697] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:29.508728 24416 raft_consensus.cc:697] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 5 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:29.522680 24761 consensus_queue.cc:237] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:29.504448 24691 consensus_queue.cc:1048] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 13:08:29.465351 24746 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:29.543469 24746 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:29.533085 22993 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:29.505517 24708 consensus_queue.cc:1048] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 13:08:29.549181 22993 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:29.549571 22879 raft_consensus.cc:2393] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 5: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:29.559186 22993 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:29.556589 24362 raft_consensus.cc:2749] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:08:29.561053 24621 raft_consensus.cc:2804] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:29.561141 24571 raft_consensus.cc:2749] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:29.514582 24771 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:29.593462 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:29.634899 24771 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:29.638914 23624 raft_consensus.cc:1240] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 2. Current term is 3. Ops: []
W20260709 13:08:29.639303 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 176421us
I20260709 13:08:29.639883 24785 consensus_queue.cc:1059] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 13:08:29.641534 23623 raft_consensus.cc:1240] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:08:29.643013 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:29.643218 23024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 1355) took 1019 ms. Trace:
I20260709 13:08:29.643548 23024 rpcz_store.cc:276] 0709 13:08:28.623570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:28.661528 (+ 37958us) service_pool.cc:224] Handling call
0709 13:08:28.661827 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:29.437240 (+775413us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:29.643022 (+205782us) spinlock_profiling.cc:243] Waited 775 ms on lock 0x7b5400359654. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:29.643168 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1781519104}
I20260709 13:08:29.644032 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:29.644646 23025 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 3.
I20260709 13:08:29.621342 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:29.646121 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:29.646538 22994 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:29.619554 24801 consensus_queue.cc:1059] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:08:29.562175 24674 consensus_queue.cc:1059] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 13:08:29.571453 24662 consensus_queue.cc:1059] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 13:08:29.622529 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:29.621389 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:29.611570 24416 consensus_queue.cc:237] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:29.651384 23588 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:29.637295 24621 raft_consensus.cc:697] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:29.655704 23588 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:29.658032 24764 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:29.659301 24621 consensus_queue.cc:237] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:29.667207 24709 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:29.664438 24587 raft_consensus.cc:2749] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:29.669037 24805 raft_consensus.cc:2804] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:29.669530 24805 raft_consensus.cc:697] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:29.670378 24805 consensus_queue.cc:237] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:29.672384 24674 consensus_queue.cc:1243] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer 5b346767a89745fa82928a5dac9cc0b2 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:08:29.672891 24662 raft_consensus.cc:3055] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:29.673413 24662 raft_consensus.cc:740] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:29.674449 24662 consensus_queue.cc:260] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:29.676129 24662 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:29.667312 24776 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:29.682221 23023 raft_consensus.cc:1240] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:29.682931 23028 raft_consensus.cc:1240] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:29.664245 23588 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:29.684342 23020 raft_consensus.cc:1275] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:08:29.692437 24455 consensus_queue.cc:1059] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:08:29.691360 23588 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:29.700439 23588 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:29.685472 24764 consensus_queue.cc:1059] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 13:08:29.691056 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:08:29.708771 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:29.694267 24490 raft_consensus.cc:2804] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:29.720748 24763 raft_consensus.cc:2749] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:08:29.729460 22871 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:29.732790 24736 raft_consensus.cc:2749] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:29.734994 23623 raft_consensus.cc:1240] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:29.735522 22994 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:29.740864 22876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 1752) took 1029 ms. Trace:
I20260709 13:08:29.707039 24587 raft_consensus.cc:2804] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:29.741482 22876 rpcz_store.cc:276] 0709 13:08:28.711859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:29.675643 (+963784us) service_pool.cc:224] Handling call
0709 13:08:29.675898 (+   255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:29.676077 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 13:08:29.676337 (+   260us) log.cc:844] Serialized 14 byte log entry
0709 13:08:29.740405 (+ 64068us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:29.740469 (+    64us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:08:29.740521 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:29.740601 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:29.740799 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":7522,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"thread_start_us":43348,"threads_started":1}
I20260709 13:08:29.770154 22878 raft_consensus.cc:1275] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:08:29.734961 23027 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:08:29.779116 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:29.783406 22949 raft_consensus.cc:1240] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:29.720496 24648 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:29.699095 24709 consensus_queue.cc:1059] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.072s
I20260709 13:08:29.752509 24806 raft_consensus.cc:2804] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:29.772349 24587 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:29.772877 24362 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
W20260709 13:08:29.799983 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 99629.5us
I20260709 13:08:29.749545 24490 raft_consensus.cc:697] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 4 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:29.743218 24467 consensus_queue.cc:1059] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 13:08:29.814198 23616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60044 (request call id 431) took 1530 ms. Trace:
I20260709 13:08:29.814632 23616 rpcz_store.cc:276] 0709 13:08:28.283779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:28.445914 (+162135us) service_pool.cc:224] Handling call
0709 13:08:28.461961 (+ 16047us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:28.688147 (+226186us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:29.808972 (+1120825us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b54002bb6d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:29.814137 (+  5165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":507974400}
I20260709 13:08:29.796339 24674 consensus_queue.cc:1048] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 13:08:29.808655 24806 raft_consensus.cc:697] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:29.817757 24806 consensus_queue.cc:237] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:29.807301 24362 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:29.831104 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:29.801908 24587 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:29.802132 24709 consensus_queue.cc:1243] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer f3c799b1249e40119d2a5864231c6c75 log is divergent from this leader: its last log entry 3.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:08:29.831944 22878 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:08:29.838660 23588 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:29.841836 24723 raft_consensus.cc:2749] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:08:29.845772 24803 log.cc:927] Time spent T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 13:08:29.850260 24490 consensus_queue.cc:237] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:29.864068 24812 log.cc:927] Time spent T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:08:29.873054 22844 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:29.876890 22844 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:29.852382 24362 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:29.885459 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:29.874943 24730 raft_consensus.cc:2749] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:08:29.890239 23624 raft_consensus.cc:1240] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:29.849691 24783 consensus_queue.cc:1048] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 13:08:29.883036 24709 raft_consensus.cc:2749] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 13:08:29.892571 24807 log.cc:927] Time spent T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.158s	user 0.000s	sys 0.001s
I20260709 13:08:29.868021 24764 consensus_queue.cc:1059] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:08:29.892693 24729 raft_consensus.cc:3055] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:08:29.917586 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:29.921957 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 443366us
I20260709 13:08:29.906268 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:29.917672 24729 raft_consensus.cc:740] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:29.931942 22872 raft_consensus.cc:1275] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:08:29.933692 22872 raft_consensus.cc:1275] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:08:29.934788 24757 raft_consensus.cc:3055] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:08:29.935209 24757 raft_consensus.cc:740] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:29.935984 22918 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:29.937924 24771 raft_consensus.cc:2804] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:29.938521 24771 raft_consensus.cc:697] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:29.939544 24771 consensus_queue.cc:237] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:29.944015 24487 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:29.944521 24487 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:29.947317 24487 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:29.947777 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:29.949123 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:29.949726 22951 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:29.950371 24487 consensus_queue.cc:1059] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:29.951596 24720 raft_consensus.cc:3055] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:29.951968 24720 raft_consensus.cc:740] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:29.952837 24720 consensus_queue.cc:260] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:29.954144 24720 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:29.923822 24709 consensus_queue.cc:1059] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:08:29.960387 23617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 395) took 1102 ms. Trace:
I20260709 13:08:29.960813 23617 rpcz_store.cc:276] 0709 13:08:28.857690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:28.986588 (+128898us) service_pool.cc:224] Handling call
0709 13:08:29.006588 (+ 20000us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:29.299062 (+292474us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:29.960160 (+661098us) spinlock_profiling.cc:243] Waited 285 ms on lock 0x7b540045bad4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:29.960329 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":654408448}
I20260709 13:08:29.963330 24729 consensus_queue.cc:260] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:29.963523 24757 consensus_queue.cc:260] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:29.964381 24729 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:29.965394 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:29.966131 22953 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:29.967654 24762 raft_consensus.cc:3055] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:29.968030 24762 raft_consensus.cc:740] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:29.973348 22953 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 5.
I20260709 13:08:29.976028 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:29.977218 22946 raft_consensus.cc:1240] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 LEADER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:08:29.978394 24707 consensus_queue.cc:1059] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:29.978734 24762 consensus_queue.cc:260] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:29.979502 24707 raft_consensus.cc:3055] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:29.979938 24707 raft_consensus.cc:740] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:29.980105 24762 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:29.982595 22873 consensus_queue.cc:260] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:29.983655 22873 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:29.984926 24757 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:29.985743 22874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53724 (request call id 232) took 1298 ms. Trace:
I20260709 13:08:29.986131 22874 rpcz_store.cc:276] 0709 13:08:28.687216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:29.018950 (+331734us) service_pool.cc:224] Handling call
0709 13:08:29.985522 (+966572us) spinlock_profiling.cc:243] Waited 183 ms on lock 0x7b540022a6d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:29.985680 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":420030208}
I20260709 13:08:29.988801 22880 raft_consensus.cc:1240] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:29.990008 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:29.991310 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:29.991441 22920 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:29.993167 22949 raft_consensus.cc:1240] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:29.995316 22951 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:08:30.023228 22953 raft_consensus.cc:1275] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:08:30.027221 22949 raft_consensus.cc:1240] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:30.019876 24587 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:30.030517 24587 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:30.030700 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:08:30.032407 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:30.033097 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:30.034616 24819 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:30.035146 24819 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:30.027849 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:30.032495 22873 raft_consensus.cc:1275] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:30.038713 22873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 1732) took 2104 ms. Trace:
I20260709 13:08:30.039134 22873 rpcz_store.cc:276] 0709 13:08:27.934051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:27.937254 (+  3203us) service_pool.cc:224] Handling call
0709 13:08:27.937591 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:28.902303 (+964712us) spinlock_profiling.cc:243] Waited 260 ms on lock 0x7b54000cbfa0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135d5cdc0 00007f6135de71c0 00007f6135de8231 00007f6135de4d8f 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c
0709 13:08:29.817836 (+915533us) spinlock_profiling.cc:243] Waited 80.1 ms on lock 0x7b54000cbfa0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135d6306f 00007f6135ddbf60 00007f6135ddd7de 00007f6135de7b49 00007f6135de8416 00007f6135de4d8f 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62
0709 13:08:29.982283 (+164447us) spinlock_profiling.cc:243] Waited 19 ms on lock 0x7b54000cbfa0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135d62520 00007f6135ddbf70 00007f6135ddd7de 00007f6135de7b49 00007f6135de8416 00007f6135de4d8f 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62
0709 13:08:30.038336 (+ 56053us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:30.038665 (+   329us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":825610752}
I20260709 13:08:30.003013 24660 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:30.043452 24674 consensus_queue.cc:1048] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:08:30.046059 24764 consensus_queue.cc:1059] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:08:30.041545 24707 consensus_queue.cc:260] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:30.051846 22877 raft_consensus.cc:1240] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 1. Current term is 2. Ops: []
W20260709 13:08:30.051981 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:30.050627 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:30.052325 23623 raft_consensus.cc:1240] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 LEADER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:30.059130 22879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35360 (request call id 1321) took 1070 ms. Trace:
W20260709 13:08:30.060603 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 84047.5us
I20260709 13:08:30.059495 22879 rpcz_store.cc:276] 0709 13:08:28.989226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:29.166892 (+177666us) service_pool.cc:224] Handling call
0709 13:08:30.058937 (+892045us) spinlock_profiling.cc:243] Waited 352 ms on lock 0x7b54001ba154. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:30.059071 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":810154752}
I20260709 13:08:30.063062 23020 raft_consensus.cc:1240] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 5 LEADER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:30.063256 23025 raft_consensus.cc:1275] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:08:30.063205 23023 raft_consensus.cc:1240] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 3. Current term is 4. Ops: []
W20260709 13:08:30.041900 24658 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 13:08:30.066174 23020 raft_consensus.cc:1240] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:30.115592 23023 raft_consensus.cc:1240] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:30.098299 22871 raft_consensus.cc:1275] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:08:30.129997 24630 consensus_queue.cc:1059] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:30.130829 24630 consensus_queue.cc:1243] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer f3c799b1249e40119d2a5864231c6c75 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:08:30.131268 24630 consensus_queue.cc:1048] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:30.132782 24720 consensus_queue.cc:1059] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:30.133484 24720 consensus_queue.cc:1243] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer f3c799b1249e40119d2a5864231c6c75 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 13:08:30.134379 24819 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:30.134559 24720 consensus_queue.cc:1059] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:30.135349 24720 consensus_queue.cc:1243] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer f3c799b1249e40119d2a5864231c6c75 log is divergent from this leader: its last log entry 3.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:08:30.135972 24720 raft_consensus.cc:3055] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:30.136463 24720 raft_consensus.cc:740] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:30.136989 24771 consensus_queue.cc:1059] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:30.137238 24743 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:30.137305 24720 consensus_queue.cc:260] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:30.138160 24738 raft_consensus.cc:3055] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:30.138500 24720 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:30.138571 24738 raft_consensus.cc:740] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:30.139592 24738 consensus_queue.cc:260] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:30.140553 24738 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:30.146045 23027 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 5.
I20260709 13:08:30.114461 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:30.067981 24783 consensus_queue.cc:1048] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.029s
W20260709 13:08:30.077477 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:30.104450 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:08:30.109026 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 117646us
I20260709 13:08:30.054716 24660 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:30.049157 24736 raft_consensus.cc:3055] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:30.150905 24736 raft_consensus.cc:740] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:30.151798 24660 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:30.151975 24736 consensus_queue.cc:260] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:30.153551 24736 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:30.154520 22920 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:30.127070 24691 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:30.159843 24691 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:30.161964 24691 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:30.127455 24822 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:30.123731 24750 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:30.162866 24750 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:30.136047 22875 raft_consensus.cc:1275] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:08:30.101930 24731 raft_consensus.cc:3055] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:08:30.158192 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:30.158679 24658 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:30.166126 22843 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:30.166119 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:30.049696 24707 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:30.167325 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:30.168015 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:30.172571 22920 leader_election.cc:400] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 3. Current term is 4.
I20260709 13:08:30.170018 24806 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:30.178423 22875 raft_consensus.cc:2468] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:30.178557 22920 leader_election.cc:403] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:30.180016 22920 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:30.199333 24571 raft_consensus.cc:3055] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:30.199770 24571 raft_consensus.cc:740] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:30.182943 22920 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:30.209573 22920 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:30.177062 24691 raft_consensus.cc:2749] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:30.193284 24750 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:30.212885 24630 raft_consensus.cc:2749] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:30.214964 22920 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:30.182020 24783 raft_consensus.cc:2764] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:08:30.228435 24783 consensus_queue.cc:1059] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:30.193881 24762 raft_consensus.cc:2749] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 3. Current term is 4.
I20260709 13:08:30.229537 24762 consensus_queue.cc:1059] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:30.230491 24762 raft_consensus.cc:3055] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:30.230829 24762 raft_consensus.cc:740] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:30.231568 24762 consensus_queue.cc:260] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:30.182173 24709 raft_consensus.cc:2804] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:08:30.232546 22920 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:30.233433 24819 consensus_queue.cc:1048] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:30.225451 24691 raft_consensus.cc:2749] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:30.222914 24738 raft_consensus.cc:2749] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:30.233002 24709 raft_consensus.cc:697] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 5 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:30.245787 24709 consensus_queue.cc:237] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:30.232832 24771 consensus_queue.cc:1048] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:08:30.249287 22871 raft_consensus.cc:1240] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:30.216373 23022 raft_consensus.cc:2393] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 3: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:30.203729 24806 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:30.251972 24817 consensus_queue.cc:1059] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:08:30.233610 24823 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:30.202698 24658 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:30.253190 24823 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:30.254791 24658 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:30.256294 24823 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:30.258767 23588 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:30.259780 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:30.260212 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:30.260404 22871 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
I20260709 13:08:30.236598 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:30.242789 22843 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:30.252285 24806 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:30.170749 24731 raft_consensus.cc:740] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:30.235141 24720 raft_consensus.cc:2749] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:30.265902 23022 raft_consensus.cc:1275] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:08:30.266623 24816 raft_consensus.cc:2749] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:08:30.267555 22951 raft_consensus.cc:1275] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:08:30.272972 22920 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:30.234437 24762 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:30.279919 22920 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:30.241055 24822 raft_consensus.cc:2749] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:30.274035 24757 raft_consensus.cc:2749] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:30.299937 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:30.284103 24816 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:30.284530 24738 raft_consensus.cc:2804] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:30.285702 24720 raft_consensus.cc:2749] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:08:30.311481 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:30.312041 22878 raft_consensus.cc:1275] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:08:30.312403 22878 pending_rounds.cc:85] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 1
I20260709 13:08:30.312633 22878 pending_rounds.cc:107] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:08:30.312968 22878 raft_consensus.cc:2889] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:30.305080 24817 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:30.304927 24738 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:30.309597 24672 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:30.321105 24738 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:30.307126 24816 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:30.344014 23023 raft_consensus.cc:1240] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:30.310005 24731 consensus_queue.cc:260] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:30.329742 24672 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:30.322647 24817 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:30.390611 24727 consensus_queue.cc:260] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:30.568701 22880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 1329) took 1572 ms. Trace:
I20260709 13:08:30.569324 22880 rpcz_store.cc:276] 0709 13:08:28.996017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:29.443613 (+447596us) service_pool.cc:224] Handling call
0709 13:08:29.988545 (+544932us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:29.989293 (+   748us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:30.568347 (+579054us) spinlock_profiling.cc:243] Waited 545 ms on lock 0x7b540022a6d0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:30.568590 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1252541696}
I20260709 13:08:30.799482 24767 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
W20260709 13:08:30.822045 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:30.823853 22843 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:08:30.825173 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:30.801486 24738 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:30.693478 24630 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:30.802615 24709 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:30.803387 24743 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:30.806733 24757 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
W20260709 13:08:30.803402 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:30.807477 24836 log.cc:927] Time spent T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.458s	user 0.003s	sys 0.000s
I20260709 13:08:30.808401 22994 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:30.808719 24816 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:30.809702 24515 consensus_queue.cc:1048] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 13:08:30.815092 24821 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:30.829648 22875 raft_consensus.cc:1240] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:30.856144 23624 raft_consensus.cc:1240] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:30.857996 22879 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:30.819432 24763 raft_consensus.cc:3055] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:08:30.891868 22879 raft_consensus.cc:1275] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:08:30.827111 24723 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:30.817021 24708 raft_consensus.cc:3055] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:30.843469 24646 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:08:30.853737 24830 log.cc:927] Time spent T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.543s	user 0.000s	sys 0.011s
W20260709 13:08:30.851414 24831 log.cc:927] Time spent T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.543s	user 0.010s	sys 0.000s
I20260709 13:08:30.898227 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:30.842694 24767 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:30.892421 24819 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:30.891718 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:30.891469 24763 raft_consensus.cc:740] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:30.861524 24630 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:30.864144 24709 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:30.901188 24708 raft_consensus.cc:740] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:30.870249 24757 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:30.867065 24743 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:30.869781 24833 consensus_queue.cc:1048] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.097s
I20260709 13:08:30.973241 24819 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:30.967235 24822 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:30.980643 24767 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:30.980428 24571 consensus_queue.cc:260] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:30.952252 24646 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:30.980614 24763 consensus_queue.cc:260] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:30.982728 24630 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:30.991930 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 137599us
I20260709 13:08:30.949298 24723 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:30.895043 24821 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:30.995151 24806 raft_consensus.cc:2804] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:31.020566 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:31.021260 22950 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:08:31.012413 23588 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:31.013804 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:31.023402 22950 raft_consensus.cc:1275] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 13:08:31.023720 22950 pending_rounds.cc:85] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting all ops after (but not including) 4
W20260709 13:08:31.023913 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 215562us
I20260709 13:08:31.024107 22951 raft_consensus.cc:1240] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:31.023929 22950 pending_rounds.cc:107] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 13:08:31.025249 22950 raft_consensus.cc:2889] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:30.993985 24660 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:30.997133 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 141111us
I20260709 13:08:31.018819 22952 raft_consensus.cc:1275] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:30.994366 24817 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:31.035883 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:31.036681 22947 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:31.045444 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:31.046211 23027 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
I20260709 13:08:31.049513 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:31.050247 23027 raft_consensus.cc:2393] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 4: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:31.013173 24571 raft_consensus.cc:3060] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:31.050669 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:31.048256 23590 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:31.014287 24813 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.025292 24806 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:31.061816 23027 raft_consensus.cc:1275] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 13:08:31.063711 24707 consensus_meta.cc:287] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.867s	user 0.009s	sys 0.004s
I20260709 13:08:31.064224 24707 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.064688 24707 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:31.029202 24763 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:30.999419 24822 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:31.037751 24832 raft_consensus.cc:2749] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:08:31.068738 24806 raft_consensus.cc:3060] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:08:31.076198 24762 consensus_meta.cc:287] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.791s	user 0.010s	sys 0.005s
I20260709 13:08:31.045567 24660 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:31.066746 24672 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:31.088222 24630 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:31.093556 24842 raft_consensus.cc:2749] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:08:31.036918 24708 consensus_queue.cc:260] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:31.069880 24813 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:31.153997 24849 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.154738 24849 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:31.102981 24841 consensus_queue.cc:1048] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:08:31.159113 24646 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:31.159883 24646 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.160431 24646 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:31.162940 24646 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:31.113996 24738 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:31.115530 24846 raft_consensus.cc:2749] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:31.118700 24844 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:31.176049 24757 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:31.177654 24743 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:31.178501 24844 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:31.182596 24819 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:31.184348 24821 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:31.193889 24844 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:31.201284 24822 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:31.209072 24813 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:31.144471 24837 log.cc:927] Time spent T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.335s	user 0.007s	sys 0.000s
I20260709 13:08:31.217429 24849 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:30.178501 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:08:31.261608 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 743037us
W20260709 13:08:31.277444 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:31.285616 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:31.286571 22947 raft_consensus.cc:1275] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:08:31.288383 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:31.289558 22948 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:08:31.060091 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:31.291213 23025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 1368) took 1225 ms. Trace:
I20260709 13:08:31.291308 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:31.291550 23022 raft_consensus.cc:2393] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 4: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:31.291715 23025 rpcz_store.cc:276] 0709 13:08:30.065311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:30.066061 (+   750us) service_pool.cc:224] Handling call
0709 13:08:30.066337 (+   276us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:30.066639 (+   302us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:30.066668 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:30.104916 (+ 38248us) log.cc:844] Serialized 26 byte log entry
0709 13:08:30.155325 (+ 50409us) spinlock_profiling.cc:243] Waited 2.77 ms on lock 0x7b18001cc540. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cdae07 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf7c02 00007f613597f20b 00007f6135984d23 00007f61359850d1 00007f6135da0a0f
0709 13:08:30.805053 (+649728us) spinlock_profiling.cc:243] Waited 1.59 ms on lock 0x7b18001cc540. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cc62a6 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f613598d4bc 00007f6132cdb48b
0709 13:08:30.806609 (+  1556us) spinlock_profiling.cc:243] Waited 2.36 ms on lock 0x7b18001cc540. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f613598d4bc 00007f6132cdb48b 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf7c02 00007f613597f20b 00007f6135984d23 00007f61359850d1
0709 13:08:30.806751 (+   142us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:08:30.806801 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:30.806870 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:31.159765 (+352895us) spinlock_profiling.cc:243] Waited 2.09 ms on lock 0x7b18001cc540. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cf83f3 00007f6132cfa7c1 00007f6132cfa779 00007f6132cfa709 00007f6132cfa6d1 00007f6132cf99cd 000000000038f354 000000000038f199 00007f6132cdb83a
0709 13:08:31.290441 (+130676us) raft_consensus.cc:1685] finished
0709 13:08:31.290504 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:31.290908 (+   404us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13117,"raft.queue_time_us":364,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":34311,"spinlock_wait_cycles":20263040,"thread_start_us":17826,"threads_started":1,"wal-append.queue_time_us":68448}
I20260709 13:08:31.285693 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:31.303642 22951 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
I20260709 13:08:31.318020 24866 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:31.320575 24866 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:31.340241 22951 raft_consensus.cc:1275] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:31.324627 24866 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:31.356592 22948 raft_consensus.cc:1275] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:08:31.412473 24813 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.412681 24851 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.413141 24852 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.413663 24813 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:31.414172 24852 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:31.415649 24864 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:31.416766 24743 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:31.416716 24864 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:31.417784 24743 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:31.414040 24851 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:31.129367 24630 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:31.404188 24866 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:31.418810 24766 raft_consensus.cc:3055] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:08:31.435542 24766 raft_consensus.cc:740] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:31.435650 22878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 1336) took 1506 ms. Trace:
I20260709 13:08:31.435856 24844 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:31.419054 24646 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.437054 24844 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:31.421983 24660 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:31.438058 24646 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:31.440142 24844 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:31.440835 24849 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.441740 24849 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:31.416800 24816 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:31.443948 24816 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:31.445222 24646 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:31.445664 24757 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.445863 24646 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
W20260709 13:08:31.426628 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:31.447844 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:31.436147 22878 rpcz_store.cc:276] 0709 13:08:29.929353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:29.930760 (+  1407us) service_pool.cc:224] Handling call
0709 13:08:29.931671 (+   911us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:30.313372 (+381701us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:31.435147 (+1121775us) spinlock_profiling.cc:243] Waited 380 ms on lock 0x7b54001bb554. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:31.435477 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":875403392}
I20260709 13:08:31.448834 24848 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.446775 24646 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:31.449698 24743 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:31.449934 24846 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:31.430660 24817 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:31.450814 24846 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:31.451476 24817 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:31.451618 24852 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:31.454017 24817 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:31.432272 24731 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:31.458343 24708 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:08:31.461876 22844 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.005s	sys 0.023s
W20260709 13:08:31.463094 22844 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.005s	sys 0.023s
I20260709 13:08:31.127055 24672 consensus_queue.cc:1048] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.339s
I20260709 13:08:31.466796 24691 consensus_queue.cc:1048] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:31.467161 24865 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.467980 24865 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:31.447988 24707 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:31.446450 24757 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:31.420794 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 141406us
W20260709 13:08:31.419239 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 144089us
I20260709 13:08:31.425344 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:31.176443 24709 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:31.476344 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:31.430204 24630 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:31.432534 24864 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:31.430352 24851 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:31.484333 24709 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:31.485185 24709 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:31.485836 22918 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:31.486094 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:31.486600 24407 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.486858 23624 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:08:31.487308 24407 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:31.487807 24417 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.488534 24417 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:31.489784 23622 raft_consensus.cc:1240] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 5. Current term is 6. Ops: [3.6-5.7]
I20260709 13:08:31.491405 24417 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:31.449714 24848 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:31.491494 24854 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:31.492583 24869 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.494591 24848 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:31.434439 24866 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:31.499287 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:31.492772 24854 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:31.429616 24727 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:31.450335 24743 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:31.504142 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:31.504354 24857 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.505081 24857 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:31.505789 22879 raft_consensus.cc:1275] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:08:31.502413 24854 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:31.513890 22948 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 3.
I20260709 13:08:31.513759 24515 consensus_queue.cc:1059] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:08:31.514480 22873 raft_consensus.cc:1275] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:08:31.515151 24515 raft_consensus.cc:3055] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:31.515528 24515 raft_consensus.cc:740] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:31.516790 24672 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:31.517405 24672 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:31.519153 23616 raft_consensus.cc:1240] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:08:31.520221 24757 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:31.520780 24757 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.521209 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:31.521356 24757 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:31.521886 22880 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 3.
I20260709 13:08:31.522840 24723 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:31.523610 24757 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:31.525631 24857 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:31.516815 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:31.500730 24709 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:31.524253 24757 consensus_queue.cc:1048] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 13:08:31.511682 24865 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:31.543958 24455 consensus_queue.cc:1048] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:31.545435 22880 raft_consensus.cc:1240] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:08:31.551064 24806 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:31.516517 24515 consensus_queue.cc:260] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:31.552498 22955 raft_consensus.cc:1275] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:08:31.555137 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:31.536016 24869 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:31.556072 23620 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 3.
I20260709 13:08:31.557057 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:31.548741 22950 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:31.508811 24844 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:31.555358 22776 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33488 (request call id 265) took 6452 ms. Trace:
I20260709 13:08:31.508678 24864 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:31.533576 24672 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:31.535367 24813 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:31.566843 24515 raft_consensus.cc:3060] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:31.567417 24407 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:31.437086 24766 consensus_queue.cc:260] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:31.568449 24766 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:31.567894 24806 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:31.566875 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:31.571782 22875 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 3.
I20260709 13:08:31.573659 24866 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:31.575457 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:31.592162 23624 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
I20260709 13:08:31.575851 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:31.576930 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:31.578426 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:31.593732 23622 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 3.
I20260709 13:08:31.594419 22947 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 3.
I20260709 13:08:31.595316 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:31.595867 22947 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 4.
I20260709 13:08:31.579193 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:31.581040 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:31.563385 23616 raft_consensus.cc:2393] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 4: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:31.597664 24854 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:31.599364 22949 raft_consensus.cc:1240] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:31.582110 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:31.582799 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:31.584561 24846 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:31.585848 24869 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:31.586555 24646 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:31.588186 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:31.588511 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:31.610069 22879 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:31.589040 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:31.610404 24709 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.610505 24646 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.611573 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:31.611220 24709 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:31.610406 24863 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:31.590077 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:31.611235 22955 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 2.
I20260709 13:08:31.612690 24863 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:31.612843 22878 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:31.589393 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:31.590147 24816 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:31.597666 22951 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 5.
I20260709 13:08:31.607537 24849 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:31.608094 22948 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 2.
I20260709 13:08:31.609418 24866 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:31.611285 24646 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:31.612133 22879 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:08:31.590996 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:31.562742 24743 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:31.621258 22874 raft_consensus.cc:1275] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:31.621567 22880 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
I20260709 13:08:31.620429 22950 raft_consensus.cc:1275] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 13:08:31.624902 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:31.628118 24646 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:31.642627 24646 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:31.643416 24646 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:31.563634 24844 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:31.626725 22844 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:31.652416 24869 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:31.653036 24771 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:31.653198 24869 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:31.653741 24771 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:31.656100 24771 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:31.656855 24738 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:31.657251 24709 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:31.628520 24743 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:31.635185 24864 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:31.658470 24720 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.561844 22776 rpcz_store.cc:276] 0709 13:08:25.102817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:25.102966 (+   149us) service_pool.cc:224] Handling call
0709 13:08:31.555077 (+6452111us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:28.524500 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:28.649399 (+124899us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:28.652174 (+  2775us) write_op.cc:435] Acquired schema lock
0709 13:08:28.655008 (+  2834us) tablet.cc:662] Decoding operations
0709 13:08:28.706376 (+ 51368us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:28.706439 (+    63us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:28.706474 (+    35us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:08:28.707135 (+   661us) tablet.cc:701] Row locks acquired
0709 13:08:28.707153 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:08:28.707293 (+   140us) write_op.cc:270] Start()
0709 13:08:28.707404 (+   111us) write_op.cc:276] Timestamp: P: 1783602508707254 usec, L: 0
0709 13:08:28.707429 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 13:08:28.729956 (+ 22527us) log.cc:844] Serialized 3190 byte log entry
0709 13:08:29.192302 (+462346us) op_driver.cc:464] REPLICATION: finished
0709 13:08:29.353286 (+160984us) write_op.cc:301] APPLY: starting
0709 13:08:29.353400 (+   114us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:29.378125 (+ 24725us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:29.378239 (+   114us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:29.379178 (+   939us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:29.379375 (+   197us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=8
0709 13:08:29.379425 (+    50us) write_op.cc:312] APPLY: finished
0709 13:08:29.379673 (+   248us) log.cc:844] Serialized 65 byte log entry
0709 13:08:29.379846 (+   173us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:29.379961 (+   115us) write_op.cc:454] Released schema lock
0709 13:08:29.380008 (+    47us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:29.380352 (+   344us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:29.380381 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":20,"child_traces":[["op",{"apply.queue_time_us":152308,"apply.run_cpu_time_us":2877,"apply.run_wall_time_us":32484,"num_ops":8,"prepare.queue_time_us":422130,"prepare.run_cpu_time_us":11423,"prepare.run_wall_time_us":327513,"raft.queue_time_us":165293,"raft.run_cpu_time_us":47668,"raft.run_wall_time_us":70051,"replication_time_us":484734,"thread_start_us":571381,"threads_started":4,"wal-append.queue_time_us":130334,"wal-append.run_cpu_time_us":86676,"wal-append.run_wall_time_us":1983511}]]}
I20260709 13:08:31.659097 24720 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:31.660163 24864 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:31.660983 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 96970.7us
I20260709 13:08:31.661588 22992 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:31.661931 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:31.663946 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:31.664086 24571 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:31.664536 24571 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:31.665010 24571 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:31.665287 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:31.665503 23024 raft_consensus.cc:1217] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 5 LEADER]: Deduplicated request from leader. Original: 2.2->[4.3-4.3]   Dedup: 4.3->[]
I20260709 13:08:31.666078 23024 raft_consensus.cc:1240] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 5 LEADER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:08:31.666163 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:31.666879 23625 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:31.666929 24691 consensus_queue.cc:1048] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:31.628712 24866 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:31.669348 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:31.669764 24866 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:31.670518 22993 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:31.670642 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:31.671172 24868 consensus_queue.cc:1059] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:31.672287 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:31.672487 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:31.672873 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:31.672307 24455 raft_consensus.cc:2804] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:31.672964 23026 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6.
I20260709 13:08:31.674690 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:31.672593 22876 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:31.674490 24455 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:31.677071 23624 raft_consensus.cc:2393] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 6: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:31.679620 24455 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:31.680894 24844 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:31.680954 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:31.681695 24863 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:31.672279 24868 raft_consensus.cc:3055] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:08:31.682334 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:31.683061 23616 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
W20260709 13:08:31.683801 24763 consensus_meta.cc:287] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.605s	user 0.014s	sys 0.001s
I20260709 13:08:31.684593 22992 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:31.685205 24783 consensus_queue.cc:1048] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:31.685511 24762 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:31.686209 24762 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:31.687157 24806 consensus_queue.cc:1048] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:31.682458 24868 raft_consensus.cc:740] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:31.696092 24868 consensus_queue.cc:260] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:31.697521 24868 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:31.700296 22874 raft_consensus.cc:1240] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:31.703814 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:31.704459 24571 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:31.677556 24869 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:31.657891 24822 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:31.705288 24790 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.706010 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:31.692726 23588 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:31.693740 24851 raft_consensus.cc:2804] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:31.708056 23021 raft_consensus.cc:2393] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 5: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:31.704962 24646 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:31.705806 24822 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:31.692881 24662 consensus_queue.cc:1048] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:31.690788 24762 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:31.706851 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 103126us
I20260709 13:08:31.708250 24851 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:31.720564 22844 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:31.723080 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:31.720870 22873 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:31.726783 23588 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:31.729359 23623 raft_consensus.cc:1240] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:31.729449 24849 raft_consensus.cc:2804] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:31.730008 24849 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:31.730415 24849 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:31.731256 24727 consensus_queue.cc:1048] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:31.731436 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:31.732853 24821 consensus_queue.cc:1059] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 13:08:31.734992 24854 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:31.705878 24790 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:31.717808 24709 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:31.740918 24727 consensus_queue.cc:1048] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:31.741581 24816 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:31.742027 24766 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.742242 23588 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:31.725674 24851 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:31.747148 22918 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:31.748509 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:31.749332 22844 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:31.749794 24822 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:31.768898 24720 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:31.657482 24738 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:31.750247 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:31.750365 24846 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:31.751156 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:31.751803 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:31.771641 23025 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
I20260709 13:08:31.741039 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:31.755173 22919 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:31.757313 22878 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 5.
I20260709 13:08:31.737946 22875 raft_consensus.cc:1275] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:31.753109 24750 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:31.743000 24763 consensus_queue.cc:1059] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:08:31.761947 23588 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:31.765136 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:31.765431 24731 raft_consensus.cc:2804] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:31.767597 23590 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:31.760505 24816 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:31.773020 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:31.769577 24727 consensus_queue.cc:1048] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:31.770635 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:31.771376 24846 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:31.758569 24709 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:31.777169 23024 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 3.
I20260709 13:08:31.787074 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:31.796939 23588 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:31.797348 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:31.798241 22920 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:31.795388 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:31.800858 22876 raft_consensus.cc:1275] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:08:31.791265 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:31.798621 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:31.803261 22880 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:31.795025 23617 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 5.
I20260709 13:08:31.805420 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:31.806811 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:31.808035 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:31.779408 24731 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:31.809381 24731 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:31.790969 24846 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:31.791162 24762 raft_consensus.cc:2804] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:31.812489 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:31.813056 24762 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:31.801467 24750 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:31.813484 24762 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:31.814878 24571 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:31.815220 24750 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:31.815709 24750 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.813766 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:31.776966 24863 raft_consensus.cc:2804] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:31.759763 24766 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:31.817651 22880 raft_consensus.cc:1240] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:31.816601 23023 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 3.
I20260709 13:08:31.783573 24708 raft_consensus.cc:2804] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:31.783134 24816 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:31.777091 23022 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:31.824441 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:31.821026 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:31.836879 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:31.791355 24709 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:31.836934 22878 raft_consensus.cc:3060] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:31.830461 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:31.838974 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:08:31.840320 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 337671us
I20260709 13:08:31.841328 22873 raft_consensus.cc:1275] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:08:31.842269 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:31.843091 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:31.777062 24407 consensus_queue.cc:1048] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:08:31.846817 22950 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:08:31.848271 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:31.848312 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:31.842727 22875 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:31.841246 22843 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:31.795768 24417 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:31.819245 24863 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:31.849870 22876 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 3.
I20260709 13:08:31.827674 24723 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:31.824091 24848 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.816216 24750 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:31.859288 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:31.862105 23021 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 3.
I20260709 13:08:31.840760 24672 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:31.836345 24708 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:08:31.862671 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:31.863421 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:08:31.864084 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:31.860003 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:31.841131 24571 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.864746 22950 raft_consensus.cc:1275] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:08:31.866256 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 68087.1us
I20260709 13:08:31.866662 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:08:31.867736 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 100380us
I20260709 13:08:31.851599 24854 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:31.857805 24863 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:31.875003 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:31.874110 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:31.878189 23028 raft_consensus.cc:1275] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:08:31.879058 22920 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:31.861704 24723 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:31.881136 23021 raft_consensus.cc:1275] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:08:31.865785 24672 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:31.883908 23025 raft_consensus.cc:1240] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 4. Current term is 5. Ops: []
W20260709 13:08:31.885820 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:31.886600 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:31.886608 24856 consensus_queue.cc:1048] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.101s
I20260709 13:08:31.910055 24720 raft_consensus.cc:2749] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:31.912132 24790 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:31.887709 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:31.913300 23621 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 3.
I20260709 13:08:31.875427 24417 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:31.888339 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:31.915079 24844 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.915448 24417 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:31.915722 24844 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:31.888460 24750 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:31.872709 24708 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:31.888940 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:31.888994 24571 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:31.938534 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:31.941418 24863 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:31.942718 22874 raft_consensus.cc:2393] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 4: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:31.948736 24862 raft_consensus.cc:2804] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:31.949265 24862 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:31.949644 24862 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:31.950222 24571 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:31.955191 24762 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:31.896277 22878 raft_consensus.cc:2468] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:08:31.898283 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:31.958961 22947 raft_consensus.cc:2393] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 5: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:31.900461 23588 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:08:31.959553 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 59121.7us
I20260709 13:08:31.959041 23029 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 3.
I20260709 13:08:31.868189 24848 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:31.904147 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:31.905524 22992 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:31.907130 23617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60044 (request call id 441) took 1052 ms. Trace:
I20260709 13:08:31.908842 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:31.908983 24813 raft_consensus.cc:3055] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:08:31.884043 23024 raft_consensus.cc:1275] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:08:31.854882 24763 consensus_queue.cc:1048] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:08:31.919245 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:08:31.919859 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 47672.9us
I20260709 13:08:31.920737 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:31.969527 22873 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 5.
I20260709 13:08:31.878224 24854 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:31.894948 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:31.903940 24490 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:31.884961 24723 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:31.960238 23588 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:31.961336 24878 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:31.961596 22872 raft_consensus.cc:1240] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:08:31.962698 23025 raft_consensus.cc:1240] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:08:31.918625 24672 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:08:31.972014 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 130881us
W20260709 13:08:31.972509 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 67067.7us
I20260709 13:08:31.963534 22877 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:31.968982 22920 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:31.982432 23624 raft_consensus.cc:2393] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 5: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
W20260709 13:08:31.985368 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 78989.9us
I20260709 13:08:31.986471 22843 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:31.987063 24849 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:31.987993 22920 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:31.961352 24783 raft_consensus.cc:3055] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:31.988969 24783 raft_consensus.cc:740] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:31.989141 22920 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:31.990134 24783 consensus_queue.cc:260] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:31.991515 24783 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:31.992828 24587 raft_consensus.cc:2804] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:31.993381 24587 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:31.993808 24587 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:31.994699 24848 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:31.995244 24848 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:31.975193 24455 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:31.998157 24731 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:31.999163 24455 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:31.978363 24738 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:32.000207 24731 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:32.003053 24846 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:32.004215 22871 raft_consensus.cc:2393] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 5: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:32.007354 24846 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:31.989173 24849 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:32.009029 23029 raft_consensus.cc:1240] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:32.011185 24762 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:31.976081 24869 raft_consensus.cc:2804] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:32.013262 24869 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:32.012872 24490 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:31.975733 24646 raft_consensus.cc:2804] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:32.015537 24646 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:32.016049 24646 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:31.978125 24857 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:32.016511 24878 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:32.017364 24857 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:32.019044 24878 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:32.024459 22920 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:32.030078 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 43659us
I20260709 13:08:31.974468 24854 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:08:32.054620 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 68347.7us
I20260709 13:08:31.971817 24813 raft_consensus.cc:740] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
W20260709 13:08:32.111799 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 61816.4us
I20260709 13:08:32.016185 24490 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:31.967450 23617 rpcz_store.cc:276] 0709 13:08:30.855168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:30.859189 (+  4021us) service_pool.cc:224] Handling call
0709 13:08:31.906886 (+1047697us) spinlock_profiling.cc:243] Waited 767 ms on lock 0x7b5400650ed4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:31.907047 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1763710720}
W20260709 13:08:32.123351 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 99060us
I20260709 13:08:32.048717 24821 raft_consensus.cc:2804] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:32.013841 24869 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:32.021261 24844 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:32.019867 24857 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:32.049499 24738 raft_consensus.cc:2804] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:32.065213 24846 raft_consensus.cc:2804] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:32.004515 24708 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:32.117977 24771 raft_consensus.cc:2804] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:08:32.146746 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 186566us
I20260709 13:08:32.148870 23588 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:32.126739 24822 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:32.134568 24821 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:32.151945 22843 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:32.146530 24738 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:08:32.146220 24855 log.cc:927] Time spent T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 13:08:32.159217 24738 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:32.160636 22994 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:32.163403 24817 raft_consensus.cc:2804] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:32.163822 24817 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:32.164170 24817 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:32.166363 22951 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
I20260709 13:08:32.147507 24846 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:32.124063 24750 consensus_queue.cc:1048] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
W20260709 13:08:32.169015 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 68465.9us
W20260709 13:08:32.168181 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 248621us
I20260709 13:08:32.163080 24851 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:32.130617 24727 consensus_queue.cc:1048] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 13:08:32.171897 22843 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:32.157938 24821 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:32.171329 24857 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:32.212591 23026 raft_consensus.cc:2393] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 5: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:32.213661 22843 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:32.165475 24771 raft_consensus.cc:697] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:32.173357 24490 raft_consensus.cc:2804] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:32.173234 24844 raft_consensus.cc:2749] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:32.173604 24846 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:08:32.228495 24883 log.cc:927] Time spent T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:08:32.226975 23590 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:32.212296 24743 raft_consensus.cc:2804] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:32.177191 24856 consensus_queue.cc:1048] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
W20260709 13:08:32.241487 24880 log.cc:927] Time spent T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.222s	user 0.000s	sys 0.001s
I20260709 13:08:32.248587 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:32.257930 22954 raft_consensus.cc:2393] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 3: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:32.249199 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:32.220589 24490 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:32.260998 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 47416.7us
I20260709 13:08:32.258458 22950 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 2.
W20260709 13:08:32.266403 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:32.266989 24771 consensus_queue.cc:237] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:32.276360 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:32.279249 24848 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:32.279685 24848 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:32.280064 24848 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:32.280608 24490 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:32.259021 24862 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:32.286862 24708 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:32.242343 24757 raft_consensus.cc:2749] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:32.287832 24862 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:32.226158 24857 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:32.252249 24743 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:32.289657 24743 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:32.291095 24857 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:32.291966 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 65063.7us
I20260709 13:08:32.293871 23590 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:32.296016 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:32.299719 24731 raft_consensus.cc:2749] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:32.302330 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:32.302625 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:32.303864 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:32.305187 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:32.168424 24813 consensus_queue.cc:260] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:32.306564 24849 consensus_queue.cc:1059] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:08:32.264261 22843 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:32.316365 24857 raft_consensus.cc:2749] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:08:32.317279 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 53157.8us
I20260709 13:08:32.317735 24851 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:32.318511 24816 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:32.319442 23622 raft_consensus.cc:1240] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:32.212116 24763 consensus_queue.cc:1048] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:08:32.322202 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:32.323213 23622 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:08:32.325040 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:32.329041 23622 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
I20260709 13:08:32.300393 24709 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:32.331202 24846 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:32.332896 24709 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:32.334669 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:32.331581 24863 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:32.336237 22843 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:32.342501 24863 raft_consensus.cc:2749] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:32.342958 24766 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:32.344216 22843 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:32.345443 24863 raft_consensus.cc:2749] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:32.337040 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:32.349113 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:32.349318 24672 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:32.337702 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:32.350931 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:32.338587 24856 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:32.351034 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:08:32.339452 24881 log.cc:927] Time spent T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.127s	user 0.000s	sys 0.004s
W20260709 13:08:32.351715 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 88928.2us
I20260709 13:08:32.352366 24889 consensus_queue.cc:1059] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:08:32.392447 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:32.340237 24587 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:32.341231 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:32.402045 22874 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:32.396525 22993 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:32.354929 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:32.355185 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:08:32.355796 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:32.358477 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:32.358428 24490 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:32.359395 24691 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:32.359622 24884 log.cc:927] Time spent T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.068s	user 0.004s	sys 0.000s
W20260709 13:08:32.367962 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 206480us
I20260709 13:08:32.370224 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:32.385138 23620 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
I20260709 13:08:32.385952 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:32.360219 24743 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:32.389902 23625 raft_consensus.cc:2393] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 3: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:32.389921 22919 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:32.352715 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:32.320569 24816 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:32.406864 22993 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:32.410754 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:32.427899 22878 raft_consensus.cc:2393] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 5: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:32.410976 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:32.413558 22871 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 4.
W20260709 13:08:32.423776 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 50641.9us
I20260709 13:08:32.424161 23622 raft_consensus.cc:2393] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 4: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:32.386605 24821 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:32.433763 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:32.430928 22992 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:32.440120 22877 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 4.
I20260709 13:08:32.413951 24750 raft_consensus.cc:2749] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:32.443224 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:32.424176 24771 raft_consensus.cc:2749] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:32.439138 24846 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:32.426860 24889 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:32.438854 22919 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.63.4:44043 timed out after 1.542s (SENT) [suppressed 10 similar messages]
I20260709 13:08:32.460537 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:32.406257 24856 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:32.446684 23618 raft_consensus.cc:2393] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 4: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
W20260709 13:08:32.472070 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 55196.3us
I20260709 13:08:32.474547 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:32.466545 23617 raft_consensus.cc:2393] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 4: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:32.425058 24691 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:32.484568 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:32.478137 22844 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:32.491561 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:32.471517 24727 raft_consensus.cc:2749] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:08:32.497696 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 56845.6us
I20260709 13:08:32.501797 22994 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:08:32.502050 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 95253.8us
I20260709 13:08:32.505196 22993 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:32.507783 22993 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:32.493067 24846 raft_consensus.cc:2804] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:08:32.503878 24888 log.cc:927] Time spent T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.133s	user 0.002s	sys 0.000s
I20260709 13:08:32.458429 24515 consensus_queue.cc:1048] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.047s
I20260709 13:08:32.470976 24889 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:32.511979 22993 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:32.513528 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:32.503451 24763 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
W20260709 13:08:32.510084 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 77804.4us
I20260709 13:08:32.512250 22994 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:32.517297 22993 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:32.508836 24455 consensus_queue.cc:1059] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 13:08:32.510782 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:32.530809 24691 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:32.506812 24867 raft_consensus.cc:2804] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:32.507169 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:32.513952 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:32.550832 24867 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:32.551115 22952 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:32.551156 23620 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:32.551535 24515 raft_consensus.cc:2749] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:32.551266 24867 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:32.552505 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:32.553498 23620 raft_consensus.cc:2368] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 3. Current term is 4.
I20260709 13:08:32.554611 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:32.555683 23620 raft_consensus.cc:2393] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 3: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:32.556473 24821 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:32.556694 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:32.513679 24723 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:32.527232 24743 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:32.526815 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:32.528028 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:32.520771 24848 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:32.521093 24646 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:32.528810 24898 log.cc:927] Time spent T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:08:32.550077 24910 raft_consensus.cc:3055] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:32.520995 22953 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 3.
I20260709 13:08:32.549912 23621 raft_consensus.cc:2393] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 6: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:32.514125 24865 raft_consensus.cc:2804] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:08:32.513929 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 68189.2us
I20260709 13:08:32.558772 22993 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:32.569233 24455 raft_consensus.cc:2749] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:32.572124 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:32.540643 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:32.587285 22874 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 2.
I20260709 13:08:32.560387 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:32.558431 22919 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:32.587554 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:32.572860 24490 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:32.587960 22876 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:32.572958 24587 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:32.588771 24720 raft_consensus.cc:2749] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:08:32.573788 24894 consensus_queue.cc:1048] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.071s
I20260709 13:08:32.561468 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:32.513357 24846 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:32.590716 24846 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:32.574590 24856 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:32.575307 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:32.592893 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:32.593959 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:32.571837 24738 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:32.594844 23623 raft_consensus.cc:2393] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 4: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:32.595603 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:32.596241 23623 raft_consensus.cc:2393] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 4: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:32.576902 23618 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 5.
I20260709 13:08:32.597218 22872 raft_consensus.cc:1240] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:32.599694 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:32.588457 23617 raft_consensus.cc:2393] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 4: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:32.600337 23618 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:08:32.515203 24727 raft_consensus.cc:2804] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:32.587443 22955 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 5.
I20260709 13:08:32.519702 24851 raft_consensus.cc:2749] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:32.601992 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:32.603089 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:32.603619 22955 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:08:32.600886 24727 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:32.605014 24727 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:32.605649 23622 raft_consensus.cc:1240] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:32.577080 24854 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:32.607471 24854 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:32.578336 24910 raft_consensus.cc:740] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:32.608729 24672 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:32.609230 24910 consensus_queue.cc:260] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:32.610512 24910 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:32.610646 23618 raft_consensus.cc:1240] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 13:08:32.524753 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:08:32.528141 24895 log.cc:927] Time spent T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 13:08:32.527454 24769 raft_consensus.cc:2804] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:32.565411 24783 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:32.562889 24763 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:32.586761 24515 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:32.590844 22879 raft_consensus.cc:2393] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 4: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:32.528479 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:32.579430 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:32.611917 22946 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 4
I20260709 13:08:32.604367 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:08:32.567334 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 55159.2us
I20260709 13:08:32.574828 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:32.608174 22954 raft_consensus.cc:1240] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:32.608179 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:32.558986 23590 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:32.613788 22993 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:32.581578 23620 raft_consensus.cc:2393] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 5: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:32.621685 22876 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6.
W20260709 13:08:32.633095 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 74182.1us
I20260709 13:08:32.577462 22952 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
I20260709 13:08:32.622469 22843 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:32.628844 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:32.579505 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:32.641285 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:32.641503 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:32.571977 24848 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:32.579487 24865 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:32.642103 22879 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:32.642449 24865 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:32.625501 22920 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:32.636236 24672 raft_consensus.cc:2804] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:32.580912 24723 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:32.636503 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:32.636895 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:32.637636 24743 raft_consensus.cc:2749] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:32.653146 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:32.653308 22844 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:32.637562 24846 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:32.638360 24854 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:32.638312 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:32.638995 24813 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:32.639665 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:32.635330 22843 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:32.644765 22920 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:32.621167 23619 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:08:32.663399 22920 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:32.646653 24821 raft_consensus.cc:2749] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:32.647480 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:32.665066 22951 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:32.664914 24819 consensus_queue.cc:1059] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:08:32.666224 24821 raft_consensus.cc:3055] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:32.666633 24821 raft_consensus.cc:740] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:32.637580 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:32.650913 23590 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:32.668650 24587 raft_consensus.cc:2804] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:32.670061 24587 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:32.670435 24587 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:32.670948 24890 raft_consensus.cc:3055] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:32.671336 24890 raft_consensus.cc:740] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:32.608350 24817 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:32.671926 22920 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:08:32.651670 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 64567.3us
I20260709 13:08:32.652326 22992 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:32.578681 24646 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:32.654443 22844 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:32.655133 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:32.655616 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:32.655297 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:32.582420 24889 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:32.627439 24769 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:32.657354 24844 raft_consensus.cc:2804] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:32.661005 24854 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:32.661666 24455 raft_consensus.cc:2804] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:32.661803 24863 raft_consensus.cc:2804] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:32.662317 24720 raft_consensus.cc:2804] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:32.664481 24691 raft_consensus.cc:2804] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:32.667456 24821 consensus_queue.cc:260] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:32.720192 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:32.673811 22843 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:32.718395 24720 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:32.676262 22946 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 4.
I20260709 13:08:32.677165 23590 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
W20260709 13:08:32.723506 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 46426.8us
W20260709 13:08:32.721179 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 47424.3us
I20260709 13:08:32.725461 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:32.726224 22946 raft_consensus.cc:2393] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 3: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:32.576017 24869 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:32.649731 24672 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:32.706177 22955 raft_consensus.cc:2393] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 5: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:32.706094 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:32.707021 22879 raft_consensus.cc:2368] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 4. Current term is 5.
I20260709 13:08:32.707418 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:32.731818 22952 raft_consensus.cc:2468] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 3.
I20260709 13:08:32.710613 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:08:32.708264 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 53876.5us
I20260709 13:08:32.733767 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:32.734083 22952 raft_consensus.cc:1240] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:32.734911 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:32.735744 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:32.713518 22875 raft_consensus.cc:2393] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 4: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:32.713368 22993 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:08:32.713902 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 61621.2us
I20260709 13:08:32.738152 23590 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:32.739018 22953 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
I20260709 13:08:32.709474 24769 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:32.713253 24743 raft_consensus.cc:2804] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:32.719276 22951 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 5.
I20260709 13:08:32.719889 24691 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:32.686342 24819 raft_consensus.cc:2804] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:32.656117 24407 consensus_queue.cc:1059] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 13:08:32.728725 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:32.727591 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:32.730614 22843 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:32.746764 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:32.747308 24856 raft_consensus.cc:2804] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:32.753801 24846 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:32.730970 24646 raft_consensus.cc:2804] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:32.709203 24844 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:32.658782 24738 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:32.716933 24455 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:32.677914 24783 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:32.716750 24863 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:32.767865 24691 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:32.765761 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:32.766551 24819 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:32.755182 24515 raft_consensus.cc:2749] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:32.769755 22947 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:32.708434 24867 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:32.803524 23020 raft_consensus.cc:1240] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 13:08:32.798468 24863 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:08:32.804602 24905 log.cc:927] Time spent T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:08:32.805147 24867 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:32.774125 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 60833.9us
I20260709 13:08:32.737946 24889 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:08:32.808873 24889 raft_consensus.cc:697] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:32.788821 24646 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:32.809461 24646 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:08:32.776585 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 46044.8us
I20260709 13:08:32.721686 24720 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:32.780567 24806 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:08:32.776989 24856 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:32.783118 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:32.783388 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:32.783427 24857 raft_consensus.cc:2749] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:08:32.738559 24672 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:32.758280 24821 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:32.764070 24743 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:32.794809 22920 leader_election.cc:400] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 4. Current term is 5.
I20260709 13:08:32.825749 22920 leader_election.cc:403] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
W20260709 13:08:32.826236 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 60205.2us
I20260709 13:08:32.685977 24894 consensus_queue.cc:1048] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.045s
I20260709 13:08:32.728722 24571 raft_consensus.cc:2804] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:08:32.725986 24727 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:32.799450 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:32.834277 23026 raft_consensus.cc:2393] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 4: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:32.835100 23024 raft_consensus.cc:1240] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
W20260709 13:08:32.813057 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 154905us
I20260709 13:08:32.813769 23028 raft_consensus.cc:2468] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 4.
I20260709 13:08:32.793041 24844 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:32.822885 23029 raft_consensus.cc:2393] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 4: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:32.795270 24455 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:32.827337 22843 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:32.838806 22843 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:32.830348 22873 raft_consensus.cc:1275] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:08:32.810675 24819 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:32.820334 24856 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:08:32.844055 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 211266us
I20260709 13:08:32.829176 24743 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:32.830610 24707 raft_consensus.cc:2749] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 4. Current term is 5.
I20260709 13:08:32.864257 22843 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:32.869125 22994 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:32.849110 24846 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:08:32.848483 24817 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:32.850742 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:32.857862 24857 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:32.877211 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:32.868511 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:32.818415 24889 consensus_queue.cc:237] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:32.850481 24766 consensus_queue.cc:1048] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 13:08:32.869493 24709 raft_consensus.cc:2804] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:32.912021 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:32.905421 22918 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:32.923100 22880 raft_consensus.cc:1240] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:08:32.924139 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:32.933642 22880 raft_consensus.cc:2393] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 4: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:32.933321 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:32.934237 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:32.939843 22878 raft_consensus.cc:2393] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 5: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:32.934206 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:32.943838 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:32.951749 22872 raft_consensus.cc:2393] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 4: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:32.919301 24709 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:32.944820 23617 raft_consensus.cc:2393] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 4: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:32.958756 22919 leader_election.cc:400] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 3. Current term is 4.
I20260709 13:08:32.964920 22919 leader_election.cc:403] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:08:32.967821 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:32.951730 24763 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:32.951977 24854 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:32.951839 24869 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:32.964222 24709 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:32.954645 24921 raft_consensus.cc:2804] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:08:32.973819 24919 log.cc:927] Time spent T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:08:32.963579 24846 raft_consensus.cc:697] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 5 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:33.009862 24571 raft_consensus.cc:697] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 5 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:08:33.053755 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 148403us
I20260709 13:08:33.058094 24854 raft_consensus.cc:2749] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 3. Current term is 4.
I20260709 13:08:33.060916 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:33.020275 24783 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:33.027558 24727 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:33.071069 23588 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:33.071141 24865 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:33.073083 24927 consensus_queue.cc:1048] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:08:33.024945 24823 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:33.019001 24926 raft_consensus.cc:2749] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:33.081359 23588 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:08:33.075743 24914 log.cc:927] Time spent T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 13:08:33.052217 24921 raft_consensus.cc:697] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:08:33.086900 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 128235us
I20260709 13:08:33.091204 23588 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:33.122900 22919 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:33.124251 23027 raft_consensus.cc:2393] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 5: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:33.079072 24920 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:33.085279 24913 log.cc:927] Time spent T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.252s	user 0.000s	sys 0.001s
I20260709 13:08:33.081813 24806 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:33.082660 24490 raft_consensus.cc:2749] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:08:33.148468 22880 raft_consensus.cc:1240] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 13:08:33.152098 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:33.151374 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:33.150612 24848 raft_consensus.cc:2749] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:08:33.080811 24766 consensus_queue.cc:1059] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 13:08:33.090267 24928 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:33.093408 24731 raft_consensus.cc:2749] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:08:33.125026 24910 consensus_meta.cc:287] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.514s	user 0.010s	sys 0.002s
I20260709 13:08:33.091382 24823 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:33.175518 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:33.202770 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:33.230966 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:33.231571 22951 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:08:33.218299 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 111507us
W20260709 13:08:33.205078 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 112620us
I20260709 13:08:33.218505 24490 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:33.231948 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:33.234428 23588 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:08:33.234746 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 292380us
I20260709 13:08:33.175118 24857 raft_consensus.cc:2749] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:33.245083 23588 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:33.157683 24920 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:33.156836 24806 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:33.175892 24918 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:33.261055 23588 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:33.175160 24455 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:33.237772 24813 raft_consensus.cc:2804] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:33.176580 24783 raft_consensus.cc:2749] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:08:33.271484 22947 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6.
I20260709 13:08:33.272348 23023 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 3.
W20260709 13:08:33.274894 24925 log.cc:927] Time spent T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:08:33.238015 24939 raft_consensus.cc:3055] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:08:33.276867 24918 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:33.277290 24848 consensus_queue.cc:1059] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:08:33.304091 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:33.287515 23022 raft_consensus.cc:2410] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:08:33.233644 24691 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:33.177619 24571 consensus_queue.cc:237] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:33.321017 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 57551.7us
I20260709 13:08:33.257408 24731 raft_consensus.cc:2804] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:33.305460 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:33.278905 24823 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:33.177543 24928 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:33.159179 24846 consensus_queue.cc:237] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:33.313591 22951 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:08:33.304013 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:33.176005 24863 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:33.315825 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:33.308938 24813 raft_consensus.cc:697] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:33.345927 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:33.281458 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:33.273422 24862 raft_consensus.cc:2749] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:08:33.175156 24844 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:33.319160 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:33.246318 24490 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:33.301692 24769 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:33.280133 24672 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:33.176065 24921 consensus_queue.cc:237] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:33.301185 24939 raft_consensus.cc:740] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
W20260709 13:08:33.344620 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 95626.4us
I20260709 13:08:33.280758 24933 consensus_queue.cc:1059] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.087s
I20260709 13:08:33.318682 24865 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:33.347980 23028 raft_consensus.cc:2393] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 4: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:33.327809 24731 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:33.361816 24691 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:33.327014 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 53517.8us
I20260709 13:08:33.266395 24515 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:33.364975 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:33.365162 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:33.367373 22873 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
I20260709 13:08:33.359149 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:33.381817 23021 raft_consensus.cc:2393] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 4: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:33.374121 24672 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:08:33.374631 24937 log.cc:927] Time spent T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 13:08:33.346731 24889 consensus_queue.cc:1059] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.022s
I20260709 13:08:33.374727 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:33.375897 24455 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:33.376335 24490 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:33.377436 24743 consensus_meta.cc:287] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.506s	user 0.012s	sys 0.000s
I20260709 13:08:33.385522 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:33.378561 24918 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:33.348157 24709 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:33.387276 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:33.390450 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:33.379884 24731 raft_consensus.cc:3060] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:33.391599 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:08:33.396188 23616 consensus_meta.cc:287] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.544s	user 0.007s	sys 0.003s
I20260709 13:08:33.393172 24515 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:33.396780 23616 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
I20260709 13:08:33.398183 24928 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:33.398792 24515 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:08:33.400130 24587 consensus_meta.cc:287] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.729s	user 0.011s	sys 0.000s
I20260709 13:08:33.400632 24587 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:33.402810 24587 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:33.403658 24813 consensus_queue.cc:237] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:33.405786 24863 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:33.406971 23020 raft_consensus.cc:2410] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:08:33.414958 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:33.416934 23619 raft_consensus.cc:2393] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 3: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:33.417181 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:33.417452 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:33.421154 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:33.422049 23022 raft_consensus.cc:2393] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 4: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:33.424710 24769 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:33.387193 24743 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:33.431957 24806 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:33.433030 24743 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:33.395054 24918 consensus_queue.cc:1059] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 13:08:33.442612 24918 raft_consensus.cc:3055] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:33.443049 24918 raft_consensus.cc:740] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:33.444959 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:33.445225 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:33.445770 23028 raft_consensus.cc:2368] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 5. Current term is 6.
W20260709 13:08:33.435775 24821 consensus_meta.cc:287] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.607s	user 0.006s	sys 0.004s
W20260709 13:08:33.448747 24720 consensus_meta.cc:287] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.623s	user 0.009s	sys 0.003s
I20260709 13:08:33.444502 23621 raft_consensus.cc:2393] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 4: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:33.449855 24720 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:33.451582 22955 raft_consensus.cc:1240] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:33.454037 24720 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:33.457229 24727 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:33.458424 24727 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:33.461851 24727 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:33.434795 24731 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:33.457599 24709 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:33.465473 24731 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:33.443996 24918 consensus_queue.cc:260] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:33.469095 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:08:33.471939 24646 consensus_meta.cc:287] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.655s	user 0.006s	sys 0.006s
I20260709 13:08:33.473204 24920 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:33.491113 24844 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:33.496631 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:33.491180 24709 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:33.513142 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:33.506459 24844 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:33.511229 24856 consensus_meta.cc:287] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.649s	user 0.010s	sys 0.000s
I20260709 13:08:33.522328 24856 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:33.503501 24646 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:33.526292 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:33.512130 24918 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:33.552961 22992 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:33.557318 22947 raft_consensus.cc:1240] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:33.529840 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:33.558705 24890 consensus_queue.cc:260] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:33.559188 24932 raft_consensus.cc:2804] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:08:33.559931 24932 raft_consensus.cc:697] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:08:33.560745 24916 log.cc:927] Time spent T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.554s	user 0.008s	sys 0.000s
I20260709 13:08:33.560848 24854 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:33.561614 24854 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:33.565479 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:33.566586 22874 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
I20260709 13:08:33.567777 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:33.568169 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:33.569258 22880 raft_consensus.cc:2393] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 3: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:33.570118 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:08:33.551638 24819 consensus_meta.cc:287] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.689s	user 0.013s	sys 0.003s
W20260709 13:08:33.573652 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 207814us
I20260709 13:08:33.577422 22776 catalog_manager.cc:5697] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:33.575699 22920 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:33.579596 22776 catalog_manager.cc:5697] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: config changed from index 3 to 4, term changed from 1 to 3, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New cstate: current_term: 3 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:33.518074 24709 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:33.584064 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:33.589023 23025 raft_consensus.cc:2393] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 4: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:33.587574 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:33.589084 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:33.590481 22946 raft_consensus.cc:2393] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 4: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:33.559934 24890 raft_consensus.cc:3060] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:33.591374 24926 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:33.595862 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:33.596091 24926 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:33.596129 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:33.596611 24926 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:33.596930 23025 raft_consensus.cc:2368] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 5. Current term is 6.
I20260709 13:08:33.528419 24844 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:33.602195 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:33.602597 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:33.603595 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:33.604233 22947 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:08:33.605139 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:33.590498 22776 catalog_manager.cc:5697] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 3 to 4, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 4 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:33.605863 23028 raft_consensus.cc:2468] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
I20260709 13:08:33.583940 24932 consensus_queue.cc:237] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:33.614265 24709 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:33.607510 22776 catalog_manager.cc:5697] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:33.621565 22776 catalog_manager.cc:5697] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 3 to 4, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 4 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:33.623965 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:33.624750 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:33.623710 22776 catalog_manager.cc:5697] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 2 to 3, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 3 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:08:33.584478 24819 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:33.609663 24854 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:33.629842 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:33.629189 22776 catalog_manager.cc:5697] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 2 to 4, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 4 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:33.631908 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:33.633826 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:33.632903 22776 catalog_manager.cc:5697] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 4 to 5, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 5 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
W20260709 13:08:33.633509 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 57160.6us
I20260709 13:08:33.636248 22950 raft_consensus.cc:2393] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 3: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:33.637480 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:33.638016 22947 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:33.642236 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:33.644287 22992 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:33.646346 22947 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6.
I20260709 13:08:33.645118 22776 catalog_manager.cc:5697] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:33.648412 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:33.650226 22947 raft_consensus.cc:2410] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:08:33.650437 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:33.651441 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:33.651996 22952 raft_consensus.cc:3060] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 5
W20260709 13:08:33.654227 23588 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:33.654807 23588 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:33.655759 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:33.652753 22776 catalog_manager.cc:5697] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 1 to 3, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 3 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:33.656942 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:33.656847 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:33.658111 22776 catalog_manager.cc:5697] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 2 to 3, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 3 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:08:33.658764 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:33.659301 23020 raft_consensus.cc:3060] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:33.659500 22776 catalog_manager.cc:5697] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 2 to 3, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 3 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:08:33.648257 24515 raft_consensus.cc:2804] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:33.642676 24662 consensus_queue.cc:1059] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 13:08:33.668697 24662 raft_consensus.cc:3055] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:33.669061 24662 raft_consensus.cc:740] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:33.669850 24662 consensus_queue.cc:260] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:33.670985 24662 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:33.673727 23029 raft_consensus.cc:1275] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 13:08:33.674098 23029 pending_rounds.cc:85] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75: Aborting all ops after (but not including) 2
I20260709 13:08:33.675673 23020 raft_consensus.cc:2468] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 5.
I20260709 13:08:33.677191 24646 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:33.661531 24856 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:33.673345 24515 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:33.679690 24515 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:33.657212 24587 raft_consensus.cc:2749] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:08:33.674328 23029 pending_rounds.cc:107] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:08:33.696995 23029 raft_consensus.cc:2889] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:33.705369 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:33.706260 22871 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:33.711043 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:33.712168 22952 raft_consensus.cc:2468] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 5.
I20260709 13:08:33.713456 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:33.713717 23588 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:33.716588 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:33.718617 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:33.720009 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:33.720346 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:33.720018 23588 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:33.727830 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:33.727259 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:33.731313 24946 raft_consensus.cc:3055] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:33.731712 24946 raft_consensus.cc:740] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:33.736694 22844 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:33.736785 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:33.738005 22876 raft_consensus.cc:2393] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 3: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:33.739718 22879 raft_consensus.cc:1240] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:33.727648 24904 raft_consensus.cc:2804] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:08:33.726275 24587 raft_consensus.cc:2804] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:08:33.733697 24844 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:33.749343 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 167615us
I20260709 13:08:33.749269 22994 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:33.741150 24646 raft_consensus.cc:2749] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:33.753149 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:33.754570 23028 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
I20260709 13:08:33.742182 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:33.746245 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:33.742009 24920 consensus_queue.cc:1059] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
I20260709 13:08:33.763242 22994 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:33.761088 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:33.760423 22843 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:33.824972 22946 raft_consensus.cc:2393] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 3: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:33.770573 23620 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:33.855013 24929 raft_consensus.cc:3055] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:33.826603 24790 raft_consensus.cc:2749] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:08:33.837440 23026 raft_consensus.cc:1240] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 13:08:33.750113 24904 raft_consensus.cc:697] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 6 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:33.762248 24819 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:33.770351 24806 raft_consensus.cc:2749] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:33.757390 24587 raft_consensus.cc:697] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
W20260709 13:08:33.837011 24917 log.cc:927] Time spent T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.825s	user 0.004s	sys 0.000s
I20260709 13:08:33.733948 24946 consensus_queue.cc:260] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:33.816370 23616 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 418) took 1482 ms (client timeout 1756 ms). Trace:
W20260709 13:08:33.822567 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 59392.5us
W20260709 13:08:33.879657 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 119368us
I20260709 13:08:33.880960 24844 raft_consensus.cc:2749] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:33.879164 24929 raft_consensus.cc:740] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:33.899169 22843 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:33.906908 22843 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:33.896651 23616 rpcz_store.cc:269] 0709 13:08:32.334252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:32.348818 (+ 14566us) service_pool.cc:224] Handling call
0709 13:08:33.816191 (+1467373us) spinlock_profiling.cc:243] Waited 487 ms on lock 0x7b540046b254. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:33.816300 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1120506624}
I20260709 13:08:33.905179 24927 raft_consensus.cc:2749] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:33.911530 22871 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
I20260709 13:08:33.917934 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:33.918187 23590 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:33.919061 22876 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:33.917393 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:33.919883 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:33.920215 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:33.920846 23625 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:33.921126 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:33.921411 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:33.921839 23623 raft_consensus.cc:2393] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 4: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:33.922056 22872 raft_consensus.cc:2393] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 5: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:33.923087 22872 raft_consensus.cc:1275] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:08:33.920260 22875 raft_consensus.cc:2393] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 3: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:33.926203 24407 raft_consensus.cc:2749] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:08:33.927831 22843 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:33.925670 22994 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:33.930107 24662 raft_consensus.cc:2749] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:33.931073 23590 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:33.930989 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:33.933547 22843 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:33.936367 22843 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:33.937556 24938 raft_consensus.cc:2749] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:33.936717 24407 raft_consensus.cc:2749] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:33.935300 24927 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:33.939713 24939 consensus_queue.cc:260] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:33.941329 23590 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:33.943061 22843 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:33.909830 24904 consensus_queue.cc:237] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:33.933661 22994 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:33.942006 22876 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6.
I20260709 13:08:33.929014 24844 raft_consensus.cc:2749] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:33.943990 22843 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:33.943917 23590 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:33.960973 22994 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:33.962453 22994 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:33.964721 22843 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:33.968853 22871 raft_consensus.cc:1275] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:08:33.974546 23625 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:08:33.975828 24941 consensus_queue.cc:1048] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:08:33.942234 24927 raft_consensus.cc:2749] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:33.945598 24938 raft_consensus.cc:2749] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:08:33.945521 24889 raft_consensus.cc:2749] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:08:33.979425 24846 raft_consensus.cc:2749] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:33.977170 24662 raft_consensus.cc:2749] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:33.992997 24938 raft_consensus.cc:2749] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:33.994626 22843 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:33.995808 22843 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:33.996019 24646 raft_consensus.cc:2749] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:08:33.978304 24867 raft_consensus.cc:2749] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:33.977687 24763 raft_consensus.cc:2749] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:33.980716 24920 raft_consensus.cc:2749] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:33.999310 24938 raft_consensus.cc:2749] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:33.999997 22843 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:34.002645 24938 raft_consensus.cc:2749] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:08:34.003742 22843 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:34.005112 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:34.006500 22843 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:34.011020 22843 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:34.012127 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:33.948392 24929 consensus_queue.cc:260] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:34.017813 24743 raft_consensus.cc:2749] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:34.017980 23022 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "5b346767a89745fa82928a5dac9cc0b2"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:08:34.011615 24889 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:34.016218 24938 raft_consensus.cc:2749] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:08:34.006695 24927 raft_consensus.cc:2749] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:33.986850 24806 consensus_queue.cc:1059] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
I20260709 13:08:34.029418 24889 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:34.031301 24929 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:34.032455 24889 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:34.029902 23022 raft_consensus.cc:606] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Received request to transfer leadership to TS 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:34.033120 23022 raft_consensus.cc:614] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:34.033874 23617 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 5.
I20260709 13:08:34.035566 24817 raft_consensus.cc:3055] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:34.035687 23620 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 3.
I20260709 13:08:34.036010 24817 raft_consensus.cc:740] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:34.039108 24817 consensus_queue.cc:260] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:34.040699 24407 consensus_queue.cc:1059] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:08:34.041529 24926 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:34.043864 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:34.046113 24926 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:33.997808 24587 consensus_queue.cc:237] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:34.061900 22843 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:34.063251 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:34.063891 22993 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:34.063990 23624 raft_consensus.cc:2393] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 4: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:34.061167 22844 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:34.068053 22843 leader_election.cc:400] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 5. Current term is 6.
I20260709 13:08:34.068243 22993 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:34.062808 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:34.057312 24817 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:34.072893 22993 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:34.066032 22844 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:34.073257 23617 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:34.077135 23618 raft_consensus.cc:1275] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:08:34.077351 24743 raft_consensus.cc:2804] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:34.078020 24743 raft_consensus.cc:697] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 3 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:34.078857 24743 consensus_queue.cc:237] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:34.070600 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:34.080453 23620 raft_consensus.cc:2393] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 5: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:34.084825 24806 raft_consensus.cc:2749] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:08:34.070789 24938 raft_consensus.cc:2749] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:08:34.072573 24907 raft_consensus.cc:2749] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:08:34.069002 24927 raft_consensus.cc:2804] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:08:34.074548 24455 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:08:34.103309 24844 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:34.072436 23621 raft_consensus.cc:2410] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:08:34.108728 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:34.069582 22843 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:34.112488 23027 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32794 (request call id 310) took 1488 ms. Trace:
I20260709 13:08:34.114081 23027 rpcz_store.cc:276] 0709 13:08:32.618565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:32.628084 (+  9519us) service_pool.cc:224] Handling call
0709 13:08:34.106687 (+1478603us) spinlock_profiling.cc:243] Waited 463 ms on lock 0x7b540010c054. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:34.106806 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1065658112}
I20260709 13:08:34.116796 24844 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:34.118407 24889 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:34.118911 24844 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:34.119038 24889 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:34.112479 24927 raft_consensus.cc:697] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 5 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:34.121572 24927 consensus_queue.cc:237] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:34.122754 24889 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:34.125494 22953 raft_consensus.cc:1275] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 13:08:34.125541 23619 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60044 (request call id 463) took 1453 ms (client timeout 1806 ms). Trace:
W20260709 13:08:34.126148 23619 rpcz_store.cc:269] 0709 13:08:32.671978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:32.676880 (+  4902us) service_pool.cc:224] Handling call
0709 13:08:34.125314 (+1448434us) spinlock_profiling.cc:243] Waited 674 ms on lock 0x7b54002bc854. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:34.125453 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1549685120}
I20260709 13:08:34.128358 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:34.130584 24455 raft_consensus.cc:697] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 5 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:08:34.131224 24924 log.cc:927] Time spent T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 1.104s	user 0.005s	sys 0.000s
I20260709 13:08:34.135025 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:34.135753 24490 raft_consensus.cc:3055] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:08:34.136184 24490 raft_consensus.cc:740] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:34.137176 24490 consensus_queue.cc:260] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:34.138381 24490 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:34.141575 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:34.133885 22918 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:08:34.133736 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 64119.3us
I20260709 13:08:34.147918 22918 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:34.150023 22918 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:34.151440 22918 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:34.153414 22918 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:34.157980 22918 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:34.155786 24821 raft_consensus.cc:2749] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:08:34.132090 24455 consensus_queue.cc:237] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:34.154503 24972 raft_consensus.cc:2749] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:08:34.168344 22950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58310 (request call id 1445) took 1586 ms. Trace:
I20260709 13:08:34.168819 22950 rpcz_store.cc:276] 0709 13:08:32.581312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:32.715146 (+133834us) service_pool.cc:224] Handling call
0709 13:08:34.167144 (+1451998us) spinlock_profiling.cc:243] Waited 866 ms on lock 0x7b5400148554. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:34.167323 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1990807296}
I20260709 13:08:34.170960 24954 raft_consensus.cc:2749] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:34.170626 24938 consensus_queue.cc:1048] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:08:34.172873 24972 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:34.173444 24972 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:34.166561 24819 raft_consensus.cc:2749] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:34.154875 24926 raft_consensus.cc:2749] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:34.157982 24930 raft_consensus.cc:2749] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:08:34.177788 24972 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:34.132745 23617 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
I20260709 13:08:34.187347 22873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35370 (request call id 1828) took 1280 ms. Trace:
I20260709 13:08:34.138396 24515 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:34.156217 24783 raft_consensus.cc:2764] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:08:34.189132 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:34.196115 22873 rpcz_store.cc:276] 0709 13:08:32.906726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:32.911731 (+  5005us) service_pool.cc:224] Handling call
0709 13:08:34.187132 (+1275401us) spinlock_profiling.cc:243] Waited 452 ms on lock 0x7b54003c69d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:34.187251 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1038735232}
I20260709 13:08:34.191649 23022 raft_consensus.cc:2393] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 5: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:34.199194 23025 raft_consensus.cc:2410] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:08:34.201503 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:34.202078 24515 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:34.170137 22918 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:34.236172 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 66122.3us
I20260709 13:08:34.207427 23023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32794 (request call id 311) took 1588 ms. Trace:
I20260709 13:08:34.230888 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:34.233356 22920 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:34.241516 22993 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:08:34.242733 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:34.236900 23023 rpcz_store.cc:276] 0709 13:08:32.618887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:32.628378 (+  9491us) service_pool.cc:224] Handling call
0709 13:08:34.207287 (+1578909us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b54001c7ad4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:34.207376 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":358635904}
I20260709 13:08:34.245525 23588 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:34.247961 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:34.249343 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:34.249428 22993 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:34.253041 22950 raft_consensus.cc:1275] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:08:34.255364 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:34.228525 24945 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:34.254652 24942 raft_consensus.cc:2749] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:08:34.267590 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:08:34.268478 24923 log.cc:927] Time spent T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 1.205s	user 0.003s	sys 0.000s
I20260709 13:08:34.260932 24738 raft_consensus.cc:2749] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:34.259442 24819 consensus_queue.cc:1048] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 13:08:34.269960 23620 raft_consensus.cc:2393] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 4: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:34.254716 24926 raft_consensus.cc:2749] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:34.253846 24790 raft_consensus.cc:2804] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:08:34.253996 24889 raft_consensus.cc:2804] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:34.262228 24972 consensus_queue.cc:1048] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 13:08:34.281278 24783 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:34.285132 24783 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:34.287365 24783 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:34.288089 23021 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32794 (request call id 309) took 1669 ms (client timeout 1594 ms). Trace:
W20260709 13:08:34.288508 23021 rpcz_store.cc:269] 0709 13:08:32.618244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:32.628565 (+ 10321us) service_pool.cc:224] Handling call
0709 13:08:34.287882 (+1659317us) spinlock_profiling.cc:243] Waited 746 ms on lock 0x7b540024a254. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:34.288015 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1716627712}
I20260709 13:08:34.289783 23024 raft_consensus.cc:1275] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:08:34.290970 23623 raft_consensus.cc:1275] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 13:08:34.291343 23623 pending_rounds.cc:85] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2: Aborting all ops after (but not including) 5
I20260709 13:08:34.291604 23623 pending_rounds.cc:107] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 13:08:34.291846 23623 raft_consensus.cc:2889] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:34.271761 23619 raft_consensus.cc:2393] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 4: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:34.301388 22918 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:08:34.302412 24819 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:34.302932 24819 raft_consensus.cc:697] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:34.303791 24819 consensus_queue.cc:237] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:34.305645 22843 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:34.307646 22878 raft_consensus.cc:1240] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:34.314498 23620 raft_consensus.cc:1240] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:34.315526 22844 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:34.318221 23588 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:34.329254 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:34.331884 22994 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:34.282459 24939 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:34.325161 24587 consensus_queue.cc:1048] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:08:34.334221 24915 consensus_queue.cc:1059] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:34.332823 24972 raft_consensus.cc:2749] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:34.312650 24790 raft_consensus.cc:697] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 6 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:34.336148 24790 consensus_queue.cc:237] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:34.338636 22948 raft_consensus.cc:1275] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:08:34.333425 24743 raft_consensus.cc:2749] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:34.341270 23588 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:34.342478 23588 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:34.343741 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:34.344087 24848 raft_consensus.cc:2749] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:34.344570 23621 raft_consensus.cc:2393] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 3: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:34.273172 24945 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:34.345726 24941 raft_consensus.cc:2749] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:34.341404 24946 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:34.344053 24862 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:34.346385 24941 consensus_queue.cc:1048] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:08:34.309309 24455 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:34.358508 22919 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:34.366500 24921 consensus_queue.cc:1048] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:08:34.368007 24731 consensus_queue.cc:1048] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:08:34.350176 24945 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:34.372028 24738 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:34.372553 24738 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:34.373572 24854 raft_consensus.cc:2749] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:34.373623 24727 consensus_queue.cc:1048] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:08:34.374763 24738 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:34.375218 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:34.376276 22951 raft_consensus.cc:1240] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:34.377779 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:34.356338 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:34.359840 24515 raft_consensus.cc:2804] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:08:34.359376 24851 raft_consensus.cc:3055] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:34.393450 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:34.359597 24889 raft_consensus.cc:697] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:34.395634 24720 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:34.409159 24720 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:34.411564 24720 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:34.394963 24851 raft_consensus.cc:740] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:34.382709 24455 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:34.398864 24951 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:34.401837 24515 raft_consensus.cc:697] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 5 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:34.445786 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:34.439896 24889 consensus_queue.cc:237] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:34.450924 22874 raft_consensus.cc:2393] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 4: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:34.443462 24455 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:34.443872 22949 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
I20260709 13:08:34.447158 22954 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:08:34.415668 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:34.415341 24750 consensus_queue.cc:1059] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 13:08:34.461678 24750 raft_consensus.cc:3055] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:08:34.462037 24750 raft_consensus.cc:740] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:34.462818 24750 consensus_queue.cc:260] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:34.457286 24978 log.cc:927] Time spent T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:08:34.468842 22919 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:34.458065 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:34.444885 24851 consensus_queue.cc:260] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:34.483916 24515 consensus_queue.cc:237] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:34.489315 22946 raft_consensus.cc:1275] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:08:34.490958 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:34.450016 24946 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:34.492803 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:34.493194 24946 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:34.495136 22877 raft_consensus.cc:1275] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:08:34.495672 23024 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:08:34.496275 23024 raft_consensus.cc:606] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Received request to transfer leadership to TS aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:34.497761 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:34.501718 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:34.505354 22992 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:34.505530 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:34.505770 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:34.506207 23026 raft_consensus.cc:2393] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 4: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:34.464591 24750 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:34.515779 22954 raft_consensus.cc:1275] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:08:34.518117 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:34.479761 24821 raft_consensus.cc:2749] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:34.519495 22950 raft_consensus.cc:1275] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:08:34.520273 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:34.505060 23621 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
W20260709 13:08:34.530889 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:34.531738 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:34.532231 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:34.532400 23021 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
I20260709 13:08:34.532828 22951 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:34.535794 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:34.536471 23021 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
I20260709 13:08:34.537068 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:34.537681 22879 raft_consensus.cc:1275] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:08:34.421847 24951 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:34.570700 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:34.526630 24922 raft_consensus.cc:2804] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:34.533766 24869 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:34.521025 24587 consensus_queue.cc:1048] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.081s
I20260709 13:08:34.527230 24817 consensus_queue.cc:1059] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.083s
I20260709 13:08:34.612564 22918 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:34.624373 22918 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:08:34.610248 22843 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:34.611454 23620 raft_consensus.cc:1275] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 13:08:34.633476 24991 log.cc:927] Time spent T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
W20260709 13:08:34.646483 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 47216.1us
I20260709 13:08:34.630018 24862 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:08:34.675508 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 60164.4us
I20260709 13:08:34.678117 24817 raft_consensus.cc:3055] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:08:34.678570 24817 raft_consensus.cc:740] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:08:34.678706 24982 log.cc:927] Time spent T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.315s	user 0.003s	sys 0.001s
I20260709 13:08:34.679569 24817 consensus_queue.cc:260] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:34.681090 24983 log.cc:927] Time spent T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.225s	user 0.000s	sys 0.001s
I20260709 13:08:34.672353 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:34.682482 23621 raft_consensus.cc:2393] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 4: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:34.682777 24817 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:34.646137 24922 raft_consensus.cc:697] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:34.686524 23623 raft_consensus.cc:1240] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 5. Current term is 6. Ops: []
W20260709 13:08:34.691030 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 80858.8us
I20260709 13:08:34.692167 22843 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:34.629631 24945 raft_consensus.cc:2749] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:08:34.690069 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:34.619510 24869 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:34.684865 24951 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:34.638849 24738 raft_consensus.cc:2804] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:34.646358 24813 consensus_queue.cc:1048] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 13:08:34.714792 22992 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:34.713436 22874 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
I20260709 13:08:34.693882 24927 raft_consensus.cc:2749] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:34.724012 22920 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:34.716244 24821 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:34.668540 24515 consensus_queue.cc:1048] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:08:34.715137 24738 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:34.743817 23022 raft_consensus.cc:1275] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:08:34.746717 24922 consensus_queue.cc:237] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:34.739365 24951 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:08:34.756719 24984 log.cc:927] Time spent T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.228s	user 0.000s	sys 0.001s
I20260709 13:08:34.756923 24951 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:34.757450 24951 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:34.758253 24869 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:34.759898 22953 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 5.
I20260709 13:08:34.760740 24935 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:34.761318 24935 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:34.763510 24935 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:34.734988 24907 raft_consensus.cc:2804] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:34.768980 24907 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:34.769719 24907 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:34.770855 22872 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:34.749408 24927 raft_consensus.cc:2749] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:34.773094 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:34.773927 22879 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 5.
I20260709 13:08:34.721773 24455 consensus_queue.cc:1048] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:08:34.732081 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:34.777647 22920 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:34.749532 24738 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:08:34.775283 24986 log.cc:927] Time spent T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
I20260709 13:08:34.791026 24819 raft_consensus.cc:2804] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:34.794034 24819 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:34.794468 24819 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:34.790405 23621 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
W20260709 13:08:34.800802 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:34.802657 24946 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:34.805945 22872 raft_consensus.cc:1275] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:08:34.808184 24946 consensus_queue.cc:1048] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260709 13:08:34.817971 24990 log.cc:927] Time spent T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 13:08:34.819408 23619 raft_consensus.cc:1240] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 4. Current term is 5. Ops: []
W20260709 13:08:34.824828 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 147221us
I20260709 13:08:34.826471 24455 consensus_queue.cc:1048] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:34.829022 22918 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:34.840335 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:34.846221 22918 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:34.749831 24821 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:34.848595 22918 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:34.852592 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 160036us
I20260709 13:08:34.854610 22955 raft_consensus.cc:1275] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:08:34.858278 23026 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:08:34.859012 23026 raft_consensus.cc:606] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Received request to transfer leadership to TS aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:08:34.904526 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 43998.3us
I20260709 13:08:34.927217 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:08:34.930982 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 151091us
I20260709 13:08:34.851588 24946 raft_consensus.cc:2749] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:34.851965 24939 raft_consensus.cc:2749] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:34.851193 24854 raft_consensus.cc:2804] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:34.932286 24848 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:34.950810 24918 raft_consensus.cc:993] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75: : Instructing follower aed3f089ec8b49af9c8cf5cdee76e8e7 to start an election
I20260709 13:08:34.951310 24918 raft_consensus.cc:1081] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Signalling peer aed3f089ec8b49af9c8cf5cdee76e8e7 to start an election
I20260709 13:08:34.950656 24848 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:34.952636 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:34.953383 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 47830.3us
W20260709 13:08:34.953691 24980 log.cc:927] Time spent T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.580s	user 0.000s	sys 0.004s
I20260709 13:08:34.943876 24854 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:08:34.957340 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:34.950592 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:35.009167 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:08:34.960629 24969 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.044s	sys 0.073s
I20260709 13:08:35.013427 22873 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 4.
I20260709 13:08:34.960359 24854 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:08:35.017576 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 166816us
W20260709 13:08:35.016150 24969 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.044s	sys 0.073s
I20260709 13:08:34.936849 24571 consensus_queue.cc:1048] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.075s
I20260709 13:08:35.026845 23619 raft_consensus.cc:1275] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:08:35.027112 24571 consensus_queue.cc:1243] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer 5b346767a89745fa82928a5dac9cc0b2 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:08:35.025730 23590 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
W20260709 13:08:35.028970 24988 log.cc:927] Time spent T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.345s	user 0.004s	sys 0.003s
I20260709 13:08:35.019732 23621 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
I20260709 13:08:35.043947 24738 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:35.045469 24819 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:35.049031 24989 log.cc:927] Time spent T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.355s	user 0.004s	sys 0.006s
W20260709 13:08:35.054791 24750 consensus_meta.cc:287] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.530s	user 0.006s	sys 0.001s
I20260709 13:08:35.057278 24904 raft_consensus.cc:2804] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:08:35.075830 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 222286us
I20260709 13:08:35.077566 22843 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:35.078289 24998 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:35.078773 24941 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:35.079478 24941 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:35.082016 24941 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:35.083201 22953 raft_consensus.cc:1275] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:08:35.086367 22843 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:35.093812 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:35.076347 24904 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:35.087935 24998 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:35.067112 24915 consensus_queue.cc:1048] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 13:08:35.104193 23619 pending_rounds.cc:85] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2: Aborting all ops after (but not including) 5
I20260709 13:08:35.108477 24854 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:35.109896 24738 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:35.112145 24819 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:35.120231 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:35.122982 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:35.125620 22952 raft_consensus.cc:1240] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:08:35.132025 23619 pending_rounds.cc:107] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:08:35.130687 24998 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:35.074689 24951 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:35.147614 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:35.119534 24904 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:08:35.149283 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 205944us
I20260709 13:08:35.150580 22955 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
I20260709 13:08:35.164166 22877 raft_consensus.cc:3055] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:08:35.170840 23022 raft_consensus.cc:1275] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:08:35.173851 22877 raft_consensus.cc:740] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:35.175422 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:35.175858 22880 raft_consensus.cc:2393] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 4: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:35.175797 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:35.170920 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:35.142668 22953 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:58320
I20260709 13:08:35.178773 23022 raft_consensus.cc:1240] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 6 LEADER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:08:35.179180 22953 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:35.179699 22953 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:35.181900 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:35.155121 23619 raft_consensus.cc:2889] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:35.198763 23025 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
I20260709 13:08:35.192988 22953 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:35.174736 22877 consensus_queue.cc:260] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:35.201301 22877 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:35.171060 24998 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:35.169093 24854 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:35.227370 22953 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:35.184962 24943 consensus_queue.cc:1048] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:08:35.208817 25008 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:35.202162 24571 consensus_queue.cc:1059] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.026s
I20260709 13:08:35.225268 24723 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:35.202337 25006 consensus_queue.cc:1048] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
W20260709 13:08:35.244621 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 54200.2us
I20260709 13:08:35.246245 22950 raft_consensus.cc:1240] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:35.250653 24723 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:35.256728 25002 log.cc:927] Time spent T 00000000000000000000000000000000 P 8d175b4ae72f4fc48971c2f31668971c: Append to log took a long time: real 0.079s	user 0.000s	sys 0.002s
I20260709 13:08:35.258116 24723 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:35.283066 23025 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:08:35.283692 23025 raft_consensus.cc:606] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Received request to transfer leadership to TS aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:35.283987 23025 raft_consensus.cc:614] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:35.287886 22872 raft_consensus.cc:1275] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 13:08:35.293354 24987 log.cc:927] Time spent T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.575s	user 0.001s	sys 0.004s
I20260709 13:08:35.290776 22918 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:35.299707 22918 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:35.238039 25008 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:35.301697 23022 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:08:35.311782 23022 raft_consensus.cc:606] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 2 LEADER]: Received request to transfer leadership to TS aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:35.303478 24821 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:35.305344 24848 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:35.306386 24946 raft_consensus.cc:2749] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:08:35.366242 24720 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:35.367028 24720 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:35.374068 24939 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:35.374864 24939 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:35.377126 24951 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:35.379891 24907 consensus_meta.cc:287] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.610s	user 0.012s	sys 0.000s
I20260709 13:08:35.382534 24951 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:35.310971 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:35.361557 24738 raft_consensus.cc:2804] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:35.384229 23025 raft_consensus.cc:3055] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:35.384584 24738 raft_consensus.cc:697] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
W20260709 13:08:35.385041 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:35.385824 24738 consensus_queue.cc:237] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:35.310268 24935 consensus_queue.cc:1048] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.087s
I20260709 13:08:35.399488 21756 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:08:35.399902 25008 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:35.400450 24727 consensus_queue.cc:1048] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:08:35.402282 24720 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:35.383270 24951 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:35.406946 24951 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:35.384580 23025 raft_consensus.cc:740] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:35.431202 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:35.432736 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:35.412424 23025 consensus_queue.cc:260] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:35.455336 22877 raft_consensus.cc:1275] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:08:35.415262 21756 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:35.455729 22877 pending_rounds.cc:85] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 6
I20260709 13:08:35.456013 22877 pending_rounds.cc:107] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:08:35.456315 22877 raft_consensus.cc:2889] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:35.430312 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:35.461454 24867 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:35.461951 24867 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:35.463917 24867 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:35.465562 23026 raft_consensus.cc:2393] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 5: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:35.466391 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:35.466992 23023 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:35.468917 22918 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:35.469156 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:35.469697 24821 raft_consensus.cc:2804] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:35.470084 22877 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:35.470628 24727 consensus_queue.cc:1048] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:08:35.472777 22994 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:35.473847 24727 raft_consensus.cc:2804] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:35.474206 24727 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:35.474555 24727 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:35.483248 24727 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:35.486923 23025 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:35.380328 24907 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:35.495203 24851 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:35.496095 24907 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:35.490612 25010 log.cc:927] Time spent T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.395s	user 0.000s	sys 0.006s
I20260709 13:08:35.470111 24821 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:35.501643 23028 raft_consensus.cc:2368] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 4. Current term is 5.
I20260709 13:08:35.424932 24750 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:35.454092 24939 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:35.519452 23025 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:35.504947 24821 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:35.521180 22918 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:08:35.529358 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 240096us
I20260709 13:08:35.536442 22947 raft_consensus.cc:1275] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:08:35.539106 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:35.539700 22953 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 4.
I20260709 13:08:35.540329 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:35.541049 22871 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:08:35.545243 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:35.546038 22947 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 5.
I20260709 13:08:35.517020 24750 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:35.565261 22949 raft_consensus.cc:1275] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:08:35.565307 23588 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:35.569217 23588 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:35.573895 22871 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
I20260709 13:08:35.548902 24848 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:35.574910 22994 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:35.607648 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:35.610273 23621 raft_consensus.cc:2393] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 4: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:35.573053 24490 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:35.556732 24738 raft_consensus.cc:2804] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:35.623142 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:35.606155 24571 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:35.624920 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:35.615065 24727 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:35.624936 24490 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:35.633060 24490 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:35.615994 24790 raft_consensus.cc:2804] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:08:35.634891 24998 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:35.635375 24790 raft_consensus.cc:697] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 5 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:35.635782 24821 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:35.636318 24790 consensus_queue.cc:237] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:35.638804 24821 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:35.576841 24848 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:35.569860 24723 consensus_queue.cc:1048] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:08:35.642809 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:35.619451 24941 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:35.612319 24904 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:35.648941 24571 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:35.654564 24848 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:35.636332 24817 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
W20260709 13:08:35.655771 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:35.655557 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:35.640786 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:35.662972 23620 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 5.
I20260709 13:08:35.664954 24998 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:35.651877 24941 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:35.672911 24941 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:35.673625 24817 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:35.646914 24738 raft_consensus.cc:697] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:35.671640 24571 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:35.677662 22950 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
I20260709 13:08:35.679353 22843 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:35.680850 24998 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:35.681380 24998 raft_consensus.cc:697] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 4 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:35.681694 24817 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:35.680464 24738 consensus_queue.cc:237] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:35.688828 23025 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:35.690022 24417 raft_consensus.cc:3055] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:08:35.690469 24417 raft_consensus.cc:740] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:35.692231 22918 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:35.695747 23024 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
W20260709 13:08:35.704819 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:35.729226 23624 raft_consensus.cc:1275] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:08:35.682211 24998 consensus_queue.cc:237] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:35.733827 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:35.734751 22874 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:35.737493 25025 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:35.738729 25025 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:35.730557 24854 raft_consensus.cc:2804] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:35.751570 24854 raft_consensus.cc:697] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:35.752709 24854 consensus_queue.cc:237] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:35.709482 25023 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:35.755450 22874 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:35.755290 25023 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:35.757875 25023 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:35.758914 24490 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:35.761047 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:35.762456 23624 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:35.762673 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:35.764600 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:35.765959 22950 raft_consensus.cc:2393] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 4: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:35.768059 22843 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:35.781656 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:35.782424 22873 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:35.774629 25023 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:35.789258 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:35.708825 24417 consensus_queue.cc:260] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:35.790298 22878 raft_consensus.cc:2393] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 3: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:35.790517 24417 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:35.789220 24821 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:35.787751 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:35.792802 22872 raft_consensus.cc:2393] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 3: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:35.781211 25004 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:35.805665 23624 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
W20260709 13:08:35.807144 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 212592us
I20260709 13:08:35.791956 24821 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:35.795261 24490 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:08:35.827172 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:35.827606 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:35.828708 22947 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 4.
I20260709 13:08:35.832212 24890 consensus_queue.cc:1048] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:08:35.832887 24894 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:35.833441 24894 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:35.833698 25033 consensus_queue.cc:1048] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 13:08:35.835493 24894 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:35.838446 24821 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:35.842401 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:35.843394 22874 raft_consensus.cc:2393] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 5: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:35.852479 23624 raft_consensus.cc:1275] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:08:35.852761 25023 raft_consensus.cc:2749] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:35.853808 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:35.854723 23624 raft_consensus.cc:2368] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 4. Current term is 5.
I20260709 13:08:35.808439 25004 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:35.861899 24904 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:35.862931 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:35.863528 22949 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:35.868023 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:35.868992 24790 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:35.869637 24821 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:35.870152 24821 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:35.869463 24790 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:35.874226 24790 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:35.875552 24821 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:35.876874 25025 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:35.879123 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:35.881526 25004 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:35.882757 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:08:35.883064 23024 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33420 (request call id 1410) took 1384 ms (client timeout 1643 ms). Trace:
I20260709 13:08:35.883646 22873 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 5.
W20260709 13:08:35.883502 23024 rpcz_store.cc:269] 0709 13:08:34.498230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:34.501451 (+  3221us) service_pool.cc:224] Handling call
0709 13:08:35.873500 (+1372049us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b540010ca54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:35.883004 (+  9504us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2673081728}
I20260709 13:08:35.864581 24941 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:35.887028 24941 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:35.889839 23621 raft_consensus.cc:1240] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 5. Current term is 6. Ops: []
I20260709 13:08:35.890393 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:35.891005 22954 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:08:35.892066 22948 raft_consensus.cc:1240] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:08:35.894840 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:35.895489 22948 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:35.897557 22953 raft_consensus.cc:1275] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:08:35.884003 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:35.899309 23028 raft_consensus.cc:2393] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 5: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:35.883363 22947 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 5.
I20260709 13:08:35.904491 22920 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:35.899638 23588 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:35.908090 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:35.909086 22844 leader_election.cc:400] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 election: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 4. Current term is 5.
I20260709 13:08:35.909317 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:35.909874 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:35.918457 22949 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 5.
W20260709 13:08:35.930130 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 146816us
I20260709 13:08:35.943161 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:35.943861 23024 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 4.
I20260709 13:08:35.949355 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:35.912357 24821 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:08:35.959290 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:35.960156 22951 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:35.989678 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:35.946596 24941 raft_consensus.cc:2804] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:35.992331 22951 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
I20260709 13:08:35.961185 24918 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:36.013453 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 104542us
I20260709 13:08:36.020951 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:36.020025 23025 raft_consensus.cc:1275] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:08:35.995903 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:36.026870 23028 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 5.
I20260709 13:08:36.023409 23588 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:35.997157 25039 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:35.999212 24941 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:36.044554 22919 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:36.011361 25006 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:36.047868 22993 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:36.036377 24856 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:35.995520 25023 consensus_queue.cc:1048] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 13:08:36.057667 22918 leader_election.cc:400] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 4. Current term is 5.
I20260709 13:08:36.058005 22918 leader_election.cc:403] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:08:36.067441 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:36.068054 23028 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:36.069962 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:36.073688 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:36.084700 23028 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 5.
I20260709 13:08:36.039667 23621 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 5.
I20260709 13:08:36.062566 24571 consensus_queue.cc:1048] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:08:36.090598 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:36.091158 23622 raft_consensus.cc:1275] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:08:36.093235 22992 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:08:36.094391 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:36.098742 24943 raft_consensus.cc:2804] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:08:36.041040 25039 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:36.101560 24823 raft_consensus.cc:2749] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:36.102360 24821 raft_consensus.cc:697] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:36.102210 24894 consensus_queue.cc:1059] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:08:36.103093 24894 raft_consensus.cc:3055] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:08:36.103590 24821 consensus_queue.cc:237] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:36.045137 24941 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:08:36.106544 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 48456.8us
W20260709 13:08:36.108629 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 51942.3us
I20260709 13:08:36.046943 24813 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:08:36.103430 24894 raft_consensus.cc:740] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:36.116223 22919 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:36.103680 25039 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:36.127810 25018 raft_consensus.cc:2804] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:36.128336 25018 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:36.128712 25018 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:36.129298 24738 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:36.129885 24738 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:36.133078 24943 raft_consensus.cc:697] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 5 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:36.133719 22919 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:36.134095 24943 consensus_queue.cc:237] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:36.141566 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 48422.4us
W20260709 13:08:36.168394 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 145931us
I20260709 13:08:36.032814 24918 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:36.140947 24854 raft_consensus.cc:2749] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:36.106226 24856 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:36.228395 22873 raft_consensus.cc:1275] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:08:36.226329 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:36.232885 24417 consensus_queue.cc:1048] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:08:36.235205 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:36.234405 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:36.236804 22878 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:36.239531 22844 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:36.241736 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 98490.4us
I20260709 13:08:36.235459 25039 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:36.245862 24790 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:36.246443 25039 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:36.246642 24790 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:36.248960 22878 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 5.
I20260709 13:08:36.250962 25008 consensus_queue.cc:1048] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:36.219328 24813 raft_consensus.cc:697] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:36.258253 24856 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:36.259436 23590 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:36.260937 25036 raft_consensus.cc:2804] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:08:36.261487 25036 raft_consensus.cc:697] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 5 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:36.262495 25036 consensus_queue.cc:237] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:36.264645 24894 consensus_queue.cc:260] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:36.266163 24894 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:36.257649 24813 consensus_queue.cc:237] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:36.267526 24738 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:36.248548 25039 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:36.273918 25039 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:36.058050 25006 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:36.278239 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:36.286013 22952 raft_consensus.cc:2393] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 5: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:36.264394 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:36.287062 22879 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 4.
I20260709 13:08:36.280551 22949 raft_consensus.cc:1240] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:36.217942 25004 raft_consensus.cc:2804] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:36.289417 23590 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:36.291286 22948 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
I20260709 13:08:36.245934 25025 raft_consensus.cc:2804] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:36.310379 25006 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:36.292657 25004 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:36.313283 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:36.315212 25050 consensus_queue.cc:1059] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:08:36.317831 24672 raft_consensus.cc:2804] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:36.302130 24935 raft_consensus.cc:2749] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 4. Current term is 5.
I20260709 13:08:36.308406 24918 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:36.276139 25039 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:36.321063 25025 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:36.339017 25004 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:36.368702 24790 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:36.369341 25025 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:36.369669 24941 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:36.372211 24927 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:36.372385 24750 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:36.329872 25018 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:36.367167 25050 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:36.339040 24672 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:36.365641 25006 raft_consensus.cc:3055] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:08:36.372802 24927 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:36.410583 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:36.414618 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:36.407786 24672 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:08:36.422513 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 203469us
I20260709 13:08:36.424129 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:36.424834 23623 raft_consensus.cc:1813] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Handling vote request from an unknown peer f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:36.426470 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:36.429500 23623 raft_consensus.cc:2393] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 4: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:36.408010 25039 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:36.431565 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:36.408411 25006 raft_consensus.cc:740] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:36.418684 25050 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:36.449234 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:36.452539 23625 raft_consensus.cc:2393] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 6: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:36.454797 24935 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:36.456248 25006 consensus_queue.cc:260] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:36.457033 24935 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:36.457751 25006 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:36.459551 24935 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:36.462728 24672 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:36.491982 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:36.477415 24672 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:36.507661 24817 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:36.491662 24571 consensus_queue.cc:1048] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:08:36.514186 24941 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:36.560458 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 93869.7us
W20260709 13:08:36.561982 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 151297us
I20260709 13:08:36.564994 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:36.565609 22947 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
W20260709 13:08:36.568138 22994 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:36.571676 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:36.572430 22871 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:36.578315 22920 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:36.580236 22843 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:36.561275 22992 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:36.585114 25039 raft_consensus.cc:2804] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:36.585667 25039 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:36.586089 25039 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:36.559430 25018 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:36.554008 24817 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:36.603922 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:36.582748 24927 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:36.591900 24945 raft_consensus.cc:2804] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:36.604635 22992 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:36.621129 25065 raft_consensus.cc:2749] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:36.622153 25065 raft_consensus.cc:2749] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:36.624723 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:36.625597 22948 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:08:36.626960 24954 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:36.627925 24953 consensus_queue.cc:1048] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:36.627779 24954 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:36.631168 23588 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:36.632442 24763 raft_consensus.cc:2749] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:08:36.634056 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:36.634827 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:36.635440 23028 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 4.
I20260709 13:08:36.635144 22992 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:36.634827 22952 raft_consensus.cc:1275] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:08:36.636811 25065 raft_consensus.cc:2804] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:36.637194 25065 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:36.637542 25065 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:36.635134 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:36.641829 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:36.635640 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:36.643314 22946 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:36.655814 22992 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:36.615902 24720 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:36.615792 24945 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:36.674013 25057 log.cc:927] Time spent T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 13:08:36.675737 25058 log.cc:927] Time spent T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 13:08:36.677242 25059 log.cc:927] Time spent T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 13:08:36.678882 25054 log.cc:927] Time spent T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
W20260709 13:08:36.679775 25048 log.cc:927] Time spent T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.246s	user 0.000s	sys 0.001s
I20260709 13:08:36.680884 22877 raft_consensus.cc:1275] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:36.681617 24817 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:36.654897 24821 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:36.682375 24821 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:36.684362 24821 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:36.672139 25050 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:36.686707 25071 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:36.655797 24927 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:36.650179 24954 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:36.719398 24417 raft_consensus.cc:993] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75: : Instructing follower aed3f089ec8b49af9c8cf5cdee76e8e7 to start an election
W20260709 13:08:37.363372 22948 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.629s	user 0.007s	sys 0.011s
W20260709 13:08:37.383407 22948 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.008s	sys 0.012s
I20260709 13:08:36.731875 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:36.690630 22877 raft_consensus.cc:1275] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:08:37.381794 23025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 358) took 1321 ms. Trace:
W20260709 13:08:37.390467 22946 consensus_meta.cc:287] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.747s	user 0.029s	sys 0.040s
I20260709 13:08:37.390898 22946 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
I20260709 13:08:37.388922 23025 rpcz_store.cc:276] 0709 13:08:36.060798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:36.061168 (+   370us) service_pool.cc:224] Handling call
0709 13:08:36.061722 (+   554us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:36.218433 (+156711us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:36.218462 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:36.229666 (+ 11204us) log.cc:844] Serialized 26 byte log entry
0709 13:08:36.445349 (+215683us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:36.445406 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:36.445497 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:37.381484 (+935987us) raft_consensus.cc:1685] finished
0709 13:08:37.381510 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:37.381725 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":156301,"raft.queue_time_us":167184,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":5382,"thread_start_us":141183,"threads_started":1,"wal-append.queue_time_us":276233}
I20260709 13:08:36.667304 24945 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:08:36.673116 25053 log.cc:927] Time spent T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
W20260709 13:08:37.218640 25071 consensus_meta.cc:287] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.531s	user 0.009s	sys 0.000s
I20260709 13:08:37.413827 24954 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:08:36.741456 25061 log.cc:927] Time spent T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:08:37.105736 24763 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:37.403393 25039 consensus_meta.cc:287] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.817s	user 0.016s	sys 0.031s
W20260709 13:08:37.212795 25066 log.cc:927] Time spent T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.144s	user 0.001s	sys 0.000s
I20260709 13:08:36.725499 25056 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:37.135448 25018 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:37.062995 24890 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:37.009020 25006 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:36.990167 25050 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:36.972604 25062 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:36.687930 24720 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:36.742059 24750 consensus_queue.cc:1048] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.682s
I20260709 13:08:36.723405 24738 consensus_queue.cc:1048] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.701s
I20260709 13:08:37.423189 25071 raft_consensus.cc:2749] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:36.721728 25025 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:36.741408 25065 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:37.394778 24927 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:36.721073 25004 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:37.444850 22955 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
I20260709 13:08:37.437211 25075 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:37.424216 24954 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:37.438388 24890 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:37.426232 24763 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:37.427695 25039 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:37.461706 22950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 276) took 1637 ms. Trace:
I20260709 13:08:37.457315 25075 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:37.462144 22950 rpcz_store.cc:276] 0709 13:08:35.824357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:35.824502 (+   145us) service_pool.cc:224] Handling call
0709 13:08:35.824843 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:35.825189 (+   346us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:35.825218 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:35.832146 (+  6928us) log.cc:844] Serialized 26 byte log entry
0709 13:08:36.510769 (+678623us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b180038fec0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cdae07 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf7c02 00007f613597f20b 00007f6135984d23 00007f61359850d1 00007f6135da0a0f
0709 13:08:36.618567 (+107798us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:36.618623 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:36.618722 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:37.461247 (+842525us) raft_consensus.cc:1685] finished
0709 13:08:37.461278 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:37.461584 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2498,"raft.queue_time_us":506,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":9676,"spinlock_wait_cycles":31105920,"thread_start_us":93359,"threads_started":1,"wal-append.queue_time_us":1169908}
I20260709 13:08:37.465262 25019 raft_consensus.cc:1081] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Signalling peer aed3f089ec8b49af9c8cf5cdee76e8e7 to start an election
I20260709 13:08:37.465499 25074 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:37.466060 25039 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:37.466086 25074 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:37.466814 25039 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:37.467080 25019 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:37.467365 25039 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:37.467630 25019 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:37.447160 25071 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:37.431829 25018 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:37.431557 25050 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:37.430734 25056 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:37.468775 24954 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:37.468531 25074 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:37.483599 24890 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:37.439623 25006 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:37.483865 25065 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:37.438344 25062 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:37.466306 25076 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:37.492328 25050 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:37.492921 25050 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:37.492208 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:37.494156 22875 raft_consensus.cc:2393] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 6: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:37.494537 22875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35370 (request call id 1917) took 1125 ms. Trace:
I20260709 13:08:37.493517 25039 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:37.493377 25050 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:37.483635 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:37.494771 22875 rpcz_store.cc:276] 0709 13:08:36.368917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:36.371690 (+  2773us) service_pool.cc:224] Handling call
0709 13:08:37.492044 (+1120354us) spinlock_profiling.cc:243] Waited 24.6 ms on lock 0x7b5800000c98. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0a22 00007f613bbd9c09 00007f613ad33ee0 00007f613adfd0ab 00007f613adfd131 00007f613adab0a0 00007f613adaa984 00007f61343256d4 00007f6134325662
0709 13:08:37.494493 (+  2449us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":56694528}
I20260709 13:08:37.488006 25074 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:37.487221 25077 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:37.501412 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:37.480008 25071 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:37.502954 23624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 462) took 1074 ms. Trace:
I20260709 13:08:37.503197 23624 rpcz_store.cc:276] 0709 13:08:36.428688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:36.428785 (+    97us) service_pool.cc:224] Handling call
0709 13:08:37.501253 (+1072468us) spinlock_profiling.cc:243] Waited 992 us on lock 0x7b58001c4d98. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0a22 00007f613bbd9c09 00007f613ad33ee0 00007f613adfd0ab 00007f613adfd131 00007f613adab0a0 00007f613adaa984 00007f61343256d4 00007f6134325662
0709 13:08:37.502904 (+  1651us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2280832}
I20260709 13:08:37.502365 25065 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:37.501151 23616 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:37.489987 25076 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:37.504607 25065 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:37.507539 25076 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:37.512928 24927 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:37.515348 23620 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 455) took 1400 ms. Trace:
I20260709 13:08:37.517119 24720 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:37.515761 23620 rpcz_store.cc:276] 0709 13:08:36.114999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:36.118870 (+  3871us) service_pool.cc:224] Handling call
0709 13:08:36.212723 (+ 93853us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:36.262944 (+ 50221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:36.262991 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:36.263635 (+   644us) log.cc:844] Serialized 26 byte log entry
0709 13:08:36.499475 (+235840us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:36.653634 (+154159us) log.cc:844] Serialized 14 byte log entry
0709 13:08:36.687886 (+ 34252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:36.727415 (+ 39529us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:37.512786 (+785371us) spinlock_profiling.cc:243] Waited 25.7 ms on lock 0x7b18002f8120. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cc62a6 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135d5adbe 00007f6135d65930 00007f6135d658d8
0709 13:08:37.514234 (+  1448us) spinlock_profiling.cc:243] Waited 14 ms on lock 0x7b540063eaa0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135d5adbe 00007f6135d65930 00007f6135d658d8 00007f6135d65858 00007f6135d65813 00007f6135d64a9c 00007f613bc50626 00007f613bc4add8 00007f6135da17cb 00007f6135da42c0
0709 13:08:37.515126 (+   892us) raft_consensus.cc:1685] finished
0709 13:08:37.515150 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:37.515277 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":166856,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":10787,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":89,"spinlock_wait_cycles":118398336,"thread_start_us":302800,"threads_started":2,"wal-append.queue_time_us":296487}
I20260709 13:08:37.520617 24763 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:37.518488 25075 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:37.525759 22993 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:37.526389 25071 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:37.521117 24939 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:37.530596 22993 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:37.507611 25065 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:37.527521 25062 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:37.505122 25074 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:37.529323 24907 raft_consensus.cc:2749] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:37.529032 25019 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:08:37.554756 22918 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.606s (SENT) [suppressed 6 similar messages]
W20260709 13:08:37.589141 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 91764.3us
I20260709 13:08:37.534322 25078 raft_consensus.cc:2749] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:37.601264 25025 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:37.509413 25077 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:37.609185 25077 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:37.539295 24939 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:37.606813 25095 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:37.610178 25095 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:37.611382 24939 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:37.611894 24939 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:37.612242 25095 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:37.612346 24939 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:37.614424 24939 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:37.618054 25077 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:37.618844 22872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 1913) took 1251 ms. Trace:
I20260709 13:08:37.618590 25077 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:37.619190 22872 rpcz_store.cc:276] 0709 13:08:36.367855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:36.374026 (+  6171us) service_pool.cc:224] Handling call
0709 13:08:37.618189 (+1244163us) spinlock_profiling.cc:243] Waited 38.2 ms on lock 0x7b5800000c98. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0a22 00007f613bbd9c09 00007f613ad33ee0 00007f613adfd0ab 00007f613adfd131 00007f613adbefc0 00007f613ada9929 00007f613ada93d2 00007f6134320bd4
0709 13:08:37.618377 (+   188us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:37.618528 (+   151us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:37.618552 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:37.618592 (+    40us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:08:37.618624 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:37.618686 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:37.618795 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":87922432}
I20260709 13:08:37.620887 25077 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:37.623310 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:37.623967 22954 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:08:37.628342 22879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 1918) took 1259 ms. Trace:
I20260709 13:08:37.603986 25080 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:37.629475 25018 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:37.629309 25080 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:37.630491 25081 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:37.631012 25081 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:37.631711 25080 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:37.623423 24490 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:37.633709 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:37.628969 22879 rpcz_store.cc:276] 0709 13:08:36.369166 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:36.454636 (+ 85470us) service_pool.cc:224] Handling call
0709 13:08:36.455637 (+  1001us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:36.489210 (+ 33573us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:36.489254 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:36.489308 (+    54us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:08:36.489352 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:37.628028 (+1138676us) spinlock_profiling.cc:243] Waited 33.3 ms on lock 0x7b5400367754. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:37.628100 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:37.628265 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":76610816}
I20260709 13:08:37.634346 22949 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
I20260709 13:08:37.635365 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:37.635680 22954 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
I20260709 13:08:37.637508 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:37.638967 25080 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:37.639508 25080 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:37.640805 25077 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:37.641276 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:37.642376 22992 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:37.642835 23026 raft_consensus.cc:1275] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:08:37.643532 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:37.642856 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:37.644701 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:37.645970 23024 raft_consensus.cc:1275] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:37.646590 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:37.643556 22880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 1915) took 1275 ms. Trace:
I20260709 13:08:37.647131 23020 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:37.645120 23025 raft_consensus.cc:2368] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 5. Current term is 6.
I20260709 13:08:37.652263 23020 raft_consensus.cc:1275] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:37.651397 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:37.669749 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:37.684150 23028 raft_consensus.cc:2393] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 5: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:37.651804 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:37.648550 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:37.649233 22992 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:37.686075 22954 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
I20260709 13:08:37.677496 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:37.687263 22878 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:37.652182 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:37.641342 25077 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:37.675859 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:37.602900 25056 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:37.693995 22950 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:58320
I20260709 13:08:37.694500 22919 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:37.632467 24490 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:37.648409 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:38.107625 22952 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
W20260709 13:08:37.798332 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 243587us
I20260709 13:08:37.645587 23021 raft_consensus.cc:2376] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4. Re-sending same reply.
I20260709 13:08:37.702606 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:38.110073 23624 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
W20260709 13:08:37.696563 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 47423.1us
W20260709 13:08:38.094659 22920 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.003s	sys 0.016s
I20260709 13:08:38.095044 22878 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
W20260709 13:08:38.111982 22920 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.004s	sys 0.016s
I20260709 13:08:37.675944 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:37.755939 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:37.647300 22880 rpcz_store.cc:276] 0709 13:08:36.368406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:36.372683 (+  4277us) service_pool.cc:224] Handling call
0709 13:08:37.642978 (+1270295us) spinlock_profiling.cc:243] Waited 24.8 ms on lock 0x7b5800000c98. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0a22 00007f613bbd9c09 00007f613ad33ee0 00007f613adfd0ab 00007f613adfd131 00007f613adbefc0 00007f613ada9929 00007f613ada93d2 00007f6134320bd4
0709 13:08:37.643148 (+   170us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:37.643292 (+   144us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:37.643316 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:37.643351 (+    35us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:08:37.643383 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:37.643428 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:37.643511 (+    83us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":56948096}
I20260709 13:08:38.115293 23027 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 5.
I20260709 13:08:37.650522 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:38.116385 22950 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:38.117187 22950 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:08:38.117347 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 422908us
I20260709 13:08:38.117553 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:38.118166 22875 raft_consensus.cc:1813] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Handling vote request from an unknown peer f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:37.669245 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:37.862553 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:38.118471 22879 raft_consensus.cc:2393] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 6: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:38.118927 22949 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:38.149884 22875 raft_consensus.cc:2368] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 4. Current term is 5.
W20260709 13:08:38.475374 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 78923.5us
I20260709 13:08:38.297811 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:38.265806 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:38.850243 22951 raft_consensus.cc:2410] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 13:08:38.298043 22876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 1916) took 1929 ms. Trace:
I20260709 13:08:38.371950 22949 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6.
I20260709 13:08:38.851708 22949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52222 (request call id 290) took 1213 ms. Trace:
I20260709 13:08:38.302640 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:38.154906 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:38.853926 22879 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:38.125424 22992 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:38.855417 22875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35370 (request call id 1929) took 1183 ms. Trace:
W20260709 13:08:38.614298 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 315744us
I20260709 13:08:38.847698 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:38.856787 22875 rpcz_store.cc:276] 0709 13:08:37.671659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:37.675593 (+  3934us) service_pool.cc:224] Handling call
0709 13:08:38.855355 (+1179762us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:38.858059 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:37.692812 25087 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:38.869043 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:38.869850 22875 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 3.
I20260709 13:08:37.882992 24921 raft_consensus.cc:993] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75: : Instructing follower aed3f089ec8b49af9c8cf5cdee76e8e7 to start an election
I20260709 13:08:38.386600 22950 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:38.119176 24943 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.791s	user 0.015s	sys 0.010s
I20260709 13:08:38.851238 22876 rpcz_store.cc:276] 0709 13:08:36.368648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:36.371535 (+  2887us) service_pool.cc:224] Handling call
0709 13:08:38.297362 (+1925827us) spinlock_profiling.cc:243] Waited 25.3 ms on lock 0x7b5800000c98. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0a22 00007f613bbd9c09 00007f613ad33ee0 00007f613adfd0ab 00007f613adfd131 00007f613adbefc0 00007f613ada9929 00007f613ada93d2 00007f6134320bd4
0709 13:08:38.297564 (+   202us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:38.297716 (+   152us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:38.297741 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:38.297786 (+    45us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:38.297813 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:38.297861 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:38.297977 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":58115968}
I20260709 13:08:38.872594 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:37.656652 24868 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:38.873651 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:38.874181 24868 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:38.875039 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:37.845098 25062 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:37.880882 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:37.841456 25086 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:38.123083 25076 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:37.844362 24743 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:37.755820 25070 raft_consensus.cc:2804] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:37.945832 25074 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:38.128207 24907 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:38.302413 25071 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
W20260709 13:08:38.798964 25097 log.cc:927] Time spent T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.216s	user 0.000s	sys 0.000s
W20260709 13:08:38.124811 24945 consensus_meta.cc:287] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.702s	user 0.008s	sys 0.008s
I20260709 13:08:38.852342 22949 rpcz_store.cc:276] 0709 13:08:37.637865 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:37.638776 (+   911us) service_pool.cc:224] Handling call
0709 13:08:38.851622 (+1212846us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:38.859691 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:37.885125 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:37.769121 25088 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:37.768507 25025 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:37.951102 24889 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:08:38.856369 24890 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.006s	sys 0.016s
W20260709 13:08:38.884553 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 759106us
I20260709 13:08:38.882251 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:37.844974 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:08:38.886340 22952 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.013s	sys 0.009s
I20260709 13:08:38.114792 24954 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:38.123399 24935 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:38.122387 25065 raft_consensus.cc:2804] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:08:38.122781 25075 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:38.887655 22950 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:37.892688 25050 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:38.122076 25090 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:37.841645 24927 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:38.134375 24972 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:38.122123 24738 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:38.122642 25039 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:37.692973 25006 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:38.374313 25096 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:38.479825 25085 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:38.142982 24998 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:38.124500 25081 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:38.382272 24490 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:38.347463 25084 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
W20260709 13:08:38.273687 23616 consensus_meta.cc:287] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.768s	user 0.007s	sys 0.003s
I20260709 13:08:38.477290 25023 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:08:38.962594 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 48979.4us
I20260709 13:08:38.964385 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:38.799546 25092 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:08:39.041536 22952 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.752s	user 0.013s	sys 0.017s
I20260709 13:08:39.040396 23623 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:08:39.042492 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 179650us
W20260709 13:08:38.876636 24763 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.004s	sys 0.029s
I20260709 13:08:39.170697 22950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:58320 (request call id 2024) took 1419 ms. Trace:
I20260709 13:08:39.171074 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:39.045914 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:39.047518 23029 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 1445) took 1400 ms. Trace:
I20260709 13:08:39.041319 22992 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:08:38.878163 24943 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.552s	user 0.020s	sys 0.038s
W20260709 13:08:39.050292 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 215264us
I20260709 13:08:39.171432 22950 rpcz_store.cc:276] 0709 13:08:37.621400 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:37.693831 (+ 72431us) service_pool.cc:224] Handling call
0709 13:08:39.040520 (+1346689us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:39.172924 22949 raft_consensus.cc:2368] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 5. Current term is 6.
W20260709 13:08:39.173530 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 132291us
W20260709 13:08:39.050845 24890 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.751s	user 0.007s	sys 0.017s
I20260709 13:08:38.899786 25019 raft_consensus.cc:2804] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:39.202483 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:38.997087 25111 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:39.205526 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:38.964855 24921 raft_consensus.cc:1081] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 2 LEADER]: Signalling peer aed3f089ec8b49af9c8cf5cdee76e8e7 to start an election
I20260709 13:08:39.207494 24417 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:39.239255 24417 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:39.206671 25050 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:39.218967 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:38.987074 25070 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:39.205729 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:08:39.207223 25082 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.680s	user 0.063s	sys 0.127s
I20260709 13:08:38.989086 24868 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:39.050803 24750 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:38.995705 25014 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:39.173147 23029 rpcz_store.cc:276] 0709 13:08:37.641764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:37.644340 (+  2576us) service_pool.cc:224] Handling call
0709 13:08:37.693382 (+ 49042us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:38.290288 (+596906us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:38.290318 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:38.290356 (+    38us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:08:38.290387 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:39.042445 (+752058us) spinlock_profiling.cc:243] Waited 597 ms on lock 0x7b54001f7cd4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:39.042491 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:39.042611 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1372269312}
W20260709 13:08:39.176952 24763 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.913s	user 0.011s	sys 0.032s
I20260709 13:08:39.172087 25025 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:39.171267 25006 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:39.170557 25088 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:38.885733 25087 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:38.893836 25062 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:39.048969 24907 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:39.003844 25076 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:39.176218 24954 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:39.056849 24945 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:39.171149 25023 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:39.176766 25081 consensus_queue.cc:1048] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.174s
I20260709 13:08:39.170269 24738 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:38.963806 25086 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:39.169620 24998 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:39.172359 25071 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:39.182516 24889 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:39.169865 24972 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:39.169660 25039 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:39.003087 24743 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:39.170866 25084 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:39.171391 25085 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:39.171015 25096 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:39.169826 25090 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:39.830813 22949 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
I20260709 13:08:39.831843 22949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58310 (request call id 1487) took 1355 ms. Trace:
I20260709 13:08:39.832418 22949 rpcz_store.cc:276] 0709 13:08:38.476495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:39.205549 (+729054us) service_pool.cc:224] Handling call
0709 13:08:39.831769 (+626220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:39.174271 24943 raft_consensus.cc:2804] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:39.050406 23616 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 5.
I20260709 13:08:39.616523 22877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 1933) took 1927 ms. Trace:
I20260709 13:08:39.662819 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:39.837061 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:39.862445 22948 raft_consensus.cc:2393] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 6: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:39.863106 22949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 291) took 1743 ms. Trace:
I20260709 13:08:39.863580 22949 rpcz_store.cc:276] 0709 13:08:38.120111 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:39.862145 (+1742034us) service_pool.cc:224] Handling call
0709 13:08:39.862517 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:39.862710 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:39.862746 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:39.862800 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:39.862844 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:39.862911 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:39.863043 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:39.864293 22948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 292) took 1743 ms. Trace:
W20260709 13:08:39.567238 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 44812.8us
I20260709 13:08:39.614888 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:39.838189 22951 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 3.
I20260709 13:08:39.867944 22951 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58310 (request call id 1490) took 1225 ms. Trace:
W20260709 13:08:39.737444 22879 consensus_meta.cc:287] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.799s	user 0.006s	sys 0.000s
I20260709 13:08:39.869547 22879 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 5.
W20260709 13:08:39.870240 22879 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53724 (request call id 312) took 1716 ms (client timeout 1553 ms). Trace:
W20260709 13:08:39.870715 22879 rpcz_store.cc:269] 0709 13:08:38.154366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:38.154562 (+   196us) service_pool.cc:224] Handling call
0709 13:08:39.870151 (+1715589us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:39.684962 22947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2027) took 2052 ms. Trace:
W20260709 13:08:39.765846 22993 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.008s	sys 0.000s
I20260709 13:08:39.217271 24571 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:39.327144 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:39.874612 22950 raft_consensus.cc:2393] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 6: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
W20260709 13:08:39.764863 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 528714us
I20260709 13:08:39.508416 23622 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60044 (request call id 483) took 1858 ms. Trace:
I20260709 13:08:39.185724 25074 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:08:39.842729 23623 consensus_meta.cc:287] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.666s	user 0.010s	sys 0.000s
I20260709 13:08:39.878576 23623 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
W20260709 13:08:39.817396 24935 consensus_meta.cc:287] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.768s	user 0.010s	sys 0.000s
I20260709 13:08:39.847043 22953 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2019) took 2308 ms. Trace:
I20260709 13:08:39.879225 23623 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 463) took 1530 ms. Trace:
W20260709 13:08:39.880523 22955 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58320 (request call id 2013) took 3121 ms (client timeout 1966 ms). Trace:
I20260709 13:08:39.882449 22950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58310 (request call id 1483) took 1513 ms. Trace:
I20260709 13:08:39.883005 22950 rpcz_store.cc:276] 0709 13:08:38.369572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:39.326797 (+957225us) service_pool.cc:224] Handling call
0709 13:08:39.882366 (+555569us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:39.844852 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:39.875674 23622 rpcz_store.cc:276] 0709 13:08:37.649848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:37.649968 (+   120us) service_pool.cc:224] Handling call
0709 13:08:37.650240 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:38.290292 (+640052us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:38.290330 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:38.290379 (+    49us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:38.290422 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:39.508121 (+1217699us) spinlock_profiling.cc:243] Waited 640 ms on lock 0x7b540045bfd4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:39.508177 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:39.508347 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1471456640}
I20260709 13:08:39.190635 25065 raft_consensus.cc:697] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 6 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:39.848235 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:39.879293 22953 rpcz_store.cc:276] 0709 13:08:37.538763 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:37.539152 (+   389us) service_pool.cc:224] Handling call
0709 13:08:37.539595 (+   443us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:37.610101 (+ 70506us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:37.610127 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:37.610166 (+    39us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:08:37.610216 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:39.846676 (+2236460us) spinlock_profiling.cc:243] Waited 70.3 ms on lock 0x7b54003c78d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:39.846739 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:39.846931 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":161723776}
I20260709 13:08:39.891829 23625 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
I20260709 13:08:39.879616 23623 rpcz_store.cc:276] 0709 13:08:38.348538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:38.881937 (+533399us) service_pool.cc:224] Handling call
0709 13:08:39.879161 (+997224us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:08:39.501580 25103 log.cc:927] Time spent T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.271s	user 0.000s	sys 0.000s
I20260709 13:08:39.868525 22951 rpcz_store.cc:276] 0709 13:08:38.642416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:39.218708 (+576292us) service_pool.cc:224] Handling call
0709 13:08:39.867830 (+649122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:39.853839 24417 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:39.895745 22876 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
W20260709 13:08:39.664311 25102 log.cc:927] Time spent T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.262s	user 0.000s	sys 0.000s
I20260709 13:08:39.859829 22877 rpcz_store.cc:276] 0709 13:08:37.689194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:37.689429 (+   235us) service_pool.cc:224] Handling call
0709 13:08:39.615675 (+1926246us) spinlock_profiling.cc:243] Waited 22.2 ms on lock 0x7b5800000c98. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0a22 00007f613bbd9c09 00007f613ad33ee0 00007f613adfd0ab 00007f613adfd131 00007f613adbefc0 00007f613ada9929 00007f613ada93d2 00007f6134320bd4
0709 13:08:39.615894 (+   219us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:39.616110 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:39.616145 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:39.616195 (+    50us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:08:39.616237 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:39.616306 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:39.616455 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":51079040}
W20260709 13:08:39.896617 22994 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.723s	user 0.003s	sys 0.022s
I20260709 13:08:39.896803 22871 raft_consensus.cc:1275] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:08:39.896816 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:08:39.897012 22994 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.723s	user 0.003s	sys 0.022s
I20260709 13:08:39.890515 23624 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
W20260709 13:08:39.833688 25082 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.338s	user 0.077s	sys 0.140s
I20260709 13:08:39.218210 25019 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:39.217830 25052 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:08:39.881449 22955 rpcz_store.cc:269] 0709 13:08:36.731533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:36.731655 (+   122us) service_pool.cc:224] Handling call
0709 13:08:39.852943 (+3121288us) spinlock_profiling.cc:243] Waited 60.2 ms on lock 0x7b5400411ed4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:39.853086 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":138507648}
I20260709 13:08:39.893342 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:08:39.877017 22993 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.009s	sys 0.001s
I20260709 13:08:39.872090 22947 rpcz_store.cc:276] 0709 13:08:37.633032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:37.633156 (+   124us) service_pool.cc:224] Handling call
0709 13:08:37.657062 (+ 23906us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:37.657516 (+   454us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:37.657555 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:37.692974 (+ 35419us) log.cc:844] Serialized 26 byte log entry
0709 13:08:38.299541 (+606567us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:08:38.299577 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:38.299651 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:39.684698 (+1385047us) raft_consensus.cc:1685] finished
0709 13:08:39.684725 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:39.684888 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":27611,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":144193,"thread_start_us":417725,"threads_started":1,"wal-append.queue_time_us":1703578}
I20260709 13:08:39.887384 22872 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:08:36.523813 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:39.235718 25077 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:39.865010 22948 rpcz_store.cc:276] 0709 13:08:38.120370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:39.863338 (+1742968us) service_pool.cc:224] Handling call
0709 13:08:39.863681 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:39.863895 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:39.863934 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:39.863987 (+    53us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:39.864035 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:39.864103 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:39.864228 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:39.564801 25079 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:39.471832 24890 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:39.237627 23620 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60044 (request call id 481) took 1584 ms (client timeout 2003 ms). Trace:
I20260709 13:08:39.565424 25089 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:39.239567 24455 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:39.898878 22873 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:08:39.506505 25080 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:39.847651 25006 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:39.848331 25050 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:39.897861 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 723952us
I20260709 13:08:39.899823 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:39.900523 22946 raft_consensus.cc:2393] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 4: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
W20260709 13:08:39.900977 22952 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.008s	sys 0.019s
W20260709 13:08:39.901118 23625 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60044 (request call id 485) took 2141 ms (client timeout 1689 ms). Trace:
I20260709 13:08:39.791836 25113 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
W20260709 13:08:39.248450 23621 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 542) took 1354 ms (client timeout 1587 ms). Trace:
I20260709 13:08:39.903357 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:39.482728 25117 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:39.380515 25106 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:39.840240 25070 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:39.841123 24943 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:39.171761 25075 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:39.848635 24720 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
W20260709 13:08:39.907657 23588 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.028s	sys 0.031s
I20260709 13:08:39.908527 22879 raft_consensus.cc:2410] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
W20260709 13:08:39.908234 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 861352us
W20260709 13:08:39.916803 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 68458.2us
I20260709 13:08:39.906517 25004 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:39.917900 25127 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:39.860886 24750 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:39.919677 25052 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:39.910619 25019 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:39.176486 25092 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:39.176651 24927 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:39.894446 24571 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:39.895375 25074 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:39.252079 25111 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:39.848778 25014 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:40.016407 22950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 294) took 1891 ms. Trace:
I20260709 13:08:40.143893 24943 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:40.144266 22950 rpcz_store.cc:276] 0709 13:08:38.124820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:39.883902 (+1759082us) service_pool.cc:224] Handling call
0709 13:08:39.884316 (+   414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:39.901843 (+ 17527us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:39.901886 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:39.901941 (+    55us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:39.901986 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:40.016120 (+114134us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b540037aad4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:40.016184 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:40.016346 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39716736}
I20260709 13:08:40.020743 24998 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:40.145747 25052 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:40.146579 23617 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 6.
I20260709 13:08:40.051189 22774 catalog_manager.cc:5697] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 3 to 4, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 4 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:40.147610 25092 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:40.147843 23617 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 538) took 2291 ms (client timeout 1878 ms). Trace:
W20260709 13:08:40.148334 23617 rpcz_store.cc:269] 0709 13:08:37.856617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:37.880474 (+ 23857us) service_pool.cc:224] Handling call
0709 13:08:40.147630 (+2267156us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b540063e7d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:40.147774 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2894336384}
I20260709 13:08:40.148847 25106 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:40.149513 24571 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:40.149714 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:40.151180 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:40.151326 25106 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:40.151602 24750 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:40.152125 25127 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:39.927112 22954 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2034) took 2286 ms. Trace:
I20260709 13:08:40.148595 22774 catalog_manager.cc:5697] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 2 to 4, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 4 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:08:40.153236 22954 rpcz_store.cc:276] 0709 13:08:37.640471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:38.122333 (+481862us) service_pool.cc:224] Handling call
0709 13:08:38.122640 (+   307us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:38.122837 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:38.122859 (+    22us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:38.123279 (+   420us) log.cc:844] Serialized 26 byte log entry
0709 13:08:38.477247 (+353968us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:08:38.477303 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:38.477393 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:39.926841 (+1449448us) raft_consensus.cc:1685] finished
0709 13:08:39.926869 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:39.927037 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":353661,"threads_started":1,"wal-append.queue_time_us":846617}
I20260709 13:08:40.154691 25127 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:40.154489 22774 catalog_manager.cc:5697] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:40.155323 22843 connection.cc:256] RPC call timeout handler was delayed by -1.2717340149999927s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1918s for a response
W20260709 13:08:40.155902 22843 connection.cc:256] RPC call timeout handler was delayed by -1.1441826860000219s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19390000000000002s for a response
I20260709 13:08:40.156466 25006 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:40.156728 23588 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.755s	user 0.029s	sys 0.031s
I20260709 13:08:40.157387 25085 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:40.157917 25085 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:40.158005 25074 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:40.158133 22843 leader_election.cc:400] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 5. Current term is 6.
I20260709 13:08:40.158672 22843 leader_election.cc:403] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:08:40.158591 25074 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:40.173179 25074 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:40.031317 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:40.174014 24998 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 5. Current term is 6.
I20260709 13:08:40.174212 23027 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:40.021598 22871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 1932) took 2349 ms. Trace:
I20260709 13:08:39.904587 25123 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:40.174845 22871 rpcz_store.cc:276] 0709 13:08:37.672583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:37.677656 (+  5073us) service_pool.cc:224] Handling call
0709 13:08:38.117658 (+440002us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:39.897495 (+1779837us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:40.021329 (+123834us) spinlock_profiling.cc:243] Waited 1.77 s on lock 0x7b5400369554. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:40.021531 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4080061824}
I20260709 13:08:40.175411 25074 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:40.175565 25123 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:40.176263 24455 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:39.891255 25115 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:40.177505 25115 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:39.960954 24907 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:39.924851 25023 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:40.160197 22880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53724 (request call id 314) took 1886 ms. Trace:
I20260709 13:08:39.922681 25084 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:40.158461 25085 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:40.178591 22880 rpcz_store.cc:276] 0709 13:08:38.274232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:38.300410 (+ 26178us) service_pool.cc:224] Handling call
0709 13:08:38.373987 (+ 73577us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:40.159370 (+1785383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:40.159397 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:40.159439 (+    42us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:40.159477 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:40.160000 (+   523us) spinlock_profiling.cc:243] Waited 1.78 s on lock 0x7b5400368654. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:40.160038 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:40.160136 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4105460096}
I20260709 13:08:40.179623 25115 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:40.027200 24972 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:40.179976 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:40.180872 22880 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
I20260709 13:08:40.181468 24945 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:40.181770 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:40.182387 22880 raft_consensus.cc:2468] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:08:40.182633 24738 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:40.183748 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:40.185004 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:40.185604 22880 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
I20260709 13:08:40.166221 25111 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:40.186389 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:40.168095 25014 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:40.187637 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:40.188300 22880 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:40.160847 24720 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:40.190749 22953 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 295) took 1935 ms. Trace:
I20260709 13:08:40.190956 24720 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:40.161588 24927 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:40.191228 22953 rpcz_store.cc:276] 0709 13:08:38.254843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:39.898969 (+1644126us) service_pool.cc:224] Handling call
0709 13:08:39.899297 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:40.189389 (+290092us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:40.189417 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:40.189456 (+    39us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:40.189487 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:40.190481 (+   994us) spinlock_profiling.cc:243] Waited 290 ms on lock 0x7b540035d4d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:40.190539 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:40.190686 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":666734336}
I20260709 13:08:40.191668 25085 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:40.019603 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:40.196424 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: true dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:40.197019 22871 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:40.148542 25117 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:40.209301 25117 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:39.996193 24921 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:40.004019 22874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53724 (request call id 310) took 3680 ms. Trace:
I20260709 13:08:40.162312 25050 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:40.216336 24921 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:39.950362 25088 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:40.216288 22874 rpcz_store.cc:276] 0709 13:08:36.323153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:36.323308 (+   155us) service_pool.cc:224] Handling call
0709 13:08:37.883337 (+1560029us) spinlock_profiling.cc:243] Waited 14.8 ms on lock 0x7b5800000c98. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0a22 00007f613bbd9c09 00007f613ad33ee0 00007f613adfd0ab 00007f613adfd131 00007f613adbefc0 00007f613ada9929 00007f613ada93d2 00007f6134320bd4
0709 13:08:38.578134 (+694797us) spinlock_profiling.cc:243] Waited 2.43 ms on lock 0x7b5800000c98. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0e9c 00007f6132cc0a5b 00007f613bbd9c09 00007f613ad33ee0 00007f613adfd0ab 00007f613adfd131 00007f613adbefc0 00007f613ada9929 00007f613ada93d2 00007f6134320bd4
0709 13:08:38.578407 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:39.896363 (+1317956us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:39.896404 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:39.903050 (+  6646us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:39.903093 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:40.003688 (+100595us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b5400412654. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:40.003767 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:40.003915 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3069973504}
I20260709 13:08:40.045495 25025 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:40.217461 25092 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:40.163866 25071 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:40.164302 25079 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:39.958217 22878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53724 (request call id 313) took 1696 ms. Trace:
I20260709 13:08:40.160562 25075 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:40.166253 23621 rpcz_store.cc:269] 0709 13:08:37.882107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:37.884561 (+  2454us) service_pool.cc:224] Handling call
0709 13:08:39.236657 (+1352096us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:40.218941 22954 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 1482) took 1927 ms. Trace:
I20260709 13:08:40.219703 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:40.219969 23029 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:40.167052 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:40.167229 25090 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:40.167306 22952 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2033) took 2527 ms. Trace:
I20260709 13:08:40.221055 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:40.166890 24954 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:40.167747 23625 rpcz_store.cc:269] 0709 13:08:37.751107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:37.751251 (+   144us) service_pool.cc:224] Handling call
0709 13:08:39.848089 (+2096838us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b58001c4d98. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0a22 00007f613bbd9c09 00007f613ad33ee0 00007f613adfd0ab 00007f613adfd131 00007f613adab0a0 00007f613adaa984 00007f61343256d4 00007f6134325662
0709 13:08:39.892362 (+ 44273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25334144}
I20260709 13:08:40.019186 25076 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:40.003057 23624 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 541) took 2121 ms (client timeout 2239 ms). Trace:
I20260709 13:08:40.221089 22952 rpcz_store.cc:276] 0709 13:08:37.640088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:38.126688 (+486600us) service_pool.cc:224] Handling call
0709 13:08:38.289475 (+162787us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:38.289704 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:08:39.051424 (+761720us) log.cc:844] Serialized 14 byte log entry
0709 13:08:40.166976 (+1115552us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:40.167026 (+    50us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:40.167070 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:40.167131 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:40.167253 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3153,"prepare.queue_time_us":706484,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":568147,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":125,"spinlock_wait_cycles":16128,"thread_start_us":1873338,"threads_started":3,"wal-append.queue_time_us":794326}
I20260709 13:08:39.932524 25086 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:40.168920 22877 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 2.
I20260709 13:08:40.222609 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:08:40.222481 23624 rpcz_store.cc:269] 0709 13:08:37.881741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:38.883376 (+1001635us) service_pool.cc:224] Handling call
0709 13:08:40.002848 (+1119472us) spinlock_profiling.cc:243] Waited 849 ms on lock 0x7b54000cadd4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:40.002986 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1953575680}
I20260709 13:08:40.223664 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:40.223841 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:40.224246 22877 raft_consensus.cc:1275] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:08:40.224679 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:40.224906 22877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53724 (request call id 316) took 1925 ms. Trace:
I20260709 13:08:40.225214 24943 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:40.225382 22877 rpcz_store.cc:276] 0709 13:08:38.299748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:40.223744 (+1923996us) service_pool.cc:224] Handling call
0709 13:08:40.223984 (+   240us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:40.224553 (+   569us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:40.224858 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:40.225817 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:40.226720 22877 raft_consensus.cc:1275] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:08:40.226898 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:40.224952 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:40.227051 22877 pending_rounds.cc:85] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 5
I20260709 13:08:40.227917 22877 pending_rounds.cc:107] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:08:40.227792 22878 rpcz_store.cc:276] 0709 13:08:38.262343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:38.262494 (+   151us) service_pool.cc:224] Handling call
0709 13:08:38.262883 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:39.895129 (+1632246us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:39.895163 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:39.895203 (+    40us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:39.895238 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:39.957918 (+ 62680us) spinlock_profiling.cc:243] Waited 1.63 s on lock 0x7b54002d7154. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:39.957974 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:39.958150 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3753217280}
I20260709 13:08:40.229306 24943 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:40.143575 25039 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:40.169329 25113 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:40.219365 22954 rpcz_store.cc:276] 0709 13:08:38.292008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:40.154817 (+1862809us) service_pool.cc:224] Handling call
0709 13:08:40.218316 (+ 63499us) spinlock_profiling.cc:243] Waited 17.2 ms on lock 0x7b5800043598. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0a22 00007f613bbd9c09 00007f613ad33ee0 00007f613adfd0ab 00007f613adfd131 00007f613adbefc0 00007f613ada9929 00007f613ada93d2 00007f6134320bd4
0709 13:08:40.218467 (+   151us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:40.218618 (+   151us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:40.218642 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:40.218720 (+    78us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:40.218752 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:40.218801 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:40.218890 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39491456}
I20260709 13:08:40.238929 25113 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:40.240896 22949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 293) took 2120 ms. Trace:
I20260709 13:08:40.241365 22949 rpcz_store.cc:276] 0709 13:08:38.120591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:39.864972 (+1744381us) service_pool.cc:224] Handling call
0709 13:08:39.865283 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:40.239978 (+374695us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:40.240020 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:40.240073 (+    53us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:40.240117 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:40.240658 (+   541us) spinlock_profiling.cc:243] Waited 374 ms on lock 0x7b54000753d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:40.240718 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:40.240831 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":861169024}
I20260709 13:08:40.170315 22776 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33478 (request call id 275) took 8417 ms. Trace:
I20260709 13:08:40.244534 23028 raft_consensus.cc:1275] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:08:40.242607 22776 rpcz_store.cc:276] 0709 13:08:31.753523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:31.771505 (+ 17982us) service_pool.cc:224] Handling call
0709 13:08:40.170234 (+8398729us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:34.141983 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:34.177205 (+ 35222us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:34.177269 (+    64us) write_op.cc:435] Acquired schema lock
0709 13:08:34.177291 (+    22us) tablet.cc:662] Decoding operations
0709 13:08:34.180521 (+  3230us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:34.180626 (+   105us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:34.180653 (+    27us) tablet.cc:685] Acquiring locks for 12 operations
0709 13:08:34.198428 (+ 17775us) tablet.cc:701] Row locks acquired
0709 13:08:34.251590 (+ 53162us) write_op.cc:260] PREPARE: finished
0709 13:08:34.271904 (+ 20314us) write_op.cc:270] Start()
0709 13:08:34.330939 (+ 59035us) write_op.cc:276] Timestamp: P: 1783602514264748 usec, L: 0
0709 13:08:34.331539 (+   600us) op_driver.cc:348] REPLICATION: starting
0709 13:08:34.365805 (+ 34266us) log.cc:844] Serialized 4704 byte log entry
0709 13:08:34.786295 (+420490us) spinlock_profiling.cc:243] Waited 15.4 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f6132cdb0ac 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf7c02 00007f613597f20b
0709 13:08:35.565341 (+779046us) op_driver.cc:464] REPLICATION: finished
0709 13:08:35.699360 (+134019us) write_op.cc:301] APPLY: starting
0709 13:08:35.699459 (+    99us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:35.741695 (+ 42236us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:35.741751 (+    56us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:35.743803 (+  2052us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:35.788062 (+ 44259us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=12
0709 13:08:35.792486 (+  4424us) write_op.cc:312] APPLY: finished
0709 13:08:35.844373 (+ 51887us) log.cc:844] Serialized 89 byte log entry
0709 13:08:35.844587 (+   214us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:35.844733 (+   146us) write_op.cc:454] Released schema lock
0709 13:08:35.844801 (+    68us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:35.845190 (+   389us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:35.845227 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4001,"reported_tablets":28,"child_traces":[["op",{"apply.queue_time_us":121098,"apply.run_cpu_time_us":4829,"apply.run_wall_time_us":147400,"num_ops":12,"prepare.queue_time_us":236466,"prepare.run_cpu_time_us":123992,"prepare.run_wall_time_us":884691,"raft.queue_time_us":258712,"raft.run_cpu_time_us":17312,"raft.run_wall_time_us":164665,"replication_time_us":1186665,"spinlock_wait_cycles":35438336,"thread_start_us":1115901,"threads_started":4,"wal-append.queue_time_us":728885,"wal-append.run_cpu_time_us":91811,"wal-append.run_wall_time_us":1741951}]]}
I20260709 13:08:40.088447 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:08:40.173516 22873 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35370 (request call id 1914) took 3805 ms (client timeout 1659 ms). Trace:
I20260709 13:08:39.896342 24935 raft_consensus.cc:2749] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:40.177891 25123 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:40.248247 22873 rpcz_store.cc:269] 0709 13:08:36.368151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:36.371320 (+  3169us) service_pool.cc:224] Handling call
0709 13:08:37.684839 (+1313519us) spinlock_profiling.cc:243] Waited 24.7 ms on lock 0x7b5800000c98. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0a22 00007f613bbd9c09 00007f613ad33ee0 00007f613adfd0ab 00007f613adfd131 00007f613adab0a0 00007f613adaa984 00007f61343256d4 00007f6134325662
0709 13:08:39.844628 (+2159789us) spinlock_profiling.cc:243] Waited 23.1 ms on lock 0x7b5800000c98. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0e9c 00007f6132cc0a5b 00007f613bbd9c09 00007f613ad33ee0 00007f613adfd0ab 00007f613adfd131 00007f613adab0a0 00007f613adaa984 00007f61343256d4 00007f6134325662
0709 13:08:40.173343 (+328715us) spinlock_profiling.cc:243] Waited 13.8 ms on lock 0x7b54003bc754. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:40.173450 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":141586816}
I20260709 13:08:40.249094 22844 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:40.249918 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:40.250535 22873 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6.
I20260709 13:08:40.251341 23618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 543) took 1736 ms. Trace:
I20260709 13:08:40.251809 22873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 1937) took 1013 ms. Trace:
I20260709 13:08:40.251718 23618 rpcz_store.cc:276] 0709 13:08:38.514721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:39.001108 (+486387us) service_pool.cc:224] Handling call
0709 13:08:39.001555 (+   447us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:39.468001 (+466446us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:39.468057 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:39.780912 (+312855us) log.cc:844] Serialized 26 byte log entry
0709 13:08:40.247014 (+466102us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:40.250632 (+  3618us) log.cc:844] Serialized 14 byte log entry
0709 13:08:40.251037 (+   405us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:40.251117 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:40.251154 (+    37us) raft_consensus.cc:1685] finished
0709 13:08:40.251170 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:40.251290 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2830,"prepare.queue_time_us":3153,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":33,"raft.queue_time_us":380571,"raft.run_cpu_time_us":311,"raft.run_wall_time_us":74772,"thread_start_us":695843,"threads_started":3,"wal-append.queue_time_us":362677}
I20260709 13:08:40.252182 22873 rpcz_store.cc:276] 0709 13:08:39.238566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:40.251079 (+1012513us) service_pool.cc:224] Handling call
0709 13:08:40.251353 (+   274us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:40.251527 (+   174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:40.251551 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:40.251588 (+    37us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:08:40.251621 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:40.251670 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:40.251763 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:40.248122 22878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 1438) took 1029 ms. Trace:
I20260709 13:08:39.934077 25062 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:40.256276 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 79077.1us
I20260709 13:08:39.959442 25065 consensus_queue.cc:237] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:40.256501 22878 rpcz_store.cc:276] 0709 13:08:39.218423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:40.228974 (+1010551us) service_pool.cc:224] Handling call
0709 13:08:40.229234 (+   260us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:40.229542 (+   308us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:40.229568 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:40.229978 (+   410us) log.cc:844] Serialized 26 byte log entry
0709 13:08:40.247832 (+ 17854us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:08:40.247865 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:40.247926 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:40.247963 (+    37us) raft_consensus.cc:1685] finished
0709 13:08:40.247976 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:40.248075 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1196,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":96,"thread_start_us":17512,"threads_started":1,"wal-append.queue_time_us":15347}
I20260709 13:08:40.262804 25086 raft_consensus.cc:2749] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:40.263917 22844 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:40.264629 25062 raft_consensus.cc:2804] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:40.265077 25062 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:40.265239 22844 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:40.273855 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:40.002676 25087 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:40.201138 25089 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:40.277019 23617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 466) took 1394 ms. Trace:
I20260709 13:08:40.277487 22994 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:40.277633 25089 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:40.277360 23617 rpcz_store.cc:276] 0709 13:08:38.883174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:40.152040 (+1268866us) service_pool.cc:224] Handling call
0709 13:08:40.152328 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:40.275310 (+122982us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:40.275339 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:40.275375 (+    36us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:40.275406 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:40.276833 (+  1427us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b5400671454. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:40.276869 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:40.276967 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":282380672}
I20260709 13:08:40.278752 25087 raft_consensus.cc:2804] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:08:40.278978 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:40.279240 25087 raft_consensus.cc:697] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 6 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:40.279608 22994 leader_election.cc:400] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 4. Current term is 5.
W20260709 13:08:39.888443 23616 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60044 (request call id 476) took 4313 ms (client timeout 1822 ms). Trace:
I20260709 13:08:40.202176 24890 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:40.191330 24455 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:40.280563 23622 raft_consensus.cc:1275] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 13:08:40.280313 25087 consensus_queue.cc:237] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:40.281872 23622 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 465) took 1723 ms. Trace:
I20260709 13:08:40.282070 25039 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:40.282508 25039 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:40.282356 23622 rpcz_store.cc:276] 0709 13:08:38.558627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:39.906670 (+1348043us) service_pool.cc:224] Handling call
0709 13:08:39.907048 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:40.280935 (+373887us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:40.281604 (+   669us) spinlock_profiling.cc:243] Waited 373 ms on lock 0x7b5400638dd4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:40.281803 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":858294656}
I20260709 13:08:40.284394 24455 consensus_queue.cc:1048] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:40.285189 24890 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:40.285666 23616 rpcz_store.cc:269] 0709 13:08:35.563918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:35.565265 (+  1347us) service_pool.cc:224] Handling call
0709 13:08:36.346613 (+781348us) spinlock_profiling.cc:243] Waited 1.25 ms on lock 0x7b58001c4d98. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0eee 0000000000375939 00007f613ad341e0 00007f613adfd0c4 00007f613adfd131 00007f613adab0a0 00007f613adaa984 00007f61343256d4 00007f6134325662
0709 13:08:37.483547 (+1136934us) spinlock_profiling.cc:243] Waited 2.36 ms on lock 0x7b58001c4d98. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0f8c 00007f6132cc0ef6 0000000000375939 00007f613ad341e0 00007f613adfd0c4 00007f613adfd131 00007f613adab0a0 00007f613adaa984 00007f61343256d4 00007f6134325662
0709 13:08:39.877673 (+2394126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8305152}
I20260709 13:08:40.020700 25124 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:40.001058 22875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53724 (request call id 315) took 1711 ms. Trace:
W20260709 13:08:39.981217 22872 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35370 (request call id 1934) took 2151 ms (client timeout 1966 ms). Trace:
I20260709 13:08:40.211287 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:40.211469 22948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 1481) took 1938 ms. Trace:
I20260709 13:08:40.213418 22953 raft_consensus.cc:1275] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:08:40.105710 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 187486us
W20260709 13:08:40.202013 23620 rpcz_store.cc:269] 0709 13:08:37.648935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:37.656528 (+  7593us) service_pool.cc:224] Handling call
0709 13:08:39.233248 (+1576720us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:40.215412 23027 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6.
I20260709 13:08:39.951864 25096 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:40.218072 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:40.295934 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:40.218616 24921 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:40.216702 25070 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:40.229589 23624 raft_consensus.cc:2376] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6. Re-sending same reply.
I20260709 13:08:40.230057 23621 raft_consensus.cc:2393] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 6: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:39.921931 23619 raft_consensus.cc:1275] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:08:40.167852 24889 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:40.231860 25079 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:40.228256 22877 raft_consensus.cc:2889] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:40.232962 25126 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:40.218029 25092 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:40.233685 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:40.295797 22948 rpcz_store.cc:276] 0709 13:08:38.272645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:39.911194 (+1638549us) service_pool.cc:224] Handling call
0709 13:08:40.210954 (+299760us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:40.211120 (+   166us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:40.211142 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:40.211192 (+    50us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:08:40.211223 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:40.211273 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:40.211393 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:40.296716 25125 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:40.318946 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:08:40.303969 22919 connection.cc:256] RPC call timeout handler was delayed by -1.0548032840000587s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1787s for a response
I20260709 13:08:40.164239 25019 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:40.265524 25062 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:40.298482 25083 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:40.331187 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:39.953215 24743 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:40.268426 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:40.269678 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:40.270257 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:40.332732 22918 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:40.336282 23022 raft_consensus.cc:2410] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:08:40.272939 22871 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 5.
I20260709 13:08:40.279717 23617 raft_consensus.cc:2393] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 5: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
W20260709 13:08:40.201989 22946 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52222 (request call id 296) took 1946 ms (client timeout 1798 ms). Trace:
I20260709 13:08:40.286837 25124 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:40.283003 25039 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:40.286810 22875 rpcz_store.cc:276] 0709 13:08:38.290021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:38.961628 (+671607us) service_pool.cc:224] Handling call
0709 13:08:38.975265 (+ 13637us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:39.891589 (+916324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:39.891620 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:39.891665 (+    45us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:39.891694 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:40.000719 (+109025us) spinlock_profiling.cc:243] Waited 916 ms on lock 0x7b54001bb054. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:40.000802 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:40.000972 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2106769664}
I20260709 13:08:40.292186 25086 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:40.295229 22953 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 302) took 1752 ms. Trace:
I20260709 13:08:40.341467 22843 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:40.320240 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:08:40.320562 22919 connection.cc:256] RPC call timeout handler was delayed by -1.0038754350000545s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20030000000000003s for a response
I20260709 13:08:40.307199 22947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 300) took 2051 ms. Trace:
I20260709 13:08:40.309528 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:40.309834 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:40.311762 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:40.313870 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:40.314258 22877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53724 (request call id 317) took 1958 ms. Trace:
I20260709 13:08:40.319008 25125 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:40.372481 23590 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:40.310324 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:40.304116 25112 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:40.310503 25114 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:40.313222 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:40.316073 25092 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:40.306716 22955 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 298) took 2051 ms. Trace:
I20260709 13:08:40.317004 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:08:40.320631 22872 rpcz_store.cc:269] 0709 13:08:37.757904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:37.758091 (+   187us) service_pool.cc:224] Handling call
0709 13:08:39.662617 (+1904526us) spinlock_profiling.cc:243] Waited 1.85 ms on lock 0x7b5800000c98. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0a22 00007f613bbd9c09 00007f613ad33ee0 00007f613adfd0ab 00007f613adfd131 00007f613adab0a0 00007f613adaa984 00007f61343256d4 00007f6134325662
0709 13:08:39.909127 (+246510us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4248320}
I20260709 13:08:40.299633 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:40.325711 22874 raft_consensus.cc:1275] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:40.308730 22951 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 297) took 2053 ms. Trace:
I20260709 13:08:40.328647 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:40.311249 23029 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 5.
I20260709 13:08:40.304366 25093 consensus_queue.cc:1048] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 13:08:40.161433 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:40.405068 22879 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
I20260709 13:08:40.405664 22879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35370 (request call id 1938) took 1166 ms. Trace:
I20260709 13:08:40.406143 22879 rpcz_store.cc:276] 0709 13:08:39.238845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:40.161203 (+922358us) service_pool.cc:224] Handling call
0709 13:08:40.405602 (+244399us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:40.407366 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:40.407404 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:40.315049 23619 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60044 (request call id 484) took 2621 ms. Trace:
I20260709 13:08:40.326138 22994 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:08:40.338052 22918 connection.cc:256] RPC call timeout handler was delayed by -2.1283340969999927s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16890000000000002s for a response
W20260709 13:08:40.414002 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 87939.1us
W20260709 13:08:40.414155 22918 leader_election.cc:336] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.643s (SENT)
I20260709 13:08:40.403365 24738 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:40.414562 22918 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:40.414733 22994 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:40.414901 25120 raft_consensus.cc:2804] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:40.415396 25120 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:08:40.415798 22918 connection.cc:256] RPC call timeout handler was delayed by -1.1048556469999085s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1865s for a response
I20260709 13:08:40.415951 22994 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:40.372336 22947 rpcz_store.cc:276] 0709 13:08:38.256058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:39.908552 (+1652494us) service_pool.cc:224] Handling call
0709 13:08:40.216291 (+307739us) spinlock_profiling.cc:243] Waited 43.2 ms on lock 0x7b5800043598. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0a22 00007f613bbd9c09 00007f613ad33ee0 00007f613adfd0ab 00007f613adfd131 00007f613adbefc0 00007f613ada9929 00007f613ada93d2 00007f6134320bd4
0709 13:08:40.216459 (+   168us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:40.216643 (+   184us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:08:40.217151 (+   508us) log.cc:844] Serialized 14 byte log entry
0709 13:08:40.306925 (+ 89774us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:40.306965 (+    40us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:40.307007 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:40.307064 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:40.307153 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58434,"prepare.queue_time_us":132,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":25,"raft.queue_time_us":59581,"raft.run_cpu_time_us":45,"raft.run_wall_time_us":45,"spinlock_wait_cycles":99364096,"thread_start_us":31047,"threads_started":1,"wal-append.queue_time_us":26756}
I20260709 13:08:40.416144 25065 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:40.377278 24954 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:40.416772 25065 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:40.416774 25096 raft_consensus.cc:2749] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:40.417318 25065 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:40.417271 24954 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:40.417811 24571 consensus_queue.cc:1048] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:40.419406 24571 raft_consensus.cc:2804] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:40.419672 24954 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:40.419770 24571 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:40.420208 24571 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:40.355415 22953 rpcz_store.cc:276] 0709 13:08:38.543798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:40.192495 (+1648697us) service_pool.cc:224] Handling call
0709 13:08:40.212955 (+ 20460us) spinlock_profiling.cc:243] Waited 19.3 ms on lock 0x7b5800043598. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0a22 00007f613bbd9c09 00007f613ad33ee0 00007f613adfd0ab 00007f613adfd131 00007f613adbefc0 00007f613ada9929 00007f613ada93d2 00007f6134320bd4
0709 13:08:40.213121 (+   166us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:40.295001 (+ 81880us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:40.295182 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":44324224}
I20260709 13:08:40.375684 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:40.375902 22949 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
I20260709 13:08:40.421213 25134 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:40.337231 22946 rpcz_store.cc:269] 0709 13:08:38.255135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:38.272507 (+ 17372us) service_pool.cc:224] Handling call
0709 13:08:40.201800 (+1929293us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400148554. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:40.201923 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2407509888}
I20260709 13:08:40.378118 23025 raft_consensus.cc:1240] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:08:40.375151 22948 raft_consensus.cc:2393] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 5: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:40.399778 22951 rpcz_store.cc:276] 0709 13:08:38.255389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:39.900985 (+1645596us) service_pool.cc:224] Handling call
0709 13:08:40.214171 (+313186us) spinlock_profiling.cc:243] Waited 43.5 ms on lock 0x7b5800043598. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0a22 00007f613bbd9c09 00007f613ad33ee0 00007f613adfd0ab 00007f613adfd131 00007f613adbefc0 00007f613ada9929 00007f613ada93d2 00007f6134320bd4
0709 13:08:40.214331 (+   160us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:40.214501 (+   170us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:08:40.214808 (+   307us) log.cc:844] Serialized 14 byte log entry
0709 13:08:40.308238 (+ 93430us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:40.308280 (+    42us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:40.308318 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:40.308562 (+   244us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:40.308684 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":60299,"prepare.queue_time_us":182,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":61358,"raft.run_cpu_time_us":65,"raft.run_wall_time_us":64,"spinlock_wait_cycles":100082560,"thread_start_us":31425,"threads_started":1,"wal-append.queue_time_us":26204}
I20260709 13:08:40.378849 22843 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:40.331777 22873 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 5.
I20260709 13:08:40.408337 22879 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 3.
W20260709 13:08:40.423305 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 44518.6us
I20260709 13:08:40.375401 22877 rpcz_store.cc:276] 0709 13:08:38.355482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:40.226176 (+1870694us) service_pool.cc:224] Handling call
0709 13:08:40.226428 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:40.314029 (+ 87601us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:40.314209 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:40.412374 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:40.425046 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:40.356266 25113 raft_consensus.cc:2804] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:40.331457 25083 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:40.415321 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:40.343894 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:40.305683 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:40.412326 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:40.431846 22871 raft_consensus.cc:2376] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 5. Re-sending same reply.
I20260709 13:08:40.404476 25076 raft_consensus.cc:2804] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:40.422665 22994 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:40.445591 22994 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:40.445988 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:40.425693 22952 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 5.
I20260709 13:08:40.426671 22950 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 1.
I20260709 13:08:40.420439 25090 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:40.415970 25120 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:40.356010 24935 consensus_queue.cc:1048] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.079s
I20260709 13:08:40.439254 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:40.424333 24954 raft_consensus.cc:2749] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:08:40.404515 25112 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:40.482496 25076 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:40.482323 23019 tablet_service.cc:1558] Processing DeleteTablet for tablet 5ab4700268fc4c67b9930087c5da71c8 with delete_type TABLET_DATA_TOMBSTONED (TS f3c799b1249e40119d2a5864231c6c75 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:33406
I20260709 13:08:40.483163 24943 raft_consensus.cc:2804] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:40.483989 23620 raft_consensus.cc:2368] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 5. Current term is 6.
I20260709 13:08:40.421752 25134 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:40.484898 24921 raft_consensus.cc:2804] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:40.401228 22955 rpcz_store.cc:276] 0709 13:08:38.255604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:39.906655 (+1651051us) service_pool.cc:224] Handling call
0709 13:08:39.906943 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:39.907270 (+   327us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:08:40.200423 (+293153us) log.cc:844] Serialized 14 byte log entry
0709 13:08:40.306324 (+105901us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:40.306367 (+    43us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:40.306508 (+   141us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:40.306565 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:40.306670 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":81264,"prepare.queue_time_us":292953,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":45,"raft.queue_time_us":81399,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":81,"thread_start_us":23973,"threads_started":1,"wal-append.queue_time_us":10517}
W20260709 13:08:40.497967 22918 leader_election.cc:336] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.606s (SENT)
I20260709 13:08:40.414746 24738 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:40.428959 24743 raft_consensus.cc:2804] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:40.489655 23622 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 5.
I20260709 13:08:40.377144 25086 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:40.429482 25113 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:40.510480 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:40.435899 25123 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:40.482352 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:08:40.504340 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 130926us
I20260709 13:08:40.314507 25126 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:40.411881 25114 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:40.515180 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:40.661016 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:40.662024 22875 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6.
W20260709 13:08:40.662665 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 217151us
I20260709 13:08:40.662986 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:40.663388 22918 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:40.663828 22875 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 5.
I20260709 13:08:40.664788 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:40.665092 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:40.520380 25112 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:40.687271 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:40.665400 22994 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:08:40.698129 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 200206us
I20260709 13:08:40.698459 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:40.666179 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:40.700264 23029 raft_consensus.cc:2410] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:08:40.702634 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:40.704063 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:40.705178 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:40.666234 24921 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:40.667284 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:40.668684 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:40.669216 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:40.669289 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:40.708297 22946 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 3.
I20260709 13:08:40.708495 22948 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
I20260709 13:08:40.510445 25076 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:08:40.636812 25152 log.cc:927] Time spent T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 13:08:40.505527 25070 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:40.702694 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:40.506462 25090 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:40.705890 23022 raft_consensus.cc:1240] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:08:40.706025 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:40.706617 22951 raft_consensus.cc:2468] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:08:40.707571 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:40.670737 24743 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:40.726271 23028 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
I20260709 13:08:40.708201 25096 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:40.711180 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:40.733283 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:40.510751 24943 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:40.711180 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:40.665714 24738 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:40.680888 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 307335us
I20260709 13:08:40.736850 25125 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:40.736979 23590 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:40.670814 25107 consensus_queue.cc:1048] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.370s
I20260709 13:08:40.711145 25151 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:40.714071 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:40.714902 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:40.740245 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:40.740557 22953 raft_consensus.cc:2410] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:08:40.670603 25093 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:40.717006 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:40.742302 22946 raft_consensus.cc:2376] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6. Re-sending same reply.
I20260709 13:08:40.743696 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:40.744062 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:40.717733 22879 raft_consensus.cc:2468] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:40.413508 23619 rpcz_store.cc:276] 0709 13:08:37.693984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:37.704038 (+ 10054us) service_pool.cc:224] Handling call
0709 13:08:39.661424 (+1957386us) spinlock_profiling.cc:243] Waited 42.1 ms on lock 0x7b58001c4d98. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0eee 0000000000375939 00007f613ad341e0 00007f613adfd0c4 00007f613adfd131 00007f613adbefc0 00007f613ada9929 00007f613ada93d2 00007f6134320bd4
0709 13:08:39.878043 (+216619us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:40.309970 (+431927us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:40.314845 (+  4875us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b54004943d0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:40.314973 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":554292864}
I20260709 13:08:40.728744 25086 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:40.730649 25126 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:40.731788 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:40.713598 24921 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:40.712538 22875 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:40.751034 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:40.670708 25113 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:40.637373 25062 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:40.710775 25133 raft_consensus.cc:2749] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:40.740465 25124 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:40.718196 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:40.727311 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:40.694054 25123 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:40.746117 22951 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 3.
I20260709 13:08:40.747035 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:40.726920 24743 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:40.744701 24890 consensus_queue.cc:1048] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:08:40.733356 25096 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:40.738212 24943 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:40.669514 25134 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:40.761355 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:40.829909 25062 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:40.765102 25153 log.cc:927] Time spent T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 13:08:40.765988 25114 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:40.767371 25019 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:40.767690 25090 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:40.734599 25065 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:40.835812 25120 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:40.744869 25151 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:40.773136 22948 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 5.
I20260709 13:08:40.782774 22875 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
I20260709 13:08:40.837605 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:40.838014 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:40.838287 22876 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 5.
I20260709 13:08:40.760644 22920 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:08:40.839313 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 78712.9us
I20260709 13:08:40.839598 22952 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 5.
I20260709 13:08:40.840099 22920 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:40.744333 25093 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:40.841259 22920 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:40.789284 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:40.792393 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:40.787263 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:40.813414 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:40.842392 22873 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 1.
I20260709 13:08:40.809217 23621 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:08:40.824508 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 393562us
I20260709 13:08:40.826609 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:40.826823 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:40.854887 22872 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6.
I20260709 13:08:40.855055 22877 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 1.
I20260709 13:08:40.762113 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:40.856211 22879 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:40.841322 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:40.817795 24889 raft_consensus.cc:2749] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:40.857220 22920 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:40.857275 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:40.850553 25114 raft_consensus.cc:2764] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:08:40.859855 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:40.866840 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:40.868340 22950 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:58320
I20260709 13:08:40.869207 22950 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:08:40.869639 22950 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:40.864147 22843 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:40.873162 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:40.875524 22843 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:40.878060 23021 raft_consensus.cc:2393] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 4: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:40.790011 25083 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:40.879316 23022 raft_consensus.cc:2368] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 6. Current term is 7.
I20260709 13:08:40.879196 22843 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:40.892575 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:40.895759 22920 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:40.881654 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:40.882304 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:40.897437 22920 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:40.884471 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:40.885839 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:40.887533 24571 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:40.891393 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:40.818346 25096 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:40.899760 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:40.900381 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:40.901249 22843 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:40.860453 25082 raft_consensus.cc:2804] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:40.860286 25090 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:08:40.916285 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 204917us
I20260709 13:08:40.903338 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:40.905030 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:40.905825 25093 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:40.910490 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:40.911502 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:40.871871 25065 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:40.915261 22843 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:40.973762 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:40.893112 25075 raft_consensus.cc:2749] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:40.975481 22947 raft_consensus.cc:2393] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 6: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:40.893947 25025 raft_consensus.cc:2749] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:08:40.977676 23022 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
W20260709 13:08:40.978111 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 62948.2us
I20260709 13:08:40.931929 22920 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:41.003424 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:40.973294 25090 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:40.975279 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:40.930635 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:41.014786 22953 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 4.
I20260709 13:08:40.977824 25114 raft_consensus.cc:2804] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:40.980971 23027 raft_consensus.cc:2393] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 5: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:40.902959 25125 raft_consensus.cc:2804] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:08:40.918501 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:40.902774 25134 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:40.902336 25088 raft_consensus.cc:2804] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:41.023429 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:41.024480 22955 raft_consensus.cc:2393] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 4: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:41.026032 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.026413 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:40.917501 25082 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:40.919297 22844 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:40.972298 25050 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:41.005128 22843 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:40.919297 22950 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:41.012043 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 80172.6us
I20260709 13:08:40.980180 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:40.972882 25083 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:41.019625 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:40.985154 25086 raft_consensus.cc:2804] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:40.904304 25080 consensus_queue.cc:1048] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.082s
I20260709 13:08:40.984138 23588 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:40.975638 25157 consensus_queue.cc:1048] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:08:40.984642 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.027653 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:41.096276 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:41.259353 22948 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:08:41.183560 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:41.260124 22920 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:41.260150 22949 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:41.119278 23623 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 468) took 1888 ms. Trace:
I20260709 13:08:41.092331 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.083927 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.261317 23020 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
I20260709 13:08:41.100409 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:08:41.100991 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:08:41.264060 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:41.073014 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.263140 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:08:41.263660 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 279549us
W20260709 13:08:41.264528 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 345273us
I20260709 13:08:41.265528 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.266271 23028 raft_consensus.cc:2410] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:08:41.049515 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.050414 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:08:41.138962 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 401999us
I20260709 13:08:41.167054 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.261014 23623 rpcz_store.cc:276] 0709 13:08:39.230512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:40.234552 (+1004040us) service_pool.cc:224] Handling call
0709 13:08:40.234799 (+   247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:40.708279 (+473480us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:08:40.708532 (+   253us) log.cc:844] Serialized 14 byte log entry
0709 13:08:40.982246 (+273714us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:40.982304 (+    58us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:40.982346 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:41.118804 (+136458us) spinlock_profiling.cc:243] Waited 473 ms on lock 0x7b54005aebd4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:41.118868 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:41.119072 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5471,"prepare.queue_time_us":511,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":2619,"raft.queue_time_us":6380,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":18788,"spinlock_wait_cycles":1088353536,"thread_start_us":267758,"threads_started":1,"wal-append.queue_time_us":397904}
I20260709 13:08:41.027275 23022 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
I20260709 13:08:41.270627 22920 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:41.271270 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.272136 23590 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:41.272640 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.280200 23590 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:41.280846 23588 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:41.174986 22953 consensus_queue.cc:237] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:41.210688 25158 tablet_replica.cc:333] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
W20260709 13:08:41.251125 25145 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.088s	sys 0.123s
I20260709 13:08:41.285981 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.286299 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.291229 23026 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
I20260709 13:08:41.286970 23590 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:41.262152 23024 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 5.
I20260709 13:08:41.293411 23590 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:41.018280 25090 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:41.288285 22920 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:41.289853 22950 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:41.045955 25039 consensus_meta.cc:287] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.643s	user 0.009s	sys 0.000s
I20260709 13:08:41.260130 25082 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:41.023564 25134 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:41.302102 25134 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:41.302245 25089 raft_consensus.cc:2749] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:41.303017 25089 raft_consensus.cc:2804] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:41.303383 25089 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:41.303750 25089 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:41.023826 25088 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:41.304911 25088 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:41.039778 25125 raft_consensus.cc:697] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:41.316449 23588 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:08:41.317301 25075 raft_consensus.cc:2804] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:41.317814 25075 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:41.317206 25125 consensus_queue.cc:237] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:41.318168 25075 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:41.292805 24954 raft_consensus.cc:2804] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:41.319180 24954 raft_consensus.cc:697] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:41.319977 24954 consensus_queue.cc:237] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:41.321171 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:41.321805 23621 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
I20260709 13:08:41.322319 23621 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60044 (request call id 495) took 1019 ms. Trace:
I20260709 13:08:41.322662 25079 raft_consensus.cc:2764] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:08:41.323045 25122 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:41.323535 25122 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:41.323926 25122 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:41.323766 24743 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:41.326133 25123 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:41.269047 25062 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:41.327189 25062 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:41.327709 25062 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:41.327957 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:41.328822 25126 consensus_queue.cc:1048] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:08:41.329416 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:41.260046 24935 raft_consensus.cc:2764] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 13:08:41.291013 25133 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:41.313298 25112 raft_consensus.cc:2804] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:08:41.298323 25145 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.089s	sys 0.123s
I20260709 13:08:41.321238 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:41.266564 25050 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:41.351392 25050 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:41.322621 23621 rpcz_store.cc:276] 0709 13:08:40.303295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:40.701011 (+397716us) service_pool.cc:224] Handling call
0709 13:08:41.322261 (+621250us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:41.327119 23588 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:41.356866 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:41.357860 24889 raft_consensus.cc:2804] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:41.358343 24889 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:41.358453 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.359356 23027 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 5.
I20260709 13:08:41.359939 25158 raft_consensus.cc:2243] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:08:41.360845 25158 raft_consensus.cc:2272] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:08:41.045614 24943 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:41.361049 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.362588 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.363341 24943 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:41.043139 25113 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:41.047411 25076 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:41.364086 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.299239 22919 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:41.300539 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.022425 25114 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:41.295866 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:41.279548 25159 raft_consensus.cc:2804] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:08:41.089923 25151 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:41.298689 24890 raft_consensus.cc:2804] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:08:41.299103 25107 raft_consensus.cc:2749] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:08:41.343219 22949 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:08:41.345745 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.082566 25120 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:41.270159 25086 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:41.347081 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.358784 24889 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:41.330879 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:41.365068 23021 raft_consensus.cc:2393] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 3: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:41.197044 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:41.360538 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:41.420295 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.037904 24571 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:41.296800 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.325768 24743 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:41.364526 25158 ts_tablet_manager.cc:1916] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:08:41.036989 24921 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:41.372414 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:41.373409 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:41.379499 22954 raft_consensus.cc:1240] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:41.381440 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:41.348031 25133 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:41.384459 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:41.460368 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:41.385567 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.348428 25112 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:08:41.387528 23589 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.776s	user 0.015s	sys 0.019s
W20260709 13:08:41.463265 23589 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.852s	user 0.015s	sys 0.019s
I20260709 13:08:41.388366 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:08:41.390623 22871 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.013s	sys 0.021s
W20260709 13:08:41.465183 22871 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.013s	sys 0.021s
I20260709 13:08:41.390481 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:08:41.391669 22995 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.007s	sys 0.023s
W20260709 13:08:41.467353 22995 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.007s	sys 0.023s
I20260709 13:08:41.468429 25076 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:41.468509 22950 raft_consensus.cc:1275] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:08:41.468597 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:41.468852 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:41.392328 23624 raft_consensus.cc:1240] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 6. Current term is 7. Ops: []
I20260709 13:08:41.341980 25039 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:41.425949 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:41.470747 23023 raft_consensus.cc:2376] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6. Re-sending same reply.
I20260709 13:08:41.473634 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:41.438301 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:41.381246 25114 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:41.450102 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:41.391176 25086 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:08:41.458935 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 193242us
W20260709 13:08:41.460424 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 455322us
I20260709 13:08:41.460984 23025 raft_consensus.cc:2376] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 5. Re-sending same reply.
I20260709 13:08:41.279875 25083 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:41.461148 22878 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
I20260709 13:08:41.474946 23623 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6.
I20260709 13:08:41.497874 23026 raft_consensus.cc:1240] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:41.487684 22843 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:41.491845 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:41.499689 22877 raft_consensus.cc:2393] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 7: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:41.505300 22878 raft_consensus.cc:1240] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:41.508153 22948 raft_consensus.cc:1275] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:08:41.509455 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:41.468624 25112 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:41.464847 25133 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:41.519271 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.508376 22843 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:41.520287 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.521061 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.521981 23028 raft_consensus.cc:2393] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 6: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:41.522650 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.523625 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:41.524107 23028 raft_consensus.cc:3055] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:08:41.524375 23028 raft_consensus.cc:740] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:41.512635 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:41.528977 22877 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:41.517644 22880 raft_consensus.cc:2393] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 5: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:41.517937 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.530476 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.531412 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.541008 25089 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:41.543298 25089 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:41.543853 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.440209 25151 consensus_queue.cc:1048] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.063s
I20260709 13:08:41.546494 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:41.551162 23028 consensus_queue.cc:260] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:41.552397 23028 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:41.549557 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.546723 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.554162 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.555002 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.561496 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.564210 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.565578 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:41.566201 23023 raft_consensus.cc:3055] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:41.566534 23023 raft_consensus.cc:740] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:41.507478 24743 raft_consensus.cc:2749] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:41.563505 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:41.576058 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:08:41.570574 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 271379us
I20260709 13:08:41.570807 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.572731 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.577543 23026 raft_consensus.cc:2393] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 7: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:41.577751 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.578799 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.579188 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.580310 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:41.581862 22878 raft_consensus.cc:2393] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 5: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:41.583451 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.583611 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.584182 23021 raft_consensus.cc:2368] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 5. Current term is 6.
I20260709 13:08:41.584532 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.585855 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.586889 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.582934 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:41.593194 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:08:41.593333 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 305097us
I20260709 13:08:41.594225 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.595099 23026 raft_consensus.cc:2393] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 6: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:41.595870 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.596899 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:41.598555 22920 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:41.548983 25070 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:41.598887 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:41.621495 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:41.621692 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:41.599820 23022 raft_consensus.cc:1240] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:08:41.599526 22920 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:41.623675 23022 raft_consensus.cc:1240] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:41.602367 22919 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:41.625294 22919 leader_election.cc:400] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 election: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 5. Current term is 6.
I20260709 13:08:41.625563 22919 leader_election.cc:403] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:08:41.630105 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:41.632974 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:41.633723 22949 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:41.595620 23023 consensus_queue.cc:260] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:41.639674 23023 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:41.568688 23616 raft_consensus.cc:1240] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 5. Current term is 6. Ops: []
W20260709 13:08:41.642505 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 270752us
I20260709 13:08:41.655364 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:41.660772 24935 raft_consensus.cc:2749] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:41.661692 24935 raft_consensus.cc:2804] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:41.662086 24935 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:41.662525 24935 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:41.669423 25080 raft_consensus.cc:2804] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:41.669976 25080 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:41.670425 25080 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:08:41.651091 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 152766us
I20260709 13:08:41.554400 25082 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:41.723845 22949 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 5.
I20260709 13:08:41.724071 23023 raft_consensus.cc:2468] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
W20260709 13:08:41.730981 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 181579us
I20260709 13:08:41.644646 24743 raft_consensus.cc:2804] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:41.733610 23028 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 5.
I20260709 13:08:41.649308 25070 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:41.708931 24890 raft_consensus.cc:697] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 5 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:41.737488 24743 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:41.742971 25151 raft_consensus.cc:2749] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:41.750810 24738 raft_consensus.cc:2749] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 5. Current term is 6.
I20260709 13:08:42.088709 25165 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:42.090106 25165 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:41.898550 25080 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:42.024986 22871 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.009s	sys 0.000s
I20260709 13:08:42.025600 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:42.024472 22878 raft_consensus.cc:1240] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 4. Current term is 5. Ops: []
W20260709 13:08:42.027333 22879 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.005s	sys 0.008s
W20260709 13:08:42.029430 22880 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53724 (request call id 324) took 1818 ms (client timeout 1939 ms). Trace:
I20260709 13:08:42.034981 22947 raft_consensus.cc:1240] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 LEADER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 4. Current term is 5. Ops: []
W20260709 13:08:42.039073 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:42.040251 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 297217us
W20260709 13:08:42.040756 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:42.052290 23024 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.002s	sys 0.002s
I20260709 13:08:41.796538 23623 raft_consensus.cc:1275] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:08:41.815858 24743 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:41.848135 24890 consensus_queue.cc:237] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:41.904304 24921 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:41.911636 24935 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:41.871542 25039 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:41.913844 25050 consensus_meta.cc:287] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.562s	user 0.008s	sys 0.000s
I20260709 13:08:41.744997 25062 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:41.888942 25070 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:41.892278 25075 consensus_meta.cc:287] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.574s	user 0.004s	sys 0.004s
I20260709 13:08:41.889991 25082 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:41.877059 25083 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:42.138127 25080 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:42.139220 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:08:42.139493 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 428471us
I20260709 13:08:41.913924 25086 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:42.144034 25086 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:41.848954 25088 consensus_meta.cc:287] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.543s	user 0.003s	sys 0.006s
I20260709 13:08:42.145588 25088 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:42.146422 22918 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:42.147883 25062 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:42.149219 22873 raft_consensus.cc:1240] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 3. Current term is 4. Ops: []
W20260709 13:08:41.893568 25090 consensus_meta.cc:287] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.544s	user 0.003s	sys 0.004s
I20260709 13:08:42.155910 22875 raft_consensus.cc:2393] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 5: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:42.138202 25075 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:42.156147 25090 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:42.157148 22872 raft_consensus.cc:1240] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:42.158578 25075 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:42.159020 25090 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:42.145588 25050 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:42.162690 25050 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:42.163499 22947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2052) took 1157 ms. Trace:
W20260709 13:08:41.891292 25096 consensus_meta.cc:287] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.918s	user 0.009s	sys 0.003s
I20260709 13:08:42.165383 25096 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:42.167093 23029 raft_consensus.cc:2368] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 5. Current term is 6.
I20260709 13:08:42.169277 22955 raft_consensus.cc:2393] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 6: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
W20260709 13:08:42.169910 23029 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33418 (request call id 1413) took 1468 ms (client timeout 1532 ms). Trace:
W20260709 13:08:42.170507 23029 rpcz_store.cc:269] 0709 13:08:40.701859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:40.702329 (+   470us) service_pool.cc:224] Handling call
0709 13:08:42.169484 (+1467155us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b5400358754. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:42.169759 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3364289792}
I20260709 13:08:41.917833 25112 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:42.172015 22992 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:42.174958 25112 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:41.739645 25113 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:42.175706 25187 consensus_queue.cc:1048] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:08:41.851915 25120 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:42.205577 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:42.206120 25120 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:42.207454 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:42.208336 22872 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:41.942736 25133 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:42.215464 23588 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:41.905900 25134 consensus_meta.cc:287] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.603s	user 0.006s	sys 0.004s
I20260709 13:08:42.217644 25158 ts_tablet_manager.cc:1929] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
W20260709 13:08:41.979415 25159 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.037s	sys 0.057s
I20260709 13:08:41.684789 25164 consensus_queue.cc:1059] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.066s
W20260709 13:08:42.218802 25159 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.769s	user 0.037s	sys 0.057s
I20260709 13:08:42.219226 25159 raft_consensus.cc:697] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 6 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:42.219885 25164 raft_consensus.cc:3055] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:08:42.111300 23024 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.686s	user 0.002s	sys 0.003s
I20260709 13:08:42.113977 25165 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:42.114233 22871 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.009s	sys 0.001s
I20260709 13:08:42.223254 25133 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:42.112110 25168 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:42.105998 22880 rpcz_store.cc:269] 0709 13:08:40.210382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:40.211052 (+   670us) service_pool.cc:224] Handling call
0709 13:08:42.028889 (+1817837us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b540030cad4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:42.029199 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2811949952}
W20260709 13:08:42.129451 22879 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.006s	sys 0.009s
I20260709 13:08:42.132381 22877 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:08:42.148370 25088 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:41.919523 25114 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:42.239023 25120 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:42.164021 22947 rpcz_store.cc:276] 0709 13:08:41.006393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:41.007240 (+   847us) service_pool.cc:224] Handling call
0709 13:08:41.011378 (+  4138us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:42.136462 (+1125084us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:42.163046 (+ 26584us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b5400285354. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:42.163333 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2352332672}
I20260709 13:08:42.192409 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:42.195248 25080 raft_consensus.cc:2749] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:42.187747 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:42.241657 25114 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:42.188887 25182 consensus_queue.cc:1059] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:08:42.180298 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:42.188180 25197 consensus_queue.cc:1059] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:08:42.181641 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:42.179942 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:42.243090 25182 raft_consensus.cc:3055] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:08:42.243597 25182 raft_consensus.cc:740] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:42.168303 25096 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:42.128118 24935 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:42.204123 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:08:41.911580 25122 consensus_meta.cc:287] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.587s	user 0.007s	sys 0.001s
W20260709 13:08:42.260133 22919 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.004s	sys 0.027s
W20260709 13:08:42.260690 22919 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.004s	sys 0.027s
W20260709 13:08:42.261242 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 636012us
I20260709 13:08:42.209163 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:42.217412 25134 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:42.112232 22994 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:08:42.219988 23588 leader_election.cc:336] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.553s (SENT)
I20260709 13:08:42.231401 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:42.220471 25164 raft_consensus.cc:740] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:42.236395 22871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 1458) took 1488 ms. Trace:
I20260709 13:08:42.238539 25168 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:42.175930 25188 raft_consensus.cc:2749] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:42.242880 23624 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
I20260709 13:08:42.177192 25070 consensus_queue.cc:1059] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:08:42.243103 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:42.226027 25158 log.cc:1199] T 5ab4700268fc4c67b9930087c5da71c8 P f3c799b1249e40119d2a5864231c6c75: Deleting WAL directory at /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-2-root/wals/5ab4700268fc4c67b9930087c5da71c8
I20260709 13:08:42.255234 25182 consensus_queue.cc:260] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:42.258667 25114 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:08:42.260120 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 113896us
I20260709 13:08:42.220556 25159 consensus_queue.cc:237] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:42.265077 22871 rpcz_store.cc:276] 0709 13:08:40.748025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:40.748134 (+   109us) service_pool.cc:224] Handling call
0709 13:08:40.758207 (+ 10073us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:40.758404 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:08:41.465517 (+707113us) log.cc:844] Serialized 14 byte log entry
0709 13:08:42.225533 (+760016us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:42.225653 (+   120us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:08:42.228330 (+  2677us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:42.235336 (+  7006us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:42.236302 (+   966us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":834062,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":1845,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":10525,"thread_start_us":1426747,"threads_started":2,"wal-append.queue_time_us":401004}
I20260709 13:08:42.269765 22918 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:42.269876 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:42.270924 22875 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 2.
I20260709 13:08:42.272037 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:42.272891 25080 raft_consensus.cc:2749] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:42.274219 22878 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:08:42.277465 24889 consensus_meta.cc:287] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.831s	user 0.000s	sys 0.017s
I20260709 13:08:42.278450 22774 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33466 (request call id 288) took 10388 ms. Trace:
I20260709 13:08:42.278196 24889 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:42.278878 25182 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:42.280047 22993 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:42.280969 22918 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:42.281620 25202 log.cc:927] Time spent T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:08:42.282846 25181 raft_consensus.cc:3055] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:42.283109 25080 raft_consensus.cc:2749] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:08:42.284370 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 112576us
I20260709 13:08:42.283082 22918 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:42.277400 22919 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:42.250057 22843 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.009s	sys 0.011s
I20260709 13:08:42.269950 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:42.281654 22775 catalog_manager.cc:5697] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 3 to 4, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 4 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:42.288514 22875 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
W20260709 13:08:42.289736 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 69796.8us
I20260709 13:08:42.290201 24889 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:42.291844 25080 raft_consensus.cc:2749] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:42.292101 25168 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:42.292893 22880 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:42.293006 22871 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
I20260709 13:08:42.294204 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:42.294903 22871 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:08:42.288136 22843 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.765s	user 0.009s	sys 0.012s
I20260709 13:08:42.260880 25122 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:42.304380 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:42.312386 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:42.313640 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:42.314955 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:42.305644 25158 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:33406 (request call id 77) took 1963 ms. Trace:
I20260709 13:08:42.306301 23621 raft_consensus.cc:1240] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:42.307266 25114 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:42.284212 25164 consensus_queue.cc:260] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:42.330945 22877 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:08:42.285769 25181 raft_consensus.cc:740] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
W20260709 13:08:42.342156 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 52630us
I20260709 13:08:42.306398 22775 catalog_manager.cc:5697] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 1 to 4, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 4 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:42.342370 22872 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
I20260709 13:08:42.343320 22880 raft_consensus.cc:2410] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:08:42.343282 22871 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:08:42.344496 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:42.345456 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:42.346168 22871 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:42.346242 22880 raft_consensus.cc:1275] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:08:42.347066 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:42.347828 22876 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:08:42.347939 23618 raft_consensus.cc:1240] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:08:42.350291 25114 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:42.351451 22843 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:42.352757 24935 raft_consensus.cc:2749] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:08:42.352852 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 73000.3us
I20260709 13:08:42.353677 25087 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:42.354183 25087 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:42.354635 25087 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:42.354975 25181 consensus_queue.cc:260] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:42.357009 25088 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:42.356587 22775 catalog_manager.cc:5697] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 3 to 4, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 4 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:08:42.357723 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 277207us
I20260709 13:08:42.357879 25088 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:42.358734 22872 raft_consensus.cc:1240] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:42.359241 25208 raft_consensus.cc:2749] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:42.360177 25088 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:42.361821 22874 raft_consensus.cc:1240] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:42.362354 22876 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6.
I20260709 13:08:42.340400 25165 raft_consensus.cc:3055] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:08:42.363705 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 86447.7us
I20260709 13:08:42.363436 22874 raft_consensus.cc:1275] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:08:42.360893 25050 consensus_queue.cc:1059] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 13:08:42.358673 22994 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:42.364046 22775 catalog_manager.cc:5697] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 2 to 3, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 3 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:42.381752 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:42.383268 22875 raft_consensus.cc:1240] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 6. Current term is 7. Ops: []
I20260709 13:08:42.384311 22992 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:42.385115 22843 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:42.361028 25168 raft_consensus.cc:2749] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:42.390043 24743 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:42.391623 22873 raft_consensus.cc:2393] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 3: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:42.396872 22877 raft_consensus.cc:1240] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:42.398572 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:42.398965 22920 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:42.399564 23616 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
I20260709 13:08:42.400955 24935 raft_consensus.cc:2749] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:08:42.401206 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:42.402803 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:42.403455 23621 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:42.388494 25181 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:42.405829 25050 raft_consensus.cc:3055] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:08:42.406289 25050 raft_consensus.cc:740] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:42.407303 25050 consensus_queue.cc:260] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:42.363437 25165 raft_consensus.cc:740] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:42.360625 25134 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:42.413370 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:42.368719 22919 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:42.415868 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 47377.8us
I20260709 13:08:42.417126 22919 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:42.417356 25114 raft_consensus.cc:2749] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:42.418658 22919 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:42.419324 24935 raft_consensus.cc:2749] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:08:42.369153 25122 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:42.418399 25114 consensus_queue.cc:1048] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:08:42.420917 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 62330.5us
I20260709 13:08:42.420995 25208 raft_consensus.cc:2749] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:42.422047 22994 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:42.423887 25187 raft_consensus.cc:2749] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:42.424021 22994 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:08:42.424281 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 50983.9us
I20260709 13:08:42.424701 25187 raft_consensus.cc:2749] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:42.425652 22843 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:42.425997 22878 raft_consensus.cc:2468] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
I20260709 13:08:42.425913 22992 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:42.427441 22878 raft_consensus.cc:1240] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 5. Current term is 6. Ops: []
I20260709 13:08:42.428174 25075 raft_consensus.cc:2749] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:42.428689 25088 raft_consensus.cc:2749] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:42.433588 22920 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:42.409929 25164 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:42.435253 24743 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:42.435986 22843 leader_election.cc:400] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 6. Current term is 7.
I20260709 13:08:42.438136 25176 raft_consensus.cc:2749] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:42.439723 22879 raft_consensus.cc:1240] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:42.442279 22875 raft_consensus.cc:1240] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:42.443651 22877 raft_consensus.cc:1275] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:08:42.278838 22774 rpcz_store.cc:276] 0709 13:08:31.889653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:31.889969 (+   316us) service_pool.cc:224] Handling call
0709 13:08:42.278198 (+10388229us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:40.235711 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:40.236451 (+   740us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:40.236500 (+    49us) write_op.cc:435] Acquired schema lock
0709 13:08:40.236526 (+    26us) tablet.cc:662] Decoding operations
0709 13:08:40.237255 (+   729us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:40.237348 (+    93us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:40.237387 (+    39us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:08:40.237794 (+   407us) tablet.cc:701] Row locks acquired
0709 13:08:40.237817 (+    23us) write_op.cc:260] PREPARE: finished
0709 13:08:40.237989 (+   172us) write_op.cc:270] Start()
0709 13:08:40.238123 (+   134us) write_op.cc:276] Timestamp: P: 1783602520237933 usec, L: 0
0709 13:08:40.238150 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 13:08:40.238968 (+   818us) log.cc:844] Serialized 1294 byte log entry
0709 13:08:40.731245 (+492277us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b18000c0be0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cdae07 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf5455 00007f6135d62190 00007f6135d5bae7 00007f6135d5add9 00007f6135d65930
0709 13:08:41.212216 (+480971us) op_driver.cc:464] REPLICATION: finished
0709 13:08:41.958914 (+746698us) write_op.cc:301] APPLY: starting
0709 13:08:41.959027 (+   113us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:41.959217 (+   190us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:41.959249 (+    32us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:41.959875 (+   626us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:41.960141 (+   266us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 13:08:41.960214 (+    73us) write_op.cc:312] APPLY: finished
0709 13:08:41.960496 (+   282us) log.cc:844] Serialized 35 byte log entry
0709 13:08:41.960733 (+   237us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:41.960860 (+   127us) write_op.cc:454] Released schema lock
0709 13:08:41.960917 (+    57us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:41.961373 (+   456us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:41.961412 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":94598,"reported_tablets":22,"child_traces":[["op",{"apply.queue_time_us":746409,"apply.run_cpu_time_us":2438,"apply.run_wall_time_us":2437,"num_ops":3,"prepare.queue_time_us":24872,"prepare.run_cpu_time_us":36164,"prepare.run_wall_time_us":85206,"raft.queue_time_us":868663,"raft.run_cpu_time_us":22451,"raft.run_wall_time_us":363416,"replication_time_us":973916,"spinlock_wait_cycles":27525888,"thread_start_us":1055579,"threads_started":4,"wal-append.queue_time_us":61568}]]}
I20260709 13:08:42.410900 22872 raft_consensus.cc:1240] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:42.408948 25050 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:42.455322 22843 leader_election.cc:403] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:42.460448 23024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 383) took 1070 ms. Trace:
I20260709 13:08:42.463585 25096 raft_consensus.cc:2804] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:42.465051 25187 raft_consensus.cc:2804] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:42.467762 25096 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:42.468133 25187 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:42.468427 25096 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:42.468940 25187 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:42.470417 22992 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:42.471365 25208 raft_consensus.cc:2749] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:42.450379 25080 raft_consensus.cc:2749] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:42.473201 22992 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:42.411201 25165 consensus_queue.cc:260] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:42.481122 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 62631.1us
W20260709 13:08:42.483140 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 200239us
I20260709 13:08:42.479219 25114 raft_consensus.cc:2804] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:42.485035 25088 raft_consensus.cc:2749] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 6. Current term is 7.
I20260709 13:08:42.489405 22919 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:42.487370 25114 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:08:42.492827 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 59346.4us
I20260709 13:08:42.464869 23024 rpcz_store.cc:276] 0709 13:08:41.386549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:41.386669 (+   120us) service_pool.cc:224] Handling call
0709 13:08:41.386926 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:41.387281 (+   355us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:08:42.230765 (+843484us) log.cc:844] Serialized 14 byte log entry
0709 13:08:42.438662 (+207897us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:42.438946 (+   284us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:42.439313 (+   367us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:42.439929 (+   616us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:42.457250 (+ 17321us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17081,"prepare.queue_time_us":550237,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":72832,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":163,"thread_start_us":909655,"threads_started":2,"wal-append.queue_time_us":220657}
I20260709 13:08:42.497578 22920 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:08:42.500293 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 56948us
W20260709 13:08:42.500998 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 77154us
I20260709 13:08:42.499615 25080 raft_consensus.cc:2749] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:42.495812 25208 raft_consensus.cc:2749] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:42.337035 25158 rpcz_store.cc:276] 0709 13:08:40.411833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:40.481980 (+ 70147us) service_pool.cc:224] Handling call
0709 13:08:41.210206 (+728226us) ts_tablet_manager.cc:1151] Deleting tablet 5ab4700268fc4c67b9930087c5da71c8
0709 13:08:41.210326 (+   120us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:08:41.917595 (+707269us) tablet_metadata.cc:665] Metadata flushed
0709 13:08:42.217352 (+299757us) tablet_metadata.cc:665] Metadata flushed
0709 13:08:42.305475 (+ 88123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":74618,"tablet-delete.queue_time_us":686379,"thread_start_us":376217,"threads_started":1}
W20260709 13:08:42.509274 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 74024.9us
I20260709 13:08:42.510303 22843 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:42.511723 25088 raft_consensus.cc:2749] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:42.515705 23621 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 6.
I20260709 13:08:42.516963 25080 raft_consensus.cc:2804] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:42.517439 25080 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:42.517845 25080 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:42.516521 22920 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:42.522532 24935 raft_consensus.cc:2804] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:42.496178 25114 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:42.511718 22843 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:42.517436 22919 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:42.526268 22920 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:42.516016 22918 leader_election.cc:400] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 5. Current term is 6.
I20260709 13:08:42.523164 24935 raft_consensus.cc:697] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:42.531423 22919 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:42.531492 24935 consensus_queue.cc:237] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:42.532449 25134 raft_consensus.cc:2749] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:42.535249 25134 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:08:42.535790 25134 raft_consensus.cc:697] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:42.536747 25134 consensus_queue.cc:237] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:42.528594 25088 raft_consensus.cc:2749] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:42.538925 22918 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:42.539559 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 66502.9us
I20260709 13:08:42.546100 25204 raft_consensus.cc:2749] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:42.534562 25087 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:42.546633 25188 consensus_queue.cc:1059] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:08:42.547020 25193 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:42.547910 25188 raft_consensus.cc:3055] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:08:42.548311 25188 raft_consensus.cc:740] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:42.549367 25188 consensus_queue.cc:260] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:42.551076 25188 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:42.551395 25087 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:42.557945 25193 raft_consensus.cc:2749] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:42.560726 25165 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:08:42.565507 25211 log.cc:927] Time spent T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 13:08:42.566669 22919 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:42.567623 25193 raft_consensus.cc:2804] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:42.568207 25193 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:42.533802 25187 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:42.568629 25193 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:08:42.574095 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:42.581396 22994 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:42.582442 25208 raft_consensus.cc:2804] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:42.582763 22994 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:42.583055 25208 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:42.583600 25070 raft_consensus.cc:2749] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:08:42.583642 25208 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:42.585094 22992 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:42.586022 25070 raft_consensus.cc:2749] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:08:42.586509 22992 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:42.587956 25124 raft_consensus.cc:2804] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:08:42.588784 25124 raft_consensus.cc:697] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:08:42.589599 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 48254us
I20260709 13:08:42.590255 22844 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:42.589896 24943 consensus_queue.cc:1048] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:42.590998 24743 consensus_queue.cc:1059] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:08:42.591558 22844 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:42.592247 24743 raft_consensus.cc:2804] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:42.592806 25075 raft_consensus.cc:3055] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:08:42.593282 25075 raft_consensus.cc:740] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:42.594228 25075 consensus_queue.cc:260] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:42.594588 25168 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:08:42.594919 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 62946.2us
I20260709 13:08:42.595498 25075 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:42.595528 25168 raft_consensus.cc:697] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 6 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:42.596047 22843 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:42.592585 24943 consensus_queue.cc:1048] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 13:08:42.596972 25086 raft_consensus.cc:2749] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:08:42.599383 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 54872.8us
I20260709 13:08:42.592615 24743 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:42.600448 24743 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:42.600076 25168 consensus_queue.cc:237] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:42.609315 22994 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:42.611321 25163 consensus_queue.cc:1059] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:08:42.610339 24943 raft_consensus.cc:2749] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:42.614043 25193 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:42.619427 22871 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:42.575255 25096 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:42.628545 25163 raft_consensus.cc:3055] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:08:42.629072 25163 raft_consensus.cc:740] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:42.629830 22876 raft_consensus.cc:1240] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:42.627595 25087 consensus_queue.cc:1059] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:08:42.630242 25163 consensus_queue.cc:260] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:42.630889 25087 raft_consensus.cc:3055] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:08:42.631454 25087 raft_consensus.cc:740] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:42.631748 25163 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:42.628872 25019 consensus_queue.cc:1059] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:42.636435 22994 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:42.643594 25019 raft_consensus.cc:3055] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:42.643959 22874 raft_consensus.cc:1240] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:08:42.644225 25019 raft_consensus.cc:740] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:42.645303 22843 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:42.645670 25019 consensus_queue.cc:260] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:42.647857 25019 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:42.648383 25075 raft_consensus.cc:2749] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:42.647879 22843 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:42.657711 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 119602us
I20260709 13:08:42.653982 25086 raft_consensus.cc:2749] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:42.615564 25124 consensus_queue.cc:237] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:42.668977 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 43593us
I20260709 13:08:42.632416 25087 consensus_queue.cc:260] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:42.694870 24921 raft_consensus.cc:2749] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:08:42.697147 22994 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:08:42.702049 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:42.688742 25187 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:08:42.705694 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 57958.4us
I20260709 13:08:42.703121 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:42.707768 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:42.709815 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:42.710664 22871 raft_consensus.cc:2393] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 6: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:42.713032 22843 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:42.687583 25080 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:42.718902 25087 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:42.719127 25114 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:42.721285 25114 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:42.724895 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 47276.2us
I20260709 13:08:42.725754 22918 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:42.727344 25134 raft_consensus.cc:2749] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:42.730232 25096 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:42.731658 22877 raft_consensus.cc:2393] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 5: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
W20260709 13:08:42.745731 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 58773.8us
I20260709 13:08:42.748292 22876 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6.
I20260709 13:08:42.743217 25070 raft_consensus.cc:2804] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:42.739034 25168 raft_consensus.cc:2749] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:42.752367 22843 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:42.727527 25204 consensus_queue.cc:1059] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 13:08:42.759351 25086 raft_consensus.cc:2804] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:42.750928 25070 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:42.762677 25086 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:42.762843 25070 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:42.763252 25086 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:42.766175 22762 catalog_manager.cc:5028] TS f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): tablet 5ab4700268fc4c67b9930087c5da71c8 (table test-workload [id=a42f2ccfbe484baca3019323a490c64a]) successfully deleted
I20260709 13:08:42.767417 23588 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:42.767902 22949 raft_consensus.cc:1240] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:42.768572 22947 raft_consensus.cc:1240] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:42.769387 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:42.769716 25181 raft_consensus.cc:2749] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:08:42.770067 22955 raft_consensus.cc:1240] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:42.770440 25181 consensus_queue.cc:1059] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:08:42.771440 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:42.771608 25070 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:42.772110 22952 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:42.772771 25122 raft_consensus.cc:3055] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:08:42.773228 25122 raft_consensus.cc:740] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:42.773566 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:42.770112 22954 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
I20260709 13:08:42.778868 22918 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:42.794373 22994 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:42.774304 25122 consensus_queue.cc:260] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:42.797756 25122 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:42.798563 22950 raft_consensus.cc:1240] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:42.777060 25070 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:42.756706 25208 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:42.805505 22948 raft_consensus.cc:1240] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:08:42.815451 22950 raft_consensus.cc:1240] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 LEADER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:42.757012 24935 consensus_queue.cc:1059] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 13:08:42.823324 22843 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:42.827771 22954 raft_consensus.cc:1240] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:08:42.827811 25019 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 13:08:42.831670 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 85088.1us
I20260709 13:08:42.813851 25193 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:42.835091 22994 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:08:42.836601 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:42.837809 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:42.837793 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:42.849116 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:42.850395 22952 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:42.835763 25019 raft_consensus.cc:697] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:42.862917 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:42.863715 22947 raft_consensus.cc:1240] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:08:42.865355 22948 raft_consensus.cc:1240] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 1. Current term is 2. Ops: []
I20260709 13:08:42.834422 25168 raft_consensus.cc:2749] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:42.872857 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:42.875844 25208 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:42.876768 22880 raft_consensus.cc:1275] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:08:42.877753 25220 log.cc:927] Time spent T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:08:42.877586 25086 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:42.872911 22955 raft_consensus.cc:1240] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 5. Current term is 6. Ops: []
I20260709 13:08:42.878309 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:42.879719 22949 raft_consensus.cc:1240] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:08:42.866816 24743 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:42.895210 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 71506.1us
I20260709 13:08:42.884383 25134 raft_consensus.cc:2749] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:42.883683 25164 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:42.901233 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:42.892037 25070 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:42.905090 22946 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 7.
W20260709 13:08:42.910303 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 96439.1us
I20260709 13:08:42.911343 22843 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:42.898855 23621 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:42.911597 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:42.921765 22948 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:42.924233 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:42.924736 25086 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:42.928815 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:42.929677 22951 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:42.931695 22955 raft_consensus.cc:1275] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:08:42.905256 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:42.941870 25176 consensus_queue.cc:1059] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:08:42.942348 23029 raft_consensus.cc:1240] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:42.942927 25176 raft_consensus.cc:3055] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:08:42.943115 22843 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:42.943310 25176 raft_consensus.cc:740] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:42.944525 22843 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:42.947086 22843 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:42.946743 25019 consensus_queue.cc:237] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:42.947624 25075 raft_consensus.cc:2749] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:42.948771 25088 raft_consensus.cc:2749] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:08:42.948390 22843 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:42.905318 25114 consensus_queue.cc:1059] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.035s
I20260709 13:08:42.955377 22947 raft_consensus.cc:1240] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:42.964393 25187 consensus_queue.cc:1059] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:08:42.964838 25208 consensus_queue.cc:1048] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:42.965569 25187 raft_consensus.cc:3055] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:08:42.965925 25187 raft_consensus.cc:740] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:42.961633 25088 raft_consensus.cc:2804] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:42.967114 25088 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:42.967507 25088 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:42.966785 25187 consensus_queue.cc:260] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:42.937191 25050 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:42.969918 25050 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:42.970463 25050 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:42.939478 25164 consensus_queue.cc:1059] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
I20260709 13:08:42.974202 25164 raft_consensus.cc:3055] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:42.948432 25075 raft_consensus.cc:2749] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:42.974692 25164 raft_consensus.cc:740] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:42.975212 25080 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:42.976706 25096 consensus_queue.cc:1048] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 13:08:42.980291 24743 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:42.981230 22952 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
W20260709 13:08:42.987279 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 46892.9us
I20260709 13:08:42.994925 22948 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 5.
I20260709 13:08:42.996837 22951 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
W20260709 13:08:42.996842 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 238271us
I20260709 13:08:42.921737 25070 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:43.000679 25182 consensus_queue.cc:1059] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:08:43.001758 25181 raft_consensus.cc:3055] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:08:43.002089 25181 raft_consensus.cc:740] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:43.002992 25181 consensus_queue.cc:260] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:43.004284 25181 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:43.005260 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:42.968608 25187 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:43.012549 23027 raft_consensus.cc:1275] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:08:43.014190 23028 raft_consensus.cc:1275] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:08:43.014498 22992 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:43.015321 25165 raft_consensus.cc:2804] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:43.015614 23023 raft_consensus.cc:1240] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:08:43.015458 23590 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:43.016528 25199 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:43.016619 25087 consensus_queue.cc:1059] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:43.017668 25087 raft_consensus.cc:3055] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:43.017994 25087 raft_consensus.cc:740] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:43.018808 25087 consensus_queue.cc:260] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:43.019892 25087 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:43.015283 25163 consensus_queue.cc:1059] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 13:08:43.020895 25096 consensus_queue.cc:1059] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:43.021803 25163 raft_consensus.cc:3055] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:43.022159 25163 raft_consensus.cc:740] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:43.022655 25096 raft_consensus.cc:3055] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:08:43.021102 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:43.023212 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:43.023005 25163 consensus_queue.cc:260] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:42.944351 25176 consensus_queue.cc:260] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:43.024255 25163 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:43.024438 25176 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:08:43.024600 25182 consensus_queue.cc:1175] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "0a407504ed1741bf89a276558adf4c0f" responder_term: 6 status { last_received { term: 6 index: 7 } last_committed_idx: 7 last_received_current_leader { term: 6 index: 7 } }
I20260709 13:08:43.023026 25096 raft_consensus.cc:740] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:43.026005 25096 consensus_queue.cc:260] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:43.026824 23590 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:43.027383 25096 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:43.030040 25193 raft_consensus.cc:3055] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:43.030942 22994 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:42.975907 25164 consensus_queue.cc:260] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:43.038038 23024 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6.
W20260709 13:08:43.041142 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 92814.2us
W20260709 13:08:43.054570 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 56263us
I20260709 13:08:43.051826 25193 raft_consensus.cc:740] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:43.055588 25088 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:43.056794 25182 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:08:43.056934 23621 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:08:43.057348 25182 raft_consensus.cc:697] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:43.032102 25165 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:43.057834 25088 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:43.058115 25165 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:43.058249 25182 consensus_queue.cc:237] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:43.056630 25193 consensus_queue.cc:260] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:43.067147 25199 raft_consensus.cc:2749] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:43.031949 24935 raft_consensus.cc:3055] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:08:43.072983 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 178386us
I20260709 13:08:43.077343 23025 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 5.
W20260709 13:08:43.082640 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 51772.2us
W20260709 13:08:43.082793 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 149406us
I20260709 13:08:43.080475 22918 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
W20260709 13:08:43.086180 23588 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:43.088526 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 50734.2us
I20260709 13:08:43.095562 25228 raft_consensus.cc:2804] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:43.079244 24935 raft_consensus.cc:740] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:43.098672 22920 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:43.085589 25080 consensus_queue.cc:1059] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 13:08:43.100934 22775 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48906 (request call id 198) took 11184 ms. Trace:
I20260709 13:08:43.107364 22777 catalog_manager.cc:5697] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:43.107414 25065 consensus_queue.cc:1059] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 13:08:43.102317 24943 consensus_queue.cc:1059] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 13:08:43.104559 25019 raft_consensus.cc:2804] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:08:43.085841 25208 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:43.086043 25199 consensus_queue.cc:1048] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
W20260709 13:08:43.156801 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 49602.4us
I20260709 13:08:43.089563 25134 consensus_queue.cc:1059] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.017s
W20260709 13:08:43.164772 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:43.093641 25229 consensus_queue.cc:1059] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.050s
I20260709 13:08:43.167708 22777 catalog_manager.cc:5697] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 3 to 4, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 4 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:43.171998 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:43.102608 22775 rpcz_store.cc:276] 0709 13:08:31.916630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:31.916976 (+   346us) service_pool.cc:224] Handling call
0709 13:08:43.100875 (+11183899us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:42.549601 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:42.550382 (+   781us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:42.550437 (+    55us) write_op.cc:435] Acquired schema lock
0709 13:08:42.550449 (+    12us) tablet.cc:662] Decoding operations
0709 13:08:42.551195 (+   746us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:42.551279 (+    84us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:42.551303 (+    24us) tablet.cc:685] Acquiring locks for 4 operations
0709 13:08:42.553253 (+  1950us) tablet.cc:701] Row locks acquired
0709 13:08:42.553305 (+    52us) write_op.cc:260] PREPARE: finished
0709 13:08:42.553503 (+   198us) write_op.cc:270] Start()
0709 13:08:42.553658 (+   155us) write_op.cc:276] Timestamp: P: 1783602522553434 usec, L: 0
0709 13:08:42.553681 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 13:08:42.594371 (+ 40690us) log.cc:844] Serialized 1662 byte log entry
0709 13:08:42.741161 (+146790us) op_driver.cc:464] REPLICATION: finished
0709 13:08:42.853914 (+112753us) write_op.cc:301] APPLY: starting
0709 13:08:42.854056 (+   142us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:42.854284 (+   228us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:42.854450 (+   166us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:42.855337 (+   887us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:42.855706 (+   369us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 13:08:42.855773 (+    67us) write_op.cc:312] APPLY: finished
0709 13:08:42.856194 (+   421us) log.cc:844] Serialized 41 byte log entry
0709 13:08:42.856452 (+   258us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:42.856578 (+   126us) write_op.cc:454] Released schema lock
0709 13:08:42.856652 (+    74us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:42.859097 (+  2445us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:42.859155 (+    58us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":8626,"reported_tablets":16,"child_traces":[["op",{"apply.queue_time_us":110084,"apply.run_cpu_time_us":3278,"apply.run_wall_time_us":5148,"num_ops":4,"prepare.queue_time_us":168077,"prepare.run_cpu_time_us":4214,"prepare.run_wall_time_us":45237,"replication_time_us":163174,"thread_start_us":154438,"threads_started":2}]]}
I20260709 13:08:43.111397 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:43.173983 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:43.209160 23624 raft_consensus.cc:2410] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:08:43.175099 23616 raft_consensus.cc:1240] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
W20260709 13:08:43.176077 25231 log.cc:927] Time spent T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:08:43.163426 25193 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:43.176527 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:43.112416 25164 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:43.163506 25228 raft_consensus.cc:697] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:43.104292 25188 consensus_queue.cc:1059] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:08:43.097900 25226 consensus_queue.cc:1048] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.066s
I20260709 13:08:43.176406 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:43.213444 22993 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:08:43.213320 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 114706us
W20260709 13:08:43.215750 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 53710.5us
W20260709 13:08:43.218770 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 137316us
I20260709 13:08:43.106515 25159 consensus_queue.cc:1059] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.073s
I20260709 13:08:43.169154 25114 raft_consensus.cc:2804] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:08:43.220991 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 108927us
I20260709 13:08:43.165614 25208 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:43.225440 22843 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:43.224961 22992 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:43.232057 23622 raft_consensus.cc:1275] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:08:43.232601 25188 raft_consensus.cc:3055] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:08:43.233035 25188 raft_consensus.cc:740] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:43.226188 23588 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:43.225590 25236 log.cc:927] Time spent T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
I20260709 13:08:43.220542 24943 consensus_queue.cc:1059] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 13:08:43.247984 22843 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:43.248737 25124 raft_consensus.cc:3055] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:08:43.249167 25124 raft_consensus.cc:740] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:43.249239 22843 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:43.250111 25124 consensus_queue.cc:260] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:43.251317 25124 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:43.251904 25240 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:08:43.252303 25168 raft_consensus.cc:2804] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:43.252420 25240 raft_consensus.cc:697] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:43.252827 24743 raft_consensus.cc:2749] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:43.257906 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:43.240995 22777 catalog_manager.cc:5697] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 3 to 4, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 4 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:43.251533 22843 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:43.257462 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:43.221799 25070 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:43.256938 25168 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
W20260709 13:08:43.296183 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 70077.4us
I20260709 13:08:43.257830 25088 raft_consensus.cc:2804] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:43.257185 24921 raft_consensus.cc:3055] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:08:43.304325 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 52863.4us
I20260709 13:08:43.304909 22777 catalog_manager.cc:5697] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 4 to 6, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 6 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:43.306579 25208 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:43.311677 22843 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:43.314832 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 169490us
I20260709 13:08:43.293375 25168 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:43.235010 25065 consensus_queue.cc:1059] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.074s
I20260709 13:08:43.233990 25188 consensus_queue.cc:260] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:43.301735 25070 raft_consensus.cc:2804] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:08:43.321223 22777 catalog_manager.cc:5697] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 4 to 5, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 5 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:43.312855 25114 raft_consensus.cc:697] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:43.312714 25075 raft_consensus.cc:2749] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:43.314085 25088 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:43.340898 25088 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:43.312848 24921 raft_consensus.cc:740] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:43.281615 25165 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:43.335541 22777 catalog_manager.cc:5697] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 4 to 5, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 5 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:43.338798 25247 raft_consensus.cc:2804] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:43.347013 23588 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:43.344740 25228 consensus_queue.cc:237] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:43.335609 24743 raft_consensus.cc:2749] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:08:43.351871 22843 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:43.359477 24921 consensus_queue.cc:260] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:43.359256 22777 catalog_manager.cc:5697] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 3 to 5, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 5 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:43.363583 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:43.365993 23621 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:43.366900 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:43.366649 25050 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:43.370364 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:43.371740 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:43.372082 25181 consensus_queue.cc:1048] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.016s
I20260709 13:08:43.372083 25182 consensus_queue.cc:1048] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:08:43.374234 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:43.374128 25159 consensus_queue.cc:1059] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:08:43.374923 23588 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:08:43.377087 25122 raft_consensus.cc:2804] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:08:43.376869 25088 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:43.344982 25240 consensus_queue.cc:237] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:43.392038 22876 raft_consensus.cc:2393] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 6: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:43.346403 25070 raft_consensus.cc:697] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 5 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:43.398949 22874 raft_consensus.cc:2393] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 5: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:43.401261 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:43.349284 25247 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:43.403782 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:43.398205 25075 raft_consensus.cc:2749] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 13:08:43.421574 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 69798.1us
I20260709 13:08:43.404016 25019 raft_consensus.cc:697] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 5 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:43.429322 25070 consensus_queue.cc:237] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:43.420033 24943 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:43.432948 24943 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:43.433068 25114 consensus_queue.cc:237] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:43.434938 24943 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:43.424650 25122 raft_consensus.cc:697] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 5 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:43.441804 25122 consensus_queue.cc:237] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:43.444247 25050 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:43.445885 22777 catalog_manager.cc:5697] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 2 to 4, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 4 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:43.449894 24935 consensus_queue.cc:260] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:43.448035 22777 catalog_manager.cc:5697] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 2 to 4, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 4 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:08:43.415696 25124 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:43.454710 22843 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:43.454905 25124 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:43.455982 22843 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:43.457511 25088 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:43.458171 22919 consensus_peers.cc:597] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Couldn't send request to peer 5b346767a89745fa82928a5dac9cc0b2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7cd8492f63fb4193a47cf9fbc574ef58. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:08:43.458653 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 284056us
I20260709 13:08:43.459136 25134 raft_consensus.cc:3055] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:43.459539 25134 raft_consensus.cc:740] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:43.460414 25165 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:43.411026 25250 consensus_queue.cc:1048] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 13:08:43.453922 25159 raft_consensus.cc:2804] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:08:43.416760 24921 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:43.466682 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:43.413455 25247 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:43.470937 25208 raft_consensus.cc:3055] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:43.471454 25208 raft_consensus.cc:740] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:43.472592 25124 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:43.472463 25208 consensus_queue.cc:260] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:43.473551 23024 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:08:43.474076 25208 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:43.474491 23024 raft_consensus.cc:606] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:08:43.474942 23024 raft_consensus.cc:614] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:43.481530 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:43.483604 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:43.484572 23617 raft_consensus.cc:2393] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 3: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:43.485010 25188 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:43.485786 25019 consensus_queue.cc:237] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:43.486407 25241 raft_consensus.cc:3055] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:43.486878 25241 raft_consensus.cc:740] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:43.487771 25241 consensus_queue.cc:260] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:43.490001 22877 raft_consensus.cc:1240] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:43.495009 22877 raft_consensus.cc:1240] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:43.495813 23623 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
I20260709 13:08:43.496027 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:43.496795 22876 raft_consensus.cc:1275] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:08:43.460419 25134 consensus_queue.cc:260] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:08:43.498486 23025 raft_consensus.cc:1240] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:08:43.498731 25134 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:08:43.500053 25235 log.cc:927] Time spent T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.283s	user 0.003s	sys 0.000s
I20260709 13:08:43.469605 25159 raft_consensus.cc:697] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 6 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:43.501753 25159 consensus_queue.cc:237] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:43.503744 25245 raft_consensus.cc:3055] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:08:43.504161 25245 raft_consensus.cc:740] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:43.505014 25245 consensus_queue.cc:260] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:43.476946 24943 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:43.506207 25245 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:43.506395 24943 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:43.513060 22876 raft_consensus.cc:1240] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:08:43.458186 25088 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:43.529958 22876 raft_consensus.cc:1240] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:43.524809 22919 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:43.532537 22992 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:08:43.539942 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:43.542877 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:08:43.541033 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 166165us
I20260709 13:08:43.543864 25124 raft_consensus.cc:2749] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:43.517303 25087 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:43.545969 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:43.546538 22874 raft_consensus.cc:1240] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:43.510883 24943 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:08:43.547050 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 54578.8us
I20260709 13:08:43.546224 22952 raft_consensus.cc:2393] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 3: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
W20260709 13:08:43.543190 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:43.491791 25075 consensus_queue.cc:1048] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:08:43.530821 25241 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:43.572954 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:43.574424 22919 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:43.477327 25168 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:43.578096 22992 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:43.588506 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:43.588569 25227 consensus_queue.cc:1059] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:08:43.599478 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:43.592341 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:43.607342 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:43.609616 22880 raft_consensus.cc:1240] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:08:43.611949 22874 raft_consensus.cc:1275] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:08:43.587244 25080 raft_consensus.cc:3055] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:08:43.637837 22920 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:43.643764 22873 raft_consensus.cc:1275] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:08:43.652612 23024 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
I20260709 13:08:43.656423 22874 raft_consensus.cc:1240] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 5. Current term is 6. Ops: []
W20260709 13:08:43.660163 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 85812.1us
I20260709 13:08:43.609220 24743 raft_consensus.cc:2749] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:08:43.671368 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:43.614941 25193 raft_consensus.cc:2804] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:43.670811 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:43.615017 25199 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:43.703466 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:43.586118 25087 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:43.609081 25254 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:43.687029 22952 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:08:43.695755 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:43.633612 25080 raft_consensus.cc:740] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:43.668828 25124 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:43.724746 25124 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:43.725219 25080 consensus_queue.cc:260] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:43.726687 25080 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:43.727278 25124 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:43.730000 25233 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:43.730683 25248 consensus_queue.cc:1059] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:08:43.731815 25248 raft_consensus.cc:3055] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:08:43.732191 25248 raft_consensus.cc:740] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:43.732753 25255 raft_consensus.cc:2749] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:08:43.730571 25233 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:43.669317 25204 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:08:43.701318 25236 log.cc:927] Time spent T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:08:43.746194 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:43.700554 25114 consensus_queue.cc:1059] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.058s
W20260709 13:08:43.741045 22843 connection.cc:256] RPC call timeout handler was delayed by -1.7903578729999481s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.167s for a response
I20260709 13:08:43.761861 23027 raft_consensus.cc:2393] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 6: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
W20260709 13:08:43.763077 22843 connection.cc:256] RPC call timeout handler was delayed by -1.7811777900000152s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18120000000000003s for a response
W20260709 13:08:43.763578 22843 connection.cc:256] RPC call timeout handler was delayed by -1.5057777900000247s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2118s for a response
W20260709 13:08:43.763980 22843 connection.cc:256] RPC call timeout handler was delayed by -1.4546578729999737s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20430000000000004s for a response
W20260709 13:08:43.764349 22843 connection.cc:256] RPC call timeout handler was delayed by -1.4168578730000263s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20850000000000002s for a response
I20260709 13:08:43.672820 25251 consensus_queue.cc:1048] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.094s
I20260709 13:08:43.746490 24889 raft_consensus.cc:3055] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:43.779098 25227 consensus_queue.cc:1059] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:08:43.778813 25237 consensus_queue.cc:1059] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
W20260709 13:08:43.800197 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 113303us
I20260709 13:08:43.786547 23024 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
I20260709 13:08:43.722074 25193 raft_consensus.cc:697] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:43.821121 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:43.821573 25193 consensus_queue.cc:237] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:43.790550 23025 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:43.794142 22952 raft_consensus.cc:1275] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:08:43.795578 22843 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:43.834899 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:43.838930 22919 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:08:43.811482 23024 raft_consensus.cc:1240] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:43.817692 22992 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:43.788501 25255 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:43.723084 25199 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:43.855903 22874 raft_consensus.cc:1275] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:08:43.858320 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:43.859112 22952 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:43.794123 24889 raft_consensus.cc:740] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:43.864354 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:43.866443 23623 raft_consensus.cc:1275] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 13:08:43.733105 25248 consensus_queue.cc:260] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:43.794590 25204 consensus_queue.cc:1048] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.068s
W20260709 13:08:43.886546 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 91057.1us
I20260709 13:08:43.887070 23024 raft_consensus.cc:1240] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:43.885468 22952 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
I20260709 13:08:43.871950 25075 raft_consensus.cc:2804] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:08:43.902945 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:43.903805 22947 raft_consensus.cc:2393] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 5: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:43.898556 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:43.907931 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:43.908385 22843 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:43.914117 25262 log.cc:927] Time spent T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
I20260709 13:08:43.915294 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:43.915912 25253 raft_consensus.cc:2749] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:08:43.908699 25075 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:43.917182 25075 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:08:43.920806 25272 consensus_queue.cc:1048] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 13:08:43.922907 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:43.923080 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:43.923517 22951 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:08:43.917263 25259 log.cc:927] Time spent T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.116s	user 0.000s	sys 0.001s
I20260709 13:08:43.928130 25226 raft_consensus.cc:3055] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:08:43.928704 23029 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 2.
I20260709 13:08:43.940162 25261 consensus_queue.cc:1048] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.006s
I20260709 13:08:43.946730 23021 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6.
I20260709 13:08:43.946796 23028 raft_consensus.cc:1275] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:08:43.850337 25247 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:43.899996 25168 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:43.997113 23028 raft_consensus.cc:1240] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 3. Current term is 4. Ops: []
W20260709 13:08:43.950304 25240 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.016s	sys 0.021s
W20260709 13:08:44.059892 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 151569us
I20260709 13:08:43.942551 25248 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:44.064749 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:43.948727 25226 raft_consensus.cc:740] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:43.872677 25255 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:44.116547 25240 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.016s	sys 0.022s
W20260709 13:08:44.162654 25265 log.cc:927] Time spent T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.213s	user 0.000s	sys 0.000s
W20260709 13:08:44.427349 23023 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.032s	sys 0.085s
I20260709 13:08:44.135667 24889 consensus_queue.cc:260] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:44.469528 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 630648us
W20260709 13:08:44.511571 23023 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.699s	user 0.032s	sys 0.085s
I20260709 13:08:44.470271 25050 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:44.470615 24743 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:44.471155 25087 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:44.472913 22951 consensus_meta.cc:287] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.549s	user 0.020s	sys 0.021s
I20260709 13:08:44.531108 22951 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 5.
W20260709 13:08:44.473143 25188 consensus_meta.cc:287] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.988s	user 0.006s	sys 0.009s
I20260709 13:08:44.473560 23621 raft_consensus.cc:1240] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 7. Current term is 8. Ops: []
I20260709 13:08:44.473785 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:44.543061 23028 raft_consensus.cc:1275] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:08:44.474493 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:44.476064 25247 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:44.476123 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:44.478051 25250 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:44.478775 25199 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:44.480181 22878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 1988) took 1099 ms. Trace:
I20260709 13:08:44.481633 25229 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:44.482450 25228 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
W20260709 13:08:44.486517 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:44.573140 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:44.486423 25168 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:44.575461 25168 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:44.578532 25168 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:44.579267 25168 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:44.489398 25254 consensus_meta.cc:287] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.724s	user 0.011s	sys 0.000s
I20260709 13:08:44.581303 25254 raft_consensus.cc:2749] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:43.940266 25252 raft_consensus.cc:2804] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:08:44.490025 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:44.478864 25088 raft_consensus.cc:2804] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:44.582923 25088 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:44.583436 25088 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
W20260709 13:08:44.492179 25241 consensus_meta.cc:287] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.902s	user 0.003s	sys 0.007s
I20260709 13:08:44.492647 23026 raft_consensus.cc:1275] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 13:08:44.491439 25245 consensus_meta.cc:287] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.984s	user 0.015s	sys 0.029s
I20260709 13:08:44.494032 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:44.494701 25233 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:08:44.497611 25080 consensus_meta.cc:287] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.771s	user 0.011s	sys 0.006s
I20260709 13:08:44.586593 23027 raft_consensus.cc:2410] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:08:44.586652 25080 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:44.587188 25080 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:44.499866 22947 raft_consensus.cc:2393] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 6: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:44.502251 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:08:44.476087 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:44.472471 25208 consensus_meta.cc:287] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.998s	user 0.008s	sys 0.007s
W20260709 13:08:44.480551 25134 consensus_meta.cc:287] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.981s	user 0.010s	sys 0.004s
I20260709 13:08:44.593070 22954 raft_consensus.cc:1275] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:08:44.593465 22954 pending_rounds.cc:85] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting all ops after (but not including) 2
I20260709 13:08:44.593663 22954 pending_rounds.cc:107] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 3.3
I20260709 13:08:44.594197 22954 raft_consensus.cc:2917] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Aborting config change with OpId 3.3: Aborted: Op aborted by new leader
I20260709 13:08:44.594623 22954 raft_consensus.cc:2889] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:44.594722 25241 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:44.540186 24889 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:44.474211 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:44.500986 25181 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:44.595295 25241 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:44.595793 25181 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:44.597683 25241 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:44.557492 22878 rpcz_store.cc:276] 0709 13:08:43.380950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:43.381933 (+   983us) service_pool.cc:224] Handling call
0709 13:08:43.382224 (+   291us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:43.382603 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:43.382628 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:43.383104 (+   476us) log.cc:844] Serialized 26 byte log entry
0709 13:08:43.608967 (+225863us) raft_consensus.cc:1645] Marking committed up to 2
0709 13:08:43.609024 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:43.609115 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:44.479281 (+870166us) raft_consensus.cc:1685] finished
0709 13:08:44.479419 (+   138us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:44.479978 (+   559us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":192,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":169,"thread_start_us":225413,"threads_started":1,"wal-append.queue_time_us":324843}
I20260709 13:08:44.598529 23029 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:44.599097 25134 consensus_queue.cc:1048] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:08:44.600683 22954 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:52206 (request call id 10) took 3495 ms. Trace:
I20260709 13:08:44.601137 22954 rpcz_store.cc:276] 0709 13:08:41.105440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:41.120913 (+ 15473us) service_pool.cc:224] Handling call
0709 13:08:41.558244 (+437331us) spinlock_profiling.cc:243] Waited 54.9 ms on lock 0x7b4c003d25c0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f61340b1a20 00007f61340c2a29 00007f6134088644 00007f61340aff2d 00007f61340afb93 00007f6135d3e1af 00007f6135d3df3e 00007f6135dc3888 00007f6135de075c 00007f6135de176a 00007f6135de091b
0709 13:08:41.620656 (+ 62412us) log.cc:844] Serialized 492 byte log entry
0709 13:08:41.975296 (+354640us) spinlock_profiling.cc:243] Waited 9.67 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:08:42.166059 (+190763us) spinlock_profiling.cc:243] Waited 9.77 ms on lock 0x7b1800133240. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cf83f3 00007f6132cfa7c1 00007f6132cfa779 00007f6132cfa709 00007f6132cfa6d1 00007f6132cf99cd 000000000038f354 000000000038f199 00007f6132cdb83a
0709 13:08:42.184726 (+ 18667us) spinlock_profiling.cc:243] Waited 16.8 ms on lock 0x7b5400084920. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135d5e142 00007f6135d3eeaa 00007f6135d45d53 00007f6135d45cd9 00007f6135d45c69 00007f6135d45c31 00007f6135d44dfd 000000000038f354 000000000038f199 00007f6132cf8475
0709 13:08:42.217084 (+ 32358us) spinlock_profiling.cc:243] Waited 31.6 ms on lock 0x7b580013afd0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4f090 00007f6135d3f43c 00007f6135d45d53 00007f6135d45cd9 00007f6135d45c69 00007f6135d45c31 00007f6135d44dfd 000000000038f354 000000000038f199 00007f6132cf8475 00007f6132cfa7c1
0709 13:08:44.600563 (+2383479us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":326934,"raft.queue_time_us":451205,"raft.run_cpu_time_us":5147,"raft.run_wall_time_us":150509,"spinlock_wait_cycles":282334208,"thread_start_us":114883,"threads_started":1,"wal-append.queue_time_us":337280,"wal-append.run_cpu_time_us":3345,"wal-append.run_wall_time_us":1019514}
W20260709 13:08:44.605223 22826 auto_rebalancer.cc:629] Failed to schedule move for tablet 7cd8492f63fb4193a47cf9fbc574ef58: Aborted: Op aborted by new leader
I20260709 13:08:44.611761 22843 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:44.597746 25181 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:44.618801 22953 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 6.
I20260709 13:08:44.621343 23028 raft_consensus.cc:1275] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:08:44.550043 24943 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:44.626411 23625 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 6.
I20260709 13:08:44.636356 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:44.638141 23028 raft_consensus.cc:1275] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:08:44.640493 22918 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:44.579869 25168 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:44.534509 25050 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:44.653756 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:44.669451 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 103727us
I20260709 13:08:44.663848 23021 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 5.
I20260709 13:08:44.541489 24743 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:44.668228 22918 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
W20260709 13:08:44.696209 25280 log.cc:927] Time spent T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.423s	user 0.000s	sys 0.008s
I20260709 13:08:44.696992 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:44.697175 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:44.697510 22872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53724 (request call id 355) took 1233 ms. Trace:
I20260709 13:08:44.697685 23618 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:08:44.697932 22872 rpcz_store.cc:276] 0709 13:08:43.464381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:43.477826 (+ 13445us) service_pool.cc:224] Handling call
0709 13:08:43.489216 (+ 11390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:43.492780 (+  3564us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:43.492814 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:43.519234 (+ 26420us) log.cc:844] Serialized 26 byte log entry
0709 13:08:43.854041 (+334807us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:43.854099 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:43.854196 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:44.697319 (+843123us) raft_consensus.cc:1685] finished
0709 13:08:44.697338 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:44.697459 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2043,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":29124,"thread_start_us":308223,"threads_started":1,"wal-append.queue_time_us":347670}
I20260709 13:08:44.699103 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:44.696563 23624 raft_consensus.cc:1275] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:08:44.700627 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:44.646977 25070 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:44.701964 22993 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:44.612686 25229 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:44.722754 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 205135us
I20260709 13:08:44.732401 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:44.711033 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:44.707268 22843 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:44.739506 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:44.740242 22875 raft_consensus.cc:2468] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:44.620600 25228 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:44.740326 23623 raft_consensus.cc:1275] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:08:44.690063 25289 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:44.742831 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:44.693404 25087 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:44.693188 25065 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:44.693609 25240 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:44.733937 23025 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
I20260709 13:08:44.735265 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:44.644985 24943 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:44.651082 25226 consensus_queue.cc:260] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:44.753233 25281 log.cc:927] Time spent T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:08:44.753106 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:08:44.753916 25248 consensus_meta.cc:287] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.593s	user 0.010s	sys 0.000s
I20260709 13:08:44.620980 25250 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:44.807041 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:44.757472 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:08:44.760776 25278 log.cc:927] Time spent T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.543s	user 0.000s	sys 0.008s
I20260709 13:08:44.761438 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:44.809068 22878 raft_consensus.cc:2393] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 5: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:44.761873 25286 raft_consensus.cc:2749] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:44.762421 22880 raft_consensus.cc:1240] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:44.762511 25288 raft_consensus.cc:2804] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:44.762734 25168 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:44.807078 25293 log.cc:927] Time spent T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.056s	user 0.002s	sys 0.000s
I20260709 13:08:44.803822 25240 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:44.765686 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:44.819765 23624 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:44.767390 25289 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:44.768074 23022 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 5.
I20260709 13:08:44.769671 25229 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:44.830516 25088 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:44.834481 25250 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:44.835932 25240 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:44.836966 25289 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:44.837535 25289 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:44.837811 22878 raft_consensus.cc:1275] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:08:44.837891 25289 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:44.838536 25289 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:44.771018 25188 raft_consensus.cc:2804] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:44.840942 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:44.841853 22878 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
I20260709 13:08:44.840981 25188 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:44.843180 25188 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:44.761469 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:44.777323 22872 raft_consensus.cc:1275] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:08:44.782810 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:44.806457 23026 raft_consensus.cc:2393] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 3: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:44.753656 25065 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:44.846406 22875 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
I20260709 13:08:44.811376 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:44.778887 25019 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:44.762904 25228 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:44.816932 25288 raft_consensus.cc:697] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:44.817435 23023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 404) took 1226 ms. Trace:
I20260709 13:08:44.817596 24743 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:44.748844 25251 consensus_queue.cc:1059] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.065s
W20260709 13:08:44.858618 25075 consensus_meta.cc:287] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.940s	user 0.012s	sys 0.004s
I20260709 13:08:44.860479 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:44.862054 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:44.733436 25070 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:44.847829 25168 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:44.890825 22880 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:08:44.819346 25087 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:44.860643 23023 rpcz_store.cc:276] 0709 13:08:43.590765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:43.604638 (+ 13873us) service_pool.cc:224] Handling call
0709 13:08:43.626260 (+ 21622us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:43.663458 (+ 37198us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:43.663501 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:43.743453 (+ 79952us) log.cc:844] Serialized 26 byte log entry
0709 13:08:44.512089 (+768636us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:44.512144 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:44.512244 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:44.817199 (+304955us) raft_consensus.cc:1685] finished
0709 13:08:44.817224 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:44.817370 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1454,"raft.queue_time_us":37459,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":32347,"thread_start_us":699259,"threads_started":1,"wal-append.queue_time_us":766821}
I20260709 13:08:44.856484 22948 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 5.
I20260709 13:08:44.899801 23028 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 5.
I20260709 13:08:44.871567 24743 raft_consensus.cc:2749] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:08:44.905429 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:44.906208 22951 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
W20260709 13:08:44.908450 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 240282us
W20260709 13:08:44.916440 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:44.894515 25088 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:44.870309 25255 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:44.860200 25019 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:44.921173 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:44.891767 25229 consensus_queue.cc:1059] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 13:08:44.909448 22952 raft_consensus.cc:1275] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:08:44.872334 25286 consensus_queue.cc:1048] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 13:08:44.894959 25168 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:44.894269 25075 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:44.920720 25296 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:44.926208 25134 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:44.921868 25252 raft_consensus.cc:697] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:44.955994 25252 consensus_queue.cc:237] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:44.925963 25065 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:44.938932 25241 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:44.965204 25070 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:44.967918 22918 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:44.970172 25075 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:44.953830 25134 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:44.950852 25288 consensus_queue.cc:237] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:44.973691 25134 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:44.974614 25134 raft_consensus.cc:3055] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:08:44.975049 25134 raft_consensus.cc:740] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
W20260709 13:08:44.976277 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:44.953941 25296 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:44.978464 25287 log.cc:927] Time spent T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.438s	user 0.000s	sys 0.003s
I20260709 13:08:44.978705 25296 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:44.979431 25296 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:44.980011 25296 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:44.976051 25134 consensus_queue.cc:260] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:44.984146 25134 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:44.985349 25296 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:44.989284 24943 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:44.936650 25050 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:44.946033 25255 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:45.028061 23617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 567) took 1090 ms. Trace:
I20260709 13:08:45.032419 25245 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:45.035120 25245 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:45.040866 25302 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:45.041608 25302 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:45.050478 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 82644.6us
I20260709 13:08:45.054160 22918 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
W20260709 13:08:45.052589 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 299046us
I20260709 13:08:45.030340 23617 rpcz_store.cc:276] 0709 13:08:43.902512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:43.920092 (+ 17580us) service_pool.cc:224] Handling call
0709 13:08:43.920399 (+   307us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:43.920797 (+   398us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:43.920824 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:43.921448 (+   624us) log.cc:844] Serialized 52 byte log entry
0709 13:08:44.168791 (+247343us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:44.168854 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:44.168944 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:44.977786 (+808842us) spinlock_profiling.cc:243] Waited 70.8 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:08:44.992106 (+ 14320us) raft_consensus.cc:1685] finished
0709 13:08:44.992133 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:44.992382 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1611,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"spinlock_wait_cycles":162892032,"thread_start_us":215359,"threads_started":1,"wal-append.queue_time_us":618385}
I20260709 13:08:44.984376 25289 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:45.066234 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 366317us
I20260709 13:08:45.070827 22994 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:08:45.064963 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 363063us
I20260709 13:08:45.077677 22880 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:08:45.109719 23588 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.033s	sys 0.038s
W20260709 13:08:45.112751 23588 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.034s	sys 0.038s
I20260709 13:08:45.085471 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:45.081228 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:45.114835 22946 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
I20260709 13:08:45.115087 22952 raft_consensus.cc:1275] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:08:45.075676 22992 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:45.119458 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:45.120235 22872 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 4.
I20260709 13:08:45.049837 25255 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:45.075178 22994 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:45.064867 23621 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 504) took 1625 ms. Trace:
I20260709 13:08:45.114320 23624 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
W20260709 13:08:45.122792 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 48243.4us
I20260709 13:08:45.116629 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:45.137033 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:45.121500 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:45.142360 22879 raft_consensus.cc:1240] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:45.143385 22873 raft_consensus.cc:1275] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:08:45.163364 22946 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:08:45.059341 25241 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:45.165131 22876 raft_consensus.cc:1275] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:45.084623 24943 raft_consensus.cc:2749] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:08:45.165427 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:45.068403 25240 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:45.083137 25075 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:45.165151 22877 raft_consensus.cc:1240] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:45.073628 25050 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:45.172534 23589 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.005s	sys 0.016s
I20260709 13:08:45.083307 25296 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:45.182773 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:45.174682 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:45.073644 25301 raft_consensus.cc:2749] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:45.176501 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:45.167492 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:45.083472 25087 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:45.176990 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:45.205842 22951 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
I20260709 13:08:45.199844 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:08:45.181548 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:45.196908 23589 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.698s	user 0.007s	sys 0.016s
I20260709 13:08:45.198320 23026 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 5.
I20260709 13:08:45.077327 25286 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:45.176772 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:45.219270 22952 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
I20260709 13:08:45.198151 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:45.202482 23022 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 7.
I20260709 13:08:45.200176 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:45.221771 22878 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6.
I20260709 13:08:45.169670 24743 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:45.068456 25188 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:45.196259 23023 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6.
I20260709 13:08:45.227603 22875 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 5.
I20260709 13:08:45.176740 25298 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:45.214655 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 160557us
I20260709 13:08:45.165997 25065 raft_consensus.cc:2804] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:45.219188 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:45.165464 25088 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:45.206700 23622 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 5.
I20260709 13:08:45.237783 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:45.136520 25070 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:45.231163 22918 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
W20260709 13:08:45.231971 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 64444.1us
I20260709 13:08:45.188524 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:45.193145 25288 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:45.241842 22880 raft_consensus.cc:1275] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:08:45.243496 22946 raft_consensus.cc:1275] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:08:45.197640 25299 raft_consensus.cc:2804] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:08:45.236977 25065 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:45.244410 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 169291us
I20260709 13:08:45.245085 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:08:45.246726 25283 log.cc:927] Time spent T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.706s	user 0.002s	sys 0.000s
I20260709 13:08:45.164832 23621 rpcz_store.cc:276] 0709 13:08:43.439427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:43.453200 (+ 13773us) service_pool.cc:224] Handling call
0709 13:08:43.453547 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:44.554054 (+1100507us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:45.057314 (+503260us) spinlock_profiling.cc:243] Waited 672 ms on lock 0x7b540049a2d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:45.063099 (+  5785us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1545234304}
I20260709 13:08:45.249135 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:45.250386 22843 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:45.252785 22994 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:45.266568 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:45.268586 22994 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:45.253746 22992 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:45.252125 22946 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 3.
I20260709 13:08:45.199868 25240 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:45.206658 25050 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:45.204734 25075 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:45.278148 22994 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:45.227344 25087 consensus_queue.cc:1048] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:08:45.224493 24921 consensus_queue.cc:1048] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 13:08:45.272855 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:45.206101 25296 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:45.230003 24743 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:45.297420 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:45.325676 22951 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6.
I20260709 13:08:45.243614 24943 raft_consensus.cc:2749] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:45.267606 25297 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:45.241088 25301 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:45.250082 25306 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:45.316192 22946 raft_consensus.cc:2393] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 7: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:45.253651 25065 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:45.254195 23616 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
I20260709 13:08:45.244176 25088 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:45.248776 25070 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:45.369534 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 90923.9us
I20260709 13:08:45.374735 22992 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:08:45.340456 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 96931.8us
I20260709 13:08:45.375905 22843 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:45.253108 25288 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:45.385623 25088 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:45.234094 25286 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:45.362423 25289 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:45.395030 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 88182.6us
I20260709 13:08:45.398327 25315 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:08:45.377854 25306 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:45.399331 25288 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:45.400863 25304 raft_consensus.cc:2804] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:45.401088 25316 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:45.401237 25326 raft_consensus.cc:2749] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:08:45.401430 25304 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:45.401610 25050 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:45.402052 25304 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:45.401778 25316 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:45.402354 25255 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:45.402164 25326 consensus_queue.cc:1048] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:45.403827 25326 raft_consensus.cc:2749] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:45.404330 25075 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:45.405110 25241 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:45.405738 25241 raft_consensus.cc:2804] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:45.405586 25326 consensus_queue.cc:1059] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:08:45.406118 25241 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:45.406498 25241 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:45.406620 25286 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:45.406877 25245 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:45.407436 25326 raft_consensus.cc:3055] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:08:45.407835 25326 raft_consensus.cc:740] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:45.409166 25070 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:45.409835 25240 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:45.408545 25326 consensus_queue.cc:260] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:45.411975 25296 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:45.412123 25188 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:45.413491 25326 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:45.360477 25309 raft_consensus.cc:2749] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:08:45.242810 25298 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:45.397444 22919 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:45.406695 24743 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:45.401109 25306 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:08:45.431457 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:45.454536 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 114703us
I20260709 13:08:45.479043 22993 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:45.388144 24943 raft_consensus.cc:2749] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:08:45.470741 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 94932.6us
I20260709 13:08:45.400642 25302 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:45.346091 25297 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:45.494261 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:45.494618 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:45.494951 23616 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
I20260709 13:08:45.495474 24743 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:45.495764 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:45.496359 23624 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:45.496775 25270 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:45.497480 25270 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:45.498153 25275 consensus_queue.cc:1048] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:08:45.499486 25275 consensus_queue.cc:1059] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:08:45.500226 25270 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:45.452024 25316 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:45.476764 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 79426.8us
I20260709 13:08:45.463845 25315 raft_consensus.cc:697] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:45.496016 24743 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:45.496876 25075 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:45.381754 25301 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:45.502933 22993 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:45.485220 22843 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:08:45.485859 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 214224us
W20260709 13:08:45.504813 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:45.482983 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 204866us
I20260709 13:08:45.500808 25289 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:45.474275 25288 consensus_queue.cc:1048] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:08:45.514814 25316 consensus_queue.cc:1048] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:08:45.515508 25303 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:45.532738 25305 raft_consensus.cc:2804] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:08:45.533306 25242 raft_consensus.cc:3055] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:08:45.533699 25242 raft_consensus.cc:740] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:45.533877 25301 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:45.534539 25242 consensus_queue.cc:260] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:45.526844 25075 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:45.536571 25319 raft_consensus.cc:2804] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:45.536993 25075 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:45.540194 22947 raft_consensus.cc:1275] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:08:45.537822 25298 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:45.538477 25325 raft_consensus.cc:2804] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:45.541870 25325 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:45.542243 25325 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:45.543661 22955 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 337) took 1028 ms. Trace:
I20260709 13:08:45.477389 25286 consensus_queue.cc:1048] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 13:08:45.544070 22955 rpcz_store.cc:276] 0709 13:08:44.515578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:44.515790 (+   212us) service_pool.cc:224] Handling call
0709 13:08:44.516058 (+   268us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:44.929854 (+413796us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:44.929899 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:44.945825 (+ 15926us) log.cc:844] Serialized 26 byte log entry
0709 13:08:45.218437 (+272612us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:45.218476 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:45.543288 (+324812us) spinlock_profiling.cc:243] Waited 26.3 ms on lock 0x7b540044af54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:45.543337 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:45.543381 (+    44us) raft_consensus.cc:1685] finished
0709 13:08:45.543392 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:45.543603 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":144933,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":103,"spinlock_wait_cycles":60500864,"thread_start_us":625863,"threads_started":2,"wal-append.queue_time_us":362731}
I20260709 13:08:45.547122 24743 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:45.547466 22994 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:45.548863 24921 raft_consensus.cc:2804] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:45.549280 24921 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:45.549227 25303 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:45.549757 25248 raft_consensus.cc:3055] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:45.550271 25248 raft_consensus.cc:740] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:45.551405 25303 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:45.551123 25248 consensus_queue.cc:260] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:45.552457 25305 raft_consensus.cc:697] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 5 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:45.552834 25270 raft_consensus.cc:3055] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:45.553227 25270 raft_consensus.cc:740] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:45.553417 25305 consensus_queue.cc:237] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:45.554088 25270 consensus_queue.cc:260] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:45.554630 24935 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:45.555330 25270 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:45.531694 25309 raft_consensus.cc:2804] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:45.496775 25296 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:45.557277 25309 raft_consensus.cc:697] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 4 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:45.557572 25296 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:45.549813 24921 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:45.491693 25299 raft_consensus.cc:697] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
W20260709 13:08:45.538067 22953 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58310 (request call id 1518) took 1674 ms (client timeout 1744 ms). Trace:
I20260709 13:08:45.551740 22994 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:45.511993 23624 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
I20260709 13:08:45.534935 22948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58310 (request call id 1524) took 1033 ms. Trace:
I20260709 13:08:45.536794 25323 raft_consensus.cc:2804] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:45.527520 22843 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:08:45.561606 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 44297us
W20260709 13:08:45.570241 22953 rpcz_store.cc:269] 0709 13:08:43.863921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:43.864064 (+   143us) service_pool.cc:224] Handling call
0709 13:08:45.537888 (+1673824us) spinlock_profiling.cc:243] Waited 723 ms on lock 0x7b5400379bd4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:45.537994 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1662420224}
I20260709 13:08:45.573917 22948 rpcz_store.cc:276] 0709 13:08:44.501901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:44.502020 (+   119us) service_pool.cc:224] Handling call
0709 13:08:45.534789 (+1032769us) spinlock_profiling.cc:243] Waited 222 ms on lock 0x7b5400074ed4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:45.534882 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":511064320}
W20260709 13:08:45.575069 23625 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 568) took 1651 ms (client timeout 1959 ms). Trace:
I20260709 13:08:45.575603 22993 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:45.583773 25246 raft_consensus.cc:2804] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:08:45.584410 25246 raft_consensus.cc:697] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:45.585423 25246 consensus_queue.cc:237] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:45.581193 23625 rpcz_store.cc:269] 0709 13:08:43.916151 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:43.922842 (+  6691us) service_pool.cc:224] Handling call
0709 13:08:45.567036 (+1644194us) spinlock_profiling.cc:243] Waited 665 ms on lock 0x7b54006383d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:45.567192 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1529544448}
I20260709 13:08:45.537070 25319 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:45.515862 25315 consensus_queue.cc:237] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:45.562914 25242 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:45.561993 25328 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:45.594406 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 66959.8us
I20260709 13:08:45.603022 22843 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:45.592784 25248 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:45.568514 25019 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:45.529928 25289 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:45.625965 22954 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2094) took 1727 ms. Trace:
I20260709 13:08:45.626453 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:45.615187 25296 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:45.599694 25319 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:45.619035 25299 consensus_queue.cc:237] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:45.558180 25309 consensus_queue.cc:237] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:45.621050 25328 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:45.621546 25297 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:45.600919 24743 raft_consensus.cc:2804] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:08:45.590080 24943 consensus_queue.cc:1048] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 13:08:45.626389 22954 rpcz_store.cc:276] 0709 13:08:43.898315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:43.899332 (+  1017us) service_pool.cc:224] Handling call
0709 13:08:43.914895 (+ 15563us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:44.606021 (+691126us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:45.625677 (+1019656us) spinlock_profiling.cc:243] Waited 678 ms on lock 0x7b540037afd4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:45.625894 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1558680832}
W20260709 13:08:45.628367 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 51555.9us
I20260709 13:08:45.648489 22994 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:45.615284 25325 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:45.606859 22918 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.676s (SENT) [suppressed 43 similar messages]
I20260709 13:08:45.631258 22843 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:45.579701 25323 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
W20260709 13:08:45.641630 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 147906us
I20260709 13:08:45.611655 25301 consensus_queue.cc:1048] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 13:08:45.639883 25320 consensus_queue.cc:1048] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
I20260709 13:08:45.683566 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:45.687753 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:45.689436 23625 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:45.691991 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:45.688967 22949 raft_consensus.cc:1275] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:08:45.674882 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:45.692590 22875 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6.
I20260709 13:08:45.668831 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:45.695899 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:45.672716 23029 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33420 (request call id 1510) took 1183 ms. Trace:
I20260709 13:08:45.696489 22875 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 5.
I20260709 13:08:45.669176 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:45.669457 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:45.696408 23029 rpcz_store.cc:276] 0709 13:08:44.489526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:44.489740 (+   214us) service_pool.cc:224] Handling call
0709 13:08:45.672535 (+1182795us) spinlock_profiling.cc:243] Waited 9.07 ms on lock 0x7b54002dafd4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:45.672635 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20857088}
I20260709 13:08:45.706238 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:45.706907 22876 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
I20260709 13:08:45.707834 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:45.687988 23590 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:45.709790 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:45.710275 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:45.711021 22875 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6.
I20260709 13:08:45.710428 23624 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6.
I20260709 13:08:45.711558 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:45.712162 22880 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
I20260709 13:08:45.679060 25065 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:45.715384 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:45.705401 25228 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:45.716495 25065 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:45.687812 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:45.716904 25228 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:45.716163 25252 consensus_queue.cc:1048] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:45.708977 25297 raft_consensus.cc:2804] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:45.720534 25122 consensus_queue.cc:1048] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:08:45.720821 23590 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:45.721124 25297 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:45.644694 25298 raft_consensus.cc:2749] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:08:45.723851 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:45.724646 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:45.655551 25296 raft_consensus.cc:2749] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:08:45.658659 25261 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:45.725446 23625 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
I20260709 13:08:45.735260 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:45.682214 25325 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:45.757155 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:45.738276 23026 raft_consensus.cc:1240] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 5 LEADER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:45.687917 25318 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:45.738349 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:45.685562 25080 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:45.671226 25168 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:45.725819 22875 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
W20260709 13:08:45.724440 25321 log.cc:927] Time spent T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.282s	user 0.000s	sys 0.001s
I20260709 13:08:45.727543 23588 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:45.730677 25308 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:45.679158 25289 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:45.759912 22876 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:08:45.709307 25114 consensus_queue.cc:1048] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.060s
I20260709 13:08:45.713353 25241 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:45.656234 24743 raft_consensus.cc:697] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 5 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:45.761479 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:45.725894 23590 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:45.810497 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:45.765650 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:45.741802 22920 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:45.766104 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:45.760695 25088 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:45.816197 23020 raft_consensus.cc:2368] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 5. Current term is 6.
I20260709 13:08:45.724951 25297 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:08:45.782752 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 55285.6us
I20260709 13:08:45.757117 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:45.785229 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:45.817121 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:45.817600 22876 raft_consensus.cc:2393] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 5: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:45.739296 23028 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
I20260709 13:08:45.817914 22871 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:45.787549 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:08:45.788072 25290 log.cc:927] Time spent T 00000000000000000000000000000000 P 8d175b4ae72f4fc48971c2f31668971c: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:08:45.788658 25303 raft_consensus.cc:2804] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:45.764788 25261 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:45.788913 22874 raft_consensus.cc:1275] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:08:45.789157 25331 raft_consensus.cc:2764] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:08:45.789748 25333 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:45.785892 25308 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:45.798141 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:45.829351 24743 consensus_queue.cc:237] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:45.802601 25226 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:45.803805 24921 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:45.763870 25252 consensus_queue.cc:1059] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 13:08:45.760859 25298 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:45.708674 25328 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:45.811331 23022 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 5.
I20260709 13:08:45.767508 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:08:45.813900 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 88078.4us
W20260709 13:08:45.811398 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 69706.9us
I20260709 13:08:45.875100 22920 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:45.877393 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:45.815097 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:45.879015 23026 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:45.880107 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:45.655820 25323 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:45.820130 23023 raft_consensus.cc:2410] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260709 13:08:45.817063 23588 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:45.779333 25296 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:45.823081 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:45.825511 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:45.826431 23024 raft_consensus.cc:2368] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 6. Current term is 7.
I20260709 13:08:45.823493 25088 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:45.820958 25329 raft_consensus.cc:2804] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:45.819278 25248 raft_consensus.cc:2804] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:45.827296 25320 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:45.825456 25303 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:45.823307 25304 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:46.033620 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 132991us
I20260709 13:08:46.332363 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:08:46.005900 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 399024us
I20260709 13:08:45.899857 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:46.341180 23020 raft_consensus.cc:2393] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 4: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
W20260709 13:08:46.335745 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:46.035449 23029 raft_consensus.cc:1240] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 3. Current term is 4. Ops: []
W20260709 13:08:46.339499 22843 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.010s	sys 0.017s
W20260709 13:08:46.343647 22843 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.010s	sys 0.017s
I20260709 13:08:46.056219 22871 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
W20260709 13:08:45.935895 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 60922.4us
I20260709 13:08:46.345911 23028 raft_consensus.cc:2393] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 6: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
W20260709 13:08:45.991007 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:45.962913 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:45.832767 25308 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:46.359135 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:46.360404 23026 raft_consensus.cc:2393] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 6: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:46.364622 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:46.365536 23020 raft_consensus.cc:2393] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 4: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:46.142915 25319 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:46.369324 25261 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:46.364943 25346 log.cc:927] Time spent T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.288s	user 0.001s	sys 0.000s
I20260709 13:08:46.371584 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:46.381319 23028 raft_consensus.cc:2468] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
I20260709 13:08:46.380319 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:46.368433 25319 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:46.387948 23027 raft_consensus.cc:2393] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 7: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:46.389413 23025 raft_consensus.cc:2368] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 7. Current term is 8.
I20260709 13:08:46.389906 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:46.081072 25328 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:46.392198 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:46.393114 23028 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 6.
I20260709 13:08:46.393953 22843 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:46.394348 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:46.395344 23028 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 2.
I20260709 13:08:46.396097 22843 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:46.397068 22871 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:08:46.397866 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:46.403867 23028 raft_consensus.cc:2393] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 3: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:46.071142 25301 raft_consensus.cc:2804] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:46.293424 25325 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:46.340334 25248 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:46.398658 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:46.344518 25329 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:46.370203 25304 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:45.962903 25242 raft_consensus.cc:3055] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:08:46.356081 25303 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:46.378937 25252 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:46.415074 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:08:46.389393 25309 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.010s	sys 0.019s
I20260709 13:08:46.416369 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:46.424044 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:45.833411 25333 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:46.415475 25324 raft_consensus.cc:2804] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:46.408627 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:45.976912 25298 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:46.385811 25296 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:46.356390 25320 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:46.406188 25301 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:46.425032 24921 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:46.456369 22871 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:08:46.454126 25329 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:46.392170 25114 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:46.480103 23028 raft_consensus.cc:2393] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 5: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:46.505061 22875 raft_consensus.cc:1275] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:08:46.506217 23028 raft_consensus.cc:1240] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:08:46.483036 23026 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:08:46.507465 23026 raft_consensus.cc:606] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:08:46.507867 23026 raft_consensus.cc:614] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:46.478426 25324 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:46.473529 25297 consensus_meta.cc:287] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.644s	user 0.009s	sys 0.000s
I20260709 13:08:46.459101 25242 raft_consensus.cc:740] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:46.414314 25328 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:46.494601 22880 raft_consensus.cc:2410] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:08:46.533352 23028 raft_consensus.cc:1275] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:46.494854 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:46.468703 25252 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:46.446316 25325 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:46.532323 25324 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:46.534564 25297 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:46.538156 25229 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:46.538720 25229 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:46.539323 25325 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:46.466746 25309 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.011s	sys 0.019s
I20260709 13:08:46.539770 25297 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:08:46.494482 25270 consensus_meta.cc:287] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.939s	user 0.012s	sys 0.000s
I20260709 13:08:46.540867 23021 raft_consensus.cc:2393] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 6: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:46.541618 23022 raft_consensus.cc:2393] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 6: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:46.541781 25328 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:46.531705 23020 raft_consensus.cc:2393] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 2: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:46.543614 25320 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:46.530774 25114 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:46.458817 25248 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:46.498783 25301 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:46.496429 25241 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:46.534693 23029 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
I20260709 13:08:46.559425 25229 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:46.610528 22871 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
W20260709 13:08:46.592401 25305 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.783s	user 0.020s	sys 0.008s
I20260709 13:08:46.676519 22993 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:08:46.679606 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 334414us
W20260709 13:08:46.665055 22874 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.018s	sys 0.029s
W20260709 13:08:46.707964 22874 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.018s	sys 0.031s
W20260709 13:08:46.665813 23617 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.827s	user 0.025s	sys 0.039s
I20260709 13:08:46.708457 25297 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:46.676702 24743 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.844s	user 0.033s	sys 0.033s
W20260709 13:08:46.709264 25355 log.cc:927] Time spent T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
I20260709 13:08:46.709139 25297 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:46.709824 25356 log.cc:927] Time spent T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 13:08:46.673828 25320 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:46.711582 25320 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:46.713567 25320 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:46.676043 25328 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:46.674212 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 857145us
W20260709 13:08:46.680167 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:46.709223 24743 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.877s	user 0.036s	sys 0.033s
I20260709 13:08:46.718040 22949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2101) took 1644 ms. Trace:
I20260709 13:08:46.718465 22949 rpcz_store.cc:276] 0709 13:08:45.073748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:45.114197 (+ 40449us) service_pool.cc:224] Handling call
0709 13:08:45.114482 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:45.693715 (+579233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:46.717737 (+1024022us) spinlock_profiling.cc:243] Waited 541 ms on lock 0x7b54000a87d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:46.717960 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1244674304}
I20260709 13:08:46.714366 25328 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:46.708637 23617 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.870s	user 0.025s	sys 0.041s
W20260709 13:08:46.748360 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 71942.4us
I20260709 13:08:46.617206 25228 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:46.636535 25305 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.827s	user 0.020s	sys 0.008s
I20260709 13:08:46.589013 25242 consensus_queue.cc:260] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:46.673669 25241 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:46.676729 25341 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:46.845047 25325 raft_consensus.cc:2804] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:08:46.900866 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:46.930519 22919 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.009s	sys 0.009s
W20260709 13:08:46.931022 22919 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.009s	sys 0.009s
I20260709 13:08:46.866361 25300 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:46.931753 22878 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.813s	user 0.000s	sys 0.020s
I20260709 13:08:46.868587 25241 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:46.932027 22844 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.828s	user 0.009s	sys 0.011s
I20260709 13:08:47.133270 22875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2000) took 1440 ms. Trace:
W20260709 13:08:47.148357 22844 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.044s	user 0.011s	sys 0.012s
W20260709 13:08:47.156569 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 1.27575e+06us
W20260709 13:08:47.111596 23590 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.776s	user 0.009s	sys 0.006s
W20260709 13:08:47.158771 23590 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.823s	user 0.009s	sys 0.006s
I20260709 13:08:47.035689 23024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33420 (request call id 1519) took 1306 ms. Trace:
I20260709 13:08:47.159775 23024 rpcz_store.cc:276] 0709 13:08:45.729051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:45.729208 (+   157us) service_pool.cc:224] Handling call
0709 13:08:47.035403 (+1306195us) spinlock_profiling.cc:243] Waited 15.3 ms on lock 0x7b540010bb54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:47.035601 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":35123584}
I20260709 13:08:47.051380 23022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33418 (request call id 1486) took 1222 ms. Trace:
I20260709 13:08:47.147182 22919 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:47.161217 23022 rpcz_store.cc:276] 0709 13:08:45.828741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:45.874185 (+ 45444us) service_pool.cc:224] Handling call
0709 13:08:47.051092 (+1176907us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b54002c9054. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:47.051301 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":443624320}
I20260709 13:08:47.099738 22877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 1514) took 1932 ms. Trace:
I20260709 13:08:47.022221 23021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33418 (request call id 1481) took 1241 ms. Trace:
I20260709 13:08:47.148638 22875 rpcz_store.cc:276] 0709 13:08:45.692977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:45.784133 (+ 91156us) service_pool.cc:224] Handling call
0709 13:08:45.786099 (+  1966us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:46.505637 (+719538us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:47.132997 (+627360us) spinlock_profiling.cc:243] Waited 718 ms on lock 0x7b54001bb554. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:47.133197 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1652447488}
W20260709 13:08:46.930454 23591 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.011s	sys 0.007s
W20260709 13:08:47.201318 23591 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.821s	user 0.013s	sys 0.008s
W20260709 13:08:47.201992 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:47.202557 22878 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.083s	user 0.000s	sys 0.022s
W20260709 13:08:47.195408 23023 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.007s	sys 0.012s
W20260709 13:08:47.204187 23023 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.008s	sys 0.012s
I20260709 13:08:47.207214 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:47.207948 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:47.209345 22844 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:47.197746 23021 rpcz_store.cc:276] 0709 13:08:45.781939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:45.812467 (+ 30528us) service_pool.cc:224] Handling call
0709 13:08:47.021904 (+1209437us) spinlock_profiling.cc:243] Waited 195 ms on lock 0x7b540018a954. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:47.022138 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":448960896}
W20260709 13:08:47.089066 25347 log.cc:927] Time spent T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 1.028s	user 0.001s	sys 0.006s
W20260709 13:08:47.035768 25318 consensus_meta.cc:287] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 1.257s	user 0.010s	sys 0.000s
I20260709 13:08:46.892637 25088 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:46.894618 25325 raft_consensus.cc:697] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 6 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:08:47.006929 25314 log.cc:927] Time spent T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 1.602s	user 0.012s	sys 0.012s
I20260709 13:08:47.227933 22878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53724 (request call id 369) took 1536 ms. Trace:
I20260709 13:08:47.229051 22992 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:47.198340 22877 rpcz_store.cc:276] 0709 13:08:45.166907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:45.205686 (+ 38779us) service_pool.cc:224] Handling call
0709 13:08:45.239429 (+ 33743us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:45.476057 (+236628us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:45.476092 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:45.476543 (+   451us) log.cc:844] Serialized 26 byte log entry
0709 13:08:45.640834 (+164291us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:08:45.640891 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:47.099286 (+1458395us) spinlock_profiling.cc:243] Waited 232 ms on lock 0x7b54001bc454. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:47.099334 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:47.099378 (+    44us) raft_consensus.cc:1685] finished
0709 13:08:47.099393 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:47.099661 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":162,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":4169,"spinlock_wait_cycles":533302528,"thread_start_us":130898,"threads_started":1,"wal-append.queue_time_us":178289,"wal-append.run_cpu_time_us":1538,"wal-append.run_wall_time_us":1002025}
I20260709 13:08:47.232280 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:08:47.176401 25323 consensus_meta.cc:287] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.796s	user 0.007s	sys 0.003s
W20260709 13:08:47.010167 25350 log.cc:927] Time spent T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.346s	user 0.000s	sys 0.001s
I20260709 13:08:47.098699 24935 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:47.124557 25291 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:47.237766 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:47.123829 25329 consensus_meta.cc:287] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.593s	user 0.010s	sys 0.000s
I20260709 13:08:47.255189 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:47.256038 22948 raft_consensus.cc:2410] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 13:08:46.931654 25075 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.112s	user 0.016s	sys 0.018s
I20260709 13:08:47.258880 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:47.198371 25333 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:46.879833 25341 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:47.261201 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:47.175675 25331 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:47.204308 25065 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:47.203397 25249 raft_consensus.cc:2804] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:47.037292 25251 consensus_queue.cc:1048] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.226s
W20260709 13:08:47.264788 25324 consensus_meta.cc:287] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.727s	user 0.011s	sys 0.000s
I20260709 13:08:47.259524 22947 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 5.
I20260709 13:08:47.213878 25275 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:47.230122 25289 raft_consensus.cc:2749] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:47.271198 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:47.099323 25301 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:47.238174 25318 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:47.175376 25320 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:47.234578 25305 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:47.233608 25304 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:47.346621 25320 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:47.346925 25305 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:47.347343 25304 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:47.260023 25075 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.447s	user 0.018s	sys 0.020s
I20260709 13:08:47.226089 25242 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:47.248687 25088 raft_consensus.cc:2764] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:08:47.254776 25114 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:47.362942 25114 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:47.276484 25065 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:47.310245 25324 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:47.286856 25331 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:47.320806 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:47.267496 25366 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:47.264477 25050 raft_consensus.cc:2749] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:08:47.318225 22873 raft_consensus.cc:1275] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:08:47.322201 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:47.203785 25300 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:47.309584 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:47.331208 22843 leader_election.cc:400] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 5. Current term is 6.
W20260709 13:08:47.279004 25226 consensus_meta.cc:287] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 1.440s	user 0.010s	sys 0.003s
I20260709 13:08:47.272796 25249 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:47.278692 25299 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.025s	sys 0.031s
I20260709 13:08:47.263981 24935 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:47.284449 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:47.337059 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:47.333118 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:47.340251 25246 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:47.341010 25325 consensus_queue.cc:237] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:47.342626 25296 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:47.344144 25318 raft_consensus.cc:2804] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:47.254607 25323 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:47.275836 25297 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:47.345323 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:47.345836 22948 raft_consensus.cc:2393] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 5: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:47.346760 25377 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:47.346792 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:47.315882 25229 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:47.358543 25378 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:47.264151 25329 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:47.307770 25370 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:47.270830 25291 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:47.310791 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:47.797546 22879 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:08:47.490044 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:47.631462 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:47.797708 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 466667us
I20260709 13:08:47.801612 22952 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
I20260709 13:08:47.414539 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:47.803272 22843 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:47.414109 23617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60044 (request call id 522) took 1698 ms. Trace:
I20260709 13:08:47.804489 22843 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:47.805260 22871 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:08:47.805742 22843 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:47.806771 22951 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:47.803290 22947 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:47.808838 25291 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:47.809402 25299 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.111s	user 0.025s	sys 0.036s
I20260709 13:08:47.810469 25249 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:47.814204 22953 raft_consensus.cc:1240] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:47.815508 22953 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 1530) took 2163 ms. Trace:
I20260709 13:08:47.815569 25323 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:47.815891 22953 rpcz_store.cc:276] 0709 13:08:45.652063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:45.652347 (+   284us) service_pool.cc:224] Handling call
0709 13:08:45.652623 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:47.814629 (+2162006us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:47.815265 (+   636us) spinlock_profiling.cc:243] Waited 2.16 s on lock 0x7b54003af054. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:47.815421 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4970616704}
I20260709 13:08:47.817782 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:47.804608 22877 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:47.823673 25249 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:47.825053 22918 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.000s	sys 0.005s
W20260709 13:08:47.825438 22918 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.000s	sys 0.005s
I20260709 13:08:47.826377 25249 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:47.340277 25369 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:47.833797 22950 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 5.
I20260709 13:08:47.507167 22876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2012) took 1405 ms. Trace:
I20260709 13:08:47.800907 22955 raft_consensus.cc:2468] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
I20260709 13:08:47.512425 23023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 1528) took 1776 ms. Trace:
I20260709 13:08:47.838212 23023 rpcz_store.cc:276] 0709 13:08:45.736384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:46.482779 (+746395us) service_pool.cc:224] Handling call
0709 13:08:46.500190 (+ 17411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:46.503623 (+  3433us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:46.503748 (+   125us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:46.517766 (+ 14018us) log.cc:844] Serialized 26 byte log entry
0709 13:08:47.204587 (+686821us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:08:47.204638 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:47.204744 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:47.512203 (+307459us) raft_consensus.cc:1685] finished
0709 13:08:47.512224 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:47.512359 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1550,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":24774,"thread_start_us":631915,"threads_started":1,"wal-append.queue_time_us":783070}
I20260709 13:08:47.344172 25252 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:47.804682 23617 rpcz_store.cc:276] 0709 13:08:45.715269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:45.717177 (+  1908us) service_pool.cc:224] Handling call
0709 13:08:45.717478 (+   301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:45.837610 (+120132us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:45.837639 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:45.838508 (+   869us) log.cc:844] Serialized 26 byte log entry
0709 13:08:47.142704 (+1304196us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:47.142747 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:47.337514 (+194767us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b5400453654. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:47.337554 (+    40us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:47.413776 (+ 76222us) raft_consensus.cc:1685] finished
0709 13:08:47.413799 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:47.414041 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1377,"raft.run_cpu_time_us":278,"raft.run_wall_time_us":59458,"spinlock_wait_cycles":275478784,"thread_start_us":906220,"threads_started":1,"wal-append.queue_time_us":1500240}
I20260709 13:08:47.835086 22876 rpcz_store.cc:276] 0709 13:08:46.101591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:46.101824 (+   233us) service_pool.cc:224] Handling call
0709 13:08:46.102268 (+   444us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:46.130154 (+ 27886us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:46.130200 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:46.306365 (+176165us) log.cc:844] Serialized 26 byte log entry
0709 13:08:47.127252 (+820887us) spinlock_profiling.cc:243] Waited 6.61 ms on lock 0x7b18003c0b80. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cdae07 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf7c02 00007f613597f20b 00007f6135984d23 00007f61359850d1 00007f6135da0a0f
0709 13:08:47.261327 (+134075us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:08:47.261368 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:47.261457 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:47.495690 (+234233us) raft_consensus.cc:1685] finished
0709 13:08:47.495717 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:47.506972 (+ 11255us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":49678,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":102673,"spinlock_wait_cycles":15212928,"thread_start_us":133880,"threads_started":1,"wal-append.queue_time_us":1149770}
I20260709 13:08:47.845027 22877 raft_consensus.cc:2468] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
W20260709 13:08:47.540006 25303 consensus_meta.cc:287] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 1.068s	user 0.006s	sys 0.006s
W20260709 13:08:47.474232 25248 consensus_meta.cc:287] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.895s	user 0.012s	sys 0.000s
I20260709 13:08:47.858642 23624 raft_consensus.cc:2393] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 7: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:47.363590 25341 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:47.691001 25308 consensus_meta.cc:287] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 1.286s	user 0.012s	sys 0.001s
I20260709 13:08:47.850549 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:47.694199 25304 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:47.411775 25357 log.cc:927] Time spent T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 13:08:47.410874 25360 log.cc:927] Time spent T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 13:08:47.851303 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 45641.9us
I20260709 13:08:47.853447 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:47.345674 25275 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:47.849858 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:47.846446 22871 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
I20260709 13:08:47.459683 25363 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:47.490099 25309 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:47.356145 25328 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:47.865216 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:48.123222 22876 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:08:47.889040 23590 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.012s	sys 0.000s
W20260709 13:08:48.124169 23590 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.882s	user 0.012s	sys 0.013s
I20260709 13:08:47.833726 24743 raft_consensus.cc:2804] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:48.123584 22843 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:47.967101 22947 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
W20260709 13:08:47.990856 22992 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.737s	user 0.020s	sys 0.015s
W20260709 13:08:47.967144 22872 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.004s	sys 0.014s
W20260709 13:08:48.144487 22992 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.891s	user 0.036s	sys 0.018s
W20260709 13:08:48.144819 22872 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.014s	sys 0.025s
W20260709 13:08:48.144925 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 915888us
W20260709 13:08:48.060801 23588 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.842s	user 0.000s	sys 0.024s
I20260709 13:08:48.145764 22843 leader_election.cc:400] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 7. Current term is 8.
W20260709 13:08:48.048153 22949 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.016s	sys 0.033s
I20260709 13:08:48.146355 22843 leader_election.cc:403] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:48.126565 22873 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 2.
W20260709 13:08:48.034452 22874 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.739s	user 0.005s	sys 0.035s
W20260709 13:08:47.920264 22920 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.001s	sys 0.013s
W20260709 13:08:48.146934 22874 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.869s	user 0.007s	sys 0.037s
W20260709 13:08:48.147032 22920 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.884s	user 0.007s	sys 0.018s
W20260709 13:08:48.124495 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 882415us
I20260709 13:08:48.147372 22843 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:48.048617 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:48.148244 22877 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:08:48.125319 22875 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
W20260709 13:08:48.145840 23588 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.927s	user 0.001s	sys 0.025s
I20260709 13:08:47.979501 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:08:48.146569 22949 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.710s	user 0.016s	sys 0.043s
W20260709 13:08:47.859529 22993 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.002s	sys 0.023s
W20260709 13:08:48.153568 22993 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.886s	user 0.003s	sys 0.023s
I20260709 13:08:47.834553 25228 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:48.156903 22994 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.903s	user 0.026s	sys 0.051s
W20260709 13:08:48.165221 22994 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.911s	user 0.026s	sys 0.051s
I20260709 13:08:47.829404 25318 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:47.541422 25368 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:47.472429 25362 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:47.435395 25298 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:48.166694 25368 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:48.166723 25362 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:48.167186 22994 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:48.169669 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:48.170156 22992 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:47.829794 25371 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:47.490439 25241 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:08:48.164849 22918 connection.cc:256] RPC call timeout handler was delayed by -1.1312807309999471s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15900000000000003s for a response
W20260709 13:08:48.176051 22918 connection.cc:256] RPC call timeout handler was delayed by -1.1204807310000433s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1602s for a response
I20260709 13:08:47.971417 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:47.990599 23618 raft_consensus.cc:2410] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 13:08:47.575760 25088 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:48.172644 22879 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
W20260709 13:08:48.025054 25367 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.666s	user 0.014s	sys 0.024s
I20260709 13:08:47.854583 25323 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:47.858467 25301 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:47.850315 25291 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:48.023180 23616 raft_consensus.cc:1240] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:08:47.858119 25372 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:48.276798 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:47.834383 25229 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:48.177326 22918 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:47.363709 25114 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:47.969441 23621 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 519) took 2396 ms. Trace:
I20260709 13:08:48.152892 25333 raft_consensus.cc:2749] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:48.094188 25331 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:48.097886 25324 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:48.160768 24743 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:48.161171 25308 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:48.394431 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:48.046517 25320 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:48.099224 25065 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:48.109808 24935 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:47.849771 25369 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:48.100428 25329 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:48.161381 25328 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:48.169219 25226 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:48.415647 22954 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
W20260709 13:08:48.427186 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 260083us
I20260709 13:08:48.173192 25298 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:48.178267 25318 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:08:48.269342 25080 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.969s	user 0.019s	sys 0.076s
I20260709 13:08:48.168625 25289 raft_consensus.cc:2804] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:47.897621 25248 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:48.278090 25367 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.919s	user 0.017s	sys 0.025s
I20260709 13:08:48.171130 25305 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:48.450316 22948 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:48.169106 25296 raft_consensus.cc:2804] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:08:48.329033 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:48.168648 25075 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 7. Current term is 8.
I20260709 13:08:48.178841 25371 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:48.433370 25304 raft_consensus.cc:2804] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:48.378669 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:48.471627 22952 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
I20260709 13:08:47.858605 25246 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:48.380434 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 203187us
I20260709 13:08:47.834846 25377 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:47.834969 25378 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:47.897475 25303 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:48.318244 23618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 525) took 1110 ms. Trace:
I20260709 13:08:47.238972 22878 rpcz_store.cc:276] 0709 13:08:45.691891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:45.692009 (+   118us) service_pool.cc:224] Handling call
0709 13:08:45.720954 (+ 28945us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:45.721134 (+   180us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:08:46.118887 (+397753us) log.cc:844] Serialized 14 byte log entry
0709 13:08:47.227273 (+1108386us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:47.227383 (+   110us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:47.227493 (+   110us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:47.227590 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:47.227823 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1839,"prepare.queue_time_us":76454,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":19907,"thread_start_us":1485466,"threads_started":2}
W20260709 13:08:48.391381 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 221273us
I20260709 13:08:47.833498 25366 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:48.171942 25309 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:48.176437 25363 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:48.173681 25228 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:48.384840 25357 log.cc:927] Time spent T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:08:47.850497 25370 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:48.320302 25291 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:48.318756 25372 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:48.285682 25088 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:48.386297 25299 raft_consensus.cc:2749] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:48.288055 22876 raft_consensus.cc:2410] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 13:08:48.427040 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:48.436350 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:48.436386 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:48.437644 22994 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:48.307338 25323 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:48.436184 25080 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.182s	user 0.021s	sys 0.078s
I20260709 13:08:48.434995 25065 raft_consensus.cc:2804] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:48.440562 25289 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:48.444026 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:48.445153 25275 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:48.445283 25248 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:48.446766 25353 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:48.436260 25328 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:48.449091 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:48.415872 24743 raft_consensus.cc:3060] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:48.379815 25383 consensus_queue.cc:1048] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.114s
W20260709 13:08:48.377213 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 208221us
I20260709 13:08:48.428573 25320 consensus_queue.cc:1048] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 13:08:48.438588 25298 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:48.437403 25226 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:48.435164 25341 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:48.482815 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:48.687373 22952 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
I20260709 13:08:48.490397 22874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53724 (request call id 374) took 1255 ms. Trace:
I20260709 13:08:48.585042 22948 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:08:48.377975 23621 rpcz_store.cc:276] 0709 13:08:45.494158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:45.495583 (+  1425us) service_pool.cc:224] Handling call
0709 13:08:45.495929 (+   346us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:45.611277 (+115348us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:45.611326 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:45.613485 (+  2159us) log.cc:844] Serialized 52 byte log entry
0709 13:08:45.757425 (+143940us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:45.765750 (+  8325us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:45.781338 (+ 15588us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:47.088469 (+1307131us) spinlock_profiling.cc:243] Waited 6.07 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:08:47.831694 (+743225us) raft_consensus.cc:1685] finished
0709 13:08:47.831723 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:47.859960 (+ 28237us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14369,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":18098,"spinlock_wait_cycles":13964544,"thread_start_us":186746,"threads_started":2,"wal-append.queue_time_us":211971,"wal-append.run_cpu_time_us":24193,"wal-append.run_wall_time_us":2567333}
I20260709 13:08:48.465444 25296 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
W20260709 13:08:48.692705 22876 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35370 (request call id 2021) took 1363 ms (client timeout 1762 ms). Trace:
W20260709 13:08:48.691100 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 253500us
I20260709 13:08:48.704103 22994 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:08:48.700773 22876 rpcz_store.cc:269] 0709 13:08:47.329163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:47.850343 (+521180us) service_pool.cc:224] Handling call
0709 13:08:48.692612 (+842269us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:48.690057 22992 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:08:48.690196 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 215611us
I20260709 13:08:48.696534 25401 consensus_queue.cc:1048] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 13:08:48.676334 25305 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:48.508951 25246 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:48.471421 25304 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:48.487978 25380 log.cc:927] Time spent T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:08:48.689714 22874 rpcz_store.cc:276] 0709 13:08:47.235302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:47.235506 (+   204us) service_pool.cc:224] Handling call
0709 13:08:47.263994 (+ 28488us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:47.277045 (+ 13051us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:47.277084 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:47.277567 (+   483us) log.cc:844] Serialized 26 byte log entry
0709 13:08:48.147407 (+869840us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:48.147444 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:48.147507 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:48.490194 (+342687us) raft_consensus.cc:1685] finished
0709 13:08:48.490228 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:48.490343 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2013,"raft.queue_time_us":13590,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":35105,"thread_start_us":869542,"threads_started":1,"wal-append.queue_time_us":1153377}
I20260709 13:08:48.710695 22918 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:48.710806 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:48.713011 23625 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:08:48.484925 25390 log.cc:927] Time spent T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:08:48.713550 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:08:48.486904 25242 consensus_meta.cc:287] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 1.123s	user 0.012s	sys 0.000s
I20260709 13:08:48.715170 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:48.715623 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:48.716838 22947 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:48.490020 25308 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:48.494391 25075 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:48.694994 25289 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:48.500996 25362 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:48.490991 25114 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:48.721817 25328 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:48.721952 25370 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:48.695475 25378 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:48.485001 25300 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:48.488893 25368 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:48.694191 25065 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:48.734448 23617 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
W20260709 13:08:48.735339 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:48.703156 25296 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:48.736629 22952 raft_consensus.cc:2393] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 7: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:48.737138 23623 raft_consensus.cc:2393] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 6: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:48.737740 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:48.701442 25372 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:48.736502 22918 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:48.692303 23618 rpcz_store.cc:276] 0709 13:08:47.208779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:47.229829 (+ 21050us) service_pool.cc:224] Handling call
0709 13:08:48.307946 (+1078117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:48.743604 25080 raft_consensus.cc:2804] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:48.781334 25080 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:08:48.781770 25404 log.cc:927] Time spent T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 13:08:48.781800 25080 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:48.781992 25226 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:48.782382 22947 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
I20260709 13:08:48.744271 25075 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:48.770933 22843 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.033s	sys 0.031s
I20260709 13:08:48.692929 25341 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:48.785006 22843 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.033s	sys 0.031s
I20260709 13:08:48.735446 25304 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:48.778769 22872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2016) took 1469 ms. Trace:
I20260709 13:08:48.779220 22952 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58320 (request call id 2120) took 1304 ms. Trace:
I20260709 13:08:48.780189 22918 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:48.722380 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:48.798676 22872 rpcz_store.cc:276] 0709 13:08:47.309502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:47.309651 (+   149us) service_pool.cc:224] Handling call
0709 13:08:47.337132 (+ 27481us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:47.337457 (+   325us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:47.337485 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:47.349166 (+ 11681us) log.cc:844] Serialized 26 byte log entry
0709 13:08:48.145448 (+796282us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:08:48.145506 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:48.145598 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:48.778588 (+632990us) raft_consensus.cc:1685] finished
0709 13:08:48.778606 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:48.778718 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2065,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":5424,"thread_start_us":703218,"threads_started":1,"wal-append.queue_time_us":909344}
I20260709 13:08:48.800220 22843 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:48.802733 22948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58320 (request call id 2122) took 1315 ms. Trace:
I20260709 13:08:48.803272 22948 rpcz_store.cc:276] 0709 13:08:47.487738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:48.169349 (+681611us) service_pool.cc:224] Handling call
0709 13:08:48.802524 (+633175us) spinlock_profiling.cc:243] Waited 280 ms on lock 0x7b54000eced4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:48.802660 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":643738880}
I20260709 13:08:48.783392 25309 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:48.804991 25309 consensus_queue.cc:1048] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:08:48.723222 25325 raft_consensus.cc:2804] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:48.806332 25323 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:48.806597 25325 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:48.806991 25325 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:48.808634 25303 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:48.806355 25309 raft_consensus.cc:2749] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:08:48.740335 25365 log.cc:927] Time spent T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 1.400s	user 0.014s	sys 0.031s
I20260709 13:08:48.790295 25291 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:48.794296 23625 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
I20260709 13:08:48.794965 25080 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:48.706962 25353 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:48.797536 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:08:48.797737 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 108091us
I20260709 13:08:48.741343 25246 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:48.817441 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:48.778142 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:48.722819 25305 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:48.774780 25065 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:48.774204 25399 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:48.859061 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:48.860126 22876 raft_consensus.cc:2393] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 4: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:48.799301 22952 rpcz_store.cc:276] 0709 13:08:47.473608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:48.698548 (+1224940us) service_pool.cc:224] Handling call
0709 13:08:48.777586 (+ 79038us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:08:48.841598 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 61471.4us
W20260709 13:08:48.825670 25395 log.cc:927] Time spent T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:08:48.841959 22992 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:48.841718 22993 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:08:48.823014 25380 log.cc:927] Time spent T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:08:48.838928 22994 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:48.844852 25298 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:48.846277 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:48.848037 25363 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:48.823181 25328 consensus_queue.cc:1048] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:08:48.848785 25303 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:48.852955 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:48.857677 25088 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:48.857812 25341 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:48.817315 25370 raft_consensus.cc:2804] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:48.840991 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:48.867636 22992 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:08:48.870940 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:48.817687 25362 consensus_queue.cc:1048] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.067s
I20260709 13:08:48.875732 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:08:48.895807 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 95653.6us
I20260709 13:08:48.897095 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:48.859443 25291 raft_consensus.cc:2749] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:08:48.877378 25384 raft_consensus.cc:2804] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:48.905258 22948 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 6.
I20260709 13:08:48.896277 25363 raft_consensus.cc:2804] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:48.906075 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:48.869336 25252 consensus_queue.cc:1048] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.065s
I20260709 13:08:48.911819 22953 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 5.
I20260709 13:08:48.917879 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:48.944583 22952 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
I20260709 13:08:48.952235 22878 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6.
I20260709 13:08:48.935936 22871 raft_consensus.cc:1240] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:48.906070 22876 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
W20260709 13:08:48.937841 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 98990.9us
I20260709 13:08:48.954573 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:48.955353 22871 raft_consensus.cc:2410] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:08:48.955242 22994 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:48.909078 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:48.956234 22879 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:48.956452 22994 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:48.941532 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:48.957125 22948 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
I20260709 13:08:48.876302 25075 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:48.893261 25246 raft_consensus.cc:2804] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:48.911392 22874 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
I20260709 13:08:48.959435 25246 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:48.959964 25246 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:48.965832 22877 raft_consensus.cc:1240] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
W20260709 13:08:48.970304 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:48.957291 22994 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:48.904984 25370 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:48.975499 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 107951us
I20260709 13:08:48.976599 22879 raft_consensus.cc:1275] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 13:08:48.972846 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 106796us
I20260709 13:08:48.919936 25228 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:48.898082 25303 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:48.910348 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:48.979025 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:48.988250 22874 raft_consensus.cc:2468] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 5.
I20260709 13:08:48.952837 25401 raft_consensus.cc:2804] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:08:48.922717 23616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60044 (request call id 525) took 1716 ms. Trace:
I20260709 13:08:48.875007 25400 consensus_queue.cc:1059] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.099s
I20260709 13:08:48.944018 25384 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:48.982666 22879 raft_consensus.cc:1240] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 5. Current term is 6. Ops: []
W20260709 13:08:48.973309 25318 consensus_meta.cc:287] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.523s	user 0.009s	sys 0.001s
I20260709 13:08:48.978389 22992 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:48.989445 22877 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:48.972268 25394 raft_consensus.cc:2804] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:48.979923 25370 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:48.913240 25088 consensus_queue.cc:1059] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 13:08:48.995278 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:48.970768 25363 raft_consensus.cc:697] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:48.990715 22949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2111) took 1731 ms. Trace:
W20260709 13:08:49.005072 23625 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 522) took 1798 ms (client timeout 2214 ms). Trace:
W20260709 13:08:49.009766 23625 rpcz_store.cc:269] 0709 13:08:47.206838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:47.206967 (+   129us) service_pool.cc:224] Handling call
0709 13:08:49.004909 (+1797942us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b54001108d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:49.005015 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3461458816}
I20260709 13:08:49.011577 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:49.013188 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:49.013520 22880 raft_consensus.cc:1275] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:08:49.013811 22878 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 5.
I20260709 13:08:49.001308 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:48.904718 25325 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:49.002683 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:49.006626 23624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 523) took 1799 ms. Trace:
I20260709 13:08:49.013674 22873 raft_consensus.cc:1275] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:08:49.006322 25328 raft_consensus.cc:2804] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:49.015404 23624 rpcz_store.cc:276] 0709 13:08:47.207118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:47.207718 (+   600us) service_pool.cc:224] Handling call
0709 13:08:49.006442 (+1798724us) spinlock_profiling.cc:243] Waited 650 ms on lock 0x7b54002bc354. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:49.006573 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1494820224}
I20260709 13:08:49.020339 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:49.014699 22879 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 5.
I20260709 13:08:49.006026 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:49.020941 22951 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 6.
I20260709 13:08:49.021317 22953 raft_consensus.cc:2376] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 6. Re-sending same reply.
I20260709 13:08:49.007779 22949 rpcz_store.cc:276] 0709 13:08:47.259378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:47.434664 (+175286us) service_pool.cc:224] Handling call
0709 13:08:47.435090 (+   426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:47.435533 (+   443us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:08:47.435881 (+   348us) log.cc:844] Serialized 14 byte log entry
0709 13:08:48.163130 (+727249us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:48.269303 (+106173us) log.cc:844] Serialized 26 byte log entry
0709 13:08:48.269446 (+   143us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:08:48.269489 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:48.269566 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:48.801271 (+531705us) spinlock_profiling.cc:243] Waited 2.39 ms on lock 0x7b180027dfc0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cf83f3 00007f6132cfa7c1 00007f6132cfa779 00007f6132cfa709 00007f6132cfa6d1 00007f6132cf99cd 000000000038f354 000000000038f199 00007f6132cdb83a
0709 13:08:48.818915 (+ 17644us) spinlock_profiling.cc:243] Waited 7.31 ms on lock 0x7b4800190158. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613ad0eaa1 00007f613ad05b37 00007f613ad059ca 00007f613adfd26f 00007f613adfcf37 00007f613ae1155c 00007f613ae11508 00007f613ae11488 00007f613ae11443 00007f613ae106ec 00007f6139d3aa96
0709 13:08:48.990431 (+171516us) raft_consensus.cc:1685] finished
0709 13:08:48.990465 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:48.990637 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13695,"prepare.queue_time_us":6181,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":22366,"raft.run_cpu_time_us":11169,"raft.run_wall_time_us":645766,"spinlock_wait_cycles":22302464,"thread_start_us":714885,"threads_started":1,"wal-append.queue_time_us":934574}
I20260709 13:08:49.026160 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:48.992039 25414 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:48.998637 25384 raft_consensus.cc:3060] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:49.000311 25394 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:49.031095 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:49.032622 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:49.000699 25401 raft_consensus.cc:697] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 6 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:49.040221 22949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58320 (request call id 2123) took 1552 ms. Trace:
W20260709 13:08:49.031769 22918 leader_election.cc:336] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.676s (SENT)
I20260709 13:08:49.037379 22877 raft_consensus.cc:1275] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:08:49.017285 25305 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:48.995652 25366 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:48.931497 25377 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:49.040632 22949 rpcz_store.cc:276] 0709 13:08:47.488100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:49.032382 (+1544282us) service_pool.cc:224] Handling call
0709 13:08:49.040148 (+  7766us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:49.040805 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:49.040838 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:49.040848 22918 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:49.041426 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:49.057490 22879 raft_consensus.cc:2368] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 6. Current term is 7.
I20260709 13:08:49.041141 22951 raft_consensus.cc:2393] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 5: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:49.016656 25369 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:49.019572 25320 raft_consensus.cc:2804] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:08:49.019042 25328 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:49.056979 25252 raft_consensus.cc:2749] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:49.031320 25414 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:49.061671 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:49.004832 25318 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:49.074786 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:49.075004 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:49.043466 25394 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:49.076053 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:49.107024 22874 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
I20260709 13:08:49.076920 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:49.082916 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:49.087975 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:49.088457 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:49.088966 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:49.089027 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:49.117177 22952 raft_consensus.cc:2376] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 5. Re-sending same reply.
I20260709 13:08:49.117362 23622 raft_consensus.cc:2393] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 8: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:49.089152 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:49.125980 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:49.126657 23620 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
I20260709 13:08:49.127120 25304 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:49.089893 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:49.128839 23625 raft_consensus.cc:2393] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 8: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:49.090576 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:49.129109 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:49.091327 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:49.092594 25377 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:49.097095 25401 consensus_queue.cc:237] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:49.149444 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:49.064536 25328 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:49.118582 22951 raft_consensus.cc:2393] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 3: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:49.108153 22949 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 6.
I20260709 13:08:49.124473 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:49.124595 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:49.039963 24743 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:49.106307 25414 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:48.996131 23616 rpcz_store.cc:276] 0709 13:08:47.206567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:47.214329 (+  7762us) service_pool.cc:224] Handling call
0709 13:08:47.279589 (+ 65260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:48.322385 (+1042796us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:48.922388 (+600003us) spinlock_profiling.cc:243] Waited 550 ms on lock 0x7b5400650ed4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:48.922624 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1265835264}
I20260709 13:08:49.150241 22879 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6.
I20260709 13:08:49.159282 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:49.104584 22843 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:08:49.162443 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 57979.2us
W20260709 13:08:49.162990 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 205815us
I20260709 13:08:49.163128 22843 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:49.064891 25320 raft_consensus.cc:697] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:49.153621 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:49.155400 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:49.153016 22877 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6.
I20260709 13:08:49.165539 22874 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6.
I20260709 13:08:49.131806 22919 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:49.148411 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:49.163950 22994 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:49.162106 22878 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 5.
I20260709 13:08:49.170723 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:49.147140 25080 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:49.173629 22873 raft_consensus.cc:2393] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 8: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:49.173808 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:08:49.174000 22950 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58310 (request call id 1522) took 5161 ms (client timeout 1891 ms). Trace:
I20260709 13:08:49.165087 22843 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:08:49.181599 22950 rpcz_store.cc:269] 0709 13:08:44.001518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:44.473979 (+472461us) service_pool.cc:224] Handling call
0709 13:08:49.162624 (+4688645us) spinlock_profiling.cc:243] Waited 3.24 s on lock 0x7b5400148f54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:49.162729 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7446844288}
I20260709 13:08:49.182176 25298 raft_consensus.cc:2804] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:08:49.182906 25298 raft_consensus.cc:697] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 5 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:49.183143 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:49.184095 25298 consensus_queue.cc:237] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:49.184544 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:49.187475 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:49.188287 22950 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
I20260709 13:08:49.190433 22950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2114) took 1921 ms. Trace:
I20260709 13:08:49.190521 22843 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:49.181628 22952 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
I20260709 13:08:49.190997 22950 rpcz_store.cc:276] 0709 13:08:47.268653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:49.189504 (+1920851us) service_pool.cc:224] Handling call
0709 13:08:49.189826 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:49.190029 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:49.190065 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:49.190118 (+    53us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:08:49.190174 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:49.190252 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:49.190367 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:49.193964 22950 raft_consensus.cc:1275] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:08:49.194098 22952 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2115) took 1922 ms. Trace:
I20260709 13:08:49.209365 22952 rpcz_store.cc:276] 0709 13:08:47.271792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:49.193137 (+1921345us) service_pool.cc:224] Handling call
0709 13:08:49.193475 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:49.193703 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:49.193741 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:49.193796 (+    55us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:49.193853 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:49.193921 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:49.194036 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:49.213757 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:49.214543 23029 raft_consensus.cc:2376] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3. Re-sending same reply.
I20260709 13:08:49.218379 22952 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2128) took 1728 ms. Trace:
I20260709 13:08:49.194828 25406 raft_consensus.cc:2804] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:49.195947 22843 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:49.177992 25304 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:49.207681 22918 leader_election.cc:336] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 2.209s (SENT)
I20260709 13:08:49.232437 22918 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:49.179153 25369 raft_consensus.cc:2804] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:49.219966 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:49.220448 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:49.194730 22950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2118) took 1827 ms. Trace:
I20260709 13:08:49.219950 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:49.224857 23020 raft_consensus.cc:1275] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 13:08:49.234777 22950 rpcz_store.cc:276] 0709 13:08:47.366991 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:49.193265 (+1826274us) service_pool.cc:224] Handling call
0709 13:08:49.193579 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:49.194461 (+   882us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:49.194678 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:49.236263 22950 raft_consensus.cc:1240] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:08:49.172444 25088 raft_consensus.cc:2804] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:49.229259 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:49.229432 22843 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:08:49.230268 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 251925us
I20260709 13:08:49.180706 25362 raft_consensus.cc:2749] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:08:49.179348 25305 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:08:49.231667 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 332134us
I20260709 13:08:49.232715 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:49.261503 23022 raft_consensus.cc:1240] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 13:08:49.261783 23029 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 5.
I20260709 13:08:49.153517 25377 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:49.238711 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:49.239207 22950 raft_consensus.cc:1240] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:08:49.232349 25075 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:49.237329 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:49.260393 25384 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:49.270227 25325 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:49.228394 23618 raft_consensus.cc:2410] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:08:49.271225 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:49.271373 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:49.271759 22874 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:49.271952 22877 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6.
I20260709 13:08:49.272783 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:49.272961 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:49.273316 22877 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:49.273651 22878 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 2.
I20260709 13:08:49.274670 22878 raft_consensus.cc:1240] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:49.237329 25369 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:49.275754 25369 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:49.242430 25080 raft_consensus.cc:2749] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:08:49.280645 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:49.247483 22949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58310 (request call id 1536) took 1077 ms. Trace:
W20260709 13:08:49.251854 25417 log.cc:927] Time spent T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.134s	user 0.000s	sys 0.000s
I20260709 13:08:49.253496 25309 consensus_queue.cc:1059] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:49.158358 25363 consensus_queue.cc:237] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:49.257575 25320 consensus_queue.cc:237] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:49.263275 22872 raft_consensus.cc:2393] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 4: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:49.263957 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:49.228173 25406 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:49.224340 22952 rpcz_store.cc:276] 0709 13:08:47.489643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:49.215409 (+1725766us) service_pool.cc:224] Handling call
0709 13:08:49.216888 (+  1479us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:49.217106 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:49.217149 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:49.218042 (+   893us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:08:49.218106 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:49.218178 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:49.218313 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:49.265202 22873 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6.
I20260709 13:08:49.268252 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:49.269577 23025 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:49.240243 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:08:49.346769 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:49.249951 22992 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:49.270241 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:49.352829 23029 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
I20260709 13:08:49.267377 22993 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:49.353264 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:49.233762 25353 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:49.281659 22879 raft_consensus.cc:2393] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 5: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:49.241751 25088 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:49.288142 22949 rpcz_store.cc:276] 0709 13:08:48.169630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:49.172176 (+1002546us) service_pool.cc:224] Handling call
0709 13:08:49.247404 (+ 75228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:49.290620 22877 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6.
I20260709 13:08:49.295019 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:49.343318 25298 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:49.278985 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:49.392148 22880 raft_consensus.cc:2368] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 7. Current term is 8.
I20260709 13:08:49.269272 25075 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:49.353914 22873 raft_consensus.cc:2468] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 2.
I20260709 13:08:49.346230 23020 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 5.
W20260709 13:08:49.354774 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 105173us
I20260709 13:08:49.354887 22918 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:49.358675 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:49.360966 25401 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:49.361438 25325 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
W20260709 13:08:49.362202 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 94883.3us
I20260709 13:08:49.362713 25421 raft_consensus.cc:2749] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:08:49.363613 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 113764us
W20260709 13:08:49.398396 25296 consensus_meta.cc:287] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.613s	user 0.007s	sys 0.003s
I20260709 13:08:49.370616 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:49.371465 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:08:49.374693 25399 consensus_meta.cc:287] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.503s	user 0.009s	sys 0.000s
I20260709 13:08:49.271394 25384 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:49.340323 25406 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:49.277102 25303 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:49.379841 25362 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:49.380546 25080 consensus_queue.cc:1048] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:49.383889 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:49.384091 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:49.384701 25251 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:49.387704 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:49.388458 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:49.390251 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:49.512096 22879 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 2.
I20260709 13:08:49.391036 22878 raft_consensus.cc:2393] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 6: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:49.393126 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:49.513489 22880 raft_consensus.cc:2393] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 5: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:49.394604 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:49.513996 22877 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
I20260709 13:08:49.295864 25050 raft_consensus.cc:2804] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:49.398917 22874 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 8.
I20260709 13:08:49.468719 25399 raft_consensus.cc:2749] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:49.403337 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:49.401214 22872 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
I20260709 13:08:49.400182 25075 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:49.406219 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:08:49.524758 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:49.369349 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:08:49.398279 25289 consensus_meta.cc:287] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.625s	user 0.006s	sys 0.006s
I20260709 13:08:49.467242 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:08:49.447022 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 92221.5us
I20260709 13:08:49.420559 25401 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:49.388586 25246 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:49.414469 24743 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:49.490496 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:49.410272 25418 raft_consensus.cc:2749] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:49.501974 25369 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:49.541481 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:49.542523 25369 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:49.542874 25427 log.cc:927] Time spent T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:08:49.543458 22950 raft_consensus.cc:2393] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 6: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:49.535367 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:49.544219 22877 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:49.497910 25362 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:49.546051 22954 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58320 (request call id 2112) took 2286 ms (client timeout 1594 ms). Trace:
I20260709 13:08:49.546312 25362 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:49.546449 22954 rpcz_store.cc:269] 0709 13:08:47.259618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:47.277203 (+ 17585us) service_pool.cc:224] Handling call
0709 13:08:49.545858 (+2268655us) spinlock_profiling.cc:243] Waited 575 ms on lock 0x7b5400284454. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:49.545963 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1322012032}
I20260709 13:08:49.548259 22952 raft_consensus.cc:1275] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:08:49.548979 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:49.549512 22879 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:49.416597 25410 raft_consensus.cc:3055] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:49.400135 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:49.397140 25298 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:49.422833 25318 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:49.391084 25088 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:49.517055 23619 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 3.
I20260709 13:08:49.527354 23021 raft_consensus.cc:2393] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 8: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:49.431119 25296 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:49.532593 23622 raft_consensus.cc:2368] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 7. Current term is 8.
I20260709 13:08:49.422780 25325 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:49.538776 25246 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:49.555984 22877 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
W20260709 13:08:49.556438 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 134881us
I20260709 13:08:49.579041 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:49.498404 25303 consensus_queue.cc:1048] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:08:49.485375 23588 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:49.511166 23616 raft_consensus.cc:2376] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 6. Re-sending same reply.
I20260709 13:08:49.573555 22955 raft_consensus.cc:1240] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:49.559836 25410 raft_consensus.cc:740] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:49.576998 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:49.536259 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:49.578557 22920 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:49.592389 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:49.592392 22994 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:49.582702 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:49.517299 25251 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:49.532207 25289 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:49.586068 22918 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:08:49.567466 25065 consensus_meta.cc:287] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.722s	user 0.010s	sys 0.000s
I20260709 13:08:49.547850 25435 raft_consensus.cc:2804] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:08:49.590674 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 105407us
I20260709 13:08:49.579685 23029 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 3.
I20260709 13:08:49.598480 23588 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:08:49.598937 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:08:49.599601 22992 connection.cc:256] RPC call timeout handler was delayed by -2.5926834299999655s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15940000000000001s for a response
I20260709 13:08:49.582577 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:49.595487 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:49.565023 25226 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:49.599865 23029 raft_consensus.cc:2368] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 4. Current term is 5.
I20260709 13:08:49.602146 22920 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:49.747401 22877 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:49.585878 22879 raft_consensus.cc:2410] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:08:49.749452 22873 raft_consensus.cc:1240] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 6. Current term is 7. Ops: []
W20260709 13:08:49.738291 22844 leader_election.cc:336] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Timed out: RequestConsensusVote RPC to 127.21.63.4:44043 timed out after 1.959s (SENT)
I20260709 13:08:49.751271 22844 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:49.750989 22774 catalog_manager.cc:5697] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:08:49.595644 25399 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:49.750130 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:49.753485 22774 catalog_manager.cc:5697] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:49.753305 25399 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:49.755064 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:49.755883 22879 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:49.756110 22774 catalog_manager.cc:5697] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 2 to 6, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 6 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:49.759076 22774 catalog_manager.cc:5697] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 1 to 3, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 3 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:08:49.762164 22774 catalog_manager.cc:5697] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:49.764752 22774 catalog_manager.cc:5697] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 3 to 6, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 6 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:49.600198 23025 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:08:49.766705 22992 connection.cc:256] RPC call timeout handler was delayed by -2.0346834299999728s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2214s for a response
W20260709 13:08:49.769670 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 177353us
I20260709 13:08:49.770792 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:49.770835 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:49.772990 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:49.573309 25309 consensus_queue.cc:1059] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.173s
I20260709 13:08:49.773228 22955 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
I20260709 13:08:49.593724 25405 raft_consensus.cc:2749] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:08:49.600571 25435 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:49.780279 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 178192us
I20260709 13:08:49.596168 25246 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:49.774597 22875 raft_consensus.cc:1240] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:08:49.773319 22994 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:08:49.776490 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 190520us
I20260709 13:08:49.778224 22954 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 6.
W20260709 13:08:49.782944 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 44715us
I20260709 13:08:49.799088 22918 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:49.768784 25304 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:49.773984 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:08:49.768492 25429 log.cc:927] Time spent T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.217s	user 0.000s	sys 0.001s
I20260709 13:08:49.785802 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:49.601765 25228 raft_consensus.cc:3055] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:49.789778 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:49.588344 25320 consensus_queue.cc:1048] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.189s
I20260709 13:08:49.772387 22879 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:08:49.852995 23025 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 6.
I20260709 13:08:49.771679 25080 raft_consensus.cc:2804] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:49.788486 25323 raft_consensus.cc:2804] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:49.800940 22918 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:08:49.862025 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 61193.8us
I20260709 13:08:49.864045 22918 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:49.803344 25325 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:49.803812 25369 raft_consensus.cc:2804] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:49.803731 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:49.878213 22954 raft_consensus.cc:2393] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 5: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:49.806351 25251 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:49.781451 25384 raft_consensus.cc:2804] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:49.815697 23620 raft_consensus.cc:2410] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:08:49.816887 25410 consensus_queue.cc:260] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:49.819924 25298 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:49.825953 25296 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:49.779990 25428 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:08:49.831801 25372 consensus_meta.cc:287] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 1.034s	user 0.007s	sys 0.005s
I20260709 13:08:49.832894 25246 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:49.835253 25289 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:49.838146 25401 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:49.841367 23029 raft_consensus.cc:2376] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4. Re-sending same reply.
I20260709 13:08:49.897468 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:49.898633 23029 raft_consensus.cc:2393] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 3: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
W20260709 13:08:49.898763 25328 consensus_meta.cc:287] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.736s	user 0.010s	sys 0.000s
I20260709 13:08:49.899291 25328 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:49.899842 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:49.842747 22920 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:49.846248 22955 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
I20260709 13:08:49.900949 23027 raft_consensus.cc:2368] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 5. Current term is 6.
I20260709 13:08:49.901880 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:49.902952 25328 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:49.903349 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:49.854923 22879 raft_consensus.cc:1240] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:49.905679 25305 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:49.906165 25305 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:49.800215 22994 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:08:49.907723 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 107592us
I20260709 13:08:49.908814 22994 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:08:49.909044 22947 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58320 (request call id 2119) took 2461 ms (client timeout 2214 ms). Trace:
W20260709 13:08:49.909596 22947 rpcz_store.cc:269] 0709 13:08:47.447676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:48.393395 (+945719us) service_pool.cc:224] Handling call
0709 13:08:49.908873 (+1515478us) spinlock_profiling.cc:243] Waited 24.7 ms on lock 0x7b54004123d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:49.908976 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":56865536}
I20260709 13:08:49.911172 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:49.911559 25401 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:49.914144 25418 raft_consensus.cc:2804] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:49.914625 25418 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:49.915042 25418 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:49.918524 22994 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:49.854269 25434 raft_consensus.cc:2749] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:08:49.777434 22877 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6.
W20260709 13:08:49.856689 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 258253us
W20260709 13:08:49.925350 22843 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.000s	sys 0.008s
W20260709 13:08:49.925766 22843 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.000s	sys 0.008s
I20260709 13:08:49.802965 25399 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:49.791414 25377 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:49.767846 25065 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:49.867138 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:49.878610 22946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2108) took 2620 ms. Trace:
I20260709 13:08:49.791975 25438 raft_consensus.cc:2804] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:49.846088 25318 raft_consensus.cc:2804] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:49.806968 25075 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:49.879606 22918 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:49.943887 25410 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:49.853130 24743 raft_consensus.cc:3055] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:08:49.779569 25226 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:49.884140 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:49.851083 25229 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:49.861400 25323 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:49.877725 25369 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:49.957707 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:49.944538 25400 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:49.849418 25435 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:49.855401 25080 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:49.850952 25228 raft_consensus.cc:740] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:49.828022 25406 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:49.852440 25304 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:49.884382 25366 raft_consensus.cc:2749] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:49.882853 25384 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:49.922793 23020 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 5.
I20260709 13:08:49.943540 25440 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:49.923128 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:49.791603 25373 raft_consensus.cc:2804] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:08:49.888252 25394 consensus_meta.cc:287] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.740s	user 0.008s	sys 0.003s
W20260709 13:08:49.926589 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 506494us
I20260709 13:08:49.971776 22955 raft_consensus.cc:2410] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
W20260709 13:08:49.928121 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 85458.2us
W20260709 13:08:49.943953 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 64472.9us
I20260709 13:08:49.972568 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:49.973704 22920 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:49.974550 22918 leader_election.cc:400] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 4. Current term is 5.
I20260709 13:08:49.974848 22918 leader_election.cc:403] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
I20260709 13:08:49.882305 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:49.975032 22920 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:49.937628 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:49.976747 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:49.882964 25439 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:49.946959 23021 raft_consensus.cc:3060] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:49.982666 25439 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:49.983075 25439 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:49.949149 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:49.984990 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:49.985121 22873 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:08:49.985546 22878 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:08:49.961907 22876 raft_consensus.cc:1240] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:08:49.986219 22920 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:49.961408 25080 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:08:49.962735 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 81343.4us
I20260709 13:08:49.919032 25428 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:49.996248 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:49.997061 23020 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:49.997272 23021 raft_consensus.cc:2468] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 5.
I20260709 13:08:49.998363 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:49.912422 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:49.930401 25388 raft_consensus.cc:2804] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:08:49.942060 25438 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:49.967060 22949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 1560) took 1054 ms. Trace:
I20260709 13:08:50.005939 22949 rpcz_store.cc:276] 0709 13:08:48.912739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:49.399812 (+487073us) service_pool.cc:224] Handling call
0709 13:08:49.414948 (+ 15136us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:49.512011 (+ 97063us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:49.512064 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:49.558741 (+ 46677us) log.cc:844] Serialized 26 byte log entry
0709 13:08:49.749812 (+191071us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:49.964504 (+214692us) log.cc:844] Serialized 14 byte log entry
0709 13:08:49.966723 (+  2219us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:49.966811 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:49.966860 (+    49us) raft_consensus.cc:1685] finished
0709 13:08:49.966876 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:49.967001 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5113,"raft.queue_time_us":146210,"raft.run_cpu_time_us":319,"raft.run_wall_time_us":27454,"thread_start_us":463378,"threads_started":3,"wal-append.queue_time_us":236641}
I20260709 13:08:49.968883 25440 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:49.970901 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:50.011924 23028 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:08:49.995350 25400 raft_consensus.cc:2749] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:08:49.960099 25369 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:49.926954 25372 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:50.015661 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:49.976125 25373 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:49.980389 25377 raft_consensus.cc:2804] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:49.980897 25436 consensus_queue.cc:1059] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:49.981571 25251 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:49.981956 25229 consensus_queue.cc:1048] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:08:49.960268 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:49.987471 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:50.020766 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:49.960577 22994 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:49.963853 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:49.944877 25318 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:50.002676 22878 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
I20260709 13:08:49.929725 22946 rpcz_store.cc:276] 0709 13:08:47.258599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:47.267434 (+  8835us) service_pool.cc:224] Handling call
0709 13:08:47.267777 (+   343us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:08:47.993002 (+725225us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:47.993047 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:08:48.113730 (+120683us) log.cc:844] Serialized 52 byte log entry
0709 13:08:48.310410 (+196680us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:08:48.310470 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:49.845308 (+1534838us) spinlock_profiling.cc:243] Waited 647 ms on lock 0x7b54000ed8d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:49.845365 (+    57us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:49.845416 (+    51us) raft_consensus.cc:1685] finished
0709 13:08:49.845434 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:49.878541 (+ 33107us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":134165,"raft.queue_time_us":107191,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":19362,"spinlock_wait_cycles":1487289088,"thread_start_us":195876,"threads_started":1,"wal-append.queue_time_us":604271,"wal-append.run_cpu_time_us":2862,"wal-append.run_wall_time_us":1192566}
I20260709 13:08:49.968259 25384 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:49.958638 25323 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:49.946357 24743 raft_consensus.cc:740] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:50.013109 23020 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 5.
W20260709 13:08:50.014207 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 85151.5us
I20260709 13:08:50.051447 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:08:50.014937 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 58465.4us
I20260709 13:08:50.016839 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:50.053134 22873 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 2.
I20260709 13:08:50.022104 22877 raft_consensus.cc:2393] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 6: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:50.021368 22843 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:50.021263 25370 consensus_meta.cc:287] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 1.000s	user 0.010s	sys 0.001s
I20260709 13:08:50.022518 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:49.987609 25433 consensus_queue.cc:1059] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 13:08:50.057157 25433 raft_consensus.cc:3055] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:08:50.057567 25433 raft_consensus.cc:740] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:50.058432 25433 consensus_queue.cc:260] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:50.059777 25433 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:50.059657 24743 consensus_queue.cc:260] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:50.060837 24743 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:08:50.022826 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 62329.4us
I20260709 13:08:50.023298 25399 raft_consensus.cc:2804] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:08:50.024500 25406 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:50.024894 25075 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:50.026492 25304 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:50.012928 25440 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:08:50.033380 25414 consensus_meta.cc:287] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.867s	user 0.010s	sys 0.000s
I20260709 13:08:50.021464 25377 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:50.035226 25372 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:50.035691 22871 raft_consensus.cc:1240] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 6. Current term is 7. Ops: []
I20260709 13:08:49.987965 25394 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:50.037487 25065 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:50.015705 25388 raft_consensus.cc:697] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 5 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:50.008803 25366 raft_consensus.cc:2804] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:50.009114 25438 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:49.995532 25291 consensus_queue.cc:1048] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 13:08:50.051627 23623 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 535) took 1140 ms. Trace:
I20260709 13:08:50.114751 23623 rpcz_store.cc:276] 0709 13:08:48.910844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:48.910979 (+   135us) service_pool.cc:224] Handling call
0709 13:08:48.911251 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:49.002747 (+ 91496us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:08:49.023013 (+ 20266us) log.cc:844] Serialized 14 byte log entry
0709 13:08:49.174743 (+151730us) log.cc:844] Serialized 14 byte log entry
0709 13:08:49.174934 (+   191us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:49.174987 (+    53us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:49.175047 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:49.917690 (+742643us) spinlock_profiling.cc:243] Waited 7.46 ms on lock 0x7b1800386f20. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cc62a6 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613ad0eaa1 00007f613ad05b37 00007f613ad059ca 00007f613adfd26f
0709 13:08:49.943990 (+ 26300us) spinlock_profiling.cc:243] Waited 19.7 ms on lock 0x7b48001be658. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613ad0eaa1 00007f613ad05b37 00007f613ad059ca 00007f613adfd26f 00007f613adfcf37 00007f613ae1155c 00007f613ae11508 00007f613ae11488 00007f613ae11443 00007f613ae106ec 00007f6139d3aa96
0709 13:08:50.051306 (+107316us) spinlock_profiling.cc:243] Waited 91.2 ms on lock 0x7b540046b254. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:50.051352 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:50.051565 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":26053,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":75,"raft.queue_time_us":17025,"raft.run_cpu_time_us":4832,"raft.run_wall_time_us":803142,"spinlock_wait_cycles":272166144,"thread_start_us":139656,"threads_started":1,"wal-append.queue_time_us":245032}
I20260709 13:08:50.052469 23617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60044 (request call id 526) took 1312 ms. Trace:
I20260709 13:08:50.116816 23617 rpcz_store.cc:276] 0709 13:08:48.739973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:48.772849 (+ 32876us) service_pool.cc:224] Handling call
0709 13:08:50.051325 (+1278476us) spinlock_profiling.cc:243] Waited 1.84 ms on lock 0x7b58001c4d98. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0a22 00007f613bbd9c09 00007f613ad33ee0 00007f613adfd0ab 00007f613adfd131 00007f613adab0a0 00007f613adaa984 00007f61343256d4 00007f6134325662
0709 13:08:50.052399 (+  1074us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4236160}
I20260709 13:08:50.011766 25418 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:50.055514 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:50.118646 22875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 1523) took 1169 ms. Trace:
I20260709 13:08:50.033469 25246 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:50.073112 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 51813.1us
I20260709 13:08:50.121157 23026 raft_consensus.cc:1240] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:50.021688 25373 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:50.124559 22843 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:50.126883 22843 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:50.047923 25318 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:50.103262 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:50.045799 25400 consensus_queue.cc:1048] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 13:08:50.049434 25436 consensus_queue.cc:1048] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 13:08:50.104759 25366 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:50.131278 25366 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:50.132907 25434 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:08:50.133615 25434 raft_consensus.cc:697] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 5 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:50.134699 25434 consensus_queue.cc:237] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:50.110829 22994 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:50.113581 22774 catalog_manager.cc:5697] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:08:50.064009 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:50.076166 25309 raft_consensus.cc:2749] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 13:08:50.119022 22875 rpcz_store.cc:276] 0709 13:08:48.949745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:48.950129 (+   384us) service_pool.cc:224] Handling call
0709 13:08:48.950536 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:49.839974 (+889438us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:50.118405 (+278431us) spinlock_profiling.cc:243] Waited 824 ms on lock 0x7b5400367754. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:50.118592 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1894554496}
W20260709 13:08:50.106504 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 102039us
I20260709 13:08:50.097543 23588 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:08:50.107005 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 132573us
W20260709 13:08:50.125036 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:50.150033 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 52580.3us
I20260709 13:08:50.128273 22880 raft_consensus.cc:2410] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:08:50.099952 25377 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:50.099570 22879 raft_consensus.cc:1275] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:08:50.150856 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:50.101473 25399 raft_consensus.cc:697] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:50.154418 22843 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:50.071044 25088 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:50.083050 25370 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:50.159838 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:50.150055 22994 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:50.174281 22946 raft_consensus.cc:2393] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 7: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:50.174587 22994 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:50.175073 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:50.160383 25402 raft_consensus.cc:2804] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:50.160555 25372 raft_consensus.cc:2804] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election won for term 5
W20260709 13:08:50.160411 22777 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33488 (request call id 266) took 11906 ms (client timeout 15000 ms). Trace:
I20260709 13:08:50.162918 22918 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:50.164701 22920 leader_election.cc:400] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 election: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 6. Current term is 7.
I20260709 13:08:50.098537 25414 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:50.172894 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:50.097036 25305 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:50.159584 23020 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 5.
I20260709 13:08:50.179629 22920 leader_election.cc:403] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
W20260709 13:08:50.171597 22843 leader_election.cc:336] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.538s (SENT)
I20260709 13:08:50.199337 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:50.107577 25080 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:50.193149 22874 raft_consensus.cc:1240] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:50.168964 25246 raft_consensus.cc:2804] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:50.166736 25401 raft_consensus.cc:2749] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:08:50.217473 23588 connection.cc:256] RPC call timeout handler was delayed by -1.7344067989999985s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.189s for a response
W20260709 13:08:50.218202 23588 connection.cc:256] RPC call timeout handler was delayed by -1.7119067990000758s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1915s for a response
I20260709 13:08:50.199249 22843 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:50.218639 23588 connection.cc:256] RPC call timeout handler was delayed by -1.6552067989999841s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1978s for a response
I20260709 13:08:50.191391 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:50.179351 25383 raft_consensus.cc:2804] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:08:50.181938 22777 rpcz_store.cc:269] 0709 13:08:38.254386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:38.254605 (+   219us) service_pool.cc:224] Handling call
0709 13:08:50.160345 (+11905740us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:43.633383 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:43.778896 (+145513us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:43.778935 (+    39us) write_op.cc:435] Acquired schema lock
0709 13:08:43.778952 (+    17us) tablet.cc:662] Decoding operations
0709 13:08:43.809146 (+ 30194us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:43.850722 (+ 41576us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:43.856196 (+  5474us) tablet.cc:685] Acquiring locks for 9 operations
0709 13:08:44.492552 (+636356us) tablet.cc:701] Row locks acquired
0709 13:08:44.492579 (+    27us) write_op.cc:260] PREPARE: finished
0709 13:08:44.492762 (+   183us) write_op.cc:270] Start()
0709 13:08:44.492898 (+   136us) write_op.cc:276] Timestamp: P: 1783602524492714 usec, L: 0
0709 13:08:44.492922 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:08:44.493838 (+   916us) log.cc:844] Serialized 3568 byte log entry
0709 13:08:45.294953 (+801115us) op_driver.cc:464] REPLICATION: finished
0709 13:08:45.615664 (+320711us) write_op.cc:301] APPLY: starting
0709 13:08:45.615754 (+    90us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:45.615957 (+   203us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:45.616084 (+   127us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:45.617167 (+  1083us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:45.617383 (+   216us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=9
0709 13:08:45.617429 (+    46us) write_op.cc:312] APPLY: finished
0709 13:08:45.617692 (+   263us) log.cc:844] Serialized 71 byte log entry
0709 13:08:45.617854 (+   162us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:45.617971 (+   117us) write_op.cc:454] Released schema lock
0709 13:08:45.618016 (+    45us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:45.618361 (+   345us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:45.618390 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1503543,"reported_tablets":29,"child_traces":[["op",{"apply.queue_time_us":285137,"apply.run_cpu_time_us":2733,"apply.run_wall_time_us":6595,"num_ops":9,"prepare.queue_time_us":126693,"prepare.run_cpu_time_us":61730,"prepare.run_wall_time_us":1001866,"raft.queue_time_us":381871,"raft.run_cpu_time_us":19641,"raft.run_wall_time_us":250511,"replication_time_us":757260,"thread_start_us":467791,"threads_started":4,"wal-append.queue_time_us":267050,"wal-append.run_cpu_time_us":29665,"wal-append.run_wall_time_us":2081193}]]}
I20260709 13:08:50.206460 23023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 1531) took 1038 ms. Trace:
I20260709 13:08:50.220691 22844 leader_election.cc:400] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 7. Current term is 8.
I20260709 13:08:50.221061 22844 leader_election.cc:403] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:50.210737 22953 raft_consensus.cc:2393] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 4: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
W20260709 13:08:50.211833 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 61897.2us
I20260709 13:08:50.220623 23023 rpcz_store.cc:276] 0709 13:08:49.167919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:49.168068 (+   149us) service_pool.cc:224] Handling call
0709 13:08:49.216384 (+ 48316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:49.216867 (+   483us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:49.216906 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:49.217487 (+   581us) log.cc:844] Serialized 26 byte log entry
0709 13:08:50.042416 (+824929us) spinlock_profiling.cc:243] Waited 1.81 ms on lock 0x7b18002cc980. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cdae07 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf7c02 00007f613597f20b 00007f6135984d23 00007f61359850d1 00007f6135da0a0f
0709 13:08:50.105181 (+ 62765us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:08:50.105216 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:50.105291 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:50.206161 (+100870us) raft_consensus.cc:1685] finished
0709 13:08:50.206180 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:50.206402 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":633,"raft.queue_time_us":6209,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":10264,"spinlock_wait_cycles":4174080,"thread_start_us":62584,"threads_started":1,"wal-append.queue_time_us":941188}
I20260709 13:08:50.222800 22920 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:50.213827 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:50.224373 22949 raft_consensus.cc:2368] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 5. Current term is 6.
I20260709 13:08:50.224359 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:50.177455 25075 raft_consensus.cc:2804] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:08:50.228935 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:50.177660 25402 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:50.229857 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:08:50.230406 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 58852.9us
W20260709 13:08:50.234754 22843 leader_election.cc:336] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.896s (SENT)
I20260709 13:08:50.235140 22843 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:50.230955 23024 raft_consensus.cc:1240] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 6. Current term is 7. Ops: []
I20260709 13:08:50.231624 22920 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:50.235577 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:50.236366 22844 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:50.186231 25065 raft_consensus.cc:2804] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:50.234109 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:50.196736 25372 raft_consensus.cc:697] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 5 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:50.237113 23020 raft_consensus.cc:2393] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 6: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:50.264190 23020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33420 (request call id 1545) took 1028 ms. Trace:
I20260709 13:08:50.264526 23020 rpcz_store.cc:276] 0709 13:08:49.236252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:50.214334 (+978082us) service_pool.cc:224] Handling call
0709 13:08:50.264134 (+ 49800us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:50.264469 25372 consensus_queue.cc:237] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:50.265686 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:50.258231 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:50.251281 23029 raft_consensus.cc:2368] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 5. Current term is 6.
I20260709 13:08:50.254622 22920 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:50.180627 25457 raft_consensus.cc:2804] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:50.258112 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:50.261539 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:50.217878 25246 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:50.266254 23020 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 4.
I20260709 13:08:50.218485 25305 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:50.217037 25394 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:50.223821 25226 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:50.221911 25444 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:50.221732 25396 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:50.267513 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:50.215719 25363 consensus_queue.cc:1048] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 13:08:50.220500 25400 raft_consensus.cc:2749] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 6. Current term is 7.
I20260709 13:08:50.238729 25050 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:50.250387 25309 raft_consensus.cc:2749] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 7. Current term is 8.
W20260709 13:08:50.296535 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 61798.4us
W20260709 13:08:50.285283 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 48991us
I20260709 13:08:50.294157 23617 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
I20260709 13:08:50.232023 25353 raft_consensus.cc:3055] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:08:50.353363 22918 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:50.333648 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:50.358125 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:50.358655 23021 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 3.
I20260709 13:08:50.294098 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:50.373863 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:50.374584 22876 raft_consensus.cc:1275] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:08:50.374967 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:50.376317 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:50.376626 22918 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:50.294196 25451 raft_consensus.cc:2749] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:50.297037 23023 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:50.238478 25406 raft_consensus.cc:2749] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:08:50.266759 25075 raft_consensus.cc:697] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 5 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:50.258991 25291 raft_consensus.cc:2749] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:08:50.221588 25455 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:50.305846 25050 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:50.388964 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:50.389516 22949 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:50.251144 25436 raft_consensus.cc:2749] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:50.306402 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:08:50.289134 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 51170.8us
I20260709 13:08:50.287591 23028 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 5.
I20260709 13:08:50.251332 25383 raft_consensus.cc:697] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 6 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:50.393409 22880 raft_consensus.cc:1275] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:08:50.287113 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:50.305464 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:50.405830 25075 consensus_queue.cc:237] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:50.406705 22877 raft_consensus.cc:1275] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 5. (term mismatch)
I20260709 13:08:50.407081 22877 pending_rounds.cc:85] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 4
I20260709 13:08:50.407330 22877 pending_rounds.cc:107] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 5.5
I20260709 13:08:50.407588 22877 raft_consensus.cc:2889] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:50.408000 25080 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:50.410166 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:50.411309 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:50.412423 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:50.413388 25366 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:50.415185 23021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33418 (request call id 1502) took 1017 ms. Trace:
I20260709 13:08:50.415479 25366 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:50.415549 23021 rpcz_store.cc:276] 0709 13:08:49.397452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:50.412228 (+1014776us) service_pool.cc:224] Handling call
0709 13:08:50.415136 (+  2908us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:50.325804 22871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 1529) took 1339 ms. Trace:
I20260709 13:08:50.423007 22871 rpcz_store.cc:276] 0709 13:08:48.986843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:48.988156 (+  1313us) service_pool.cc:224] Handling call
0709 13:08:48.988544 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:50.096645 (+1108101us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:50.325598 (+228953us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b5400302354. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:50.325748 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2407678208}
W20260709 13:08:50.326447 22951 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58310 (request call id 1544) took 1843 ms (client timeout 2449 ms). Trace:
I20260709 13:08:50.330427 23022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 1532) took 1162 ms. Trace:
I20260709 13:08:50.425035 23022 rpcz_store.cc:276] 0709 13:08:49.168258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:49.168390 (+   132us) service_pool.cc:224] Handling call
0709 13:08:49.220857 (+ 52467us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:49.261909 (+ 41052us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:50.330185 (+1068276us) spinlock_profiling.cc:243] Waited 40.4 ms on lock 0x7b540018a954. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:50.330363 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":92916480}
I20260709 13:08:50.333627 25414 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:50.231626 25369 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:50.335601 25320 consensus_queue.cc:1059] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:50.338668 23027 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33420 (request call id 1539) took 1141 ms. Trace:
I20260709 13:08:50.289582 25246 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:50.303841 25399 consensus_queue.cc:237] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:50.342141 25418 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:50.342589 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:50.447398 22947 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
I20260709 13:08:50.342209 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:50.448009 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:50.347616 25428 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:50.208175 25388 consensus_queue.cc:237] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:50.294144 25289 raft_consensus.cc:2749] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:08:50.241104 25402 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:50.352705 25439 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:50.296041 25252 raft_consensus.cc:2804] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:50.288262 22920 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:08:50.481530 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 193346us
I20260709 13:08:50.382712 22918 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:50.385172 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:50.385766 22955 raft_consensus.cc:2393] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 6: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:50.387738 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:50.394560 22844 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:50.305995 25396 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:50.396452 23590 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:50.398753 22880 raft_consensus.cc:1240] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 6. Current term is 7. Ops: []
I20260709 13:08:50.391585 25400 raft_consensus.cc:2749] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:50.400421 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:50.400583 25455 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:50.407428 25440 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:50.323361 25426 log.cc:927] Time spent T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.830s	user 0.004s	sys 0.000s
I20260709 13:08:50.324939 23026 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 1537) took 1155 ms. Trace:
I20260709 13:08:50.432145 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:50.377257 25458 raft_consensus.cc:2749] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:08:50.424358 22951 rpcz_store.cc:269] 0709 13:08:48.482643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:49.058752 (+576109us) service_pool.cc:224] Handling call
0709 13:08:50.326272 (+1267520us) spinlock_profiling.cc:243] Waited 18.7 ms on lock 0x7b5400400454. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:50.326394 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42962816}
I20260709 13:08:50.352582 25353 raft_consensus.cc:740] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:50.434980 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:50.439975 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:50.436931 23027 rpcz_store.cc:276] 0709 13:08:49.197164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:49.220149 (+ 22985us) service_pool.cc:224] Handling call
0709 13:08:50.338470 (+1118321us) spinlock_profiling.cc:243] Waited 666 ms on lock 0x7b54002ca954. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:50.338606 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1531689728}
I20260709 13:08:50.444801 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:08:50.449409 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:50.394351 23621 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 3.
I20260709 13:08:50.378876 25371 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:50.266306 25065 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:50.461196 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:50.368464 25435 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:50.452071 25305 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:50.467008 25369 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:50.470119 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:50.304293 25444 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:50.448768 25309 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:50.295771 25457 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:50.482345 22920 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:50.626070 22876 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:08:50.522835 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 129905us
W20260709 13:08:50.638926 23588 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.978s (SENT) [suppressed 49 similar messages]
I20260709 13:08:50.639609 23588 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:08:50.640503 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 158223us
I20260709 13:08:50.640848 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:50.641129 22920 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:50.631886 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:50.627205 23020 raft_consensus.cc:2368] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 7. Current term is 8.
W20260709 13:08:50.626787 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 244127us
I20260709 13:08:50.644191 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:50.641933 23027 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33420 (request call id 1544) took 1411 ms. Trace:
I20260709 13:08:50.627558 22871 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:08:50.509407 22953 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58310 (request call id 1550) took 1721 ms (client timeout 1884 ms). Trace:
W20260709 13:08:50.628301 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 233813us
I20260709 13:08:50.645155 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:50.629211 23022 raft_consensus.cc:2393] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 7: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:50.453145 25328 raft_consensus.cc:3055] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:08:50.645292 22955 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 7.
I20260709 13:08:50.654259 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:50.629602 23026 rpcz_store.cc:276] 0709 13:08:49.169833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:49.169963 (+   130us) service_pool.cc:224] Handling call
0709 13:08:49.170315 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:50.121730 (+951415us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:50.324699 (+202969us) spinlock_profiling.cc:243] Waited 948 ms on lock 0x7b540010c554. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:50.324876 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2180291456}
W20260709 13:08:50.630019 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 233609us
I20260709 13:08:50.628116 22949 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 3.
I20260709 13:08:50.623598 22843 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:50.642282 23588 leader_election.cc:400] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 5. Current term is 6.
I20260709 13:08:50.627450 23021 raft_consensus.cc:2393] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 4: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:50.562165 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:50.481997 25252 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
W20260709 13:08:50.648087 22953 rpcz_store.cc:269] 0709 13:08:48.783674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:49.026244 (+242570us) service_pool.cc:224] Handling call
0709 13:08:50.504715 (+1478471us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b540037a5d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:50.504861 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2656856960}
I20260709 13:08:50.563582 25080 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:50.654903 23624 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 8.
I20260709 13:08:50.646235 23027 rpcz_store.cc:276] 0709 13:08:49.230849 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:50.640445 (+1409596us) service_pool.cc:224] Handling call
0709 13:08:50.641860 (+  1415us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:50.657855 22950 raft_consensus.cc:2393] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 6: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:50.655788 23590 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:50.650302 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:50.649688 25446 raft_consensus.cc:2804] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:50.630661 25065 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:50.637285 25291 raft_consensus.cc:2749] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:08:50.656116 23588 leader_election.cc:403] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:50.631708 25410 raft_consensus.cc:2764] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:08:50.705042 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:50.674160 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:50.658985 22954 raft_consensus.cc:1240] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:08:50.653112 25371 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:50.663482 22920 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:50.712024 22949 raft_consensus.cc:2410] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 13:08:50.712237 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 48852.7us
W20260709 13:08:50.702553 25438 consensus_meta.cc:287] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.592s	user 0.006s	sys 0.003s
I20260709 13:08:50.646353 25457 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:50.699105 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:50.700466 22871 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:08:50.670697 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:50.701264 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:50.644207 25447 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:50.702821 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:50.625092 25377 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:50.723824 23028 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:50.721572 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:50.699949 22918 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:50.674360 25451 raft_consensus.cc:2804] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:50.702152 25464 raft_consensus.cc:2804] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:50.655905 25434 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:50.644273 25320 consensus_queue.cc:1048] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.069s
I20260709 13:08:50.703787 25252 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:50.659658 22946 raft_consensus.cc:2368] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 7. Current term is 8.
I20260709 13:08:50.652002 25369 raft_consensus.cc:2804] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:08:50.712138 25318 consensus_meta.cc:287] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.584s	user 0.009s	sys 0.001s
I20260709 13:08:50.725451 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:50.775357 22949 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:50.725160 25451 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:50.653180 25328 raft_consensus.cc:740] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:50.704595 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:50.782294 23028 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 5.
I20260709 13:08:50.783138 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:50.730567 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:08:50.703413 25251 consensus_meta.cc:287] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.658s	user 0.010s	sys 0.000s
I20260709 13:08:50.729156 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:50.738947 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:50.725472 25464 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:50.726402 22955 raft_consensus.cc:2393] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 5: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:50.645140 25394 consensus_queue.cc:1048] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.067s
I20260709 13:08:50.795503 22880 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6.
I20260709 13:08:50.697463 25400 consensus_queue.cc:1059] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 13:08:50.768963 22843 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:50.722738 23027 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 5.
I20260709 13:08:50.483922 25228 consensus_queue.cc:260] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:50.700870 25080 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:50.636948 25305 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:50.697654 25446 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:50.581290 25383 consensus_queue.cc:237] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:50.772284 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:50.801707 23020 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:50.772367 23024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 1533) took 1603 ms. Trace:
I20260709 13:08:50.651126 25309 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:50.772765 22873 raft_consensus.cc:1240] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:08:50.638212 25366 consensus_queue.cc:1059] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 13:08:50.806133 25400 raft_consensus.cc:3055] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:08:50.806690 25400 raft_consensus.cc:740] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:50.806862 25366 raft_consensus.cc:3055] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:08:50.807273 25366 raft_consensus.cc:740] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:50.807534 25400 consensus_queue.cc:260] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:50.808097 25366 consensus_queue.cc:260] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:50.808816 25400 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:50.809306 25366 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:50.776319 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:50.809612 25455 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:50.810199 22946 raft_consensus.cc:2393] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 7: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:50.811031 25309 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:50.811079 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:50.812281 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:08:50.777505 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 80536.4us
I20260709 13:08:50.722682 25447 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:50.722411 25325 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:50.714542 25371 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:50.783672 23028 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:50.765775 25436 raft_consensus.cc:2804] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:08:50.781391 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 81486.9us
I20260709 13:08:50.765769 25229 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:50.786692 22878 raft_consensus.cc:1240] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:08:50.790364 25370 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:50.792523 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 163787us
I20260709 13:08:50.864074 22992 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:50.786470 25251 raft_consensus.cc:2749] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 4. Current term is 5.
I20260709 13:08:50.792801 25463 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:08:50.711535 22876 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:50.772987 25434 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:50.788942 25439 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:50.795087 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:08:50.767867 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 125669us
I20260709 13:08:50.772936 25369 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:50.777357 25458 raft_consensus.cc:2804] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:50.798098 22843 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:50.798569 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:50.907908 22880 raft_consensus.cc:2393] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 7: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:50.765887 25433 consensus_queue.cc:1059] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 13:08:50.782017 25451 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:50.799938 23027 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:08:50.911110 23027 raft_consensus.cc:606] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Received request to transfer leadership to TS aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:50.911612 23027 raft_consensus.cc:614] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:50.800537 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:50.800462 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:50.914240 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:50.768781 25226 consensus_queue.cc:1059] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 13:08:50.802379 23024 rpcz_store.cc:276] 0709 13:08:49.168566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:49.168718 (+   152us) service_pool.cc:224] Handling call
0709 13:08:49.215792 (+ 47074us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:50.235815 (+1020023us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:50.772147 (+536332us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54002c9054. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:50.772305 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2334149504}
I20260709 13:08:50.918268 23024 raft_consensus.cc:1240] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 6 LEADER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:08:50.918884 23024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 1499) took 1562 ms. Trace:
I20260709 13:08:50.919212 23024 rpcz_store.cc:276] 0709 13:08:49.356299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:50.917755 (+1561456us) service_pool.cc:224] Handling call
0709 13:08:50.918079 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:50.918653 (+   574us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:50.918831 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:50.726090 25438 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:50.768143 25471 consensus_queue.cc:1059] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 13:08:50.910293 25370 consensus_queue.cc:1059] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260709 13:08:50.806072 25228 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:50.922698 25438 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:50.925031 25433 raft_consensus.cc:3055] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:50.925514 25433 raft_consensus.cc:740] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
W20260709 13:08:50.812932 25445 log.cc:927] Time spent T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.790s	user 0.000s	sys 0.008s
I20260709 13:08:50.712757 25353 consensus_queue.cc:260] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:50.926949 25353 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:50.926520 25433 consensus_queue.cc:260] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:50.927991 25433 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:50.929039 25226 raft_consensus.cc:3055] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:08:50.929476 25226 raft_consensus.cc:740] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:50.930409 25226 consensus_queue.cc:260] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:50.776108 22877 raft_consensus.cc:1240] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:08:50.931726 25226 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:50.817224 22952 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 1553) took 2032 ms. Trace:
I20260709 13:08:50.794492 25464 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:50.932498 22952 rpcz_store.cc:276] 0709 13:08:48.784498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:49.354352 (+569854us) service_pool.cc:224] Handling call
0709 13:08:49.547999 (+193647us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:49.548575 (+   576us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:50.817060 (+1268485us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b5400284450. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:50.817177 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":444686592}
I20260709 13:08:50.821442 23590 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:50.800154 25080 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:50.799427 25410 raft_consensus.cc:2749] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:50.938917 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:50.892854 25371 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:50.827256 25435 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:50.828027 25440 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:50.828989 25396 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:50.924295 25246 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:50.829886 25305 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:50.829999 25088 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:50.970147 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:50.972213 22994 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:50.833881 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:50.783850 25318 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:50.833475 25328 consensus_queue.cc:260] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:50.871306 25377 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:50.873276 25373 consensus_meta.cc:287] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.722s	user 0.009s	sys 0.000s
W20260709 13:08:50.880898 25384 consensus_meta.cc:287] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.830s	user 0.005s	sys 0.005s
I20260709 13:08:50.880153 25229 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:50.882011 25436 raft_consensus.cc:697] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:50.803736 25446 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:50.890055 22875 raft_consensus.cc:1240] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:08:50.890861 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:50.892293 22992 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
W20260709 13:08:50.990085 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 97854.4us
I20260709 13:08:50.990823 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:50.990958 22992 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:50.893546 23028 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 3.
I20260709 13:08:50.897773 22918 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:50.904978 23020 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
I20260709 13:08:50.995247 23021 raft_consensus.cc:1240] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 13:08:50.995347 22874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53724 (request call id 390) took 1591 ms. Trace:
I20260709 13:08:50.995919 22874 rpcz_store.cc:276] 0709 13:08:49.404199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:49.516441 (+112242us) service_pool.cc:224] Handling call
0709 13:08:49.516808 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:50.200482 (+683674us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:50.995073 (+794591us) spinlock_profiling.cc:243] Waited 676 ms on lock 0x7b5400368654. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:50.995280 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1554716032}
I20260709 13:08:50.905874 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:08:50.833698 25323 consensus_meta.cc:287] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.742s	user 0.008s	sys 0.004s
I20260709 13:08:50.910512 25458 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:50.933701 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:51.000067 22952 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:50.920452 23024 raft_consensus.cc:1240] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:50.935434 23588 leader_election.cc:400] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 7. Current term is 8.
I20260709 13:08:51.000892 23588 leader_election.cc:403] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:08:51.001108 23024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 1501) took 1607 ms. Trace:
W20260709 13:08:51.001364 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 66126.4us
I20260709 13:08:51.001549 23024 rpcz_store.cc:276] 0709 13:08:49.394452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:50.919984 (+1525532us) service_pool.cc:224] Handling call
0709 13:08:50.920277 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:51.000866 (+ 80589us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:51.001060 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:50.955216 25465 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:50.938160 23027 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:60044 (request call id 20) took 1557 ms. Trace:
I20260709 13:08:50.825050 25325 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:50.939790 22880 raft_consensus.cc:2393] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 5: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:50.967793 25455 raft_consensus.cc:2804] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:50.825971 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:08:50.942505 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 144464us
I20260709 13:08:51.017829 22843 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:50.944105 23022 raft_consensus.cc:1240] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:08:51.019126 23022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 1497) took 1663 ms. Trace:
I20260709 13:08:51.019565 23022 rpcz_store.cc:276] 0709 13:08:49.355820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:50.912956 (+1557136us) service_pool.cc:224] Handling call
0709 13:08:50.943939 (+ 30983us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:51.018823 (+ 74884us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:51.019060 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:50.944228 23026 raft_consensus.cc:1240] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 13:08:50.823189 25447 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:51.029084 22872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2035) took 1598 ms. Trace:
I20260709 13:08:50.948951 22920 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:51.030033 23026 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 1558) took 1442 ms. Trace:
W20260709 13:08:51.030450 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 81574.7us
I20260709 13:08:51.030375 23026 rpcz_store.cc:276] 0709 13:08:49.587554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:50.943719 (+1356165us) service_pool.cc:224] Handling call
0709 13:08:50.943992 (+   273us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:51.029788 (+ 85796us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:51.029983 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":11648}
I20260709 13:08:50.910702 25369 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:50.949870 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:51.032711 22843 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:51.029481 22872 rpcz_store.cc:276] 0709 13:08:49.430367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:49.523447 (+ 93080us) service_pool.cc:224] Handling call
0709 13:08:49.549541 (+ 26094us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:49.551601 (+  2060us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:49.551629 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:49.574697 (+ 23068us) log.cc:844] Serialized 26 byte log entry
0709 13:08:49.883578 (+308881us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:49.883632 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:49.883706 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:50.812341 (+928635us) spinlock_profiling.cc:243] Waited 23.3 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:08:51.006056 (+193715us) raft_consensus.cc:1685] finished
0709 13:08:51.006088 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:51.029022 (+ 22934us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8756,"raft.queue_time_us":52443,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":32408,"spinlock_wait_cycles":53545216,"thread_start_us":282945,"threads_started":1,"wal-append.queue_time_us":406638}
I20260709 13:08:51.033800 22843 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:50.940770 25472 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:50.951787 25402 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:50.944036 25406 raft_consensus.cc:2749] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:50.965678 22844 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:50.995110 25418 raft_consensus.cc:2764] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:08:50.973475 22994 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:08:51.055951 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 82570.2us
I20260709 13:08:50.984403 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:50.992362 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:51.004015 25458 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:51.002179 23588 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:50.935510 23023 consensus_meta.cc:287] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.556s	user 0.010s	sys 0.000s
I20260709 13:08:51.059012 23023 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 7.
I20260709 13:08:51.003453 25363 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:51.060322 23023 raft_consensus.cc:1240] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:08:51.061010 23023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 424) took 1264 ms. Trace:
I20260709 13:08:51.006619 23020 raft_consensus.cc:1240] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:08:50.981030 25383 consensus_queue.cc:1048] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 13:08:51.062175 23020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 419) took 1288 ms. Trace:
I20260709 13:08:51.015319 25377 raft_consensus.cc:2749] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:08:50.977833 25420 consensus_queue.cc:1059] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:08:51.004933 23027 rpcz_store.cc:276] 0709 13:08:49.355507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:50.799823 (+1444316us) service_pool.cc:224] Handling call
0709 13:08:50.912099 (+112276us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:08:51.028229 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 53717.5us
I20260709 13:08:51.034226 22955 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:08:50.987883 25414 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:08:51.040123 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:08:50.967686 23029 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33418 (request call id 1507) took 1484 ms (client timeout 1896 ms). Trace:
I20260709 13:08:51.051812 22952 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
W20260709 13:08:51.075297 23029 rpcz_store.cc:269] 0709 13:08:49.482898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:49.937293 (+454395us) service_pool.cc:224] Handling call
0709 13:08:50.967450 (+1030157us) spinlock_profiling.cc:243] Waited 275 ms on lock 0x7b5400359154. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:50.967614 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":633120512}
I20260709 13:08:51.076092 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:51.077054 22952 raft_consensus.cc:2393] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 5: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:51.058066 23022 raft_consensus.cc:1240] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:51.062695 23020 rpcz_store.cc:276] 0709 13:08:49.775747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:51.006015 (+1230268us) service_pool.cc:224] Handling call
0709 13:08:51.006396 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:51.061864 (+ 55468us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:51.062113 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:08:51.007445 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 109732us
I20260709 13:08:51.078837 22918 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:51.061429 23023 rpcz_store.cc:276] 0709 13:08:49.797989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:51.059750 (+1261761us) service_pool.cc:224] Handling call
0709 13:08:51.060110 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:51.060795 (+   685us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:51.060960 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:08:51.064955 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 62842.5us
I20260709 13:08:51.082518 23588 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:51.027994 25455 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:51.045070 25050 consensus_meta.cc:287] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.601s	user 0.005s	sys 0.005s
I20260709 13:08:51.028091 25241 raft_consensus.cc:2749] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:08:51.042361 25309 raft_consensus.cc:2749] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:08:50.987401 25252 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:51.041862 25372 raft_consensus.cc:2804] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:08:51.071416 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:51.133147 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:51.073062 23027 raft_consensus.cc:1240] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:08:51.133534 23020 raft_consensus.cc:1240] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 13:08:51.134325 23027 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 425) took 1292 ms. Trace:
I20260709 13:08:51.134471 23020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 1562) took 1024 ms. Trace:
I20260709 13:08:51.134853 23027 rpcz_store.cc:276] 0709 13:08:49.842414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:51.072515 (+1230101us) service_pool.cc:224] Handling call
0709 13:08:51.072832 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:51.133975 (+ 61143us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:51.134251 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:51.135003 23020 rpcz_store.cc:276] 0709 13:08:50.110158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:51.077710 (+967552us) service_pool.cc:224] Handling call
0709 13:08:51.133223 (+ 55513us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:51.134114 (+   891us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:51.134400 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:51.069386 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:51.041397 25366 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:51.061604 25362 raft_consensus.cc:2749] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:08:51.074795 22950 raft_consensus.cc:2376] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8. Re-sending same reply.
I20260709 13:08:50.995843 25373 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:51.085683 23022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 421) took 1309 ms. Trace:
I20260709 13:08:51.143487 23022 rpcz_store.cc:276] 0709 13:08:49.776289 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:51.020515 (+1244226us) service_pool.cc:224] Handling call
0709 13:08:51.057895 (+ 37380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:51.077535 (+ 19640us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:51.085627 (+  8092us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:51.085717 23029 raft_consensus.cc:1275] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:08:51.021698 25444 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:50.993958 25384 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:51.086854 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 265482us
I20260709 13:08:51.146517 23021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 1555) took 1868 ms. Trace:
I20260709 13:08:51.146906 23590 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:51.146888 23021 rpcz_store.cc:276] 0709 13:08:49.278262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:50.812981 (+1534719us) service_pool.cc:224] Handling call
0709 13:08:50.992614 (+179633us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:51.001300 (+  8686us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:51.146338 (+145038us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b54001798d0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:51.146464 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":412365696}
W20260709 13:08:51.086943 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 96081.9us
I20260709 13:08:51.150717 22992 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:51.080255 22918 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:08:51.151808 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 71639.5us
I20260709 13:08:51.084472 23588 leader_election.cc:400] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 election: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 7. Current term is 8.
W20260709 13:08:51.152882 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 68586.4us
I20260709 13:08:51.014765 25323 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:51.045708 25440 consensus_queue.cc:1059] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:08:51.070689 25388 raft_consensus.cc:2804] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:51.063324 25088 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:51.144680 23029 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 1518) took 1036 ms. Trace:
I20260709 13:08:51.174207 23029 rpcz_store.cc:276] 0709 13:08:50.108215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:51.076068 (+967853us) service_pool.cc:224] Handling call
0709 13:08:51.076303 (+   235us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:51.144421 (+ 68118us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:51.144605 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:51.152616 22918 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:08:51.173976 25088 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:51.176154 25088 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:51.157573 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:51.158124 23590 leader_election.cc:400] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 election: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 7. Current term is 8.
I20260709 13:08:51.181665 22952 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 5.
I20260709 13:08:51.129972 25414 raft_consensus.cc:697] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:51.159953 23028 raft_consensus.cc:1240] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 6. Current term is 7. Ops: []
I20260709 13:08:51.071660 25406 raft_consensus.cc:2804] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:08:51.088181 25455 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:51.190948 23026 raft_consensus.cc:1240] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:51.033109 25465 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:51.085809 25405 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 7. Current term is 8.
I20260709 13:08:51.065815 25363 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:51.142966 25482 raft_consensus.cc:2749] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:08:51.171480 22992 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:08:51.223203 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 51997.8us
I20260709 13:08:51.189688 22949 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:08:51.141664 25372 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:51.044865 25472 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:51.190613 25406 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:51.184752 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:51.145830 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:51.138140 25436 consensus_queue.cc:237] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:51.216434 25439 raft_consensus.cc:2804] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:08:51.211678 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 145565us
I20260709 13:08:51.212888 22948 raft_consensus.cc:2393] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 4: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:51.143030 25366 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:51.087998 25435 raft_consensus.cc:2749] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:08:51.217432 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 183681us
I20260709 13:08:51.164136 25377 raft_consensus.cc:2804] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:08:51.226871 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:51.232832 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:51.257051 22949 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:51.257781 22950 raft_consensus.cc:2410] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:08:51.166361 25444 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:08:51.089217 25050 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:51.235634 22992 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:08:51.200285 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 47742us
I20260709 13:08:51.192317 25484 raft_consensus.cc:3055] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:08:51.242081 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:51.237314 25418 raft_consensus.cc:2749] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:51.260653 22952 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6.
W20260709 13:08:51.260914 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 78568.8us
I20260709 13:08:51.261650 25318 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:51.244268 23023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 1565) took 1046 ms. Trace:
I20260709 13:08:51.157349 25471 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:51.236575 25406 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:51.214548 25388 raft_consensus.cc:697] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:51.250380 22843 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:51.252478 25465 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:51.258330 23024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 420) took 1480 ms. Trace:
W20260709 13:08:51.243777 25478 log.cc:927] Time spent T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.216s	user 0.000s	sys 0.001s
I20260709 13:08:51.236843 25372 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:51.193495 25320 raft_consensus.cc:2749] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:51.264004 22918 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:51.263892 23023 rpcz_store.cc:276] 0709 13:08:50.193755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:51.089575 (+895820us) service_pool.cc:224] Handling call
0709 13:08:51.089869 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:51.090063 (+   194us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:08:51.140021 (+ 49958us) log.cc:844] Serialized 14 byte log entry
0709 13:08:51.239546 (+ 99525us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:51.239603 (+    57us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:51.239654 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:51.239723 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:51.239843 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2164,"prepare.queue_time_us":52748,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":13464,"thread_start_us":97256,"threads_started":1}
I20260709 13:08:51.266110 23590 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:51.307006 25251 raft_consensus.cc:3055] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:51.309319 22946 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
I20260709 13:08:51.375699 22946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58320 (request call id 2137) took 1019 ms. Trace:
I20260709 13:08:51.376786 22946 rpcz_store.cc:276] 0709 13:08:50.356177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:50.984065 (+627888us) service_pool.cc:224] Handling call
0709 13:08:51.375635 (+391570us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:08:51.377357 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 127039us
I20260709 13:08:51.378057 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:51.378153 22843 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:51.378309 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 114378us
I20260709 13:08:51.256610 25366 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:51.378055 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:51.249750 25439 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:51.374292 23024 rpcz_store.cc:276] 0709 13:08:49.776036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:51.003546 (+1227510us) service_pool.cc:224] Handling call
0709 13:08:51.038992 (+ 35446us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:51.048911 (+  9919us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:51.048955 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:51.059093 (+ 10138us) log.cc:844] Serialized 26 byte log entry
0709 13:08:51.249438 (+190345us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:51.250041 (+   603us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:51.250109 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:51.256162 (+  6053us) raft_consensus.cc:1685] finished
0709 13:08:51.256181 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:51.256326 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9435,"raft.queue_time_us":10116,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":8086,"thread_start_us":100057,"threads_started":1,"wal-append.queue_time_us":164700}
I20260709 13:08:51.259040 25383 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:51.392349 22843 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:51.345647 23616 raft_consensus.cc:2368] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 7. Current term is 8.
I20260709 13:08:51.398566 22954 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 5.
I20260709 13:08:51.399108 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:51.399437 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:51.399653 22950 raft_consensus.cc:2376] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 5. Re-sending same reply.
I20260709 13:08:51.265997 25484 raft_consensus.cc:740] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:51.400741 22954 raft_consensus.cc:1240] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:08:51.262902 25246 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:51.258762 25495 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:51.308506 25229 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:51.309003 25444 raft_consensus.cc:697] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 6 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:51.426036 22954 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2136) took 1070 ms. Trace:
I20260709 13:08:51.308523 25377 raft_consensus.cc:697] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:51.428473 22954 rpcz_store.cc:276] 0709 13:08:50.355998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:51.400197 (+1044199us) service_pool.cc:224] Handling call
0709 13:08:51.400502 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:51.425757 (+ 25255us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:51.425980 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:51.389940 25439 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:51.387498 25440 raft_consensus.cc:2804] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:51.390105 25325 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:51.393707 22843 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:51.375330 25251 raft_consensus.cc:740] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
W20260709 13:08:51.436632 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 471055us
W20260709 13:08:51.401919 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 166384us
I20260709 13:08:51.466792 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:51.467499 22872 raft_consensus.cc:2393] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 5: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:51.437705 22954 raft_consensus.cc:1240] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 13:08:51.311690 25370 consensus_queue.cc:1059] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.115s
I20260709 13:08:51.474313 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:51.453884 25440 raft_consensus.cc:697] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:51.400311 25309 raft_consensus.cc:2804] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:51.389259 25486 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:08:51.430323 25475 log.cc:927] Time spent T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.375s	user 0.001s	sys 0.000s
I20260709 13:08:51.451323 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:51.399683 25228 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:51.452706 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:08:51.460227 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 66583.1us
I20260709 13:08:51.495764 22843 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:51.460750 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:51.496960 23024 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:51.385334 25400 consensus_queue.cc:1059] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.062s
I20260709 13:08:51.401561 25485 raft_consensus.cc:2749] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:51.456832 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:51.273353 25471 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:51.502419 25471 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:51.468290 22844 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:51.431543 25492 raft_consensus.cc:2804] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:08:51.503307 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:51.503860 25492 raft_consensus.cc:697] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 8 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:51.437222 25402 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:51.505019 25402 consensus_queue.cc:1059] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:08:51.505173 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:51.505884 25402 consensus_queue.cc:1059] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:08:51.474812 22954 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 362) took 1034 ms. Trace:
I20260709 13:08:51.507026 22954 rpcz_store.cc:276] 0709 13:08:50.440858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:51.437146 (+996288us) service_pool.cc:224] Handling call
0709 13:08:51.437500 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:51.467886 (+ 30386us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:51.474761 (+  6875us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:51.437673 25434 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:51.394150 25383 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:51.431586 25487 raft_consensus.cc:2749] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:08:51.347322 25414 consensus_queue.cc:237] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:51.517464 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:51.488685 23020 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ec334666b7964c969125af95d9c797ed"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:08:51.498512 22843 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:51.465312 25305 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:51.465543 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:51.465842 25410 raft_consensus.cc:2749] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:08:51.461741 25252 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:51.520571 23020 raft_consensus.cc:606] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 5 LEADER]: Received request to transfer leadership to TS aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:51.486459 25309 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:51.436784 25495 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:51.528522 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:08:51.510494 25080 consensus_meta.cc:287] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.565s	user 0.010s	sys 0.000s
I20260709 13:08:51.497539 25323 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:51.521369 25384 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:51.508380 25373 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:51.510067 25492 consensus_queue.cc:237] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:51.519940 25370 raft_consensus.cc:3055] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:08:51.556453 25476 log.cc:927] Time spent T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.371s	user 0.000s	sys 0.001s
I20260709 13:08:51.544386 25305 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:51.498711 25428 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
W20260709 13:08:51.562752 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 87117.4us
W20260709 13:08:51.551430 25353 consensus_meta.cc:287] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.619s	user 0.009s	sys 0.001s
I20260709 13:08:51.510480 25465 consensus_queue.cc:1059] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.044s
W20260709 13:08:51.593761 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:51.500584 25228 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:51.553613 25309 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:51.553470 23618 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 6.
I20260709 13:08:51.621498 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:51.559768 25080 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:51.557171 25496 raft_consensus.cc:2749] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:08:51.530481 25485 raft_consensus.cc:2804] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:51.487314 25388 consensus_queue.cc:237] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:51.631086 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:51.495978 25484 consensus_queue.cc:260] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:51.645694 22955 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58320 (request call id 2135) took 1290 ms. Trace:
W20260709 13:08:51.650035 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 151581us
I20260709 13:08:51.668447 25472 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:08:51.672245 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 406178us
W20260709 13:08:51.628826 25488 log.cc:927] Time spent T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.358s	user 0.001s	sys 0.000s
I20260709 13:08:51.682476 23024 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6.
I20260709 13:08:51.657359 25050 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:51.550635 25377 consensus_queue.cc:237] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:51.527017 25251 consensus_queue.cc:260] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:51.556944 25444 consensus_queue.cc:237] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:51.669526 25373 raft_consensus.cc:3055] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:08:51.692366 22955 rpcz_store.cc:276] 0709 13:08:50.355747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:50.913919 (+558172us) service_pool.cc:224] Handling call
0709 13:08:51.645414 (+731495us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b540044b954. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:51.645601 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":273954048}
I20260709 13:08:51.692826 22843 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:51.651989 25370 raft_consensus.cc:740] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
W20260709 13:08:51.699379 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:51.694002 22950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2140) took 1307 ms. Trace:
W20260709 13:08:51.711099 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:51.618435 25440 consensus_queue.cc:237] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:51.683455 25457 consensus_meta.cc:287] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.912s	user 0.010s	sys 0.017s
I20260709 13:08:51.705559 22918 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:51.711012 22955 raft_consensus.cc:1240] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:51.705439 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:08:51.689052 25491 log.cc:927] Time spent T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.282s	user 0.000s	sys 0.001s
W20260709 13:08:51.713310 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:51.686221 25485 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:51.711010 22950 rpcz_store.cc:276] 0709 13:08:50.386989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:51.400237 (+1013248us) service_pool.cc:224] Handling call
0709 13:08:51.438413 (+ 38176us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:51.438609 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:08:51.501022 (+ 62413us) log.cc:844] Serialized 14 byte log entry
0709 13:08:51.693432 (+192410us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:51.693504 (+    72us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:08:51.693663 (+   159us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:51.693765 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:51.693930 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":93841,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":3574,"raft.queue_time_us":9315,"thread_start_us":252597,"threads_started":2}
I20260709 13:08:51.734318 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:51.732975 22955 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 364) took 1260 ms. Trace:
I20260709 13:08:51.718429 22843 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:51.685716 23618 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 592) took 1452 ms (client timeout 1817 ms). Trace:
W20260709 13:08:51.709472 25065 consensus_meta.cc:287] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.998s	user 0.011s	sys 0.015s
I20260709 13:08:51.735863 22955 rpcz_store.cc:276] 0709 13:08:50.472861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:51.710525 (+1237664us) service_pool.cc:224] Handling call
0709 13:08:51.710836 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:51.732654 (+ 21818us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:51.732911 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:51.738626 22955 raft_consensus.cc:1240] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:08:51.740408 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:51.741544 22878 raft_consensus.cc:2393] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 5: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:51.741667 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:08:51.741055 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 229393us
I20260709 13:08:51.742995 22955 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 366) took 1242 ms. Trace:
I20260709 13:08:51.743360 22955 rpcz_store.cc:276] 0709 13:08:50.500249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:51.738032 (+1237783us) service_pool.cc:224] Handling call
0709 13:08:51.738393 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:51.742771 (+  4378us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:51.742947 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:08:51.743690 22844 leader_election.cc:336] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Timed out: RequestConsensusVote RPC to 127.21.63.4:44043 timed out after 2.060s (SENT)
W20260709 13:08:51.753511 25466 log.cc:927] Time spent T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.886s	user 0.000s	sys 0.010s
W20260709 13:08:51.753926 25489 log.cc:927] Time spent T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.262s	user 0.001s	sys 0.000s
I20260709 13:08:51.752987 25508 raft_consensus.cc:3055] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:08:51.735034 25251 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:51.756287 22950 raft_consensus.cc:1240] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:08:51.797365 22950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 365) took 1314 ms. Trace:
I20260709 13:08:51.797730 22950 rpcz_store.cc:276] 0709 13:08:50.483130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:51.735590 (+1252460us) service_pool.cc:224] Handling call
0709 13:08:51.756094 (+ 20504us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:51.797081 (+ 40987us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:51.797303 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:51.754390 25507 raft_consensus.cc:3055] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:08:51.801707 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:51.713930 25050 raft_consensus.cc:2749] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:51.758334 22843 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:51.712827 25373 raft_consensus.cc:740] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
W20260709 13:08:51.811685 25503 log.cc:927] Time spent T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 13:08:51.714079 25353 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:51.813091 25353 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:51.813742 25507 raft_consensus.cc:740] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:51.814169 25495 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:51.756851 25496 raft_consensus.cc:2749] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:08:51.741143 25400 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:08:51.738438 25485 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:08:51.737178 23618 rpcz_store.cc:269] 0709 13:08:50.230897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:50.232115 (+  1218us) service_pool.cc:224] Handling call
0709 13:08:51.455186 (+1223071us) spinlock_profiling.cc:243] Waited 2.41 ms on lock 0x7b58001c4d98. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0a22 00007f613bbd9c09 00007f613ad33ee0 00007f613adfd0ab 00007f613adfd131 00007f613adab0a0 00007f613adaa984 00007f61343256d4 00007f6134325662
0709 13:08:51.683551 (+228365us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5538048}
I20260709 13:08:51.747812 25383 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:51.815492 22950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2144) took 1190 ms. Trace:
I20260709 13:08:51.840202 22950 rpcz_store.cc:276] 0709 13:08:50.625464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:51.798527 (+1173063us) service_pool.cc:224] Handling call
0709 13:08:51.814973 (+ 16446us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:51.815147 (+   174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:51.815169 (+    22us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:51.815222 (+    53us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:51.815255 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:51.815328 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:51.815440 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:51.744872 22955 raft_consensus.cc:1240] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 4. Current term is 5. Ops: []
W20260709 13:08:51.811735 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 53492.2us
I20260709 13:08:51.842577 22843 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:08:51.831740 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 88080.3us
I20260709 13:08:51.832316 25353 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:51.835067 23024 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:60044
W20260709 13:08:51.836644 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 163614us
I20260709 13:08:51.742129 25065 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:51.739246 25465 consensus_queue.cc:1059] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 13:08:51.845849 22880 raft_consensus.cc:1275] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:08:51.846305 23588 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:51.846776 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 152848us
I20260709 13:08:51.810575 25508 raft_consensus.cc:740] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:51.790704 25410 consensus_queue.cc:1048] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.043s
I20260709 13:08:51.849517 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:51.850486 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:51.868883 22876 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:51.869117 23020 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 7.
I20260709 13:08:51.852761 22955 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 367) took 1323 ms. Trace:
W20260709 13:08:51.854337 25500 log.cc:927] Time spent T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.278s	user 0.001s	sys 0.000s
I20260709 13:08:51.854624 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:08:51.856004 25499 log.cc:927] Time spent T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
I20260709 13:08:51.746699 25457 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:51.857620 25303 raft_consensus.cc:2749] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:08:51.861815 25371 consensus_meta.cc:287] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.908s	user 0.009s	sys 0.002s
I20260709 13:08:51.862183 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:51.890882 22843 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:51.891695 22880 raft_consensus.cc:2410] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
W20260709 13:08:51.861821 25088 consensus_meta.cc:287] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.685s	user 0.006s	sys 0.009s
I20260709 13:08:51.863619 25366 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:51.848636 25505 raft_consensus.cc:2749] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:08:51.880892 25506 log.cc:927] Time spent T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 13:08:51.881223 25446 consensus_meta.cc:287] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.860s	user 0.017s	sys 0.000s
I20260709 13:08:51.880045 22955 rpcz_store.cc:276] 0709 13:08:50.524546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:51.744316 (+1219770us) service_pool.cc:224] Handling call
0709 13:08:51.744642 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:51.841688 (+ 97046us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:51.847879 (+  6191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:51.883812 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:51.902561 23020 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 8.
W20260709 13:08:51.890532 25464 consensus_meta.cc:287] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.956s	user 0.004s	sys 0.008s
I20260709 13:08:51.883805 23025 raft_consensus.cc:2368] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 3. Current term is 4.
W20260709 13:08:51.857760 25369 consensus_meta.cc:287] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.803s	user 0.007s	sys 0.002s
I20260709 13:08:51.839560 25400 raft_consensus.cc:697] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:51.854492 25353 raft_consensus.cc:2804] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:51.889086 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:51.853215 25305 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:51.903813 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:51.923978 23020 raft_consensus.cc:2393] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 6: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:51.900410 25228 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:51.820739 25370 consensus_queue.cc:260] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:51.909001 22954 raft_consensus.cc:1240] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:08:51.814662 25507 consensus_queue.cc:260] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:51.904534 22951 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
I20260709 13:08:51.893399 25496 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:51.925076 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:51.884634 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:51.936111 25075 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:51.945150 25075 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:51.906169 25080 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:51.835731 25373 consensus_queue.cc:260] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:51.918248 25353 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:51.895224 25088 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:51.916981 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:51.898159 25371 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:51.909022 25318 consensus_queue.cc:1048] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:08:51.959224 25509 consensus_queue.cc:1059] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 13:08:51.937736 25305 raft_consensus.cc:2804] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:08:51.905221 25446 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:51.965529 25226 consensus_meta.cc:287] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 1.033s	user 0.011s	sys 0.009s
I20260709 13:08:51.972365 23026 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 423) took 2184 ms. Trace:
I20260709 13:08:51.993503 23026 rpcz_store.cc:276] 0709 13:08:49.788437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:51.031260 (+1242823us) service_pool.cc:224] Handling call
0709 13:08:51.031512 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:51.191337 (+159825us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:51.972056 (+780719us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b54002c9f54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:51.972290 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":366266752}
I20260709 13:08:51.994581 22920 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:51.995338 23026 raft_consensus.cc:1240] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:08:51.995939 22920 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:51.977098 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:51.956074 23021 raft_consensus.cc:1275] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:08:51.962167 25353 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:51.917306 25369 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:52.004101 25369 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:52.004971 22947 raft_consensus.cc:2368] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 6. Current term is 7.
I20260709 13:08:51.966830 25370 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:51.961005 25496 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:52.014293 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:52.000747 25436 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:52.001232 25080 raft_consensus.cc:2749] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:08:51.909361 25464 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:51.960577 25512 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:51.955401 25463 consensus_meta.cc:287] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 1.058s	user 0.006s	sys 0.005s
I20260709 13:08:51.937681 25439 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:52.008975 22876 raft_consensus.cc:2468] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 3.
I20260709 13:08:52.012022 22953 raft_consensus.cc:2393] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 7: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:52.014494 22952 raft_consensus.cc:2410] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:08:51.988792 25435 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:52.006176 25228 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:51.948344 25075 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:51.980324 25291 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:52.017272 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 126563us
W20260709 13:08:51.965529 25451 consensus_meta.cc:287] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 1.025s	user 0.011s	sys 0.000s
W20260709 13:08:52.014997 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 140646us
W20260709 13:08:52.018605 25406 consensus_meta.cc:287] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.643s	user 0.006s	sys 0.005s
W20260709 13:08:52.019090 25433 consensus_meta.cc:287] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 1.091s	user 0.011s	sys 0.004s
I20260709 13:08:52.028062 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:52.013502 25369 raft_consensus.cc:2749] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:08:52.026436 22994 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:52.050433 22946 raft_consensus.cc:2410] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:08:52.032754 25435 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:52.033253 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:52.024375 25050 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:52.039304 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:52.073982 23622 raft_consensus.cc:2368] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 4. Current term is 5.
I20260709 13:08:52.040432 22843 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:52.041698 25291 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:52.042073 25075 consensus_queue.cc:1059] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:52.042429 25451 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:52.042761 25504 consensus_queue.cc:1059] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:08:52.043017 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:52.043442 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:52.032775 25464 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:52.043524 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:52.016347 25305 raft_consensus.cc:697] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 3 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:52.044467 23021 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:08:52.044831 23025 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33418 (request call id 1522) took 1935 ms (client timeout 1946 ms). Trace:
I20260709 13:08:52.052977 22872 raft_consensus.cc:1240] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 5. Current term is 6. Ops: []
I20260709 13:08:52.053469 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:08:52.037127 25455 consensus_meta.cc:287] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.827s	user 0.010s	sys 0.012s
I20260709 13:08:52.026021 25463 raft_consensus.cc:2749] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 5. Current term is 6.
I20260709 13:08:52.077189 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:52.079666 22843 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:08:52.071457 25458 consensus_meta.cc:287] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.997s	user 0.012s	sys 0.015s
I20260709 13:08:51.992877 25400 consensus_queue.cc:237] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:52.020356 25436 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:52.043708 25512 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:52.044081 25228 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:52.061488 25406 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:52.110704 22871 raft_consensus.cc:2393] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 4: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:52.080706 23621 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
I20260709 13:08:52.081342 25050 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:52.194868 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:52.308773 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:52.320348 22949 raft_consensus.cc:1275] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 13:08:52.320919 22949 pending_rounds.cc:85] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting all ops after (but not including) 2
I20260709 13:08:52.321152 22949 pending_rounds.cc:107] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 13:08:52.321369 22949 raft_consensus.cc:2889] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:52.321275 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:52.322909 25305 consensus_queue.cc:237] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:52.323504 25436 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:52.324106 25458 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:52.182673 25447 consensus_meta.cc:287] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 1.135s	user 0.001s	sys 0.010s
I20260709 13:08:52.325310 25447 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:52.186466 25396 raft_consensus.cc:3055] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:08:52.326618 25458 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:52.327205 23623 raft_consensus.cc:2393] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 7: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:52.116487 25510 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:52.169255 25518 log.cc:927] Time spent T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 13:08:52.327888 25510 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:52.126720 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:52.329592 25447 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:52.329768 25487 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:52.330636 25510 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:52.330344 25487 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:52.220641 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:52.331367 25505 raft_consensus.cc:2804] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:52.106878 25366 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:52.331734 25505 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:52.178278 25291 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:52.332258 25505 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:08:52.333102 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 253528us
I20260709 13:08:52.330461 25455 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:52.168380 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:52.334806 23023 raft_consensus.cc:1240] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
W20260709 13:08:52.095582 23025 rpcz_store.cc:269] 0709 13:08:50.109863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:50.333382 (+223519us) service_pool.cc:224] Handling call
0709 13:08:52.044616 (+1711234us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b5400135454. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:52.044771 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3509247360}
I20260709 13:08:52.316648 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:52.131440 23021 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
W20260709 13:08:52.319242 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:52.093089 25434 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:08:52.073452 25418 raft_consensus.cc:2804] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 13:08:52.201897 25499 log.cc:927] Time spent T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 13:08:52.327025 25396 raft_consensus.cc:740] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:52.341944 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:08:52.333990 25428 consensus_meta.cc:287] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.663s	user 0.005s	sys 0.009s
W20260709 13:08:52.190584 25372 consensus_meta.cc:287] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.810s	user 0.015s	sys 0.011s
I20260709 13:08:52.182030 25496 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:52.348706 23625 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
I20260709 13:08:52.351593 22874 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:52.114655 25519 consensus_queue.cc:1048] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.226s
I20260709 13:08:52.092614 25065 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:52.339475 25075 consensus_queue.cc:1059] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.007s
I20260709 13:08:52.332849 25487 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:52.338037 25388 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:52.319664 23027 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
I20260709 13:08:52.349033 25363 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:52.324520 25228 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:52.319167 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:52.376986 23620 raft_consensus.cc:2393] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 7: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:52.377434 25446 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:52.347091 25440 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:52.382489 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:52.354295 25418 raft_consensus.cc:697] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:52.402096 23025 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:52.383275 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:52.351815 25434 raft_consensus.cc:697] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 7 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
W20260709 13:08:52.357573 25486 consensus_meta.cc:287] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.856s	user 0.012s	sys 0.002s
W20260709 13:08:52.390607 25485 consensus_meta.cc:287] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.543s	user 0.000s	sys 0.011s
I20260709 13:08:52.352928 25428 raft_consensus.cc:2749] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:08:52.187073 25439 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:52.176204 25495 consensus_queue.cc:1059] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.181s
I20260709 13:08:52.392150 25328 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:52.408316 23617 raft_consensus.cc:2393] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 4: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:52.409358 25451 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:52.388796 25088 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:52.421518 23026 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:08:52.359189 25372 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:52.395121 25435 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:52.434685 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:52.451906 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:52.421186 25486 raft_consensus.cc:2749] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:52.395197 25388 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:52.470316 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:52.471549 23617 raft_consensus.cc:2393] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 5: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:52.471927 25512 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:52.407758 25396 consensus_queue.cc:260] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:52.472692 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:52.473453 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:52.480957 25434 consensus_queue.cc:237] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:52.487949 25251 consensus_meta.cc:287] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.688s	user 0.012s	sys 0.001s
I20260709 13:08:52.445784 23619 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6.
I20260709 13:08:52.410451 25440 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:52.502828 25440 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:52.424064 25485 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:52.507469 25388 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:52.611215 25353 consensus_meta.cc:287] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.609s	user 0.005s	sys 0.005s
I20260709 13:08:52.644603 25507 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:08:52.461503 25515 log.cc:927] Time spent T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.295s	user 0.001s	sys 0.001s
I20260709 13:08:52.645370 25353 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:52.629892 25513 raft_consensus.cc:3055] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:08:52.520905 25521 raft_consensus.cc:3055] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:08:52.527673 25496 raft_consensus.cc:3055] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:08:52.525137 25418 consensus_queue.cc:237] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:52.650851 25373 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:52.651328 25485 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:52.466715 25471 consensus_meta.cc:287] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.964s	user 0.017s	sys 0.026s
I20260709 13:08:52.644603 25484 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:52.608618 25505 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:52.645296 25455 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:52.654947 25521 raft_consensus.cc:740] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:52.658809 25513 raft_consensus.cc:740] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
W20260709 13:08:52.599448 25524 log.cc:927] Time spent T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:08:52.612708 25457 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:52.704902 22844 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.697s	user 0.005s	sys 0.007s
I20260709 13:08:52.641712 25406 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:52.808000 22844 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.801s	user 0.006s	sys 0.007s
W20260709 13:08:52.706214 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 370976us
I20260709 13:08:52.375497 22874 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
W20260709 13:08:52.707612 22880 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.629s	user 0.000s	sys 0.015s
I20260709 13:08:52.708048 22879 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:52.709797 22878 raft_consensus.cc:1240] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 8. Current term is 9. Ops: []
W20260709 13:08:52.710860 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:52.714058 22947 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58320 (request call id 2143) took 2271 ms (client timeout 1979 ms). Trace:
I20260709 13:08:52.715507 22950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2146) took 2077 ms. Trace:
I20260709 13:08:52.719451 22954 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 363) took 2275 ms. Trace:
W20260709 13:08:52.720902 22951 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58320 (request call id 2141) took 2328 ms (client timeout 1980 ms). Trace:
W20260709 13:08:52.721196 22993 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.004s	sys 0.007s
W20260709 13:08:52.721796 22992 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.777s	user 0.000s	sys 0.022s
W20260709 13:08:52.722962 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:52.726708 22918 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:08:52.729457 22948 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58320 (request call id 2129) took 2388 ms (client timeout 1612 ms). Trace:
I20260709 13:08:52.729889 22952 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58310 (request call id 1580) took 1989 ms. Trace:
I20260709 13:08:52.737562 23023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 1577) took 1282 ms. Trace:
I20260709 13:08:52.740549 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:52.745158 23022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:32782 (request call id 10) took 2879 ms. Trace:
I20260709 13:08:52.745581 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:52.746119 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:52.747805 23029 raft_consensus.cc:2410] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 13:08:52.775346 23588 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.005s	sys 0.033s
W20260709 13:08:52.778861 23616 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 590) took 2548 ms (client timeout 2060 ms). Trace:
W20260709 13:08:52.780241 23623 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 595) took 2545 ms (client timeout 2033 ms). Trace:
W20260709 13:08:52.784076 22871 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35360 (request call id 1557) took 2005 ms (client timeout 2053 ms). Trace:
I20260709 13:08:52.784950 22946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58310 (request call id 1582) took 2030 ms. Trace:
I20260709 13:08:52.786813 22949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2152) took 2061 ms. Trace:
W20260709 13:08:52.785808 22953 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52222 (request call id 358) took 2416 ms (client timeout 2075 ms). Trace:
I20260709 13:08:52.790907 25050 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:52.791937 23617 raft_consensus.cc:1240] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 5. Current term is 6. Ops: []
W20260709 13:08:52.794988 22919 connection.cc:256] RPC call timeout handler was delayed by -1.2263116849999278s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2191s for a response
I20260709 13:08:52.798563 23028 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 1561) took 3206 ms. Trace:
I20260709 13:08:52.515272 25438 raft_consensus.cc:3055] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:08:52.656862 25496 raft_consensus.cc:740] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
W20260709 13:08:52.826287 23588 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.006s	sys 0.033s
I20260709 13:08:52.512141 25508 consensus_queue.cc:260] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:52.821754 23616 rpcz_store.cc:269] 0709 13:08:50.230196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:50.231681 (+  1485us) service_pool.cc:224] Handling call
0709 13:08:51.069226 (+837545us) spinlock_profiling.cc:243] Waited 1.79 ms on lock 0x7b58001c4d98. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0a22 00007f613bbd9c09 00007f613ad33ee0 00007f613adfd0ab 00007f613adfd131 00007f613adab0a0 00007f613adaa984 00007f61343256d4 00007f6134325662
0709 13:08:52.778212 (+1708986us) spinlock_profiling.cc:243] Waited 92.2 ms on lock 0x7b54001108d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:52.778552 (+   340us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":216222848}
I20260709 13:08:52.833555 25508 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:52.833879 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:52.835278 23616 raft_consensus.cc:2393] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 4: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:52.836107 23026 raft_consensus.cc:2410] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 9 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 13:08:52.822666 23623 rpcz_store.cc:269] 0709 13:08:50.234484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:50.236042 (+  1558us) service_pool.cc:224] Handling call
0709 13:08:50.645021 (+408979us) spinlock_profiling.cc:243] Waited 18.6 ms on lock 0x7b58001c4d98. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0a22 00007f613bbd9c09 00007f613ad33ee0 00007f613adfd0ab 00007f613adfd131 00007f613adab0a0 00007f613adaa984 00007f61343256d4 00007f6134325662
0709 13:08:52.611108 (+1966087us) spinlock_profiling.cc:243] Waited 1.68 s on lock 0x7b540045d654. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:52.780032 (+168924us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3897129600}
W20260709 13:08:52.836910 25384 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.060s	sys 0.103s
W20260709 13:08:52.837453 25384 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.060s	sys 0.103s
I20260709 13:08:52.837612 23621 raft_consensus.cc:1275] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:08:52.837920 23623 raft_consensus.cc:1275] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:08:52.818627 22954 rpcz_store.cc:276] 0709 13:08:50.443631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:51.507846 (+1064215us) service_pool.cc:224] Handling call
0709 13:08:51.508138 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:51.937841 (+429703us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:52.718879 (+781038us) spinlock_profiling.cc:243] Waited 401 ms on lock 0x7b540037b4d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:52.719250 (+   371us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":921236224}
W20260709 13:08:52.840313 23029 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33420 (request call id 1574) took 1388 ms (client timeout 1819 ms). Trace:
I20260709 13:08:52.840433 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:52.841358 22954 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 8.
I20260709 13:08:52.841544 22955 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
I20260709 13:08:52.611943 25363 raft_consensus.cc:3055] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:08:52.808753 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 867769us
I20260709 13:08:52.842790 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:52.842976 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:52.820367 23022 rpcz_store.cc:276] 0709 13:08:49.879888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:51.144391 (+1264503us) service_pool.cc:224] Handling call
0709 13:08:52.744585 (+1600194us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b540010c554. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135deb735 00007f613adabd29 00007f613432a1d4 00007f613432a162 00007f613432a091 00007f613432a00c 00007f61343292c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:52.744900 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2647850368}
I20260709 13:08:52.844148 22955 raft_consensus.cc:2393] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 5: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:52.662703 25471 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:52.845120 25514 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:52.813149 25080 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:52.820125 23023 rpcz_store.cc:276] 0709 13:08:51.454871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:51.499470 (+ 44599us) service_pool.cc:224] Handling call
0709 13:08:51.499849 (+   379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:52.336422 (+836573us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:52.736684 (+400262us) spinlock_profiling.cc:243] Waited 835 ms on lock 0x7b540010c554. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:52.737264 (+   580us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1919590272}
I20260709 13:08:52.819976 22952 rpcz_store.cc:276] 0709 13:08:50.753658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:51.274659 (+521001us) service_pool.cc:224] Handling call
0709 13:08:52.729412 (+1454753us) spinlock_profiling.cc:243] Waited 635 ms on lock 0x7b5400411ed4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:52.729716 (+   304us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1461195136}
W20260709 13:08:52.819001 22951 rpcz_store.cc:269] 0709 13:08:50.392413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:50.630366 (+237953us) service_pool.cc:224] Handling call
0709 13:08:52.720435 (+2090069us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b5400149454. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:52.720654 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2901630720}
I20260709 13:08:52.814273 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:08:52.508558 25309 consensus_meta.cc:287] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.825s	user 0.010s	sys 0.000s
W20260709 13:08:52.820756 22992 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.876s	user 0.000s	sys 0.023s
W20260709 13:08:52.823210 22993 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.004s	sys 0.007s
I20260709 13:08:52.605912 25372 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:52.828199 22880 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.750s	user 0.000s	sys 0.016s
W20260709 13:08:52.831743 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 603437us
I20260709 13:08:52.841648 22878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2069) took 1095 ms. Trace:
I20260709 13:08:52.669605 25521 consensus_queue.cc:260] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:52.818536 22950 rpcz_store.cc:276] 0709 13:08:50.637915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:51.841440 (+1203525us) service_pool.cc:224] Handling call
0709 13:08:51.882213 (+ 40773us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:51.882673 (+   460us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:08:51.895526 (+ 12853us) log.cc:844] Serialized 14 byte log entry
0709 13:08:52.129470 (+233944us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:52.130060 (+   590us) log.cc:844] Serialized 26 byte log entry
0709 13:08:52.312825 (+182765us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:08:52.313146 (+   321us) log.cc:844] Serialized 14 byte log entry
0709 13:08:52.313439 (+   293us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:52.313544 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:52.634317 (+320773us) spinlock_profiling.cc:243] Waited 3.21 ms on lock 0x7b18003156a0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cf83f3 00007f6132cfa7c1 00007f6132cfa779 00007f6132cfa709 00007f6132cfa6d1 00007f6132cf99cd 000000000038f354 000000000038f199 00007f6132cdb83a
0709 13:08:52.714896 (+ 80579us) raft_consensus.cc:1685] finished
0709 13:08:52.714945 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:52.715303 (+   358us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":337395,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":67,"raft.queue_time_us":27459,"raft.run_cpu_time_us":347,"raft.run_wall_time_us":12062,"spinlock_wait_cycles":7377152,"thread_start_us":191014,"threads_started":1,"wal-append.queue_time_us":515620}
I20260709 13:08:52.863235 25521 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:52.863924 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:08:52.822275 22953 rpcz_store.cc:269] 0709 13:08:50.369134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:50.700769 (+331635us) service_pool.cc:224] Handling call
0709 13:08:52.745100 (+2044331us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b54000758d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:52.785565 (+ 40465us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2958918400}
W20260709 13:08:52.524125 25370 consensus_meta.cc:287] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.506s	user 0.009s	sys 0.000s
W20260709 13:08:52.818236 22947 rpcz_store.cc:269] 0709 13:08:50.456460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:50.638145 (+181685us) service_pool.cc:224] Handling call
0709 13:08:52.713507 (+2075362us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b540037a0d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:52.713822 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3103978240}
I20260709 13:08:52.822019 22949 rpcz_store.cc:276] 0709 13:08:50.725749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:51.971645 (+1245896us) service_pool.cc:224] Handling call
0709 13:08:52.006786 (+ 35141us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:52.322002 (+315216us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:52.717507 (+395505us) spinlock_profiling.cc:243] Waited 313 ms on lock 0x7b540037b9d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:52.786651 (+ 69144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":720346496}
I20260709 13:08:52.825476 23028 rpcz_store.cc:276] 0709 13:08:49.592119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:50.993220 (+1401101us) service_pool.cc:224] Handling call
0709 13:08:51.064920 (+ 71700us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:51.184797 (+119877us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:52.752945 (+1568148us) spinlock_profiling.cc:243] Waited 94.8 ms on lock 0x7b5400358754. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:52.798075 (+ 45130us) spinlock_profiling.cc:243] Waited 35.7 ms on lock 0x7b5400358750. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:52.798333 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":300243456}
W20260709 13:08:52.840957 23029 rpcz_store.cc:269] 0709 13:08:51.452331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:51.452460 (+   129us) service_pool.cc:224] Handling call
0709 13:08:52.839991 (+1387531us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54000bc7d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:52.840236 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2879682816}
I20260709 13:08:52.821887 22946 rpcz_store.cc:276] 0709 13:08:50.754302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:51.377736 (+623434us) service_pool.cc:224] Handling call
0709 13:08:52.731334 (+1353598us) spinlock_profiling.cc:243] Waited 666 ms on lock 0x7b5400148054. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:52.784745 (+ 53411us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1530885632}
W20260709 13:08:52.819684 22948 rpcz_store.cc:269] 0709 13:08:50.340599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:50.376103 (+ 35504us) service_pool.cc:224] Handling call
0709 13:08:52.728835 (+2352732us) spinlock_profiling.cc:243] Waited 836 ms on lock 0x7b54000ed3d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:52.729202 (+   367us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1922051840}
W20260709 13:08:52.855674 25497 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.006s	sys 0.035s
W20260709 13:08:52.822458 22871 rpcz_store.cc:269] 0709 13:08:50.778824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:50.794840 (+ 16016us) service_pool.cc:224] Handling call
0709 13:08:52.754249 (+1959409us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54002d6254. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:52.783837 (+ 29588us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2767837056}
W20260709 13:08:52.857560 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:52.861294 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:52.862696 25396 raft_consensus.cc:3060] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:52.852639 22878 rpcz_store.cc:276] 0709 13:08:51.746184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:51.746392 (+   208us) service_pool.cc:224] Handling call
0709 13:08:51.746703 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:52.821968 (+1075265us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:52.841337 (+ 19369us) spinlock_profiling.cc:243] Waited 962 ms on lock 0x7b54003c69d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:52.841577 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2213409664}
I20260709 13:08:52.599814 25371 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:52.843858 22954 raft_consensus.cc:2410] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 13:08:52.808431 25519 consensus_queue.cc:1048] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:08:52.842989 25363 raft_consensus.cc:740] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:52.831841 25226 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:52.835534 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:52.876101 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:52.881132 25458 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:52.881502 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:52.882007 25458 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:52.882719 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 938702us
I20260709 13:08:52.883284 25363 consensus_queue.cc:260] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:52.884712 25510 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:52.884943 25226 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:52.885378 25510 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:52.887626 23021 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:08:52.889245 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:52.889915 25372 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:52.890691 25372 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:52.846243 25514 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:52.892822 25309 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:52.893723 25440 raft_consensus.cc:2804] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:08:52.894348 25440 raft_consensus.cc:697] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:52.893041 25291 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:52.878509 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 152438us
I20260709 13:08:52.897063 25228 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:52.895183 22951 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 7.
W20260709 13:08:52.893101 25497 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.806s	user 0.006s	sys 0.037s
I20260709 13:08:52.855981 25513 consensus_queue.cc:260] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:52.903923 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:52.912422 25496 consensus_queue.cc:260] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:52.912912 25228 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:52.916956 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:52.917752 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:52.918655 25309 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:52.919008 22951 raft_consensus.cc:1240] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 5. Current term is 6. Ops: [5.5-5.5]
I20260709 13:08:52.919836 22951 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 1587) took 2164 ms. Trace:
I20260709 13:08:52.914665 25496 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:52.920188 22951 rpcz_store.cc:276] 0709 13:08:50.755758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:52.918137 (+2162379us) service_pool.cc:224] Handling call
0709 13:08:52.918635 (+   498us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:52.919531 (+   896us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:52.919742 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:52.920809 25291 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:52.928009 22955 raft_consensus.cc:2393] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 5: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:52.927919 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:52.925621 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:52.931145 23022 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6.
I20260709 13:08:52.923058 23616 raft_consensus.cc:1240] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 7. Current term is 8. Ops: []
I20260709 13:08:52.900014 25305 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:52.875124 25080 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:52.929049 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:52.905845 25438 raft_consensus.cc:740] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:52.941627 22950 raft_consensus.cc:2410] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:08:52.942260 25438 consensus_queue.cc:260] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:52.943176 22993 leader_election.cc:336] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Timed out: RequestConsensusVote RPC to 127.21.63.4:44043 timed out after 2.214s (SENT)
I20260709 13:08:52.943536 25438 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:52.945077 22953 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:52.947116 22947 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:52.958159 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:52.959586 22872 raft_consensus.cc:2393] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 9: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:52.960397 22875 raft_consensus.cc:1275] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 13:08:52.929680 25394 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.826s	user 0.009s	sys 0.000s
I20260709 13:08:52.963363 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:52.944316 25080 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:52.958225 22951 raft_consensus.cc:1240] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 5. Current term is 6. Ops: []
I20260709 13:08:52.966876 25513 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:52.967135 25505 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:52.967459 23625 raft_consensus.cc:1275] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:08:52.967657 25370 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:52.975761 22952 raft_consensus.cc:1275] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:08:52.977298 22951 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2153) took 2207 ms. Trace:
I20260709 13:08:52.978402 22919 leader_election.cc:400] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 4. Current term is 5.
I20260709 13:08:52.980160 22874 raft_consensus.cc:1240] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 5. Current term is 6. Ops: []
I20260709 13:08:52.980425 25493 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:52.980613 25414 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:52.988322 22948 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:08:52.993152 25414 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:52.994256 22948 raft_consensus.cc:1275] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:08:52.994752 22880 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
I20260709 13:08:52.981894 22951 rpcz_store.cc:276] 0709 13:08:50.765243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:52.926854 (+2161611us) service_pool.cc:224] Handling call
0709 13:08:52.957829 (+ 30975us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:52.972076 (+ 14247us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:52.972278 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:52.982419 22919 leader_election.cc:403] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:08:52.986013 22949 raft_consensus.cc:1240] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:08:52.939507 23619 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:08:52.906427 25440 consensus_queue.cc:237] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:52.985285 22952 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2154) took 2214 ms. Trace:
I20260709 13:08:52.995451 22954 raft_consensus.cc:1240] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 5. Current term is 6. Ops: [5.4-5.4]
W20260709 13:08:53.088476 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:53.094467 22952 rpcz_store.cc:276] 0709 13:08:50.771217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:52.932321 (+2161104us) service_pool.cc:224] Handling call
0709 13:08:52.965707 (+ 33386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:52.981773 (+ 16066us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:52.985235 (+  3462us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:52.994904 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:53.045176 23021 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6.
I20260709 13:08:52.994958 22948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2159) took 2128 ms. Trace:
I20260709 13:08:52.995888 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:53.000464 22950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 368) took 2092 ms. Trace:
I20260709 13:08:53.102037 22954 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 1586) took 2342 ms. Trace:
I20260709 13:08:53.000535 22918 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:52.995783 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:53.103158 22954 rpcz_store.cc:276] 0709 13:08:50.755486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:52.931620 (+2176134us) service_pool.cc:224] Handling call
0709 13:08:52.947734 (+ 16114us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:53.097736 (+150002us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:53.097957 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:53.103154 22950 rpcz_store.cc:276] 0709 13:08:50.907900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:52.999260 (+2091360us) service_pool.cc:224] Handling call
0709 13:08:52.999679 (+   419us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:53.000059 (+   380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:53.000092 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:53.000161 (+    69us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:53.000198 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:53.000267 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:53.000400 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:53.099261 22948 rpcz_store.cc:276] 0709 13:08:50.866058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:52.993428 (+2127370us) service_pool.cc:224] Handling call
0709 13:08:52.993843 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:52.994688 (+   845us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:52.994895 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:53.103303 22949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 1583) took 2348 ms. Trace:
I20260709 13:08:53.107091 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:53.112188 23622 raft_consensus.cc:2410] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:08:53.112578 22879 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:08:53.113862 22949 rpcz_store.cc:276] 0709 13:08:50.754588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:52.879965 (+2125377us) service_pool.cc:224] Handling call
0709 13:08:52.933766 (+ 53801us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:53.100019 (+166253us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:53.103251 (+  3232us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:53.113420 22947 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 6.
W20260709 13:08:52.996289 25394 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.982s	user 0.010s	sys 0.000s
I20260709 13:08:53.114904 22947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58310 (request call id 1596) took 1122 ms. Trace:
I20260709 13:08:53.118741 22953 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:08:53.120699 22954 raft_consensus.cc:1240] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 5. Current term is 6. Ops: []
I20260709 13:08:53.096772 25414 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:53.102515 25363 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:53.097206 25291 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:53.088025 25514 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:53.124788 22955 raft_consensus.cc:1240] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 6. Current term is 7. Ops: []
I20260709 13:08:53.121447 22947 rpcz_store.cc:276] 0709 13:08:51.992937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:52.877002 (+884065us) service_pool.cc:224] Handling call
0709 13:08:53.114852 (+237850us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":762752}
I20260709 13:08:53.099308 25353 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:53.126736 22843 leader_election.cc:336] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.946s (SENT)
I20260709 13:08:53.117406 25372 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:53.130474 22955 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2155) took 2355 ms. Trace:
I20260709 13:08:53.131647 25228 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:53.130757 22955 rpcz_store.cc:276] 0709 13:08:50.774838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:52.938021 (+2163183us) service_pool.cc:224] Handling call
0709 13:08:53.095348 (+157327us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:53.130237 (+ 34889us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:53.130428 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:08:53.134657 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 187553us
W20260709 13:08:53.135460 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:53.135146 23028 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:52.996109 25305 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:52.998592 25370 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:53.143301 22878 raft_consensus.cc:1240] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 5 LEADER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:53.145677 25291 raft_consensus.cc:2804] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:53.146242 25291 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:53.146680 25291 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:52.998139 25493 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:53.153703 22872 raft_consensus.cc:1240] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 2. Current term is 3. Ops: []
I20260709 13:08:53.151707 22918 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:53.152172 22993 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:53.153671 22871 raft_consensus.cc:2468] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 4.
I20260709 13:08:53.160180 22875 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 4.
W20260709 13:08:53.164207 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 186115us
I20260709 13:08:53.164865 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:53.171432 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:53.171568 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:53.169806 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:53.176314 23622 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 538) took 1100 ms. Trace:
I20260709 13:08:53.177651 23622 rpcz_store.cc:276] 0709 13:08:52.076170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:52.076910 (+   740us) service_pool.cc:224] Handling call
0709 13:08:53.176042 (+1099132us) spinlock_profiling.cc:243] Waited 776 ms on lock 0x7b5400671454. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:53.176242 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1785350912}
I20260709 13:08:53.177500 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:53.178911 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:53.184710 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:08:53.188623 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 62043.2us
I20260709 13:08:53.187717 25318 raft_consensus.cc:2749] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:08:53.173409 25371 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 5
I20260709 13:08:53.190021 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:53.192094 23022 raft_consensus.cc:2393] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 4: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:53.191753 23028 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 6.
I20260709 13:08:53.191675 23021 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 6.
W20260709 13:08:53.195915 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 69767.7us
I20260709 13:08:53.196033 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:53.196579 22877 raft_consensus.cc:1240] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:08:53.197503 22993 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:53.197788 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:53.198478 25318 raft_consensus.cc:2749] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:08:53.198544 22992 connection.cc:256] RPC call timeout handler was delayed by -1.1323966259999452s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2214s for a response
I20260709 13:08:53.198711 23625 raft_consensus.cc:2393] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 8: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:53.197518 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:53.200453 23623 raft_consensus.cc:2393] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 7: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:53.185205 25458 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:53.200557 25318 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:53.201261 25318 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:53.201779 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:53.202992 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:08:53.203141 22826 auto_rebalancer.cc:629] Failed to schedule move for tablet f603724630e44ef78014e7719f096cb3: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:08:53.203295 25305 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:53.203939 22955 raft_consensus.cc:2410] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:08:53.188191 25228 raft_consensus.cc:2804] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:08:53.188477 25514 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:53.216849 22919 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:53.213626 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:53.214176 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:53.221556 22874 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 4.
I20260709 13:08:53.221750 23028 raft_consensus.cc:1240] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:08:53.222925 25318 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:53.211731 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:53.225234 23622 raft_consensus.cc:2410] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 9 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 13:08:53.231462 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:53.232800 23621 raft_consensus.cc:1275] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:08:53.234324 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:53.235075 22952 raft_consensus.cc:2376] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7. Re-sending same reply.
I20260709 13:08:53.235978 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:08:53.237267 23025 consensus_meta.cc:287] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.834s	user 0.019s	sys 0.003s
I20260709 13:08:53.237957 23025 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6.
I20260709 13:08:53.238983 23023 raft_consensus.cc:1240] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 13:08:53.240093 25414 consensus_queue.cc:1048] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:08:53.240001 25318 consensus_queue.cc:1048] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:08:53.242031 25414 consensus_queue.cc:1059] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:08:53.218446 25228 raft_consensus.cc:697] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:53.243197 25318 raft_consensus.cc:3055] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:08:53.243737 25318 raft_consensus.cc:740] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:53.244825 22918 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:53.245574 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:53.246068 23028 raft_consensus.cc:1275] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:08:53.246980 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:53.247213 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:53.248139 23623 raft_consensus.cc:2368] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 3. Current term is 4.
I20260709 13:08:53.219102 25514 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:53.248976 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:53.249755 23623 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 6.
I20260709 13:08:53.250097 22879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35370 (request call id 2066) took 1516 ms. Trace:
I20260709 13:08:53.250559 23029 raft_consensus.cc:2368] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 3. Current term is 4.
I20260709 13:08:53.250634 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:53.250936 25226 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:53.237316 22954 raft_consensus.cc:1275] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:08:53.249015 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:53.248574 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:53.251302 23625 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:53.250476 22879 rpcz_store.cc:276] 0709 13:08:51.733965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:51.734068 (+   103us) service_pool.cc:224] Handling call
0709 13:08:53.249907 (+1515839us) spinlock_profiling.cc:243] Waited 972 ms on lock 0x7b54001bba54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:53.250047 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2234854272}
I20260709 13:08:53.257407 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:53.248222 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:53.248150 23588 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:53.271368 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:53.275447 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:53.277467 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:53.268782 23025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33420 (request call id 1584) took 1064 ms. Trace:
I20260709 13:08:53.278791 23026 raft_consensus.cc:1240] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:08:53.281706 22952 raft_consensus.cc:2368] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 5. Current term is 6.
I20260709 13:08:53.278991 23025 rpcz_store.cc:276] 0709 13:08:52.194420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:52.349557 (+155137us) service_pool.cc:224] Handling call
0709 13:08:53.251940 (+902383us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:08:53.282368 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:53.259876 25458 raft_consensus.cc:2749] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:08:53.283989 22955 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
I20260709 13:08:53.244412 25228 consensus_queue.cc:237] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:53.300343 22993 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:53.302127 22872 raft_consensus.cc:2393] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 4: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
W20260709 13:08:53.295347 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 47442.2us
I20260709 13:08:53.287400 25440 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:53.289137 25305 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:53.306838 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:53.307085 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:53.307581 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 101439us
I20260709 13:08:53.308471 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:53.296118 25372 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:53.300094 25252 raft_consensus.cc:2749] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:08:53.278954 25363 consensus_queue.cc:1048] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260709 13:08:53.319205 22879 raft_consensus.cc:1240] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 4. Current term is 5. Ops: []
W20260709 13:08:53.314134 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:53.315970 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:53.323467 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:53.324293 23621 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
I20260709 13:08:53.326130 25372 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:53.293330 25493 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:08:53.328337 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:53.329136 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:53.332226 25328 raft_consensus.cc:2804] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:53.332803 25328 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:53.332892 23022 raft_consensus.cc:1240] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:08:53.333024 25372 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:53.333377 25328 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:53.334270 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:53.334308 25371 raft_consensus.cc:2749] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 4. Current term is 5.
I20260709 13:08:53.327373 25370 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:53.336964 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:53.341221 25373 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:53.341833 25373 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:53.342293 25373 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:53.345129 25370 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:53.345770 25370 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:53.346982 22994 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:08:53.347416 25484 consensus_meta.cc:287] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.685s	user 0.010s	sys 0.003s
I20260709 13:08:53.348151 25484 raft_consensus.cc:2804] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:53.348944 23024 raft_consensus.cc:606] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:08:53.349167 25370 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:53.349356 23024 raft_consensus.cc:614] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:08:53.349762 25305 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:53.351004 25514 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:53.352036 23024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:60044 (request call id 22) took 1517 ms. Trace:
I20260709 13:08:53.352698 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:53.351969 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:53.353631 22951 raft_consensus.cc:2393] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 4: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:53.353885 23025 raft_consensus.cc:2393] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 4: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:53.352506 23024 rpcz_store.cc:276] 0709 13:08:51.834633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:51.834915 (+   282us) service_pool.cc:224] Handling call
0709 13:08:53.351830 (+1516915us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b5400189f54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135ddf853 00007f613adad60a 00007f61343337d4 00007f6134333762 00007f6134333691 00007f613433360c 00007f61343328c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:53.351950 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3461486464}
I20260709 13:08:53.360198 23026 raft_consensus.cc:1240] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 6. Current term is 7. Ops: []
W20260709 13:08:53.362450 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 48589.5us
I20260709 13:08:53.362613 25514 consensus_queue.cc:1059] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:08:53.364533 25521 raft_consensus.cc:3055] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:08:53.364970 25521 raft_consensus.cc:740] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:53.366142 25521 consensus_queue.cc:260] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:53.367856 25396 consensus_queue.cc:1059] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:08:53.369091 22874 raft_consensus.cc:2393] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 7: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:53.369536 25396 raft_consensus.cc:3055] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:08:53.369268 25464 consensus_queue.cc:1059] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:08:53.370007 25396 raft_consensus.cc:740] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:53.370772 25464 raft_consensus.cc:3055] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:08:53.371162 25464 raft_consensus.cc:740] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:53.371085 25396 consensus_queue.cc:260] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:53.376502 22918 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:08:53.379556 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:53.380353 25371 raft_consensus.cc:2804] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:53.380857 25371 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:53.381286 25371 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:53.372263 25464 consensus_queue.cc:260] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:53.387465 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 170704us
I20260709 13:08:53.387923 25464 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:53.389571 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:53.397709 25521 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:53.352030 25305 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:53.410943 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 117977us
I20260709 13:08:53.429013 25510 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:53.454964 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:53.413873 25484 raft_consensus.cc:697] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:08:53.480391 25551 log.cc:927] Time spent T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
W20260709 13:08:53.539857 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:53.569108 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:53.570142 23619 raft_consensus.cc:2393] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 7: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:53.448403 25396 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:08:53.539803 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 333395us
I20260709 13:08:53.422019 25440 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:53.558193 23625 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 9.
I20260709 13:08:53.563216 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:08:53.564960 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 209669us
I20260709 13:08:53.578078 25471 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:53.558472 25371 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:53.584468 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:08:53.586467 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:53.588873 22919 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:53.589787 22947 raft_consensus.cc:1275] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 13:08:53.596169 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:53.590284 22947 pending_rounds.cc:85] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting all ops after (but not including) 1
I20260709 13:08:53.599632 22871 raft_consensus.cc:2393] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 7: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:53.593371 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:53.547673 25328 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:53.604367 23619 raft_consensus.cc:1240] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:08:53.603328 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:53.597750 22947 pending_rounds.cc:107] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:08:53.596827 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:53.593809 25228 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:53.609165 25383 consensus_queue.cc:1059] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:08:53.610002 22993 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:53.614243 25373 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:53.615005 25418 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:53.616469 23029 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 8.
I20260709 13:08:53.616969 22874 raft_consensus.cc:1240] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:53.564692 25291 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:53.619148 22947 raft_consensus.cc:2889] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:53.627602 22843 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:53.624737 22994 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:53.636824 23020 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:53.603829 25513 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:53.637809 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:53.641100 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:53.638441 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:53.599485 25493 consensus_queue.cc:1048] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 13:08:53.626572 25226 raft_consensus.cc:2749] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:08:53.638996 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:53.641882 22950 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 6.
I20260709 13:08:53.647889 22871 raft_consensus.cc:2393] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 5: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
W20260709 13:08:53.648865 25507 consensus_meta.cc:287] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.961s	user 0.017s	sys 0.005s
I20260709 13:08:53.649235 22951 raft_consensus.cc:2410] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:08:53.649223 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:53.649741 23618 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 6.
I20260709 13:08:53.649899 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:53.650352 22949 raft_consensus.cc:2393] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 7: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:53.650547 22953 raft_consensus.cc:2368] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 4. Current term is 5.
I20260709 13:08:53.628877 25372 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:53.656303 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:53.650607 25471 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:53.657675 25471 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:53.658739 25328 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:53.659443 25328 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:53.659972 25328 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:53.660283 25553 log.cc:927] Time spent T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 13:08:53.661087 25371 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:53.606863 25241 consensus_queue.cc:1059] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.104s
I20260709 13:08:53.661993 22879 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 8.
I20260709 13:08:53.662289 25241 raft_consensus.cc:3055] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:08:53.662813 25241 raft_consensus.cc:740] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:53.663157 25484 consensus_queue.cc:237] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:53.640255 25228 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:53.637398 25370 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:53.648881 23620 raft_consensus.cc:2393] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 4: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:53.658990 25363 raft_consensus.cc:2804] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:53.626012 25508 consensus_queue.cc:1059] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.023s
I20260709 13:08:53.660382 25328 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:53.668190 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:53.659482 23624 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:08:53.649523 25353 raft_consensus.cc:2749] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:53.676509 22948 raft_consensus.cc:1240] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 LEADER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 13:08:53.679551 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 102180us
I20260709 13:08:53.683276 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:53.663919 25241 consensus_queue.cc:260] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:53.607573 25318 consensus_queue.cc:260] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:53.688974 25373 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:53.689131 23625 raft_consensus.cc:3055] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:08:53.689570 23625 raft_consensus.cc:740] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:53.674368 25363 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:53.690330 25363 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:53.690486 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:53.691406 22947 raft_consensus.cc:2393] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 6: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:53.678731 25418 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:53.695883 25418 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:53.647001 25513 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:53.680416 25372 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:53.696949 23618 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 600) took 1828 ms (client timeout 1691 ms). Trace:
W20260709 13:08:53.697515 23618 rpcz_store.cc:269] 0709 13:08:51.873854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:51.875709 (+  1855us) service_pool.cc:224] Handling call
0709 13:08:53.696651 (+1820942us) spinlock_profiling.cc:243] Waited 1.68 s on lock 0x7b5400572454. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:53.696873 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3868373888}
I20260709 13:08:53.698405 25372 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:53.698807 23023 raft_consensus.cc:1240] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:53.699090 25318 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:53.700100 25228 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:53.686535 25370 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:53.688992 22952 raft_consensus.cc:2393] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 5: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:53.702638 25370 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:53.688380 22775 catalog_manager.cc:5697] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 3 to 4, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 4 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
W20260709 13:08:53.703349 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 85880.7us
I20260709 13:08:53.705418 22843 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:53.705353 22775 catalog_manager.cc:5697] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 3 to 5, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 5 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:53.706539 22843 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:53.689880 25291 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:53.707278 23624 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 547) took 1325 ms (client timeout 1693 ms). Trace:
I20260709 13:08:53.707587 25353 raft_consensus.cc:2804] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:53.708135 25353 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:53.707880 23624 rpcz_store.cc:269] 0709 13:08:52.382002 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:52.434391 (+ 52389us) service_pool.cc:224] Handling call
0709 13:08:53.707051 (+1272660us) spinlock_profiling.cc:243] Waited 794 ms on lock 0x7b5400670f54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:53.707210 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1825451904}
I20260709 13:08:53.708715 25353 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:53.669476 25438 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:53.711369 23029 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 8.
I20260709 13:08:53.690486 23625 consensus_queue.cc:260] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:53.714622 22774 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33466 (request call id 289) took 9700 ms. Trace:
I20260709 13:08:53.707283 22775 catalog_manager.cc:5697] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:08:53.721372 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:53.722399 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:53.715997 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:53.704648 25444 raft_consensus.cc:3055] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:08:53.698405 25513 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:53.733913 23625 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:08:53.731915 23023 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 7.
I20260709 13:08:53.712589 25372 raft_consensus.cc:2804] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:53.731230 22879 raft_consensus.cc:2393] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 7: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:53.741354 22946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2158) took 2875 ms. Trace:
I20260709 13:08:53.739006 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:53.746459 23020 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 5.
W20260709 13:08:53.748539 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:53.752491 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 43930us
I20260709 13:08:53.734576 25444 raft_consensus.cc:740] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:53.744635 22775 catalog_manager.cc:5697] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 1 to 5, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 5 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:53.767841 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:53.741830 22946 rpcz_store.cc:276] 0709 13:08:50.865805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:53.025780 (+2159975us) service_pool.cc:224] Handling call
0709 13:08:53.120332 (+ 94552us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:53.133901 (+ 13569us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:08:53.282250 (+148349us) log.cc:844] Serialized 14 byte log entry
0709 13:08:53.738514 (+456264us) log.cc:844] Serialized 14 byte log entry
0709 13:08:53.740738 (+  2224us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:53.740805 (+    67us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:08:53.740863 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:53.740943 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:53.741288 (+   345us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9081,"prepare.queue_time_us":180180,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":9336,"spinlock_wait_cycles":5468544,"thread_start_us":308118,"threads_started":2}
I20260709 13:08:53.771525 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:53.771926 22992 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:53.740021 25291 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:53.715651 25438 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:53.715169 22774 rpcz_store.cc:276] 0709 13:08:43.917939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:43.918149 (+   210us) service_pool.cc:224] Handling call
0709 13:08:53.616648 (+9698499us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:50.213168 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:50.256512 (+ 43344us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:50.256550 (+    38us) write_op.cc:435] Acquired schema lock
0709 13:08:50.256567 (+    17us) tablet.cc:662] Decoding operations
0709 13:08:50.257513 (+   946us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:50.257588 (+    75us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:50.257629 (+    41us) tablet.cc:685] Acquiring locks for 7 operations
0709 13:08:50.416505 (+158876us) tablet.cc:701] Row locks acquired
0709 13:08:50.416529 (+    24us) write_op.cc:260] PREPARE: finished
0709 13:08:50.416708 (+   179us) write_op.cc:270] Start()
0709 13:08:50.416834 (+   126us) write_op.cc:276] Timestamp: P: 1783602530416635 usec, L: 0
0709 13:08:50.416868 (+    34us) op_driver.cc:348] REPLICATION: starting
0709 13:08:50.482484 (+ 65616us) log.cc:844] Serialized 2803 byte log entry
0709 13:08:51.065350 (+582866us) op_driver.cc:464] REPLICATION: finished
0709 13:08:51.687089 (+621739us) write_op.cc:301] APPLY: starting
0709 13:08:51.692365 (+  5276us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:51.747125 (+ 54760us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:51.824161 (+ 77036us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:51.989261 (+165100us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:52.005527 (+ 16266us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=7
0709 13:08:52.005589 (+    62us) write_op.cc:312] APPLY: finished
0709 13:08:52.044598 (+ 39009us) log.cc:844] Serialized 59 byte log entry
0709 13:08:52.957702 (+913104us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:52.958058 (+   356us) write_op.cc:454] Released schema lock
0709 13:08:52.958202 (+   144us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:52.959229 (+  1027us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:52.959267 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2021509,"reported_tablets":29,"child_traces":[["op",{"apply.queue_time_us":597088,"apply.run_cpu_time_us":49887,"apply.run_wall_time_us":1278088,"num_ops":7,"prepare.queue_time_us":21635,"prepare.run_cpu_time_us":25343,"prepare.run_wall_time_us":617169,"raft.queue_time_us":77245,"raft.run_cpu_time_us":23166,"raft.run_wall_time_us":472161,"replication_time_us":628659,"thread_start_us":1462739,"threads_started":5,"wal-append.queue_time_us":993959,"wal-append.run_cpu_time_us":5851,"wal-append.run_wall_time_us":2103582}]]}
I20260709 13:08:53.780165 22874 raft_consensus.cc:2410] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:08:53.782331 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:53.783212 22879 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
I20260709 13:08:53.784807 22871 raft_consensus.cc:1240] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:08:53.784869 22843 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:53.787545 22843 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:53.789755 25440 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:53.791735 25444 consensus_queue.cc:260] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:53.739578 25372 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:53.793857 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:53.800067 25444 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:53.801887 22878 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:53.803540 25471 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:53.804092 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 93736.2us
I20260709 13:08:53.803954 25291 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:53.806542 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:53.809059 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:08:53.809463 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 43942.3us
I20260709 13:08:53.809880 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:53.810452 25493 raft_consensus.cc:2804] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:08:53.811815 25438 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:53.811762 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:08:53.824280 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 222900us
I20260709 13:08:53.824841 22871 raft_consensus.cc:2393] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 7: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:53.769006 25241 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:53.812615 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:53.813231 25513 consensus_queue.cc:1059] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:53.813573 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:53.823776 22872 raft_consensus.cc:2410] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 13:08:53.825655 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:08:53.826560 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:53.815176 25372 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:08:53.823308 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 215230us
W20260709 13:08:53.830314 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:53.834419 23625 raft_consensus.cc:1275] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260709 13:08:53.835002 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:53.829802 22876 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6.
I20260709 13:08:53.834353 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:53.834069 22877 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:08:53.841271 22992 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:53.846861 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:53.852210 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:53.853251 22872 raft_consensus.cc:2393] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 4: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:53.854246 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:53.856494 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:53.821513 25363 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:53.857877 22879 raft_consensus.cc:1275] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:08:53.850405 22919 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:53.835444 25493 raft_consensus.cc:697] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:53.861811 22877 raft_consensus.cc:1275] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:08:53.865422 22775 catalog_manager.cc:5697] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 4 to 5, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 5 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:53.873384 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:53.842033 25438 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:53.879607 25246 raft_consensus.cc:2764] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 13:08:53.878108 25438 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:53.882709 25438 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:53.887274 25493 consensus_queue.cc:237] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:53.896180 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 65758.1us
I20260709 13:08:53.874392 25226 raft_consensus.cc:2749] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:08:53.900739 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 65827.4us
W20260709 13:08:53.904835 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 296550us
I20260709 13:08:53.912216 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:08:53.915400 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 87041.3us
I20260709 13:08:53.914685 22994 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:53.916237 22919 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:53.918599 23588 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:53.919389 23588 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:53.924562 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:53.925087 25363 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:53.925427 22873 raft_consensus.cc:2393] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 6: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:53.924758 25438 raft_consensus.cc:2749] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:08:53.927069 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 139576us
I20260709 13:08:53.927870 22843 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:53.924602 25565 consensus_queue.cc:1059] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 13:08:53.928715 25471 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:53.929055 25568 raft_consensus.cc:2804] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:08:53.929314 25471 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:53.931514 25471 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:53.928201 25414 raft_consensus.cc:3055] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:08:53.932214 25414 raft_consensus.cc:740] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:53.933265 25414 consensus_queue.cc:260] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:53.934435 25438 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:53.935070 25438 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:53.931860 25418 raft_consensus.cc:2804] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:53.938360 25438 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:53.954625 22878 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
W20260709 13:08:53.958441 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:53.958841 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:53.959807 22878 raft_consensus.cc:2393] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 4: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:53.959517 22775 catalog_manager.cc:5697] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 3 to 4, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 4 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
W20260709 13:08:53.962071 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:53.963725 25246 consensus_queue.cc:1059] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:08:53.964402 25565 raft_consensus.cc:3055] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 13:08:53.972946 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 56836.1us
I20260709 13:08:53.974401 22992 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:53.959980 25305 raft_consensus.cc:2804] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:53.973371 22775 catalog_manager.cc:5697] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:53.974655 25568 raft_consensus.cc:697] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 6 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:53.988361 25414 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:53.988555 22992 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:53.989205 25560 raft_consensus.cc:2749] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:08:53.990738 23026 raft_consensus.cc:1240] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:08:53.964641 25246 raft_consensus.cc:3055] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:08:53.992535 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:53.974195 25383 consensus_queue.cc:1048] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 13:08:53.993310 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:53.964972 25565 raft_consensus.cc:740] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:54.008741 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:53.967149 25353 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:53.994405 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:54.017491 25565 consensus_queue.cc:260] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:53.996073 23027 raft_consensus.cc:1275] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:08:54.019078 25565 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:54.019475 22874 raft_consensus.cc:2393] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 9: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:54.019181 25372 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:53.993968 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:54.005579 23023 raft_consensus.cc:2393] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 6: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:54.019199 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:54.020253 23028 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
I20260709 13:08:54.020756 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:54.021714 22992 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:08:54.002401 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 45284us
I20260709 13:08:54.023433 25318 raft_consensus.cc:2749] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:08:54.004035 25246 raft_consensus.cc:740] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:54.007513 25484 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:54.024168 25318 consensus_queue.cc:1059] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:08:54.024847 25560 consensus_queue.cc:1048] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:54.025802 25318 raft_consensus.cc:3055] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:08:54.026062 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:54.024612 25246 consensus_queue.cc:260] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:54.007524 25573 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:54.024701 25561 consensus_queue.cc:1048] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 13:08:54.027666 25573 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:54.028179 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:54.008152 23025 raft_consensus.cc:1275] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:08:54.028854 22871 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6.
I20260709 13:08:54.005596 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:54.030778 25573 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:54.031735 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:54.031919 25484 raft_consensus.cc:2749] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:54.008641 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:54.033052 23027 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 5.
I20260709 13:08:54.008054 22775 catalog_manager.cc:5697] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 1 to 3, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 3 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:54.033535 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:54.034216 22872 raft_consensus.cc:1275] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:08:54.008992 25559 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:54.034767 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:54.035115 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:54.010134 25570 raft_consensus.cc:2804] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:54.035480 22871 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
I20260709 13:08:54.035763 25570 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:54.036191 25570 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:54.009497 25396 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:54.036238 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:53.967501 25418 raft_consensus.cc:697] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:54.016310 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:54.018105 23022 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6.
I20260709 13:08:53.991568 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:08:54.026952 25557 log.cc:927] Time spent T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
I20260709 13:08:54.008322 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:54.028993 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:54.007586 25568 consensus_queue.cc:237] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:54.032367 22879 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 3.
I20260709 13:08:54.032994 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:54.026911 22877 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 5.
I20260709 13:08:54.012498 22919 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:54.032945 23026 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:54.034513 22775 catalog_manager.cc:5697] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 1 to 5. New cstate: current_term: 5 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:08:54.036456 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:54.055269 22875 raft_consensus.cc:2393] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 5: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:54.055722 22871 raft_consensus.cc:2410] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:08:54.055965 22880 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:54.053320 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:54.023694 22993 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:54.056725 22878 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:54.057227 22872 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
W20260709 13:08:54.005153 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 77419us
I20260709 13:08:54.061925 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:54.037464 25246 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:54.026202 25318 raft_consensus.cc:740] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:54.063709 23020 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:08:54.063781 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 51445.1us
I20260709 13:08:54.065534 22879 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
I20260709 13:08:54.068853 22878 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 5.
I20260709 13:08:54.069535 22843 leader_election.cc:400] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 election: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 4. Current term is 5.
I20260709 13:08:54.069993 22843 leader_election.cc:403] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
W20260709 13:08:54.070458 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 43977.9us
I20260709 13:08:54.069167 22775 catalog_manager.cc:5697] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 4 to 6, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 6 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:54.019972 25353 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:54.075943 22992 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:54.064846 25438 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:54.070729 25226 consensus_queue.cc:1048] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 13:08:54.077665 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:54.078289 23029 raft_consensus.cc:2376] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6. Re-sending same reply.
I20260709 13:08:54.079216 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:54.079600 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:54.080441 22952 raft_consensus.cc:2393] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 4: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:54.080787 22876 raft_consensus.cc:2393] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 5: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:54.081391 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:54.083511 22920 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:54.064139 25396 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:54.085729 25396 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:54.085695 22920 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:54.086210 25226 raft_consensus.cc:2749] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:08:54.086933 25438 raft_consensus.cc:2749] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:54.088795 25572 raft_consensus.cc:3055] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:08:54.089260 25572 raft_consensus.cc:740] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:54.090117 25572 consensus_queue.cc:260] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:54.092331 25572 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:54.000990 25328 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:54.093022 25438 consensus_queue.cc:1048] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:08:54.094172 22992 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:54.117400 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:54.094754 25560 raft_consensus.cc:2804] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:08:54.055047 25305 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:08:54.097064 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 73458.6us
I20260709 13:08:54.098275 25372 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:54.119534 22775 catalog_manager.cc:5697] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 3 to 4, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 4 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:54.098327 23590 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:54.098673 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:54.073876 25484 raft_consensus.cc:2749] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:08:54.102356 25418 consensus_queue.cc:237] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:54.035403 25559 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:54.103757 23026 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
I20260709 13:08:54.104305 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:54.105540 23022 raft_consensus.cc:1275] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:08:54.105801 23021 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6.
I20260709 13:08:54.137964 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:54.039647 25291 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:54.112720 23020 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 5.
I20260709 13:08:54.115279 22880 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
I20260709 13:08:54.115556 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:54.116163 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:08:54.117139 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:54.094350 25226 consensus_queue.cc:1048] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:08:54.140461 23027 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:08:54.118492 23622 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:08:54.141479 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:54.143111 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:54.118397 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 49002.9us
I20260709 13:08:54.126952 25318 consensus_queue.cc:260] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:54.146142 25318 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:54.146401 23622 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 8.
I20260709 13:08:54.146636 22843 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:54.148084 25372 raft_consensus.cc:2749] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:08:54.121220 23625 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:54.121914 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:54.119575 25353 raft_consensus.cc:2749] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 4. Current term is 5.
I20260709 13:08:54.100946 22920 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:54.122825 25560 raft_consensus.cc:697] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:54.122298 22775 catalog_manager.cc:5697] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:54.151783 25560 consensus_queue.cc:237] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:54.152559 22775 catalog_manager.cc:5697] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 1 to 2, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 2 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:08:54.155313 22947 raft_consensus.cc:2393] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 7: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:54.125342 25328 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:54.125906 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:54.126888 22844 leader_election.cc:400] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 election: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 3. Current term is 4.
I20260709 13:08:54.122493 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:08:54.118979 25557 log.cc:927] Time spent T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:08:54.127815 25305 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:54.130527 25565 raft_consensus.cc:2749] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:54.136731 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:54.138602 23028 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
I20260709 13:08:54.150861 25507 raft_consensus.cc:2749] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:54.172537 23621 raft_consensus.cc:1275] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 13:08:54.123679 22992 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:54.155486 23625 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 9.
W20260709 13:08:54.155580 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 54708.4us
I20260709 13:08:54.156795 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:54.157711 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:54.158290 22918 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:08:54.176353 22918 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:54.176652 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:54.177475 22949 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:54.158699 25458 raft_consensus.cc:2804] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:54.159224 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:54.178610 25458 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:54.165185 22844 leader_election.cc:403] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 13:08:54.179055 25458 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:54.179140 25226 raft_consensus.cc:2804] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:08:54.179602 25226 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:54.179648 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 52938.1us
I20260709 13:08:54.180104 25226 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:54.180770 22844 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:54.181923 25353 raft_consensus.cc:2749] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 3. Current term is 4.
I20260709 13:08:54.182786 25353 raft_consensus.cc:2804] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:54.183213 25353 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:54.183310 25438 raft_consensus.cc:2749] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:54.183769 25353 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:54.184109 25438 consensus_queue.cc:1048] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:54.179068 22947 raft_consensus.cc:2393] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 7: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:54.185420 23025 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 5.
I20260709 13:08:54.177991 22918 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:08:54.186157 23020 raft_consensus.cc:2376] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6. Re-sending same reply.
I20260709 13:08:54.187652 22876 raft_consensus.cc:1275] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 13:08:54.191140 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 67524.2us
I20260709 13:08:54.187458 25246 consensus_queue.cc:1048] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260709 13:08:54.193192 22992 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:54.193658 25246 raft_consensus.cc:2749] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:54.194015 25559 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:54.194299 22992 leader_election.cc:400] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 election: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 5. Current term is 6.
I20260709 13:08:54.194377 25246 raft_consensus.cc:2804] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:08:54.194689 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 96563us
I20260709 13:08:54.194902 25246 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:54.195292 25246 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:54.195467 23590 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:54.169014 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:54.169370 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:54.169139 21756 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:08:54.170524 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:54.194092 25383 consensus_queue.cc:1048] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:54.196498 22873 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
I20260709 13:08:54.195446 22992 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:54.167618 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:54.205427 22952 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 8.
I20260709 13:08:54.206422 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:54.207274 22947 raft_consensus.cc:2393] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 7: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:54.205824 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:54.209342 22948 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 5.
I20260709 13:08:54.211684 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:54.212298 22955 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:54.213527 21756 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:08:54.205533 25568 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:54.215082 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:54.215592 22872 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:54.209898 22918 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:54.214416 25568 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:54.217605 25571 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:54.204406 25291 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:54.220088 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:54.219288 22844 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:54.222923 22953 raft_consensus.cc:2376] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 5. Re-sending same reply.
I20260709 13:08:54.224187 25568 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:54.225759 23590 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:54.226605 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:54.227326 23022 raft_consensus.cc:2393] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 7: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:54.227392 23590 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:54.228111 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:54.205485 25507 raft_consensus.cc:2749] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:08:54.221660 22918 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:54.229588 23022 raft_consensus.cc:2410] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:08:54.231612 22952 raft_consensus.cc:1240] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:54.232980 22994 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:54.234799 22918 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:54.235248 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:54.230592 25252 raft_consensus.cc:2804] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:54.250494 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:54.252825 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:54.222180 25370 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:54.257138 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:54.258143 22954 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:54.257555 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:54.252878 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:54.257081 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:54.259393 23027 raft_consensus.cc:2468] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 6.
I20260709 13:08:54.257330 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:54.222714 25291 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:54.260119 22948 raft_consensus.cc:1240] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:08:54.260636 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:54.229224 25328 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:54.217720 25440 consensus_queue.cc:1048] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 13:08:54.295960 22950 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 5.
I20260709 13:08:54.264582 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:54.293927 23029 raft_consensus.cc:2393] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 5: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:54.274555 23590 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:54.297048 22947 raft_consensus.cc:1275] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:08:54.296991 23026 raft_consensus.cc:1275] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:08:54.246841 25396 raft_consensus.cc:2804] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:08:54.300060 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 104696us
I20260709 13:08:54.252475 25252 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:54.298532 22947 pending_rounds.cc:85] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting all ops after (but not including) 7
I20260709 13:08:54.302026 25370 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:54.318626 22947 pending_rounds.cc:107] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:08:54.303858 22843 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:54.319180 22947 raft_consensus.cc:2889] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:54.320073 22843 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:54.252969 25438 raft_consensus.cc:2749] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:54.263979 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:54.317967 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:54.274438 22844 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:54.322292 22951 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:54.253357 25363 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:54.324476 22953 raft_consensus.cc:2468] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
I20260709 13:08:54.236406 25571 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:54.332594 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:54.341956 22952 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 7.
I20260709 13:08:54.275190 25559 raft_consensus.cc:2804] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:08:54.342633 22872 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:08:54.247427 25383 consensus_queue.cc:1048] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.065s
I20260709 13:08:54.250020 25228 raft_consensus.cc:2804] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:54.352134 22878 raft_consensus.cc:1275] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 13:08:54.273837 25291 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:54.352970 22994 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:54.355955 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:54.357450 25372 raft_consensus.cc:2804] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:08:54.357787 25559 raft_consensus.cc:697] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:54.357960 25444 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:54.358054 25372 raft_consensus.cc:697] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 5 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:54.358573 25444 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:54.359172 25372 consensus_queue.cc:237] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:54.299384 25574 raft_consensus.cc:2749] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:08:54.360517 25444 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:54.365638 25226 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:54.373642 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:54.332060 25363 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:54.375538 25363 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:54.361016 25559 consensus_queue.cc:237] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:54.360460 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:54.371869 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:08:54.380395 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 121093us
I20260709 13:08:54.327277 25507 raft_consensus.cc:2804] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:54.383602 25507 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:54.384024 25507 raft_consensus.cc:3060] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:54.303743 25396 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:54.324952 25252 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:54.390822 22954 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 5.
I20260709 13:08:54.393360 22946 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 5.
W20260709 13:08:54.394173 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 119815us
I20260709 13:08:54.363878 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:54.327720 25568 raft_consensus.cc:2804] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:54.407563 22844 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:54.400667 23028 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 6.
W20260709 13:08:54.406071 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 86083.7us
I20260709 13:08:54.411113 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:54.412823 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:54.413807 23622 raft_consensus.cc:2393] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 6: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:54.414899 23020 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:54.417048 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:54.417788 23619 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:08:54.420590 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 185838us
I20260709 13:08:54.425457 22843 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:54.426630 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:54.424906 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:54.436091 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:54.436919 23617 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:54.355182 25228 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:08:54.439013 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:54.439921 22918 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:54.442005 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:54.362418 25370 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:54.357502 25513 raft_consensus.cc:2749] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:54.435336 25226 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:54.408097 25561 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:08:54.449641 22918 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:54.405670 25396 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:54.454334 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:54.425241 25241 consensus_queue.cc:1048] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 13:08:54.467679 22955 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 8.
W20260709 13:08:54.468860 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 61389.5us
I20260709 13:08:54.473624 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:54.407968 25568 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:54.400017 25414 consensus_queue.cc:1048] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 13:08:54.448800 25228 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:54.478534 22946 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
I20260709 13:08:54.483031 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:54.483793 22954 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:54.455436 25440 raft_consensus.cc:2804] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:08:54.487725 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 128883us
I20260709 13:08:54.481482 22843 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:54.491811 22844 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:54.472131 25573 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:08:54.493485 25438 consensus_queue.cc:1048] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:54.498307 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:54.499907 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:54.477969 25568 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:54.500914 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:54.503921 22918 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:08:54.504016 23588 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:54.487426 25575 raft_consensus.cc:2804] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:08:54.480424 25561 raft_consensus.cc:697] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:54.519313 23020 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 8.
I20260709 13:08:54.489431 25226 raft_consensus.cc:2804] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:08:54.522847 22918 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:54.529129 22954 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
I20260709 13:08:54.539628 22918 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:08:54.540941 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 59572.4us
W20260709 13:08:54.541989 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 61082.7us
I20260709 13:08:54.545233 22844 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:54.510497 25418 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:54.557310 22876 raft_consensus.cc:1275] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:08:54.559120 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:54.560807 22920 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:54.521692 25573 raft_consensus.cc:2804] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:54.567303 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:54.569867 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:54.570830 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:08:54.572901 22826 auto_rebalancer.cc:629] Failed to schedule move for tablet ac5cc94926a742e6a0c1340ec33c0ff0: Illegal state: Replica f3c799b1249e40119d2a5864231c6c75 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 4 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:08:54.524158 25440 raft_consensus.cc:697] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:54.532347 25574 raft_consensus.cc:2764] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:08:54.530148 25383 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:54.594632 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:54.593452 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:54.574788 22878 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 5.
W20260709 13:08:54.574168 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 71309.8us
I20260709 13:08:54.593233 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:54.592640 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:54.542129 25575 raft_consensus.cc:697] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 9 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:54.596570 23026 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6.
I20260709 13:08:54.597656 25575 consensus_queue.cc:237] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:54.600262 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:54.601357 22874 raft_consensus.cc:2393] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 7: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:54.602451 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:54.595415 23021 raft_consensus.cc:2393] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 8: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:54.604030 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:54.604916 22875 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:08:54.606145 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:54.606873 22874 pending_rounds.cc:85] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 2
I20260709 13:08:54.607373 22874 pending_rounds.cc:107] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 13:08:54.595264 22876 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 5.
I20260709 13:08:54.570932 22918 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:54.607808 22874 raft_consensus.cc:2889] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:54.552405 25559 raft_consensus.cc:2749] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:08:54.605736 25562 log.cc:927] Time spent T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:08:54.645932 22873 raft_consensus.cc:1240] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:08:54.648423 22992 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:54.606809 25573 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:54.653635 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:54.654223 22992 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:08:54.668467 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:54.690708 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 94193.7us
I20260709 13:08:54.592658 25513 raft_consensus.cc:2804] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:08:54.692623 23590 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:54.694170 23619 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
W20260709 13:08:54.701116 22949 consensus_meta.cc:287] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.523s	user 0.001s	sys 0.018s
I20260709 13:08:54.701488 22949 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 6.
I20260709 13:08:54.691882 22992 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:54.566393 25418 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:54.710126 22953 raft_consensus.cc:1275] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
W20260709 13:08:54.714125 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 162128us
I20260709 13:08:54.721010 22953 pending_rounds.cc:85] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting all ops after (but not including) 4
I20260709 13:08:54.653128 25592 raft_consensus.cc:2764] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:08:54.733412 22953 pending_rounds.cc:107] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 13:08:54.738209 22878 raft_consensus.cc:1275] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:08:54.738286 22953 raft_consensus.cc:2889] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:54.653182 25573 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:08:54.743011 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 177433us
I20260709 13:08:54.746018 22843 leader_election.cc:400] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 3 election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 3. Current term is 4.
I20260709 13:08:54.615828 25383 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:54.719409 25444 raft_consensus.cc:2804] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:08:54.756702 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:54.732434 25572 consensus_meta.cc:287] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.611s	user 0.010s	sys 0.011s
I20260709 13:08:54.719899 25571 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:54.738822 22878 pending_rounds.cc:85] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 3
I20260709 13:08:54.767670 22878 pending_rounds.cc:107] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:08:54.768025 22878 raft_consensus.cc:2889] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:54.709869 25599 raft_consensus.cc:2764] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
W20260709 13:08:54.790140 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 229454us
W20260709 13:08:54.792392 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 93798.7us
I20260709 13:08:54.793313 23588 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:08:54.794994 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 103170us
I20260709 13:08:54.659708 25574 consensus_queue.cc:1048] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.116s
I20260709 13:08:54.765818 25444 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:54.803999 23588 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:54.804572 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 55228.1us
I20260709 13:08:54.769953 25370 raft_consensus.cc:2804] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:08:54.805335 22920 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:54.710853 25588 consensus_queue.cc:1048] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.084s
I20260709 13:08:54.833267 22843 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:54.805203 25353 consensus_meta.cc:287] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.621s	user 0.011s	sys 0.001s
I20260709 13:08:54.802502 25559 raft_consensus.cc:2764] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:08:54.764894 25363 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:54.836319 25444 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:08:54.878185 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 72950.4us
I20260709 13:08:54.899860 22920 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:54.874137 22843 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:54.898378 22878 raft_consensus.cc:1275] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:08:54.892998 22826 auto_rebalancer.cc:629] Failed to schedule move for tablet f3b08335f3564868a8160dc1b5ade501: Illegal state: Replica f3c799b1249e40119d2a5864231c6c75 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:08:54.903285 23617 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
I20260709 13:08:54.734714 25561 consensus_queue.cc:237] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:54.842550 25583 raft_consensus.cc:2804] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:54.905604 22843 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:54.849166 25590 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:54.897647 25458 consensus_meta.cc:287] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.718s	user 0.006s	sys 0.009s
I20260709 13:08:54.849511 25565 raft_consensus.cc:2749] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:54.764695 25440 consensus_queue.cc:237] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:54.863234 25600 raft_consensus.cc:2749] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:08:54.897279 25594 log.cc:927] Time spent T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.183s	user 0.000s	sys 0.001s
I20260709 13:08:54.802762 25601 consensus_queue.cc:1048] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.115s
W20260709 13:08:54.950933 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 122615us
I20260709 13:08:54.879669 25598 raft_consensus.cc:2749] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:08:54.954265 22951 consensus_meta.cc:287] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.601s	user 0.006s	sys 0.001s
I20260709 13:08:54.897982 25370 raft_consensus.cc:697] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:54.960990 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:54.961325 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:54.961663 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:54.918789 22920 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:54.913492 25583 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:54.877481 25353 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:55.005125 22919 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.004s	sys 0.003s
W20260709 13:08:54.961990 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 56128.6us
W20260709 13:08:55.039867 22919 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.006s	sys 0.004s
W20260709 13:08:54.967981 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 396672us
I20260709 13:08:55.040500 22843 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:54.964496 22951 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 9.
I20260709 13:08:54.913990 25610 raft_consensus.cc:2804] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 13:08:54.956317 25570 consensus_meta.cc:287] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.885s	user 0.006s	sys 0.005s
I20260709 13:08:54.961012 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:55.044364 22843 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:54.911597 25595 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:54.910459 25607 consensus_queue.cc:1048] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.104s
W20260709 13:08:55.024735 25305 consensus_meta.cc:287] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.837s	user 0.011s	sys 0.002s
I20260709 13:08:55.054415 22843 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:54.930786 25559 consensus_queue.cc:1059] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.097s
I20260709 13:08:54.958921 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:08:55.056488 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 46766.1us
I20260709 13:08:55.043731 25601 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.043447 25600 raft_consensus.cc:2749] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 13:08:55.061463 25318 consensus_meta.cc:287] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.915s	user 0.006s	sys 0.007s
I20260709 13:08:54.968128 23625 raft_consensus.cc:2393] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 7: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
W20260709 13:08:55.160920 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 242189us
I20260709 13:08:54.968726 25563 raft_consensus.cc:2804] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:08:55.050446 25610 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:55.164611 22919 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:54.969419 25418 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:55.043514 25583 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:55.184515 22879 raft_consensus.cc:1240] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 6. Current term is 7. Ops: []
I20260709 13:08:55.162945 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:54.930250 25590 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:55.230877 22951 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 5.
W20260709 13:08:55.214865 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 50196.2us
I20260709 13:08:55.021286 25574 raft_consensus.cc:2749] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:08:55.222306 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 167969us
W20260709 13:08:55.237078 25585 log.cc:927] Time spent T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.431s	user 0.001s	sys 0.000s
W20260709 13:08:55.237310 25587 log.cc:927] Time spent T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.413s	user 0.001s	sys 0.000s
I20260709 13:08:55.238447 25602 raft_consensus.cc:2804] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:55.238826 23025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 1545) took 1014 ms. Trace:
I20260709 13:08:55.238914 25602 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:55.239714 25616 raft_consensus.cc:2749] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:55.240507 25624 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.239389 25602 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:55.241182 25624 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:55.243830 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:55.184432 25595 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:55.246915 23021 raft_consensus.cc:1275] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 13:08:55.247387 23021 pending_rounds.cc:85] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75: Aborting all ops after (but not including) 2
I20260709 13:08:55.247686 23021 pending_rounds.cc:107] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 13:08:55.248075 23021 raft_consensus.cc:2889] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:55.239269 23025 rpcz_store.cc:276] 0709 13:08:54.224794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:54.237165 (+ 12371us) service_pool.cc:224] Handling call
0709 13:08:54.258602 (+ 21437us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:54.273965 (+ 15363us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:54.273995 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:54.274410 (+   415us) log.cc:844] Serialized 26 byte log entry
0709 13:08:54.449056 (+174646us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:54.449102 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:54.449164 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:55.238576 (+789412us) raft_consensus.cc:1685] finished
0709 13:08:55.238597 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:55.238747 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1715,"raft.queue_time_us":36017,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":15035,"thread_start_us":91046,"threads_started":1,"wal-append.queue_time_us":195016}
I20260709 13:08:54.930505 25458 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:54.965230 25246 consensus_meta.cc:287] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.770s	user 0.009s	sys 0.008s
I20260709 13:08:55.239389 22843 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:55.243799 25605 consensus_queue.cc:1059] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.277s
I20260709 13:08:55.273402 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:08:55.229921 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 268529us
W20260709 13:08:55.232656 25252 consensus_meta.cc:287] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.794s	user 0.009s	sys 0.001s
W20260709 13:08:55.277763 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 55135.5us
I20260709 13:08:55.278671 22920 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:55.279649 22872 raft_consensus.cc:1275] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:08:55.227268 25563 raft_consensus.cc:697] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:55.281561 25563 consensus_queue.cc:237] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:55.002734 25471 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:55.231206 25589 log.cc:927] Time spent T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.315s	user 0.001s	sys 0.000s
I20260709 13:08:55.283186 23588 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:08:55.270457 25513 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.001s	sys 0.005s
I20260709 13:08:55.098191 25570 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:55.277566 25252 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:55.231979 25418 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.284708 22880 raft_consensus.cc:2393] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 7: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:55.284981 25418 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:55.285530 25252 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:55.285877 25605 consensus_queue.cc:1048] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:08:55.286273 25252 raft_consensus.cc:3060] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:55.287207 25605 raft_consensus.cc:2804] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:08:55.287662 25605 raft_consensus.cc:697] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 6 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:55.289011 25628 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:55.289445 25628 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:55.289788 25628 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:08:55.292459 25618 log.cc:927] Time spent T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:08:55.285530 25570 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:55.292994 23588 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:55.293591 25570 consensus_queue.cc:1059] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:55.294516 25574 raft_consensus.cc:3055] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:08:55.295001 25574 raft_consensus.cc:740] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:55.218879 25610 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:55.298030 22843 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:55.330968 22843 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:55.299527 22920 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:55.333539 22920 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:54.984686 25228 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:55.307830 22872 raft_consensus.cc:1275] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:08:55.267138 22871 raft_consensus.cc:1240] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 8. Current term is 9. Ops: []
I20260709 13:08:55.265218 23620 raft_consensus.cc:2393] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 8: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:55.338780 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:55.296152 25574 consensus_queue.cc:260] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:55.381294 25574 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:08:55.382579 25507 consensus_meta.cc:287] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.998s	user 0.009s	sys 0.000s
I20260709 13:08:55.383044 25507 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:55.384244 23623 raft_consensus.cc:1240] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:55.342612 25627 raft_consensus.cc:2804] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:55.288477 25605 consensus_queue.cc:237] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:55.298713 25513 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.001s	sys 0.005s
I20260709 13:08:55.346148 25590 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:55.285435 25614 raft_consensus.cc:2749] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:55.349720 25621 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.351562 25458 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:55.236241 25383 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:55.351429 25628 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:55.352593 25559 raft_consensus.cc:3055] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:08:55.357206 25568 consensus_meta.cc:287] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.837s	user 0.010s	sys 0.000s
I20260709 13:08:55.225124 25601 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:55.226838 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:55.276235 25246 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:55.361653 25328 consensus_meta.cc:287] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 1.041s	user 0.007s	sys 0.003s
W20260709 13:08:55.362375 25573 consensus_meta.cc:287] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.572s	user 0.006s	sys 0.001s
I20260709 13:08:55.439447 25573 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:55.362460 25396 consensus_meta.cc:287] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.873s	user 0.007s	sys 0.002s
I20260709 13:08:55.274691 25588 consensus_queue.cc:1059] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.096s
I20260709 13:08:55.451134 25396 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:55.451673 25588 raft_consensus.cc:3055] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:08:55.452050 25588 raft_consensus.cc:740] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:55.452348 23624 raft_consensus.cc:1240] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:55.453127 25590 raft_consensus.cc:2749] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:08:55.453399 25396 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:55.453876 23624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60044 (request call id 561) took 1065 ms. Trace:
I20260709 13:08:55.454305 23624 rpcz_store.cc:276] 0709 13:08:54.388115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:54.388225 (+   110us) service_pool.cc:224] Handling call
0709 13:08:54.416574 (+ 28349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:55.452780 (+1036206us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:55.453626 (+   846us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b54002bc854. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:55.453794 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2381467392}
W20260709 13:08:55.291270 25291 consensus_meta.cc:287] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.872s	user 0.006s	sys 0.003s
I20260709 13:08:55.456915 25291 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:55.458935 25291 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:55.459378 25602 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:55.408871 25621 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:55.463044 25621 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:55.409144 25571 raft_consensus.cc:2749] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:55.364336 25600 raft_consensus.cc:2749] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:55.466490 22948 raft_consensus.cc:2393] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 8: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:55.467689 22948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58320 (request call id 2175) took 1026 ms. Trace:
I20260709 13:08:55.468055 22948 rpcz_store.cc:276] 0709 13:08:54.441196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:54.451183 (+  9987us) service_pool.cc:224] Handling call
0709 13:08:55.467491 (+1016308us) spinlock_profiling.cc:243] Waited 992 ms on lock 0x7b5400411ed4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:55.467632 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2281195264}
I20260709 13:08:55.370416 25228 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:55.371060 22950 raft_consensus.cc:2393] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 6: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:55.372031 25353 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:55.373010 25603 raft_consensus.cc:2804] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:08:55.374176 25629 raft_consensus.cc:2764] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:08:55.374969 25596 raft_consensus.cc:2749] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:08:55.344467 25444 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:55.489912 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:55.379237 25363 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:08:55.385177 22993 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.819s	user 0.009s	sys 0.022s
W20260709 13:08:55.491243 22993 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.925s	user 0.010s	sys 0.022s
I20260709 13:08:55.492132 23619 raft_consensus.cc:2368] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 5. Current term is 6.
I20260709 13:08:55.492846 25603 raft_consensus.cc:697] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 5 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:55.345752 25471 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:55.493315 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:08:55.386173 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:55.400532 22946 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.007s	sys 0.022s
I20260709 13:08:55.493824 25603 consensus_queue.cc:237] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:55.405913 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 113003us
I20260709 13:08:55.385341 25593 consensus_queue.cc:1048] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.484s
I20260709 13:08:55.385334 25627 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:55.433352 25507 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:55.419466 25568 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:55.434717 25252 raft_consensus.cc:2749] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:55.274114 25370 consensus_queue.cc:237] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:55.435738 25601 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:55.436369 23618 raft_consensus.cc:2393] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 7: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:55.436757 25636 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.438508 25246 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:55.438968 25328 raft_consensus.cc:2749] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:55.450474 25628 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:55.452958 25588 consensus_queue.cc:260] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:55.211400 25305 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:55.388321 25513 raft_consensus.cc:697] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 8 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:55.559013 23588 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:55.470752 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:55.560027 22951 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
I20260709 13:08:55.559906 25513 consensus_queue.cc:237] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:55.560884 25305 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:55.560987 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:55.561640 22951 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:55.471849 22953 raft_consensus.cc:1275] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 13:08:55.473088 22949 raft_consensus.cc:1275] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:08:55.366356 22952 raft_consensus.cc:1240] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
W20260709 13:08:55.476765 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 143307us
I20260709 13:08:55.413956 25559 raft_consensus.cc:740] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:55.265908 25616 raft_consensus.cc:2804] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:55.491108 25444 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:08:55.400614 22955 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.018s	sys 0.014s
I20260709 13:08:55.246935 25595 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:55.540210 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:55.541236 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:55.535508 23028 raft_consensus.cc:1240] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 6. Current term is 7. Ops: []
W20260709 13:08:55.542030 22946 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.713s	user 0.011s	sys 0.022s
I20260709 13:08:55.493912 22947 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
I20260709 13:08:55.536576 22993 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:55.543759 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:55.622587 22871 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6.
I20260709 13:08:55.492473 25624 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:55.549826 25636 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:55.555621 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:55.557098 25634 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.633771 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:55.557307 25573 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:55.550817 25596 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.547278 25484 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:55.642258 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 105736us
I20260709 13:08:55.544561 25627 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:55.563429 25588 raft_consensus.cc:3060] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:55.547475 25438 consensus_queue.cc:1059] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 13:08:55.448172 25583 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:55.616775 22951 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:08:55.573442 22953 raft_consensus.cc:1240] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 8. Current term is 9. Ops: []
I20260709 13:08:55.617530 23588 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:55.620749 22992 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:08:55.578420 22955 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.020s	sys 0.015s
W20260709 13:08:55.624550 22995 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.013s	sys 0.027s
I20260709 13:08:55.624789 25601 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.624919 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:55.612952 25629 raft_consensus.cc:2804] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:55.670632 23617 raft_consensus.cc:2393] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 6: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:55.633671 25471 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:55.627270 25610 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:55.628882 25602 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:55.620646 23590 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:55.629628 25228 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.631208 23623 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 566) took 1241 ms. Trace:
I20260709 13:08:55.685966 23623 rpcz_store.cc:276] 0709 13:08:54.389720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:54.404628 (+ 14908us) service_pool.cc:224] Handling call
0709 13:08:54.435445 (+ 30817us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:55.384651 (+949206us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:55.630934 (+246283us) spinlock_profiling.cc:243] Waited 948 ms on lock 0x7b54003d90d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:55.631139 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2181461248}
I20260709 13:08:55.632196 22948 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:55.564816 23618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60044 (request call id 562) took 1164 ms. Trace:
I20260709 13:08:55.635576 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:55.636650 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:55.571704 25318 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.571534 25616 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:55.641844 23625 raft_consensus.cc:1240] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:08:55.566375 22920 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:08:55.646847 22995 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.793s	user 0.013s	sys 0.027s
I20260709 13:08:55.639389 25561 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.643376 25596 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:55.679179 25438 raft_consensus.cc:3055] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:08:55.679849 25636 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:55.680540 25559 consensus_queue.cc:260] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:55.639667 25634 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:55.650370 22992 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:55.695971 25593 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:55.697956 22876 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:08:55.700747 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 133015us
I20260709 13:08:55.649230 22874 raft_consensus.cc:1240] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 5 LEADER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:55.703323 25604 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:55.705138 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 84588.5us
I20260709 13:08:55.634681 25560 consensus_queue.cc:1048] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 13:08:55.700440 25228 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:55.710106 22871 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 5.
I20260709 13:08:55.668851 25363 raft_consensus.cc:2804] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:08:55.670693 25629 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:55.700357 25241 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.719733 22949 raft_consensus.cc:1240] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:08:55.685359 25573 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:08:55.721912 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 71624.4us
W20260709 13:08:55.642092 25638 log.cc:927] Time spent T 00000000000000000000000000000000 P 8d175b4ae72f4fc48971c2f31668971c: Append to log took a long time: real 0.164s	user 0.000s	sys 0.002s
I20260709 13:08:55.696445 25291 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.696702 25603 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:55.649578 25484 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:55.672159 25601 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:56.163955 22948 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
I20260709 13:08:56.070333 23624 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
W20260709 13:08:55.717250 25226 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.160s	user 0.014s	sys 0.019s
W20260709 13:08:56.171672 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 605276us
I20260709 13:08:56.170693 23588 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:55.715487 25637 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:08:56.293766 22844 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.63.4:44043 timed out after 1.625s (SENT) [suppressed 29 similar messages]
W20260709 13:08:56.129055 25640 log.cc:927] Time spent T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.433s	user 0.001s	sys 0.000s
W20260709 13:08:56.385221 22875 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.755s	user 0.004s	sys 0.014s
W20260709 13:08:56.406209 22875 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.776s	user 0.005s	sys 0.014s
W20260709 13:08:56.338835 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 172569us
I20260709 13:08:55.704943 25616 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:56.160233 25634 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:56.415089 25634 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.415419 22876 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6.
I20260709 13:08:56.415678 25634 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:56.416179 25657 consensus_queue.cc:1048] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 13:08:56.417091 25565 consensus_queue.cc:1059] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:08:56.419292 22954 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2183) took 1864 ms. Trace:
I20260709 13:08:56.419715 22954 rpcz_store.cc:276] 0709 13:08:54.554558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:54.569926 (+ 15368us) service_pool.cc:224] Handling call
0709 13:08:54.570223 (+   297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:55.503149 (+932926us) spinlock_profiling.cc:243] Waited 32.1 ms on lock 0x7b18003dd440. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f613598d4bc 00007f6132cdb48b 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf5455 00007f6135ddd060 00007f6135dddbfb 00007f6135de864b
0709 13:08:55.503377 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:55.503410 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:55.544869 (+ 41459us) log.cc:844] Serialized 26 byte log entry
0709 13:08:55.715368 (+170499us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:08:56.385564 (+670196us) log.cc:844] Serialized 14 byte log entry
0709 13:08:56.385871 (+   307us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:56.385969 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:56.418941 (+ 32972us) raft_consensus.cc:1685] finished
0709 13:08:56.419059 (+   118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:56.419226 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4149,"raft.queue_time_us":204942,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":35609,"spinlock_wait_cycles":73838464,"thread_start_us":740886,"threads_started":3,"wal-append.queue_time_us":862662}
I20260709 13:08:56.422257 25440 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.422856 25440 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:56.385488 25596 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:56.425398 25596 raft_consensus.cc:2749] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:56.425597 25440 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:56.162262 25583 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:56.080014 25648 log.cc:927] Time spent T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 13:08:56.429039 25440 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:56.429180 25651 log.cc:927] Time spent T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.233s	user 0.000s	sys 0.001s
I20260709 13:08:56.429559 25440 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:56.431813 25440 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:56.432370 25440 consensus_queue.cc:1059] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:08:56.434244 25658 raft_consensus.cc:3055] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:08:56.434660 25658 raft_consensus.cc:740] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:56.435565 25658 consensus_queue.cc:260] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:55.718477 25471 raft_consensus.cc:2804] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:08:55.723294 25624 raft_consensus.cc:2804] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:55.715687 25438 raft_consensus.cc:740] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:56.174938 25629 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:56.019001 25636 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.352936 25639 raft_consensus.cc:2804] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:56.303043 25595 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:55.984768 25644 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.018675 25444 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:55.979823 25574 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.299932 25628 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:56.326107 25372 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.327816 25602 consensus_queue.cc:1048] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.124s
I20260709 13:08:55.925180 25328 raft_consensus.cc:2804] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:08:56.413131 25645 log.cc:927] Time spent T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 13:08:56.196138 25291 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:56.207230 25603 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:56.200873 25226 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.648s	user 0.053s	sys 0.034s
I20260709 13:08:56.190673 25559 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:55.697423 23618 rpcz_store.cc:276] 0709 13:08:54.388428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:54.398991 (+ 10563us) service_pool.cc:224] Handling call
0709 13:08:55.184236 (+785245us) spinlock_profiling.cc:243] Waited 22.3 ms on lock 0x7b58001c4d98. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0a22 00007f613bbd9c09 00007f613ad33ee0 00007f613adfd0ab 00007f613adfd131 00007f613adab0a0 00007f613adaa984 00007f61343256d4 00007f6134325662
0709 13:08:55.552740 (+368504us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":51200768}
I20260709 13:08:56.198760 25241 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:56.197687 25575 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.331161 25623 raft_consensus.cc:2749] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:08:56.327229 25610 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:55.719692 25318 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:56.513310 25575 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:56.514707 25241 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:56.515275 25318 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:56.515578 25241 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.515857 25318 raft_consensus.cc:3055] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:08:56.516237 25318 raft_consensus.cc:740] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:56.517185 25318 consensus_queue.cc:260] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:56.453114 22955 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.768s	user 0.137s	sys 0.174s
W20260709 13:08:56.518256 22955 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.833s	user 0.137s	sys 0.174s
I20260709 13:08:56.482793 25372 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:55.717798 25561 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:56.519980 25372 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:56.520336 25561 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:56.312494 25568 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:56.446494 25624 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:56.521159 25624 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:56.482668 23590 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:56.525837 23590 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:56.481201 25328 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:56.526435 25328 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:56.526521 22875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 1609) took 1925 ms. Trace:
I20260709 13:08:56.450958 25658 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:56.527132 22875 rpcz_store.cc:276] 0709 13:08:54.600850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:54.732492 (+131642us) service_pool.cc:224] Handling call
0709 13:08:54.745596 (+ 13104us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:55.259647 (+514051us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:55.259692 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:55.629750 (+370058us) spinlock_profiling.cc:243] Waited 7.1 ms on lock 0x7b2c000b6b68. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f61346dea11 00007f61346ee372 00007f6135e20a01 00007f6135de543a 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2
0709 13:08:55.630105 (+   355us) log.cc:844] Serialized 26 byte log entry
0709 13:08:56.406683 (+776578us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:56.406733 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:56.408305 (+  1572us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:56.526117 (+117812us) raft_consensus.cc:1685] finished
0709 13:08:56.526150 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:56.526445 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19556,"raft.queue_time_us":222803,"raft.run_cpu_time_us":288,"raft.run_wall_time_us":57327,"spinlock_wait_cycles":16334720,"thread_start_us":1236321,"threads_started":2,"wal-append.queue_time_us":858519}
I20260709 13:08:56.530083 25291 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:56.198261 25604 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:56.532343 25604 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:56.532835 25596 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:56.533200 25596 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:56.533505 25596 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:08:56.533881 25588 consensus_meta.cc:287] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.863s	user 0.005s	sys 0.005s
I20260709 13:08:56.534438 25588 raft_consensus.cc:2804] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:56.534775 25588 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:56.535087 25588 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:56.483959 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:56.536334 23619 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:08:56.484818 23625 raft_consensus.cc:1275] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 13:08:56.445727 25637 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:56.198596 25573 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:56.452369 25639 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:56.454459 25471 raft_consensus.cc:697] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:56.538050 25639 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:56.538542 25637 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:56.538965 25471 consensus_queue.cc:237] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:56.488942 25226 raft_consensus.cc:697] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:56.458549 25444 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:56.541606 25226 consensus_queue.cc:237] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:56.542670 25444 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:56.482898 25628 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:56.457815 25644 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:56.492089 25601 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:56.458666 25574 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:56.487196 25484 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:56.455878 25595 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:56.498057 25438 consensus_queue.cc:260] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:56.499514 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:56.505843 25634 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:56.540419 25603 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:56.545761 22919 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:56.455098 25636 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:56.558199 23616 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:08:56.556921 25610 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:56.574851 25636 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:56.552337 25291 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.577239 25291 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:56.574285 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 373127us
W20260709 13:08:56.559496 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 361653us
W20260709 13:08:56.562011 25627 consensus_meta.cc:287] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.890s	user 0.009s	sys 0.000s
I20260709 13:08:56.563558 25644 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:56.563647 25595 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:56.568986 25438 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:56.571595 25573 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:56.571880 25593 raft_consensus.cc:2749] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:56.516250 25241 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:56.623160 25291 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:56.624456 25241 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:08:56.588340 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 417677us
I20260709 13:08:56.625872 23624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 572) took 1009 ms. Trace:
I20260709 13:08:56.626394 23624 rpcz_store.cc:276] 0709 13:08:55.616652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:55.617949 (+  1297us) service_pool.cc:224] Handling call
0709 13:08:56.625652 (+1007703us) spinlock_profiling.cc:243] Waited 405 ms on lock 0x7b54005aebd4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:56.625799 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":930449280}
I20260709 13:08:56.628108 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:56.633642 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:56.633803 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:56.634552 22953 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 7.
I20260709 13:08:56.634703 22951 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:56.617041 25636 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.639180 22954 raft_consensus.cc:1240] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 7. Current term is 8. Ops: []
I20260709 13:08:56.612258 25628 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:56.643332 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:56.643680 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:56.649521 22947 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 5.
I20260709 13:08:56.649677 22953 raft_consensus.cc:2393] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 7: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:56.609277 25363 raft_consensus.cc:697] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:56.613749 25629 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:56.660475 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 1.61865e+06us
I20260709 13:08:56.655282 25328 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:56.661671 22918 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
W20260709 13:08:56.664754 23588 connection.cc:256] RPC call timeout handler was delayed by -1.4779022279999481s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1724s for a response
I20260709 13:08:56.665076 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:56.666275 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:56.666987 22953 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 7.
I20260709 13:08:56.679847 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:56.681128 22947 raft_consensus.cc:2393] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 7: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:56.681655 22874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53724 (request call id 418) took 1144 ms. Trace:
I20260709 13:08:56.664271 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:56.684206 22843 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:56.682118 22874 rpcz_store.cc:276] 0709 13:08:55.537508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:55.537647 (+   139us) service_pool.cc:224] Handling call
0709 13:08:55.538049 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:56.165654 (+627605us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:56.681441 (+515787us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b540022a6d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:56.681600 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":255209344}
I20260709 13:08:56.619537 25627 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:56.699800 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 406061us
I20260709 13:08:56.700464 22873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53724 (request call id 413) took 2121 ms. Trace:
W20260709 13:08:56.698809 23588 connection.cc:256] RPC call timeout handler was delayed by -1.2939657449999231s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15400000000000003s for a response
I20260709 13:08:56.700991 22954 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6.
I20260709 13:08:56.621824 25639 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:56.705752 23588 connection.cc:256] RPC call timeout handler was delayed by -1.2385022279999021s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.199s for a response
I20260709 13:08:56.699445 25573 raft_consensus.cc:2804] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:08:56.703971 22873 rpcz_store.cc:276] 0709 13:08:54.559044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:54.559152 (+   108us) service_pool.cc:224] Handling call
0709 13:08:54.592598 (+ 33446us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:54.650466 (+ 57868us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:56.680682 (+2030216us) spinlock_profiling.cc:243] Waited 47.8 ms on lock 0x7b540030cad4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:56.680848 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":110016896}
I20260709 13:08:56.657702 25629 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:56.702571 23620 raft_consensus.cc:1240] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 7. Current term is 8. Ops: []
I20260709 13:08:56.706647 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:08:56.707006 23588 connection.cc:256] RPC call timeout handler was delayed by -1.2054162689998975s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1655s for a response
W20260709 13:08:56.719931 23588 connection.cc:256] RPC call timeout handler was delayed by -1.1475655899998856s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17210000000000003s for a response
W20260709 13:08:56.721133 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 59519.7us
I20260709 13:08:56.702137 25634 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.723057 22918 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:56.649427 25616 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:56.710206 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:56.716833 25573 raft_consensus.cc:697] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 9 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:56.703142 25628 consensus_queue.cc:1059] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:08:56.664109 25636 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:56.707612 25623 raft_consensus.cc:2804] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 13:08:56.709108 22844 leader_election.cc:336] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Timed out: RequestConsensusVote RPC to 127.21.63.4:44043 timed out after 1.891s (SENT)
I20260709 13:08:56.736413 22955 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2181) took 2220 ms. Trace:
I20260709 13:08:56.717464 25658 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:08:56.722743 25601 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:08:56.734099 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 206678us
I20260709 13:08:56.655834 25363 consensus_queue.cc:237] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:56.737898 22951 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6.
I20260709 13:08:56.740109 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:56.736440 22844 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:08:56.742666 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:56.743010 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:56.738927 23588 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:56.743425 23590 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:56.750049 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:56.742952 22955 rpcz_store.cc:276] 0709 13:08:54.512890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:54.513709 (+   819us) service_pool.cc:224] Handling call
0709 13:08:54.530403 (+ 16694us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:54.770237 (+239834us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:54.770286 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:54.800750 (+ 30464us) log.cc:844] Serialized 26 byte log entry
0709 13:08:55.646870 (+846120us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:08:56.518622 (+871752us) log.cc:844] Serialized 14 byte log entry
0709 13:08:56.733051 (+214429us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:56.733209 (+   158us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:56.733267 (+    58us) raft_consensus.cc:1685] finished
0709 13:08:56.733281 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:56.733440 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50463,"prepare.queue_time_us":923466,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":27,"raft.queue_time_us":243720,"raft.run_cpu_time_us":408,"raft.run_wall_time_us":41545,"thread_start_us":2009262,"threads_started":4,"wal-append.queue_time_us":624349,"wal-append.run_cpu_time_us":2148,"wal-append.run_wall_time_us":1012346}
W20260709 13:08:56.705797 25632 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.004s	sys 0.012s
I20260709 13:08:56.733300 25650 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:56.734169 25660 raft_consensus.cc:2749] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 13:08:56.757006 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 47931.6us
I20260709 13:08:56.762719 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:56.763581 22951 raft_consensus.cc:2393] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 6: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:56.741351 25664 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:56.752898 25632 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.004s	sys 0.012s
I20260709 13:08:56.772337 22844 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:56.729372 25634 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:56.776022 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:56.749548 25560 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.755627 25663 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.756893 25444 raft_consensus.cc:2804] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:08:56.781545 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:56.772382 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:56.756121 25574 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:56.784770 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:56.757256 25623 raft_consensus.cc:697] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:56.765483 25628 raft_consensus.cc:2749] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:08:56.740947 25629 consensus_queue.cc:1048] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:08:56.788468 22844 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:56.740967 25596 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:56.781932 25593 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:56.786962 25513 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:56.791391 22951 raft_consensus.cc:2393] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 6: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
W20260709 13:08:56.793952 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 55091.8us
I20260709 13:08:56.785962 25328 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:56.766229 25659 raft_consensus.cc:2749] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:08:56.753712 25601 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:56.797114 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:56.819880 22955 raft_consensus.cc:2368] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 5. Current term is 6.
I20260709 13:08:56.783653 25625 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:56.802626 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:08:56.795192 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 44130.7us
I20260709 13:08:56.805341 23588 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:56.788813 25353 raft_consensus.cc:2749] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:08:56.759572 25658 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:56.760293 25662 consensus_queue.cc:1048] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:08:56.792152 25610 raft_consensus.cc:2749] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:08:56.827111 22954 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 7.
I20260709 13:08:56.822816 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:08:56.850587 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 66009.2us
I20260709 13:08:56.850852 22951 raft_consensus.cc:2393] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 8: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:56.790323 25573 consensus_queue.cc:237] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:56.824692 25601 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:56.794679 25444 raft_consensus.cc:697] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 6 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:56.784574 25664 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:56.852195 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:56.856810 22954 raft_consensus.cc:1240] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:56.857232 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:56.858008 22955 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
W20260709 13:08:56.858224 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 52963.3us
I20260709 13:08:56.859525 23588 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:56.860775 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:56.861336 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:56.862080 22872 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:56.848410 25595 raft_consensus.cc:3055] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:08:56.865546 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:56.789631 25560 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:56.767252 25650 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:56.819722 25605 raft_consensus.cc:2749] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:08:56.873497 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:56.889621 22871 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 7.
I20260709 13:08:56.872804 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:56.890727 22875 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
I20260709 13:08:56.891522 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:56.892112 22875 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:56.892418 22955 raft_consensus.cc:1240] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 6. Current term is 7. Ops: []
I20260709 13:08:56.891440 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:56.893847 22874 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:56.874477 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:56.894791 22873 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 5.
I20260709 13:08:56.846630 25418 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:08:56.899760 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 186471us
I20260709 13:08:56.843887 25634 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:08:56.909607 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 50183.8us
I20260709 13:08:56.902204 22954 raft_consensus.cc:1240] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:08:56.905525 23024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 1558) took 1659 ms. Trace:
I20260709 13:08:56.906344 25596 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:56.854405 25440 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:56.907202 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:56.910912 22871 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6.
I20260709 13:08:56.855439 25588 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:56.912703 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 85406.7us
I20260709 13:08:56.911896 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:56.865183 25595 raft_consensus.cc:740] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:56.912632 22954 raft_consensus.cc:1275] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 13:08:56.933720 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:56.821321 25513 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:56.902024 25639 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:56.910529 23024 rpcz_store.cc:276] 0709 13:08:55.246049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:55.260157 (+ 14108us) service_pool.cc:224] Handling call
0709 13:08:55.271046 (+ 10889us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:55.271371 (+   325us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:55.271397 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:56.411156 (+1139759us) spinlock_profiling.cc:243] Waited 3.6 ms on lock 0x7b2c000bc418. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f61346dea11 00007f61346ee372 00007f6135e20a01 00007f6135de543a 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2
0709 13:08:56.411577 (+   421us) log.cc:844] Serialized 26 byte log entry
0709 13:08:56.735400 (+323823us) raft_consensus.cc:1645] Marking committed up to 1
0709 13:08:56.735438 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:56.735506 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:56.905252 (+169746us) raft_consensus.cc:1685] finished
0709 13:08:56.905271 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:56.905467 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19937,"raft.queue_time_us":57948,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":115,"spinlock_wait_cycles":8289536,"thread_start_us":323476,"threads_started":1,"wal-append.queue_time_us":385398}
I20260709 13:08:56.915323 23623 raft_consensus.cc:1275] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:08:56.916263 22920 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:56.937444 22875 raft_consensus.cc:2410] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:08:56.916936 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:56.938190 22920 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:56.920012 22946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2182) took 2388 ms. Trace:
I20260709 13:08:56.856339 25659 consensus_queue.cc:1059] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:08:56.921124 22952 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2184) took 2324 ms. Trace:
I20260709 13:08:56.921676 25658 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:56.873044 25678 raft_consensus.cc:2749] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:56.817000 25593 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:56.923923 25623 consensus_queue.cc:237] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:56.924968 25627 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:56.926795 25664 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:56.929452 22955 raft_consensus.cc:1275] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:08:56.794689 25663 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:56.872299 25637 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:56.934741 22873 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6.
I20260709 13:08:56.946019 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:56.945578 22952 rpcz_store.cc:276] 0709 13:08:54.596546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:54.597750 (+  1204us) service_pool.cc:224] Handling call
0709 13:08:54.598226 (+   476us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:55.565329 (+967103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:56.920879 (+1355550us) spinlock_profiling.cc:243] Waited 767 ms on lock 0x7b5400285854. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:56.921062 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1764248832}
I20260709 13:08:56.947249 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:56.949678 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:56.938961 22946 rpcz_store.cc:276] 0709 13:08:54.531314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:54.531418 (+   104us) service_pool.cc:224] Handling call
0709 13:08:54.556099 (+ 24681us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:54.758433 (+202334us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:54.758490 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:54.769716 (+ 11226us) log.cc:844] Serialized 26 byte log entry
0709 13:08:55.633726 (+864010us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:08:56.038449 (+404723us) log.cc:844] Serialized 14 byte log entry
0709 13:08:56.131393 (+ 92944us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:56.131515 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:56.919503 (+787988us) spinlock_profiling.cc:243] Waited 4.43 ms on lock 0x7b5400170320. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135d5adbe 00007f6135d65930 00007f6135d658d8 00007f6135d65858 00007f6135d65813 00007f6135d64a9c 00007f613bc50626 00007f613bc4add8 00007f6135da17cb 00007f6135da42c0
0709 13:08:56.919730 (+   227us) raft_consensus.cc:1685] finished
0709 13:08:56.919830 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:56.919961 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3441,"prepare.queue_time_us":924159,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":58,"raft.queue_time_us":233616,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":32357,"spinlock_wait_cycles":72477184,"thread_start_us":1248849,"threads_started":3,"wal-append.queue_time_us":745684}
I20260709 13:08:56.930824 25559 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:08:56.999841 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:57.000605 22871 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
I20260709 13:08:57.001885 23624 raft_consensus.cc:2410] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 8 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 13:08:56.865875 25625 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:56.946199 25440 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:57.042236 25637 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:56.963980 25624 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:57.044095 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:57.045265 22879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2102) took 2693 ms. Trace:
I20260709 13:08:57.060439 22955 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:57.060379 22879 rpcz_store.cc:276] 0709 13:08:54.352279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:54.352418 (+   139us) service_pool.cc:224] Handling call
0709 13:08:54.360983 (+  8565us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:55.254263 (+893280us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:57.045011 (+1790748us) spinlock_profiling.cc:243] Waited 800 ms on lock 0x7b54003bc754. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:57.045212 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1840935296}
I20260709 13:08:56.999730 25616 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:57.045945 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:57.049134 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:56.974670 22878 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:57.011993 23621 raft_consensus.cc:2393] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 8: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:56.947595 25660 raft_consensus.cc:2804] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:57.053900 22872 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6.
I20260709 13:08:56.947755 25680 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:57.057816 22874 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
W20260709 13:08:57.058121 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 117806us
I20260709 13:08:57.073957 25627 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.078166 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:56.963367 25659 consensus_queue.cc:1243] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer 0a407504ed1741bf89a276558adf4c0f log is divergent from this leader: its last log entry 5.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:08:57.084045 22872 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:08:57.085769 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:57.084230 22951 raft_consensus.cc:1240] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:56.924986 25444 consensus_queue.cc:237] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:57.054474 25559 raft_consensus.cc:697] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 8 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:57.054864 25636 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:57.127566 22952 raft_consensus.cc:1275] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:08:57.086943 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:57.120613 22872 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 8.
I20260709 13:08:57.191392 22878 raft_consensus.cc:2410] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 13:08:57.192967 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 110012us
I20260709 13:08:57.129562 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:57.058280 25595 consensus_queue.cc:260] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:57.195796 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:57.196966 23623 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:57.197001 22879 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:57.084770 25678 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.202688 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:57.237238 23624 raft_consensus.cc:2368] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 7. Current term is 8.
I20260709 13:08:57.237025 25678 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:57.238212 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:57.238816 23618 raft_consensus.cc:1240] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 8. Current term is 9. Ops: []
I20260709 13:08:57.239355 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:57.239786 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:57.202795 25559 consensus_queue.cc:237] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:57.203924 23622 raft_consensus.cc:1240] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 6. Current term is 7. Ops: []
I20260709 13:08:57.166841 25679 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:57.241683 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:57.241420 25679 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:57.242972 23622 raft_consensus.cc:3055] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:08:57.243312 23622 raft_consensus.cc:740] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:57.244417 25679 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:57.089099 25627 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:57.245244 25593 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:57.240454 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:57.247363 23624 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 7.
I20260709 13:08:57.079406 25680 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:57.205788 23588 leader_election.cc:336] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Timed out: RequestConsensusVote RPC to 127.21.63.2:33309 timed out after 1.540s (SENT)
I20260709 13:08:57.209712 22955 raft_consensus.cc:2468] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 5.
I20260709 13:08:57.212347 23028 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 455) took 2658 ms. Trace:
I20260709 13:08:57.246037 23622 consensus_queue.cc:260] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:57.257203 23028 rpcz_store.cc:276] 0709 13:08:54.553897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:54.554035 (+   138us) service_pool.cc:224] Handling call
0709 13:08:54.594823 (+ 40788us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:55.578393 (+983570us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:57.212086 (+1633693us) spinlock_profiling.cc:243] Waited 940 ms on lock 0x7b5400359154. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:57.212261 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2162682624}
I20260709 13:08:57.216074 25624 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:57.218719 22992 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.018s	user 0.011s	sys 0.012s
I20260709 13:08:57.259732 23588 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:57.260180 22992 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.060s	user 0.012s	sys 0.012s
W20260709 13:08:57.261410 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 1.08764e+06us
I20260709 13:08:57.227304 25663 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:57.230015 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:08:57.231241 25681 log.cc:927] Time spent T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.155s	user 0.000s	sys 0.000s
W20260709 13:08:57.231590 22877 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53724 (request call id 414) took 2672 ms (client timeout 1945 ms). Trace:
I20260709 13:08:57.233060 25575 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:57.235049 22843 leader_election.cc:400] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 5. Current term is 6.
I20260709 13:08:57.240420 25678 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:57.266191 22877 rpcz_store.cc:269] 0709 13:08:54.559341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:54.569980 (+ 10639us) service_pool.cc:224] Handling call
0709 13:08:57.231364 (+2661384us) spinlock_profiling.cc:243] Waited 1.56 s on lock 0x7b5400265554. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:57.231497 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3581217024}
I20260709 13:08:57.240499 23618 raft_consensus.cc:2368] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 6. Current term is 7.
I20260709 13:08:57.079145 25660 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:57.257865 23623 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 7.
W20260709 13:08:57.214944 25438 consensus_meta.cc:287] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.586s	user 0.004s	sys 0.004s
I20260709 13:08:57.258251 23622 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:57.166522 25228 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:57.257385 25595 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:57.266474 22843 leader_election.cc:403] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 13:08:57.204833 22952 raft_consensus.cc:1240] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:08:57.266397 22992 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:57.084404 23616 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
I20260709 13:08:57.265506 22874 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 7.
I20260709 13:08:57.275754 22992 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:57.278111 25678 raft_consensus.cc:2749] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:08:57.258033 25370 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.274901 22843 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:57.280913 22992 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:57.257741 25513 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:57.282316 22843 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:57.268735 25560 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:57.270967 25660 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:57.258368 25680 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:57.303758 23622 raft_consensus.cc:1275] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 13:08:57.282866 25575 raft_consensus.cc:2749] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 5. Current term is 6.
I20260709 13:08:57.270578 25246 consensus_queue.cc:1048] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 13:08:57.310549 22955 raft_consensus.cc:1275] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
W20260709 13:08:57.315860 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 110089us
W20260709 13:08:57.315861 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:57.315922 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:57.332098 25353 consensus_queue.cc:1048] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 13:08:57.310400 25615 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:08:57.325155 23622 pending_rounds.cc:85] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2: Aborting all ops after (but not including) 5
I20260709 13:08:57.311843 25665 consensus_queue.cc:1059] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 13:08:57.312000 25328 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:57.309896 22879 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 7.
I20260709 13:08:57.328840 22948 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 5.
W20260709 13:08:57.489003 25690 log.cc:927] Time spent T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.157s	user 0.000s	sys 0.001s
I20260709 13:08:57.491252 23622 pending_rounds.cc:107] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 13:08:57.491884 23622 raft_consensus.cc:2889] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:57.494666 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:57.323969 25573 consensus_queue.cc:1059] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.171s
I20260709 13:08:57.507263 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:57.309324 25636 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:57.502480 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:57.511667 23619 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:57.503355 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:57.304467 25363 raft_consensus.cc:2804] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:08:57.504741 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:57.304171 25661 raft_consensus.cc:2804] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:08:57.304226 25565 consensus_queue.cc:1059] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.190s
I20260709 13:08:57.499087 25328 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:57.373983 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:57.510861 25620 log.cc:927] Time spent T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 2.302s	user 0.000s	sys 0.021s
I20260709 13:08:57.500821 25692 consensus_queue.cc:1059] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:08:57.311770 25513 raft_consensus.cc:2804] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:08:57.509351 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:57.513309 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:57.570186 22955 raft_consensus.cc:2468] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
I20260709 13:08:57.513872 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:57.515723 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:57.571379 23622 raft_consensus.cc:2393] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 6: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:57.516506 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:57.571841 23022 raft_consensus.cc:2468] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6.
I20260709 13:08:57.516849 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:57.486100 25637 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:57.462620 25440 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:57.420082 25689 log.cc:927] Time spent T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:08:57.523728 23588 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:57.546370 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:57.547528 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:57.502092 25246 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.325982 25616 consensus_queue.cc:1059] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.185s
I20260709 13:08:57.310801 25370 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:57.511530 25636 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:57.615489 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:57.567226 23619 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:08:57.568125 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:57.616912 22953 raft_consensus.cc:2393] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 6: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:57.518095 25363 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:57.570245 23026 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:57.513317 23025 raft_consensus.cc:2410] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:08:57.572885 23021 raft_consensus.cc:2393] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 7: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:57.328612 25605 consensus_queue.cc:1059] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.214s
I20260709 13:08:57.571467 25513 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:57.615561 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 91919.1us
I20260709 13:08:57.624320 23588 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:57.625304 25444 raft_consensus.cc:2804] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:57.625679 25444 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:57.626011 25444 raft_consensus.cc:3060] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:08:57.626724 23588 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:57.509193 25625 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:57.614993 25246 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:57.633780 25444 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:57.634353 23026 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 7.
I20260709 13:08:57.635104 25246 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:57.636693 25444 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:57.641002 25328 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:57.641746 25627 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:57.649973 25662 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.650096 25246 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.650605 25662 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:57.650873 25246 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:57.618929 25636 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:57.653762 25246 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:57.612191 25659 raft_consensus.cc:2749] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:08:57.619838 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:57.575555 25692 consensus_queue.cc:1243] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer f3c799b1249e40119d2a5864231c6c75 log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:08:57.568954 25661 raft_consensus.cc:697] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:57.652957 25605 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:57.658227 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:57.659312 23588 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:57.669703 22953 raft_consensus.cc:2393] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 4: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:57.670958 23026 raft_consensus.cc:3060] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:57.671648 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:57.673965 22953 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:57.671974 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:57.675618 23619 raft_consensus.cc:2393] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 5: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:57.622354 25363 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:57.659726 25513 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:57.656589 25662 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:57.690635 23026 raft_consensus.cc:2468] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 7.
I20260709 13:08:57.675510 25692 raft_consensus.cc:2804] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:08:57.697358 23588 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:57.678853 25573 consensus_queue.cc:1243] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer aed3f089ec8b49af9c8cf5cdee76e8e7 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
W20260709 13:08:57.724498 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:57.685169 25328 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.697804 25305 raft_consensus.cc:3055] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:08:57.696979 25703 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.699667 25659 raft_consensus.cc:2804] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:08:57.739672 22953 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:08:57.701977 25662 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.700552 25665 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.674873 25605 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:57.748831 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:57.699785 25370 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:57.755808 22953 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
I20260709 13:08:57.756909 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:57.757263 23588 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:57.758490 23588 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:57.727651 25573 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:08:57.759972 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:57.727636 25692 raft_consensus.cc:697] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 7 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:57.764030 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:57.733407 25305 raft_consensus.cc:740] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:57.766094 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:57.731884 25615 raft_consensus.cc:697] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 6 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:08:57.772989 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 216758us
I20260709 13:08:57.765105 25559 raft_consensus.cc:2764] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:08:57.778494 22844 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:57.780553 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:57.762102 25659 raft_consensus.cc:697] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:57.760646 25565 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.763134 25444 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:57.734375 25328 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:57.786441 25444 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:57.786965 25444 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:57.761073 25704 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:57.787583 25704 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:57.788165 25328 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:08:57.788549 25698 log.cc:927] Time spent T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
W20260709 13:08:57.789057 25699 log.cc:927] Time spent T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.144s	user 0.000s	sys 0.005s
I20260709 13:08:57.784976 25625 consensus_queue.cc:1059] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 13:08:57.789921 25704 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:57.758316 25662 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:57.790887 25700 log.cc:927] Time spent T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 13:08:57.792500 25662 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:57.752166 25703 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:57.793193 25305 consensus_queue.cc:260] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:57.794410 25305 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:57.794874 25703 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:57.795121 25715 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.760872 25665 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:57.795708 25715 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:57.776638 25440 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:57.800083 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:08:57.802378 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 43994.3us
I20260709 13:08:57.782841 22953 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:57.774467 25370 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.800940 23025 raft_consensus.cc:2410] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:08:57.800987 22919 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:57.804388 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:57.804960 23021 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:57.805946 22919 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:57.798650 25328 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:57.807104 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:57.808947 23027 raft_consensus.cc:1240] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:08:57.807061 25353 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:57.811641 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:57.812255 23028 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:57.813555 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:57.766991 25573 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:57.817637 22880 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35370 (request call id 2090) took 3660 ms (client timeout 2990 ms). Trace:
I20260709 13:08:57.818632 22953 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 8.
I20260709 13:08:57.819072 23023 raft_consensus.cc:1275] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:08:57.819862 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:57.820036 23588 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:57.820422 22953 raft_consensus.cc:2468] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6.
I20260709 13:08:57.821256 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:08:57.821343 22950 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52222 (request call id 385) took 3503 ms (client timeout 1540 ms). Trace:
I20260709 13:08:57.821766 22953 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 9.
W20260709 13:08:57.821720 22950 rpcz_store.cc:269] 0709 13:08:54.317546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:54.325778 (+  8232us) service_pool.cc:224] Handling call
0709 13:08:57.821171 (+3495393us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b540023d7d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:57.821287 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2331366784}
I20260709 13:08:57.823032 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:08:57.818660 22880 rpcz_store.cc:269] 0709 13:08:54.157341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:54.157499 (+   158us) service_pool.cc:224] Handling call
0709 13:08:57.817449 (+3659950us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b540030d4d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:57.817571 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2551319808}
I20260709 13:08:57.825634 25559 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:08:57.826141 25559 raft_consensus.cc:697] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:57.826977 25559 consensus_queue.cc:237] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:57.827617 25444 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:57.827612 25680 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:57.830874 25680 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:57.815742 25328 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:57.832301 25318 raft_consensus.cc:3060] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:57.832890 22952 raft_consensus.cc:1240] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:08:57.833572 25605 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:57.807626 25599 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:57.834169 25599 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:57.834491 25599 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:57.834812 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:57.837553 25328 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:57.837191 25692 consensus_queue.cc:237] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:57.838829 23590 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:57.839886 23590 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:57.840538 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:57.761343 25661 consensus_queue.cc:237] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:57.841653 23590 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:57.843923 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:57.846956 23621 raft_consensus.cc:2393] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 5: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:57.845603 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:57.785048 25705 consensus_queue.cc:1048] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 13:08:57.799856 25565 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:57.861682 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:57.863168 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:57.864508 23622 raft_consensus.cc:1275] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:08:57.865396 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:57.816241 25252 raft_consensus.cc:2804] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:08:57.819964 25665 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:57.806604 25440 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:57.885159 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:57.872978 23021 raft_consensus.cc:2410] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:08:57.808043 25660 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:57.808988 25370 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:57.820581 25701 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:57.954852 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:57.850298 25678 raft_consensus.cc:2749] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:57.976975 23025 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:57.886101 25252 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:57.978835 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:57.999645 23021 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:58.000955 22952 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:57.898842 23028 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6.
I20260709 13:08:58.004804 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:58.005537 23028 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:08:58.006538 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:58.007162 23028 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6.
I20260709 13:08:58.008082 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:58.009039 22955 raft_consensus.cc:2393] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 7: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:57.814535 25353 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:57.946295 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 99716.7us
I20260709 13:08:57.982430 22953 raft_consensus.cc:2376] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 5. Re-sending same reply.
W20260709 13:08:57.892954 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 51349.9us
I20260709 13:08:57.849670 25679 consensus_queue.cc:1059] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.127s
I20260709 13:08:57.998682 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:58.019425 23028 raft_consensus.cc:1275] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:08:58.020797 23025 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:08:58.021904 23021 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6.
I20260709 13:08:57.855975 25616 consensus_queue.cc:1048] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.141s
I20260709 13:08:57.978256 25701 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:58.015110 23590 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:08:58.013725 22920 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:57.893617 25716 raft_consensus.cc:2804] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:58.037204 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:58.037915 22953 raft_consensus.cc:2393] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 7: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:58.037963 22993 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:58.039122 22993 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:58.012780 25252 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:58.023381 25592 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:58.041240 25657 consensus_queue.cc:1059] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:08:58.045198 25507 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:58.045634 25507 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:58.045976 25507 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:58.037061 25444 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:08:58.049463 25711 log.cc:927] Time spent T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:08:58.051430 25353 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:58.030108 25692 raft_consensus.cc:2749] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:58.057242 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:58.058400 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:58.059368 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:58.059875 22950 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:08:58.060565 25634 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 13:08:58.061697 25370 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:58.062264 25370 raft_consensus.cc:2764] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:08:58.071024 23024 raft_consensus.cc:2468] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 5.
I20260709 13:08:58.072505 23023 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6.
I20260709 13:08:58.073974 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:58.074945 22953 raft_consensus.cc:2393] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 6: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:58.030900 25701 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:57.988144 25707 consensus_queue.cc:1048] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.091s
I20260709 13:08:57.875380 25615 consensus_queue.cc:237] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:58.038818 25716 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:57.886843 25659 consensus_queue.cc:237] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:58.103899 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:58.043017 25708 raft_consensus.cc:2804] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:58.063298 25665 raft_consensus.cc:2804] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:08:58.105283 23588 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:58.108311 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:58.113181 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:08:58.105926 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:58.115211 22953 raft_consensus.cc:3060] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:58.106516 25708 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:58.041759 25328 consensus_queue.cc:1048] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
I20260709 13:08:58.043488 25705 consensus_queue.cc:1048] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 13:08:58.122517 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:58.121384 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:58.072211 25560 consensus_queue.cc:1059] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.044s
W20260709 13:08:58.124082 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 133388us
W20260709 13:08:58.133527 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:58.134006 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:58.103104 25716 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:58.115355 25559 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:58.164973 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:58.166105 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:58.166735 25305 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:58.168196 25719 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:58.197373 25305 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:58.197566 25719 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:58.199633 25305 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:58.116469 25715 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:58.174278 23619 raft_consensus.cc:1240] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 6. Current term is 7. Ops: []
I20260709 13:08:58.168247 22843 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:58.121939 25708 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:08:58.076009 25592 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:58.206022 25592 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:58.186848 25599 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:58.119292 25665 raft_consensus.cc:697] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:58.208770 25665 consensus_queue.cc:237] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:58.211066 25599 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:58.212909 25724 raft_consensus.cc:2804] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:08:58.213547 25724 raft_consensus.cc:697] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 5 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:58.214463 25724 consensus_queue.cc:237] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:58.218588 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:58.219357 22954 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 7.
I20260709 13:08:58.189064 25565 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:58.105576 25657 consensus_queue.cc:1048] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:08:58.192348 25712 consensus_queue.cc:1048] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:08:58.223505 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:58.084573 25636 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:58.235944 22950 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6.
I20260709 13:08:58.241247 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:58.243191 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:58.244532 23027 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:08:58.247642 22954 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 7.
W20260709 13:08:58.245849 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 201471us
I20260709 13:08:58.250706 23020 raft_consensus.cc:1275] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:08:58.250901 23023 raft_consensus.cc:1240] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 7. Current term is 8. Ops: [7.5-7.5]
W20260709 13:08:58.251932 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:58.251770 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:58.245872 22952 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 8.
I20260709 13:08:58.253007 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:58.254376 22876 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:58.252928 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:58.253566 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:58.255717 22878 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 7.
I20260709 13:08:58.256304 22879 raft_consensus.cc:1275] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 13:08:58.256551 22879 pending_rounds.cc:85] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 3
I20260709 13:08:58.256764 22879 pending_rounds.cc:107] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:08:58.200383 25719 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:58.304333 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:08:58.242204 25513 consensus_meta.cc:287] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.545s	user 0.005s	sys 0.005s
I20260709 13:08:58.308930 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:58.309877 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:58.257898 22953 raft_consensus.cc:2468] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 7.
I20260709 13:08:58.310803 22876 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6.
I20260709 13:08:58.298758 22879 raft_consensus.cc:2889] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:58.177994 25573 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:58.254400 22950 raft_consensus.cc:1240] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:58.304091 22875 raft_consensus.cc:1275] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:08:58.305346 23020 raft_consensus.cc:2368] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 6. Current term is 7.
I20260709 13:08:58.299823 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:58.315553 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:58.315728 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:08:58.303236 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 289573us
I20260709 13:08:58.307369 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:58.307518 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:58.312227 22953 raft_consensus.cc:1275] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:08:58.313295 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:58.316458 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:58.317629 22877 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:58.250551 25715 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:58.322098 22954 raft_consensus.cc:2393] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 6: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:58.324419 22920 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:58.326635 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:58.325906 22920 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:58.328312 22879 raft_consensus.cc:1275] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:08:58.329576 22844 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:58.306926 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:58.221805 25559 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:58.251399 25507 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:58.318748 25639 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:58.319295 25660 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:58.336444 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:58.336308 22920 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:58.354596 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:08:58.355216 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 56547.8us
I20260709 13:08:58.355266 22880 raft_consensus.cc:2393] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 8: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:58.339781 23619 raft_consensus.cc:1275] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:08:58.347378 23027 raft_consensus.cc:1240] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:08:58.329025 25592 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:58.359892 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:58.360039 23621 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
W20260709 13:08:58.339452 25363 consensus_meta.cc:287] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.655s	user 0.009s	sys 0.001s
I20260709 13:08:58.333969 25573 raft_consensus.cc:2749] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:08:58.344271 25624 raft_consensus.cc:2804] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:08:58.318339 25513 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:58.398432 22877 raft_consensus.cc:2410] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:08:58.328043 25715 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:58.400080 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:58.401621 22949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2190) took 2920 ms. Trace:
I20260709 13:08:58.402017 22949 rpcz_store.cc:276] 0709 13:08:55.481076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:55.573749 (+ 92673us) service_pool.cc:224] Handling call
0709 13:08:55.613147 (+ 39398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:56.174082 (+560935us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:58.401388 (+2227306us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b54000a96d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:58.401574 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":244329728}
I20260709 13:08:58.361307 25592 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:58.344998 23616 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
I20260709 13:08:58.396037 25370 raft_consensus.cc:2804] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:08:58.357848 25639 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:58.344795 25727 consensus_queue.cc:1048] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.075s
I20260709 13:08:58.401944 25624 raft_consensus.cc:697] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:58.413301 25592 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:58.394327 25363 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:58.454232 25717 log.cc:927] Time spent T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.205s	user 0.000s	sys 0.001s
I20260709 13:08:58.469995 23021 raft_consensus.cc:1275] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:08:58.362947 25728 consensus_queue.cc:1048] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
W20260709 13:08:58.456547 25732 log.cc:927] Time spent T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
W20260709 13:08:58.506263 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 44653.2us
I20260709 13:08:58.399606 25252 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:58.516628 25716 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:58.539108 23022 raft_consensus.cc:2393] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 7: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
W20260709 13:08:58.541977 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 138771us
I20260709 13:08:58.542555 25370 raft_consensus.cc:697] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:58.543519 25624 consensus_queue.cc:237] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:58.469909 25718 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:58.547385 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:58.549064 23021 raft_consensus.cc:2368] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 5. Current term is 6.
I20260709 13:08:58.555881 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:58.556500 23021 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 6.
I20260709 13:08:58.565431 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:58.570025 25716 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:58.571624 25363 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:58.572249 25715 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:58.573210 25507 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:58.574241 22873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 1627) took 1704 ms. Trace:
I20260709 13:08:58.574618 22873 rpcz_store.cc:276] 0709 13:08:56.869548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:57.017076 (+147528us) service_pool.cc:224] Handling call
0709 13:08:57.060552 (+ 43476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:57.087796 (+ 27244us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:08:57.334899 (+247103us) log.cc:844] Serialized 14 byte log entry
0709 13:08:57.609561 (+274662us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:57.609629 (+    68us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:08:57.609671 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:58.574012 (+964341us) spinlock_profiling.cc:243] Waited 26.9 ms on lock 0x7b540022abd4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:58.574067 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:58.574187 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7558,"prepare.queue_time_us":225016,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":645,"raft.queue_time_us":94270,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":2894,"spinlock_wait_cycles":61968640,"thread_start_us":433906,"threads_started":2,"wal-append.queue_time_us":279120}
I20260709 13:08:58.575929 25708 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:58.577144 23023 raft_consensus.cc:1240] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 9. Current term is 10. Ops: []
I20260709 13:08:58.581792 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:58.582530 23028 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 6.
I20260709 13:08:58.583763 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:58.584350 23029 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 5.
I20260709 13:08:58.587951 25650 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:08:58.588171 25697 log.cc:927] Time spent T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.942s	user 0.000s	sys 0.005s
I20260709 13:08:58.588028 25559 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:58.525195 25701 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:58.591557 25701 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:58.593015 22871 raft_consensus.cc:2393] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 8: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:58.593223 25513 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:58.553344 25718 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:58.604064 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:58.607306 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:58.549746 25370 consensus_queue.cc:237] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:58.591642 25602 consensus_queue.cc:1048] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 13:08:58.630968 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:58.641760 22992 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:58.644068 22992 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:58.631606 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:58.645334 23022 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
I20260709 13:08:58.645848 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:58.622252 25252 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:58.632915 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:58.634286 25663 raft_consensus.cc:3055] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:08:58.636957 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:58.637678 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:58.638026 25637 raft_consensus.cc:3055] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:08:58.648108 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:58.648703 25708 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:58.649677 25595 raft_consensus.cc:3055] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:08:58.624814 25639 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:58.659297 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:58.663139 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:58.664964 25595 raft_consensus.cc:740] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:58.670500 22950 raft_consensus.cc:1240] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:58.671849 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:58.672904 22950 raft_consensus.cc:2393] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 9: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:58.677333 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:58.679242 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:58.656597 25560 raft_consensus.cc:2804] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:58.684410 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:58.685968 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:58.686942 22950 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
I20260709 13:08:58.688339 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:58.659103 25615 raft_consensus.cc:2804] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:58.656919 25663 raft_consensus.cc:740] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:58.698935 22949 raft_consensus.cc:1275] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:08:58.699445 22949 pending_rounds.cc:85] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting all ops after (but not including) 8
I20260709 13:08:58.699769 22949 pending_rounds.cc:107] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:08:58.700057 22949 raft_consensus.cc:2889] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:58.701052 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:58.659986 25637 raft_consensus.cc:740] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:08:58.668823 25718 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:58.653687 25715 consensus_queue.cc:1048] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:08:58.680261 23620 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 573) took 3048 ms. Trace:
W20260709 13:08:58.714882 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 169156us
I20260709 13:08:58.661926 25363 consensus_queue.cc:1059] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:08:58.715962 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:58.666978 25565 consensus_queue.cc:1059] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 13:08:58.694779 25372 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:58.734745 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:58.734579 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:58.741482 23028 raft_consensus.cc:2368] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 6. Current term is 7.
I20260709 13:08:58.741714 22953 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
I20260709 13:08:58.702919 25615 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:58.697244 25560 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:58.698447 25639 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:08:58.745600 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:58.757072 22872 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 7
I20260709 13:08:58.746271 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:58.749020 25246 raft_consensus.cc:3055] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:08:58.749998 25599 consensus_queue.cc:1048] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:08:58.751233 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:58.752380 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:58.754168 22992 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:58.761468 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:58.770152 23028 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 6.
I20260709 13:08:58.768741 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:58.768159 22872 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
I20260709 13:08:58.772543 22992 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:58.754129 25615 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:58.761382 25560 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:58.761982 25750 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:58.762537 25246 raft_consensus.cc:740] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:58.797955 25707 raft_consensus.cc:2804] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:58.814321 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:58.756285 25372 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:58.738461 25595 consensus_queue.cc:260] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:58.800185 25328 consensus_queue.cc:1048] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:08:58.761947 25703 consensus_queue.cc:1059] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.050s
I20260709 13:08:58.741369 25637 consensus_queue.cc:260] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:58.801304 25661 raft_consensus.cc:2804] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:58.828357 23021 raft_consensus.cc:1275] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 13:08:58.849077 25637 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:08:58.837997 22951 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 392) took 2207 ms. Trace:
I20260709 13:08:58.850394 22951 rpcz_store.cc:276] 0709 13:08:56.630201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:56.856893 (+226692us) service_pool.cc:224] Handling call
0709 13:08:56.857153 (+   260us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:57.189684 (+332531us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:58.837792 (+1648108us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b54000ed3d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:58.837936 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":521791872}
I20260709 13:08:58.851449 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:58.855294 25372 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:58.827104 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:58.856175 23020 raft_consensus.cc:2376] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 5. Re-sending same reply.
I20260709 13:08:58.856739 25707 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:58.829078 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:58.857107 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:58.857600 23022 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 4.
W20260709 13:08:58.824885 23024 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32794 (request call id 461) took 1333 ms (client timeout 1671 ms). Trace:
I20260709 13:08:58.830345 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:08:58.776932 25639 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:58.857141 25661 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:59.003705 23588 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:58.815738 25750 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:59.003829 25661 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:58.837497 23023 raft_consensus.cc:1240] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:08:58.819424 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:08:58.969722 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:58.873920 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:58.716303 23620 rpcz_store.cc:276] 0709 13:08:55.618281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:55.618650 (+   369us) service_pool.cc:224] Handling call
0709 13:08:55.668590 (+ 49940us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:56.722608 (+1054018us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:58.645399 (+1922791us) spinlock_profiling.cc:243] Waited 990 ms on lock 0x7b5400670f54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:58.666779 (+ 21380us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2275896576}
I20260709 13:08:58.862349 22951 raft_consensus.cc:1240] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:08:59.009981 23622 raft_consensus.cc:2410] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 9 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260709 13:08:59.012012 23024 rpcz_store.cc:269] 0709 13:08:57.491382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:57.508747 (+ 17365us) service_pool.cc:224] Handling call
0709 13:08:58.824651 (+1315904us) spinlock_profiling.cc:243] Waited 475 ms on lock 0x7b54001f86d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:58.824812 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1092147584}
I20260709 13:08:59.018443 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:59.049391 23023 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 7.
I20260709 13:08:59.050619 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:08:59.061123 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:08:58.840723 22980 heartbeater.cc:646] Failed to heartbeat to 127.21.63.62:33399 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.21.63.62:33399 timed out after 15.000s (SENT)
I20260709 13:08:59.063539 22951 raft_consensus.cc:1240] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:08:59.063643 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:59.065954 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:58.834713 25636 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:59.008085 25707 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:58.829425 25246 consensus_queue.cc:260] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:58.811901 25663 consensus_queue.cc:260] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:59.068320 25639 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:59.055527 25599 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:59.068677 25663 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:59.069023 25750 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:59.069408 25595 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:59.069542 22950 raft_consensus.cc:1240] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 6. Current term is 7. Ops: []
W20260709 13:08:59.069901 25759 log.cc:927] Time spent T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 13:08:59.083106 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:59.070001 22951 raft_consensus.cc:1275] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:08:59.072336 25605 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:59.074172 23617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60044 (request call id 568) took 2617 ms. Trace:
I20260709 13:08:59.075320 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:58.973588 25660 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:58.960641 25718 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:59.079452 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:59.080606 23023 raft_consensus.cc:1240] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:58.822736 25592 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:59.109906 23023 raft_consensus.cc:1275] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:08:59.058707 25593 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.505s	user 0.019s	sys 0.054s
I20260709 13:08:59.081770 23024 raft_consensus.cc:1240] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 13:08:59.082285 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:59.119366 22872 raft_consensus.cc:2393] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 6: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:59.083982 22875 raft_consensus.cc:2393] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 7: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:59.093624 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:59.120419 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:59.121548 22872 raft_consensus.cc:2410] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:08:59.091754 22876 raft_consensus.cc:1240] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 7. Current term is 8. Ops: []
I20260709 13:08:59.125049 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:59.128908 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:59.083935 25599 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:59.133273 25599 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:59.136044 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:59.082943 25636 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:59.118284 25593 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.569s	user 0.020s	sys 0.054s
I20260709 13:08:59.097935 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:59.002504 25615 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:59.085033 23617 rpcz_store.cc:276] 0709 13:08:56.456541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:56.456685 (+   144us) service_pool.cc:224] Handling call
0709 13:08:56.457026 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:56.545418 (+ 88392us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:08:56.754966 (+209548us) log.cc:844] Serialized 14 byte log entry
0709 13:08:56.888785 (+133819us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:08:56.888843 (+    58us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:08:56.888900 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:59.073976 (+2185076us) spinlock_profiling.cc:243] Waited 88.1 ms on lock 0x7b540045bfd4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:59.074019 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:59.074122 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9030,"prepare.queue_time_us":256623,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":11653,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":4067,"spinlock_wait_cycles":202578304,"thread_start_us":339226,"threads_started":2,"wal-append.queue_time_us":288029,"wal-append.run_cpu_time_us":1986,"wal-append.run_wall_time_us":1255196}
W20260709 13:08:59.115172 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 77571.6us
I20260709 13:08:59.143083 23020 raft_consensus.cc:1275] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:08:59.144599 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:59.152807 22874 raft_consensus.cc:2393] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 7: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:59.096133 25660 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:59.092769 25605 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:59.106472 25718 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:59.159370 23021 raft_consensus.cc:1240] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:08:59.166221 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:59.165418 25639 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:59.163753 25724 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:59.168085 25639 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:59.168339 25724 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:59.173384 22874 raft_consensus.cc:2410] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:08:59.171003 25724 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:59.178653 25639 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:59.145421 25636 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:59.180292 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:59.179733 25761 consensus_queue.cc:1059] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
W20260709 13:08:59.232118 22994 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:59.246786 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:59.250367 22876 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:59.250938 22872 raft_consensus.cc:1240] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
W20260709 13:08:59.235611 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 1.20716e+06us
I20260709 13:08:59.157166 25593 raft_consensus.cc:3055] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 13:08:59.255988 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:59.256656 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:59.258181 22872 raft_consensus.cc:2468] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
I20260709 13:08:59.265885 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:59.266331 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:59.273175 22876 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:08:59.275941 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:59.276962 22947 raft_consensus.cc:2393] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 8: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:59.260175 25750 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:59.279490 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:59.279284 25750 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:59.279295 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:59.241279 25615 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:59.189918 25605 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:59.280500 23020 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 8.
I20260709 13:08:59.259450 25593 raft_consensus.cc:740] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:59.239061 25718 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:59.245335 25592 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:59.304520 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:59.299328 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:59.306398 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:59.306741 23024 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 6.
I20260709 13:08:59.307202 22875 raft_consensus.cc:2393] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 6: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:59.307638 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:59.308218 23024 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:59.308422 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:59.309841 25560 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:59.310937 23029 raft_consensus.cc:2368] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 6. Current term is 7.
I20260709 13:08:59.302714 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:59.311538 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:59.311924 22947 raft_consensus.cc:2393] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 8: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:59.311985 25560 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:59.245536 25707 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:59.313156 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:59.313437 25636 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:59.313808 22947 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 6.
I20260709 13:08:59.314143 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:59.314699 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:59.314870 22872 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:08:59.315757 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:59.315886 25660 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:59.303057 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:59.317330 22878 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:08:59.303298 25661 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:59.303665 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:59.316812 22872 raft_consensus.cc:1275] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:08:59.318250 22877 raft_consensus.cc:1275] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 5. (term mismatch)
W20260709 13:08:59.318264 22918 connection.cc:256] RPC call timeout handler was delayed by -1.0960156840000082s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1623s for a response
I20260709 13:08:59.324060 22877 pending_rounds.cc:85] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 4
I20260709 13:08:59.323073 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:59.324236 23024 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
I20260709 13:08:59.323868 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:59.324371 22877 pending_rounds.cc:107] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 5.5
I20260709 13:08:59.324842 22878 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
I20260709 13:08:59.325187 22877 raft_consensus.cc:2889] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:08:59.325594 22918 connection.cc:256] RPC call timeout handler was delayed by -1.0852156839999907s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1635s for a response
I20260709 13:08:59.323237 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:59.327303 22918 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:59.331316 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:59.332088 22947 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:59.336804 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:59.338027 22876 raft_consensus.cc:2410] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:08:59.339139 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:59.347091 22947 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 8.
I20260709 13:08:59.348680 22877 raft_consensus.cc:1240] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 3. Current term is 4. Ops: []
I20260709 13:08:59.349507 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:59.349738 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:59.381978 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:59.352841 22918 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:59.322853 25680 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:59.368594 25762 log.cc:927] Time spent T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 13:08:59.373202 22878 raft_consensus.cc:1240] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:08:59.383057 22872 raft_consensus.cc:2393] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 8: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:59.385869 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:59.384172 22918 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:59.367672 25624 raft_consensus.cc:2749] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:08:59.431936 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:59.389055 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:59.437736 23020 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:08:59.393879 22918 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:59.392063 22947 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 4.
I20260709 13:08:59.439450 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:59.433900 22872 raft_consensus.cc:1240] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 5 LEADER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 4. Current term is 5. Ops: []
I20260709 13:08:59.435411 22878 raft_consensus.cc:1240] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 6. Current term is 7. Ops: []
I20260709 13:08:59.387379 25599 raft_consensus.cc:2749] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:08:59.367376 25660 consensus_queue.cc:1048] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.063s
I20260709 13:08:59.437894 25418 raft_consensus.cc:3055] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:08:59.440385 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:08:59.450212 22948 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52222 (request call id 388) took 2822 ms (client timeout 1692 ms). Trace:
I20260709 13:08:59.500932 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:59.459894 22947 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 9.
W20260709 13:08:59.501045 22948 rpcz_store.cc:269] 0709 13:08:56.627585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:56.627713 (+   128us) service_pool.cc:224] Handling call
0709 13:08:59.448348 (+2820635us) spinlock_profiling.cc:243] Waited 646 ms on lock 0x7b5400148554. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:59.450102 (+  1754us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1486595968}
W20260709 13:08:59.502025 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 224139us
I20260709 13:08:59.502127 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:08:59.503053 22843 leader_election.cc:336] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.521s (SENT)
I20260709 13:08:59.502974 22947 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 8.
I20260709 13:08:59.503549 22843 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:59.504141 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:08:59.504765 22843 leader_election.cc:336] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.526s (SENT)
I20260709 13:08:59.435304 25592 raft_consensus.cc:2749] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:08:59.388792 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 197459us
I20260709 13:08:59.495944 25624 raft_consensus.cc:2749] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 13:08:59.459120 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 65338.9us
I20260709 13:08:59.505296 22843 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:59.489993 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:59.505300 22947 raft_consensus.cc:2393] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 9: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:59.505381 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:08:59.507459 23588 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:59.395915 25680 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:59.509050 22948 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 9.
I20260709 13:08:59.508329 22918 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
W20260709 13:08:59.509450 23588 leader_election.cc:336] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Timed out: RequestConsensusVote RPC to 127.21.63.2:33309 timed out after 2.061s (SENT)
I20260709 13:08:59.510048 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:59.513144 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:08:59.513824 22947 raft_consensus.cc:2393] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 8: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:08:59.389863 25593 consensus_queue.cc:260] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:59.516575 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:59.514359 22948 raft_consensus.cc:1240] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:08:59.500864 25418 raft_consensus.cc:740] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:59.514699 25637 raft_consensus.cc:2749] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:08:59.518877 25418 consensus_queue.cc:260] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:59.520170 25418 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:59.526171 22947 raft_consensus.cc:1240] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:08:59.527545 22947 raft_consensus.cc:1275] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:08:59.528813 22873 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
I20260709 13:08:59.529599 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:59.530393 22878 raft_consensus.cc:1240] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
W20260709 13:08:59.530930 25767 log.cc:927] Time spent T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:08:59.532907 22951 raft_consensus.cc:1275] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:08:59.535583 23588 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:59.536293 25663 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:08:59.537261 25766 log.cc:927] Time spent T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:08:59.536990 25663 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:59.539363 25663 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:59.542764 25661 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:59.543287 22993 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:59.546566 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:59.530392 23026 raft_consensus.cc:2393] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 7: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:08:59.556679 23590 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:59.559118 23588 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:59.559300 22993 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:59.547246 25370 consensus_queue.cc:1048] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.014s
I20260709 13:08:59.574427 25708 raft_consensus.cc:3055] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:08:59.574833 25680 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:59.574972 25708 raft_consensus.cc:740] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:59.575690 25615 raft_consensus.cc:2749] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:59.577071 25639 raft_consensus.cc:2749] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:08:59.556977 25718 consensus_queue.cc:1048] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:08:59.577733 25615 consensus_queue.cc:1048] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:08:59.545284 25754 raft_consensus.cc:2804] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:08:59.579458 25754 raft_consensus.cc:697] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:59.579689 22993 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:59.580642 25615 raft_consensus.cc:2749] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:59.580348 25754 consensus_queue.cc:237] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:59.562053 23020 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 9.
I20260709 13:08:59.581928 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:08:59.582751 23020 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 4.
I20260709 13:08:59.562016 25750 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:59.584086 23020 raft_consensus.cc:1275] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:08:59.584429 25661 raft_consensus.cc:2749] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:59.559698 25663 raft_consensus.cc:2804] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:08:59.586328 25661 consensus_queue.cc:1059] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:08:59.586627 25663 raft_consensus.cc:697] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 6 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:59.562263 22843 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:59.507566 25636 raft_consensus.cc:2749] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:08:59.564910 25573 consensus_queue.cc:1048] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 13:08:59.565289 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:59.602373 23028 raft_consensus.cc:1275] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:08:59.565598 22878 raft_consensus.cc:1275] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:08:59.604302 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:08:59.605608 23590 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:59.607529 23590 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:59.609599 22872 raft_consensus.cc:1240] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:08:59.606876 22920 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:59.565966 25588 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:08:59.565928 25318 consensus_queue.cc:1059] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 13:08:59.566560 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:59.566638 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:59.567579 22880 raft_consensus.cc:1275] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:08:59.587847 25663 consensus_queue.cc:237] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:08:59.568301 25760 log.cc:927] Time spent T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.771s	user 0.000s	sys 0.001s
I20260709 13:08:59.571205 23619 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 594) took 1209 ms. Trace:
I20260709 13:08:59.621299 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:59.622481 22875 raft_consensus.cc:2410] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 8 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:08:59.548851 22948 raft_consensus.cc:1275] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:08:59.623021 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:59.572746 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:59.612594 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:59.544589 25593 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:59.617498 22877 raft_consensus.cc:2393] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 8: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:59.621516 23619 rpcz_store.cc:276] 0709 13:08:58.361966 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:58.362106 (+   140us) service_pool.cc:224] Handling call
0709 13:08:58.362455 (+   349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:58.508389 (+145934us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:08:58.508432 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:08:58.517368 (+  8936us) log.cc:844] Serialized 26 byte log entry
0709 13:08:58.692347 (+174979us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:08:58.692409 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:08:59.570814 (+878405us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b54002bc354. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:08:59.570864 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:08:59.570912 (+    48us) raft_consensus.cc:1685] finished
0709 13:08:59.570925 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:08:59.571146 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":647,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":112,"spinlock_wait_cycles":334458624,"thread_start_us":174536,"threads_started":1,"wal-append.queue_time_us":222551}
W20260709 13:08:59.618153 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 55964.3us
I20260709 13:08:59.618402 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:59.624555 22874 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6.
I20260709 13:08:59.628141 22843 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:59.625550 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 191172us
I20260709 13:08:59.623822 22879 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 7.
I20260709 13:08:59.624758 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:59.630029 22878 raft_consensus.cc:2468] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
I20260709 13:08:59.614372 25718 raft_consensus.cc:2804] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:59.642247 22920 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:59.646154 22843 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:59.646773 22920 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:59.576231 25708 consensus_queue.cc:260] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:59.626366 25570 raft_consensus.cc:2804] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:59.666251 25570 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:59.666793 25570 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:59.667397 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:59.668104 22872 raft_consensus.cc:2393] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 6: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:59.656744 25636 raft_consensus.cc:2749] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:59.652863 25588 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:08:59.669544 25588 raft_consensus.cc:2749] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:08:59.653090 25624 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:08:59.671139 25624 raft_consensus.cc:697] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:59.655951 22843 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:59.672952 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:59.673931 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:59.673971 22843 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:59.674554 22872 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
I20260709 13:08:59.675841 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:59.674798 25636 raft_consensus.cc:2749] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:59.674932 25637 raft_consensus.cc:2749] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:59.666154 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:59.678130 22874 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:08:59.678928 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:59.679646 22874 raft_consensus.cc:2393] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 7: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:08:59.672058 25624 consensus_queue.cc:237] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:59.617136 25639 consensus_queue.cc:1048] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 13:08:59.680981 25370 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:08:59.681591 25370 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:59.684278 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:59.685760 25680 consensus_queue.cc:1048] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:59.686816 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:59.685750 22872 raft_consensus.cc:1275] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 13:08:59.688033 22843 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:59.688467 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 129390us
W20260709 13:08:59.689788 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 176613us
I20260709 13:08:59.688550 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:59.693562 22843 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:59.694126 22874 raft_consensus.cc:1275] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:08:59.695031 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:08:59.695539 22874 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:59.697068 25709 consensus_queue.cc:1059] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:08:59.697995 25615 raft_consensus.cc:3055] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:08:59.698375 25615 raft_consensus.cc:740] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:08:59.698453 22994 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:59.699260 25615 consensus_queue.cc:260] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:59.699846 25318 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:59.700395 25615 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:59.701076 22994 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:59.701934 25661 raft_consensus.cc:2749] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:59.704003 22994 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:59.705286 25661 raft_consensus.cc:2749] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:08:59.707165 22994 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:59.708386 25661 raft_consensus.cc:2749] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:08:59.711418 22994 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:59.712549 25661 raft_consensus.cc:2749] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:59.714689 22918 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:08:59.688037 22872 pending_rounds.cc:85] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 3
I20260709 13:08:59.691959 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:59.726624 22872 pending_rounds.cc:107] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 13:08:59.726918 22872 raft_consensus.cc:2889] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:08:59.727311 22879 raft_consensus.cc:2410] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:08:59.641273 25750 raft_consensus.cc:2804] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:08:59.719576 22874 raft_consensus.cc:2468] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
W20260709 13:08:59.720283 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:59.691606 23588 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:59.722051 22878 raft_consensus.cc:2393] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 7: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:08:59.723439 22843 leader_election.cc:400] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 6. Current term is 7.
I20260709 13:08:59.645809 25718 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:08:59.652863 25595 raft_consensus.cc:2749] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:59.715551 22994 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:08:59.725734 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 118281us
I20260709 13:08:59.730129 22918 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:08:59.731339 22843 leader_election.cc:403] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:08:59.737603 25573 raft_consensus.cc:2749] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 6. Current term is 7.
I20260709 13:08:59.738004 22843 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:59.739269 22843 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:59.739452 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:08:59.690697 25599 raft_consensus.cc:2804] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:08:59.746721 25709 raft_consensus.cc:2804] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:59.747185 25709 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:59.747538 25709 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:59.747841 25570 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:59.750737 25570 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:08:59.692396 25639 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:59.752516 25761 consensus_queue.cc:1048] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:08:59.755230 22843 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:59.756424 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 64889.4us
I20260709 13:08:59.727950 25572 raft_consensus.cc:2749] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:08:59.757126 23588 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:59.762189 23588 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:59.746132 22918 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:59.727950 25708 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:59.723928 25637 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:08:59.736366 25718 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:59.734586 25660 raft_consensus.cc:2804] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:08:59.763456 22843 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:59.739778 25750 raft_consensus.cc:697] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:59.755348 25599 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:59.778559 25781 consensus_queue.cc:1059] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:08:59.783952 25572 raft_consensus.cc:2804] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:08:59.784261 22843 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:59.754482 25370 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:08:59.754714 25636 raft_consensus.cc:2804] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:59.760479 25570 raft_consensus.cc:2749] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:08:59.761240 25663 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:08:59.799302 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 149887us
I20260709 13:08:59.752970 25639 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:59.803632 22843 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:59.791011 25599 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:08:59.765272 25573 raft_consensus.cc:2749] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:08:59.757594 25605 consensus_queue.cc:1059] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:08:59.760627 25661 consensus_queue.cc:1059] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 13:08:59.765763 25588 consensus_queue.cc:1048] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260709 13:08:59.764506 25418 consensus_queue.cc:1059] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 13:08:59.804805 25637 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:59.780160 25777 consensus_queue.cc:1048] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.097s
I20260709 13:08:59.806485 25636 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:59.871038 25780 raft_consensus.cc:2804] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 13:08:59.898100 22955 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58320 (request call id 2202) took 2573 ms (client timeout 1715 ms). Trace:
W20260709 13:08:59.905524 22955 rpcz_store.cc:269] 0709 13:08:57.324355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:57.571145 (+246790us) service_pool.cc:224] Handling call
0709 13:08:59.897876 (+2326731us) spinlock_profiling.cc:243] Waited 392 ms on lock 0x7b5400425754. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:59.898033 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":902408576}
I20260709 13:08:59.898566 22954 raft_consensus.cc:1240] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 7. Current term is 8. Ops: [7.9-7.9]
W20260709 13:08:59.897919 22952 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52222 (request call id 396) took 2126 ms (client timeout 1853 ms). Trace:
I20260709 13:08:59.907471 22919 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
W20260709 13:08:59.908154 22952 rpcz_store.cc:269] 0709 13:08:57.771631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:57.834561 (+ 62930us) service_pool.cc:224] Handling call
0709 13:08:59.897736 (+2063175us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b540044af54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:59.897851 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":380763520}
I20260709 13:08:59.876256 25573 raft_consensus.cc:2749] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:08:59.806717 25595 raft_consensus.cc:3060] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:59.858019 25370 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:08:59.886246 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 142779us
I20260709 13:08:59.905427 25636 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:08:59.889587 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 127448us
W20260709 13:08:59.936787 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 74581.2us
I20260709 13:08:59.910174 23023 raft_consensus.cc:1275] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:08:59.886273 25637 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:08:59.798803 25572 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:08:59.935724 22994 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:08:59.889000 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:59.938602 22920 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:08:59.907047 25783 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:59.943094 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:08:59.906567 25780 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:08:59.929073 25782 log.cc:927] Time spent T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 13:08:59.951411 25799 raft_consensus.cc:2804] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:08:59.951674 22994 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:59.951891 25799 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:08:59.952236 25799 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:08:59.952627 25418 raft_consensus.cc:2749] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:08:59.952999 22994 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:59.953850 25418 raft_consensus.cc:2764] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 13:08:59.954674 22994 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:08:59.955415 25418 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:08:59.945375 22955 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 7.
I20260709 13:08:59.956453 25761 consensus_queue.cc:1048] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:08:59.957041 22994 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:08:59.947896 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 144372us
I20260709 13:08:59.917346 25593 raft_consensus.cc:3055] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 13:08:59.958590 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 212497us
I20260709 13:08:59.960733 22843 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:59.961445 22918 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:59.936916 25663 raft_consensus.cc:697] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:08:59.943831 25572 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:08:59.941318 25370 raft_consensus.cc:697] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:59.967731 22843 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:08:59.942930 25784 raft_consensus.cc:2749] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:08:59.961359 22994 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:08:59.975806 23029 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33418 (request call id 1584) took 1343 ms (client timeout 1543 ms). Trace:
W20260709 13:09:00.093662 23029 rpcz_store.cc:269] 0709 13:08:58.631925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:58.657606 (+ 25681us) service_pool.cc:224] Handling call
0709 13:08:59.975537 (+1317931us) spinlock_profiling.cc:243] Waited 633 ms on lock 0x7b5400358254. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:08:59.975726 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1455307008}
I20260709 13:08:59.978598 22843 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:08:59.941133 25660 raft_consensus.cc:697] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:08:59.950652 25780 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:08:59.958700 25781 raft_consensus.cc:2804] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:08:59.906651 25750 consensus_queue.cc:237] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:08:59.959400 25593 raft_consensus.cc:740] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:00.099071 23625 raft_consensus.cc:1275] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
W20260709 13:09:00.098503 23025 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33420 (request call id 1627) took 2287 ms (client timeout 1635 ms). Trace:
I20260709 13:09:00.108091 23621 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60044 (request call id 576) took 1999 ms. Trace:
I20260709 13:09:00.113137 25729 raft_consensus.cc:3055] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:09:00.099218 22871 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53724 (request call id 435) took 1844 ms (client timeout 1851 ms). Trace:
I20260709 13:09:00.118669 25363 raft_consensus.cc:2749] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:09:00.120535 23021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 477) took 1291 ms. Trace:
W20260709 13:09:00.124029 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:08:59.984820 25605 consensus_queue.cc:1048] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.161s
W20260709 13:09:00.123966 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 162590us
I20260709 13:08:59.967068 25573 raft_consensus.cc:2749] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:09:00.019098 25797 raft_consensus.cc:3055] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:08:59.968835 25418 consensus_queue.cc:1048] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.160s
W20260709 13:09:00.137915 22871 rpcz_store.cc:269] 0709 13:08:58.254977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:58.255080 (+   103us) service_pool.cc:224] Handling call
0709 13:09:00.099026 (+1843946us) spinlock_profiling.cc:243] Waited 277 ms on lock 0x7b54002d7154. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:00.099161 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":636190976}
I20260709 13:09:00.173046 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:00.150940 25761 raft_consensus.cc:2804] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:09:00.287241 23623 raft_consensus.cc:2393] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 8: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:00.131062 23621 rpcz_store.cc:276] 0709 13:08:58.108705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:58.112887 (+  4182us) service_pool.cc:224] Handling call
0709 13:09:00.107867 (+1994980us) spinlock_profiling.cc:243] Waited 236 ms on lock 0x7b54005901d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:00.108020 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":543054592}
I20260709 13:09:00.288246 25761 raft_consensus.cc:697] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:00.288554 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:00.288676 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:00.289206 25761 consensus_queue.cc:237] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:00.289700 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:00.289958 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:00.137181 25796 consensus_queue.cc:1059] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.161s
I20260709 13:09:00.291589 23623 raft_consensus.cc:1240] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:09:00.291688 25796 raft_consensus.cc:3055] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:09:00.292162 25796 raft_consensus.cc:740] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:00.290230 23621 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:00.129105 23625 pending_rounds.cc:85] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2: Aborting all ops after (but not including) 6
I20260709 13:09:00.125885 25791 consensus_queue.cc:1059] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.176s
I20260709 13:09:00.129923 25664 raft_consensus.cc:3055] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:09:00.147413 25639 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:00.139945 25573 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:00.139634 25605 raft_consensus.cc:2749] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:09:00.137028 25440 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:00.127694 25781 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:00.166966 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 228880us
I20260709 13:09:00.315907 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:00.160838 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 113785us
I20260709 13:09:00.320586 22919 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:00.172772 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:00.192044 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:00.171717 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:00.330196 23618 raft_consensus.cc:2410] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 10 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 13:09:00.200909 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:00.181039 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:00.294498 23625 pending_rounds.cc:107] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:09:00.332080 23625 raft_consensus.cc:2889] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:00.332111 23617 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:09:00.125609 25704 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.811s	user 0.023s	sys 0.028s
I20260709 13:09:00.181711 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
W20260709 13:09:00.129235 23025 rpcz_store.cc:269] 0709 13:08:57.806408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:57.813341 (+  6933us) service_pool.cc:224] Handling call
0709 13:08:59.981792 (+2168451us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b5400189f54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:00.093337 (+111545us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":374436608}
I20260709 13:09:00.322999 22948 raft_consensus.cc:1240] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:09:00.322503 22919 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:00.335518 23619 raft_consensus.cc:2393] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 8: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:00.138147 23021 rpcz_store.cc:276] 0709 13:08:58.828744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:58.850083 (+ 21339us) service_pool.cc:224] Handling call
0709 13:08:58.850498 (+   415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:59.160048 (+309550us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:00.120184 (+960136us) spinlock_profiling.cc:243] Waited 309 ms on lock 0x7b54002c9a54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:00.120457 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":709625728}
I20260709 13:09:00.177682 25754 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:00.361771 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:00.336287 23625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 591) took 2030 ms. Trace:
W20260709 13:09:00.392697 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 70256.6us
I20260709 13:09:00.393724 22919 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:00.398132 22950 raft_consensus.cc:1240] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 13:09:00.343613 23621 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:09:00.128809 25588 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:09:00.176743 25785 log.cc:927] Time spent T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.287s	user 0.001s	sys 0.000s
I20260709 13:09:00.361292 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:00.386535 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:00.361184 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:00.386739 25440 raft_consensus.cc:2804] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:09:00.465858 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:00.391008 23625 rpcz_store.cc:276] 0709 13:08:58.306073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:58.306314 (+   241us) service_pool.cc:224] Handling call
0709 13:09:00.098473 (+1792159us) spinlock_profiling.cc:243] Waited 1.58 ms on lock 0x7b58001c4d98. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0a22 00007f613bbd9c09 00007f613ad33ee0 00007f613adfd0ab 00007f613adfd131 00007f613adbefc0 00007f613ada9929 00007f613ada93d2 00007f6134320bd4
0709 13:09:00.098707 (+   234us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:00.332854 (+234147us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:00.336228 (+  3374us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3637504}
I20260709 13:09:00.412807 25794 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:00.305137 25573 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:00.177559 25795 raft_consensus.cc:2749] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:09:00.137748 25729 raft_consensus.cc:740] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:00.304584 25639 raft_consensus.cc:3055] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:09:00.412468 25793 log.cc:927] Time spent T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.285s	user 0.001s	sys 0.000s
I20260709 13:09:00.303992 25664 raft_consensus.cc:740] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:09:00.421703 22826 auto_rebalancer.cc:629] Failed to schedule move for tablet b7633b7b4c334b039db57d77774df345: Illegal state: Replica aed3f089ec8b49af9c8cf5cdee76e8e7 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 6 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:09:00.159375 25790 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:09:00.337013 25788 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:00.176832 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:00.309163 25781 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:00.145941 25797 raft_consensus.cc:740] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
W20260709 13:09:00.337719 25704 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.023s	user 0.024s	sys 0.029s
I20260709 13:09:00.335901 25801 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:00.308794 25800 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:00.451555 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:00.335371 25784 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:00.461547 23617 raft_consensus.cc:2410] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 13:09:00.508494 25639 raft_consensus.cc:740] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
W20260709 13:09:00.493420 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 532066us
I20260709 13:09:00.492622 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:00.329504 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:00.421154 25637 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:00.503943 23588 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.032s	sys 0.077s
W20260709 13:09:00.544260 23588 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.033s	sys 0.077s
W20260709 13:09:00.544631 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 607300us
I20260709 13:09:00.546804 23588 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:00.176007 25593 consensus_queue.cc:260] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:08:59.728091 25318 consensus_queue.cc:1175] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "0a407504ed1741bf89a276558adf4c0f" responder_term: 8 status { last_received { term: 5 index: 4 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 7. Current term is 8. Ops: [7.5-7.5]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:09:00.451225 25807 raft_consensus.cc:2749] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:09:00.516109 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 211191us
I20260709 13:09:00.492026 25440 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:00.169853 25663 consensus_queue.cc:237] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:00.518024 22953 raft_consensus.cc:1275] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:09:00.500000 25573 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:09:00.519909 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 541328us
I20260709 13:09:00.561424 22843 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:00.420511 25636 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:00.450093 25588 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:00.177258 25370 consensus_queue.cc:237] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:00.402871 25754 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:00.328783 25660 consensus_queue.cc:237] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:00.532282 22946 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:00.533129 22948 raft_consensus.cc:1240] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 4. Current term is 5. Ops: []
W20260709 13:09:00.439061 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:00.535740 23617 raft_consensus.cc:1240] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260709 13:09:00.536244 23625 raft_consensus.cc:1240] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:00.516963 25790 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:00.557338 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:00.517510 25812 raft_consensus.cc:2749] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:09:00.542709 25595 consensus_meta.cc:287] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.606s	user 0.010s	sys 0.000s
I20260709 13:09:00.562799 22843 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:00.502836 25794 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:00.560295 25718 consensus_meta.cc:287] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.762s	user 0.008s	sys 0.002s
I20260709 13:09:00.599419 22843 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:00.536382 25811 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:00.532133 23616 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 5.
W20260709 13:09:00.587743 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:00.604676 22843 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:00.593856 22871 raft_consensus.cc:1240] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:09:00.555208 23620 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
I20260709 13:09:00.603600 22949 raft_consensus.cc:1240] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:09:00.576043 25637 raft_consensus.cc:2804] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:09:00.561614 25440 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:00.522714 25788 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:00.592681 25709 consensus_meta.cc:287] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.845s	user 0.012s	sys 0.000s
I20260709 13:09:00.541712 25800 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:00.560824 25777 consensus_queue.cc:1048] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:09:00.562090 25804 consensus_queue.cc:1048] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
W20260709 13:09:00.653209 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 48640.7us
I20260709 13:09:00.591269 25790 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:09:00.614221 25783 consensus_meta.cc:287] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.655s	user 0.011s	sys 0.000s
I20260709 13:09:00.544781 25784 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:00.607571 25595 raft_consensus.cc:2749] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:09:00.618018 22877 raft_consensus.cc:1275] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:09:00.670571 22946 raft_consensus.cc:2468] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:09:00.612468 25605 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:00.614869 25637 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:00.575196 25805 consensus_queue.cc:1059] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.111s
I20260709 13:09:00.541625 25801 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:00.717522 23590 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:00.728824 25828 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:00.572444 25664 consensus_queue.cc:260] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:00.663008 25810 raft_consensus.cc:2749] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:09:00.729282 22872 raft_consensus.cc:1275] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 13:09:00.733160 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 186432us
I20260709 13:09:00.662475 25819 raft_consensus.cc:2749] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:09:00.699721 25799 consensus_meta.cc:287] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.744s	user 0.011s	sys 0.000s
I20260709 13:09:00.555395 25729 consensus_queue.cc:260] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:00.669739 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:00.754251 22877 raft_consensus.cc:1240] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 6. Current term is 7. Ops: []
W20260709 13:09:00.757153 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:00.616930 22992 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:00.615550 25811 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:00.786535 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 169702us
I20260709 13:09:00.787756 22992 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:00.664042 25817 consensus_queue.cc:1059] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.098s
I20260709 13:09:00.670570 25639 consensus_queue.cc:260] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:00.683954 25709 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:00.740617 25783 raft_consensus.cc:2749] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:09:00.756886 25637 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:00.800411 25588 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:00.699584 25813 consensus_queue.cc:1048] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.080s
W20260709 13:09:00.800766 25815 log.cc:927] Time spent T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
W20260709 13:09:00.801712 25821 log.cc:927] Time spent T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:09:00.762576 25828 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:00.802485 25828 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:09:00.777163 25572 consensus_meta.cc:287] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.757s	user 0.011s	sys 0.000s
I20260709 13:09:00.611760 25718 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:00.804036 22952 raft_consensus.cc:2410] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 13:09:00.812203 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 86587.2us
W20260709 13:09:00.801766 25814 log.cc:927] Time spent T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:09:00.820894 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:00.797060 25662 raft_consensus.cc:2749] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:09:00.825655 22843 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:00.804867 25729 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:00.829658 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:00.830317 23025 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 3.
I20260709 13:09:00.838622 22843 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:00.815299 25639 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:00.812906 25703 raft_consensus.cc:2749] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:09:00.848846 25754 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:00.852798 25810 raft_consensus.cc:2749] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:09:00.852986 25816 log.cc:927] Time spent T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
I20260709 13:09:00.853543 25810 raft_consensus.cc:2749] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:09:00.822302 25813 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:00.857156 25813 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:00.859882 25813 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:00.827837 25636 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:00.861244 25784 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:00.864070 25823 log.cc:927] Time spent T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 13:09:00.811475 25818 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:00.864955 25818 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:00.865330 25817 consensus_queue.cc:1048] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:00.783039 25799 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:00.867074 25818 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:00.867574 25599 consensus_meta.cc:287] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 1.006s	user 0.010s	sys 0.000s
I20260709 13:09:00.867988 25599 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:00.822546 25588 raft_consensus.cc:2804] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:00.815426 25572 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:00.899674 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 182269us
W20260709 13:09:00.905656 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 133256us
I20260709 13:09:00.843514 25662 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:00.872222 25588 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:00.916694 23588 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
W20260709 13:09:00.919318 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:00.875765 25799 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:00.836758 25819 consensus_queue.cc:1048] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 13:09:00.925046 22877 raft_consensus.cc:1240] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 7. Current term is 8. Ops: []
W20260709 13:09:00.929410 25827 log.cc:927] Time spent T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 13:09:00.929795 25570 raft_consensus.cc:3055] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:09:00.930438 25570 raft_consensus.cc:740] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:00.931274 25570 consensus_queue.cc:260] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:00.932301 25570 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:09:00.932850 25825 log.cc:927] Time spent T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 13:09:00.934754 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:00.935606 22878 raft_consensus.cc:2393] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 8: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:00.852442 25781 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:00.936537 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:00.937350 22878 raft_consensus.cc:2393] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 8: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:00.938284 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:00.938962 23622 raft_consensus.cc:2393] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 10: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:00.929747 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:00.938292 25781 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:00.939983 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:00.940546 22947 raft_consensus.cc:2376] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 8. Re-sending same reply.
I20260709 13:09:00.941376 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:00.942255 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:00.936327 25828 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:00.945605 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:00.946229 25599 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:00.946614 25788 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:00.922459 25588 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:00.919595 25794 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:00.921949 25573 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:00.955531 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:00.956871 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 118318us
I20260709 13:09:00.961920 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:00.946811 25828 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:00.949293 22947 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 7.
W20260709 13:09:00.966945 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 145622us
I20260709 13:09:00.961134 22918 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:00.971325 22843 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:00.980651 22918 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:00.981498 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:00.982163 22918 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:09:00.982412 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 65821.4us
I20260709 13:09:00.996608 23025 raft_consensus.cc:1240] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 6. Current term is 7. Ops: []
I20260709 13:09:00.997809 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:00.994724 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:00.998595 22878 raft_consensus.cc:2468] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
I20260709 13:09:00.995160 23588 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:00.999610 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:01.002281 25761 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:01.002671 25718 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:01.002801 25761 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:01.006160 25599 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:00.955941 25624 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:00.955317 25592 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:01.007647 22994 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:01.010460 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:01.011637 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:01.012236 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:01.013952 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:09:01.014576 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:01.015756 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:01.017170 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:01.006273 25761 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:01.024005 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:00.969811 25800 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:01.014683 23021 raft_consensus.cc:2376] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8. Re-sending same reply.
I20260709 13:09:01.033390 25818 consensus_queue.cc:1048] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:01.029253 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:00.995608 25813 raft_consensus.cc:2804] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:09:00.922622 25662 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:01.035996 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:01.031069 23588 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:01.040733 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:01.007817 25798 raft_consensus.cc:2749] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:09:01.033520 25663 raft_consensus.cc:2749] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:01.045974 22873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35360 (request call id 1644) took 1971 ms. Trace:
I20260709 13:09:01.046267 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:01.046489 22873 rpcz_store.cc:276] 0709 13:08:59.074778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:59.075069 (+   291us) service_pool.cc:224] Handling call
0709 13:09:01.045784 (+1970715us) spinlock_profiling.cc:243] Waited 436 ms on lock 0x7b5400367c54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:01.045908 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1003800448}
I20260709 13:09:01.032025 22878 raft_consensus.cc:1240] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 13:09:00.990152 25819 raft_consensus.cc:2804] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:01.019155 25788 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:09:01.051909 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:01.048453 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:01.054343 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:01.058399 22878 raft_consensus.cc:1275] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:09:01.059409 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:01.062346 23021 raft_consensus.cc:2368] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 5. Current term is 6.
I20260709 13:09:01.066798 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:01.035241 25829 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.067659 23588 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:01.067775 23026 raft_consensus.cc:2368] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 6. Current term is 7.
I20260709 13:09:01.076155 23025 raft_consensus.cc:1275] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:09:01.045871 25602 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.089931 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:01.075481 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:00.948552 25796 consensus_queue.cc:260] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:01.085551 25777 consensus_queue.cc:1059] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:09:01.086177 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:01.025262 25592 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:01.023437 25624 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:01.051545 25819 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:01.030987 25680 consensus_queue.cc:1048] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.038s
W20260709 13:09:01.092324 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 67861us
W20260709 13:09:01.062120 25780 consensus_meta.cc:287] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.930s	user 0.008s	sys 0.004s
I20260709 13:09:01.090626 22948 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:09:01.080446 25811 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:01.038532 25781 consensus_queue.cc:1048] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:09:01.051270 25833 raft_consensus.cc:2749] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:09:01.064649 25813 raft_consensus.cc:697] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:01.072098 25709 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:01.115844 23617 raft_consensus.cc:1240] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:01.073113 25788 raft_consensus.cc:697] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:01.137943 22918 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:01.150393 25806 raft_consensus.cc:2749] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:09:01.103085 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:00.552338 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:09:01.128424 25831 log.cc:927] Time spent T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
I20260709 13:09:01.128019 25440 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:01.129261 25835 log.cc:927] Time spent T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 13:09:01.133888 25796 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:01.136368 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:01.136811 23025 raft_consensus.cc:1240] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:09:01.136844 23026 raft_consensus.cc:1275] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:09:01.136976 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:01.165601 23027 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 5.
I20260709 13:09:01.146160 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:01.168040 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:01.169085 22872 raft_consensus.cc:2393] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 5: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:01.138281 22843 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:01.170503 22843 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:01.139282 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:01.108891 25826 raft_consensus.cc:2804] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:09:01.172395 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:01.173448 22952 raft_consensus.cc:2368] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 6. Current term is 7.
I20260709 13:09:01.139652 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:09:01.124917 25708 consensus_meta.cc:287] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 1.334s	user 0.010s	sys 0.003s
I20260709 13:09:01.153906 22878 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:09:01.155534 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 86747.1us
I20260709 13:09:01.156759 22920 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:01.124696 25819 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:01.158965 22880 raft_consensus.cc:1275] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:09:01.159474 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:01.161703 25806 consensus_queue.cc:1048] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:09:01.162688 25592 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:01.174710 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:01.191442 22874 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6.
I20260709 13:09:01.192155 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:01.099753 25602 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:01.182061 22948 raft_consensus.cc:2393] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 7: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:01.107748 25794 consensus_queue.cc:1048] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:09:01.089775 25829 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:01.184270 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:01.186630 22843 leader_election.cc:400] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 5. Current term is 6.
I20260709 13:09:01.133098 25680 consensus_queue.cc:1059] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:09:01.188391 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:01.189301 23590 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:01.189586 22920 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:01.193638 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:01.187870 25662 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:01.222190 22878 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 8.
I20260709 13:09:01.198688 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:01.157847 25777 raft_consensus.cc:3055] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:09:01.156137 25572 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:01.125036 25637 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:01.233930 23617 raft_consensus.cc:1275] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:09:01.211807 22843 leader_election.cc:400] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 6. Current term is 7.
I20260709 13:09:01.212050 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:01.211391 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:01.235157 22843 leader_election.cc:403] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:09:01.235024 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:01.235674 22953 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 6.
I20260709 13:09:01.194818 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:01.182446 25826 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:01.134538 25780 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:01.198119 25595 raft_consensus.cc:2749] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:09:01.197475 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:01.226581 23590 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:01.195237 25818 raft_consensus.cc:2749] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:09:01.228729 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:01.232559 25840 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:09:01.241528 22952 raft_consensus.cc:1275] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:09:01.238579 23020 raft_consensus.cc:2393] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 7: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:01.239584 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:01.240710 22948 raft_consensus.cc:2468] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:09:01.261000 25802 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.261473 25780 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:01.262426 22880 raft_consensus.cc:1240] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 6. Current term is 7. Ops: []
I20260709 13:09:01.264235 22952 raft_consensus.cc:1275] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:09:01.258559 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:01.234313 25777 raft_consensus.cc:740] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:01.234894 25605 raft_consensus.cc:2749] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:09:01.238729 25826 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:01.225020 25573 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:01.240561 25680 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:01.283651 22994 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:01.282370 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:01.291567 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:01.262143 25780 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:09:01.261550 25802 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:01.321358 25662 raft_consensus.cc:2749] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:09:01.334408 25821 log.cc:927] Time spent T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
W20260709 13:09:01.322386 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:01.335005 25852 consensus_queue.cc:1048] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.101s
W20260709 13:09:01.324934 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:01.234169 25708 consensus_queue.cc:1059] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:09:01.333695 25838 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.320503 25798 consensus_queue.cc:1059] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:01.234108 25813 consensus_queue.cc:237] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:01.336603 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 53031.3us
I20260709 13:09:01.337550 25850 raft_consensus.cc:2749] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:09:01.337221 25838 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:01.338234 22994 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:01.338980 25662 raft_consensus.cc:2749] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:09:01.339058 25842 log.cc:927] Time spent T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 13:09:01.340571 23020 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:01.341408 22877 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:01.342232 22994 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:09:01.342676 25849 log.cc:927] Time spent T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:09:01.265340 23021 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:01.347185 25246 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:01.342958 25818 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
W20260709 13:09:01.342854 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 61588.5us
I20260709 13:09:01.361977 25818 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:01.362784 22920 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:01.364198 25798 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:01.364375 25818 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:01.347522 23620 raft_consensus.cc:1240] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 7. Current term is 8. Ops: []
I20260709 13:09:01.365335 25798 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:01.366163 25798 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:01.366976 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:01.368036 23620 raft_consensus.cc:2393] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 7: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:01.369310 25818 consensus_queue.cc:1059] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:09:01.369781 25783 consensus_queue.cc:1048] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:01.370193 25818 raft_consensus.cc:3055] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:09:01.370651 25818 raft_consensus.cc:740] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:01.380995 23021 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 8.
W20260709 13:09:01.388468 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:01.395118 25838 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:01.395752 25588 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:01.397727 25836 log.cc:927] Time spent T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.281s	user 0.001s	sys 0.000s
I20260709 13:09:01.397938 25588 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:01.398772 25843 log.cc:927] Time spent T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.197s	user 0.000s	sys 0.000s
I20260709 13:09:01.400319 25707 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:01.401644 25845 log.cc:927] Time spent T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.253s	user 0.001s	sys 0.000s
W20260709 13:09:01.402089 25853 log.cc:927] Time spent T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 13:09:01.401989 25829 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:01.403406 25847 log.cc:927] Time spent T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:09:01.405282 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:01.406018 23024 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:01.340647 25615 raft_consensus.cc:3055] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:09:01.407238 25615 raft_consensus.cc:740] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:01.407323 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:01.407989 23619 raft_consensus.cc:2376] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10. Re-sending same reply.
I20260709 13:09:01.408108 25615 consensus_queue.cc:260] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:01.408959 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:01.409276 25615 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:01.410084 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:01.259192 25788 consensus_queue.cc:237] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:01.411140 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:09:01.411782 25790 consensus_meta.cc:287] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.671s	user 0.007s	sys 0.004s
I20260709 13:09:01.412168 25790 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:01.413051 25624 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:01.413439 22951 raft_consensus.cc:1240] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 6. Current term is 7. Ops: []
I20260709 13:09:01.280767 25806 consensus_queue.cc:1048] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.133s
I20260709 13:09:01.336877 25832 raft_consensus.cc:2749] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 6. Current term is 7.
I20260709 13:09:01.415719 23020 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 9.
I20260709 13:09:01.416782 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:09:01.417486 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 75325us
W20260709 13:09:01.418038 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 45794.2us
I20260709 13:09:01.419649 22920 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:01.344352 22844 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:01.457906 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:01.469560 22952 raft_consensus.cc:1275] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 13:09:01.470549 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 51000us
I20260709 13:09:01.474140 22920 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:01.470765 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:01.483577 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 139300us
I20260709 13:09:01.485893 23024 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 5.
W20260709 13:09:01.494361 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 236529us
W20260709 13:09:01.495416 22844 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.63.4:44043 timed out after 1.947s (SENT) [suppressed 44 similar messages]
W20260709 13:09:01.495566 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:01.497656 25573 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.498224 25573 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:01.479125 22952 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:09:01.502599 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:01.503204 22952 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
I20260709 13:09:01.351931 25660 raft_consensus.cc:3055] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:09:01.504544 25660 raft_consensus.cc:740] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:01.505609 25660 consensus_queue.cc:260] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:01.507195 25660 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:09:01.510550 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:01.359100 25802 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:01.512446 25624 consensus_queue.cc:1048] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:09:01.512959 25802 consensus_queue.cc:1059] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:09:01.346486 25599 raft_consensus.cc:2749] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:09:01.525954 25780 raft_consensus.cc:697] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:01.570529 22877 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6.
I20260709 13:09:01.582948 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:01.469255 25832 raft_consensus.cc:2749] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:09:01.469825 25707 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.469285 25838 raft_consensus.cc:2804] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:09:01.527676 25780 consensus_queue.cc:237] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:01.489907 22920 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:01.487597 25817 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:01.589473 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 99646.3us
I20260709 13:09:01.492110 25806 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:01.487109 25680 raft_consensus.cc:2749] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:09:01.594616 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:01.596686 23020 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 6.
W20260709 13:09:01.587249 23028 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.017s	sys 0.026s
I20260709 13:09:01.534106 25662 raft_consensus.cc:3055] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 13:09:01.601593 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 106172us
I20260709 13:09:01.585048 25573 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:01.515552 25605 consensus_queue.cc:1048] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.106s
I20260709 13:09:01.595775 25624 raft_consensus.cc:2804] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:09:01.635658 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 138694us
I20260709 13:09:01.592288 25838 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:01.634766 22879 raft_consensus.cc:1275] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:09:01.601200 25805 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.350279 25777 consensus_queue.cc:260] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:01.602380 25662 raft_consensus.cc:740] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:01.648325 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:01.596681 25707 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:01.660025 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 64114.5us
I20260709 13:09:01.658128 22876 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:01.635695 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:01.635903 25624 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:01.661926 22843 leader_election.cc:400] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 election: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 6. Current term is 7.
I20260709 13:09:01.674589 23023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 1606) took 2396 ms. Trace:
I20260709 13:09:01.597471 25817 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:01.675107 23023 rpcz_store.cc:276] 0709 13:08:59.277974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:59.278536 (+   562us) service_pool.cc:224] Handling call
0709 13:08:59.278869 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:08:59.937737 (+658868us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:01.674334 (+1736597us) spinlock_profiling.cc:243] Waited 631 ms on lock 0x7b54001f8bd4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:01.674528 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1450968448}
I20260709 13:09:01.678161 23023 raft_consensus.cc:1240] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:09:01.678422 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:01.667716 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:01.680773 22878 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:09:01.668313 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:01.669682 25662 consensus_queue.cc:260] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:01.674762 22843 leader_election.cc:403] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 13:09:01.641093 25838 raft_consensus.cc:3060] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:01.719470 22843 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:01.677204 25624 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:01.680600 23620 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:01.740195 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:01.719715 22920 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:01.680394 25790 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:01.684213 25662 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:09:01.753827 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 66771.9us
I20260709 13:09:01.775110 22843 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:01.777369 22843 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:01.772938 25811 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:01.674652 25639 consensus_queue.cc:1048] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 13:09:01.735808 25840 raft_consensus.cc:2749] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:09:01.650910 25805 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:01.718240 25783 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.723209 25573 raft_consensus.cc:2749] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 6. Current term is 7.
I20260709 13:09:01.781598 25811 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:01.785313 25840 raft_consensus.cc:2749] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:09:01.785386 25811 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:01.788028 22876 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 8.
I20260709 13:09:01.782305 25810 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:01.801971 23022 raft_consensus.cc:1275] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:09:01.802515 23022 pending_rounds.cc:85] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75: Aborting all ops after (but not including) 7
I20260709 13:09:01.802783 23022 pending_rounds.cc:107] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:09:01.803086 23022 raft_consensus.cc:2889] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:01.815105 25707 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:09:01.805507 25861 log.cc:927] Time spent T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 13:09:01.786460 25818 consensus_queue.cc:260] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:01.804353 25363 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.784976 25826 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:01.787369 25783 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:01.825919 25859 log.cc:927] Time spent T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.230s	user 0.000s	sys 0.001s
I20260709 13:09:01.842944 25818 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:09:01.851037 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:01.832392 25363 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:01.841251 25857 log.cc:927] Time spent T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.230s	user 0.000s	sys 0.001s
I20260709 13:09:01.879297 25783 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:01.882897 22954 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2208) took 3214 ms. Trace:
I20260709 13:09:01.886124 22954 rpcz_store.cc:276] 0709 13:08:58.668801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:58.668938 (+   137us) service_pool.cc:224] Handling call
0709 13:08:59.898276 (+1229338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:08:59.909756 (+ 11480us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:01.882647 (+1972891us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b540044af50. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:01.882825 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2766024576}
I20260709 13:09:01.889379 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:09:01.881294 25819 consensus_meta.cc:287] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.679s	user 0.011s	sys 0.001s
W20260709 13:09:01.901865 25867 log.cc:927] Time spent T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
W20260709 13:09:01.902364 25871 log.cc:927] Time spent T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
W20260709 13:09:01.905263 25868 log.cc:927] Time spent T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
W20260709 13:09:01.903551 25798 consensus_meta.cc:287] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.537s	user 0.008s	sys 0.003s
I20260709 13:09:01.906656 25798 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:01.915879 25865 log.cc:927] Time spent T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.233s	user 0.001s	sys 0.000s
I20260709 13:09:01.908864 25798 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:01.922763 25806 raft_consensus.cc:2749] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:09:01.914412 25819 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:01.927436 25796 consensus_meta.cc:287] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.739s	user 0.008s	sys 0.000s
W20260709 13:09:01.943475 25872 log.cc:927] Time spent T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 13:09:01.945966 22948 raft_consensus.cc:1240] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 6. Current term is 7. Ops: []
I20260709 13:09:01.914243 25817 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:09:01.930260 25866 log.cc:927] Time spent T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 13:09:01.970937 25878 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:01.971704 25878 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:01.930609 23620 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
W20260709 13:09:02.002081 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:02.010622 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 350891us
I20260709 13:09:02.035213 25826 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:02.036762 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:02.038241 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:02.039372 22876 raft_consensus.cc:2410] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:09:02.040328 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:09:02.041004 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 263669us
I20260709 13:09:02.043267 22843 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:02.044618 22876 raft_consensus.cc:2393] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 8: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:01.988044 25624 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:01.987988 25855 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:02.034900 25819 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:02.034003 25805 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:02.048167 25796 raft_consensus.cc:2804] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:09:02.048374 25572 consensus_queue.cc:1048] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 13:09:02.061338 25796 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
W20260709 13:09:02.083007 25615 consensus_meta.cc:287] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.673s	user 0.008s	sys 0.008s
I20260709 13:09:02.083374 25796 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:02.057586 25855 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:02.085216 25573 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:02.094734 25573 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:02.095569 25797 consensus_queue.cc:260] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:02.096832 25573 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:02.097503 25624 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:02.098920 23622 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60044 (request call id 582) took 1926 ms (client timeout 2062 ms). Trace:
W20260709 13:09:02.099349 23622 rpcz_store.cc:269] 0709 13:09:00.172463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:00.172550 (+    87us) service_pool.cc:224] Handling call
0709 13:09:02.098701 (+1926151us) spinlock_profiling.cc:243] Waited 609 ms on lock 0x7b540049a2d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:02.098862 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1400927488}
I20260709 13:09:02.100613 23624 raft_consensus.cc:1240] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:09:02.102838 25855 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:02.104741 25819 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:02.105252 23027 raft_consensus.cc:1240] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 7. Current term is 8. Ops: [5.3-7.4]
I20260709 13:09:02.113855 25878 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:02.107338 25811 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:02.085278 25840 consensus_queue.cc:1059] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 13:09:02.118032 25838 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:02.119164 25637 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:02.120240 25838 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:02.105264 25819 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:02.123232 23588 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:02.123443 25819 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:02.124202 25602 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:02.124398 23588 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:02.125360 23588 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:02.125910 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:02.127750 23027 raft_consensus.cc:1275] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:09:02.129527 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:02.130111 22952 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
I20260709 13:09:02.130754 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:02.131147 22952 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:02.134331 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:02.134157 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:02.135780 25855 raft_consensus.cc:2749] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:09:02.167151 25862 log.cc:927] Time spent T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.528s	user 0.000s	sys 0.007s
I20260709 13:09:02.169270 25811 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:02.135676 22955 raft_consensus.cc:2468] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
I20260709 13:09:02.150884 23623 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 598) took 1864 ms. Trace:
I20260709 13:09:02.176160 22952 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 10.
I20260709 13:09:02.172849 22843 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:02.135262 25718 raft_consensus.cc:2804] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:09:02.176460 23623 rpcz_store.cc:276] 0709 13:09:00.286094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:00.292649 (+  6555us) service_pool.cc:224] Handling call
0709 13:09:00.292934 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:00.601183 (+308249us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:09:00.814754 (+213571us) log.cc:844] Serialized 14 byte log entry
0709 13:09:00.984589 (+169835us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:00.984639 (+    50us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:09:00.984707 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:02.150629 (+1165922us) spinlock_profiling.cc:243] Waited 308 ms on lock 0x7b54005a23d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:02.150676 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:02.150816 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28186,"prepare.queue_time_us":306651,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":2558,"raft.queue_time_us":44760,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":87,"spinlock_wait_cycles":708297984,"thread_start_us":309788,"threads_started":2,"wal-append.queue_time_us":251885}
I20260709 13:09:02.154277 25810 raft_consensus.cc:2749] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:09:02.133150 25878 raft_consensus.cc:2749] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:09:02.198459 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:02.199491 22955 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 8.
I20260709 13:09:02.193130 25718 raft_consensus.cc:697] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:02.204187 25718 consensus_queue.cc:237] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:02.185220 25819 raft_consensus.cc:2804] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:09:02.216918 25819 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:02.220831 25819 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:02.222666 22954 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 1672) took 1070 ms. Trace:
I20260709 13:09:02.223635 22954 rpcz_store.cc:276] 0709 13:09:01.152635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:01.892013 (+739378us) service_pool.cc:224] Handling call
0709 13:09:01.892349 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:01.892612 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:09:02.186342 (+293730us) log.cc:844] Serialized 14 byte log entry
0709 13:09:02.222245 (+ 35903us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:02.222293 (+    48us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:09:02.222338 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:02.222397 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:02.222613 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":293685,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":81,"thread_start_us":321477,"threads_started":2}
I20260709 13:09:02.228554 22877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35360 (request call id 1665) took 1083 ms. Trace:
I20260709 13:09:02.229075 22877 rpcz_store.cc:276] 0709 13:09:01.148085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:01.188130 (+ 40045us) service_pool.cc:224] Handling call
0709 13:09:02.228363 (+1040233us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b5400301e54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:02.228488 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":293584640}
I20260709 13:09:02.233676 22946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 423) took 1461 ms. Trace:
I20260709 13:09:02.234094 22946 rpcz_store.cc:276] 0709 13:09:00.771989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:01.110892 (+338903us) service_pool.cc:224] Handling call
0709 13:09:01.123456 (+ 12564us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:01.123668 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:01.123705 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:01.123759 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:09:01.123803 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:01.123871 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:02.233503 (+1109632us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b540044b950. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:02.233612 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":28053760}
I20260709 13:09:02.242189 23588 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:02.243330 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:02.244374 23621 raft_consensus.cc:2410] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 8 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 13:09:02.247922 22919 connection.cc:256] RPC call timeout handler was delayed by -1.37151135900001s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17210000000000003s for a response
W20260709 13:09:02.248595 22919 connection.cc:256] RPC call timeout handler was delayed by -1.3231767629999922s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1847s for a response
I20260709 13:09:02.250044 23623 raft_consensus.cc:1275] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 6. (term mismatch)
I20260709 13:09:02.250537 23622 raft_consensus.cc:1240] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 6. Current term is 7. Ops: []
I20260709 13:09:02.251379 23588 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:02.252123 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:02.250560 23623 pending_rounds.cc:85] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2: Aborting all ops after (but not including) 5
I20260709 13:09:02.253232 23623 pending_rounds.cc:107] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:09:02.253490 23623 raft_consensus.cc:2889] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:02.254717 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:02.255400 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:02.255684 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:02.255946 23619 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:02.256378 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:02.256846 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:02.257385 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:02.257925 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:02.258571 23618 raft_consensus.cc:1240] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 6. Current term is 7. Ops: []
I20260709 13:09:02.259505 25877 raft_consensus.cc:2804] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:09:02.260025 25877 raft_consensus.cc:697] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 8 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:02.260885 25877 consensus_queue.cc:237] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:02.265261 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:02.270928 23028 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 1653) took 1286 ms. Trace:
I20260709 13:09:02.272261 22993 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:02.271365 23028 rpcz_store.cc:276] 0709 13:09:00.984287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:00.984406 (+   119us) service_pool.cc:224] Handling call
0709 13:09:01.001533 (+ 17127us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:01.001758 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:09:01.074313 (+ 72555us) log.cc:844] Serialized 14 byte log entry
0709 13:09:02.270406 (+1196093us) spinlock_profiling.cc:243] Waited 3.5 ms on lock 0x7b180004fba0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132c72c0a 00007f613c3b2219 00007f6132cfaa40 00007f6132cf54e4 00007f6132cf5455 00007f6135ddd060 00007f6135def716 00007f6135e00287 00007f6135e00208
0709 13:09:02.270560 (+   154us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:02.270617 (+    57us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:09:02.270675 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:02.270745 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:02.270870 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38996,"prepare.queue_time_us":191495,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"spinlock_wait_cycles":8049792,"thread_start_us":596456,"threads_started":2,"wal-append.queue_time_us":560486}
I20260709 13:09:02.275547 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:09:02.249127 22919 connection.cc:256] RPC call timeout handler was delayed by -1.2547767629999953s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19230000000000003s for a response
W20260709 13:09:02.279093 22919 connection.cc:256] RPC call timeout handler was delayed by -1.1296767629999067s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2062s for a response
I20260709 13:09:02.290369 23619 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6.
I20260709 13:09:02.291507 22954 raft_consensus.cc:1240] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 6. Current term is 7. Ops: []
W20260709 13:09:02.293215 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:02.216499 25796 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:02.314855 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:02.322516 22954 raft_consensus.cc:2393] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 5: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:02.337504 25873 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:02.338042 25873 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:02.340911 25246 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:02.341382 25246 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:02.343256 25246 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:02.343701 25246 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:02.344079 22950 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:02.345526 25876 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:02.346026 25876 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:02.349325 25661 raft_consensus.cc:3055] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:09:02.349746 25661 raft_consensus.cc:740] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:02.350518 25661 consensus_queue.cc:260] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:02.291925 25602 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:02.352921 25602 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:02.354903 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:02.336696 25810 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:02.357554 25810 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:02.358621 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:02.359344 23025 raft_consensus.cc:2393] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 7: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:02.359977 25810 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:02.361146 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 88952.4us
I20260709 13:09:02.360560 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:02.376051 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:02.394369 23025 raft_consensus.cc:2410] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 11 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:09:02.313623 25805 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:02.344149 25246 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:02.314723 25883 raft_consensus.cc:2804] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:02.423102 25838 raft_consensus.cc:2804] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:09:02.431116 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:02.431069 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:02.421265 25819 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:02.440119 23029 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 1651) took 1456 ms. Trace:
I20260709 13:09:02.415252 25810 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:02.445075 25810 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:02.451889 25246 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:09:02.437117 25892 log.cc:927] Time spent T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:09:02.419817 22950 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
I20260709 13:09:02.845088 23025 raft_consensus.cc:1240] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 5. Current term is 6. Ops: []
W20260709 13:09:02.817036 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:02.524142 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:02.440865 25883 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:02.537859 22871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 1654) took 2221 ms. Trace:
W20260709 13:09:02.865218 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:02.816781 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 493474us
I20260709 13:09:02.866014 22919 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:02.816923 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:02.440610 23029 rpcz_store.cc:276] 0709 13:09:00.983683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:00.983782 (+    99us) service_pool.cc:224] Handling call
0709 13:09:01.022282 (+ 38500us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:01.039503 (+ 17221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:01.039580 (+    77us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:01.061097 (+ 21517us) log.cc:844] Serialized 52 byte log entry
0709 13:09:01.845825 (+784728us) spinlock_profiling.cc:243] Waited 4.95 ms on lock 0x7b1800041200. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cdae07 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf7c02 00007f613597f20b 00007f6135984d23 00007f61359850d1 00007f6135da0a0f
0709 13:09:02.202819 (+356994us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:09:02.313323 (+110504us) log.cc:844] Serialized 14 byte log entry
0709 13:09:02.424684 (+111361us) log.cc:844] Serialized 14 byte log entry
0709 13:09:02.424941 (+   257us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:02.425037 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:02.439806 (+ 14769us) raft_consensus.cc:1685] finished
0709 13:09:02.439911 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:02.440061 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":119925,"raft.queue_time_us":142126,"raft.run_cpu_time_us":360,"raft.run_wall_time_us":21384,"spinlock_wait_cycles":11390720,"thread_start_us":422646,"threads_started":2,"wal-append.queue_time_us":1140864}
W20260709 13:09:02.818923 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:02.818177 22993 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:02.866307 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:02.868507 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:02.816474 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 565068us
I20260709 13:09:02.852109 22843 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:09:02.869168 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 51056.2us
W20260709 13:09:02.759150 25718 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.005s	sys 0.012s
I20260709 13:09:02.454250 25363 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:02.824921 25593 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:02.431676 25805 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:02.874459 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 55197.5us
I20260709 13:09:02.699141 25810 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:02.836963 25661 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:02.868912 25801 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:02.978811 25800 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:02.539306 25592 raft_consensus.cc:3055] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:09:02.868939 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:02.515417 25796 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:02.516567 23616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 604) took 1459 ms. Trace:
I20260709 13:09:02.864781 22871 rpcz_store.cc:276] 0709 13:09:00.316231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:00.389295 (+ 73064us) service_pool.cc:224] Handling call
0709 13:09:00.416380 (+ 27085us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:00.610880 (+194500us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:02.537533 (+1926653us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b540022bad4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:02.537796 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":407427840}
I20260709 13:09:02.824029 25595 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:02.998183 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:03.027336 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:03.027601 23029 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:02.988325 22993 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:02.983944 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:02.816587 25873 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:09:02.903749 25718 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.697s	user 0.009s	sys 0.013s
I20260709 13:09:02.848202 25370 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:02.455766 25838 raft_consensus.cc:697] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 10 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:09:02.978653 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 112718us
I20260709 13:09:02.903357 25704 raft_consensus.cc:2749] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:09:02.979410 25809 raft_consensus.cc:2749] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:09:02.867597 25883 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:02.884030 25876 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:02.871820 25840 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:02.871416 25637 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:02.982898 25870 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:03.035496 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:03.023994 25246 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:03.054639 22919 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:03.054018 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:03.088968 22954 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:03.015348 25810 raft_consensus.cc:2804] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:09:03.027213 25813 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:03.022504 25592 raft_consensus.cc:740] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:03.026536 23616 rpcz_store.cc:276] 0709 13:09:01.057190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:01.061221 (+  4031us) service_pool.cc:224] Handling call
0709 13:09:02.516316 (+1455095us) spinlock_profiling.cc:243] Waited 534 ms on lock 0x7b54003d90d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:02.516505 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1227541248}
I20260709 13:09:03.059870 25819 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:03.098610 22919 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:03.101055 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:03.080566 25796 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:03.132804 22919 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:03.035190 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 46983.2us
I20260709 13:09:03.053439 25595 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:03.034868 25602 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:03.061002 25370 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:03.097124 25874 raft_consensus.cc:2749] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:09:03.088871 25899 raft_consensus.cc:2804] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:09:03.164769 22954 raft_consensus.cc:2410] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:09:03.174634 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:03.175763 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:03.165474 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:03.176679 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:03.026292 25800 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:03.099774 25810 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:03.085817 25840 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:03.088023 25637 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:03.157871 25812 raft_consensus.cc:2749] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:09:03.096511 25246 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:03.179203 25809 raft_consensus.cc:2749] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:03.092438 25870 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:03.190024 25810 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:09:03.222640 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 352197us
I20260709 13:09:03.104506 25813 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:03.163483 25784 consensus_queue.cc:1048] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:09:03.157704 25796 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:03.178921 25602 consensus_queue.cc:1059] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:09:03.225835 25363 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:03.173492 25838 consensus_queue.cc:237] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:03.205884 25592 consensus_queue.cc:260] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:03.245710 25805 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:03.280339 25840 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:03.264286 25704 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
W20260709 13:09:03.271463 25895 log.cc:927] Time spent T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
I20260709 13:09:03.269537 25592 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:03.255774 25363 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:03.263340 25418 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:03.282889 25588 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:03.290328 25418 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:03.290584 25588 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:03.291004 25637 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:03.292384 25418 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:03.292384 25588 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:03.292917 25904 log.cc:927] Time spent T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
W20260709 13:09:03.294826 25900 log.cc:927] Time spent T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260709 13:09:03.296985 25800 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:03.298892 25813 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:03.300457 25370 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:03.308878 25812 raft_consensus.cc:3055] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:09:03.309510 25812 raft_consensus.cc:740] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:03.310580 25812 consensus_queue.cc:260] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:03.315331 25905 log.cc:927] Time spent T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
W20260709 13:09:03.317044 25901 log.cc:927] Time spent T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.214s	user 0.000s	sys 0.001s
W20260709 13:09:03.319007 25908 log.cc:927] Time spent T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260709 13:09:03.333165 25863 log.cc:927] Time spent T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 1.674s	user 0.003s	sys 0.003s
I20260709 13:09:03.342638 25870 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:09:03.342671 25906 log.cc:927] Time spent T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
W20260709 13:09:03.345638 25897 log.cc:927] Time spent T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
W20260709 13:09:03.349215 25907 log.cc:927] Time spent T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
W20260709 13:09:03.382481 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 353380us
I20260709 13:09:03.386168 22876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 1675) took 1714 ms. Trace:
I20260709 13:09:03.164021 22919 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:03.387624 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:09:03.388978 22921 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.021s	sys 0.017s
W20260709 13:09:03.389457 22921 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.541s	user 0.022s	sys 0.017s
W20260709 13:09:03.390365 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:03.390408 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:03.392644 22951 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2230) took 3064 ms. Trace:
I20260709 13:09:03.392930 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:09:03.393298 22843 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.003s	sys 0.006s
W20260709 13:09:03.394047 22843 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.003s	sys 0.006s
I20260709 13:09:03.393245 22951 rpcz_store.cc:276] 0709 13:09:00.328248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:00.329127 (+   879us) service_pool.cc:224] Handling call
0709 13:09:00.329396 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:01.413936 (+1084540us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:03.392016 (+1978080us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b540037afd4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:03.392395 (+   379us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2492501888}
I20260709 13:09:03.394387 22954 raft_consensus.cc:2393] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 8: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:03.395040 22954 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52222 (request call id 431) took 1112 ms. Trace:
I20260709 13:09:03.395526 22954 rpcz_store.cc:276] 0709 13:09:02.282681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:03.392231 (+1109550us) service_pool.cc:224] Handling call
0709 13:09:03.394956 (+  2725us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:03.396955 22951 raft_consensus.cc:1240] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 8. Current term is 10. Ops: [8.8-8.8]
I20260709 13:09:03.397814 22951 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 432) took 1091 ms. Trace:
I20260709 13:09:03.398206 22954 raft_consensus.cc:1240] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
W20260709 13:09:03.398511 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:03.398274 22951 rpcz_store.cc:276] 0709 13:09:02.306951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:03.395562 (+1088611us) service_pool.cc:224] Handling call
0709 13:09:03.396294 (+   732us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:03.397454 (+  1160us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:03.397735 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:03.399070 22954 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2256) took 1040 ms. Trace:
I20260709 13:09:03.399521 22954 rpcz_store.cc:276] 0709 13:09:02.358256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:03.397225 (+1038969us) service_pool.cc:224] Handling call
0709 13:09:03.397778 (+   553us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:03.398819 (+  1041us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:03.398997 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:09:03.400270 22918 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.006s	sys 0.007s
W20260709 13:09:03.400760 22918 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.007s	sys 0.007s
I20260709 13:09:03.401473 22955 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 427) took 1120 ms. Trace:
I20260709 13:09:03.401567 22947 raft_consensus.cc:1240] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:09:03.402014 22955 rpcz_store.cc:276] 0709 13:09:02.281651 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:02.281787 (+   136us) service_pool.cc:224] Handling call
0709 13:09:02.321742 (+ 39955us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:02.827776 (+506034us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:02.827832 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:02.828254 (+   422us) log.cc:844] Serialized 26 byte log entry
0709 13:09:03.071350 (+243096us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:09:03.071406 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:03.071499 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:03.400887 (+329388us) raft_consensus.cc:1685] finished
0709 13:09:03.400935 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:03.401253 (+   318us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":505253,"raft.queue_time_us":506032,"raft.run_cpu_time_us":60,"raft.run_wall_time_us":60,"thread_start_us":239424,"threads_started":1,"wal-append.queue_time_us":364533}
I20260709 13:09:03.403474 22947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 422) took 2631 ms. Trace:
I20260709 13:09:03.407040 22952 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 428) took 1125 ms. Trace:
W20260709 13:09:03.409569 22826 auto_rebalancer.cc:629] Failed to schedule move for tablet 5cbb1e724f1d4e3f97c0d29d6b2ce944: Illegal state: Leader has not yet committed an operation in its own term
W20260709 13:09:03.410235 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 246179us
I20260709 13:09:03.412176 22919 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:03.446060 23023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 1664) took 2304 ms. Trace:
I20260709 13:09:03.447815 22919 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:09:03.413431 22950 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58320 (request call id 2236) took 3010 ms (client timeout 3232 ms). Trace:
I20260709 13:09:03.386866 22876 rpcz_store.cc:276] 0709 13:09:01.671027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:02.049484 (+378457us) service_pool.cc:224] Handling call
0709 13:09:02.058467 (+  8983us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:02.066827 (+  8360us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:02.066846 (+    19us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:02.079023 (+ 12177us) log.cc:844] Serialized 26 byte log entry
0709 13:09:02.541880 (+462857us) spinlock_profiling.cc:243] Waited 2.98 ms on lock 0x7b1800485b00. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cdae07 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf7c02 00007f613597f20b 00007f6135984d23 00007f61359850d1 00007f6135da0a0f
0709 13:09:02.867935 (+326055us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:09:02.867974 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:02.868034 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:03.385129 (+517095us) raft_consensus.cc:1685] finished
0709 13:09:03.385194 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:03.385788 (+   594us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7919,"raft.queue_time_us":19378,"raft.run_cpu_time_us":183,"raft.run_wall_time_us":5816,"spinlock_wait_cycles":6859008,"thread_start_us":325737,"threads_started":1,"wal-append.queue_time_us":953499}
I20260709 13:09:03.415275 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:03.452283 22950 rpcz_store.cc:269] 0709 13:09:00.390413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:00.407037 (+ 16624us) service_pool.cc:224] Handling call
0709 13:09:03.400321 (+2993284us) spinlock_profiling.cc:243] Waited 1.81 s on lock 0x7b54000753d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:03.400554 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4159676672}
I20260709 13:09:03.453243 22955 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:09:03.416175 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:03.454016 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:03.403918 22947 rpcz_store.cc:276] 0709 13:09:00.771641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:00.972174 (+200533us) service_pool.cc:224] Handling call
0709 13:09:03.400870 (+2428696us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:03.402200 (+  1330us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:03.403204 (+  1004us) spinlock_profiling.cc:243] Waited 2.4 s on lock 0x7b54000753d0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:03.403416 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5509417344}
I20260709 13:09:03.407502 22952 rpcz_store.cc:276] 0709 13:09:02.281929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:02.282041 (+   112us) service_pool.cc:224] Handling call
0709 13:09:02.282349 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:02.817670 (+535321us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:02.817702 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:02.849622 (+ 31920us) log.cc:844] Serialized 26 byte log entry
0709 13:09:03.406400 (+556778us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:09:03.406466 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:03.406610 (+   144us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:03.406654 (+    44us) raft_consensus.cc:1685] finished
0709 13:09:03.406667 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:03.406873 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":545162,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":137,"thread_start_us":400744,"threads_started":1,"wal-append.queue_time_us":371833}
I20260709 13:09:03.420640 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:03.456262 22948 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
I20260709 13:09:03.422808 22953 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2249) took 2300 ms. Trace:
I20260709 13:09:03.457337 22953 rpcz_store.cc:276] 0709 13:09:01.122276 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:01.263109 (+140833us) service_pool.cc:224] Handling call
0709 13:09:01.263355 (+   246us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:01.271888 (+  8533us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:01.281376 (+  9488us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:01.281503 (+   127us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:09:01.281637 (+   134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:03.422239 (+2140602us) spinlock_profiling.cc:243] Waited 8.36 ms on lock 0x7b54000ed8d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:03.422323 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:03.422621 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19218688}
W20260709 13:09:03.396458 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 544386us
W20260709 13:09:03.435518 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 434244us
I20260709 13:09:03.459949 22992 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:03.436210 23027 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 489) took 2089 ms. Trace:
I20260709 13:09:03.441720 23020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 1616) took 2380 ms. Trace:
I20260709 13:09:03.460759 22843 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:03.460928 23020 rpcz_store.cc:276] 0709 13:09:01.060892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:01.603173 (+542281us) service_pool.cc:224] Handling call
0709 13:09:01.603605 (+   432us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:02.414309 (+810704us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:02.414338 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:02.828780 (+414442us) log.cc:844] Serialized 52 byte log entry
0709 13:09:03.035219 (+206439us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:03.035305 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:03.035391 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:03.440963 (+405572us) raft_consensus.cc:1685] finished
0709 13:09:03.441016 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:03.441468 (+   452us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":744525,"raft.queue_time_us":715208,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":15750,"thread_start_us":311993,"threads_started":2,"wal-append.queue_time_us":401792}
I20260709 13:09:03.463330 22843 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:03.463449 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:03.443125 23021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 1667) took 2250 ms. Trace:
I20260709 13:09:03.465764 22843 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:03.466094 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:03.464820 23021 rpcz_store.cc:276] 0709 13:09:01.192778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:02.114315 (+921537us) service_pool.cc:224] Handling call
0709 13:09:02.114715 (+   400us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:02.352325 (+237610us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:02.352367 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:02.826098 (+473731us) log.cc:844] Serialized 52 byte log entry
0709 13:09:02.934306 (+108208us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:09:02.934360 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:02.934458 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:03.442475 (+508017us) raft_consensus.cc:1685] finished
0709 13:09:03.442531 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:03.442898 (+   367us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":239004,"raft.queue_time_us":237481,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":115,"spinlock_wait_cycles":21376,"thread_start_us":107729,"threads_started":1,"wal-append.queue_time_us":347145}
I20260709 13:09:03.467489 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:03.472182 23021 raft_consensus.cc:1240] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:09:03.481019 25418 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:03.481992 25418 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:03.483558 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:03.485401 23021 raft_consensus.cc:2410] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:09:03.488289 25602 consensus_queue.cc:1059] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:09:03.460801 23027 rpcz_store.cc:276] 0709 13:09:01.346457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:02.182472 (+836015us) service_pool.cc:224] Handling call
0709 13:09:02.182982 (+   510us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:02.353098 (+170116us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:02.353153 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:02.827678 (+474525us) log.cc:844] Serialized 26 byte log entry
0709 13:09:03.073531 (+245853us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:09:03.073569 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:03.073645 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:03.342144 (+268499us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:09:03.435467 (+ 93323us) raft_consensus.cc:1685] finished
0709 13:09:03.435512 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:03.435850 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":169603,"raft.queue_time_us":170174,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":82,"spinlock_wait_cycles":23817600,"thread_start_us":245489,"threads_started":1,"wal-append.queue_time_us":329689}
I20260709 13:09:03.490001 25602 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:03.490598 23027 raft_consensus.cc:1275] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:09:03.490372 25572 consensus_queue.cc:1059] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:09:03.490787 25602 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:03.491632 25572 raft_consensus.cc:2749] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:09:03.387660 22880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 1664) took 2243 ms. Trace:
I20260709 13:09:03.492832 22880 rpcz_store.cc:276] 0709 13:09:01.144414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:01.211675 (+ 67261us) service_pool.cc:224] Handling call
0709 13:09:01.233579 (+ 21904us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:01.262851 (+ 29272us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:03.386997 (+2124146us) spinlock_profiling.cc:243] Waited 28.6 ms on lock 0x7b54001bc454. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:03.387436 (+   439us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":65861376}
I20260709 13:09:03.493580 25602 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:03.494215 25602 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:03.488386 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:03.477263 22992 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:03.477807 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:03.497112 23618 raft_consensus.cc:2410] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:09:03.498099 25796 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:03.499039 25796 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:09:03.499694 25796 raft_consensus.cc:697] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 9 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:03.500612 25595 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:03.501324 25796 consensus_queue.cc:237] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:03.287568 25704 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:03.477983 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:03.478336 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:03.479012 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:03.485410 25418 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:03.486158 25246 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:03.486333 25599 raft_consensus.cc:2749] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:09:03.488848 25363 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:03.494928 25602 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:03.476809 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:03.446666 23023 rpcz_store.cc:276] 0709 13:09:01.141680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:01.679677 (+537997us) service_pool.cc:224] Handling call
0709 13:09:01.679968 (+   291us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:02.294082 (+614114us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:02.294127 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:02.826098 (+531971us) log.cc:844] Serialized 52 byte log entry
0709 13:09:03.025421 (+199323us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:09:03.444171 (+418750us) log.cc:844] Serialized 14 byte log entry
0709 13:09:03.445133 (+   962us) log.cc:844] Serialized 14 byte log entry
0709 13:09:03.445437 (+   304us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:03.445563 (+   126us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:03.445633 (+    70us) raft_consensus.cc:1685] finished
0709 13:09:03.445645 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:03.445879 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":628127,"prepare.queue_time_us":114283,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":665464,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":97,"thread_start_us":611362,"threads_started":2,"wal-append.queue_time_us":270287}
I20260709 13:09:03.473657 23022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 1605) took 4195 ms. Trace:
I20260709 13:09:03.516812 25704 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:03.517976 25754 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:03.521519 25812 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:03.516579 23620 raft_consensus.cc:2410] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:09:03.551934 23624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 599) took 3249 ms. Trace:
I20260709 13:09:03.551589 25754 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:03.554108 25784 consensus_queue.cc:1048] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:03.558058 23020 raft_consensus.cc:2393] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 5: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:03.552443 23624 rpcz_store.cc:276] 0709 13:09:00.302508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:00.402402 (+ 99894us) service_pool.cc:224] Handling call
0709 13:09:02.100372 (+1697970us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:02.101034 (+   662us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:03.551393 (+1450359us) spinlock_profiling.cc:243] Waited 1.69 s on lock 0x7b540049a2d0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:03.551699 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3882344832}
I20260709 13:09:03.565891 25883 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:03.574602 25777 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:03.545804 25920 raft_consensus.cc:2749] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:09:03.468091 22843 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:03.492591 25572 raft_consensus.cc:2804] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:09:03.579787 25797 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:03.582657 25910 raft_consensus.cc:3055] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:09:03.583771 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 136210us
I20260709 13:09:03.584194 25811 raft_consensus.cc:2804] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:09:03.584684 23616 raft_consensus.cc:1240] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 7. Current term is 8. Ops: []
I20260709 13:09:03.519110 22776 catalog_manager.cc:5697] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 5 to 6, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 6 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:03.543067 23022 rpcz_store.cc:276] 0709 13:08:59.277700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:59.277941 (+   241us) service_pool.cc:224] Handling call
0709 13:08:59.296929 (+ 18988us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:01.803634 (+2506705us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:03.473123 (+1669489us) spinlock_profiling.cc:243] Waited 2.5 s on lock 0x7b54000bc2d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:03.473455 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5759760256}
I20260709 13:09:03.589178 25881 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:03.612336 25881 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:03.601660 25910 raft_consensus.cc:740] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:03.618175 25910 consensus_queue.cc:260] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:03.620069 25910 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:03.588366 25418 raft_consensus.cc:2749] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:09:03.623468 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:03.583787 25880 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:03.622962 25832 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:03.587910 25440 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:03.620894 25810 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:03.625087 25754 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:03.631543 25572 raft_consensus.cc:697] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 8 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:03.638813 25811 raft_consensus.cc:697] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
W20260709 13:09:03.677102 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 222902us
I20260709 13:09:03.649550 25883 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:03.669889 25595 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:03.694952 25914 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:03.695798 25914 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:03.698473 25914 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:03.700853 22995 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.006s	sys 0.009s
W20260709 13:09:03.701352 22995 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.006s	sys 0.009s
W20260709 13:09:03.702304 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:03.702983 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:03.703891 25832 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:03.705269 25800 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:03.706045 25800 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:03.706911 25832 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:03.702785 22920 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:09:03.709229 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 241347us
I20260709 13:09:03.713636 25840 raft_consensus.cc:2804] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:09:03.714110 25840 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:03.714486 25840 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:03.708364 25800 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:03.730134 22949 raft_consensus.cc:2410] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
W20260709 13:09:03.743135 23029 consensus_meta.cc:287] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.657s	user 0.007s	sys 0.012s
I20260709 13:09:03.743386 25602 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:03.743644 23029 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 7.
I20260709 13:09:03.744143 25637 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:03.744906 25637 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:03.769080 25810 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:03.756901 25926 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:03.776314 25754 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:03.768752 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 66352.4us
I20260709 13:09:03.783273 25926 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:03.783658 25754 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:03.786525 25926 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:03.757014 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:03.787709 22871 raft_consensus.cc:3060] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:09:03.772073 22993 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.005s	sys 0.024s
W20260709 13:09:03.789132 22993 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.006s	sys 0.024s
W20260709 13:09:03.741025 25661 consensus_meta.cc:287] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.701s	user 0.001s	sys 0.014s
I20260709 13:09:03.766597 25592 consensus_queue.cc:1048] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:09:03.752550 25829 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:03.790520 25912 consensus_queue.cc:1059] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:03.790683 25829 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:03.753652 25664 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:03.773502 25914 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:03.792654 25914 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:03.736925 25595 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:03.754910 25929 raft_consensus.cc:2749] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:09:03.757418 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:03.797266 25595 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:03.724718 25880 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:03.797487 22876 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:09:03.798185 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:03.766959 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:03.802997 22880 raft_consensus.cc:1275] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 13:09:03.806238 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 329185us
I20260709 13:09:03.755985 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:03.818459 25912 consensus_queue.cc:1243] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer 0a407504ed1741bf89a276558adf4c0f log is divergent from this leader: its last log entry 5.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:09:03.757160 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:03.766686 22918 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:03.817967 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 44893.1us
I20260709 13:09:03.756510 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:03.822012 22874 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
I20260709 13:09:03.776381 25832 consensus_queue.cc:1048] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 13:09:03.828872 25661 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:03.829545 25661 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:03.831895 25661 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:09:03.754273 25593 consensus_meta.cc:287] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.752s	user 0.006s	sys 0.013s
I20260709 13:09:03.833663 22992 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:03.833698 22953 raft_consensus.cc:1275] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 13:09:03.834385 25899 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.511s	user 0.005s	sys 0.008s
I20260709 13:09:03.803586 25754 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:03.834751 23025 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
I20260709 13:09:03.833930 22872 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
I20260709 13:09:03.753387 25811 consensus_queue.cc:237] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:03.836854 25898 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:03.837392 25898 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:03.838841 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:03.769172 25572 consensus_queue.cc:237] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:03.841152 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:03.841780 22876 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
W20260709 13:09:03.842474 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 96512.4us
I20260709 13:09:03.843502 25796 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:03.843768 25593 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:03.776624 25809 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:03.844183 25796 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:03.844606 25593 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:03.845153 25809 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:03.845489 22992 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:03.846606 25926 raft_consensus.cc:2804] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:09:03.847083 25926 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:03.847292 25809 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:03.847533 25926 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:03.847704 25777 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:03.848507 25777 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:03.849833 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 83278.2us
I20260709 13:09:03.806252 25637 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:03.805686 25812 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:03.832437 25661 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:03.847211 25593 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:03.806578 25881 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:03.847738 25837 raft_consensus.cc:2749] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:09:03.864634 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:09:03.901459 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 59274.3us
I20260709 13:09:03.903270 22871 raft_consensus.cc:2468] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
I20260709 13:09:03.865566 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:03.864156 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:03.905246 25812 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:03.840751 25883 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:03.901842 22918 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:03.834790 25899 raft_consensus.cc:697] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:03.846554 25796 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:03.920418 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 68860.1us
I20260709 13:09:03.864451 25922 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:03.864419 25777 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:03.865622 25754 raft_consensus.cc:2749] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:09:03.938102 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:03.948758 23619 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
I20260709 13:09:03.951308 25440 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:03.951926 25440 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:04.048870 23621 raft_consensus.cc:2393] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 4: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
W20260709 13:09:03.954224 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 54741.6us
I20260709 13:09:04.065711 22876 raft_consensus.cc:2393] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 7: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
W20260709 13:09:03.952427 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 93360.6us
I20260709 13:09:04.067085 22992 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:04.067626 22918 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:04.069267 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:04.069236 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:04.072863 22872 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:09:03.954032 25663 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:03.947978 25570 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.410s	user 0.017s	sys 0.010s
I20260709 13:09:04.083511 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:04.084389 22871 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:09:04.085203 25876 consensus_queue.cc:1048] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.131s
I20260709 13:09:04.086813 25440 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:04.082418 25570 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.609s	user 0.024s	sys 0.012s
I20260709 13:09:04.089555 23625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 643) took 2907 ms. Trace:
I20260709 13:09:04.089401 25570 raft_consensus.cc:3055] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:09:04.091821 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:04.089632 25794 consensus_queue.cc:1059] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 13:09:03.952744 25661 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:04.105710 25661 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:04.091254 25784 raft_consensus.cc:2749] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:09:04.106681 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 160119us
I20260709 13:09:04.107609 23622 raft_consensus.cc:1275] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 13:09:04.107723 23028 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:04.060612 25910 raft_consensus.cc:2749] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:09:04.110415 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 51407.9us
W20260709 13:09:04.113109 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:04.108093 23622 pending_rounds.cc:85] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2: Aborting all ops after (but not including) 8
I20260709 13:09:04.117044 25663 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:04.119410 25663 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:04.114487 23622 pending_rounds.cc:107] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:09:04.124928 23622 raft_consensus.cc:2889] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:09:04.125636 25931 log.cc:927] Time spent T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.197s	user 0.000s	sys 0.000s
I20260709 13:09:04.128258 22843 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:04.128325 25899 consensus_queue.cc:237] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:04.129519 25637 raft_consensus.cc:2804] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:09:04.130085 25637 raft_consensus.cc:697] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 7 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:04.131111 25637 consensus_queue.cc:237] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:04.150450 25370 raft_consensus.cc:2749] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:04.151532 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:04.152182 22874 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:09:04.152726 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:04.153339 22950 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 8.
I20260709 13:09:04.154404 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:04.154508 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:04.154783 25914 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:04.155001 22952 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 6.
I20260709 13:09:04.155118 22950 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
I20260709 13:09:04.154806 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:04.155565 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 71579.6us
W20260709 13:09:04.157317 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:04.157408 25913 raft_consensus.cc:3055] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:09:04.157883 25913 raft_consensus.cc:740] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:04.159032 25913 consensus_queue.cc:260] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:04.160854 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 53286.9us
I20260709 13:09:04.161005 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:04.161154 25799 consensus_queue.cc:1048] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:09:04.161593 23619 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 10.
I20260709 13:09:04.162488 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:04.162710 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:09:04.164028 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 140346us
I20260709 13:09:04.165127 22949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52222 (request call id 434) took 1851 ms. Trace:
I20260709 13:09:04.165563 22949 rpcz_store.cc:276] 0709 13:09:02.317330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:03.389871 (+1072541us) service_pool.cc:224] Handling call
0709 13:09:04.164898 (+775027us) spinlock_profiling.cc:243] Waited 329 ms on lock 0x7b540023d7d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:04.165041 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":757210368}
I20260709 13:09:04.167201 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:04.167846 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:04.169075 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:04.170101 23022 raft_consensus.cc:2393] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 8: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:04.170790 23617 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 9.
I20260709 13:09:04.171448 25880 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:04.171517 25791 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:04.172298 25791 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:04.173246 25837 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:04.173938 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:04.173862 25837 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:04.175480 25898 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:04.160358 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:04.100000 25570 raft_consensus.cc:740] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:04.071862 25883 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:04.072062 25922 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:04.160315 25913 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:09:04.342518 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 228455us
W20260709 13:09:04.377410 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:04.531136 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:04.176749 22776 catalog_manager.cc:5697] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 4 to 6, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 6 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
W20260709 13:09:04.553619 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:04.179210 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:04.351187 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:04.579311 22874 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:09:04.178077 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:04.219612 23028 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 8.
I20260709 13:09:04.177650 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:04.570379 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:09:04.507474 22875 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.032s	sys 0.047s
W20260709 13:09:04.580914 22875 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.715s	user 0.037s	sys 0.052s
I20260709 13:09:04.298038 23588 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:04.090060 23625 rpcz_store.cc:276] 0709 13:09:01.141939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:01.142550 (+   611us) service_pool.cc:224] Handling call
0709 13:09:01.142902 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:01.149962 (+  7060us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:09:01.200859 (+ 50897us) log.cc:844] Serialized 14 byte log entry
0709 13:09:01.497955 (+297096us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:01.498036 (+    81us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:01.498103 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:03.917022 (+2418919us) spinlock_profiling.cc:243] Waited 6.84 ms on lock 0x7b54003d9ad4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:03.927116 (+ 10094us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:04.049185 (+122069us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":207551,"prepare.queue_time_us":73524,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":1270,"raft.queue_time_us":229354,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":102,"spinlock_wait_cycles":15726464,"thread_start_us":81119,"threads_started":1,"wal-append.queue_time_us":381364,"wal-append.run_cpu_time_us":10549,"wal-append.run_wall_time_us":1395712}
I20260709 13:09:04.582836 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:04.388653 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:04.584097 23622 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:04.303710 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:04.184029 22946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 425) took 1903 ms. Trace:
I20260709 13:09:04.574517 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:04.590752 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:04.420154 25840 consensus_meta.cc:287] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.705s	user 0.014s	sys 0.000s
I20260709 13:09:04.179911 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:04.587193 22947 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 8.
I20260709 13:09:04.590752 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:04.591820 22951 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 7.
I20260709 13:09:04.594260 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:04.596434 22874 raft_consensus.cc:2393] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 8: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:04.597491 25707 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:04.597783 23025 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33418 (request call id 1623) took 1734 ms (client timeout 1775 ms). Trace:
W20260709 13:09:04.598269 23025 rpcz_store.cc:269] 0709 13:09:02.863704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:02.866049 (+  2345us) service_pool.cc:224] Handling call
0709 13:09:04.597510 (+1731461us) spinlock_profiling.cc:243] Waited 966 ms on lock 0x7b540010c554. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:04.597692 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2221202688}
I20260709 13:09:04.598326 25707 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:04.599534 23023 raft_consensus.cc:1240] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:09:04.186537 25937 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:04.600792 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:04.601101 25937 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:04.602885 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:04.603325 25937 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:04.597506 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:04.186226 25914 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:04.605105 22876 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:04.605228 25914 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:04.607692 25914 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:04.610642 22872 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
I20260709 13:09:04.266961 25810 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:04.268754 25880 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:04.099031 25796 consensus_queue.cc:1048] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.487s
I20260709 13:09:04.185897 25572 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:04.586376 22993 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:04.629043 23022 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 7.
W20260709 13:09:04.629674 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 562071us
I20260709 13:09:04.268554 25832 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:04.629992 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:04.185508 25923 raft_consensus.cc:2804] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:04.497301 25812 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:04.630820 23026 raft_consensus.cc:2410] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:09:04.257838 25926 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:04.302009 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:04.302282 25919 consensus_queue.cc:1059] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.335s
I20260709 13:09:04.639122 22873 raft_consensus.cc:2393] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 7: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:04.179836 25829 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:04.636039 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 474417us
I20260709 13:09:04.267107 25800 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:04.639257 23617 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 646) took 3403 ms (client timeout 1947 ms). Trace:
I20260709 13:09:04.641597 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:04.642930 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:04.368638 25837 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:04.448413 25791 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:09:04.657882 23617 rpcz_store.cc:269] 0709 13:09:01.223949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:01.234768 (+ 10819us) service_pool.cc:224] Handling call
0709 13:09:04.533778 (+3299010us) spinlock_profiling.cc:243] Waited 2.69 s on lock 0x7b5400650ed4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:04.627212 (+ 93434us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6191856000}
I20260709 13:09:04.644949 25923 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:04.586294 22946 rpcz_store.cc:276] 0709 13:09:02.280164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:02.280323 (+   159us) service_pool.cc:224] Handling call
0709 13:09:02.817332 (+537009us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:03.165813 (+348481us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:09:03.710726 (+544913us) log.cc:844] Serialized 14 byte log entry
0709 13:09:03.948878 (+238152us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:04.076894 (+128016us) log.cc:844] Serialized 26 byte log entry
0709 13:09:04.077096 (+   202us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:04.077143 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:04.102493 (+ 25350us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:04.183634 (+ 81141us) raft_consensus.cc:1685] finished
0709 13:09:04.183659 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:04.183831 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41567,"prepare.queue_time_us":538622,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":59863,"raft.run_cpu_time_us":374,"raft.run_wall_time_us":139765,"thread_start_us":697796,"threads_started":3,"wal-append.queue_time_us":439791}
I20260709 13:09:04.649084 25832 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:04.655566 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:04.655973 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 78047.1us
I20260709 13:09:04.656258 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:04.605388 25707 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:04.661430 23024 raft_consensus.cc:1240] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 6. Current term is 7. Ops: []
I20260709 13:09:04.663172 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:04.626801 25840 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:04.666724 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:04.667068 23026 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:04.671111 22873 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
I20260709 13:09:04.638557 25572 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:04.654325 22879 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
W20260709 13:09:04.674659 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:04.660246 25883 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:04.679893 23029 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 8.
W20260709 13:09:04.680562 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:04.677013 22992 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:04.664634 25923 raft_consensus.cc:3060] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:04.669420 25794 raft_consensus.cc:2804] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:04.669029 25912 consensus_queue.cc:1048] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 13:09:04.692498 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:04.695202 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:04.697057 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:04.691767 25707 raft_consensus.cc:2804] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:04.698064 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:09:04.700489 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:04.703933 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:04.665019 25880 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:04.701663 22879 raft_consensus.cc:2376] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 8. Re-sending same reply.
I20260709 13:09:04.708781 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:04.712414 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:04.714704 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:04.715132 23618 raft_consensus.cc:1275] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:09:04.715372 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:04.699976 25800 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:04.716003 23029 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
I20260709 13:09:04.716626 25708 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:04.716871 22992 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:04.717211 25708 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:04.708077 25570 consensus_queue.cc:260] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:04.717962 25800 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:04.718636 25570 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:04.719255 25708 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:04.719658 25832 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:04.720464 22874 raft_consensus.cc:2376] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6. Re-sending same reply.
I20260709 13:09:04.720872 25899 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:04.725966 25899 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:04.728789 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:04.729436 22947 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:04.730253 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:04.731420 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:04.733572 22954 raft_consensus.cc:1240] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:04.734709 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:04.735586 22946 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 9.
I20260709 13:09:04.694060 25876 consensus_queue.cc:1059] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:09:04.738168 25661 raft_consensus.cc:3055] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:09:04.738749 25661 raft_consensus.cc:740] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:04.739866 25781 raft_consensus.cc:2749] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:09:04.739674 25661 consensus_queue.cc:260] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:04.738224 25572 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:04.741056 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:04.715840 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:04.732462 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:04.723165 22992 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:04.752425 25880 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:04.753687 25926 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:04.754055 23028 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32794 (request call id 490) took 2398 ms (client timeout 1636 ms). Trace:
I20260709 13:09:04.709431 23625 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
I20260709 13:09:04.742686 22879 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 5.
I20260709 13:09:04.728394 25899 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:04.724840 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:04.755493 22871 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:09:04.694653 25794 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:04.759232 23622 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 10.
W20260709 13:09:04.754474 23028 rpcz_store.cc:269] 0709 13:09:02.355996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:02.363367 (+  7371us) service_pool.cc:224] Handling call
0709 13:09:04.753789 (+2390422us) spinlock_profiling.cc:243] Waited 1.67 s on lock 0x7b5400249854. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:04.753934 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3841113344}
I20260709 13:09:04.654575 25810 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:04.783130 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 623231us
I20260709 13:09:04.701097 25707 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:04.790850 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:04.794612 23617 raft_consensus.cc:2468] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
I20260709 13:09:04.791355 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:04.793176 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:04.799341 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:04.724347 25873 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:04.806367 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:04.792796 25707 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:04.783490 25794 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:04.813016 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:04.821540 23028 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 8.
W20260709 13:09:04.791963 25950 log.cc:927] Time spent T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 13:09:04.811053 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:04.828393 22876 raft_consensus.cc:2410] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:09:04.829809 22873 raft_consensus.cc:2410] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:09:04.829591 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:04.831182 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:04.830683 22880 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:09:04.832929 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:04.807672 25873 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:04.835952 25943 log.cc:927] Time spent T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.359s	user 0.002s	sys 0.002s
I20260709 13:09:04.836089 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:04.836277 22919 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:04.838239 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:04.840523 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:04.836588 25910 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:04.841964 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:04.842630 22946 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:36566
I20260709 13:09:04.855638 22946 raft_consensus.cc:606] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:04.843359 22947 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 8.
I20260709 13:09:04.832350 25922 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:04.861132 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:04.862320 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:04.792078 25899 consensus_queue.cc:1048] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:09:04.863305 22874 raft_consensus.cc:2410] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:09:04.890380 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:04.891096 22872 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
I20260709 13:09:04.896034 23026 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 6.
W20260709 13:09:04.863691 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 45962.5us
I20260709 13:09:04.864485 22880 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 7.
I20260709 13:09:04.829404 25797 raft_consensus.cc:3055] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:09:04.865165 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:04.902850 22879 raft_consensus.cc:2393] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 10: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:04.866845 22948 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 7.
I20260709 13:09:04.867312 25910 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:04.906620 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:04.907503 22872 raft_consensus.cc:2410] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:09:04.869115 22992 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:04.868867 22954 consensus_queue.cc:237] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:09:04.869923 25874 raft_consensus.cc:2749] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:09:04.829430 25812 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:04.877815 25593 consensus_queue.cc:1059] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:04.921989 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:04.833925 25914 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:04.886476 23024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 1669) took 1446 ms. Trace:
I20260709 13:09:04.810618 25370 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:04.826359 25661 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:04.887516 23590 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:04.902994 22873 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6.
I20260709 13:09:04.897930 23624 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
I20260709 13:09:04.854652 25873 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:04.905512 22875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2187) took 1103 ms. Trace:
I20260709 13:09:04.909763 22876 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:09:04.911924 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:04.912607 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:04.914165 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:04.925066 23024 rpcz_store.cc:276] 0709 13:09:03.440617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:03.467370 (+ 26753us) service_pool.cc:224] Handling call
0709 13:09:03.467948 (+   578us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:04.666745 (+1198797us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:04.886161 (+219416us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b540010b654. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:04.886402 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2740439424}
I20260709 13:09:04.932138 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:04.902756 25797 raft_consensus.cc:740] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
W20260709 13:09:04.938313 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 50932.4us
I20260709 13:09:04.941756 25370 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:04.943501 25707 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:04.943960 25370 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:04.944633 25874 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:04.944811 25883 raft_consensus.cc:2804] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:09:04.945397 25883 raft_consensus.cc:697] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 8 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:04.945243 25874 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:04.946332 22873 raft_consensus.cc:1240] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 9 LEADER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:04.946408 25883 consensus_queue.cc:237] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:04.947925 25874 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:04.949927 25707 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:04.952322 25707 raft_consensus.cc:2804] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:09:04.952688 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:04.952956 25707 raft_consensus.cc:697] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 6 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:04.953343 22953 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:04.954107 25707 consensus_queue.cc:237] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:04.956521 22877 raft_consensus.cc:1240] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 6. Current term is 7. Ops: []
I20260709 13:09:04.957050 22880 raft_consensus.cc:1240] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 6. Current term is 7. Ops: []
I20260709 13:09:04.911000 23616 raft_consensus.cc:1275] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:09:04.962188 22872 raft_consensus.cc:1240] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 5. Current term is 6. Ops: []
W20260709 13:09:04.967152 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 98124us
I20260709 13:09:04.971808 23588 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:04.940045 25910 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:04.964927 25593 raft_consensus.cc:3055] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:09:04.940378 22875 rpcz_store.cc:276] 0709 13:09:03.801739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:03.857008 (+ 55269us) service_pool.cc:224] Handling call
0709 13:09:03.864675 (+  7667us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:03.865052 (+   377us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:09:04.585413 (+720361us) log.cc:844] Serialized 14 byte log entry
0709 13:09:04.904181 (+318768us) log.cc:844] Serialized 14 byte log entry
0709 13:09:04.904499 (+   318us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:04.904540 (+    41us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:09:04.904610 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:04.905365 (+   755us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:04.905472 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7928,"prepare.queue_time_us":290594,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":27,"raft.queue_time_us":16859,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":19419,"spinlock_wait_cycles":34874496,"thread_start_us":1006697,"threads_started":2}
I20260709 13:09:04.966420 25840 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:04.991586 22776 catalog_manager.cc:5697] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:04.993912 25781 raft_consensus.cc:2749] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:09:04.995177 22872 raft_consensus.cc:1240] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 5. Current term is 6. Ops: []
I20260709 13:09:04.995853 25873 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:04.996392 22872 raft_consensus.cc:1240] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 8. Current term is 10. Ops: []
I20260709 13:09:04.997517 22872 raft_consensus.cc:1240] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:09:04.998659 22872 raft_consensus.cc:1275] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:09:04.980698 25593 raft_consensus.cc:740] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:05.001856 25593 consensus_queue.cc:260] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:05.003213 25593 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:04.935014 25812 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:05.006449 25812 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:04.938109 25914 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:04.943516 25797 consensus_queue.cc:260] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:05.007858 25797 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:04.993549 22776 catalog_manager.cc:5697] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 3 to 5, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 5 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:05.015422 22953 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 5.
I20260709 13:09:05.016906 22992 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:05.017947 22992 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:05.010533 22776 catalog_manager.cc:5697] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:05.026043 23590 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:05.016697 25923 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:05.046778 22992 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:09:05.051110 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:05.051932 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:05.038633 25781 raft_consensus.cc:2804] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:09:05.057740 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:05.056501 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 127315us
I20260709 13:09:05.058017 22875 raft_consensus.cc:1240] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 6. Current term is 7. Ops: []
I20260709 13:09:05.058542 22876 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 9.
I20260709 13:09:05.058588 23588 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:05.067451 22880 raft_consensus.cc:1240] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:09:05.034895 25794 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:05.068192 22845 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.745s	user 0.009s	sys 0.056s
W20260709 13:09:05.074887 22845 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.751s	user 0.010s	sys 0.056s
I20260709 13:09:05.059623 25781 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:05.076619 22875 raft_consensus.cc:1275] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:09:05.091333 22992 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:05.094970 23020 raft_consensus.cc:2368] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 8. Current term is 9.
I20260709 13:09:05.073664 25791 raft_consensus.cc:2749] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:09:05.074961 25661 raft_consensus.cc:2804] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:09:05.096292 25781 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:09:05.124579 22846 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.006s	sys 0.026s
W20260709 13:09:05.125044 22846 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.006s	sys 0.026s
W20260709 13:09:05.127581 25913 consensus_meta.cc:287] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.541s	user 0.009s	sys 0.002s
I20260709 13:09:05.100072 25661 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:05.150031 25873 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:05.166249 25876 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:05.166392 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:05.169134 22776 catalog_manager.cc:5697] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 5 to 6, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 6 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:05.140127 25661 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:05.176167 22871 raft_consensus.cc:2393] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 5: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:05.177536 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:05.178136 22877 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
I20260709 13:09:05.179936 22880 raft_consensus.cc:1275] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:09:05.180248 22880 pending_rounds.cc:85] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 8
I20260709 13:09:05.186070 22880 pending_rounds.cc:107] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:09:05.186831 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:05.187899 22877 raft_consensus.cc:2393] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 6: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:05.188323 22880 raft_consensus.cc:2889] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:09:05.178748 25637 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.780s	user 0.006s	sys 0.057s
I20260709 13:09:05.175204 25873 raft_consensus.cc:2749] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:09:05.195372 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 452727us
I20260709 13:09:05.142467 25797 consensus_queue.cc:1048] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:09:05.197916 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:05.192555 25637 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.805s	user 0.007s	sys 0.057s
I20260709 13:09:05.191578 25914 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:05.203951 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:05.206240 22875 raft_consensus.cc:2393] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 7: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
W20260709 13:09:05.208966 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 342524us
I20260709 13:09:05.207465 22993 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:09:05.215957 22843 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.684s	user 0.004s	sys 0.035s
W20260709 13:09:05.216379 22843 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.004s	sys 0.035s
I20260709 13:09:05.230085 23619 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 656) took 1061 ms. Trace:
W20260709 13:09:05.253691 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:05.278335 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:05.279877 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:05.280972 22875 raft_consensus.cc:2410] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 11 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 13:09:05.284719 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:05.230600 23619 rpcz_store.cc:276] 0709 13:09:04.158630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:04.163472 (+  4842us) service_pool.cc:224] Handling call
0709 13:09:04.163803 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:04.597475 (+433672us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:09:04.597822 (+   347us) log.cc:844] Serialized 14 byte log entry
0709 13:09:05.037410 (+439588us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:05.037486 (+    76us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:09:05.037547 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:05.220147 (+182600us) spinlock_profiling.cc:243] Waited 430 ms on lock 0x7b5400671454. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:05.220211 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:05.220481 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1066,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":125,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":6044,"spinlock_wait_cycles":988990208,"thread_start_us":438897,"threads_started":1,"wal-append.queue_time_us":604839}
I20260709 13:09:05.290263 22920 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:05.291265 22920 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:05.292101 22843 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:05.283093 25370 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:05.293053 22843 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:05.293066 25370 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:05.295670 25370 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:05.296228 25370 raft_consensus.cc:2804] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:05.296609 25370 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:05.297492 25370 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:09:05.301011 22775 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33478 (request call id 276) took 21887 ms (client timeout 15000 ms). Trace:
I20260709 13:09:05.306842 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:05.307704 22954 raft_consensus.cc:2410] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:09:05.307477 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:05.301563 22775 rpcz_store.cc:269] 0709 13:08:43.419219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:43.432755 (+ 13536us) service_pool.cc:224] Handling call
0709 13:09:05.289517 (+21856762us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:08:54.550789 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:08:54.652891 (+102102us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:08:54.652944 (+    53us) write_op.cc:435] Acquired schema lock
0709 13:08:54.652967 (+    23us) tablet.cc:662] Decoding operations
0709 13:08:55.002376 (+349409us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:08:55.058733 (+ 56357us) write_op.cc:641] Partition lock acquired for write op
0709 13:08:55.101926 (+ 43193us) tablet.cc:685] Acquiring locks for 13 operations
0709 13:08:55.272041 (+170115us) tablet.cc:701] Row locks acquired
0709 13:08:55.272087 (+    46us) write_op.cc:260] PREPARE: finished
0709 13:08:55.272255 (+   168us) write_op.cc:270] Start()
0709 13:08:55.272386 (+   131us) write_op.cc:276] Timestamp: P: 1783602535272194 usec, L: 0
0709 13:08:55.272406 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:08:55.277843 (+  5437us) log.cc:844] Serialized 5090 byte log entry
0709 13:08:56.815096 (+1537253us) op_driver.cc:464] REPLICATION: finished
0709 13:08:57.278409 (+463313us) write_op.cc:301] APPLY: starting
0709 13:08:57.278514 (+   105us) tablet.cc:1367] starting BulkCheckPresence
0709 13:08:57.278879 (+   365us) tablet.cc:1370] finished BulkCheckPresence
0709 13:08:57.279051 (+   172us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:08:57.338511 (+ 59460us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:08:57.338874 (+   363us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=13
0709 13:08:57.338928 (+    54us) write_op.cc:312] APPLY: finished
0709 13:08:57.339311 (+   383us) log.cc:844] Serialized 95 byte log entry
0709 13:08:57.339486 (+   175us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:57.339629 (+   143us) write_op.cc:454] Released schema lock
0709 13:08:57.339678 (+    49us) write_op.cc:341] FINISH: Updating metrics
0709 13:08:57.413093 (+ 73415us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:08:57.413140 (+    47us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":13299,"reported_tablets":34,"child_traces":[["op",{"apply.queue_time_us":441404,"apply.run_cpu_time_us":5040,"apply.run_wall_time_us":139137,"num_ops":13,"prepare.queue_time_us":164327,"prepare.run_cpu_time_us":19128,"prepare.run_wall_time_us":866294,"raft.queue_time_us":396657,"raft.run_cpu_time_us":41631,"raft.run_wall_time_us":322389,"replication_time_us":1517977,"thread_start_us":717076,"threads_started":4,"wal-append.queue_time_us":199874,"wal-append.run_cpu_time_us":41826,"wal-append.run_wall_time_us":2862366}]]}
I20260709 13:09:05.308794 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:05.312944 25964 raft_consensus.cc:2804] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:05.313537 25964 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:05.313992 25964 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:05.314986 25370 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:05.225139 25794 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:05.321151 22953 raft_consensus.cc:2368] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 7. Current term is 8.
I20260709 13:09:05.292101 25874 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:05.322868 25874 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:05.329679 25370 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:05.318011 25910 raft_consensus.cc:2749] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:09:05.340950 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 47051.6us
I20260709 13:09:05.271643 25922 raft_consensus.cc:2804] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:09:05.233628 25963 raft_consensus.cc:2804] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:09:05.349604 22878 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.930s	user 0.013s	sys 0.047s
W20260709 13:09:05.364766 22878 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.966s	user 0.014s	sys 0.048s
W20260709 13:09:05.364904 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 528668us
I20260709 13:09:05.367286 22878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2183) took 1567 ms. Trace:
I20260709 13:09:05.374090 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:05.375814 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:05.354300 25963 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:05.393441 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:05.368098 22878 rpcz_store.cc:276] 0709 13:09:03.800222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:03.811481 (+ 11259us) service_pool.cc:224] Handling call
0709 13:09:03.864308 (+ 52827us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:03.864631 (+   323us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:09:04.019243 (+154612us) log.cc:844] Serialized 14 byte log entry
0709 13:09:05.366680 (+1347437us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:05.366761 (+    81us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:05.366831 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:05.366916 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:05.367073 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":286404,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":28,"raft.queue_time_us":206024,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":27266,"thread_start_us":1358872,"threads_started":3,"wal-append.queue_time_us":744331}
I20260709 13:09:05.365620 25874 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:05.365954 25572 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:05.367631 25840 raft_consensus.cc:2804] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:09:05.372009 25910 consensus_queue.cc:1048] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:09:05.405274 25910 consensus_queue.cc:1048] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:09:05.409430 22843 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:05.367781 25957 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:05.427598 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 314093us
I20260709 13:09:05.390153 25914 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:05.390813 25963 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:05.426338 25595 raft_consensus.cc:2749] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:09:05.431330 25840 raft_consensus.cc:697] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 10 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:05.453032 23588 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:05.450713 25926 raft_consensus.cc:3055] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:09:05.457309 25926 raft_consensus.cc:740] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:09:05.463524 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:05.463331 22843 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:05.445376 25962 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:05.441910 25873 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:05.433439 25874 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:05.462306 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 251610us
W20260709 13:09:05.469148 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 178016us
I20260709 13:09:05.468425 23588 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:05.425437 25572 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:05.471372 22843 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:05.475841 22920 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:05.476145 23588 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:05.482757 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:05.483034 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:05.428757 25957 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:05.488786 22993 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:05.469311 25962 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:05.489768 25707 raft_consensus.cc:2804] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:09:05.490175 25707 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:05.471621 25637 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:05.490859 25707 raft_consensus.cc:3060] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:05.490939 25637 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:05.491339 25962 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:05.491456 25637 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:09:05.493381 25971 log.cc:927] Time spent T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 13:09:05.499814 25599 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:05.483595 23020 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 9.
I20260709 13:09:05.490178 22993 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:05.500924 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:05.502979 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:05.451529 25919 consensus_queue.cc:1059] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 13:09:05.505369 25876 raft_consensus.cc:2804] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:05.505692 25919 raft_consensus.cc:2804] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:09:05.505893 25876 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:05.506307 25919 raft_consensus.cc:697] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 8 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:05.477497 25797 raft_consensus.cc:2804] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:05.506371 25876 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:05.507431 25919 consensus_queue.cc:237] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:05.507051 25797 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:05.486600 25874 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:05.509773 22843 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:05.514724 25978 raft_consensus.cc:3055] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:09:05.515208 25978 raft_consensus.cc:740] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
W20260709 13:09:05.515751 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:05.516266 25978 consensus_queue.cc:260] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:05.517838 25978 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:05.518061 25913 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:05.518585 25913 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:05.520610 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:05.521318 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:05.522184 22947 raft_consensus.cc:2410] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 13:09:05.525900 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 277083us
I20260709 13:09:05.469362 25873 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:05.530243 23026 raft_consensus.cc:1240] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:09:05.509161 25883 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:05.532732 22952 raft_consensus.cc:1240] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:05.534132 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:05.513913 25797 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:05.511081 25968 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:05.513303 25570 raft_consensus.cc:2804] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:09:05.535755 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:05.540859 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:05.533367 22843 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:05.541904 23029 raft_consensus.cc:1275] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
W20260709 13:09:05.542202 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 152922us
I20260709 13:09:05.544751 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:05.537926 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:05.528354 25970 raft_consensus.cc:2804] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:09:05.538740 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:05.557603 25970 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:05.539309 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:05.525717 25810 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:05.550489 22947 raft_consensus.cc:1275] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 13:09:05.500448 25599 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:05.550629 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:05.550733 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:05.551205 25968 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:05.541363 25570 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:05.552269 25840 consensus_queue.cc:237] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:05.539821 23023 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 9.
I20260709 13:09:05.554366 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:05.570106 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:05.570235 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:05.555568 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:05.555517 25873 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:05.555754 25370 raft_consensus.cc:2804] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:09:05.556084 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:05.584470 25370 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:05.584703 23026 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
I20260709 13:09:05.584949 25370 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:05.585606 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:05.556840 25913 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:05.556905 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:05.545462 23027 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:09:05.557215 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:09:05.567157 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 45740.6us
I20260709 13:09:05.582222 22951 raft_consensus.cc:2393] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 5: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:05.557927 22949 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
W20260709 13:09:05.566039 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 90255.5us
I20260709 13:09:05.572768 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:05.588304 22843 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:05.589262 23027 raft_consensus.cc:1240] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:05.589727 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:05.589967 22843 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:05.590116 22948 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:05.590440 22947 raft_consensus.cc:2393] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 5: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:05.591003 22843 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:05.591153 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:05.591727 22947 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
I20260709 13:09:05.591909 25810 raft_consensus.cc:2804] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:05.592278 25810 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:05.592599 25810 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:05.592501 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:05.573740 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:05.593432 22947 raft_consensus.cc:3060] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:05.593825 25796 raft_consensus.cc:2804] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:05.594303 25796 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:05.594671 25796 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:05.596994 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:05.597705 22949 raft_consensus.cc:2393] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 10: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:05.598511 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:05.599851 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:05.575389 25964 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:05.576192 25959 log.cc:927] Time spent T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.375s	user 0.001s	sys 0.004s
I20260709 13:09:05.576222 25960 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:05.557991 25970 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:05.578756 25923 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:05.579754 22992 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:05.567363 22878 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
I20260709 13:09:05.569682 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:05.583191 22844 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:05.617650 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:05.602741 22920 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:05.571261 25570 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:05.593817 23023 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
I20260709 13:09:05.607020 22947 raft_consensus.cc:2468] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
I20260709 13:09:05.583983 25874 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:05.618593 22946 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:05.619443 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:05.619392 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:05.621003 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:05.629876 23023 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 5.
I20260709 13:09:05.621217 23020 raft_consensus.cc:2393] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 10: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:05.621726 23027 raft_consensus.cc:1240] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:09:05.622205 22948 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
I20260709 13:09:05.605563 25957 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:05.626317 25661 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:05.627259 22949 raft_consensus.cc:1240] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 7. Current term is 8. Ops: [5.7-7.8]
W20260709 13:09:05.636438 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:05.626024 25637 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:05.633710 22952 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6.
I20260709 13:09:05.613341 25960 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:05.634296 23029 raft_consensus.cc:2393] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 9: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:05.634328 22951 raft_consensus.cc:1275] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 13:09:05.630587 23588 connection.cc:256] RPC call timeout handler was delayed by -1.6184313559999737s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1965s for a response
W20260709 13:09:05.642198 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 62511.9us
W20260709 13:09:05.643885 25987 log.cc:927] Time spent T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:09:05.628701 25595 raft_consensus.cc:2804] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:09:05.632622 25593 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:05.648303 25796 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:05.638767 25829 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:05.668213 25829 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:05.669094 25964 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:05.670460 25829 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:05.672047 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:05.675644 25370 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:05.675938 25926 consensus_queue.cc:260] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:05.641824 25874 raft_consensus.cc:2749] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:09:05.676492 22992 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:05.649536 25960 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:05.690559 22950 raft_consensus.cc:2393] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 6: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:05.656876 25957 raft_consensus.cc:2804] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:05.697600 25923 raft_consensus.cc:2804] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:09:05.702137 25876 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:05.703359 23026 raft_consensus.cc:2368] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 8. Current term is 9.
I20260709 13:09:05.704720 25926 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:05.706238 25922 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:05.706715 25922 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:05.708555 25960 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:05.708972 25572 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:05.709761 25572 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:05.709976 25661 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:05.710199 25572 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:05.712770 23028 raft_consensus.cc:2410] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 13:09:05.713436 25963 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:05.698149 25923 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
W20260709 13:09:05.726456 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:05.699055 25957 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:05.689386 25883 consensus_queue.cc:1059] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:09:06.015471 23026 raft_consensus.cc:1240] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 8. Current term is 9. Ops: []
W20260709 13:09:05.773684 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:06.025883 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:05.811197 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 239698us
I20260709 13:09:06.215998 23028 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 9.
W20260709 13:09:05.804044 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 201353us
W20260709 13:09:05.767303 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:06.013479 22946 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
W20260709 13:09:05.822327 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 164649us
I20260709 13:09:05.723320 25923 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:09:05.935885 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:06.216286 22844 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:05.936139 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:06.196779 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 500658us
I20260709 13:09:05.933799 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:06.059713 23022 raft_consensus.cc:1240] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:06.221695 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:06.221997 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:06.223067 25874 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
W20260709 13:09:06.191479 23023 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.026s	sys 0.036s
W20260709 13:09:06.253552 23023 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.031s	sys 0.037s
I20260709 13:09:06.255028 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:06.276860 22950 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:09:06.256623 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:06.218108 23588 connection.cc:256] RPC call timeout handler was delayed by -1.4926484420000179s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21960000000000002s for a response
W20260709 13:09:06.060489 25992 log.cc:927] Time spent T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.260s	user 0.001s	sys 0.000s
I20260709 13:09:06.249114 22920 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:06.285511 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:06.294646 22952 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
W20260709 13:09:06.294821 23588 connection.cc:256] RPC call timeout handler was delayed by -1.1648337809999703s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.31170000000000003s for a response
I20260709 13:09:06.295527 23588 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:06.296805 23588 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:05.839787 25964 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:06.299738 25964 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:06.300916 22952 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
I20260709 13:09:06.301914 25964 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:05.729770 25957 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:09:05.738399 25994 log.cc:927] Time spent T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:09:06.287402 22953 raft_consensus.cc:1275] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:09:05.823818 25595 raft_consensus.cc:697] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 5 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:06.020454 25370 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:05.985917 25637 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:05.813372 25781 consensus_meta.cc:287] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.670s	user 0.012s	sys 0.000s
I20260709 13:09:05.712484 25797 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:06.101922 25960 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:05.993678 25796 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:06.026157 25914 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:05.779716 25968 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:05.731900 25599 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:06.291738 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:06.132436 25876 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:06.285113 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 68913.4us
I20260709 13:09:06.225010 25812 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:06.077417 25926 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:06.302842 22953 pending_rounds.cc:85] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting all ops after (but not including) 6
I20260709 13:09:06.369462 22953 pending_rounds.cc:107] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:09:06.369757 22953 raft_consensus.cc:2889] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:09:06.063697 25707 consensus_meta.cc:287] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.572s	user 0.009s	sys 0.000s
I20260709 13:09:06.222918 25910 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:06.302915 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:06.223630 25874 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:05.779565 25570 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:06.289819 25572 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:06.318971 22947 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.596s	user 0.028s	sys 0.018s
I20260709 13:09:06.071148 25810 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:06.106515 25922 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:05.731045 25970 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:06.327684 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:06.304759 25840 raft_consensus.cc:2749] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:09:06.291414 25883 consensus_queue.cc:1059] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:09:06.533990 23029 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
I20260709 13:09:06.313256 25963 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:06.314287 25791 raft_consensus.cc:2749] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:09:06.320307 25914 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:09:06.534533 22947 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.812s	user 0.042s	sys 0.026s
I20260709 13:09:06.365936 22844 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:06.362735 25919 raft_consensus.cc:2749] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:06.543946 23022 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:09:06.545786 23025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 1680) took 1002 ms. Trace:
W20260709 13:09:06.549669 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 183885us
I20260709 13:09:06.550613 25970 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:06.550833 25572 raft_consensus.cc:2749] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:09:06.551096 25978 raft_consensus.cc:3055] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:09:06.551573 25978 raft_consensus.cc:740] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:06.551617 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:06.552445 23029 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:09:06.536890 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:06.548859 23025 rpcz_store.cc:276] 0709 13:09:05.543555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:05.543743 (+   188us) service_pool.cc:224] Handling call
0709 13:09:05.559106 (+ 15363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:05.569369 (+ 10263us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:05.569404 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:05.572786 (+  3382us) log.cc:844] Serialized 26 byte log entry
0709 13:09:05.621747 (+ 48961us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:09:05.621804 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:05.621891 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:06.545494 (+923603us) raft_consensus.cc:1685] finished
0709 13:09:06.545524 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:06.545711 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1752,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":3275,"thread_start_us":47876,"threads_started":1,"wal-append.queue_time_us":123606}
W20260709 13:09:06.553381 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:06.554075 25978 consensus_queue.cc:260] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:06.319957 25781 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:06.546430 25593 consensus_meta.cc:287] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.882s	user 0.012s	sys 0.000s
I20260709 13:09:06.367373 25968 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:06.545560 25876 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:06.541985 25599 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:06.570703 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:06.555339 22992 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:06.555918 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:06.558987 22950 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:09:06.429421 25964 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:06.370205 25370 raft_consensus.cc:2749] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:09:06.589171 23028 raft_consensus.cc:2393] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 9: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:06.563951 25593 raft_consensus.cc:2749] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:09:06.424782 25707 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:06.429941 25926 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:06.607604 23022 raft_consensus.cc:1240] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 5. Current term is 6. Ops: []
I20260709 13:09:06.607302 25964 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:06.609038 25926 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:09:06.609252 26000 log.cc:927] Time spent T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.239s	user 0.000s	sys 0.000s
I20260709 13:09:06.609808 25964 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:06.586609 25978 raft_consensus.cc:3060] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:06.541347 25910 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:06.618263 23029 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:09:06.590075 22992 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:06.614759 23026 raft_consensus.cc:1240] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:09:06.373483 25812 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:06.619581 23021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 1687) took 1020 ms. Trace:
I20260709 13:09:06.600188 25791 raft_consensus.cc:3055] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:09:06.597029 25963 raft_consensus.cc:2804] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:09:06.629360 23024 raft_consensus.cc:2393] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 6: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
W20260709 13:09:06.630648 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 62302.4us
I20260709 13:09:06.598096 25599 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:06.629495 25791 raft_consensus.cc:740] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
W20260709 13:09:06.654192 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:06.562505 25595 consensus_queue.cc:237] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:06.661244 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:06.661743 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:06.617919 25919 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:06.587559 25876 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:06.662534 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 384529us
W20260709 13:09:06.667109 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:06.570302 25968 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:06.673157 22992 leader_election.cc:400] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 7. Current term is 8.
I20260709 13:09:06.673585 22992 leader_election.cc:403] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:09:06.658210 25926 raft_consensus.cc:2804] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:09:06.660861 25963 raft_consensus.cc:697] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:09:06.689711 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 259869us
W20260709 13:09:06.692591 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:06.692006 22918 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.688s (SENT) [suppressed 19 similar messages]
I20260709 13:09:06.697088 22950 raft_consensus.cc:1275] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:09:06.698462 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:06.698962 22950 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:06.704187 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:06.705619 23029 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
W20260709 13:09:06.708758 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:06.709396 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:06.690713 25797 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:06.711634 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:06.712414 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:06.717113 23027 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:06.716387 22953 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
W20260709 13:09:06.730368 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 513099us
I20260709 13:09:06.734464 22920 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:06.736351 25370 raft_consensus.cc:2764] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:09:06.672190 25919 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:06.718838 25810 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:06.735853 22950 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
I20260709 13:09:06.714937 25791 consensus_queue.cc:260] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:06.729606 25570 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:06.739640 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 442842us
I20260709 13:09:06.754525 22920 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:06.740443 23029 raft_consensus.cc:1275] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 13:09:06.771009 22843 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.008s	sys 0.027s
W20260709 13:09:06.782267 22843 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.008s	sys 0.027s
I20260709 13:09:06.746145 23027 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6.
I20260709 13:09:06.780380 23588 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:06.782829 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 550640us
W20260709 13:09:06.745337 26001 log.cc:927] Time spent T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.409s	user 0.001s	sys 0.001s
W20260709 13:09:06.745267 26002 log.cc:927] Time spent T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 13:09:06.807992 23024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32794 (request call id 499) took 1325 ms. Trace:
I20260709 13:09:06.808744 23024 rpcz_store.cc:276] 0709 13:09:05.483049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:05.492369 (+  9320us) service_pool.cc:224] Handling call
0709 13:09:06.807700 (+1315331us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54001c70d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:06.807916 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2593040768}
I20260709 13:09:06.739935 25922 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:09:06.747054 26005 log.cc:927] Time spent T 00000000000000000000000000000000 P 8d175b4ae72f4fc48971c2f31668971c: Append to log took a long time: real 0.402s	user 0.000s	sys 0.002s
W20260709 13:09:06.810757 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 70074.8us
W20260709 13:09:06.811118 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 94319.4us
I20260709 13:09:06.749631 25791 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:06.748159 25968 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:06.748265 25781 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:06.748948 25874 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:06.749564 25812 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:06.750075 25910 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:06.783988 25707 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:06.834466 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 91283.4us
I20260709 13:09:06.835999 22920 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:06.741425 25797 consensus_queue.cc:1048] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 13:09:06.837033 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:06.838936 23624 raft_consensus.cc:1240] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 8 LEADER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 13:09:06.807752 25599 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:06.779686 25923 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:06.844240 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:06.844822 23623 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:06.847411 23619 raft_consensus.cc:1275] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
I20260709 13:09:06.848480 23619 pending_rounds.cc:85] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2: Aborting all ops after (but not including) 5
I20260709 13:09:06.848927 23619 pending_rounds.cc:107] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:09:06.849463 23619 raft_consensus.cc:2889] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:06.822350 26010 raft_consensus.cc:2749] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:09:06.843761 26006 log.cc:927] Time spent T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
I20260709 13:09:06.852155 23020 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
I20260709 13:09:06.840427 25876 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:06.852757 22843 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:06.762046 25963 consensus_queue.cc:237] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:06.839459 25926 raft_consensus.cc:697] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:06.848906 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:06.858069 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:06.858610 23624 raft_consensus.cc:2393] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 10: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
W20260709 13:09:06.855297 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 191892us
I20260709 13:09:06.856393 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:06.840057 26008 raft_consensus.cc:2749] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:09:06.859274 22992 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:06.860466 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:06.865157 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:06.870559 25926 consensus_queue.cc:237] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:06.855830 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:06.860308 25570 raft_consensus.cc:2804] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:06.854534 26015 raft_consensus.cc:2749] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 7. Current term is 8.
W20260709 13:09:06.883921 22843 connection.cc:256] RPC call timeout handler was delayed by -1.0540423700000474s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1879s for a response
I20260709 13:09:06.888401 22992 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:06.889075 23588 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:06.889848 25797 raft_consensus.cc:2749] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:09:06.890661 25840 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:06.891187 25840 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:06.891585 25572 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:06.892168 25572 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:06.893923 25840 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:06.896409 26015 raft_consensus.cc:2804] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:09:06.897007 26015 raft_consensus.cc:697] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 5 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:06.898025 25572 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:06.897889 26015 consensus_queue.cc:237] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:06.899928 25572 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:06.900384 25572 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:06.901759 22992 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:06.891160 22994 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:06.895153 26019 consensus_queue.cc:1048] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:09:06.911372 25707 raft_consensus.cc:2804] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:09:06.912197 26018 consensus_queue.cc:1048] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:09:06.913619 26018 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:06.627857 23021 rpcz_store.cc:276] 0709 13:09:05.599334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:05.607528 (+  8194us) service_pool.cc:224] Handling call
0709 13:09:05.619357 (+ 11829us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:05.619545 (+   188us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:09:06.197093 (+577548us) log.cc:844] Serialized 14 byte log entry
0709 13:09:06.613848 (+416755us) log.cc:844] Serialized 14 byte log entry
0709 13:09:06.619146 (+  5298us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:06.619217 (+    71us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:09:06.619283 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:06.619366 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:06.619514 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":162125,"prepare.queue_time_us":185458,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":153,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":9371,"thread_start_us":496632,"threads_started":2}
I20260709 13:09:06.914032 26018 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:06.894196 25957 consensus_meta.cc:287] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.584s	user 0.004s	sys 0.005s
I20260709 13:09:06.902899 26021 consensus_queue.cc:1048] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:09:06.916985 25919 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:06.918390 25923 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:06.919044 25923 raft_consensus.cc:2804] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:06.919437 25923 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:06.919575 22994 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:06.919862 25923 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:06.921038 22994 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:06.878216 25570 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:06.882130 25876 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:09:06.918119 25957 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:06.902485 25572 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:06.977576 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 88558us
I20260709 13:09:06.915961 25914 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:06.993221 23588 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:06.916101 26018 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:06.987231 22947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 451) took 1356 ms. Trace:
I20260709 13:09:06.995026 22947 rpcz_store.cc:276] 0709 13:09:05.630569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:05.632271 (+  1702us) service_pool.cc:224] Handling call
0709 13:09:05.632564 (+   293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:05.637451 (+  4887us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:05.637487 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:05.657137 (+ 19650us) log.cc:844] Serialized 26 byte log entry
0709 13:09:06.564967 (+907830us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:06.565026 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:06.565112 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:06.987007 (+421895us) raft_consensus.cc:1685] finished
0709 13:09:06.987028 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:06.987175 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2164,"raft.queue_time_us":2275,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":13367,"thread_start_us":824823,"threads_started":1,"wal-append.queue_time_us":911361}
I20260709 13:09:06.990661 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:06.993922 22992 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:06.973721 25840 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:06.999701 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:07.000635 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:07.005832 22953 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:06.981827 25570 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:06.998540 22992 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:07.001822 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:07.009369 22946 raft_consensus.cc:2410] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
W20260709 13:09:07.010483 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 174541us
I20260709 13:09:06.985378 25919 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:07.019316 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:06.994186 25876 raft_consensus.cc:697] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:07.022236 22949 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
I20260709 13:09:06.994518 26016 raft_consensus.cc:2749] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:09:06.999924 26021 raft_consensus.cc:2804] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:09:07.033475 23588 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:07.014834 26018 raft_consensus.cc:2804] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:09:07.020915 25812 raft_consensus.cc:2804] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:07.025838 25797 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:07.027482 25572 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:07.029727 25964 raft_consensus.cc:2804] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:09:07.039919 26007 consensus_queue.cc:1059] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 13:09:07.045147 25876 consensus_queue.cc:237] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:07.046396 25914 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:07.054162 25957 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:07.061059 22948 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:09:07.014011 25840 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:07.076305 23623 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
I20260709 13:09:07.079774 22953 raft_consensus.cc:2468] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
W20260709 13:09:07.083074 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 46725.6us
I20260709 13:09:07.041452 25797 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:07.036577 26021 raft_consensus.cc:697] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 6 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:07.024938 25919 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:07.039616 26008 raft_consensus.cc:2749] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 13:09:07.103232 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:07.035461 25883 consensus_queue.cc:1059] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 13:09:07.104849 26008 raft_consensus.cc:3055] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:09:07.105218 26008 raft_consensus.cc:740] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:07.106020 26008 consensus_queue.cc:260] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:07.107201 25840 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:07.107409 26008 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:07.111318 22992 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:07.112287 22992 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:07.113138 22992 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:07.113927 26019 raft_consensus.cc:2804] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:07.114319 26019 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:07.114727 26019 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:07.114856 25926 raft_consensus.cc:2804] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:09:07.115440 25926 raft_consensus.cc:697] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:07.039985 25812 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:07.117285 25812 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:07.117393 26016 raft_consensus.cc:2804] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:09:07.117911 26016 raft_consensus.cc:697] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 6 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:07.119650 26016 consensus_queue.cc:237] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:07.097563 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 63493.3us
I20260709 13:09:07.035264 25781 consensus_queue.cc:1048] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
W20260709 13:09:07.125572 25999 log.cc:927] Time spent T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.580s	user 0.006s	sys 0.000s
I20260709 13:09:07.126691 26021 consensus_queue.cc:237] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:07.127213 25570 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:07.090279 25964 raft_consensus.cc:697] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:07.127246 23023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 1691) took 1498 ms. Trace:
I20260709 13:09:07.129674 25964 consensus_queue.cc:237] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:07.131052 23622 raft_consensus.cc:1240] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:09:07.131938 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:07.082178 25572 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:07.133190 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:07.133876 23617 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
I20260709 13:09:07.134562 25572 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:07.098449 25797 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:09:07.098315 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 177341us
I20260709 13:09:07.099192 23588 leader_election.cc:400] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 8. Current term is 9.
I20260709 13:09:07.141636 23588 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:07.037032 25874 consensus_queue.cc:1059] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.054s
I20260709 13:09:07.116413 25926 consensus_queue.cc:237] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:07.146587 23020 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33420 (request call id 1694) took 1485 ms (client timeout 1820 ms). Trace:
I20260709 13:09:07.035989 26018 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:07.152979 23588 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:07.129288 23023 rpcz_store.cc:276] 0709 13:09:05.629081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:05.630470 (+  1389us) service_pool.cc:224] Handling call
0709 13:09:05.630747 (+   277us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:05.630951 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:05.630977 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:05.631457 (+   480us) log.cc:844] Serialized 26 byte log entry
0709 13:09:06.254794 (+623337us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:09:06.254839 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:06.254937 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:07.125011 (+870074us) spinlock_profiling.cc:243] Waited 8.71 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:09:07.126844 (+  1833us) raft_consensus.cc:1685] finished
0709 13:09:07.126884 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:07.127179 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20029824,"thread_start_us":622916,"threads_started":1,"wal-append.queue_time_us":675797}
I20260709 13:09:07.140298 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:09:07.149855 23020 rpcz_store.cc:269] 0709 13:09:05.661722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:05.840252 (+178530us) service_pool.cc:224] Handling call
0709 13:09:07.146381 (+1306129us) spinlock_profiling.cc:243] Waited 631 ms on lock 0x7b54001f7cd4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:07.146515 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1451774080}
I20260709 13:09:07.151783 25963 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:07.161222 22994 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:07.166103 23623 raft_consensus.cc:2410] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 11 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:09:07.152840 25791 consensus_queue.cc:1059] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:09:07.150044 25781 raft_consensus.cc:2749] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:07.167230 25883 raft_consensus.cc:3055] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:09:07.169152 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:07.167838 25883 raft_consensus.cc:740] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:07.176007 25572 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:07.176561 25572 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:07.143174 25707 raft_consensus.cc:697] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 10 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:07.178128 25910 raft_consensus.cc:3055] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:09:07.176441 25883 consensus_queue.cc:260] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:07.177980 25707 consensus_queue.cc:237] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:07.145440 25968 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:07.149807 25370 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:07.164757 26018 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:07.146740 23616 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 9.
I20260709 13:09:07.202834 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:07.198629 25874 consensus_queue.cc:1243] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer 0a407504ed1741bf89a276558adf4c0f log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:09:07.203478 25963 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:07.178544 25910 raft_consensus.cc:740] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:07.159029 26008 consensus_queue.cc:1059] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 13:09:07.197980 25593 raft_consensus.cc:2804] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:09:07.265312 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:07.269127 25572 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:07.271454 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 110351us
I20260709 13:09:07.272243 22994 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:09:07.272415 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 119847us
W20260709 13:09:07.273406 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 120496us
I20260709 13:09:07.274997 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:07.275766 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:07.276420 23623 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 7.
W20260709 13:09:07.279531 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:07.280619 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:07.254393 25781 raft_consensus.cc:3060] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:07.178910 26033 consensus_queue.cc:1059] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.125s
I20260709 13:09:07.281813 23623 raft_consensus.cc:2368] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 8. Current term is 9.
I20260709 13:09:07.285769 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:07.290876 23620 raft_consensus.cc:1240] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:09:07.271157 25593 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:07.299645 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:07.213227 25370 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:07.302582 23625 raft_consensus.cc:2393] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 7: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:07.284343 26016 raft_consensus.cc:2804] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:09:07.284204 26008 consensus_queue.cc:1243] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer f3c799b1249e40119d2a5864231c6c75 log is divergent from this leader: its last log entry 7.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:09:07.207793 25968 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:07.303539 22920 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:07.289330 25883 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:07.308976 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:07.305284 25910 consensus_queue.cc:260] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:07.289217 25923 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:07.308146 26016 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:07.410810 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:07.421834 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:07.331936 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:07.423170 22879 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:07.411154 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:07.297228 26010 consensus_queue.cc:1048] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 13:09:07.422425 23618 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 8.
I20260709 13:09:07.421955 22880 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
I20260709 13:09:07.439239 22872 raft_consensus.cc:1275] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:09:07.441303 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:07.442155 22876 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
I20260709 13:09:07.300542 25593 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:07.346820 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:07.403249 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:07.419374 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:07.424115 22875 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:07.405037 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:07.450933 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:07.428066 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:07.431152 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:07.432204 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:07.448930 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
W20260709 13:09:07.451933 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 160573us
I20260709 13:09:07.451960 22953 raft_consensus.cc:2393] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 8: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:07.453392 23619 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 7.
I20260709 13:09:07.454504 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:07.454384 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:07.455732 22871 raft_consensus.cc:2410] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:09:07.455840 22951 raft_consensus.cc:3055] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:09:07.450281 22878 raft_consensus.cc:2376] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 9. Re-sending same reply.
I20260709 13:09:07.443971 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:07.459203 22873 raft_consensus.cc:2410] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 8 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:09:07.333775 25370 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:07.460970 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:07.331637 26007 raft_consensus.cc:2804] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:07.463285 22874 raft_consensus.cc:2393] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 9: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:07.388453 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:07.463732 23624 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 7.
I20260709 13:09:07.464685 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:07.331008 26042 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:07.473198 26042 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:07.473661 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:07.474505 23619 raft_consensus.cc:2410] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:09:07.475543 26042 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:07.478516 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:07.480242 23620 raft_consensus.cc:1275] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:09:07.481932 25968 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:07.456151 22951 raft_consensus.cc:740] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:07.484513 22949 raft_consensus.cc:1275] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:09:07.469868 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:07.429744 26016 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:07.421406 25570 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:07.413268 25963 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:07.489807 22879 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:09:07.491804 22880 raft_consensus.cc:1240] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 6. Current term is 7. Ops: []
I20260709 13:09:07.470628 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:07.471004 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:07.492899 23618 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 7.
I20260709 13:09:07.488416 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:07.489208 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:07.494640 23624 raft_consensus.cc:2393] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 7: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:07.460844 25910 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:09:07.495813 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 186165us
I20260709 13:09:07.465930 26007 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:07.500640 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:07.506827 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:07.574266 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:09:07.491070 26039 log.cc:927] Time spent T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 13:09:07.500140 22994 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:07.548988 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:07.500001 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:07.565667 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:07.490437 25919 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:07.572188 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:07.570035 22951 consensus_queue.cc:260] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:07.580719 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 127388us
I20260709 13:09:07.496168 25957 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:07.586344 26018 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:07.577665 22953 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 7.
I20260709 13:09:07.577917 23624 raft_consensus.cc:2368] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 8. Current term is 9.
I20260709 13:09:07.597631 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:07.598097 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:07.598281 23616 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
I20260709 13:09:07.599123 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:07.599283 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:07.600129 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:07.576982 22949 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 7.
I20260709 13:09:07.579345 22946 raft_consensus.cc:1240] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:09:07.587114 25923 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:07.589092 22951 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:07.590440 22952 raft_consensus.cc:1275] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:09:07.574913 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:09:07.595672 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 95637.3us
I20260709 13:09:07.595201 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:07.596230 25922 raft_consensus.cc:2749] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:09:07.601828 22875 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6.
I20260709 13:09:07.606941 23625 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
I20260709 13:09:07.602298 23620 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
I20260709 13:09:07.622366 26018 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:07.627724 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:07.629383 23588 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:07.574371 26007 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:07.639225 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:07.646859 22844 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:07.649789 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:07.650633 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:07.653127 25791 raft_consensus.cc:3055] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:09:07.653506 25791 raft_consensus.cc:740] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:07.654381 25791 consensus_queue.cc:260] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:07.661453 22946 raft_consensus.cc:1240] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:09:07.661293 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:09:07.667456 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 43741.2us
I20260709 13:09:07.671598 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:07.671893 23624 raft_consensus.cc:3055] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:09:07.672248 22994 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:07.672871 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:07.627954 25957 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:07.649613 26011 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:07.688628 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:07.688711 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:09:07.689301 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:07.692256 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:07.693208 22948 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 7.
I20260709 13:09:07.695887 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:07.676791 23624 raft_consensus.cc:740] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:07.679127 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:07.681229 26015 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:09:07.697124 26015 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:07.697547 26015 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:07.697278 23624 consensus_queue.cc:260] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:07.697818 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:07.682761 22994 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:07.698289 23624 raft_consensus.cc:3060] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:09:07.686806 25812 consensus_meta.cc:287] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.569s	user 0.009s	sys 0.000s
I20260709 13:09:07.696426 22948 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 5.
W20260709 13:09:07.685451 25797 consensus_meta.cc:287] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.550s	user 0.010s	sys 0.000s
W20260709 13:09:07.699510 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:07.646059 26008 consensus_queue.cc:1048] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:09:07.701422 22994 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:07.705574 22919 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:07.704700 26042 consensus_queue.cc:1059] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:09:07.704388 25797 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:07.707288 23617 raft_consensus.cc:1240] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:09:07.707247 22994 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:07.703313 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:07.707578 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:07.708505 22952 raft_consensus.cc:2393] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 10: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
W20260709 13:09:07.702390 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 55642.1us
W20260709 13:09:07.703809 26019 consensus_meta.cc:287] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.589s	user 0.018s	sys 0.000s
I20260709 13:09:07.687649 26011 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:07.679142 25570 raft_consensus.cc:2804] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:07.675972 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:09:07.709656 22919 consensus_peers.cc:597] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Couldn't send request to peer 5b346767a89745fa82928a5dac9cc0b2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7426119f4da44ffaa1513b481a866a01. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:07.734972 22919 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:07.738529 25968 consensus_queue.cc:1059] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:09:07.730559 25874 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:07.739588 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:07.740415 22952 raft_consensus.cc:2393] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 9: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:07.727706 26047 raft_consensus.cc:2804] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:07.741097 26047 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:07.741509 26047 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:07.705394 25812 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:07.727650 26046 raft_consensus.cc:2749] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:07.742586 25957 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:07.743970 25812 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:07.745313 23622 raft_consensus.cc:2410] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:09:07.702602 26018 raft_consensus.cc:2764] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:09:07.739869 25874 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:07.736385 22994 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:07.747246 22919 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:09:07.752590 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 123320us
W20260709 13:09:07.747697 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:07.757773 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 192563us
I20260709 13:09:07.732897 26042 raft_consensus.cc:3055] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:09:07.761963 22919 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:07.744733 25883 raft_consensus.cc:2749] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:07.765280 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:07.766027 25797 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:07.766088 23026 raft_consensus.cc:1240] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:09:07.766534 25964 consensus_queue.cc:1048] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:09:07.766885 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:07.767442 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:07.781086 22948 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:09:07.781402 23024 raft_consensus.cc:1240] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:09:07.745126 25922 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:07.730695 25570 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:07.784849 25570 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:07.768424 22994 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:07.791039 26016 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:07.784945 23616 raft_consensus.cc:2410] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
W20260709 13:09:07.796972 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 43647.5us
I20260709 13:09:07.761400 26045 raft_consensus.cc:2749] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:09:07.797691 23588 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:07.739638 26038 raft_consensus.cc:3055] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:09:07.800230 22994 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:07.800299 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:07.807662 22955 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 7.
W20260709 13:09:07.776611 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 48199.9us
I20260709 13:09:07.796128 25593 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:07.803532 22948 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
I20260709 13:09:07.814119 22844 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:07.817066 22844 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:07.770547 25781 raft_consensus.cc:2749] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:09:07.826268 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:07.773036 26046 raft_consensus.cc:3055] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:09:07.773290 25370 raft_consensus.cc:2804] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:07.767901 25874 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:07.850831 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:07.815080 22993 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:07.851114 22992 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:07.851354 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:07.829829 22948 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:09:07.851884 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:07.851998 22952 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 7.
I20260709 13:09:07.769908 26042 raft_consensus.cc:740] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:07.773295 26033 raft_consensus.cc:2804] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:07.866310 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:07.867102 22878 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 7.
I20260709 13:09:07.840428 22951 raft_consensus.cc:1275] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:09:07.870543 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:07.871587 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:07.841079 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:07.873880 22872 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:07.799461 22774 catalog_manager.cc:5697] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 6 to 9, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 9 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:07.843480 23624 raft_consensus.cc:1275] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:09:07.846354 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:07.874579 22875 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:07.871280 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:07.846988 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:07.875622 22876 raft_consensus.cc:1240] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:09:07.875913 22879 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:09:07.875993 22880 raft_consensus.cc:2393] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 9: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:07.848510 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:07.876906 22873 raft_consensus.cc:1240] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:07.877245 22874 raft_consensus.cc:2410] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 13:09:07.813289 25964 raft_consensus.cc:2749] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:07.730789 26019 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:07.881227 23619 raft_consensus.cc:1240] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:09:07.842625 25370 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:07.814415 26056 raft_consensus.cc:2804] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:09:07.807585 26038 raft_consensus.cc:740] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:07.885632 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:07.886206 22878 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:07.886335 26038 consensus_queue.cc:260] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:07.887357 26038 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:07.875188 25707 raft_consensus.cc:2749] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:09:07.836577 25923 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:07.898610 26019 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:07.899406 23023 raft_consensus.cc:1240] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 8. Current term is 9. Ops: []
W20260709 13:09:07.900696 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 138818us
I20260709 13:09:07.901536 22919 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:07.903048 25910 raft_consensus.cc:2804] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:09:07.903491 25910 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:07.903877 25910 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:07.904688 25874 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:07.875648 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:07.909150 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:07.830062 26058 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:07.909889 26058 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:07.910151 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:07.910976 22880 raft_consensus.cc:2393] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 5: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:07.911437 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:07.911778 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:07.912460 22880 raft_consensus.cc:2393] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 5: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:07.912580 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:07.862218 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 47267.7us
I20260709 13:09:07.839538 26046 raft_consensus.cc:740] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:07.842396 25957 raft_consensus.cc:2749] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:07.885516 26056 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:07.924110 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:07.896236 26043 raft_consensus.cc:2804] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:09:07.879094 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:07.910328 26058 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:09:07.878857 25599 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 13:09:07.913671 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:07.814678 25919 consensus_queue.cc:1048] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 13:09:07.823616 25595 consensus_queue.cc:1048] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 13:09:07.865907 26033 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:07.914719 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 49178.6us
I20260709 13:09:07.940395 22877 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 8.
I20260709 13:09:07.918354 22875 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
I20260709 13:09:07.924983 22873 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
I20260709 13:09:07.927629 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:07.932051 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:07.932273 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:07.942956 22952 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 7.
I20260709 13:09:07.932979 26046 consensus_queue.cc:260] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:07.933532 25926 consensus_queue.cc:1048] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:09:07.934650 26047 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:07.926995 26056 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:07.936901 25570 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:07.934420 22993 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:07.939514 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:09:07.956174 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 53579.5us
I20260709 13:09:07.956439 22951 raft_consensus.cc:2393] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 5: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:07.890285 25370 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:07.943774 22954 raft_consensus.cc:2393] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 8: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:07.944212 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:07.949536 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:07.943989 26033 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:07.957983 22948 raft_consensus.cc:2410] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:09:07.964959 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:07.965574 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:07.966565 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:07.967211 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:07.970227 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:07.970660 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:07.971380 23624 raft_consensus.cc:2376] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9. Re-sending same reply.
I20260709 13:09:07.971596 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:07.972018 22872 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 7.
I20260709 13:09:07.972569 22951 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:07.973094 22879 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
I20260709 13:09:07.975919 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:07.976274 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:07.978098 22880 raft_consensus.cc:1275] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:09:07.954960 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:07.979178 22950 raft_consensus.cc:2376] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 7. Re-sending same reply.
I20260709 13:09:07.979971 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:07.944830 26043 raft_consensus.cc:697] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:07.982297 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:07.983098 22948 raft_consensus.cc:2393] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 7: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:07.981534 22947 raft_consensus.cc:2393] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 8: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:07.975610 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:07.984743 22946 raft_consensus.cc:2393] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 8: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:07.985313 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:07.984444 22993 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:07.985592 22920 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:07.987355 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:07.987637 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:07.988385 22950 raft_consensus.cc:2410] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 13:09:07.988461 23620 raft_consensus.cc:2410] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:09:07.990665 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:07.991551 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:07.964838 26045 raft_consensus.cc:2749] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:07.992151 22947 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:07.997426 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:07.998023 22946 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 9.
I20260709 13:09:08.000017 22946 raft_consensus.cc:1275] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 13:09:08.001358 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 46996.6us
I20260709 13:09:08.002254 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:09:07.993703 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 194250us
I20260709 13:09:07.994503 22873 raft_consensus.cc:2393] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 7: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:08.001530 22950 raft_consensus.cc:1275] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:09:07.993419 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:09:08.003608 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 154665us
I20260709 13:09:07.975111 26046 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:08.003983 22872 raft_consensus.cc:2393] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 9: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:08.005153 22948 raft_consensus.cc:1275] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:09:08.005720 22954 raft_consensus.cc:2393] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 9: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:08.013413 22918 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:07.994112 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:08.017169 22872 raft_consensus.cc:1240] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:09:08.015167 22947 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
I20260709 13:09:08.017777 22952 raft_consensus.cc:1275] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:09:08.017198 22880 raft_consensus.cc:1275] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:09:08.010351 23588 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:08.017653 22879 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:09:08.021068 22951 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
I20260709 13:09:08.021422 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:08.021999 23026 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 8.
I20260709 13:09:08.023615 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:08.024227 22879 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:08.029539 22879 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
I20260709 13:09:08.017699 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:08.031144 22873 raft_consensus.cc:1275] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:09:08.033677 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:08.036345 23027 raft_consensus.cc:1240] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:09:07.972148 25963 consensus_queue.cc:1048] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:09:08.041517 22878 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
I20260709 13:09:08.019295 23588 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:08.042558 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:08.006683 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:08.022837 26045 raft_consensus.cc:2804] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:09:08.015229 25922 raft_consensus.cc:2749] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:09:08.046156 23020 raft_consensus.cc:1275] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:09:08.046739 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:08.047497 23026 raft_consensus.cc:1275] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:09:08.054028 22878 raft_consensus.cc:2393] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 8: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:08.045426 23024 raft_consensus.cc:2393] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 8: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:08.054412 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:08.052374 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:08.036078 25968 raft_consensus.cc:2804] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:08.055094 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:08.043262 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:08.061801 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:08.055274 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:08.053910 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:08.056653 26045 raft_consensus.cc:697] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:08.062098 23021 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 5.
I20260709 13:09:08.056797 23616 raft_consensus.cc:2393] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 10: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:08.056792 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:08.071269 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:09:08.074470 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 44510.6us
I20260709 13:09:08.076573 23026 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 5.
I20260709 13:09:08.004508 26072 consensus_queue.cc:1059] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 13:09:08.080305 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:09:08.083447 26065 log.cc:927] Time spent T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:09:08.083814 26068 raft_consensus.cc:2804] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:08.083509 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:08.093895 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:08.068928 25968 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:08.100348 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:08.104730 23028 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 7.
I20260709 13:09:08.105546 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:08.092483 23022 raft_consensus.cc:2376] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 6. Re-sending same reply.
I20260709 13:09:08.051790 25840 consensus_queue.cc:1048] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:09:08.111269 23020 raft_consensus.cc:2393] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 5: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:08.077162 25781 raft_consensus.cc:2804] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:09:08.055187 25595 consensus_queue.cc:1059] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 13:09:08.117247 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:08.117362 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:08.119524 23028 raft_consensus.cc:2393] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 8: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:08.119616 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:08.097437 26068 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:08.057411 26064 consensus_queue.cc:1048] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 13:09:08.120594 23021 raft_consensus.cc:2393] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 9: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
W20260709 13:09:08.122637 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 74996.9us
I20260709 13:09:08.123368 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:08.108875 25968 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:08.124938 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:08.088327 26007 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:08.132812 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 113579us
I20260709 13:09:08.126653 22843 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:08.150465 25595 raft_consensus.cc:2804] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:08.150897 25595 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:08.151270 25595 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:08.157110 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:08.157760 23028 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 7.
I20260709 13:09:08.115201 25781 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:08.159794 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:08.159835 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:08.121963 26068 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:08.115437 26071 raft_consensus.cc:2804] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:09:08.166337 23024 raft_consensus.cc:1275] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:09:08.124516 26064 consensus_queue.cc:1243] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer aed3f089ec8b49af9c8cf5cdee76e8e7 log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:09:08.117962 26072 consensus_queue.cc:1048] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 13:09:08.098716 26058 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:08.171558 22949 raft_consensus.cc:2393] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 8: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:08.162178 25781 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:08.174955 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:08.176971 23028 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 5.
I20260709 13:09:08.161041 25570 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:08.122018 26075 consensus_queue.cc:1059] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 13:09:08.180794 26076 consensus_queue.cc:1048] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:09:08.181589 26076 consensus_queue.cc:1243] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer aed3f089ec8b49af9c8cf5cdee76e8e7 log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:09:08.099066 26045 consensus_queue.cc:237] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:08.188930 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:08.189718 23024 raft_consensus.cc:2393] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 8: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:08.190719 22947 raft_consensus.cc:1240] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:09:08.194788 23617 raft_consensus.cc:2468] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
I20260709 13:09:08.197001 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:08.200019 26058 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:08.200942 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:08.102881 26043 consensus_queue.cc:237] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:08.207949 26033 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:08.213249 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:08.214038 23028 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 7.
I20260709 13:09:08.214381 25923 raft_consensus.cc:2749] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:09:08.215184 22955 raft_consensus.cc:2393] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 8: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:08.219520 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:08.219750 25370 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:08.220232 23617 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:08.245922 23027 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 7.
I20260709 13:09:08.250578 26064 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:08.251266 26064 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:08.261729 23617 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
W20260709 13:09:08.221628 26015 consensus_meta.cc:287] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.524s	user 0.011s	sys 0.000s
I20260709 13:09:08.247176 26033 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:08.278414 23588 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:08.279790 26082 raft_consensus.cc:2749] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:09:08.279736 23588 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:08.288908 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:08.290767 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:08.292588 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:08.293103 25595 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:08.294335 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:08.296038 23021 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:09:08.297040 23021 raft_consensus.cc:606] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:08.297506 23021 raft_consensus.cc:614] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:08.298856 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:08.304585 25781 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:08.299656 23021 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
I20260709 13:09:08.308358 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:08.293515 23022 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 8.
I20260709 13:09:08.276993 25910 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:08.274881 26015 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:08.309357 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 146348us
W20260709 13:09:08.369694 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:08.313716 26038 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.351105 26056 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:08.349833 22875 raft_consensus.cc:1240] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 7. Current term is 8. Ops: []
I20260709 13:09:08.393541 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:08.335429 22878 raft_consensus.cc:1275] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:09:08.394618 23619 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 671) took 1060 ms. Trace:
I20260709 13:09:08.395000 23619 rpcz_store.cc:276] 0709 13:09:07.334709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:07.475091 (+140382us) service_pool.cc:224] Handling call
0709 13:09:07.487114 (+ 12023us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:07.890585 (+403471us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:08.394410 (+503825us) spinlock_profiling.cc:243] Waited 390 ms on lock 0x7b54003d9ad4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:08.394565 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":897668352}
I20260709 13:09:08.314559 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:08.384701 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 399147us
I20260709 13:09:08.387269 22951 raft_consensus.cc:1240] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 7. Current term is 8. Ops: []
I20260709 13:09:08.393108 23623 raft_consensus.cc:1240] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:09:08.397259 22843 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:08.386801 23023 raft_consensus.cc:2393] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 7: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:08.392884 23625 raft_consensus.cc:1240] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:09:08.402225 22920 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:08.392989 25595 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:08.403271 25370 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:08.320554 26008 raft_consensus.cc:2749] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:09:08.381390 26068 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:08.404682 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:08.405665 26068 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:08.405820 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:08.406451 23024 raft_consensus.cc:2376] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6. Re-sending same reply.
I20260709 13:09:08.387849 26042 consensus_queue.cc:260] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:08.361416 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:08.408219 23021 raft_consensus.cc:2376] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8. Re-sending same reply.
I20260709 13:09:08.357910 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:08.407354 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:08.409858 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:08.410287 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:08.390937 26038 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:08.338861 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:08.411243 26085 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:09:08.411717 22920 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:08.410445 23625 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:08.412492 25370 raft_consensus.cc:2749] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:09:08.413378 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:08.413532 26074 consensus_queue.cc:1048] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:09:08.414693 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:08.415416 23022 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:09:08.413388 22920 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:08.411832 26085 raft_consensus.cc:697] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:08.416899 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:08.416985 22920 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:08.417477 26085 consensus_queue.cc:237] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:08.417877 25370 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:08.420897 26091 raft_consensus.cc:2804] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:09:08.421409 26091 raft_consensus.cc:697] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
W20260709 13:09:08.421921 22776 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48906 (request call id 199) took 23244 ms (client timeout 15000 ms). Trace:
I20260709 13:09:08.422255 26091 consensus_queue.cc:237] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:08.425827 25370 raft_consensus.cc:2749] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:08.410981 23024 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
I20260709 13:09:08.426206 22774 catalog_manager.cc:5697] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:09:08.427464 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:08.428557 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:08.428771 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:08.429409 25595 raft_consensus.cc:2749] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:09:08.429455 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:08.430193 23021 raft_consensus.cc:2376] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8. Re-sending same reply.
I20260709 13:09:08.428733 26007 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:08.431030 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:08.431551 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:08.432034 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:08.370600 25781 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:08.432861 25595 consensus_queue.cc:1059] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:09:08.381734 26064 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:08.434016 25570 raft_consensus.cc:3055] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:09:08.434403 25570 raft_consensus.cc:740] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:08.435252 25570 consensus_queue.cc:260] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:08.436942 25570 raft_consensus.cc:3060] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:08.437068 23027 raft_consensus.cc:1240] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:09:08.437772 26094 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.438118 25781 consensus_queue.cc:1059] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:09:08.438494 23590 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:08.439095 25781 raft_consensus.cc:3055] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:09:08.439061 25883 consensus_queue.cc:1059] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:09:08.439565 26008 raft_consensus.cc:2804] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:09:08.440196 25919 raft_consensus.cc:3055] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:09:08.440260 26008 raft_consensus.cc:697] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 7 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:08.440721 25919 raft_consensus.cc:740] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:08.428048 22774 catalog_manager.cc:5697] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 2 to 6, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 6 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:08.439613 25781 raft_consensus.cc:740] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:08.438266 26094 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:08.384313 25968 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:08.445734 25919 consensus_queue.cc:260] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:08.446033 25781 consensus_queue.cc:260] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:08.447324 25919 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:08.447585 25781 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:08.443933 23625 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11.
I20260709 13:09:08.448196 22844 leader_election.cc:400] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 8. Current term is 9.
W20260709 13:09:08.350756 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 71087.4us
I20260709 13:09:08.387777 25910 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:08.447399 26094 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:08.448823 22844 leader_election.cc:403] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:09:08.441799 23022 raft_consensus.cc:2410] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:09:08.431555 23021 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:08.452289 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:08.441792 22774 catalog_manager.cc:5697] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 4 to 5, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 5 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:08.453088 23616 raft_consensus.cc:2410] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:09:08.451889 22844 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:08.454109 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:08.455118 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:08.455667 23618 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:08.455096 22774 catalog_manager.cc:5697] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 2 to 7, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 7 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:09:08.457391 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:08.458838 22947 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:36566
I20260709 13:09:08.459347 22947 raft_consensus.cc:606] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:08.459667 22947 raft_consensus.cc:614] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:08.460081 22879 raft_consensus.cc:1240] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:09:08.450462 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:08.460777 22880 raft_consensus.cc:1240] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:09:08.462220 23625 raft_consensus.cc:2393] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 8: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:08.462805 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:08.462677 23617 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
I20260709 13:09:08.463420 23616 raft_consensus.cc:2468] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 3.
I20260709 13:09:08.463577 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:08.455780 22918 leader_election.cc:336] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.569s (SENT)
I20260709 13:09:08.465615 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:08.466008 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:08.466233 22918 leader_election.cc:336] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.618s (SENT)
I20260709 13:09:08.466626 22874 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 9.
I20260709 13:09:08.466707 22918 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:08.466609 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:08.467888 22954 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:08.468609 22843 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:08.469894 23619 raft_consensus.cc:2393] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 8: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:08.384786 26071 raft_consensus.cc:697] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:08.468917 22774 catalog_manager.cc:5697] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:09:08.473448 22880 raft_consensus.cc:1240] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 5. Current term is 6. Ops: []
I20260709 13:09:08.473690 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:09:08.422297 22776 rpcz_store.cc:269] 0709 13:08:45.177325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:45.229415 (+ 52090us) service_pool.cc:224] Handling call
0709 13:08:45.562250 (+332835us) spinlock_profiling.cc:243] Waited 243 ms on lock 0x7b54002b6e54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4f090 00007f6135dedf3a 00007f613bba41cc 00007f613bca237e 00007f613a1632a4 00007f613a163232 00007f613a163161 00007f613a1630dc 00007f613a162392 00007f613412048c 00007f613411f8c6
0709 13:09:08.421846 (+22859596us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:09:05.587725 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:09:05.749121 (+161396us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:09:05.749183 (+    62us) write_op.cc:435] Acquired schema lock
0709 13:09:05.749211 (+    28us) tablet.cc:662] Decoding operations
0709 13:09:05.750148 (+   937us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:09:05.750243 (+    95us) write_op.cc:641] Partition lock acquired for write op
0709 13:09:05.750280 (+    37us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:09:05.750950 (+   670us) tablet.cc:701] Row locks acquired
0709 13:09:05.751001 (+    51us) write_op.cc:260] PREPARE: finished
0709 13:09:05.751202 (+   201us) write_op.cc:270] Start()
0709 13:09:05.751360 (+   158us) write_op.cc:276] Timestamp: P: 1783602545751121 usec, L: 0
0709 13:09:05.751380 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 13:09:05.940789 (+189409us) log.cc:844] Serialized 2477 byte log entry
0709 13:09:06.983075 (+1042286us) op_driver.cc:464] REPLICATION: finished
0709 13:09:07.085810 (+102735us) write_op.cc:301] APPLY: starting
0709 13:09:07.085904 (+    94us) tablet.cc:1367] starting BulkCheckPresence
0709 13:09:07.086080 (+   176us) tablet.cc:1370] finished BulkCheckPresence
0709 13:09:07.086243 (+   163us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:09:07.110426 (+ 24183us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:09:07.110774 (+   348us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0709 13:09:07.110831 (+    57us) write_op.cc:312] APPLY: finished
0709 13:09:07.111157 (+   326us) log.cc:844] Serialized 53 byte log entry
0709 13:09:07.111350 (+   193us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:07.111468 (+   118us) write_op.cc:454] Released schema lock
0709 13:09:07.111527 (+    59us) write_op.cc:341] FINISH: Updating metrics
0709 13:09:07.111979 (+   452us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:07.112004 (+    25us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2583382,"reported_tablets":23,"spinlock_wait_cycles":559553792,"child_traces":[["op",{"apply.queue_time_us":102441,"apply.run_cpu_time_us":6146,"apply.run_wall_time_us":26157,"num_ops":6,"prepare.queue_time_us":292511,"prepare.run_cpu_time_us":20990,"prepare.run_wall_time_us":702427,"raft.queue_time_us":96615,"raft.run_cpu_time_us":27058,"raft.run_wall_time_us":51338,"replication_time_us":1231530,"thread_start_us":618414,"threads_started":4,"wal-append.queue_time_us":376656,"wal-append.run_cpu_time_us":38507,"wal-append.run_wall_time_us":1799439}]]}
I20260709 13:09:08.471720 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:08.474663 22873 raft_consensus.cc:2393] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 7: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:08.465301 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:08.474905 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:08.475237 22872 raft_consensus.cc:2393] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 9: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:08.475565 22880 raft_consensus.cc:2393] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 8: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:08.474109 22879 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:08.476161 23622 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 8.
I20260709 13:09:08.477103 22953 raft_consensus.cc:1240] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:08.478080 23024 raft_consensus.cc:1240] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 7. Current term is 8. Ops: []
W20260709 13:09:08.478368 22918 leader_election.cc:336] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.555s (SENT)
I20260709 13:09:08.478624 22949 raft_consensus.cc:1240] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:08.478282 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:08.475978 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:08.480543 22872 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:08.480649 22947 pending_rounds.cc:85] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting all ops after (but not including) 4
I20260709 13:09:08.481026 22947 pending_rounds.cc:107] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 13:09:08.481284 22947 raft_consensus.cc:2889] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:08.481454 23622 raft_consensus.cc:1240] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:09:08.483251 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:08.484067 23622 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 7.
I20260709 13:09:08.484967 22879 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:09:08.477852 25910 raft_consensus.cc:2749] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:09:08.486650 23590 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:08.487669 26083 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:08.488098 26083 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:08.488433 26083 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:08.451100 25968 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:08.461203 26064 consensus_queue.cc:1048] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 13:09:08.490384 26064 raft_consensus.cc:2749] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:09:08.491628 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:08.492295 22946 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:08.492945 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:08.493705 22880 raft_consensus.cc:2393] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 7: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:08.462019 25883 consensus_queue.cc:1059] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:09:08.482290 22950 pending_rounds.cc:85] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting all ops after (but not including) 4
I20260709 13:09:08.500298 22950 pending_rounds.cc:107] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 13:09:08.500541 22950 raft_consensus.cc:2889] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:08.499734 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:08.461763 26058 raft_consensus.cc:2749] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:09:08.505275 22873 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:08.517356 22872 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:09:08.518273 22872 raft_consensus.cc:1240] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 4. Current term is 5. Ops: []
I20260709 13:09:08.506469 23022 raft_consensus.cc:1240] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:09:08.507625 22844 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:08.471467 26076 consensus_queue.cc:1048] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:09:08.512879 25968 consensus_queue.cc:1059] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:09:08.461838 25957 raft_consensus.cc:2749] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 8. Current term is 9.
I20260709 13:09:08.520893 22844 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:08.522985 23618 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 8.
I20260709 13:09:08.461199 26068 consensus_queue.cc:1059] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 13:09:08.513231 26083 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:08.527550 26094 raft_consensus.cc:2804] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:09:08.473340 26015 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:08.494530 26038 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:08.542941 22844 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:08.543789 22844 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:08.544085 22920 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:08.505787 26056 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:08.516752 22774 catalog_manager.cc:5697] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 6 to 9, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 9 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:08.558805 22946 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 8.
I20260709 13:09:08.560290 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:08.561095 22876 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:09:08.563562 22918 leader_election.cc:336] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.688s (SENT)
W20260709 13:09:08.565141 23588 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:08.567159 22920 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:09:08.568082 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 99550.2us
I20260709 13:09:08.569095 22843 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:08.542732 26076 raft_consensus.cc:2804] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:09:08.571149 22774 catalog_manager.cc:5697] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 3 to 6, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 6 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:08.540028 26094 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:08.575709 26094 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:08.555325 26064 raft_consensus.cc:2804] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:08.557525 26058 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:09:08.494861 26008 consensus_queue.cc:237] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:08.588274 22876 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:09:08.582403 22920 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:08.593531 22920 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:08.596937 22774 catalog_manager.cc:5697] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:09:08.613677 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:08.614909 22876 raft_consensus.cc:2410] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 9 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 13:09:08.556239 25840 consensus_queue.cc:1059] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:09:08.560052 26080 consensus_queue.cc:1059] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 13:09:08.584537 26064 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:08.590526 26038 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.591400 25968 raft_consensus.cc:2804] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:09:08.628163 22774 catalog_manager.cc:5697] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 4 to 5, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 5 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
W20260709 13:09:08.632757 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 63768.9us
I20260709 13:09:08.623111 26091 raft_consensus.cc:2804] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:09:08.624024 26068 consensus_queue.cc:1243] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer f3c799b1249e40119d2a5864231c6c75 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:09:08.634135 22843 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:08.635615 22774 catalog_manager.cc:5697] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:09:08.625571 26064 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:09:08.642781 26059 log.cc:927] Time spent T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.772s	user 0.000s	sys 0.003s
I20260709 13:09:08.645673 26092 raft_consensus.cc:2749] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:08.646322 26093 consensus_queue.cc:1059] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 13:09:08.653286 23590 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:08.619263 26007 consensus_queue.cc:1059] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.017s
I20260709 13:09:08.627393 25968 raft_consensus.cc:697] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:08.632109 26097 raft_consensus.cc:2749] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:09:08.635210 26087 raft_consensus.cc:2804] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:09:08.673079 25957 raft_consensus.cc:3055] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:09:08.673300 26105 raft_consensus.cc:2764] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:09:08.673774 26083 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:08.674587 26104 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:08.697515 25968 consensus_queue.cc:237] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:08.698206 26104 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:08.698735 26104 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:08.678170 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:08.686645 23021 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:09:08.695889 22774 catalog_manager.cc:5697] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 1 to 3, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 3 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:08.686832 22954 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 8.
I20260709 13:09:08.687892 23027 raft_consensus.cc:1240] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:09:08.690635 22873 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:09:08.631156 26038 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:08.657449 26091 raft_consensus.cc:697] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:08.695411 25957 raft_consensus.cc:740] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:08.705654 26011 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:08.585215 26071 consensus_queue.cc:237] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:08.707458 25883 raft_consensus.cc:3055] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:09:08.708349 26016 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:08.725178 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:08.725381 26091 consensus_queue.cc:237] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:08.709408 23025 raft_consensus.cc:2393] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 5: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:08.713387 26074 raft_consensus.cc:3055] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:09:08.713804 26087 raft_consensus.cc:697] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:08.715091 26064 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:08.716135 25922 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.742081 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:08.702143 22774 catalog_manager.cc:5697] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:09:08.742816 22873 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:09:08.723060 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:08.727923 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:08.719878 25797 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.739713 25957 consensus_queue.cc:260] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:08.731390 26094 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:08.727159 25883 raft_consensus.cc:740] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:08.749373 23622 raft_consensus.cc:1240] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 7. Current term is 8. Ops: []
I20260709 13:09:08.749953 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:08.758597 23623 raft_consensus.cc:1275] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:09:08.758667 22876 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:09:08.751325 22918 leader_election.cc:336] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.827s (SENT)
I20260709 13:09:08.717981 22919 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:08.756484 22774 catalog_manager.cc:5697] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 6 to 7, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 7 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:08.761710 22873 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 8.
I20260709 13:09:08.731560 26074 raft_consensus.cc:740] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:08.763104 23590 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
W20260709 13:09:08.765311 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 47396.5us
I20260709 13:09:08.770066 22919 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:08.760741 22774 catalog_manager.cc:5697] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 4 to 6, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 6 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:09:08.774547 22774 catalog_manager.cc:5697] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 3 to 5, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 5 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:09:08.773244 22919 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:08.799266 22876 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 10.
I20260709 13:09:08.801155 23590 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:08.758054 26058 raft_consensus.cc:697] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 8 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:08.771116 26007 raft_consensus.cc:2804] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:09:08.806989 22918 leader_election.cc:336] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.820s (SENT)
I20260709 13:09:08.753391 25922 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:08.781373 26092 raft_consensus.cc:2804] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:09:08.778772 25919 raft_consensus.cc:2804] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:09:08.755767 26104 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:08.755652 25797 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:08.788499 25957 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:09:08.820916 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 186794us
I20260709 13:09:08.809043 26011 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:09:08.822134 22843 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:08.778264 25812 consensus_queue.cc:1059] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:09:08.819954 26092 raft_consensus.cc:697] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:08.831360 26105 raft_consensus.cc:2804] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:09:08.832168 26011 raft_consensus.cc:697] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:08.822492 25919 raft_consensus.cc:697] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 8 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:08.842267 23623 raft_consensus.cc:1275] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 7. (term mismatch)
I20260709 13:09:08.842397 26092 consensus_queue.cc:237] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:08.803689 25923 consensus_queue.cc:1059] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:09:08.818357 26007 raft_consensus.cc:697] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:08.850823 22843 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:08.852243 23623 pending_rounds.cc:85] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2: Aborting all ops after (but not including) 6
I20260709 13:09:08.878582 23623 pending_rounds.cc:107] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 13:09:08.878935 23623 raft_consensus.cc:2889] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:08.876961 22994 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:08.880519 22843 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:08.881943 22843 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:08.882676 25572 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:09:08.883419 25572 raft_consensus.cc:2749] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:09:08.883597 22843 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:08.884157 25572 raft_consensus.cc:2749] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:08.813308 26016 consensus_queue.cc:1048] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:09:08.884433 25840 raft_consensus.cc:2764] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:09:08.884585 22843 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:08.886615 22843 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:08.886552 25572 consensus_queue.cc:1048] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:08.813225 26021 consensus_queue.cc:1059] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:09:08.889606 22843 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:08.888314 26111 raft_consensus.cc:2804] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:09:08.885650 25840 raft_consensus.cc:2749] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:09:08.849735 26105 raft_consensus.cc:697] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 6 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:08.893852 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:08.895675 23618 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:08.894245 22994 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:08.877835 25370 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.877960 25593 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:08.825587 25883 consensus_queue.cc:260] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:08.892460 26111 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:08.907095 23618 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 8.
I20260709 13:09:08.893435 26016 raft_consensus.cc:2749] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:08.892899 26064 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:08.916594 22994 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:08.821687 26074 consensus_queue.cc:260] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:08.897555 25572 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:09:08.918965 22844 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:08.819661 26087 consensus_queue.cc:237] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:08.907945 26011 consensus_queue.cc:237] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:08.923324 25781 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:08.929271 26108 log.cc:927] Time spent T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:09:08.929347 25922 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:08.925246 25572 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:08.930511 25572 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:08.906404 26038 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:08.925805 22994 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:08.929056 25781 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:08.906091 25964 consensus_queue.cc:1048] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 13:09:08.924022 25570 raft_consensus.cc:2804] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:09:08.926478 22843 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:08.932940 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:08.879421 26058 consensus_queue.cc:237] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:08.934962 26094 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:08.925829 25593 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.908525 25370 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:08.935861 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:08.925515 25876 raft_consensus.cc:2804] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:09:08.950027 25876 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
W20260709 13:09:08.950222 22918 leader_election.cc:336] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.897s (SENT)
I20260709 13:09:08.950487 22843 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:08.950507 25876 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:08.950738 22918 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:08.913542 26083 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.953572 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:08.954172 22872 raft_consensus.cc:2468] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6.
I20260709 13:09:08.930847 26105 consensus_queue.cc:237] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:08.956408 22946 raft_consensus.cc:1275] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:09:08.915472 26111 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:08.986642 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:08.941872 22994 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:08.932535 25963 raft_consensus.cc:2749] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:08.931674 26056 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:08.955502 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:08.961611 22843 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:08.975356 22992 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:08.914443 26084 raft_consensus.cc:2804] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:09:08.955775 23590 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:08.997771 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:08.939427 25922 raft_consensus.cc:2749] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:09:08.999303 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 57503us
W20260709 13:09:08.964932 22918 leader_election.cc:336] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.940s (SENT)
I20260709 13:09:08.973130 22954 raft_consensus.cc:1275] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:09:09.001000 22918 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:08.981616 26094 raft_consensus.cc:2749] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:09:08.946969 26097 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:09:08.948920 26074 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:08.937953 26007 consensus_queue.cc:237] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:08.995610 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:08.995929 22843 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:09:09.041966 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 46104.6us
I20260709 13:09:09.042680 22843 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:08.948792 25570 raft_consensus.cc:697] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 8 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:08.997047 22955 raft_consensus.cc:1275] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:09:09.047500 22954 raft_consensus.cc:1275] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:09:08.997321 22949 raft_consensus.cc:1240] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 7. Current term is 8. Ops: []
I20260709 13:09:08.951686 26104 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:08.973405 26091 raft_consensus.cc:2749] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:08.945461 26016 consensus_queue.cc:1048] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
W20260709 13:09:09.008808 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 43913.1us
I20260709 13:09:08.993691 26030 consensus_queue.cc:1048] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 13:09:09.057121 22874 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6.
I20260709 13:09:09.010514 26047 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:08.991125 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:09.017469 26011 raft_consensus.cc:2804] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:09:09.021899 25791 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:09.023393 26033 raft_consensus.cc:3055] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:09:09.024168 26110 raft_consensus.cc:2749] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:09:09.048285 25957 raft_consensus.cc:2764] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 13:09:09.026646 26015 consensus_queue.cc:1048] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:09:09.027354 25963 consensus_queue.cc:1059] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:09:09.024893 22992 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:09.000702 26045 raft_consensus.cc:2804] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:09:09.031282 26111 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:09.048187 22843 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:09.051446 26038 raft_consensus.cc:2749] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:09:09.001641 26064 raft_consensus.cc:2749] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:09:08.973528 25593 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:08.982372 26083 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:09.085505 26110 raft_consensus.cc:2804] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:09:09.085975 26047 raft_consensus.cc:3055] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:09:09.086329 26047 raft_consensus.cc:740] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:09.086655 26083 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:09.007928 26084 raft_consensus.cc:697] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:09.024876 25797 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:08.999511 26056 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:09.050617 25840 raft_consensus.cc:2749] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:09:08.945575 25919 consensus_queue.cc:237] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:09.069945 26010 raft_consensus.cc:3055] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:09:09.070120 23588 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:09.070982 25781 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:09.066807 26033 raft_consensus.cc:740] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:09.073145 25570 consensus_queue.cc:237] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:09.075201 26079 log.cc:927] Time spent T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.824s	user 0.000s	sys 0.002s
I20260709 13:09:09.078904 22843 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:09.049955 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 94250.3us
W20260709 13:09:09.081668 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 56876.3us
W20260709 13:09:09.083940 22918 leader_election.cc:336] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 2.045s (SENT)
I20260709 13:09:09.118655 22918 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:09.119025 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:09.119745 22871 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 9.
I20260709 13:09:09.121170 23590 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:09.121624 23618 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:09:09.069245 25923 raft_consensus.cc:3055] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:09:09.067204 26011 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:09.081154 26097 raft_consensus.cc:2749] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:09:09.104101 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 101386us
I20260709 13:09:09.122490 22843 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:09.128019 22994 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:09.128247 22843 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:09.089478 26110 raft_consensus.cc:697] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:09.088171 26045 raft_consensus.cc:697] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 6 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:09.129251 22843 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:09.087116 26047 consensus_queue.cc:260] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:09.133127 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:09.133714 22994 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:09.135730 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:09.136622 22875 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 9.
I20260709 13:09:09.137101 22843 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:09.113780 26010 raft_consensus.cc:740] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:09.147442 22843 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:09.126415 25968 raft_consensus.cc:2749] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:09:09.132579 25797 raft_consensus.cc:2804] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:09:09.151328 26120 log.cc:927] Time spent T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:09:09.154522 25370 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:09.155265 26111 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:09.126044 26064 raft_consensus.cc:2749] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:09:09.155869 26106 consensus_queue.cc:1048] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:09:09.156287 25797 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:09.156694 25797 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:09.157351 26097 consensus_queue.cc:1048] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:09:09.159585 22955 raft_consensus.cc:1275] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 6. (term mismatch)
I20260709 13:09:09.159924 22955 pending_rounds.cc:85] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting all ops after (but not including) 5
I20260709 13:09:09.160138 22955 pending_rounds.cc:107] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting uncommitted NO_OP operation due to leader change: 6.6
I20260709 13:09:09.160365 22955 raft_consensus.cc:2889] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:09.131752 25957 raft_consensus.cc:2749] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:09:09.164098 26056 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:09.163858 25797 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:09.164875 26033 consensus_queue.cc:260] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:09.130439 25964 consensus_queue.cc:1059] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:09:09.166146 25797 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:09.166527 25593 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:09.166882 23622 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc"
dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:50814
I20260709 13:09:09.150198 25595 raft_consensus.cc:2749] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:09:09.167493 23622 raft_consensus.cc:606] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:09.167905 23622 raft_consensus.cc:614] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:09.170063 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:09.174741 25572 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:09.175220 23621 raft_consensus.cc:1275] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:09:09.150561 26084 consensus_queue.cc:237] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:09.153950 22843 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:09.126420 25923 raft_consensus.cc:740] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:09.130089 26011 raft_consensus.cc:3060] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:09.134032 25840 raft_consensus.cc:2804] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:09:09.184340 22843 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:09.188175 25957 raft_consensus.cc:2749] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:09:09.132982 26007 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.197175 26045 consensus_queue.cc:237] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:09.198653 26110 consensus_queue.cc:237] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:09.136214 26058 raft_consensus.cc:2749] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:09:09.201958 22843 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:09.149690 26094 raft_consensus.cc:2764] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:09:09.210070 22843 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:09.186050 26047 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:09.149716 25922 consensus_queue.cc:1048] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.065s
I20260709 13:09:09.200838 25595 raft_consensus.cc:2749] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:09:09.218230 23622 raft_consensus.cc:1275] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:09:09.207950 26033 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:09.229231 25923 consensus_queue.cc:260] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:09.230257 26125 raft_consensus.cc:2804] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:09.230648 25923 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:09.164384 25876 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:09.228860 25572 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:09.213014 26030 raft_consensus.cc:2804] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:09:09.230702 26125 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:09.219023 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 92655.7us
I20260709 13:09:09.219949 25957 raft_consensus.cc:2804] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:09:09.233381 23026 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33418 (request call id 1663) took 1185 ms. Trace:
I20260709 13:09:09.211018 25840 raft_consensus.cc:697] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 7 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:09.235577 23028 raft_consensus.cc:1275] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 13:09:09.212785 26097 raft_consensus.cc:2749] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:09:09.210788 26015 consensus_queue.cc:1048] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 13:09:09.226081 25922 consensus_queue.cc:1243] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer aed3f089ec8b49af9c8cf5cdee76e8e7 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
W20260709 13:09:09.228243 22843 leader_election.cc:336] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.775s (SENT)
I20260709 13:09:09.242205 23026 rpcz_store.cc:276] 0709 13:09:08.048311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:08.054205 (+  5894us) service_pool.cc:224] Handling call
0709 13:09:09.233205 (+1179000us) spinlock_profiling.cc:243] Waited 21.5 ms on lock 0x7b540010c554. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:09.233329 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":49553280}
I20260709 13:09:09.208871 26007 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:09.166427 26010 consensus_queue.cc:260] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:09.248548 26098 log.cc:927] Time spent T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.611s	user 0.003s	sys 0.000s
I20260709 13:09:09.248875 26127 consensus_queue.cc:1048] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:09.262328 23619 raft_consensus.cc:1240] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 LEADER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:09.262732 23588 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:09.263406 26030 raft_consensus.cc:697] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 5 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:09.263851 26138 consensus_queue.cc:1048] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:09:09.264063 25910 consensus_queue.cc:1059] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:09:09.265017 25910 consensus_queue.cc:1243] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer 5b346767a89745fa82928a5dac9cc0b2 log is divergent from this leader: its last log entry 10.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:09:09.264549 26030 consensus_queue.cc:237] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:09.265233 23620 raft_consensus.cc:1240] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:09:09.265507 25910 raft_consensus.cc:3055] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:09:09.249825 25876 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:09.240563 26125 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:09:09.252027 26109 log.cc:927] Time spent T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.359s	user 0.003s	sys 0.000s
I20260709 13:09:09.252719 25840 consensus_queue.cc:237] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:09.265851 25910 raft_consensus.cc:740] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:09.277163 26093 raft_consensus.cc:2749] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:09:09.277798 26080 consensus_queue.cc:1048] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:09:09.280296 23619 raft_consensus.cc:1240] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 13:09:09.257700 25957 raft_consensus.cc:697] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 8 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
W20260709 13:09:09.287246 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:09.289011 26007 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:09.290140 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 49280.7us
I20260709 13:09:09.292594 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:09:09.279191 26131 log.cc:927] Time spent T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
W20260709 13:09:09.294193 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 561554us
I20260709 13:09:09.316968 26011 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:09.281880 26010 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:09.301501 26136 consensus_queue.cc:1048] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:09:09.302888 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:09.325251 23026 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:09.303671 23029 raft_consensus.cc:1217] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Deduplicated request from leader. Original: 3.6->[4.7-8.9]   Dedup: 6.8->[8.9-8.9]
I20260709 13:09:09.305415 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:09.307657 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:09.312059 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 178412us
I20260709 13:09:09.326315 23029 raft_consensus.cc:1240] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 8. Current term is 9. Ops: [4.7-8.9]
I20260709 13:09:09.313033 25910 consensus_queue.cc:260] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:09.327059 22994 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:09.329382 23623 raft_consensus.cc:1240] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:09:09.330178 22994 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:09.330924 23022 raft_consensus.cc:1240] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 6 LEADER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:09:09.323122 26074 raft_consensus.cc:2955] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 LEADER]: Committing config change with OpId 6.5: config changed from index -1 to 5, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:09:09.349610 26069 raft_consensus.cc:3055] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:09:09.350111 26069 raft_consensus.cc:740] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:09.351022 26069 consensus_queue.cc:260] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:09.351985 26069 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:09.359813 22994 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:09.362532 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:09.366271 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:09.366966 23024 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 4.
I20260709 13:09:09.368018 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:09.379134 22843 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:09.379454 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:09.381156 26058 raft_consensus.cc:2804] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:09.381764 26058 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:09.353641 26007 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:09.384019 22871 raft_consensus.cc:1240] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 7. Current term is 8. Ops: []
I20260709 13:09:09.385807 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:09.387317 22994 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:09.387838 22877 raft_consensus.cc:1275] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:09:09.385848 22872 raft_consensus.cc:1240] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 6. Current term is 7. Ops: []
I20260709 13:09:09.361418 26045 raft_consensus.cc:2804] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:09:09.390175 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:09.391284 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:09.392553 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:09.394142 23026 raft_consensus.cc:2410] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
W20260709 13:09:09.401140 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:09.382609 26072 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.405054 26143 raft_consensus.cc:2749] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:09:09.405000 26072 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:09.406924 26142 log.cc:927] Time spent T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:09:09.407313 26045 raft_consensus.cc:697] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:09.408869 26049 consensus_queue.cc:1059] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:09:09.408255 26045 consensus_queue.cc:237] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:09.410221 26049 raft_consensus.cc:3055] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:09:09.410648 26049 raft_consensus.cc:740] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:09.411559 26049 consensus_queue.cc:260] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:09.389457 26058 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:09.414736 26049 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:09.372558 26015 raft_consensus.cc:2749] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:09:09.416569 22878 raft_consensus.cc:1240] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 8 LEADER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 7. Current term is 8. Ops: []
I20260709 13:09:09.403249 22874 raft_consensus.cc:2410] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 9 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:09:09.352972 25963 consensus_queue.cc:1048] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.033s
I20260709 13:09:09.418913 22876 raft_consensus.cc:1240] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 7. Current term is 8. Ops: []
I20260709 13:09:09.419370 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:09.423786 23026 raft_consensus.cc:2393] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 8: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:09.384831 26125 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:09.426268 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:09.427013 26125 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:09.427604 23025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33420 (request call id 1709) took 1138 ms. Trace:
I20260709 13:09:09.427711 23021 raft_consensus.cc:1240] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 4. Current term is 5. Ops: []
I20260709 13:09:09.428011 23025 rpcz_store.cc:276] 0709 13:09:08.289465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:08.293975 (+  4510us) service_pool.cc:224] Handling call
0709 13:09:09.427400 (+1133425us) spinlock_profiling.cc:243] Waited 414 ms on lock 0x7b540024a254. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:09.427546 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":952387456}
I20260709 13:09:09.427619 26074 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:52206 (request call id 13) took 4572 ms. Trace:
I20260709 13:09:09.429534 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:09.446066 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:09.429178 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:09.383229 25910 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:09.448745 23026 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ec334666b7964c969125af95d9c797ed"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:09:09.313071 25957 consensus_queue.cc:237] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:09.449569 23026 raft_consensus.cc:606] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Received request to transfer leadership to TS aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:09.459168 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:09.461247 23020 raft_consensus.cc:1240] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 6 LEADER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:09:09.464090 25919 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:09:09.469000 26101 log.cc:927] Time spent T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.822s	user 0.002s	sys 0.000s
I20260709 13:09:09.473135 22877 raft_consensus.cc:1275] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:09:09.384629 26030 consensus_queue.cc:1048] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.071s
W20260709 13:09:09.476274 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:09.482707 22918 leader_election.cc:336] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 2.140s (SENT)
W20260709 13:09:09.485123 22918 leader_election.cc:336] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.837s (SENT)
I20260709 13:09:09.485595 22918 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:09.490016 22918 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:09.402859 26007 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:09.491613 22918 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:09.492080 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:09.474179 25919 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:09.492736 26084 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:09.495306 25919 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:09.498086 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:09.499593 23026 raft_consensus.cc:2393] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 8: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:09.500880 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:09.493014 22948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 1737) took 1020 ms. Trace:
I20260709 13:09:09.501660 22879 raft_consensus.cc:2393] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 8: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:09.501508 22948 rpcz_store.cc:276] 0709 13:09:08.472214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:08.472325 (+   111us) service_pool.cc:224] Handling call
0709 13:09:08.472596 (+   271us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:08.495416 (+ 22820us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:08.495459 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:08.508738 (+ 13279us) log.cc:844] Serialized 26 byte log entry
0709 13:09:08.554320 (+ 45582us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:09:08.554374 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:08.554465 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:09.468391 (+913926us) spinlock_profiling.cc:243] Waited 48.2 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:09:09.472845 (+  4454us) raft_consensus.cc:1685] finished
0709 13:09:09.472956 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:09.492951 (+ 19995us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12467,"raft.queue_time_us":522,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":10567,"spinlock_wait_cycles":110933888,"thread_start_us":45151,"threads_started":1,"wal-append.queue_time_us":103554}
I20260709 13:09:09.503793 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:09.505743 22948 raft_consensus.cc:2410] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
W20260709 13:09:09.507856 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:09.484341 25781 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:09.512835 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:09.509316 22918 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:09.511030 22878 raft_consensus.cc:1275] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:09:09.493067 25968 raft_consensus.cc:2749] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:09.512830 22843 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:09.498383 25812 raft_consensus.cc:2749] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:09:09.493672 26141 ts_tablet_manager.cc:933] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2: Initiating tablet copy from peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:09.525485 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:09.429360 26074 rpcz_store.cc:276] 0709 13:09:04.855066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:04.855217 (+   151us) service_pool.cc:224] Handling call
0709 13:09:04.914410 (+ 59193us) log.cc:844] Serialized 492 byte log entry
0709 13:09:05.499003 (+584593us) spinlock_profiling.cc:243] Waited 42.8 ms on lock 0x7b540027d1a0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135d5e142 00007f6135d3eeaa 00007f6135d45d53 00007f6135d45cd9 00007f6135d45c69 00007f6135d45c31 00007f6135d44dfd 000000000038f354 000000000038f199 00007f6132cf8475
0709 13:09:05.579938 (+ 80935us) spinlock_profiling.cc:243] Waited 5.63 ms on lock 0x7b18003dbfa0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cc62a6 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135d65b26 00007f6135d5f49a 00007f6135d5e714
0709 13:09:06.057888 (+477950us) spinlock_profiling.cc:243] Waited 6.75 ms on lock 0x7b540027d1a0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135d65b26 00007f6135d5f49a 00007f6135d5e714 00007f6135d3eeaa 00007f6135d45d53 00007f6135d45cd9 00007f6135d45c69 00007f6135d45c31 00007f6135d44dfd 000000000038f354
0709 13:09:09.427526 (+3369638us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2542,"raft.queue_time_us":40591,"raft.run_cpu_time_us":9367,"raft.run_wall_time_us":1223166,"spinlock_wait_cycles":126934016,"thread_start_us":290900,"threads_started":1,"wal-append.queue_time_us":475419,"wal-append.run_cpu_time_us":2532,"wal-append.run_wall_time_us":1127981}
I20260709 13:09:09.648430 26033 raft_consensus.cc:2749] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:09:09.555303 26076 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.932s	user 0.008s	sys 0.029s
W20260709 13:09:09.649480 26076 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.030s	user 0.009s	sys 0.029s
I20260709 13:09:09.649422 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:09.649917 26076 raft_consensus.cc:697] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 8 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
W20260709 13:09:09.652417 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 143138us
I20260709 13:09:09.653959 26033 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.654495 26033 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:09.655140 22918 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:09.656508 26140 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.657074 26140 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:09.657433 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:09.658449 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:09.658959 23023 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:09.659101 26140 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:09.659281 22918 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:09.660090 25874 raft_consensus.cc:2749] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:09:09.655220 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:09.660779 26087 consensus_queue.cc:1059] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:09:09.662014 26087 raft_consensus.cc:2749] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:09:09.662653 26087 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:09.663800 23621 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 648) took 1903 ms. Trace:
I20260709 13:09:09.663594 26087 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:09.664139 23621 rpcz_store.cc:276] 0709 13:09:07.760295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:07.760436 (+   141us) service_pool.cc:224] Handling call
0709 13:09:07.768019 (+  7583us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:09.175745 (+1407726us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:09.663591 (+487846us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b5400670f54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:09.663749 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3235114752}
I20260709 13:09:09.666998 23023 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11.
I20260709 13:09:09.667900 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:09.668154 26011 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:09.668555 23023 raft_consensus.cc:2393] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 6: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:09.494293 26058 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:09.676906 23025 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 8.
I20260709 13:09:09.677114 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:09.677930 26058 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:09.678558 26082 raft_consensus.cc:3055] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:09:09.678433 23023 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
I20260709 13:09:09.679584 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:09.515970 25781 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:09.679836 26043 raft_consensus.cc:3055] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:09:09.680296 26043 raft_consensus.cc:740] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:09.680408 23023 raft_consensus.cc:2410] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 6 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 13:09:09.681375 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:09.681620 25781 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:09.685269 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:09.686416 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:09.687402 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:09.688293 23023 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 7.
I20260709 13:09:09.689402 23028 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 1679) took 1422 ms. Trace:
I20260709 13:09:09.689766 23028 rpcz_store.cc:276] 0709 13:09:08.266746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:08.266914 (+   168us) service_pool.cc:224] Handling call
0709 13:09:09.233613 (+966699us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:09.243355 (+  9742us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:09.689243 (+445888us) spinlock_profiling.cc:243] Waited 950 ms on lock 0x7b540010c550. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:09.689353 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2185027328}
I20260709 13:09:09.690202 26007 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:09.693336 23026 raft_consensus.cc:2393] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 5: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:09.695137 23021 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 8.
I20260709 13:09:09.697849 22948 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 7.
I20260709 13:09:09.648427 21756 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 13:09:09.698971 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 186233us
I20260709 13:09:09.678938 26082 raft_consensus.cc:740] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:09.704514 26082 consensus_queue.cc:260] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:09.705694 26082 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:09.700186 21756 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:09:09.716965 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:09.721143 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 190045us
I20260709 13:09:09.703229 25572 raft_consensus.cc:2749] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:09:09.653400 26018 consensus_queue.cc:1059] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.063s
I20260709 13:09:09.731869 22994 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:09.734009 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:09.735322 23024 raft_consensus.cc:1240] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:09:09.735365 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:09.736637 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:09.736558 23024 raft_consensus.cc:1240] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:09.736961 25812 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.737568 25812 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:09.737869 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:09.738250 23024 raft_consensus.cc:1240] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 5. Current term is 6. Ops: []
I20260709 13:09:09.738502 22951 raft_consensus.cc:2393] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 7: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:09.739594 25812 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:09.739768 23024 raft_consensus.cc:1275] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:09:09.740921 22951 raft_consensus.cc:1240] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:09:09.741109 23024 raft_consensus.cc:1275] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:09:09.716312 25840 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.760861 23027 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 1730) took 1454 ms. Trace:
W20260709 13:09:09.728937 26149 log.cc:927] Time spent T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 13:09:09.730307 26030 consensus_queue.cc:1048] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:09:09.697491 26043 consensus_queue.cc:260] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:09.761338 23027 rpcz_store.cc:276] 0709 13:09:08.305944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:08.437816 (+131872us) service_pool.cc:224] Handling call
0709 13:09:08.687375 (+249559us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:08.701595 (+ 14220us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:09.757759 (+1056164us) spinlock_profiling.cc:243] Waited 247 ms on lock 0x7b54001f72d0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:09.760794 (+  3035us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":567282944}
I20260709 13:09:09.731024 25593 consensus_queue.cc:1059] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:09:09.813248 22843 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:09.813165 22918 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:09.815754 22918 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:09.764077 25963 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:09:09.823328 22843 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:09.819551 25812 raft_consensus.cc:2804] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:09.820089 26136 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
W20260709 13:09:09.823757 26134 log.cc:927] Time spent T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:09:09.819695 26074 raft_consensus.cc:2749] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:09.834968 26007 consensus_queue.cc:1059] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 13:09:09.820920 26016 consensus_queue.cc:1059] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.019s
I20260709 13:09:09.837216 26067 consensus_queue.cc:1048] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 13:09:09.840927 25968 consensus_queue.cc:1048] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:09:09.832158 26084 raft_consensus.cc:2749] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:09:09.844028 26103 log.cc:927] Time spent T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 1.209s	user 0.003s	sys 0.000s
I20260709 13:09:09.826288 25874 consensus_queue.cc:1059] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.020s
W20260709 13:09:09.846055 26096 log.cc:927] Time spent T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 1.094s	user 0.004s	sys 0.000s
I20260709 13:09:09.846768 25874 raft_consensus.cc:3055] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:09:09.847262 25874 raft_consensus.cc:740] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:09.849057 22950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 1739) took 1368 ms. Trace:
W20260709 13:09:09.850392 26153 log.cc:927] Time spent T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.326s	user 0.001s	sys 0.000s
I20260709 13:09:09.849464 22950 rpcz_store.cc:276] 0709 13:09:08.480344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:08.481854 (+  1510us) service_pool.cc:224] Handling call
0709 13:09:08.482126 (+   272us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:08.515017 (+ 32891us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:08.515043 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:08.515574 (+   531us) log.cc:844] Serialized 52 byte log entry
0709 13:09:08.617715 (+102141us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:09:08.617768 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:08.617860 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:09.843428 (+1225568us) spinlock_profiling.cc:243] Waited 55 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:09:09.848741 (+  5313us) raft_consensus.cc:1685] finished
0709 13:09:09.848771 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:09.848997 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11955,"raft.queue_time_us":80,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":3346,"spinlock_wait_cycles":126568192,"thread_start_us":49381,"threads_started":1,"wal-append.queue_time_us":93947}
I20260709 13:09:09.819362 26015 consensus_queue.cc:1048] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:09:09.853050 25707 raft_consensus.cc:3055] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:09:09.853593 25707 raft_consensus.cc:740] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:09:09.854153 26150 log.cc:927] Time spent T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.200s	user 0.001s	sys 0.000s
I20260709 13:09:09.827415 25812 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:09.824527 26125 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.761703 26076 consensus_queue.cc:237] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:09.824339 26097 raft_consensus.cc:2749] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 13:09:09.854871 25812 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:09.855088 26125 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:09.861146 26007 raft_consensus.cc:3055] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:09:09.833976 25572 raft_consensus.cc:2749] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:09:09.817435 25840 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:09.868287 22843 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:09.867519 26007 raft_consensus.cc:740] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:09.873564 26097 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:09:09.854768 25707 consensus_queue.cc:260] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:09.848424 25874 consensus_queue.cc:260] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:09.859018 26136 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:09.873824 25595 consensus_queue.cc:1048] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 13:09:09.923349 26007 consensus_queue.cc:260] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:09.924233 25707 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:09.924535 25840 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:09.924732 26007 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:09.924993 25874 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:09.874717 25968 consensus_queue.cc:1059] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
W20260709 13:09:09.961789 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:09.922714 26097 raft_consensus.cc:697] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 7 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:09.968811 25910 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.970619 25781 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:09.991425 25910 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:09.992949 25781 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:10.028046 26156 log.cc:927] Time spent T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
I20260709 13:09:09.998054 26141 tablet_copy_client.cc:323] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.63.2:33309
W20260709 13:09:10.037169 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:10.040524 26136 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:10.041383 26125 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:10.028450 26097 consensus_queue.cc:237] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:10.050220 25883 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:09:10.079988 22918 leader_election.cc:336] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.587s (SENT)
I20260709 13:09:10.080566 22918 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:10.093745 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:10.057446 25812 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:10.112329 25910 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:10.117511 22951 raft_consensus.cc:1275] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:09:10.119285 22951 raft_consensus.cc:1275] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:09:10.124078 26172 consensus_queue.cc:1048] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:09:10.125171 26076 consensus_queue.cc:1048] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:09:10.104523 26084 raft_consensus.cc:2749] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:09:10.145705 22993 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:10.153733 22951 raft_consensus.cc:1240] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 7. Current term is 8. Ops: []
I20260709 13:09:10.150815 26125 raft_consensus.cc:2804] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:09:10.149559 25883 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.150691 26136 consensus_queue.cc:1048] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.014s
I20260709 13:09:10.179491 26136 consensus_queue.cc:1243] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer 5b346767a89745fa82928a5dac9cc0b2 log is divergent from this leader: its last log entry 8.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:09:10.182108 25781 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:10.176179 26072 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:10.188614 25812 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:10.189481 26033 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:10.189612 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 109661us
W20260709 13:09:10.190994 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:10.163019 26125 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:10.195065 22992 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:09:10.192176 26092 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.258s	user 0.012s	sys 0.008s
I20260709 13:09:10.206354 22992 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:10.207415 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:10.213690 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:10.232335 22950 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
W20260709 13:09:10.212383 26092 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.278s	user 0.012s	sys 0.009s
I20260709 13:09:10.184195 25883 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:10.213768 26082 consensus_meta.cc:287] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.508s	user 0.014s	sys 0.000s
I20260709 13:09:10.243237 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:10.207451 26125 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:10.242028 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:10.231412 26047 raft_consensus.cc:2749] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 13:09:10.210489 26136 raft_consensus.cc:2749] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:10.231141 26068 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.249898 22992 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:10.203074 26049 consensus_queue.cc:1048] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 13:09:10.254613 26045 raft_consensus.cc:2804] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:09:10.256930 25883 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:10.263087 26140 raft_consensus.cc:3055] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:09:10.263568 26140 raft_consensus.cc:740] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:10.269582 25781 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.259320 26068 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:10.270177 25781 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:10.275574 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:10.285233 23623 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 665) took 1166 ms. Trace:
W20260709 13:09:10.287465 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 51760.1us
I20260709 13:09:10.261821 26045 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:10.285668 23623 rpcz_store.cc:276] 0709 13:09:09.118994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:09.119591 (+   597us) service_pool.cc:224] Handling call
0709 13:09:09.119879 (+   288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:09.354630 (+234751us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:10.284977 (+930347us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b54003d90d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:10.285177 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":468221824}
I20260709 13:09:10.296603 26094 consensus_queue.cc:1059] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:10.302981 26192 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:10.303643 22954 raft_consensus.cc:1275] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:09:10.303817 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:10.305878 23618 raft_consensus.cc:1275] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
I20260709 13:09:10.307725 23618 pending_rounds.cc:85] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2: Aborting all ops after (but not including) 5
I20260709 13:09:10.307996 23618 pending_rounds.cc:107] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 13:09:10.308233 23618 raft_consensus.cc:2889] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:10.308764 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:10.309921 23624 raft_consensus.cc:1240] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 3. Current term is 4. Ops: []
I20260709 13:09:10.313959 22993 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:10.315074 26068 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:10.315980 23622 raft_consensus.cc:1240] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:10.316007 22949 raft_consensus.cc:1275] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 13:09:10.304737 26045 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:10.312582 22954 raft_consensus.cc:1275] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:09:10.304867 26094 raft_consensus.cc:3055] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:09:10.325433 22952 raft_consensus.cc:1275] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 13:09:10.319111 26154 log.cc:927] Time spent T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.496s	user 0.000s	sys 0.002s
I20260709 13:09:10.264540 26140 consensus_queue.cc:260] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:10.326042 26136 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:10.348387 22952 raft_consensus.cc:1275] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:09:10.329744 26140 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:10.326733 26094 raft_consensus.cc:740] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:10.310704 26192 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:10.363292 26043 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:10.364123 26192 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:10.328637 26066 consensus_queue.cc:1048] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 13:09:10.356266 26042 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:10.357208 25781 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:10.316164 26191 consensus_queue.cc:1059] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
W20260709 13:09:10.374552 22994 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:10.383608 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:10.385671 23023 raft_consensus.cc:1275] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:09:10.386200 23025 raft_consensus.cc:1275] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:09:10.388512 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:10.389071 23028 raft_consensus.cc:2376] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 8. Re-sending same reply.
I20260709 13:09:10.389820 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:10.390339 23028 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
I20260709 13:09:10.402550 22874 raft_consensus.cc:1275] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:09:10.403034 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:10.403620 23026 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
I20260709 13:09:10.404739 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:10.404994 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:10.405519 23624 raft_consensus.cc:1275] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:09:10.405514 23026 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:09:10.407447 23026 raft_consensus.cc:1275] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:09:10.409969 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:10.410537 23029 raft_consensus.cc:2468] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 6.
I20260709 13:09:10.410405 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:10.410830 23027 raft_consensus.cc:1275] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:09:10.414476 23024 raft_consensus.cc:1275] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:09:10.414775 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:10.417940 22874 raft_consensus.cc:1240] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 6. Current term is 7. Ops: []
I20260709 13:09:10.421100 22874 raft_consensus.cc:1275] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:09:10.426033 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:10.431841 25876 raft_consensus.cc:3055] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:09:10.432412 25876 raft_consensus.cc:740] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:10.433538 25876 consensus_queue.cc:260] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:10.435031 25876 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:09:10.448447 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:10.451825 22826 auto_rebalancer.cc:629] Failed to schedule move for tablet 80ab3f0c16e747e3b1917985c734eed4: Illegal state: Replica 0a407504ed1741bf89a276558adf4c0f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 8 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
W20260709 13:09:10.452584 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:10.454169 23590 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:10.454873 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:10.455061 23590 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:10.456194 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:10.456892 25883 raft_consensus.cc:2804] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:10.457233 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:10.457731 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:10.457593 25572 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:10.458488 26192 raft_consensus.cc:2804] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:09:10.458913 25595 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:10.454375 22876 raft_consensus.cc:1240] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 5. Current term is 6. Ops: []
I20260709 13:09:10.459805 22872 raft_consensus.cc:1275] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:09:10.459556 25595 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:10.460206 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:10.457335 25883 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:10.460697 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:10.460953 25883 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:10.461198 22877 raft_consensus.cc:1240] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:10.461434 22872 raft_consensus.cc:2393] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 11: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:10.461777 25595 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:10.462183 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:10.462273 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:10.463124 22872 raft_consensus.cc:2410] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 6 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 13:09:10.463205 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:10.463812 22877 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:10.463994 22871 raft_consensus.cc:1275] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:09:10.458205 25572 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:10.464929 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:10.465422 23624 raft_consensus.cc:1275] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:09:10.465617 22871 raft_consensus.cc:2393] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 6: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:10.465888 26038 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.466207 25572 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:10.464395 25599 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.467172 23616 raft_consensus.cc:1240] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:09:10.467128 25599 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:10.459100 22878 raft_consensus.cc:2955] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Committing config change with OpId 6.5: config changed from index -1 to 5, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:09:10.469333 25599 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:10.469594 26074 raft_consensus.cc:3055] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:09:10.470026 26074 raft_consensus.cc:740] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:10.471030 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:10.470827 26074 consensus_queue.cc:260] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:09:10.471719 22876 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 9.
I20260709 13:09:10.472219 26074 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:10.458343 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:10.466511 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:10.458974 26192 raft_consensus.cc:697] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:10.472630 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:10.473187 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:10.456936 23619 raft_consensus.cc:2393] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 7: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:10.477137 23022 raft_consensus.cc:1275] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:09:10.473541 22871 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 9.
W20260709 13:09:10.477715 26173 log.cc:927] Time spent T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.299s	user 0.003s	sys 0.000s
W20260709 13:09:10.476063 26165 log.cc:927] Time spent T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.335s	user 0.000s	sys 0.002s
I20260709 13:09:10.500648 22871 raft_consensus.cc:1275] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:09:10.476372 22872 raft_consensus.cc:1275] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:09:10.481040 26087 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:10.480737 26192 consensus_queue.cc:237] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:10.511291 26016 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.456303 23021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33418 (request call id 1684) took 1066 ms. Trace:
W20260709 13:09:10.498042 26151 log.cc:927] Time spent T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:09:10.496152 22948 raft_consensus.cc:1240] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 13:09:10.497903 22874 raft_consensus.cc:1275] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:09:10.491257 22876 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
W20260709 13:09:10.478111 23588 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:10.519394 23588 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:10.423120 25781 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:10.500198 22877 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 10.
I20260709 13:09:10.466396 26038 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:10.504078 22871 raft_consensus.cc:1275] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:09:10.503798 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:10.509203 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:10.509266 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:10.529222 23024 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 11.
W20260709 13:09:10.512871 26181 log.cc:927] Time spent T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.200s	user 0.001s	sys 0.000s
I20260709 13:09:10.514469 23021 rpcz_store.cc:276] 0709 13:09:09.390008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:09.429304 (+ 39296us) service_pool.cc:224] Handling call
0709 13:09:10.456134 (+1026830us) spinlock_profiling.cc:243] Waited 249 ms on lock 0x7b5400359154. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:10.456250 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":573387520}
I20260709 13:09:10.494870 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:10.515825 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:10.537554 22873 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
I20260709 13:09:10.538610 23590 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:10.482915 26069 consensus_queue.cc:1048] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 13:09:10.522933 23588 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:10.504223 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:10.523414 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:10.545298 22947 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 9.
I20260709 13:09:10.523684 23020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 1688) took 1132 ms. Trace:
I20260709 13:09:10.546452 23020 rpcz_store.cc:276] 0709 13:09:09.391432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:09.463439 (+ 72007us) service_pool.cc:224] Handling call
0709 13:09:09.463926 (+   487us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:09:09.464495 (+   569us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:09.464523 (+    28us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:09:09.485209 (+ 20686us) log.cc:844] Serialized 78 byte log entry
0709 13:09:09.544549 (+ 59340us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:09:09.670187 (+125638us) log.cc:844] Serialized 14 byte log entry
0709 13:09:09.672438 (+  2251us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:09.673314 (+   876us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:10.318475 (+645161us) spinlock_profiling.cc:243] Waited 1.36 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:09:10.483716 (+165241us) raft_consensus.cc:1685] finished
0709 13:09:10.523486 (+ 39770us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:10.523631 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":124475,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":537,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":219,"spinlock_wait_cycles":3130240,"thread_start_us":47209,"threads_started":1,"wal-append.queue_time_us":233772}
W20260709 13:09:10.490150 26148 log.cc:927] Time spent T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:09:10.536728 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:10.543556 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:10.548681 22948 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 9.
I20260709 13:09:10.550374 22953 raft_consensus.cc:1275] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:09:10.426568 26015 consensus_queue.cc:1048] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.089s
I20260709 13:09:10.491515 26058 consensus_queue.cc:1048] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260709 13:09:10.526824 26068 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.527215 26082 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.520973 26016 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:10.561687 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:10.605731 22948 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 9.
I20260709 13:09:10.426671 26094 consensus_queue.cc:260] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:10.562218 22994 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:10.600250 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:10.606896 25923 raft_consensus.cc:2749] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:09:10.607024 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 44889.9us
I20260709 13:09:10.599685 22919 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:10.603087 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:10.618089 22953 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 9.
I20260709 13:09:10.562299 22952 raft_consensus.cc:1275] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:09:10.618871 26016 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:10.557480 26015 consensus_queue.cc:1243] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer 5b346767a89745fa82928a5dac9cc0b2 log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:09:10.619872 26208 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:10.620352 26208 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:10.620729 26208 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:10.624626 26176 consensus_queue.cc:1048] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:09:10.625360 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:10.625766 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 82763.4us
I20260709 13:09:10.536135 25781 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:10.558318 22965 tablet_copy_service.cc:140] P aed3f089ec8b49af9c8cf5cdee76e8e7: Received BeginTabletCopySession request for tablet 7426119f4da44ffaa1513b481a866a01 from peer 5b346767a89745fa82928a5dac9cc0b2 ({username='slave'} at 127.0.0.1:52222)
W20260709 13:09:10.628223 26173 log.cc:927] Time spent T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.071s	user 0.001s	sys 0.004s
I20260709 13:09:10.675711 23590 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:10.561928 25572 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:10.680626 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 52573.9us
I20260709 13:09:10.673842 26074 consensus_queue.cc:1059] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:09:10.562206 25876 consensus_queue.cc:1048] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.113s
I20260709 13:09:10.736570 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:10.628738 26080 raft_consensus.cc:2749] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:09:10.737874 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:10.781109 22871 raft_consensus.cc:2410] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 9 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 13:09:10.739761 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:10.740423 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 56310.3us
I20260709 13:09:10.599454 25707 consensus_queue.cc:1048] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.090s
I20260709 13:09:10.775975 23029 raft_consensus.cc:2393] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 8: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:10.737049 22918 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:10.779036 23028 raft_consensus.cc:1275] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:09:10.676577 26016 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.612304 26082 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:10.732378 22965 tablet_copy_service.cc:161] P aed3f089ec8b49af9c8cf5cdee76e8e7: Beginning new tablet copy session on tablet 7426119f4da44ffaa1513b481a866a01 from peer 5b346767a89745fa82928a5dac9cc0b2 at {username='slave'} at 127.0.0.1:52222: session id = 5b346767a89745fa82928a5dac9cc0b2-7426119f4da44ffaa1513b481a866a01
I20260709 13:09:10.605779 26068 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:10.630865 26073 consensus_queue.cc:1059] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.158s
W20260709 13:09:10.612583 26076 consensus_queue.cc:1175] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f3c799b1249e40119d2a5864231c6c75" responder_term: 7 status { last_received { term: 7 index: 6 } last_committed_idx: 3 last_received_current_leader { term: 7 index: 6 } }
I20260709 13:09:10.776314 25599 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.690673 26047 consensus_queue.cc:1048] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.120s
I20260709 13:09:10.780850 26069 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.815060 26080 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:10.827316 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:10.807523 26097 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.823321 26074 raft_consensus.cc:2804] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:10.850725 22843 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:10.826359 26210 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:10.779345 25572 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:10.809147 26016 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:10.874959 26140 consensus_meta.cc:287] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.517s	user 0.008s	sys 0.000s
W20260709 13:09:10.852569 26194 log.cc:927] Time spent T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.352s	user 0.001s	sys 0.000s
I20260709 13:09:10.880766 22843 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:10.854475 26074 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
W20260709 13:09:10.870428 26196 log.cc:927] Time spent T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.266s	user 0.001s	sys 0.000s
I20260709 13:09:10.851703 26058 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.855036 26073 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:10.861631 26210 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:10.830852 25599 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:10.840281 26069 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:10.897377 25595 raft_consensus.cc:2749] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:09:10.851424 26080 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:10.851598 26097 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:10.917873 26087 raft_consensus.cc:2749] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:10.918912 26038 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:10.974785 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 155117us
I20260709 13:09:10.926564 26074 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:10.899595 26211 consensus_queue.cc:1048] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.099s
I20260709 13:09:10.913829 25883 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:11.031176 26199 log.cc:927] Time spent T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.432s	user 0.001s	sys 0.000s
I20260709 13:09:11.035010 26214 raft_consensus.cc:2804] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:09:11.060232 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 323247us
W20260709 13:09:11.036538 26194 log.cc:927] Time spent T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.106s	user 0.000s	sys 0.000s
I20260709 13:09:10.974129 26210 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:11.029135 26016 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:11.031635 26202 log.cc:927] Time spent T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.410s	user 0.001s	sys 0.000s
I20260709 13:09:11.088027 25570 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.088631 25570 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:11.090759 25570 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:11.047248 23588 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.009s	sys 0.002s
W20260709 13:09:11.091820 23588 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.010s	sys 0.002s
W20260709 13:09:11.092351 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 569422us
I20260709 13:09:11.093132 23588 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:11.094223 23588 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:11.097543 26210 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:11.098042 26042 consensus_meta.cc:287] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.728s	user 0.010s	sys 0.000s
I20260709 13:09:11.098654 22946 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 4.
I20260709 13:09:11.100345 26210 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:11.101536 26210 raft_consensus.cc:2804] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:09:11.101794 25874 raft_consensus.cc:3055] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:09:11.101887 26210 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:11.102213 25874 raft_consensus.cc:740] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:11.102316 26210 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:11.103065 25874 consensus_queue.cc:260] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:11.104323 25874 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:11.080461 22918 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:11.109491 22918 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:11.109994 25599 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:11.110780 26176 raft_consensus.cc:2804] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:09:11.110987 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:11.111240 26176 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:11.111601 26176 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:09:11.116554 26205 log.cc:927] Time spent T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.205s	user 0.001s	sys 0.000s
I20260709 13:09:11.034391 25781 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:11.058266 22965 tablet_copy_source_session.cc:215] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:09:11.058503 26214 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:11.083830 26201 log.cc:927] Time spent T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.270s	user 0.000s	sys 0.001s
I20260709 13:09:11.117722 25968 raft_consensus.cc:2804] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:11.132505 26176 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:11.134830 25883 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:11.135857 26210 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:11.136497 26080 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:11.137259 26218 raft_consensus.cc:2749] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:09:11.175382 26210 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:11.139031 26097 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:11.049649 25595 consensus_queue.cc:1048] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.090s
I20260709 13:09:11.164954 25968 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:11.176785 25968 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:11.166326 25599 consensus_queue.cc:1048] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.014s
W20260709 13:09:11.177901 26045 consensus_meta.cc:287] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.833s	user 0.010s	sys 0.000s
I20260709 13:09:11.166507 26042 consensus_queue.cc:1048] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.012s
I20260709 13:09:11.178431 26045 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:11.182297 26176 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:11.164829 25964 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.184976 25964 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:11.187294 25964 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:11.142830 25572 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:11.143584 26068 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:11.145294 26082 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:10.977137 26058 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:11.194562 26058 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:11.035784 26212 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.195358 26212 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:11.197467 26212 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:11.155824 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 235278us
I20260709 13:09:10.977311 26073 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:11.148437 26125 consensus_meta.cc:287] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.889s	user 0.009s	sys 0.000s
I20260709 13:09:11.048933 26038 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.124182 25910 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:11.153102 26043 consensus_meta.cc:287] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.789s	user 0.010s	sys 0.002s
I20260709 13:09:11.165961 26214 raft_consensus.cc:3060] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:11.141403 26069 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:11.202316 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 419851us
I20260709 13:09:11.211642 23590 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:11.189254 26045 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:11.198630 26091 raft_consensus.cc:3055] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:09:11.203887 26125 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:11.220515 26218 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:11.224092 26125 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:11.224457 26218 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:11.226189 26073 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:11.225994 26074 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:11.226552 26218 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:11.208755 25910 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:11.227897 26074 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:11.221251 26091 raft_consensus.cc:740] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:11.218652 26082 consensus_queue.cc:1048] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:09:11.229084 26091 consensus_queue.cc:260] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:11.204268 26038 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:11.230206 26091 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:11.231395 26038 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:11.220719 26210 consensus_queue.cc:1048] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 13:09:11.224874 25883 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.183779 26208 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:11.214746 26192 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.700s	user 0.006s	sys 0.004s
I20260709 13:09:11.218448 26080 raft_consensus.cc:2804] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:09:11.221499 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:11.235764 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 141614us
I20260709 13:09:11.235986 25883 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:11.237882 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:11.243518 23024 raft_consensus.cc:2376] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 8. Re-sending same reply.
I20260709 13:09:11.238098 23027 raft_consensus.cc:1275] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:09:11.239334 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:11.244572 23029 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 8.
I20260709 13:09:11.240198 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:11.234423 26021 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
W20260709 13:09:11.235634 26192 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.722s	user 0.006s	sys 0.004s
I20260709 13:09:11.247489 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:11.254871 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:11.255558 23620 raft_consensus.cc:2410] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:09:11.245515 23020 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 8.
I20260709 13:09:11.258957 23588 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:11.260113 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:11.260934 23024 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 8.
I20260709 13:09:11.261200 25883 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:11.261941 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:11.255721 26082 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.266883 26082 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:11.269212 26082 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:11.255916 26080 raft_consensus.cc:697] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 10 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:11.271679 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:11.271252 26080 consensus_queue.cc:237] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:11.272395 23619 raft_consensus.cc:2393] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 10: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:11.242882 26045 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:11.278424 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:11.278026 25964 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:11.281172 25964 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:11.289216 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:11.291299 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:11.292382 23024 raft_consensus.cc:2410] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:09:11.293063 23619 raft_consensus.cc:2393] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 7: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:11.279711 26192 raft_consensus.cc:2749] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:09:11.257359 26021 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:11.264879 26218 consensus_queue.cc:1059] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 13:09:11.305207 23621 pending_rounds.cc:85] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2: Aborting all ops after (but not including) 8
I20260709 13:09:11.306481 23621 pending_rounds.cc:107] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 13:09:11.306735 23621 raft_consensus.cc:2889] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:09:11.308890 22845 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.710s	user 0.014s	sys 0.039s
W20260709 13:09:11.309667 22845 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.711s	user 0.015s	sys 0.039s
W20260709 13:09:11.308890 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 49982.8us
I20260709 13:09:11.310828 23588 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:11.312636 26058 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.312775 26208 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:11.313226 26058 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:11.315258 26058 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:11.289762 26045 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:11.318827 22949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2340) took 1046 ms. Trace:
I20260709 13:09:11.321592 26192 raft_consensus.cc:3055] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:09:11.322110 26192 raft_consensus.cc:740] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
W20260709 13:09:11.325950 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:11.319226 22949 rpcz_store.cc:276] 0709 13:09:10.272365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:10.321966 (+ 49601us) service_pool.cc:224] Handling call
0709 13:09:10.345575 (+ 23609us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:10.346400 (+   825us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:10.346446 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:10.353322 (+  6876us) log.cc:844] Serialized 52 byte log entry
0709 13:09:10.431242 (+ 77920us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:09:10.975680 (+544438us) log.cc:844] Serialized 14 byte log entry
0709 13:09:11.093876 (+118196us) spinlock_profiling.cc:243] Waited 52.2 ms on lock 0x7b18003de580. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cf83f3 00007f6132cfa7c1 00007f6132cfa779 00007f6132cfa709 00007f6132cfa6d1 00007f6132cf99cd 000000000038f354 000000000038f199 00007f6132cdb83a
0709 13:09:11.318155 (+224279us) spinlock_profiling.cc:243] Waited 51.8 ms on lock 0x7b18003de580. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132c72c0a 00007f613c3b2219 00007f6132cfaa40 00007f6132cf54e4 00007f6132cf5455 00007f61397dd3c7 00007f6135def9a7 00007f6135dff0bf 00007f6135dff038
0709 13:09:11.318359 (+   204us) log.cc:844] Serialized 14 byte log entry
0709 13:09:11.318547 (+   188us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:11.318615 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:11.318651 (+    36us) raft_consensus.cc:1685] finished
0709 13:09:11.318662 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:11.318770 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":495075,"prepare.queue_time_us":547646,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":1572,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":203,"spinlock_wait_cycles":239173248,"thread_start_us":37960,"threads_started":1,"wal-append.queue_time_us":190810}
I20260709 13:09:11.330794 23619 pending_rounds.cc:85] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2: Aborting all ops after (but not including) 7
I20260709 13:09:11.331089 23619 pending_rounds.cc:107] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:09:11.331321 23619 raft_consensus.cc:2889] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:11.330538 26225 consensus_queue.cc:1048] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:09:11.332381 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:11.334450 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:11.335233 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:11.335348 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:11.335958 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:11.337163 22954 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 8.
I20260709 13:09:11.338483 22947 raft_consensus.cc:2410] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:09:11.359666 23588 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:11.360395 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:11.361775 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:11.362924 22954 raft_consensus.cc:2410] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
W20260709 13:09:11.363731 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:11.364593 26038 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.364804 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:11.365147 26038 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:11.366458 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:11.366700 26069 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.367184 22952 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:11.367528 26082 raft_consensus.cc:2804] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:11.367986 26082 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:11.368342 26082 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:11.368468 23588 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:11.369556 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:11.343572 22920 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:11.294039 25968 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:11.376032 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:11.377511 22951 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 9.
I20260709 13:09:11.353229 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:11.378962 22953 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 8.
I20260709 13:09:11.380483 22948 raft_consensus.cc:1275] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:09:11.381906 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:11.383688 23617 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:09:11.379235 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:11.385279 22879 raft_consensus.cc:1275] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:09:11.386214 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:11.388149 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:11.382329 22949 raft_consensus.cc:1275] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:09:11.382699 22950 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:11.389577 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:11.390753 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:11.397025 22949 pending_rounds.cc:85] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting all ops after (but not including) 7
I20260709 13:09:11.397356 22949 pending_rounds.cc:107] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:09:11.397682 22949 raft_consensus.cc:2889] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:11.401908 22873 raft_consensus.cc:1275] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:09:11.318840 26045 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:11.407546 22920 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:11.323165 25964 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:11.336803 26084 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.338210 22955 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:11.411767 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:11.360867 26210 raft_consensus.cc:2804] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:11.414609 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:11.415217 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:11.415479 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:11.428848 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:11.415843 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:11.416607 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:11.430600 22876 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:11.416776 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:11.425671 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:11.426321 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:11.431797 22879 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 9.
I20260709 13:09:11.433645 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:11.431957 23027 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 8.
I20260709 13:09:11.435521 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:11.435782 23617 raft_consensus.cc:1240] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:11.436043 23027 raft_consensus.cc:2468] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6.
I20260709 13:09:11.367204 26038 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:11.436851 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:11.427546 22872 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
I20260709 13:09:11.450084 23027 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 8.
I20260709 13:09:11.446655 22992 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:11.444223 22876 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 10.
I20260709 13:09:11.450903 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:11.387579 26138 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.453207 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:11.403501 26074 raft_consensus.cc:2749] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:09:11.333073 26214 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:11.453984 23616 raft_consensus.cc:2393] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 5: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:11.454144 23027 raft_consensus.cc:2393] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 5: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:11.462174 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:11.463032 23027 raft_consensus.cc:2410] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 12 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
W20260709 13:09:11.457314 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 49850.1us
I20260709 13:09:11.464023 22920 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:11.453778 22952 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 10.
I20260709 13:09:11.338223 26010 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:11.412560 26058 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:11.367208 26069 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:11.427232 26210 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:09:11.486686 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 366650us
I20260709 13:09:11.508198 22873 raft_consensus.cc:2393] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 7: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:11.474534 23617 raft_consensus.cc:1240] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:09:11.409684 26091 consensus_queue.cc:1048] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.064s
I20260709 13:09:11.448724 26172 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.523097 22879 raft_consensus.cc:1240] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 3. Current term is 4. Ops: []
I20260709 13:09:11.453754 26224 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.422902 26073 consensus_queue.cc:1059] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 13:09:11.422644 26176 consensus_queue.cc:1048] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 13:09:11.526964 22955 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 10.
I20260709 13:09:11.476446 25926 raft_consensus.cc:2749] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:09:11.535089 22878 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 5.
I20260709 13:09:11.483340 26038 raft_consensus.cc:2749] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:09:11.486197 25707 consensus_queue.cc:1048] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 13:09:11.425384 26084 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:11.506352 26058 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:11.547531 26084 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:11.547904 26058 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:11.518227 26210 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:11.449348 26226 consensus_queue.cc:1048] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.103s
I20260709 13:09:11.487852 26074 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:11.481972 26140 consensus_queue.cc:1048] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260709 13:09:11.482410 26127 consensus_queue.cc:1048] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 13:09:11.532143 25572 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.485946 26049 consensus_queue.cc:1048] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 13:09:11.473902 26138 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:11.587814 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:11.486318 26010 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:11.548293 26084 raft_consensus.cc:2749] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:11.570250 26226 consensus_queue.cc:1243] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer aed3f089ec8b49af9c8cf5cdee76e8e7 log is divergent from this leader: its last log entry 9.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:09:11.534626 26172 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:11.455856 26192 consensus_queue.cc:260] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:11.585253 26058 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.584199 26038 consensus_queue.cc:1048] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 13:09:11.595368 26222 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.539862 26224 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:11.583665 26074 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:11.782905 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:11.790017 22949 pending_rounds.cc:85] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting all ops after (but not including) 1
I20260709 13:09:11.790390 22949 pending_rounds.cc:107] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 13:09:11.790683 22949 raft_consensus.cc:2889] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:09:11.792955 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:11.795063 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 315594us
I20260709 13:09:11.704631 26021 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:11.801631 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:11.799046 22920 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:11.712785 26097 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.593513 25572 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:11.805264 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:11.802320 22947 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:09:11.634191 26091 consensus_queue.cc:1048] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.165s
I20260709 13:09:11.880801 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:11.881052 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:09:11.809937 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 439883us
I20260709 13:09:11.773553 26176 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.826619 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:09:11.893599 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 94652.6us
I20260709 13:09:11.878664 26069 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:11.879582 26230 log.cc:927] Time spent T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.289s	user 0.000s	sys 0.000s
I20260709 13:09:11.881527 22947 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
I20260709 13:09:11.884999 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:11.774960 26058 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:11.887058 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:11.887874 22992 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:11.777136 26218 raft_consensus.cc:2804] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:11.773334 25964 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:11.886384 23588 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:11.918412 22992 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:11.792023 26222 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:11.807102 22952 raft_consensus.cc:1275] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:09:11.903699 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:11.989812 23621 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 678) took 1681 ms. Trace:
W20260709 13:09:12.003705 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 117392us
I20260709 13:09:12.004881 23588 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:11.919678 22948 raft_consensus.cc:2393] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 8: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:12.001116 23621 rpcz_store.cc:276] 0709 13:09:10.308728 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:10.309458 (+   730us) service_pool.cc:224] Handling call
0709 13:09:11.304843 (+995385us) spinlock_profiling.cc:243] Waited 1.11 ms on lock 0x7b58001c4d98. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0a22 00007f613bbd9c09 00007f613ad33ee0 00007f613adfd0ab 00007f613adfd131 00007f613adbefc0 00007f613ada9929 00007f613ada93d2 00007f6134320bd4
0709 13:09:11.305015 (+   172us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:11.359524 (+ 54509us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:11.359553 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:11.360075 (+   522us) log.cc:844] Serialized 52 byte log entry
0709 13:09:11.771743 (+411668us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:09:11.771799 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:11.771871 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:11.989230 (+217359us) raft_consensus.cc:1685] finished
0709 13:09:11.989249 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:11.989727 (+   478us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43331,"raft.queue_time_us":49814,"spinlock_wait_cycles":20733696,"thread_start_us":400981,"threads_started":1,"wal-append.queue_time_us":508013}
I20260709 13:09:11.883018 26211 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:11.827858 25968 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:12.016986 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 98628.6us
W20260709 13:09:11.914886 26199 log.cc:927] Time spent T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 13:09:12.026292 22992 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:11.896822 26140 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:11.949846 26021 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:11.919268 26016 raft_consensus.cc:2749] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:12.040529 23623 raft_consensus.cc:1240] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 10 LEADER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:11.960955 26214 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:11.880322 26097 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:12.016572 25572 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:12.017247 26091 raft_consensus.cc:2804] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:11.992425 26236 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:12.016871 26138 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:12.021601 26049 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:12.016192 26233 raft_consensus.cc:2749] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:12.018805 26127 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:12.022658 26010 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:12.032691 26241 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:12.070233 26241 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:12.070567 26097 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:12.059280 26049 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:12.038465 25968 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:12.071626 26049 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:12.071942 25968 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:11.927515 26176 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:12.041214 25926 raft_consensus.cc:2804] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:09:12.060497 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 165738us
I20260709 13:09:12.015915 25964 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:12.068897 26127 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:12.075997 25968 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:12.046150 25781 raft_consensus.cc:2804] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:12.084772 22920 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:12.057559 26091 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:12.048163 26021 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:12.090389 22843 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:12.093861 22843 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:09:12.097518 23025 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.723s	user 0.010s	sys 0.012s
I20260709 13:09:12.076921 26239 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:12.045657 26140 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:12.078290 26138 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.080271 25926 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:12.059686 26016 consensus_queue.cc:1048] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
I20260709 13:09:12.033809 26211 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:12.099839 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:12.110649 25926 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:12.111275 26138 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:12.112054 25964 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:09:12.082054 26073 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.006s	sys 0.015s
W20260709 13:09:12.117328 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 112559us
W20260709 13:09:12.114285 23025 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.724s	user 0.010s	sys 0.012s
I20260709 13:09:12.087700 25781 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:12.086733 26091 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:12.091105 25968 raft_consensus.cc:2764] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:09:12.168851 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:12.061586 26236 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:12.169303 26244 log.cc:927] Time spent T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 13:09:12.203313 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:12.077425 26238 consensus_queue.cc:1048] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.054s
I20260709 13:09:12.171442 23588 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:09:12.216142 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 44768.4us
I20260709 13:09:12.217018 23588 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:12.116137 26239 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:12.172937 26235 log.cc:927] Time spent T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.129s	user 0.001s	sys 0.000s
I20260709 13:09:12.173192 25781 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:12.174463 26127 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:12.175274 26222 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:12.177614 26141 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7426119f4da44ffaa1513b481a866a01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:12.179140 26138 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:12.114499 26172 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:12.185813 26245 log.cc:927] Time spent T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 13:09:12.185659 26233 consensus_queue.cc:1048] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:09:12.188500 26224 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:09:12.160406 26073 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.006s	sys 0.016s
W20260709 13:09:12.189787 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 155918us
I20260709 13:09:12.191236 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:12.188033 26049 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:12.101272 25840 raft_consensus.cc:2804] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:12.085637 26074 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:12.204702 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 110884us
I20260709 13:09:12.098416 25595 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:12.207407 23022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 1696) took 1823 ms. Trace:
I20260709 13:09:12.220482 23616 raft_consensus.cc:1275] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:09:12.220639 22871 raft_consensus.cc:2393] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 7: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:12.221145 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:12.247022 23621 raft_consensus.cc:2468] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 6.
I20260709 13:09:12.247069 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:12.248404 25595 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:12.205237 26140 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:12.246289 23022 rpcz_store.cc:276] 0709 13:09:10.383922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:10.384742 (+   820us) service_pool.cc:224] Handling call
0709 13:09:10.476837 (+ 92095us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:10.483827 (+  6990us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:12.207123 (+1723296us) spinlock_profiling.cc:243] Waited 91.8 ms on lock 0x7b5400359150. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:12.207309 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":211189504}
I20260709 13:09:12.099097 26252 consensus_queue.cc:1048] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.123s
I20260709 13:09:12.226402 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:12.252152 23625 raft_consensus.cc:2393] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 9: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:12.252844 26176 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:12.203719 26241 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:12.133975 26058 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:12.229499 23619 raft_consensus.cc:1275] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:09:12.224133 25883 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:09:12.224023 26080 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:09:12.255645 22920 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.63.1:33929 timed out after 1.679s (SENT) [suppressed 52 similar messages]
I20260709 13:09:12.256153 23024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 1714) took 1017 ms. Trace:
I20260709 13:09:12.242641 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:12.258459 22879 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6.
I20260709 13:09:12.259368 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:12.259938 22879 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:12.245189 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:12.261389 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:12.256553 23024 rpcz_store.cc:276] 0709 13:09:11.238970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:11.297036 (+ 58066us) service_pool.cc:224] Handling call
0709 13:09:11.306303 (+  9267us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:11.335383 (+ 29080us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:11.335408 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:11.335822 (+   414us) log.cc:844] Serialized 26 byte log entry
0709 13:09:11.737534 (+401712us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:09:11.737591 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:11.737697 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:12.255953 (+518256us) raft_consensus.cc:1685] finished
0709 13:09:12.255975 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:12.256097 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":28910,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":22562,"thread_start_us":326880,"threads_started":1,"wal-append.queue_time_us":612165}
I20260709 13:09:12.269645 26211 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:12.274225 23625 raft_consensus.cc:1275] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:09:12.241122 26019 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:12.245620 25840 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:12.298687 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:12.420552 23623 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 684) took 1111 ms. Trace:
I20260709 13:09:12.584053 22919 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:12.257846 26256 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
W20260709 13:09:12.577298 22946 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58320 (request call id 2322) took 3580 ms (client timeout 1866 ms). Trace:
I20260709 13:09:12.592911 22873 raft_consensus.cc:1240] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
W20260709 13:09:12.375361 26082 consensus_meta.cc:287] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 1.006s	user 0.005s	sys 0.006s
I20260709 13:09:12.292213 26074 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:12.633128 25599 raft_consensus.cc:2749] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:12.365038 26021 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:12.633482 26082 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:12.634718 23624 raft_consensus.cc:1240] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 7. Current term is 8. Ops: []
W20260709 13:09:12.632879 22946 rpcz_store.cc:269] 0709 13:09:08.996483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:08.997550 (+  1067us) service_pool.cc:224] Handling call
0709 13:09:12.577077 (+3579527us) spinlock_profiling.cc:243] Waited 2.1 s on lock 0x7b540035d754. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:12.577210 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4829938048}
I20260709 13:09:12.635885 26082 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:12.577625 26042 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.633574 26074 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:12.580000 26189 raft_consensus.cc:3055] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:09:12.374744 25926 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:12.638286 26189 raft_consensus.cc:740] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:12.580138 22872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2267) took 1072 ms. Trace:
W20260709 13:09:12.406952 26210 consensus_meta.cc:287] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.823s	user 0.007s	sys 0.005s
W20260709 13:09:12.595691 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 340035us
I20260709 13:09:12.631049 26016 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.598114 23623 rpcz_store.cc:276] 0709 13:09:11.309419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:11.309541 (+   122us) service_pool.cc:224] Handling call
0709 13:09:11.309805 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:12.044193 (+734388us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:12.420206 (+376013us) spinlock_profiling.cc:243] Waited 730 ms on lock 0x7b540048bf54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:12.420465 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1679994112}
I20260709 13:09:12.640048 22872 rpcz_store.cc:276] 0709 13:09:11.507316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:12.579378 (+1072062us) service_pool.cc:224] Handling call
0709 13:09:12.579669 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:12.579841 (+   172us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:12.579864 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:12.579901 (+    37us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:09:12.579932 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:12.579983 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:12.580086 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:09:12.406160 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:12.633734 22873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2265) took 1154 ms. Trace:
I20260709 13:09:12.298887 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:12.645491 22879 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 9.
I20260709 13:09:12.637143 25781 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:12.636994 26042 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:12.598795 25840 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:09:12.363655 26260 log.cc:927] Time spent T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 13:09:12.579988 26038 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.658404 26038 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:09:12.659437 22877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2262) took 1180 ms. Trace:
I20260709 13:09:12.661001 26038 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:12.641844 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 49137.8us
I20260709 13:09:12.666196 23588 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:12.667323 23588 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:12.668155 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:12.668901 22946 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 10.
I20260709 13:09:12.669103 23026 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 1753) took 2279 ms. Trace:
I20260709 13:09:12.671209 26038 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.669659 23026 rpcz_store.cc:276] 0709 13:09:10.389476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:10.408383 (+ 18907us) service_pool.cc:224] Handling call
0709 13:09:10.408703 (+   320us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:10.423270 (+ 14567us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:09:10.493212 (+ 69942us) log.cc:844] Serialized 14 byte log entry
0709 13:09:11.106066 (+612854us) spinlock_profiling.cc:243] Waited 7.77 ms on lock 0x7b48001bc9d8. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613ad0eaa1 00007f613ad05b37 00007f613ad059ca 00007f613adfd26f 00007f613adfcf37 00007f613ae1155c 00007f613ae11508 00007f613ae11488 00007f613ae11443 00007f613ae106ec 00007f6139d3aa96
0709 13:09:12.203443 (+1097377us) spinlock_profiling.cc:243] Waited 1.57 ms on lock 0x7b1800407da0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cdae07 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf7c02 00007f613597f20b 00007f6135984d23 00007f613598523a 00007f6135defa2a
0709 13:09:12.232486 (+ 29043us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:12.237577 (+  5091us) log.cc:844] Serialized 26 byte log entry
0709 13:09:12.237715 (+   138us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:09:12.237747 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:12.237835 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:12.668755 (+430920us) raft_consensus.cc:1685] finished
0709 13:09:12.668791 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:12.668990 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14095,"prepare.queue_time_us":69940,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":34,"raft.queue_time_us":17670,"raft.run_cpu_time_us":13715,"raft.run_wall_time_us":623168,"spinlock_wait_cycles":21486720,"thread_start_us":24139,"threads_started":1,"wal-append.queue_time_us":1727704}
I20260709 13:09:12.671748 26038 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:12.606230 25883 raft_consensus.cc:697] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:12.673574 25883 consensus_queue.cc:237] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:12.674141 26038 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:12.642861 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:12.643124 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:12.639578 26189 consensus_queue.cc:260] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:12.364368 26257 log.cc:927] Time spent T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
W20260709 13:09:12.317361 26246 log.cc:927] Time spent T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:09:12.645133 22873 rpcz_store.cc:276] 0709 13:09:11.479290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:12.375452 (+896162us) service_pool.cc:224] Handling call
0709 13:09:12.592636 (+217184us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:12.633455 (+ 40819us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:12.633679 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:12.642796 26256 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:12.641134 26261 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.292213 26080 raft_consensus.cc:697] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:12.706061 26255 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.551483 26091 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:12.592743 26192 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:12.675681 23588 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:12.715562 23588 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:12.659853 22877 rpcz_store.cc:276] 0709 13:09:11.478568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:11.480090 (+  1522us) service_pool.cc:224] Handling call
0709 13:09:11.480328 (+   238us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:11.546910 (+ 66582us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:09:11.547197 (+   287us) log.cc:844] Serialized 14 byte log entry
0709 13:09:11.977740 (+430543us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:11.977815 (+    75us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:11.977881 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:12.658633 (+680752us) spinlock_profiling.cc:243] Waited 3.56 ms on lock 0x7b1800249280. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cc62a6 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419
0709 13:09:12.659194 (+   561us) spinlock_profiling.cc:243] Waited 66.4 ms on lock 0x7b54002d6c54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:12.659250 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:12.659383 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1492,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":1162,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":4006,"spinlock_wait_cycles":168884608,"thread_start_us":428834,"threads_started":1,"wal-append.queue_time_us":498958}
I20260709 13:09:12.716691 26233 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:09:12.716773 22994 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:12.717175 26233 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:12.717272 26262 raft_consensus.cc:2749] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:09:12.657060 26236 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:12.642294 26049 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:12.640069 25926 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:12.717566 26233 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:12.721901 26042 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:12.722208 26256 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:12.707389 26189 raft_consensus.cc:3060] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:12.703805 26208 raft_consensus.cc:2804] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:12.641664 26019 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:12.709417 26094 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:12.719930 23588 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:12.640605 26210 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:12.699643 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 82735.2us
I20260709 13:09:12.727032 22949 raft_consensus.cc:1240] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:09:12.731695 23588 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:12.677078 26016 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:12.709766 26264 raft_consensus.cc:2804] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:09:12.717653 26084 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:12.737246 22993 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:12.740916 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:12.745154 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:12.747502 23622 raft_consensus.cc:2410] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:09:12.726613 26208 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:12.706061 26261 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:12.717993 26080 consensus_queue.cc:237] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:12.735237 26262 raft_consensus.cc:2749] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:12.733700 26074 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:12.737661 26264 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:12.714943 26255 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:12.742590 26211 raft_consensus.cc:2804] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:12.759028 26208 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:12.772177 26268 raft_consensus.cc:2749] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:09:12.744905 26256 consensus_queue.cc:1059] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 13:09:12.746591 26084 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:12.787994 26016 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:12.826562 26233 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:12.770808 26211 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:12.828888 26211 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:12.771472 26264 raft_consensus.cc:3060] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:12.834008 26074 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:12.834645 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:12.834715 26074 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:12.794413 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:12.837010 22952 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:12.839321 26074 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:12.804247 26210 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:12.805569 25781 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:12.805924 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:12.805568 26084 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:12.812346 23028 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 1709) took 1574 ms. Trace:
I20260709 13:09:12.813517 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:12.814204 26255 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:12.816115 26261 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:12.824520 26269 raft_consensus.cc:3055] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:09:12.846225 26269 raft_consensus.cc:740] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:12.845257 23028 rpcz_store.cc:276] 0709 13:09:11.237713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:11.238393 (+   680us) service_pool.cc:224] Handling call
0709 13:09:11.238744 (+   351us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:11.242742 (+  3998us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:11.242775 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:11.262067 (+ 19292us) log.cc:844] Serialized 26 byte log entry
0709 13:09:12.418426 (+1156359us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b1800414d00. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cdae07 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf7c02 00007f613597f20b 00007f6135984d23 00007f61359850d1 00007f6135da0a0f
0709 13:09:12.718741 (+300315us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:09:12.718784 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:12.718853 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:12.811939 (+ 93086us) raft_consensus.cc:1685] finished
0709 13:09:12.811971 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:12.812259 (+   288us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17820,"raft.queue_time_us":4062,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":17263,"spinlock_wait_cycles":25410560,"thread_start_us":123296,"threads_started":1,"wal-append.queue_time_us":1507169}
I20260709 13:09:12.847105 26269 consensus_queue.cc:260] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:12.809477 26091 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:12.852290 26269 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:12.856070 23025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 1705) took 1619 ms. Trace:
I20260709 13:09:12.856503 23025 rpcz_store.cc:276] 0709 13:09:11.236478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:11.236610 (+   132us) service_pool.cc:224] Handling call
0709 13:09:11.238608 (+  1998us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:11.335367 (+ 96759us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:09:12.161065 (+825698us) log.cc:844] Serialized 14 byte log entry
0709 13:09:12.224767 (+ 63702us) log.cc:844] Serialized 14 byte log entry
0709 13:09:12.228383 (+  3616us) log.cc:844] Serialized 14 byte log entry
0709 13:09:12.322233 (+ 93850us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:12.322313 (+    80us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:09:12.322381 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:12.855756 (+533375us) spinlock_profiling.cc:243] Waited 96.5 ms on lock 0x7b5400358754. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:12.855796 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:12.856011 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3179,"prepare.queue_time_us":684801,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":3094,"raft.queue_time_us":9812,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":7623,"spinlock_wait_cycles":257811584,"thread_start_us":943588,"threads_started":3,"wal-append.queue_time_us":156726}
W20260709 13:09:12.857983 26229 log.cc:927] Time spent T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 1.059s	user 0.003s	sys 0.009s
W20260709 13:09:12.860244 26207 log.cc:927] Time spent T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 2.171s	user 0.009s	sys 0.005s
I20260709 13:09:12.861621 22875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2249) took 2466 ms. Trace:
I20260709 13:09:12.861958 22875 rpcz_store.cc:276] 0709 13:09:10.395024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:10.395964 (+   940us) service_pool.cc:224] Handling call
0709 13:09:10.417128 (+ 21164us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:10.422546 (+  5418us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:10.422573 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:10.429754 (+  7181us) log.cc:844] Serialized 26 byte log entry
0709 13:09:10.553987 (+124233us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:10.554030 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:10.554344 (+   314us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:12.859358 (+2305014us) spinlock_profiling.cc:243] Waited 7.26 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:09:12.861349 (+  1991us) raft_consensus.cc:1685] finished
0709 13:09:12.861369 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:12.861565 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5014,"raft.queue_time_us":5589,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":63017,"spinlock_wait_cycles":16686976,"thread_start_us":39285,"threads_started":1,"wal-append.queue_time_us":174465}
I20260709 13:09:12.870698 26218 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:12.871299 26218 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:09:12.874809 22878 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35360 (request call id 1760) took 1485 ms (client timeout 1780 ms). Trace:
W20260709 13:09:12.875253 22878 rpcz_store.cc:269] 0709 13:09:11.389277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:11.389376 (+    99us) service_pool.cc:224] Handling call
0709 13:09:12.874596 (+1485220us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b54002d6254. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:12.874717 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":333226240}
I20260709 13:09:12.877776 23620 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60044 (request call id 643) took 1613 ms. Trace:
I20260709 13:09:12.878185 23620 rpcz_store.cc:276] 0709 13:09:11.264149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:11.264259 (+   110us) service_pool.cc:224] Handling call
0709 13:09:11.264592 (+   333us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:11.277849 (+ 13257us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:11.277876 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:11.281138 (+  3262us) log.cc:844] Serialized 52 byte log entry
0709 13:09:11.511699 (+230561us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:09:11.511752 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:12.857164 (+1345412us) spinlock_profiling.cc:243] Waited 81.3 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:09:12.877460 (+ 20296us) spinlock_profiling.cc:243] Waited 3.33 ms on lock 0x7b540049a2d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:12.877509 (+    49us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:12.877564 (+    55us) raft_consensus.cc:1685] finished
0709 13:09:12.877580 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:12.877719 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9505,"raft.queue_time_us":10148,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":1514,"spinlock_wait_cycles":194753536,"thread_start_us":229629,"threads_started":1,"wal-append.queue_time_us":347482}
I20260709 13:09:12.880000 26256 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.883697 26218 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:12.884426 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:12.885233 22947 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
I20260709 13:09:12.885710 26261 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:12.886210 26261 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:12.891597 23588 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:12.886164 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:12.893194 26049 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:12.888413 26261 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:12.894842 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:12.896996 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:12.898378 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:12.898988 23023 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:12.898882 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:12.900570 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:12.902064 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:12.902779 23027 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:12.922442 23022 raft_consensus.cc:2393] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 11: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:12.923000 22952 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 10.
I20260709 13:09:12.924124 23021 raft_consensus.cc:1275] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:09:12.924708 23021 pending_rounds.cc:85] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75: Aborting all ops after (but not including) 4
I20260709 13:09:12.925164 23021 pending_rounds.cc:107] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:09:12.924732 23588 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:12.925503 23021 raft_consensus.cc:2889] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:12.926419 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:12.926934 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:12.925380 23024 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 11.
I20260709 13:09:12.927568 23021 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6.
I20260709 13:09:12.928346 23588 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:12.928954 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:12.934731 22952 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:09:12.935627 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:12.936252 22952 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
I20260709 13:09:12.937100 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:12.928263 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:12.938381 23027 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 9.
I20260709 13:09:12.938532 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:12.939229 23024 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
I20260709 13:09:12.941071 22952 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
I20260709 13:09:12.941946 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:12.944761 22952 raft_consensus.cc:2410] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:09:12.893500 25572 raft_consensus.cc:2804] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:12.953672 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:12.954550 22947 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
I20260709 13:09:12.860321 26208 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:12.945850 23029 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
I20260709 13:09:12.930052 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:12.928409 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:12.880585 26256 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:12.948967 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 672456us
I20260709 13:09:12.966382 23618 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:12.961386 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:13.006531 23021 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6.
I20260709 13:09:12.986720 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:12.961911 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:12.961858 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 46837.8us
I20260709 13:09:13.008360 22953 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:13.010242 22880 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:09:12.945586 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:12.923413 26068 raft_consensus.cc:2749] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:09:13.013510 22952 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
I20260709 13:09:12.923612 26255 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:12.993570 22992 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:13.035936 22992 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:12.947324 26210 raft_consensus.cc:2804] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:09:12.958387 25572 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:13.020646 23618 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
I20260709 13:09:12.940145 25781 raft_consensus.cc:2804] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:09:13.085853 22918 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:13.093987 22918 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:13.095280 22918 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:13.089814 23023 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:09:12.934309 26049 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:12.979278 26262 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:12.982717 25595 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:13.011355 26219 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:13.104182 22951 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58310 (request call id 1762) took 1776 ms (client timeout 1778 ms). Trace:
I20260709 13:09:13.112056 22918 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:13.103190 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:09:13.116315 22951 rpcz_store.cc:269] 0709 13:09:11.327342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:11.328151 (+   809us) service_pool.cc:224] Handling call
0709 13:09:13.103934 (+1775783us) spinlock_profiling.cc:243] Waited 3.68 ms on lock 0x7b54000758d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:13.104066 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8472320}
I20260709 13:09:13.116760 22949 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:13.131742 22918 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:13.083472 25572 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:13.138085 22953 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
I20260709 13:09:13.097553 26091 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:13.109140 26084 raft_consensus.cc:2804] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:09:13.128834 26045 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:13.109616 26074 raft_consensus.cc:2804] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:09:13.067660 26255 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:13.119736 26042 raft_consensus.cc:2804] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:09:13.067682 26192 consensus_queue.cc:1059] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 13:09:13.098219 26210 raft_consensus.cc:697] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 11 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:13.133900 22918 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:13.129069 26267 raft_consensus.cc:2804] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:13.117141 25781 raft_consensus.cc:697] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 10 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:13.129158 26219 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:13.066318 25874 consensus_queue.cc:1048] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.071s
I20260709 13:09:13.098105 26261 consensus_queue.cc:1059] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 13:09:13.136999 26249 raft_consensus.cc:2804] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:09:13.184537 26267 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:13.195271 23624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 693) took 1686 ms. Trace:
I20260709 13:09:13.141124 26141 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:60044 (request call id 631) took 3876 ms. Trace:
I20260709 13:09:13.157330 26045 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:13.204890 23624 rpcz_store.cc:276] 0709 13:09:11.509144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:11.509251 (+   107us) service_pool.cc:224] Handling call
0709 13:09:11.509544 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:12.635128 (+1125584us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:13.195045 (+559917us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b5400638dd4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:13.195210 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2587153792}
I20260709 13:09:13.164937 26074 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:13.174762 26042 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:13.120800 26262 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:13.172472 26084 raft_consensus.cc:697] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
W20260709 13:09:13.210821 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 470620us
I20260709 13:09:13.201931 26236 raft_consensus.cc:2804] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:09:13.157935 26264 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:13.169018 26091 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:13.158293 26049 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:13.207314 26249 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:13.227849 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:13.207140 26267 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:13.159906 22953 raft_consensus.cc:1240] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 9. Current term is 10. Ops: [5.7-9.9]
I20260709 13:09:13.206535 26226 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:13.223016 26042 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:13.243901 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:13.213893 26074 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:13.211105 26045 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:09:13.239480 26189 consensus_meta.cc:287] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.512s	user 0.004s	sys 0.005s
I20260709 13:09:13.245684 26249 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:13.300580 22953 pending_rounds.cc:85] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting all ops after (but not including) 6
I20260709 13:09:13.301064 22953 pending_rounds.cc:107] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 13:09:13.301419 22953 raft_consensus.cc:2889] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:13.257452 22949 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11.
I20260709 13:09:13.298102 26218 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:13.285236 26226 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:13.293377 26211 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:13.311352 26256 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:13.316277 25910 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:13.326107 26208 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:13.294694 26210 consensus_queue.cc:237] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:13.297178 25781 consensus_queue.cc:237] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:13.372012 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 336139us
I20260709 13:09:13.372834 22992 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:13.378726 22992 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:13.353786 26189 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:13.323942 26084 consensus_queue.cc:237] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:13.356967 26067 raft_consensus.cc:2749] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:13.393900 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:13.394130 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:13.412125 23023 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:13.394994 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:09:13.396417 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 262558us
I20260709 13:09:13.412914 23021 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:09:13.400426 22994 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:13.207546 26141 rpcz_store.cc:276] 0709 13:09:09.261061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:09.263862 (+  2801us) service_pool.cc:224] Handling call
0709 13:09:09.648428 (+384566us) ts_tablet_manager.cc:934] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2: Initiating tablet copy from peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
0709 13:09:12.176191 (+2527763us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:09:12.546548 (+370357us) tablet_metadata.cc:665] Metadata flushed
0709 13:09:12.546720 (+   172us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:09:12.984417 (+437697us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:09:13.043366 (+ 58949us) tablet_replica.cc:206] Creating consensus instance
0709 13:09:13.131644 (+ 88278us) ts_tablet_manager.cc:989] Replica registered.
0709 13:09:13.135109 (+  3465us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":12327,"tablet-copy.queue_time_us":193784,"thread_start_us":396835,"threads_started":2}
I20260709 13:09:13.492221 23021 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 9.
I20260709 13:09:13.379714 26256 raft_consensus.cc:2804] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:13.483485 26208 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:13.524562 23023 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 9.
I20260709 13:09:13.495492 26219 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:13.501564 26256 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:13.507129 26271 log.cc:927] Time spent T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.271s	user 0.001s	sys 0.000s
I20260709 13:09:13.502635 26255 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:13.394718 26189 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:13.522174 26272 log.cc:927] Time spent T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.198s	user 0.001s	sys 0.001s
I20260709 13:09:13.522727 26141 tablet_copy_client.cc:806] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 0 data blocks...
I20260709 13:09:13.536819 26091 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:13.535570 26256 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:09:13.581802 26275 log.cc:927] Time spent T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
W20260709 13:09:13.582994 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 93169.2us
W20260709 13:09:13.572921 26283 log.cc:927] Time spent T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.184s	user 0.000s	sys 0.001s
I20260709 13:09:13.535455 26236 raft_consensus.cc:697] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:13.539490 26208 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:13.615463 26290 log.cc:927] Time spent T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:09:13.617069 26140 raft_consensus.cc:3055] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:09:13.602160 26141 tablet_copy_client.cc:670] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 1 WAL segments...
W20260709 13:09:13.624279 26281 log.cc:927] Time spent T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 13:09:13.623054 26303 consensus_queue.cc:1059] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.002s
I20260709 13:09:13.603749 26226 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:13.617656 26140 raft_consensus.cc:740] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:13.640466 26227 raft_consensus.cc:3055] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:09:13.640807 26278 log.cc:927] Time spent T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:09:13.641060 26068 raft_consensus.cc:3055] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:09:13.641528 26068 raft_consensus.cc:740] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
W20260709 13:09:13.642563 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 1.3964e+06us
I20260709 13:09:13.642361 26068 consensus_queue.cc:260] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:13.643513 26068 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:13.647395 26208 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:13.651228 23023 raft_consensus.cc:1240] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:13.654443 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:13.655617 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:13.656155 23021 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 6.
I20260709 13:09:13.656956 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:13.657630 23029 raft_consensus.cc:3060] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:13.622360 26211 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:13.672611 23029 raft_consensus.cc:2468] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 7.
I20260709 13:09:13.675169 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:13.686167 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 675997us
I20260709 13:09:13.686367 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:13.660815 26276 log.cc:927] Time spent T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.260s	user 0.001s	sys 0.001s
I20260709 13:09:13.693184 23020 raft_consensus.cc:2468] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 7.
I20260709 13:09:13.640996 26227 raft_consensus.cc:740] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:13.695564 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:13.682274 26058 consensus_queue.cc:1048] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:13.727612 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:13.728188 22844 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:13.732591 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:13.727294 26236 consensus_queue.cc:237] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:13.737486 26287 log.cc:927] Time spent T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.164s	user 0.001s	sys 0.000s
W20260709 13:09:13.737949 26296 log.cc:927] Time spent T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:09:13.693691 26094 consensus_queue.cc:1048] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
W20260709 13:09:13.737505 26300 log.cc:927] Time spent T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:09:13.741374 26258 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:13.741983 26258 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:09:13.741565 26058 consensus_queue.cc:1048] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:09:13.744436 26258 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:13.726773 26221 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:13.754609 26267 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:13.735807 25840 consensus_meta.cc:287] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 1.037s	user 0.013s	sys 0.000s
I20260709 13:09:13.755952 26094 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:13.719715 26140 consensus_queue.cc:260] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:13.741488 26227 consensus_queue.cc:260] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:13.789361 26227 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:13.762567 26076 raft_consensus.cc:2749] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:09:13.794080 26239 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:13.773795 26094 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:13.801731 26242 log.cc:927] Time spent T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 1.719s	user 0.003s	sys 0.016s
I20260709 13:09:13.805902 26262 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:13.807946 26074 consensus_meta.cc:287] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.512s	user 0.008s	sys 0.000s
I20260709 13:09:13.810293 26074 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:13.770725 25840 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:13.761883 26042 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:13.819000 26042 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:13.769290 26222 consensus_queue.cc:1048] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:09:13.764918 26069 consensus_queue.cc:1048] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:09:13.820312 26279 raft_consensus.cc:3055] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:09:13.820730 26279 raft_consensus.cc:740] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:13.821543 26279 consensus_queue.cc:260] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:13.809818 26094 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:13.824640 26249 consensus_meta.cc:287] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.507s	user 0.005s	sys 0.005s
I20260709 13:09:13.825095 26249 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:13.827864 23618 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 689) took 2394 ms (client timeout 1968 ms). Trace:
W20260709 13:09:13.828445 23618 rpcz_store.cc:269] 0709 13:09:11.433325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:11.433446 (+   121us) service_pool.cc:224] Handling call
0709 13:09:13.827602 (+2394156us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b540046a854. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:13.827754 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3523540224}
I20260709 13:09:13.831173 26233 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:13.832777 25572 consensus_meta.cc:287] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.662s	user 0.000s	sys 0.011s
I20260709 13:09:13.833362 25572 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:13.835915 25572 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:13.837980 26267 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:13.770582 26045 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:13.763005 26221 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:13.843570 26019 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:13.845214 26189 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:13.854188 26140 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:13.854561 26279 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:13.870004 26264 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:13.870610 26221 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:13.854416 26256 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:13.858914 26249 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:13.903854 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 244633us
W20260709 13:09:13.910861 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 149205us
I20260709 13:09:13.911680 22871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2264) took 2432 ms. Trace:
W20260709 13:09:13.916550 22880 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35360 (request call id 1755) took 3459 ms (client timeout 1679 ms). Trace:
I20260709 13:09:13.916774 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:13.917132 22880 rpcz_store.cc:269] 0709 13:09:10.457085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:10.458126 (+  1041us) service_pool.cc:224] Handling call
0709 13:09:13.916013 (+3457887us) spinlock_profiling.cc:243] Waited 2.5 s on lock 0x7b54001bc454. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:13.916257 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5743810432}
I20260709 13:09:13.918394 22876 raft_consensus.cc:2393] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 9: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:13.919407 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:13.920490 22874 raft_consensus.cc:2393] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 8: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:13.926554 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:13.912266 22871 rpcz_store.cc:276] 0709 13:09:11.479061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:12.264320 (+785259us) service_pool.cc:224] Handling call
0709 13:09:12.876313 (+611993us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:13.655566 (+779253us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:13.655613 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:13.665129 (+  9516us) log.cc:844] Serialized 26 byte log entry
0709 13:09:13.762717 (+ 97588us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:09:13.762772 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:13.762846 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:13.849324 (+ 86478us) spinlock_profiling.cc:243] Waited 23.5 ms on lock 0x7b4800002958. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613ad0eaa1 00007f613ad05b37 00007f613ad059ca 00007f613adfd26f 00007f613adfcf37 00007f613ae1155c 00007f613ae11508 00007f613ae11488 00007f613ae11443 00007f613ae106ec 00007f6139d3aa96
0709 13:09:13.906129 (+ 56805us) raft_consensus.cc:1685] finished
0709 13:09:13.906209 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:13.911141 (+  4932us) spinlock_profiling.cc:243] Waited 300 ms on lock 0x7b54002d6250. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:13.911399 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":778819,"raft.queue_time_us":814030,"raft.run_cpu_time_us":1837,"raft.run_wall_time_us":156005,"spinlock_wait_cycles":743339136,"thread_start_us":80746,"threads_started":1,"wal-append.queue_time_us":107498}
I20260709 13:09:13.932705 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:13.933017 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:13.947340 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:13.949944 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:13.951598 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:13.933879 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:13.953279 22872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 1765) took 1858 ms. Trace:
I20260709 13:09:13.953598 22878 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:09:13.954622 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:13.953768 22872 rpcz_store.cc:276] 0709 13:09:12.094790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:12.644881 (+550091us) service_pool.cc:224] Handling call
0709 13:09:12.645190 (+   309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:13.693210 (+1048020us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 13:09:13.813766 (+120556us) spinlock_profiling.cc:243] Waited 23.5 ms on lock 0x7b4800002958. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613ad0eaa1 00007f613ad05b37 00007f613ad059ca 00007f613adfd26f 00007f613adfcf37 00007f613ae1155c 00007f613ae11508 00007f613ae11488 00007f613ae11443 00007f613ae106ec 00007f6139d3aa96
0709 13:09:13.907875 (+ 94109us) log.cc:844] Serialized 14 byte log entry
0709 13:09:13.937858 (+ 29983us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:13.938700 (+   842us) log.cc:844] Serialized 26 byte log entry
0709 13:09:13.938888 (+   188us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:09:13.939150 (+   262us) log.cc:844] Serialized 14 byte log entry
0709 13:09:13.939339 (+   189us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:13.939448 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:13.952861 (+ 13413us) raft_consensus.cc:1685] finished
0709 13:09:13.952884 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:13.953081 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":989372,"prepare.queue_time_us":181305,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":27,"raft.queue_time_us":1029702,"raft.run_cpu_time_us":1882,"raft.run_wall_time_us":119955,"spinlock_wait_cycles":54092672,"thread_start_us":250242,"threads_started":3,"wal-append.queue_time_us":27887}
I20260709 13:09:13.955538 22878 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
I20260709 13:09:13.934842 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:13.957245 22877 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:13.937211 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:13.933506 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:10.897394 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:13.952481 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:13.964390 22994 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:13.977696 22874 raft_consensus.cc:2410] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:09:13.937615 22843 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:13.981869 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 44540.3us
I20260709 13:09:13.991885 25572 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:13.993325 26097 raft_consensus.cc:2749] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:09:13.995608 26045 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:09:13.998819 26212 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.867s	user 0.006s	sys 0.012s
W20260709 13:09:13.999482 26212 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.880s	user 0.006s	sys 0.012s
I20260709 13:09:13.992779 25572 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:14.007009 25840 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:14.007099 23020 raft_consensus.cc:2393] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 8: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:14.010179 25910 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:14.010962 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:14.011456 25910 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:14.013106 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:14.013886 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:14.014413 23027 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
I20260709 13:09:14.015051 23023 raft_consensus.cc:2368] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 5. Current term is 6.
I20260709 13:09:14.015167 26074 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:14.016438 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:14.018294 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:14.019233 22992 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:14.019644 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:14.020884 26045 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:14.021163 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:14.007477 26256 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:14.024775 23025 raft_consensus.cc:2393] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 5: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:14.020542 22992 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:09:14.031301 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 1.32524e+06us
I20260709 13:09:14.025686 22843 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:09:14.049614 22994 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:14.051573 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:14.046833 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 50487.9us
I20260709 13:09:14.049770 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:14.057314 26021 raft_consensus.cc:2804] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:09:14.057883 26021 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:14.058262 26021 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:14.060272 23022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 1716) took 1036 ms. Trace:
I20260709 13:09:14.054369 26045 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:14.062309 23024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 1774) took 1157 ms. Trace:
I20260709 13:09:14.064265 26045 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:14.066193 25910 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:14.068521 26176 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:14.070317 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:14.062845 23024 rpcz_store.cc:276] 0709 13:09:12.904887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:12.940282 (+ 35395us) service_pool.cc:224] Handling call
0709 13:09:12.940537 (+   255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:12.940771 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:09:12.969992 (+ 29221us) log.cc:844] Serialized 14 byte log entry
0709 13:09:14.061190 (+1091198us) log.cc:844] Serialized 14 byte log entry
0709 13:09:14.061565 (+   375us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:14.061668 (+   103us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:09:14.061802 (+   134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:14.061908 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:14.062099 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":714628,"prepare.queue_time_us":743504,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":79,"raft.queue_time_us":2325,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":3293,"thread_start_us":375549,"threads_started":1,"wal-append.queue_time_us":572362}
I20260709 13:09:14.073122 22877 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
I20260709 13:09:14.055773 23026 raft_consensus.cc:2376] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8. Re-sending same reply.
I20260709 13:09:14.054885 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:14.089020 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:14.089780 22876 raft_consensus.cc:2376] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8. Re-sending same reply.
I20260709 13:09:14.076382 26269 raft_consensus.cc:2804] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:14.090858 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:14.092051 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 71744.3us
I20260709 13:09:14.091516 22876 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6.
I20260709 13:09:14.108872 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:14.110697 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:14.112250 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:14.117398 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:14.120352 22871 raft_consensus.cc:1240] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 3. Current term is 4. Ops: []
I20260709 13:09:14.121737 22874 raft_consensus.cc:1275] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:09:14.121909 26264 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:14.122128 22879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 1766) took 2026 ms. Trace:
I20260709 13:09:14.122848 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:14.122695 26264 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:14.125039 22873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 1772) took 1773 ms. Trace:
I20260709 13:09:14.125489 22873 rpcz_store.cc:276] 0709 13:09:12.254490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:12.701113 (+446623us) service_pool.cc:224] Handling call
0709 13:09:12.701564 (+   451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:13.852893 (+1151329us) spinlock_profiling.cc:243] Waited 23.5 ms on lock 0x7b4800002958. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613ad0eaa1 00007f613ad05b37 00007f613ad059ca 00007f613adfd26f 00007f613adfcf37 00007f613ae1155c 00007f613ae11508 00007f613ae11488 00007f613ae11443 00007f613ae106ec 00007f6139d3aa96
0709 13:09:13.908877 (+ 55984us) spinlock_profiling.cc:243] Waited 1.44 ms on lock 0x7b18003683e0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f613598d4bc 00007f6132cdb48b 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf5455 00007f6135ddd060 00007f6135dddbfb 00007f6135de864b
0709 13:09:13.909424 (+   547us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:09:13.910192 (+   768us) log.cc:844] Serialized 14 byte log entry
0709 13:09:13.972515 (+ 62323us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:13.973371 (+   856us) log.cc:844] Serialized 26 byte log entry
0709 13:09:13.973555 (+   184us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:09:13.973601 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:13.973742 (+   141us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:14.001064 (+ 27322us) raft_consensus.cc:1685] finished
0709 13:09:14.001082 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:14.028136 (+ 27054us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1083279,"prepare.queue_time_us":87384,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":1009194,"raft.run_cpu_time_us":1519,"raft.run_wall_time_us":160234,"spinlock_wait_cycles":57442048,"thread_start_us":104982,"threads_started":2,"wal-append.queue_time_us":61963}
I20260709 13:09:14.128038 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:14.128846 26318 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:14.129645 26318 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:14.171690 23025 raft_consensus.cc:2410] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 13:09:14.136864 22876 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:14.060621 23022 rpcz_store.cc:276] 0709 13:09:13.024273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:13.036802 (+ 12529us) service_pool.cc:224] Handling call
0709 13:09:13.037213 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:13.037497 (+   284us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:09:13.106260 (+ 68763us) log.cc:844] Serialized 14 byte log entry
0709 13:09:14.059072 (+952812us) spinlock_profiling.cc:243] Waited 5.15 ms on lock 0x7b18003e95c0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132c72c0a 00007f613c3b2219 00007f6132cfaa40 00007f6132cf54e4 00007f6132cf5455 00007f61397dd3c7 00007f6135def9a7 00007f6135dff0bf 00007f6135dff038
0709 13:09:14.059400 (+   328us) log.cc:844] Serialized 14 byte log entry
0709 13:09:14.059675 (+   275us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:14.059732 (+    57us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:09:14.059841 (+   109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:14.059930 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:14.060089 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":83602,"prepare.queue_time_us":830071,"prepare.run_cpu_time_us":105,"prepare.run_wall_time_us":107,"raft.queue_time_us":7990,"raft.run_cpu_time_us":428,"raft.run_wall_time_us":5372,"spinlock_wait_cycles":11855104,"thread_start_us":228966,"threads_started":1,"wal-append.queue_time_us":567934}
I20260709 13:09:14.177107 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:14.189518 22877 raft_consensus.cc:2393] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 9: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:14.131062 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:14.192539 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:14.198800 22873 raft_consensus.cc:2410] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 9 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:09:14.170943 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:14.130331 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:14.132205 26270 raft_consensus.cc:2804] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:09:14.122486 22879 rpcz_store.cc:276] 0709 13:09:12.095419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:12.646397 (+550978us) service_pool.cc:224] Handling call
0709 13:09:12.653301 (+  6904us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:13.913439 (+1260138us) spinlock_profiling.cc:243] Waited 1.4 ms on lock 0x7b180042cdc0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f613598d4bc 00007f6132cdb48b 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf5455 00007f6135ddd060 00007f6135dddbfb 00007f6135de864b
0709 13:09:13.913763 (+   324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:13.913803 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:13.914959 (+  1156us) log.cc:844] Serialized 52 byte log entry
0709 13:09:14.023912 (+108953us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:09:14.023960 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:14.040994 (+ 17034us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:14.091881 (+ 50887us) raft_consensus.cc:1685] finished
0709 13:09:14.091901 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:14.121960 (+ 30059us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1012133,"raft.queue_time_us":1035395,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":2786,"spinlock_wait_cycles":3212672,"thread_start_us":151011,"threads_started":2,"wal-append.queue_time_us":137328}
I20260709 13:09:14.208348 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:14.209007 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:14.132822 26314 raft_consensus.cc:2804] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:14.211294 22953 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:09:14.211752 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:14.301733 22877 raft_consensus.cc:2393] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 7: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:14.211848 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:14.214246 22992 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:14.217779 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:14.221083 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:14.221570 26264 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:14.221777 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 44684.8us
I20260709 13:09:14.222743 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:14.222994 22954 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2343) took 2448 ms. Trace:
I20260709 13:09:14.224973 26269 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:14.200222 26176 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:14.194757 22777 catalog_manager.cc:5697] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:14.230727 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:14.231585 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:14.232079 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:14.135023 26016 consensus_queue.cc:1059] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.082s
I20260709 13:09:14.300622 22950 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
I20260709 13:09:14.321662 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:14.339778 23025 raft_consensus.cc:2368] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 7. Current term is 8.
I20260709 13:09:14.341367 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:14.340945 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 315563us
I20260709 13:09:14.344877 22876 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:09:14.346035 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:14.336058 23590 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:14.318778 26269 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:14.346640 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:14.355463 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:14.356208 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:14.357200 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:14.359861 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:14.364442 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:14.338847 25840 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:14.303298 22954 rpcz_store.cc:276] 0709 13:09:11.774899 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:11.777100 (+  2201us) service_pool.cc:224] Handling call
0709 13:09:11.788114 (+ 11014us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:11.788364 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:09:11.882047 (+ 93683us) log.cc:844] Serialized 14 byte log entry
0709 13:09:13.962513 (+2080466us) spinlock_profiling.cc:243] Waited 1.52 ms on lock 0x7b18003d7980. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cdae07 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf7c02 00007f613597f20b 00007f6135984d23 00007f613598523a 00007f6135defa2a
0709 13:09:14.201763 (+239250us) log.cc:844] Serialized 14 byte log entry
0709 13:09:14.222403 (+ 20640us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:14.222472 (+    69us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:09:14.222541 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:14.222658 (+   117us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:14.222827 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":123311,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":5626,"raft.queue_time_us":51727,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":15277,"spinlock_wait_cycles":3504000,"thread_start_us":60856,"threads_started":1}
I20260709 13:09:14.362457 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:14.329850 26318 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:14.379073 22871 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:14.377215 22949 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:14.369051 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:14.383152 22948 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
I20260709 13:09:14.365512 25595 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:09:14.394006 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 200864us
I20260709 13:09:14.394150 26084 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:14.394337 26068 raft_consensus.cc:2749] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:14.394845 26084 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:14.395901 22920 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:14.397260 22879 raft_consensus.cc:1240] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:14.397353 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:14.401746 23026 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:14.365782 26264 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:14.387120 22918 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:14.404503 26236 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:14.405282 26236 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:14.412034 26249 raft_consensus.cc:2749] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:09:14.413022 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 96296.4us
I20260709 13:09:14.413905 22918 leader_election.cc:400] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 5. Current term is 6.
I20260709 13:09:14.414223 22918 leader_election.cc:403] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:09:14.416083 26227 consensus_meta.cc:287] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.617s	user 0.013s	sys 0.008s
I20260709 13:09:14.417526 22873 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:14.380806 26270 raft_consensus.cc:697] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:14.427137 26270 consensus_queue.cc:237] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:14.428318 26176 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:14.394025 25595 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:14.429872 25840 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:14.396570 26189 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:09:14.418495 22918 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:14.437340 22843 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:14.450191 22879 raft_consensus.cc:1240] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:09:14.417183 26314 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:14.455742 26314 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:14.456220 22777 catalog_manager.cc:5697] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 4 to 7, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 7 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:14.459156 26212 raft_consensus.cc:2804] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:14.459403 26236 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:14.459645 26212 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:14.460122 26212 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:09:14.460286 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 278568us
I20260709 13:09:14.462088 23590 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:14.463063 26264 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:14.466331 26264 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:14.468813 22880 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 8.
W20260709 13:09:14.473122 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 48076.7us
I20260709 13:09:14.476702 22949 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:09:14.436888 25572 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:14.449426 26074 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 6
W20260709 13:09:14.521785 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 64009.8us
I20260709 13:09:14.474969 26227 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:14.483256 26189 raft_consensus.cc:697] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:09:14.533850 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 138100us
I20260709 13:09:14.531003 26227 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:14.601534 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 220457us
W20260709 13:09:14.603888 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 56643.5us
I20260709 13:09:14.565445 26227 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:09:14.613109 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 60973.9us
W20260709 13:09:14.633816 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 217208us
W20260709 13:09:14.630244 26021 consensus_meta.cc:287] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.571s	user 0.013s	sys 0.000s
I20260709 13:09:14.689954 23026 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
I20260709 13:09:14.645758 26222 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:14.647709 22919 leader_election.cc:336] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Timed out: RequestConsensusVote RPC to 127.21.63.4:44043 timed out after 1.830s (SENT)
W20260709 13:09:14.648337 26140 consensus_meta.cc:287] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.687s	user 0.006s	sys 0.011s
W20260709 13:09:14.652972 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 190982us
I20260709 13:09:14.660161 25707 raft_consensus.cc:3055] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:09:14.662104 26084 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:14.676570 22871 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
I20260709 13:09:14.539113 26236 raft_consensus.cc:2804] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:09:14.693470 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:14.694283 22919 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:14.663843 26045 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:14.704412 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 100697us
I20260709 13:09:14.705564 22992 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:09:14.714905 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 209446us
I20260709 13:09:14.712148 26084 raft_consensus.cc:2749] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:09:14.694969 25707 raft_consensus.cc:740] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:14.700433 25840 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:14.704473 26045 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:14.720675 26279 consensus_meta.cc:287] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.770s	user 0.009s	sys 0.010s
I20260709 13:09:14.721113 23590 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:14.720964 26222 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:14.721510 26345 raft_consensus.cc:2749] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:09:14.721812 23021 raft_consensus.cc:2468] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
I20260709 13:09:14.719444 25910 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:14.695927 26021 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:14.749981 22843 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:14.755064 23590 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:14.757359 23590 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:14.726964 26189 consensus_queue.cc:237] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:14.758942 25910 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:14.761539 25910 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:14.761935 26222 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:14.766606 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:14.767544 23618 raft_consensus.cc:2393] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 7: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:14.768790 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:14.769631 23624 raft_consensus.cc:2393] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 7: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:14.770283 26045 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:14.770416 26021 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:14.770615 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:14.771389 23622 raft_consensus.cc:2468] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 3.
I20260709 13:09:14.772651 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:14.773470 23620 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 8.
I20260709 13:09:14.776504 23619 raft_consensus.cc:1240] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:09:14.777067 23024 raft_consensus.cc:2393] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 7: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:14.754882 25840 consensus_queue.cc:1048] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:09:14.751936 22994 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:14.785763 26344 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:14.786697 26347 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:14.786504 26344 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:14.787039 23623 raft_consensus.cc:1240] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:09:14.787555 26347 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:14.789762 22918 leader_election.cc:400] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 7. Current term is 8.
I20260709 13:09:14.790115 22918 leader_election.cc:403] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 13:09:14.791015 25910 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:14.791821 26258 raft_consensus.cc:2749] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:09:14.792250 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:14.793483 26258 consensus_queue.cc:1048] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:09:14.797441 26074 raft_consensus.cc:2749] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 5. Current term is 6.
I20260709 13:09:14.797606 26141 tablet_copy_client.cc:538] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:09:14.790114 26314 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:14.800587 26314 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:14.801460 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:14.802282 23618 raft_consensus.cc:2393] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 10: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:14.802910 23588 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:14.803215 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:14.803932 23620 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 4.
I20260709 13:09:14.805100 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:09:14.792558 26327 log.cc:927] Time spent T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.365s	user 0.000s	sys 0.004s
I20260709 13:09:14.766713 26140 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:14.811264 22873 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 10.
I20260709 13:09:14.812655 26312 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:14.812922 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 77971.7us
I20260709 13:09:14.813862 26140 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:14.777798 26236 raft_consensus.cc:697] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
W20260709 13:09:14.823299 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 125485us
I20260709 13:09:14.819571 22992 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:14.813282 26312 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:14.790053 26212 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:14.826813 26212 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:14.826956 22843 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:14.827515 26212 consensus_queue.cc:1059] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:09:14.827898 26314 raft_consensus.cc:2764] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:09:14.810895 26269 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:14.814399 26140 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:09:14.833153 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 50261.4us
I20260709 13:09:14.832896 22920 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:14.833356 26016 raft_consensus.cc:2804] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:09:14.831185 26264 raft_consensus.cc:2749] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:14.864595 22843 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:14.831194 26318 raft_consensus.cc:3055] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:09:14.831642 26256 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:14.872417 22992 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:14.871605 25707 consensus_queue.cc:260] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:14.897939 26269 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:14.829074 26344 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:14.902053 22843 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:14.870757 26318 raft_consensus.cc:740] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
W20260709 13:09:14.902789 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 69984.6us
I20260709 13:09:14.869246 26016 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:14.828614 26347 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:14.875876 26074 raft_consensus.cc:2749] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:14.831758 26236 consensus_queue.cc:237] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:14.916119 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 49761.4us
I20260709 13:09:14.910375 26016 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:14.914171 25707 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:14.915737 26264 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:14.927784 22992 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:14.922264 25910 raft_consensus.cc:2749] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 7. Current term is 8.
I20260709 13:09:14.929679 26264 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:14.928315 26269 raft_consensus.cc:2804] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:14.924525 26314 raft_consensus.cc:2804] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:09:14.937952 26264 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:09:14.951184 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 148443us
I20260709 13:09:14.939651 26269 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:14.966012 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 55466.5us
I20260709 13:09:14.951310 26314 raft_consensus.cc:697] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 7 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:14.969036 26269 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:14.985189 26267 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:09:14.986979 26314 consensus_queue.cc:237] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:14.988229 26349 raft_consensus.cc:2749] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:09:14.951522 26227 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
W20260709 13:09:14.988884 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 231610us
I20260709 13:09:14.992893 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:14.994874 22947 raft_consensus.cc:1240] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 9. Current term is 10. Ops: [5.7-9.9]
I20260709 13:09:14.970384 26315 raft_consensus.cc:2804] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:09:15.006634 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:15.007331 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:14.983160 26318 consensus_queue.cc:260] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:15.013518 23588 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:15.014866 23021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33418 (request call id 1725) took 1359 ms. Trace:
I20260709 13:09:15.034741 22950 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:15.036602 23021 rpcz_store.cc:276] 0709 13:09:13.656355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:13.674901 (+ 18546us) service_pool.cc:224] Handling call
0709 13:09:15.014316 (+1339415us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54002c9f54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:15.014639 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2333703424}
I20260709 13:09:15.070379 22955 raft_consensus.cc:2393] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 7: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
W20260709 13:09:15.073493 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 100744us
I20260709 13:09:15.074502 22949 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
I20260709 13:09:15.076110 22920 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:15.034389 26315 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:15.036535 26267 raft_consensus.cc:697] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 10 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:15.069471 26311 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:15.090896 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 54831.9us
I20260709 13:09:15.084414 26315 raft_consensus.cc:3060] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:15.084204 26349 raft_consensus.cc:2804] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:09:15.098114 23588 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:15.099752 25910 raft_consensus.cc:2749] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:09:15.113669 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:15.103257 26318 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:15.100442 25595 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:15.106360 26279 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:15.107679 26349 raft_consensus.cc:697] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 9 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:15.119177 22920 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:15.128118 26346 raft_consensus.cc:2749] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:15.128799 26359 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:09:15.141798 22949 raft_consensus.cc:1275] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:09:15.106113 26311 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:15.144116 25910 raft_consensus.cc:2804] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:09:15.145012 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:15.145632 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:15.145687 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:15.152469 22878 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 4.
I20260709 13:09:15.145738 22920 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:15.146335 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:15.132939 26140 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:15.147392 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:15.141242 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:15.151275 22873 raft_consensus.cc:1275] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:09:15.153919 22950 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
I20260709 13:09:15.154254 22955 raft_consensus.cc:1275] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:09:15.156710 22874 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 8.
W20260709 13:09:15.158630 26352 log.cc:927] Time spent T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
W20260709 13:09:15.148999 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 322923us
I20260709 13:09:15.135167 26359 raft_consensus.cc:697] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 9 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:15.170181 26140 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:15.170369 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:15.171221 22953 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 4.
I20260709 13:09:15.172225 26227 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:15.141660 26279 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:15.173004 22994 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:15.172873 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:15.164003 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:15.179318 22920 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:15.179474 22877 raft_consensus.cc:1275] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:09:15.172442 26256 raft_consensus.cc:2749] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:09:15.183332 22871 raft_consensus.cc:2393] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 11: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:15.136467 26267 consensus_queue.cc:237] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:15.187858 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:15.190214 26233 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:15.190796 26233 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:15.192780 26269 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:15.165182 25707 consensus_queue.cc:1059] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:09:15.172782 26359 consensus_queue.cc:237] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:15.187727 23024 raft_consensus.cc:2410] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:09:15.193908 26074 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:09:15.175217 25910 raft_consensus.cc:697] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:15.220554 26212 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:15.220916 26189 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:09:15.185050 26347 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:09:15.202767 22774 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33466 (request call id 290) took 15483 ms (client timeout 15000 ms). Trace:
I20260709 13:09:15.221293 26212 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:15.223592 22947 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 9.
I20260709 13:09:15.204233 22954 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
I20260709 13:09:15.186728 26355 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:15.224691 26312 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:15.224961 26355 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:15.225368 26212 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:15.229818 22777 catalog_manager.cc:5697] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:15.198043 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:15.232463 25707 raft_consensus.cc:3055] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:09:15.233021 25707 raft_consensus.cc:740] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:15.233258 26355 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:15.234341 26141 tablet_bootstrap.cc:492] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap starting.
I20260709 13:09:15.234197 25707 consensus_queue.cc:260] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:15.235559 22919 leader_election.cc:336] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Timed out: RequestConsensusVote RPC to 127.21.63.4:44043 timed out after 1.777s (SENT)
I20260709 13:09:15.235829 25707 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:15.236081 22919 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:15.243510 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 100019us
I20260709 13:09:15.257076 22777 catalog_manager.cc:5697] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 5 to 7, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 7 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:15.174507 26264 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:15.253716 22844 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:15.263429 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:15.261521 22777 catalog_manager.cc:5697] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:09:15.264258 23026 raft_consensus.cc:2468] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 4.
I20260709 13:09:15.270496 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:15.275007 26279 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:15.269577 26233 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:15.276180 26363 log.cc:927] Time spent T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.155s	user 0.001s	sys 0.001s
W20260709 13:09:15.278791 23590 connection.cc:256] RPC call timeout handler was delayed by -1.629432526999949s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2183s for a response
I20260709 13:09:15.279954 22880 raft_consensus.cc:2410] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:09:15.280483 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:09:15.286626 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:15.289412 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 53895.3us
W20260709 13:09:15.293125 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 48123.6us
I20260709 13:09:15.295583 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:15.296895 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:09:15.297029 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 124160us
I20260709 13:09:15.296929 23622 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:09:15.297618 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
W20260709 13:09:15.299778 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 125390us
I20260709 13:09:15.284608 26346 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:15.297921 22844 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:15.245553 25910 consensus_queue.cc:237] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:15.290529 26249 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:15.302764 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:15.312836 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:15.222404 26349 consensus_queue.cc:237] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:15.317456 23618 raft_consensus.cc:2410] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:09:15.299407 26355 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:15.305518 26318 raft_consensus.cc:2804] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:09:15.319741 26357 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:15.316982 22777 catalog_manager.cc:5697] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 2 to 4, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 4 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:15.320272 26289 raft_consensus.cc:2749] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:15.323323 23620 raft_consensus.cc:1240] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 10. Current term is 11. Ops: []
W20260709 13:09:15.324985 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:15.324931 26176 consensus_queue.cc:1048] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:09:15.326747 26249 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:15.316358 26016 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:15.223613 22774 rpcz_store.cc:269] 0709 13:08:59.719750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:08:59.720011 (+   261us) service_pool.cc:224] Handling call
0709 13:09:15.202449 (+15482438us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:09:09.006312 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:09:09.018166 (+ 11854us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:09:09.018241 (+    75us) write_op.cc:435] Acquired schema lock
0709 13:09:09.018268 (+    27us) tablet.cc:662] Decoding operations
0709 13:09:09.020282 (+  2014us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:09:09.020392 (+   110us) write_op.cc:641] Partition lock acquired for write op
0709 13:09:09.020433 (+    41us) tablet.cc:685] Acquiring locks for 16 operations
0709 13:09:09.036238 (+ 15805us) tablet.cc:701] Row locks acquired
0709 13:09:09.036256 (+    18us) write_op.cc:260] PREPARE: finished
0709 13:09:09.036394 (+   138us) write_op.cc:270] Start()
0709 13:09:09.036504 (+   110us) write_op.cc:276] Timestamp: P: 1783602549036348 usec, L: 0
0709 13:09:09.036523 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 13:09:09.039033 (+  2510us) log.cc:844] Serialized 6302 byte log entry
0709 13:09:09.653919 (+614886us) op_driver.cc:464] REPLICATION: finished
0709 13:09:10.023087 (+369168us) write_op.cc:301] APPLY: starting
0709 13:09:10.027303 (+  4216us) tablet.cc:1367] starting BulkCheckPresence
0709 13:09:10.027620 (+   317us) tablet.cc:1370] finished BulkCheckPresence
0709 13:09:10.027734 (+   114us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:09:10.033796 (+  6062us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:09:10.034096 (+   300us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=16
0709 13:09:10.034151 (+    55us) write_op.cc:312] APPLY: finished
0709 13:09:10.034502 (+   351us) log.cc:844] Serialized 113 byte log entry
0709 13:09:10.034667 (+   165us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:10.034837 (+   170us) write_op.cc:454] Released schema lock
0709 13:09:10.034891 (+    54us) write_op.cc:341] FINISH: Updating metrics
0709 13:09:10.035262 (+   371us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:10.035292 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":889353,"reported_tablets":30,"child_traces":[["op",{"apply.queue_time_us":354014,"apply.run_cpu_time_us":4039,"apply.run_wall_time_us":19448,"num_ops":16,"prepare.queue_time_us":135588,"prepare.run_cpu_time_us":42736,"prepare.run_wall_time_us":207168,"raft.queue_time_us":203403,"raft.run_cpu_time_us":21101,"raft.run_wall_time_us":471109,"replication_time_us":477170,"spinlock_wait_cycles":824576,"thread_start_us":535500,"threads_started":4,"wal-append.queue_time_us":202864,"wal-append.run_cpu_time_us":17713,"wal-append.run_wall_time_us":1799367}]]}
I20260709 13:09:15.329474 22994 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:09:15.333701 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:15.338003 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:09:15.317435 26364 log.cc:927] Time spent T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.244s	user 0.004s	sys 0.001s
I20260709 13:09:15.341097 26357 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:15.385231 26016 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:15.386015 23026 raft_consensus.cc:2393] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 7: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
W20260709 13:09:15.386243 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 57637.3us
I20260709 13:09:15.387056 26357 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:15.316078 26212 raft_consensus.cc:2804] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:15.387871 26212 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:15.383652 23618 raft_consensus.cc:1240] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:15.389214 26279 raft_consensus.cc:2749] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:09:15.388311 26212 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:15.393296 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:15.394004 22877 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
I20260709 13:09:15.394299 23026 raft_consensus.cc:1275] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:09:15.395588 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:15.396221 23023 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 5.
I20260709 13:09:15.396484 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:15.383661 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:15.383493 26312 consensus_queue.cc:1059] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:09:15.398036 26312 raft_consensus.cc:3055] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:09:15.398393 26312 raft_consensus.cc:740] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:15.399240 26312 consensus_queue.cc:260] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:15.383767 22777 catalog_manager.cc:5697] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 2 to 3, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 3 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:09:15.401031 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:15.333137 26318 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:15.401932 26318 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:15.402153 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:15.401710 22777 catalog_manager.cc:5697] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 3 to 6, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 6 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:15.402777 22877 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
I20260709 13:09:15.400609 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:15.401387 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:15.403707 22872 raft_consensus.cc:2393] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 7: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:15.403688 22777 catalog_manager.cc:5697] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:15.405263 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:15.405972 22871 raft_consensus.cc:2393] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 9: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:15.405773 22777 catalog_manager.cc:5697] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:15.406519 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:15.407464 23021 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 9.
I20260709 13:09:15.401714 23024 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6.
I20260709 13:09:15.400563 26312 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:15.409556 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:15.409987 26043 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:15.411725 26043 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:15.413232 26311 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:15.413862 26043 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:15.414322 26264 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:15.415225 26269 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:15.415364 22871 raft_consensus.cc:1275] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:09:15.417182 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:15.318895 26346 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:15.418330 26212 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:15.419003 23623 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:15.418998 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:15.365873 22844 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:15.420110 26346 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:15.420768 26311 raft_consensus.cc:2804] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:09:15.421150 26311 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:15.421521 26311 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:15.421440 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:15.422887 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 57118.5us
I20260709 13:09:15.434294 22920 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:15.435024 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:15.436872 26355 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:15.441465 22920 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:15.442212 26176 raft_consensus.cc:2749] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:09:15.444900 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:15.445984 22871 raft_consensus.cc:2393] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 9: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:15.441498 22777 catalog_manager.cc:5697] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:09:15.452988 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 147153us
I20260709 13:09:15.422322 26315 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:15.456523 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:15.456748 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:15.467670 22878 raft_consensus.cc:1275] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:09:15.474313 23021 raft_consensus.cc:2393] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 7: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:15.474671 23623 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11.
I20260709 13:09:15.475420 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:15.454515 26267 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:15.480638 22871 raft_consensus.cc:2410] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 8 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 13:09:15.467741 22777 catalog_manager.cc:5697] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 5 to 8, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 8 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:15.494671 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:15.495229 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:15.495672 22879 raft_consensus.cc:1275] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:09:15.457406 26314 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:15.657769 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:15.592778 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 184765us
I20260709 13:09:15.749925 23029 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 8.
I20260709 13:09:15.638564 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:15.500375 26311 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:15.657533 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:15.752162 23028 raft_consensus.cc:2393] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 9: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:15.752298 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:15.507112 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:15.752969 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:15.753083 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:15.753535 23620 raft_consensus.cc:2376] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6. Re-sending same reply.
W20260709 13:09:15.604084 22993 leader_election.cc:336] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Timed out: RequestConsensusVote RPC to 127.21.63.4:44043 timed out after 1.968s (SENT)
I20260709 13:09:15.496305 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:15.577996 23617 raft_consensus.cc:1240] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 10 LEADER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:15.499513 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:15.500128 22777 catalog_manager.cc:5697] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 6 to 7, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 7 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:15.747475 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:15.501935 23588 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:15.501008 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:15.760238 22993 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:15.747239 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:09:15.772846 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 331447us
I20260709 13:09:15.753268 23621 raft_consensus.cc:2410] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 13:09:15.469254 26279 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:15.507962 23026 raft_consensus.cc:1275] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:09:15.772100 23624 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:09:15.763103 23025 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 10.
W20260709 13:09:15.793354 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:15.658912 26377 log.cc:927] Time spent T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
W20260709 13:09:15.814512 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:15.815407 22918 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:15.759326 22777 catalog_manager.cc:5697] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 5 to 7, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 7 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:15.804540 22920 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:15.639271 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:15.808173 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:15.807396 23028 raft_consensus.cc:1240] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 13:09:15.812448 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:15.816999 23622 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:15.812908 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:15.816690 23616 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:09:15.821424 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:15.749348 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:15.822098 22952 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6.
I20260709 13:09:15.821682 22777 catalog_manager.cc:5697] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 2 to 3, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 3 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:15.646063 26212 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:15.566107 25595 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:15.826668 22949 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 7.
I20260709 13:09:15.745704 26381 raft_consensus.cc:2804] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:09:15.830673 22946 raft_consensus.cc:1275] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:09:15.822525 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:15.790287 26311 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:15.835762 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:15.841643 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 339737us
I20260709 13:09:15.844200 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:15.806420 26269 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:15.749423 26314 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:15.847110 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:15.817687 26349 raft_consensus.cc:2749] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:09:15.857694 23621 raft_consensus.cc:2468] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:09:15.852716 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:15.860877 22952 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:15.862573 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:15.863943 23028 raft_consensus.cc:1240] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 10 LEADER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:15.856144 23588 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:15.865002 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:15.511943 26267 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:15.867364 23026 raft_consensus.cc:1240] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:15.866555 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:15.868636 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:15.868637 23025 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 8.
I20260709 13:09:15.872014 23588 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:15.873411 23619 raft_consensus.cc:1275] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:09:15.867298 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:15.873231 26043 raft_consensus.cc:2749] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:09:15.879115 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:15.879081 23028 raft_consensus.cc:1240] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 6. Current term is 7. Ops: []
I20260709 13:09:15.879134 22946 raft_consensus.cc:2393] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 11: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:15.847177 26347 raft_consensus.cc:2749] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 13:09:15.880965 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 65652.3us
I20260709 13:09:15.857122 25595 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:15.890754 25595 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:15.892371 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:15.862252 26311 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:15.898160 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:15.898782 26267 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:15.901798 23622 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
I20260709 13:09:15.905200 23617 raft_consensus.cc:2368] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 5. Current term is 6.
I20260709 13:09:15.910534 26357 raft_consensus.cc:2749] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:09:15.913034 23625 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:09:15.874212 26381 raft_consensus.cc:697] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:15.914260 26314 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:15.844818 26279 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:15.918049 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:15.881677 26359 raft_consensus.cc:2804] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 13:09:15.922338 22918 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:15.921514 22920 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:15.863530 26269 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:15.926627 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:15.934676 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 62754.1us
I20260709 13:09:15.939339 23588 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:15.956589 23588 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:15.960304 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:15.915233 26314 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:15.925176 25840 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:15.962826 26357 raft_consensus.cc:2804] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:09:15.924099 26359 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:15.964368 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:15.970508 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:15.917397 26311 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:15.925632 26315 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:15.935901 26016 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:15.971202 26343 raft_consensus.cc:2804] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:09:15.994035 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:15.995393 22992 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:15.970479 26357 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:15.998754 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:15.985316 26359 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:09:16.001559 22992 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:15.959869 26349 consensus_queue.cc:1048] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:09:16.004865 22992 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:15.915367 26381 consensus_queue.cc:237] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:16.003151 26357 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:16.024432 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:16.025611 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:16.026280 22954 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:15.970814 26314 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:15.998961 26227 raft_consensus.cc:2804] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 13:09:16.018047 25707 consensus_meta.cc:287] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.782s	user 0.014s	sys 0.001s
I20260709 13:09:16.016438 26289 raft_consensus.cc:2804] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:09:15.996938 26343 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:16.034495 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 113166us
W20260709 13:09:16.036769 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 432702us
I20260709 13:09:16.037828 23024 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 8.
I20260709 13:09:15.993688 26016 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:16.039376 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 77165.9us
I20260709 13:09:16.042696 22953 raft_consensus.cc:1275] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:09:16.045521 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:16.046416 23619 raft_consensus.cc:2393] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 10: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:15.996729 25840 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:16.054850 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 132589us
I20260709 13:09:16.032630 25595 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:16.023638 26384 raft_consensus.cc:2749] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:09:16.062536 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:16.063603 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:16.065315 22949 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:16.065270 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:16.067076 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:16.063133 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:09:16.043782 26318 consensus_meta.cc:287] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.641s	user 0.012s	sys 0.000s
I20260709 13:09:16.039539 26343 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:16.039136 26043 raft_consensus.cc:2804] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:09:16.071997 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:16.047889 25707 raft_consensus.cc:2804] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:09:16.081471 23622 raft_consensus.cc:2376] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8. Re-sending same reply.
I20260709 13:09:16.072625 22951 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:16.086400 22954 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
I20260709 13:09:16.046742 25910 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:16.060178 26249 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:16.063606 26227 raft_consensus.cc:697] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:16.064982 26289 raft_consensus.cc:697] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:16.099097 25707 raft_consensus.cc:697] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:09:16.114979 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 110140us
I20260709 13:09:16.099164 26043 raft_consensus.cc:697] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:16.080794 25595 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:16.081382 26318 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:16.115972 26383 log.cc:927] Time spent T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
I20260709 13:09:16.169857 22949 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
W20260709 13:09:16.198989 22919 leader_election.cc:336] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Timed out: RequestConsensusVote RPC to 127.21.63.4:44043 timed out after 1.641s (SENT)
I20260709 13:09:16.200407 22950 raft_consensus.cc:1240] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 7. Current term is 8. Ops: []
I20260709 13:09:16.206122 26391 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:16.207226 26385 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:16.206894 26391 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:16.207811 26385 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:16.210318 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:16.214916 26355 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:16.215570 26355 consensus_queue.cc:1048] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:09:16.112860 26349 consensus_queue.cc:1048] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.073s
I20260709 13:09:16.100907 25910 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:16.346539 22951 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
I20260709 13:09:16.344820 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:16.246078 22873 raft_consensus.cc:2368] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 4. Current term is 5.
I20260709 13:09:16.209615 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:09:16.276121 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:16.352301 23022 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:16.222254 23027 raft_consensus.cc:1240] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 7. Current term is 8. Ops: []
I20260709 13:09:16.223047 23023 raft_consensus.cc:1240] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:09:16.180984 26279 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:16.236768 22919 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:16.351783 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 230870us
I20260709 13:09:16.325675 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:16.223707 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:16.366906 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:16.259563 26380 log.cc:927] Time spent T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.626s	user 0.005s	sys 0.009s
I20260709 13:09:16.217195 26269 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:16.277865 26343 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:16.374464 26016 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:16.374284 26405 consensus_queue.cc:1048] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:09:16.262108 26403 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
W20260709 13:09:16.377714 22993 leader_election.cc:336] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Timed out: RequestConsensusVote RPC to 127.21.63.4:44043 timed out after 1.680s (SENT)
I20260709 13:09:16.378436 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:16.384749 23619 raft_consensus.cc:2393] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 10: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:16.379428 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:16.357868 26349 consensus_queue.cc:1243] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer f3c799b1249e40119d2a5864231c6c75 log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:09:16.383463 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:16.389853 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:16.390832 23622 raft_consensus.cc:2368] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 5. Current term is 7.
I20260709 13:09:16.226327 26289 consensus_queue.cc:237] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:16.349619 25707 consensus_queue.cc:237] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:16.399266 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:16.217593 26227 consensus_queue.cc:237] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:16.259284 26043 consensus_queue.cc:237] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:16.422036 23020 raft_consensus.cc:2410] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:09:16.403215 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:16.410413 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:16.400499 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:09:16.422487 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 223523us
W20260709 13:09:16.424057 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 222823us
W20260709 13:09:16.444607 22919 leader_election.cc:336] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Timed out: RequestConsensusVote RPC to 127.21.63.4:44043 timed out after 1.808s (SENT)
I20260709 13:09:16.447639 22919 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:16.444927 23588 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:16.445845 22953 raft_consensus.cc:2393] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 8: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:16.446393 23023 raft_consensus.cc:1240] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 7. Current term is 8. Ops: []
W20260709 13:09:16.424525 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 46838.9us
I20260709 13:09:16.446076 23590 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:16.426154 26267 raft_consensus.cc:2804] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:16.388288 26016 consensus_queue.cc:1048] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.064s
I20260709 13:09:16.453462 22950 raft_consensus.cc:1240] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 7. Current term is 8. Ops: [4.3-7.5]
I20260709 13:09:16.458639 23022 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 9.
W20260709 13:09:16.459103 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 84912.8us
I20260709 13:09:16.392706 26347 raft_consensus.cc:2749] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:16.460207 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:16.460180 22992 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:16.394274 26385 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:16.464618 23020 raft_consensus.cc:1240] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:16.462777 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:16.466562 22949 raft_consensus.cc:1240] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:16.471632 23590 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:16.472604 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:16.473435 22950 raft_consensus.cc:2368] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 8. Current term is 9.
I20260709 13:09:16.459082 26399 raft_consensus.cc:2749] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:16.464134 26176 raft_consensus.cc:2749] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:16.477537 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:16.392894 26403 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:16.481683 22876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53724 (request call id 531) took 1020 ms. Trace:
I20260709 13:09:16.481446 22992 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:16.495733 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:16.496554 22950 raft_consensus.cc:2368] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 7. Current term is 8.
I20260709 13:09:16.464221 26312 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:16.496649 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:16.498250 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:16.496099 22876 rpcz_store.cc:276] 0709 13:09:15.460941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:15.461045 (+   104us) service_pool.cc:224] Handling call
0709 13:09:16.477399 (+1016354us) spinlock_profiling.cc:243] Waited 4.15 ms on lock 0x7b5800000c98. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0a22 00007f613bbd9c09 00007f613ad33ee0 00007f613adfd0ab 00007f613adfd131 00007f613adab0a0 00007f613adaa984 00007f61343256d4 00007f6134325662
0709 13:09:16.481623 (+  4224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9539456}
I20260709 13:09:16.499679 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:16.483357 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:16.472255 26355 raft_consensus.cc:2804] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 13:09:16.471470 26407 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:09:16.504377 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:16.506320 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:16.476260 26343 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:16.507735 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:16.508234 22950 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 10.
I20260709 13:09:16.480528 26359 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:16.513382 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:16.514536 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:16.514385 26357 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:16.515278 22948 raft_consensus.cc:2393] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 9: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:16.515517 23617 raft_consensus.cc:2368] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 5. Current term is 6.
I20260709 13:09:16.516036 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:16.502029 26279 raft_consensus.cc:2804] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:09:16.502138 26355 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:16.529394 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:16.533278 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 67578.9us
I20260709 13:09:16.533386 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:09:16.533699 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 61810.1us
I20260709 13:09:16.534418 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:16.536263 26357 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:16.501643 26408 raft_consensus.cc:2804] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:16.531267 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:16.531927 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:16.503809 26312 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:16.538442 26267 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:16.539855 22949 raft_consensus.cc:2410] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 11 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:09:16.516644 22948 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 9.
I20260709 13:09:16.534824 23588 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:16.542326 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:16.570999 23022 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:16.547895 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:16.572295 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:16.551379 26391 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:16.553634 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:16.557751 26403 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:16.544380 26408 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:16.581645 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:16.582384 22879 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
I20260709 13:09:16.582415 25840 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:16.564019 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:16.583464 22871 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
I20260709 13:09:16.572947 22880 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:09:16.560922 26343 consensus_queue.cc:1048] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:09:16.581667 26408 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:16.585721 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:16.587420 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 52692us
I20260709 13:09:16.587908 22946 raft_consensus.cc:1240] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:09:16.588843 26403 raft_consensus.cc:2804] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:16.589186 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:16.540345 26355 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:16.530449 25910 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:16.572710 26267 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:16.575783 22878 raft_consensus.cc:1240] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:09:16.576995 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:16.577617 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:16.607295 22877 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:09:16.607564 22880 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:09:16.578210 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:16.579398 22872 raft_consensus.cc:1275] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:09:16.581110 26312 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:16.583499 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:16.612861 22879 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 9.
I20260709 13:09:16.590036 23029 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:16.573515 22954 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 9.
W20260709 13:09:16.571733 22952 consensus_meta.cc:287] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.708s	user 0.006s	sys 0.003s
I20260709 13:09:16.588500 23588 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:16.614854 22952 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 10.
I20260709 13:09:16.615602 23588 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:16.601649 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:16.614041 22919 leader_election.cc:336] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Timed out: RequestConsensusVote RPC to 127.21.63.4:44043 timed out after 1.697s (SENT)
I20260709 13:09:16.617678 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:16.609037 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:16.618652 22876 raft_consensus.cc:2410] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:09:16.544600 26279 raft_consensus.cc:697] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:16.540874 26407 raft_consensus.cc:697] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:16.618610 23023 raft_consensus.cc:1275] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:09:16.589233 26403 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:16.612917 22920 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:16.624522 22950 raft_consensus.cc:1240] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:16.625537 23623 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 697) took 1329 ms. Trace:
W20260709 13:09:16.626298 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 169274us
I20260709 13:09:16.628063 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
W20260709 13:09:16.631908 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:16.633164 23616 raft_consensus.cc:1275] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:09:16.633603 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:16.634265 22878 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 9.
I20260709 13:09:16.635201 22878 raft_consensus.cc:1240] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:16.630149 23623 rpcz_store.cc:276] 0709 13:09:15.281086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:15.287627 (+  6541us) service_pool.cc:224] Handling call
0709 13:09:16.610218 (+1322591us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b540046b254. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:16.610353 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":273140480}
I20260709 13:09:16.637346 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:16.642907 22920 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:16.637856 23023 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:09:16.647616 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:16.648816 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:16.625178 26398 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:16.649168 22878 raft_consensus.cc:1275] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:09:16.649880 23625 raft_consensus.cc:2393] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 7: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:16.649859 22873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53724 (request call id 527) took 1181 ms. Trace:
I20260709 13:09:16.654243 23620 raft_consensus.cc:1275] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:09:16.657953 22873 rpcz_store.cc:276] 0709 13:09:15.458774 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:15.458962 (+   188us) service_pool.cc:224] Handling call
0709 13:09:16.636140 (+1177178us) spinlock_profiling.cc:243] Waited 472 ms on lock 0x7b54002d6754. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:16.640048 (+  3908us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1086351488}
W20260709 13:09:16.644784 22993 leader_election.cc:336] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Timed out: RequestConsensusVote RPC to 127.21.63.4:44043 timed out after 1.593s (SENT)
I20260709 13:09:16.628491 26349 raft_consensus.cc:2804] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:16.659323 22993 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:16.655015 23022 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:09:16.660248 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:16.656950 23588 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:16.666740 25595 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:16.667125 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:16.668715 22955 raft_consensus.cc:1240] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 LEADER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 6. Current term is 7. Ops: []
I20260709 13:09:16.670047 22873 raft_consensus.cc:1240] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:16.675628 26389 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:16.676126 26389 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:16.677547 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:16.678495 26389 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:16.678722 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:16.680517 26318 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:16.681198 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:16.682600 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:16.683799 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:16.684962 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:16.685602 26311 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:16.686301 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:16.686992 22954 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 4 FOLLOWER]: Advancing to term 6
I20260709 13:09:16.689116 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:16.689770 22949 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:16.689841 23023 raft_consensus.cc:1275] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:09:16.635421 26343 raft_consensus.cc:2749] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:16.636080 26403 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:16.668587 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:16.695446 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:16.649487 26074 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:16.680351 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:16.709502 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:16.715500 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:16.740048 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:16.741613 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 197713us
I20260709 13:09:16.658502 26347 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:16.663669 26349 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:16.746716 22920 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:16.747964 22920 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:16.662442 26401 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:16.702790 26357 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:09:16.635563 26419 consensus_queue.cc:1048] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.083s
I20260709 13:09:16.711608 26074 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:16.812852 22880 raft_consensus.cc:3055] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:09:16.709082 26016 raft_consensus.cc:2749] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:16.802304 22992 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:16.796028 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:16.809687 22920 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:16.813367 22880 raft_consensus.cc:740] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:16.815295 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:16.820814 23021 raft_consensus.cc:2393] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 8: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
W20260709 13:09:16.821324 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 78174.8us
I20260709 13:09:16.814525 23027 pending_rounds.cc:85] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75: Aborting all ops after (but not including) 7
I20260709 13:09:16.817152 23022 raft_consensus.cc:1275] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:09:16.817859 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:16.825021 23025 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:16.825506 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:16.826260 23024 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:16.828058 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:16.820932 22880 consensus_queue.cc:260] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:16.749907 26347 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:16.828871 22879 raft_consensus.cc:2468] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
W20260709 13:09:16.839761 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 194993us
I20260709 13:09:16.843945 22954 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6.
I20260709 13:09:16.752045 26349 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:16.823040 23027 pending_rounds.cc:107] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:09:16.663857 26398 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:16.809000 26236 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:16.808197 25910 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:09:16.716841 26318 consensus_queue.cc:1048] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.103s
I20260709 13:09:16.819465 26074 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:16.861137 23027 raft_consensus.cc:2889] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:09:16.855101 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 104310us
I20260709 13:09:16.871032 22880 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:16.857578 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:16.821563 26379 log.cc:927] Time spent T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 1.189s	user 0.003s	sys 0.014s
I20260709 13:09:16.886155 22949 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 7.
I20260709 13:09:16.759690 26289 raft_consensus.cc:2764] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:09:16.812207 26357 raft_consensus.cc:697] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:16.808024 26176 consensus_queue.cc:1048] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
W20260709 13:09:16.895540 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:16.895784 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 73034.8us
I20260709 13:09:16.880052 26359 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:16.905962 22920 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:16.895157 22993 leader_election.cc:336] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Timed out: RequestConsensusVote RPC to 127.21.63.4:44043 timed out after 1.526s (SENT)
I20260709 13:09:16.907785 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:16.909126 22946 raft_consensus.cc:1275] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:09:16.909761 22877 raft_consensus.cc:2410] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:09:16.809185 26401 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:16.816340 26233 consensus_queue.cc:1048] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.062s
I20260709 13:09:16.818261 26249 consensus_queue.cc:1059] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.071s
W20260709 13:09:16.918730 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 60752us
I20260709 13:09:16.839607 26347 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:16.857836 26409 raft_consensus.cc:2749] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:16.966760 23023 raft_consensus.cc:1275] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:09:16.914108 26406 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:16.921904 23590 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:16.756791 26410 consensus_queue.cc:1048] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.151s
I20260709 13:09:16.924597 22993 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:16.756880 26385 consensus_queue.cc:1059] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.151s
I20260709 13:09:16.924212 22880 raft_consensus.cc:1275] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:09:16.870965 26311 consensus_queue.cc:1059] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:09:16.759662 26399 consensus_queue.cc:1048] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.160s
I20260709 13:09:16.927452 22872 raft_consensus.cc:2393] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 9: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
W20260709 13:09:16.954844 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 49018.9us
I20260709 13:09:17.003060 22920 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:16.908293 25910 raft_consensus.cc:697] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:16.956096 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:17.004411 22878 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:09:16.955124 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:17.005518 22877 raft_consensus.cc:2393] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 9: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:16.818702 26279 consensus_queue.cc:237] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:16.839897 26395 consensus_queue.cc:1059] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.089s
W20260709 13:09:16.969537 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 137634us
I20260709 13:09:16.900521 26236 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:16.958143 26227 raft_consensus.cc:2749] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:16.978138 26398 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:16.978338 26424 log.cc:927] Time spent T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 13:09:16.979439 26415 log.cc:927] Time spent T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.240s	user 0.001s	sys 0.000s
W20260709 13:09:16.980134 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 58307.7us
I20260709 13:09:17.055265 23590 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:16.968855 22843 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:16.984071 26438 raft_consensus.cc:2749] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:09:17.057193 26438 raft_consensus.cc:2749] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:09:16.984522 26359 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:16.985414 26289 raft_consensus.cc:3055] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:09:16.985818 26385 raft_consensus.cc:3055] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:09:17.065330 26289 raft_consensus.cc:740] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:17.065587 26385 raft_consensus.cc:740] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:16.970661 26406 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:16.986505 26314 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:16.951785 26433 consensus_queue.cc:1048] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.063s
I20260709 13:09:17.065359 26409 raft_consensus.cc:3055] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:09:17.092119 26409 raft_consensus.cc:740] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:16.990036 26267 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:17.095120 26267 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:17.096288 26409 consensus_queue.cc:260] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:17.098090 26409 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:09:16.999420 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 197215us
I20260709 13:09:17.103358 23024 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 7.
W20260709 13:09:17.105229 26421 log.cc:927] Time spent T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.184s	user 0.000s	sys 0.001s
I20260709 13:09:17.070262 26311 consensus_queue.cc:1243] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer aed3f089ec8b49af9c8cf5cdee76e8e7 log is divergent from this leader: its last log entry 8.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:09:17.070618 26400 raft_consensus.cc:2804] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:17.110419 26400 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:17.110908 26400 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:09:17.112818 26355 consensus_meta.cc:287] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.500s	user 0.009s	sys 0.000s
I20260709 13:09:17.113379 26355 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:17.116067 26355 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:17.117533 26430 log.cc:927] Time spent T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.153s	user 0.000s	sys 0.000s
I20260709 13:09:16.858670 26407 consensus_queue.cc:237] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:17.000747 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 105632us
I20260709 13:09:17.006567 23025 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6.
W20260709 13:09:17.126978 22993 leader_election.cc:336] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Timed out: RequestConsensusVote RPC to 127.21.63.4:44043 timed out after 2.198s (SENT)
I20260709 13:09:17.127555 22993 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:09:17.046021 22994 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:16.966835 26428 raft_consensus.cc:2804] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:09:17.009933 22918 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:17.057471 25910 consensus_queue.cc:237] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:09:16.989331 26425 raft_consensus.cc:2749] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:17.219477 23616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 714) took 1579 ms. Trace:
W20260709 13:09:17.226754 26408 consensus_meta.cc:287] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.642s	user 0.009s	sys 0.000s
I20260709 13:09:17.227233 26408 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:17.058176 26359 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:17.228709 23622 raft_consensus.cc:2393] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 9: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:17.231016 26359 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:17.246755 23617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60044 (request call id 657) took 1319 ms. Trace:
I20260709 13:09:17.220084 23616 rpcz_store.cc:276] 0709 13:09:15.639760 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:15.830760 (+191000us) service_pool.cc:224] Handling call
0709 13:09:16.610551 (+779791us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:16.633526 (+ 22975us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:17.219137 (+585611us) spinlock_profiling.cc:243] Waited 753 ms on lock 0x7b540046b250. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:17.219368 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1732114304}
I20260709 13:09:17.247321 23617 rpcz_store.cc:276] 0709 13:09:15.927295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:15.927462 (+   167us) service_pool.cc:224] Handling call
0709 13:09:17.246482 (+1319020us) spinlock_profiling.cc:243] Waited 554 ms on lock 0x7b5400453654. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:17.246667 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1274829056}
I20260709 13:09:16.970515 26357 consensus_queue.cc:237] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:17.391971 26392 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:17.392925 26392 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:17.396145 26408 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:09:17.403527 26403 consensus_meta.cc:287] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.663s	user 0.008s	sys 0.000s
I20260709 13:09:17.406631 26349 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:17.455101 22955 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 1777) took 1340 ms. Trace:
I20260709 13:09:17.404038 26403 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:17.455724 22955 rpcz_store.cc:276] 0709 13:09:16.109491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:16.109765 (+   274us) service_pool.cc:224] Handling call
0709 13:09:16.111358 (+  1593us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:16.669288 (+557930us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:17.449991 (+780703us) spinlock_profiling.cc:243] Waited 557 ms on lock 0x7b54003c7154. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:17.450270 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1281118080}
W20260709 13:09:17.493155 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 367120us
I20260709 13:09:17.525501 23621 raft_consensus.cc:1240] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 9. Current term is 10. Ops: []
W20260709 13:09:17.533757 26347 consensus_meta.cc:287] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.553s	user 0.008s	sys 0.001s
I20260709 13:09:17.744277 22871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 1801) took 1178 ms. Trace:
W20260709 13:09:17.745415 23029 consensus_meta.cc:287] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 1.131s	user 0.024s	sys 0.011s
I20260709 13:09:17.745841 23029 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 9.
W20260709 13:09:17.746364 23029 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33418 (request call id 1740) took 2099 ms (client timeout 1610 ms). Trace:
I20260709 13:09:17.744899 22871 rpcz_store.cc:276] 0709 13:09:16.565496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:16.584386 (+ 18890us) service_pool.cc:224] Handling call
0709 13:09:16.584639 (+   253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:16.584861 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:09:16.710307 (+125446us) log.cc:844] Serialized 14 byte log entry
0709 13:09:17.743915 (+1033608us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:17.743969 (+    54us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:09:17.744017 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:17.744081 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:17.744216 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1983,"prepare.queue_time_us":233923,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"thread_start_us":421955,"threads_started":2}
W20260709 13:09:17.746726 23029 rpcz_store.cc:269] 0709 13:09:15.646806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:15.751126 (+104320us) service_pool.cc:224] Handling call
0709 13:09:16.589056 (+837930us) spinlock_profiling.cc:243] Waited 14.5 ms on lock 0x7b58001ca198. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0a22 00007f613bbd9c09 00007f613ad33ee0 00007f613adfd0ab 00007f613adfd131 00007f613adab0a0 00007f613adaa984 00007f61343256d4 00007f6134325662
0709 13:09:17.746290 (+1157234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":33272064}
I20260709 13:09:17.774237 25595 raft_consensus.cc:3055] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:09:17.792587 26386 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:09:17.792757 25595 raft_consensus.cc:740] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:17.794492 22994 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:17.794739 26267 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:17.795735 26267 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:17.796140 26437 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:17.793942 25595 consensus_queue.cc:260] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:17.796689 26437 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:17.798770 26437 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:17.797067 25595 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:17.799248 26428 raft_consensus.cc:697] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 6 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:17.801034 26425 raft_consensus.cc:2804] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:17.801489 26425 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:17.801856 26425 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:17.796408 26267 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:17.805130 26403 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:17.800606 26428 consensus_queue.cc:237] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:17.720433 26347 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:17.817158 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:17.831933 26386 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:17.832597 26386 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:17.798949 26074 consensus_meta.cc:287] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.885s	user 0.007s	sys 0.002s
I20260709 13:09:17.835510 26074 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:17.839380 22952 raft_consensus.cc:1240] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:09:17.764940 26401 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:09:17.837229 26426 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.007s	sys 0.015s
W20260709 13:09:17.845664 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 842626us
I20260709 13:09:17.846400 22953 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 1785) took 1394 ms. Trace:
I20260709 13:09:17.765012 26236 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:17.845407 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 876543us
I20260709 13:09:17.850870 22953 rpcz_store.cc:276] 0709 13:09:16.452258 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:16.457135 (+  4877us) service_pool.cc:224] Handling call
0709 13:09:16.457437 (+   302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:16.494371 (+ 36934us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:16.494400 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:16.494845 (+   445us) log.cc:844] Serialized 26 byte log entry
0709 13:09:16.540308 (+ 45463us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:16.540348 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:16.540413 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:17.846119 (+1305706us) raft_consensus.cc:1685] finished
0709 13:09:17.846148 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:17.846329 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1221,"raft.queue_time_us":19372,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":4116,"thread_start_us":27292,"threads_started":1,"wal-append.queue_time_us":123787}
W20260709 13:09:17.870491 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 860535us
I20260709 13:09:17.828882 26311 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:17.883136 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:17.883711 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 137272us
W20260709 13:09:17.886870 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 831607us
W20260709 13:09:17.859362 26409 consensus_meta.cc:287] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.761s	user 0.002s	sys 0.009s
I20260709 13:09:17.860616 26425 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:17.907083 26425 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:17.908179 26267 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:17.848367 26406 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:09:17.848480 26426 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.739s	user 0.007s	sys 0.016s
I20260709 13:09:17.901582 26395 raft_consensus.cc:2804] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:17.917834 26395 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:17.918200 26395 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:17.845351 26386 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:17.902508 26400 consensus_meta.cc:287] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.791s	user 0.006s	sys 0.007s
I20260709 13:09:17.949234 22946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2409) took 1196 ms. Trace:
I20260709 13:09:17.904238 26311 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:17.955245 26349 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:17.956331 22946 rpcz_store.cc:276] 0709 13:09:16.752472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:16.752630 (+   158us) service_pool.cc:224] Handling call
0709 13:09:16.752961 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:16.925106 (+172145us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:17.948896 (+1023790us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b5400411ed4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:17.949155 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":358357376}
W20260709 13:09:18.051671 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 255918us
I20260709 13:09:18.080101 22994 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:18.065145 26400 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:18.147920 26389 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:18.165748 22843 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:18.166155 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 294722us
I20260709 13:09:18.168689 23590 leader_election.cc:400] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 4. Current term is 5.
W20260709 13:09:18.148079 26357 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.746s	user 0.011s	sys 0.034s
W20260709 13:09:18.180644 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 1.05359e+06us
W20260709 13:09:18.172818 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 308898us
I20260709 13:09:18.164386 26400 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:18.174649 26141 log.cc:927] Time spent T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:09:18.185192 22871 raft_consensus.cc:1275] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:09:18.185431 26392 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:18.186573 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 134069us
I20260709 13:09:18.186821 25707 raft_consensus.cc:2749] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:18.187309 22994 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:18.187580 25595 consensus_queue.cc:1059] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:09:18.188496 22994 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:09:18.180615 26357 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.786s	user 0.012s	sys 0.034s
I20260709 13:09:18.192610 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:18.198894 22871 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 8.
I20260709 13:09:18.182634 26389 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:18.182910 26386 raft_consensus.cc:2749] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:18.192071 26409 consensus_queue.cc:1048] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.009s
I20260709 13:09:18.188735 26212 raft_consensus.cc:3055] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:09:18.202445 26343 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:18.194928 26311 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:18.203054 26343 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:18.208266 22871 raft_consensus.cc:1275] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:09:18.183540 26349 consensus_queue.cc:1048] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:09:18.209040 22871 pending_rounds.cc:85] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 7
I20260709 13:09:18.209501 22871 pending_rounds.cc:107] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:09:18.209847 22871 raft_consensus.cc:2889] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:18.211699 26236 consensus_queue.cc:1048] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:09:18.196144 26436 consensus_queue.cc:1059] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:09:18.190193 25707 raft_consensus.cc:2749] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:09:18.213346 23622 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 724) took 1596 ms. Trace:
I20260709 13:09:18.213726 23622 rpcz_store.cc:276] 0709 13:09:16.617073 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:16.617442 (+   369us) service_pool.cc:224] Handling call
0709 13:09:18.213154 (+1595712us) spinlock_profiling.cc:243] Waited 610 ms on lock 0x7b540045bfd4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:18.213286 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1403455744}
I20260709 13:09:18.214627 26389 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:18.197405 26392 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:18.217559 26392 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:18.220800 26407 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:18.202596 26212 raft_consensus.cc:740] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:18.221297 26407 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:18.221333 26074 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:18.221705 26392 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:18.219146 22871 raft_consensus.cc:1275] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 8. (term mismatch)
I20260709 13:09:18.223323 22994 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:18.223718 22843 leader_election.cc:400] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 election: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 8. Current term is 9.
I20260709 13:09:18.225236 22843 leader_election.cc:403] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
W20260709 13:09:18.222913 22993 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.63.4:44043 timed out after 2.221s (SENT) [suppressed 34 similar messages]
I20260709 13:09:18.226106 22843 leader_election.cc:400] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 7. Current term is 8.
I20260709 13:09:18.226434 22843 leader_election.cc:403] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:09:18.225004 22871 pending_rounds.cc:85] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 7
I20260709 13:09:18.228605 22871 pending_rounds.cc:107] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 13:09:18.229022 22871 raft_consensus.cc:2889] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:18.231563 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:18.232147 22878 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:18.222697 26437 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:18.233316 25707 raft_consensus.cc:2804] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:18.233698 25707 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:18.234040 25707 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:09:18.238577 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 315674us
I20260709 13:09:18.239856 23588 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:18.240103 22994 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:18.226181 26176 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:18.245119 23588 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:18.245558 22843 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:18.229019 26074 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:18.251287 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:18.244222 26401 consensus_queue.cc:1048] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 13:09:18.239014 26311 raft_consensus.cc:2749] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 8. Current term is 9.
I20260709 13:09:18.265304 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:18.267079 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:18.268537 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:18.269687 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:18.269948 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:18.270875 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:18.271102 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:18.271842 23622 raft_consensus.cc:2393] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 11: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:18.273002 26357 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:18.273072 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:18.273294 26176 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:18.273613 26357 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:18.274019 23625 raft_consensus.cc:2376] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8. Re-sending same reply.
I20260709 13:09:18.241822 26267 consensus_queue.cc:1048] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:09:18.222244 26212 consensus_queue.cc:260] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:18.275281 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:18.255299 26449 raft_consensus.cc:2804] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:09:18.276291 26212 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:18.276651 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:18.257126 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:18.276604 26449 raft_consensus.cc:697] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:18.278800 26449 consensus_queue.cc:237] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:18.250428 26269 raft_consensus.cc:2749] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:18.281208 26395 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:18.283298 26395 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:18.284076 26343 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:18.284739 23028 raft_consensus.cc:1275] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 13:09:18.276234 23623 raft_consensus.cc:2376] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11. Re-sending same reply.
I20260709 13:09:18.285717 26407 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:18.285852 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:18.287067 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:18.257400 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:18.258030 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:18.258230 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:18.288072 23021 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 8.
I20260709 13:09:18.288523 23025 raft_consensus.cc:2468] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 7.
I20260709 13:09:18.256009 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:18.293166 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:18.294422 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:18.296864 22878 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11.
I20260709 13:09:18.293974 23025 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:18.243582 26389 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 7. Current term is 8.
I20260709 13:09:18.302605 26176 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:18.287693 26074 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:18.304879 22880 raft_consensus.cc:3055] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:09:18.305308 22880 raft_consensus.cc:740] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:18.305436 26074 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:18.306090 22880 consensus_queue.cc:260] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:18.307060 22871 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:18.307735 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:18.294179 23028 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 9.
I20260709 13:09:18.318502 23022 raft_consensus.cc:2468] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 4.
I20260709 13:09:18.341310 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:18.342483 23590 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:18.343099 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:18.343924 22878 raft_consensus.cc:2393] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 5: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:18.345737 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:18.346204 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:18.265573 26405 raft_consensus.cc:2804] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:09:18.262219 26409 raft_consensus.cc:3055] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:09:18.350903 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:09:18.359819 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 136922us
I20260709 13:09:18.360435 22878 raft_consensus.cc:2368] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 4. Current term is 5.
I20260709 13:09:18.275458 26357 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:18.306607 26386 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:18.372313 23588 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:18.400754 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:18.402468 22954 raft_consensus.cc:1275] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:09:18.355883 22880 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:18.381373 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:18.404222 22949 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
W20260709 13:09:18.387910 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:18.393537 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:18.378398 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:18.406193 22946 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:09:18.416251 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 175217us
I20260709 13:09:18.419044 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:18.419420 22949 raft_consensus.cc:1275] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:09:18.421162 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:18.425014 22843 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:18.435253 22955 raft_consensus.cc:1240] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:18.438709 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:18.358879 26409 raft_consensus.cc:740] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:18.441530 22843 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:18.362828 26074 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:18.449389 23025 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 10.
I20260709 13:09:18.451334 22871 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
I20260709 13:09:18.452091 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:18.354049 26343 consensus_queue.cc:1048] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.079s
I20260709 13:09:18.453859 22952 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 520) took 1802 ms. Trace:
W20260709 13:09:18.436195 26141 log.cc:927] Time spent T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:09:18.457598 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:18.457311 22879 raft_consensus.cc:2393] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 9: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:18.435056 26311 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:18.460120 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:18.456389 22952 rpcz_store.cc:276] 0709 13:09:16.651672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:16.651818 (+   146us) service_pool.cc:224] Handling call
0709 13:09:16.662651 (+ 10833us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:17.839897 (+1177246us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:18.453529 (+613632us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b540037b4d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:18.453772 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2705378176}
I20260709 13:09:18.419384 26405 raft_consensus.cc:697] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 6 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
W20260709 13:09:18.472289 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 128967us
I20260709 13:09:18.429124 26453 raft_consensus.cc:2804] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:18.473802 23029 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 8.
I20260709 13:09:18.347500 26385 consensus_queue.cc:260] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:18.474777 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:18.478313 23025 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:18.473690 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:18.478894 22955 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:09:18.475454 22880 raft_consensus.cc:1275] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:09:18.453197 26267 consensus_queue.cc:1059] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:09:18.482465 26267 raft_consensus.cc:3055] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:09:18.482805 26267 raft_consensus.cc:740] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:18.483625 26267 consensus_queue.cc:260] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:18.461179 26401 consensus_queue.cc:1059] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:09:18.484979 26343 raft_consensus.cc:2804] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:09:18.485505 26401 raft_consensus.cc:3055] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:09:18.485889 26401 raft_consensus.cc:740] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:18.485723 26409 consensus_queue.cc:260] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:18.486718 26401 consensus_queue.cc:260] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:18.487142 26409 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:18.487326 26141 tablet_bootstrap.cc:492] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:18.462378 26454 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:18.487932 26401 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:18.488497 26141 tablet_bootstrap.cc:492] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap complete.
I20260709 13:09:18.488212 26454 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:18.489058 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:18.489774 23618 raft_consensus.cc:2393] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 9: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:18.485526 26343 raft_consensus.cc:697] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 4 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:18.490365 26454 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:18.490701 26343 consensus_queue.cc:237] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:18.492805 22946 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
I20260709 13:09:18.492834 26267 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:18.429243 26357 consensus_queue.cc:1048] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:09:18.491842 26141 ts_tablet_manager.cc:1403] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2: Time spent bootstrapping tablet: real 3.258s	user 0.031s	sys 0.043s
I20260709 13:09:18.457171 26403 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:18.496721 26403 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:18.498443 26455 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:18.471158 26311 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:18.460563 25595 raft_consensus.cc:2804] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:09:18.507210 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 135016us
I20260709 13:09:18.509253 23590 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:18.485744 22993 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:18.509557 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:18.509657 23588 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:18.510632 23590 leader_election.cc:400] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 4 election: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 4. Current term is 5.
I20260709 13:09:18.511395 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:18.492213 26212 consensus_queue.cc:1059] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:09:18.521155 23027 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 563) took 1731 ms. Trace:
I20260709 13:09:18.522681 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:18.521289 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:18.524142 22879 raft_consensus.cc:1240] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 10. Current term is 11. Ops: [8.8-10.9]
I20260709 13:09:18.474085 26453 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:18.499558 26437 raft_consensus.cc:2749] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:09:18.529968 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 50849.4us
I20260709 13:09:18.454608 26074 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:18.522739 23027 rpcz_store.cc:276] 0709 13:09:16.789581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:16.813869 (+ 24288us) service_pool.cc:224] Handling call
0709 13:09:16.814287 (+   418us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:16.961417 (+147130us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:16.961458 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:16.968573 (+  7115us) log.cc:844] Serialized 52 byte log entry
0709 13:09:18.174470 (+1205897us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b1800412f00. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cdae07 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf7c02 00007f613597f20b 00007f6135984d23 00007f61359850d1 00007f6135da0a0f
0709 13:09:18.453356 (+278886us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:18.453392 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:18.453454 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:18.501803 (+ 48349us) raft_consensus.cc:1685] finished
0709 13:09:18.501940 (+   137us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:18.521084 (+ 19144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":677155,"raft.queue_time_us":17949,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":55524,"spinlock_wait_cycles":24248576,"thread_start_us":278705,"threads_started":1,"wal-append.queue_time_us":1515969}
I20260709 13:09:18.498965 26403 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:18.509012 26311 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:18.530653 22843 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:18.508695 25595 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:18.498965 26455 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:18.527417 26449 raft_consensus.cc:2749] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:09:18.528918 26385 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:18.527765 26453 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:18.525267 26357 consensus_queue.cc:1048] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:09:18.542936 25595 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:09:18.562882 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 77204.1us
W20260709 13:09:18.568581 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 81046.7us
I20260709 13:09:18.546940 26269 raft_consensus.cc:2804] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:18.546775 26403 raft_consensus.cc:2804] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:18.534098 25707 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:18.561383 26458 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:18.583132 22993 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:09:18.584995 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 54425.4us
I20260709 13:09:18.575667 26269 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:18.586498 23025 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 9.
I20260709 13:09:18.579531 26403 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:18.585664 22843 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:18.546679 26405 consensus_queue.cc:237] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:18.596419 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:18.607035 23617 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:09:18.590787 26403 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:18.601135 26141 raft_consensus.cc:359] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:09:18.585914 26458 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:18.579161 26425 consensus_queue.cc:1048] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.012s
I20260709 13:09:18.590534 26460 raft_consensus.cc:2749] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:18.605724 26074 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:18.593956 22843 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:18.588411 26269 raft_consensus.cc:3060] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:18.589380 26440 raft_consensus.cc:2804] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:09:18.605724 26457 log.cc:927] Time spent T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:09:18.645794 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:09:18.652872 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 151221us
I20260709 13:09:18.654101 22992 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:18.630698 26141 raft_consensus.cc:740] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 LEARNER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Initialized, Role: LEARNER
I20260709 13:09:18.666661 26141 consensus_queue.cc:260] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:09:18.669730 26141 ts_tablet_manager.cc:1434] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2: Time spent starting tablet: real 0.171s	user 0.006s	sys 0.003s
I20260709 13:09:18.633847 26464 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
W20260709 13:09:18.673981 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 80068.2us
I20260709 13:09:18.674779 22843 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:18.675644 22843 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:18.654297 26357 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:18.684170 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:18.659981 26467 raft_consensus.cc:2804] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:09:18.686728 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 103751us
I20260709 13:09:18.687914 22844 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:18.658209 26437 consensus_queue.cc:1048] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
W20260709 13:09:18.694088 26465 log.cc:927] Time spent T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:09:18.697592 26458 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:18.698372 26458 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:18.698866 26458 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:18.700382 25707 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:18.701983 26074 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:18.702541 26074 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:18.715087 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:18.715721 23620 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:18.704487 26074 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:18.717504 26074 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:18.718006 26074 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:18.718683 26442 log.cc:927] Time spent T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.597s	user 0.009s	sys 0.003s
I20260709 13:09:18.718521 26403 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:18.719871 22950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2410) took 1965 ms. Trace:
I20260709 13:09:18.720207 26074 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:18.721351 23621 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 730) took 2088 ms. Trace:
I20260709 13:09:18.721457 25595 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:18.721741 23621 rpcz_store.cc:276] 0709 13:09:16.632735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:16.632866 (+   131us) service_pool.cc:224] Handling call
0709 13:09:16.654646 (+ 21780us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:17.526073 (+871427us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:18.721139 (+1195066us) spinlock_profiling.cc:243] Waited 870 ms on lock 0x7b54001108d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:18.721299 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2001646848}
I20260709 13:09:18.723464 25595 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:18.723605 26444 log.cc:927] Time spent T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.860s	user 0.003s	sys 0.007s
I20260709 13:09:18.724107 25595 raft_consensus.cc:2749] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:18.660456 26460 consensus_queue.cc:1059] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:09:18.724327 26269 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:18.720217 22950 rpcz_store.cc:276] 0709 13:09:16.754766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:16.754949 (+   183us) service_pool.cc:224] Handling call
0709 13:09:16.756408 (+  1459us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:16.760317 (+  3909us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:16.760354 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:16.829539 (+ 69185us) log.cc:844] Serialized 26 byte log entry
0709 13:09:17.073963 (+244424us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:09:17.074023 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:17.074122 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:18.718071 (+1643949us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:09:18.719569 (+  1498us) raft_consensus.cc:1685] finished
0709 13:09:18.719587 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:18.719812 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3439,"raft.queue_time_us":4030,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":29079,"spinlock_wait_cycles":39781248,"thread_start_us":165764,"threads_started":1,"wal-append.queue_time_us":1263862}
I20260709 13:09:18.688195 26467 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:18.685705 26357 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:18.753831 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 66247.2us
I20260709 13:09:18.727525 25595 raft_consensus.cc:2749] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:09:18.679538 26464 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:18.751412 26467 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:18.755470 26477 raft_consensus.cc:3055] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:09:18.733382 26478 raft_consensus.cc:2804] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:18.780632 26458 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:18.789428 26212 raft_consensus.cc:2749] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:09:18.823757 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 108429us
I20260709 13:09:18.826419 23023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 1807) took 2070 ms. Trace:
I20260709 13:09:18.792096 26471 consensus_queue.cc:1059] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 13:09:18.834127 26458 raft_consensus.cc:3055] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:09:18.834692 26458 raft_consensus.cc:740] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
W20260709 13:09:18.834827 26476 log.cc:927] Time spent T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:09:18.828329 23023 rpcz_store.cc:276] 0709 13:09:16.759807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:16.765043 (+  5236us) service_pool.cc:224] Handling call
0709 13:09:16.818271 (+ 53228us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:16.967179 (+148908us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:18.826086 (+1858907us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b54001c75d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:18.826343 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":340553984}
W20260709 13:09:18.837669 26469 log.cc:927] Time spent T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:09:18.806556 26477 raft_consensus.cc:740] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:18.817703 26478 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:18.849365 26478 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:18.850670 26357 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:18.852844 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:18.854061 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:18.854630 22872 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 9.
W20260709 13:09:18.857884 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:18.858577 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:18.859764 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 184144us
I20260709 13:09:18.847925 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:18.849016 23620 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11.
W20260709 13:09:18.870551 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 211703us
I20260709 13:09:18.873338 25707 consensus_queue.cc:1048] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:09:18.873564 26492 consensus_queue.cc:1059] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:09:18.875568 25707 raft_consensus.cc:3055] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:09:18.869197 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 70882.8us
I20260709 13:09:18.835911 26458 consensus_queue.cc:260] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:18.880172 22993 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:18.881588 23590 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:18.857875 26478 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:18.880409 22992 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:18.888340 22844 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:18.875948 25707 raft_consensus.cc:740] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:18.903980 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:09:18.907275 26431 log.cc:927] Time spent T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 1.374s	user 0.006s	sys 0.013s
W20260709 13:09:18.909970 26446 log.cc:927] Time spent T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.796s	user 0.017s	sys 0.003s
I20260709 13:09:18.910400 26458 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:18.911682 23619 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 736) took 1923 ms. Trace:
I20260709 13:09:18.912825 26480 raft_consensus.cc:2749] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:18.915282 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:18.915498 26492 raft_consensus.cc:2764] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:09:18.915896 22875 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
I20260709 13:09:18.896270 26425 raft_consensus.cc:2804] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:09:18.916487 26481 consensus_queue.cc:1059] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.076s
I20260709 13:09:18.917233 26425 raft_consensus.cc:697] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:18.917565 26481 raft_consensus.cc:3055] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:09:18.917974 26481 raft_consensus.cc:740] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:18.918560 26425 consensus_queue.cc:237] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:18.970649 23588 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:18.943722 26403 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:19.105947 26269 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:19.106621 26477 consensus_queue.cc:260] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:18.891520 26472 raft_consensus.cc:2749] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:09:19.109526 25707 consensus_queue.cc:260] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:19.110877 25707 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:18.944188 26440 raft_consensus.cc:697] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:19.114835 26440 consensus_queue.cc:237] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:18.918790 26481 consensus_queue.cc:260] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:19.108002 26477 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:18.898681 26405 raft_consensus.cc:2764] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:09:18.912083 23619 rpcz_store.cc:276] 0709 13:09:16.988725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:16.988874 (+   149us) service_pool.cc:224] Handling call
0709 13:09:16.989184 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:17.072220 (+ 83036us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:17.072276 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:17.136602 (+ 64326us) log.cc:844] Serialized 26 byte log entry
0709 13:09:17.850113 (+713511us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:17.850157 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:18.791333 (+941176us) spinlock_profiling.cc:243] Waited 71.4 ms on lock 0x7b54003d9ad4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:18.791403 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:18.909285 (+117882us) spinlock_profiling.cc:243] Waited 56.2 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:09:18.911449 (+  2164us) raft_consensus.cc:1685] finished
0709 13:09:18.911472 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:18.911623 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10911,"raft.queue_time_us":13460,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":66676,"spinlock_wait_cycles":293525120,"thread_start_us":399674,"threads_started":1,"wal-append.queue_time_us":834800}
W20260709 13:09:19.129029 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:19.127565 26453 consensus_meta.cc:287] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.557s	user 0.000s	sys 0.011s
W20260709 13:09:19.130493 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:19.137959 26478 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:19.141325 26481 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:09:19.146113 22843 connection.cc:256] RPC call timeout handler was delayed by -2.1746108940000113s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1966s for a response
W20260709 13:09:19.148475 26311 consensus_meta.cc:287] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.601s	user 0.009s	sys 0.001s
I20260709 13:09:19.162834 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:19.142221 26453 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:19.189386 26386 consensus_meta.cc:287] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.771s	user 0.007s	sys 0.003s
I20260709 13:09:19.189985 26386 raft_consensus.cc:2749] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:19.190752 22879 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:19.198544 22879 raft_consensus.cc:1275] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:09:19.206775 23619 raft_consensus.cc:1240] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:19.215826 26467 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:19.218976 26467 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:19.224256 26448 log.cc:927] Time spent T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 1.021s	user 0.004s	sys 0.003s
I20260709 13:09:19.229215 26464 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:19.231971 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:19.237551 26462 raft_consensus.cc:3055] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:09:19.238113 26462 raft_consensus.cc:740] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:19.239151 26462 consensus_queue.cc:260] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:19.242388 22874 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 10.
I20260709 13:09:19.243177 22874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53724 (request call id 534) took 1049 ms. Trace:
I20260709 13:09:19.243842 22874 rpcz_store.cc:276] 0709 13:09:18.193453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:19.231693 (+1038240us) service_pool.cc:224] Handling call
0709 13:09:19.243108 (+ 11415us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:19.244880 26347 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:19.246984 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:19.250540 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:19.251070 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:19.252650 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:19.253396 22879 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:09:19.254565 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:19.255228 22879 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:19.256790 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:19.240427 26462 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 6
W20260709 13:09:19.258186 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 377734us
W20260709 13:09:19.260021 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 45373.8us
I20260709 13:09:19.260073 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:19.260896 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:19.262754 22879 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
I20260709 13:09:19.263831 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:19.265342 22879 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:19.278229 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:19.279258 22873 raft_consensus.cc:2393] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 5: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:19.280005 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:19.277827 22992 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:19.171478 26311 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:19.302577 22946 raft_consensus.cc:1275] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:09:19.301092 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:19.306643 22953 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:19.305836 23588 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:19.299737 26267 raft_consensus.cc:2804] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:09:19.321889 22876 raft_consensus.cc:1240] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:09:19.269318 26467 consensus_queue.cc:1059] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 13:09:19.327435 22879 raft_consensus.cc:1275] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:09:19.325321 23616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60044 (request call id 671) took 1058 ms. Trace:
I20260709 13:09:19.326186 26349 raft_consensus.cc:2804] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:09:19.328728 23616 rpcz_store.cc:276] 0709 13:09:18.267061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:18.271811 (+  4750us) service_pool.cc:224] Handling call
0709 13:09:18.272087 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:18.272308 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:09:18.455968 (+183660us) log.cc:844] Serialized 14 byte log entry
0709 13:09:18.566961 (+110993us) log.cc:844] Serialized 14 byte log entry
0709 13:09:19.187651 (+620690us) spinlock_profiling.cc:243] Waited 5.89 ms on lock 0x7b18001b4a20. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cdae07 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf5455 00007f6135ddd060 00007f6135def716 00007f6135e00287 00007f6135e00208
0709 13:09:19.324871 (+137220us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:19.324936 (+    65us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:09:19.325003 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:19.325085 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:19.325241 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15756,"prepare.queue_time_us":279700,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":70,"raft.queue_time_us":36667,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":4690,"spinlock_wait_cycles":28710400,"thread_start_us":373410,"threads_started":3,"wal-append.queue_time_us":112385}
I20260709 13:09:19.345525 26347 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:19.345281 26236 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:19.365969 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:19.366954 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
W20260709 13:09:19.371276 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 224420us
W20260709 13:09:19.350383 26381 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.433s	user 0.034s	sys 0.021s
W20260709 13:09:19.380098 26381 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.463s	user 0.034s	sys 0.021s
W20260709 13:09:19.385737 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:19.390697 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:19.393193 23616 raft_consensus.cc:3060] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:09:19.396245 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:19.396637 22843 connection.cc:256] RPC call timeout handler was delayed by -1.9723374320000175s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1948s for a response
W20260709 13:09:19.397008 22843 connection.cc:256] RPC call timeout handler was delayed by -1.9615256670000463s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20120000000000002s for a response
W20260709 13:09:19.397313 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:19.397778 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:19.398250 22843 connection.cc:256] RPC call timeout handler was delayed by -1.5720627819999891s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1573s for a response
W20260709 13:09:19.398535 22843 connection.cc:256] RPC call timeout handler was delayed by -1.5392108940000071s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.26720000000000005s for a response
W20260709 13:09:19.398818 22843 connection.cc:256] RPC call timeout handler was delayed by -1.3489860619999945s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20579999999999998s for a response
I20260709 13:09:19.396636 23621 raft_consensus.cc:1240] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 4. Current term is 5. Ops: []
W20260709 13:09:19.399125 22843 connection.cc:256] RPC call timeout handler was delayed by -1.1468108940000548s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3108s for a response
W20260709 13:09:19.399580 22843 connection.cc:256] RPC call timeout handler was delayed by -1.1085627820000354s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2088s for a response
W20260709 13:09:19.399955 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:19.399963 22843 connection.cc:256] RPC call timeout handler was delayed by -1.0455627820000473s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2158s for a response
W20260709 13:09:19.401129 22843 connection.cc:256] RPC call timeout handler was delayed by -1.0194627820000051s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2187s for a response
I20260709 13:09:19.402127 22843 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:19.372991 26279 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.628s	user 0.010s	sys 0.021s
I20260709 13:09:19.369642 26349 raft_consensus.cc:697] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 9 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
W20260709 13:09:19.456089 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 54052.5us
W20260709 13:09:19.429087 26434 log.cc:927] Time spent T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 1.926s	user 0.016s	sys 0.012s
I20260709 13:09:19.470042 22843 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:19.473255 23623 raft_consensus.cc:1240] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:19.473434 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:19.431192 26482 raft_consensus.cc:2749] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:09:19.473953 22919 leader_election.cc:336] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Timed out: RequestConsensusVote RPC to 127.21.63.4:44043 timed out after 1.814s (SENT)
I20260709 13:09:19.476330 23588 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:19.485348 23616 raft_consensus.cc:2468] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
I20260709 13:09:19.403297 26471 consensus_queue.cc:1048] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 13:09:19.486733 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:19.487469 26311 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:19.489163 22878 raft_consensus.cc:1275] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:09:19.489614 23621 raft_consensus.cc:1240] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:09:19.487520 26381 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:19.491636 22843 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:19.492199 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:19.497623 23619 raft_consensus.cc:2410] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
W20260709 13:09:19.466853 26279 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.722s	user 0.010s	sys 0.021s
I20260709 13:09:19.457330 26267 raft_consensus.cc:697] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:19.494100 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:19.372439 26347 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:19.512966 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:19.513541 23025 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:19.499447 22953 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11.
I20260709 13:09:19.500088 26267 consensus_queue.cc:237] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:19.478843 26289 consensus_queue.cc:260] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:19.507269 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:19.526924 23028 raft_consensus.cc:1240] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:19.527925 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:19.511952 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:19.529090 23029 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 11.
I20260709 13:09:19.394065 26236 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:19.510632 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:19.509613 22993 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:19.491348 26455 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:19.496563 25595 raft_consensus.cc:2749] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:09:19.497459 26405 raft_consensus.cc:2749] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:09:19.551594 22843 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:19.547577 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:19.553897 23024 raft_consensus.cc:2468] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 7.
W20260709 13:09:19.536401 26490 log.cc:927] Time spent T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:09:19.525935 26482 raft_consensus.cc:2749] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:09:19.556509 26389 raft_consensus.cc:2804] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:19.498569 25707 consensus_queue.cc:1059] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 13:09:19.559490 23024 raft_consensus.cc:1275] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 13:09:19.560928 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:19.561203 22843 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:19.538705 26471 consensus_queue.cc:1059] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
W20260709 13:09:19.568491 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 58955.9us
W20260709 13:09:19.576207 22871 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35360 (request call id 1812) took 1338 ms (client timeout 1562 ms). Trace:
I20260709 13:09:19.592568 22843 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:19.576979 26349 consensus_queue.cc:237] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:19.515453 26381 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:19.568604 26440 raft_consensus.cc:2804] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:09:19.564160 26389 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:19.567381 23024 raft_consensus.cc:1275] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 13:09:19.581231 22876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53724 (request call id 544) took 1235 ms. Trace:
W20260709 13:09:19.604192 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:19.603467 22871 rpcz_store.cc:269] 0709 13:09:18.238233 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:18.251032 (+ 12799us) service_pool.cc:224] Handling call
0709 13:09:19.575944 (+1324912us) spinlock_profiling.cc:243] Waited 48.7 ms on lock 0x7b5400369554. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:19.576116 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":111964544}
I20260709 13:09:19.621409 22876 rpcz_store.cc:276] 0709 13:09:18.345345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:18.660246 (+314901us) service_pool.cc:224] Handling call
0709 13:09:18.677849 (+ 17603us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:19.324528 (+646679us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:19.580850 (+256322us) spinlock_profiling.cc:243] Waited 644 ms on lock 0x7b5400369054. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:19.581146 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1480551808}
I20260709 13:09:19.602488 23025 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 9.
I20260709 13:09:19.595103 25707 raft_consensus.cc:3055] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:09:19.591588 26481 raft_consensus.cc:2749] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:19.607780 22992 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:19.607767 22843 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:19.608886 23588 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:19.628387 23588 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:19.595913 26289 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:19.602646 26495 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:19.619410 25595 raft_consensus.cc:2804] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:09:19.621989 26389 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:09:19.631011 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 122278us
I20260709 13:09:19.635069 26405 raft_consensus.cc:2804] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:19.625787 25707 raft_consensus.cc:740] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:19.625173 26440 raft_consensus.cc:697] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:19.630559 22843 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:09:19.638175 26443 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.447s	user 0.003s	sys 0.025s
I20260709 13:09:19.624514 26482 consensus_queue.cc:1048] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:09:19.637053 26488 raft_consensus.cc:2804] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:19.656976 22843 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:19.606096 26505 consensus_queue.cc:1048] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.046s
I20260709 13:09:19.665743 25707 consensus_queue.cc:260] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:19.666236 26311 raft_consensus.cc:2749] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:09:19.667462 26488 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:19.667922 26488 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:19.671337 26381 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:19.664563 22843 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:19.674602 26482 raft_consensus.cc:2749] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 13:09:19.660682 26515 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:19.677877 26515 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:19.680230 26515 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:19.679286 22843 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:19.640487 26496 raft_consensus.cc:2804] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:19.681141 26496 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:19.681509 26496 raft_consensus.cc:3060] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:19.681738 26482 raft_consensus.cc:2804] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:19.682111 26482 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:19.682448 26482 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:19.639986 26487 raft_consensus.cc:2804] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:09:19.685127 26487 raft_consensus.cc:697] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:09:19.685429 26513 log.cc:927] Time spent T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:09:19.685992 26487 consensus_queue.cc:237] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:19.660807 22919 connection.cc:256] RPC call timeout handler was delayed by -2.59220222700003s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2118s for a response
I20260709 13:09:19.662326 26481 raft_consensus.cc:2804] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:09:19.710269 26481 raft_consensus.cc:697] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 7 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
W20260709 13:09:19.706902 22919 connection.cc:256] RPC call timeout handler was delayed by -2.4269823040000347s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2167s for a response
W20260709 13:09:19.659456 26443 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.473s	user 0.004s	sys 0.025s
I20260709 13:09:19.711640 26481 consensus_queue.cc:237] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:19.654228 26405 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:19.638623 26449 consensus_queue.cc:1059] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.073s
W20260709 13:09:19.713213 22919 connection.cc:256] RPC call timeout handler was delayed by -1.9424991920000139s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1878s for a response
W20260709 13:09:19.729000 22919 connection.cc:256] RPC call timeout handler was delayed by -1.4014644879999878s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1505s for a response
W20260709 13:09:19.729391 22919 connection.cc:256] RPC call timeout handler was delayed by -1.2477245850000145s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1839s for a response
I20260709 13:09:19.730822 22953 raft_consensus.cc:1240] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:09:19.635393 26502 consensus_queue.cc:1048] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 13:09:19.663609 25595 raft_consensus.cc:697] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 9 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:19.706547 26515 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
W20260709 13:09:19.738580 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 130863us
I20260709 13:09:19.742523 22953 raft_consensus.cc:1240] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 10. Current term is 11. Ops: [5.7-10.10]
I20260709 13:09:19.744431 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:19.645217 26495 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:19.749536 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:19.751022 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:19.728497 26405 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:19.757759 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:19.751829 22955 raft_consensus.cc:2410] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:09:19.757093 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:19.759490 22950 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:09:19.760612 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:19.761543 22946 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 8.
I20260709 13:09:19.767403 23588 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:19.768620 22920 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:19.770727 23024 raft_consensus.cc:1275] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:09:19.771337 22948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2425) took 1353 ms. Trace:
I20260709 13:09:19.771440 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:19.772087 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:19.772589 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:19.773151 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:19.773835 22952 raft_consensus.cc:2393] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 10: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:19.773653 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:19.771368 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:19.771764 22948 rpcz_store.cc:276] 0709 13:09:18.417958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:18.440753 (+ 22795us) service_pool.cc:224] Handling call
0709 13:09:18.454762 (+ 14009us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:18.550189 (+ 95427us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:18.550229 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:18.550286 (+    57us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:09:18.550342 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:19.771121 (+1220779us) spinlock_profiling.cc:243] Waited 95.1 ms on lock 0x7b54000a87d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:19.771175 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:19.771283 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":218743168}
I20260709 13:09:19.778333 22949 raft_consensus.cc:2393] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 11: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:19.781867 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:19.782851 22951 raft_consensus.cc:2410] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 13:09:19.769500 26347 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:19.799759 22949 raft_consensus.cc:1240] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:09:19.799978 22955 raft_consensus.cc:1240] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 8. Current term is 9. Ops: []
W20260709 13:09:19.800967 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:19.780102 26464 raft_consensus.cc:2804] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:09:19.779911 26517 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:09:19.775373 26436 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:19.748831 26515 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:19.799572 22965 tablet_copy_service.cc:342] P aed3f089ec8b49af9c8cf5cdee76e8e7: Request end of tablet copy session 5b346767a89745fa82928a5dac9cc0b2-7426119f4da44ffaa1513b481a866a01 received from {username='slave'} at 127.0.0.1:52222
W20260709 13:09:19.794691 26428 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.982s	user 0.048s	sys 0.114s
I20260709 13:09:19.811414 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:19.833446 22953 raft_consensus.cc:1275] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 13:09:19.833813 22953 pending_rounds.cc:85] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting all ops after (but not including) 6
I20260709 13:09:19.834125 22953 pending_rounds.cc:107] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260709 13:09:19.834045 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:19.834493 22953 raft_consensus.cc:2889] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:19.778307 26489 consensus_queue.cc:1048] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
W20260709 13:09:19.820814 26524 log.cc:927] Time spent T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 13:09:19.822130 26495 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:19.780964 26440 consensus_queue.cc:237] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:19.785501 26486 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:19.824450 22919 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:19.830137 22950 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 9.
I20260709 13:09:19.765503 26488 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:19.812608 26521 log.cc:927] Time spent T 00000000000000000000000000000000 P 8d175b4ae72f4fc48971c2f31668971c: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:09:19.836711 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:19.834657 22955 raft_consensus.cc:2376] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 7. Re-sending same reply.
W20260709 13:09:19.834520 26522 log.cc:927] Time spent T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
W20260709 13:09:19.835989 26428 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.023s	user 0.048s	sys 0.114s
I20260709 13:09:19.794245 26398 consensus_queue.cc:1048] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:09:19.848804 22954 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:19.848522 22919 leader_election.cc:400] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 5. Current term is 7.
I20260709 13:09:19.854290 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:19.863765 22955 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
I20260709 13:09:19.865342 22955 raft_consensus.cc:1275] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:09:19.867137 22992 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:19.868587 26505 raft_consensus.cc:2804] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:09:19.869179 26505 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:19.869623 26505 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:19.880913 22919 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:09:19.853091 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 82601.4us
I20260709 13:09:19.882850 22920 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:19.883879 26289 raft_consensus.cc:2749] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:19.885778 22919 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:19.887176 26289 raft_consensus.cc:2749] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:09:19.837651 26517 raft_consensus.cc:697] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:19.860461 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:19.892627 22843 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:19.820034 26482 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:19.893715 22919 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:19.847903 22965 tablet_copy_service.cc:434] P aed3f089ec8b49af9c8cf5cdee76e8e7: ending tablet copy session 5b346767a89745fa82928a5dac9cc0b2-7426119f4da44ffaa1513b481a866a01 on tablet 7426119f4da44ffaa1513b481a866a01 with peer 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:19.837811 26436 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:19.906883 26381 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:19.848119 26486 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:19.945161 26464 raft_consensus.cc:697] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 9 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:19.946559 26515 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:19.946889 26471 raft_consensus.cc:3055] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:09:19.825469 25595 consensus_queue.cc:237] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:19.914053 26462 raft_consensus.cc:2749] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:09:19.894430 26289 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:19.968153 22954 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 10.
W20260709 13:09:19.968691 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 66439.5us
W20260709 13:09:19.966023 26439 log.cc:927] Time spent T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 1.792s	user 0.009s	sys 0.009s
W20260709 13:09:19.996392 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 181346us
I20260709 13:09:19.995519 22919 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:09:19.998694 23588 leader_election.cc:336] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Timed out: RequestConsensusVote RPC to 127.21.63.2:33309 timed out after 1.670s (SENT)
I20260709 13:09:19.978917 26471 raft_consensus.cc:740] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:19.953399 26311 consensus_queue.cc:1059] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 13:09:20.012101 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:20.013492 23621 raft_consensus.cc:2410] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:09:20.012362 26539 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:20.014815 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:20.015045 26539 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:20.015597 23616 raft_consensus.cc:2393] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 10: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:20.016397 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:20.017431 26539 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:20.018142 26539 raft_consensus.cc:2749] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:09:20.018754 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:20.019707 26531 consensus_queue.cc:1059] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 13:09:20.018713 26539 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:19.974870 26389 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:20.012174 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:20.025954 23624 raft_consensus.cc:2393] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 10: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:20.016198 26464 consensus_queue.cc:237] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:20.015759 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:20.029543 23619 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
I20260709 13:09:20.021566 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:19.976209 26505 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:20.023123 23588 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:20.023123 23623 raft_consensus.cc:1275] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:09:19.996902 26289 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:20.024844 26489 consensus_queue.cc:1059] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 13:09:20.033494 26436 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:20.034173 26515 raft_consensus.cc:3055] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:09:20.038105 26486 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:20.040328 26389 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:20.041427 23020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 560) took 3282 ms. Trace:
I20260709 13:09:20.051915 23588 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:20.060827 23020 rpcz_store.cc:276] 0709 13:09:16.758798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:16.758926 (+   128us) service_pool.cc:224] Handling call
0709 13:09:16.759235 (+   309us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:16.812156 (+ 52921us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:16.812208 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:16.868588 (+ 56380us) log.cc:844] Serialized 26 byte log entry
0709 13:09:17.248220 (+379632us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:09:17.248273 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:17.248390 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:19.809677 (+2561287us) spinlock_profiling.cc:243] Waited 6.81 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:09:20.041122 (+231445us) raft_consensus.cc:1685] finished
0709 13:09:20.041143 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:20.041350 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":98149,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":18792,"spinlock_wait_cycles":15657216,"thread_start_us":321881,"threads_started":1,"wal-append.queue_time_us":1256536}
I20260709 13:09:20.067119 23588 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:20.069800 23588 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:20.053346 26289 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:20.076422 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:20.053637 26349 raft_consensus.cc:2749] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:20.083102 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:20.077762 23623 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 LEARNER]: Advancing to term 8
W20260709 13:09:20.078166 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:20.080181 26530 log.cc:927] Time spent T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 13:09:20.058748 26515 raft_consensus.cc:740] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:20.080596 26505 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:20.092115 23588 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:20.092650 22872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53724 (request call id 551) took 1296 ms. Trace:
I20260709 13:09:20.051860 26539 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:20.074923 26482 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:20.093024 22872 rpcz_store.cc:276] 0709 13:09:18.796138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:18.855153 (+ 59015us) service_pool.cc:224] Handling call
0709 13:09:18.855442 (+   289us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:18.855788 (+   346us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:18.855815 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:18.856238 (+   423us) log.cc:844] Serialized 26 byte log entry
0709 13:09:19.244461 (+388223us) spinlock_profiling.cc:243] Waited 6.7 ms on lock 0x7b1800349120. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cf83f3 00007f6132cfa7c1 00007f6132cfa779 00007f6132cfa709 00007f6132cfa6d1 00007f6132cf99cd 000000000038f354 000000000038f199 00007f6132cdb83a
0709 13:09:20.092186 (+847725us) spinlock_profiling.cc:243] Waited 10.7 ms on lock 0x7b1800349120. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f613598d4bc 00007f6132cdb48b 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf7c02 00007f613597f20b 00007f6135984d23 00007f61359850d1
0709 13:09:20.092337 (+   151us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:20.092370 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:20.092433 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:20.092481 (+    48us) raft_consensus.cc:1685] finished
0709 13:09:20.092493 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:20.092601 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10351,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":13108,"spinlock_wait_cycles":39994496,"thread_start_us":26467,"threads_started":1,"wal-append.queue_time_us":26384}
I20260709 13:09:20.096220 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:20.098049 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:20.071841 26357 raft_consensus.cc:2804] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:20.078575 26536 raft_consensus.cc:2749] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:09:20.082360 26486 raft_consensus.cc:2749] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:20.002912 26517 consensus_queue.cc:237] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:20.114087 23623 raft_consensus.cc:1275] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 LEARNER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:09:20.115216 26532 consensus_queue.cc:1048] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 13:09:20.117518 26533 raft_consensus.cc:2749] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:09:20.118068 26539 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:20.118312 23621 raft_consensus.cc:1240] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:09:20.101926 26357 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:20.123062 26498 raft_consensus.cc:3055] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:09:20.123656 26498 raft_consensus.cc:740] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:20.124915 26498 consensus_queue.cc:260] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:20.126781 26498 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:20.129144 26472 raft_consensus.cc:3055] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:09:20.129652 26472 raft_consensus.cc:740] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:20.130646 26472 consensus_queue.cc:260] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:20.132171 26472 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:20.141166 26496 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:20.118328 26538 log.cc:927] Time spent T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:09:20.148911 26357 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:20.151350 26453 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:20.106927 26488 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:20.181739 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 88538.6us
W20260709 13:09:20.186686 22947 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52222 (request call id 516) took 4367 ms (client timeout 1670 ms). Trace:
W20260709 13:09:20.187131 22947 rpcz_store.cc:269] 0709 13:09:15.819191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:15.819356 (+   165us) service_pool.cc:224] Handling call
0709 13:09:20.186437 (+4367081us) spinlock_profiling.cc:243] Waited 3.54 s on lock 0x7b5400148554. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:20.186593 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8136702336}
I20260709 13:09:20.178695 26496 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:20.192613 26515 consensus_queue.cc:260] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:20.193737 26515 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:20.194903 26405 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:20.197772 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:20.202486 22919 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:09:20.203131 22918 leader_election.cc:336] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.732s (SENT)
I20260709 13:09:20.203529 22918 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:20.208429 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:20.208698 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 107351us
I20260709 13:09:20.210206 23024 raft_consensus.cc:1275] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:09:20.210381 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:20.210848 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:20.211134 23028 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:20.212172 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:20.212319 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:20.215095 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:20.216157 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:20.218161 22918 leader_election.cc:336] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.969s (SENT)
I20260709 13:09:20.207228 26455 raft_consensus.cc:2804] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:20.218612 22918 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:20.218986 26455 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:20.219381 26455 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:20.207456 26385 raft_consensus.cc:2749] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:20.224704 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:20.225430 26385 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:20.226950 23021 raft_consensus.cc:1275] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:09:20.227537 26289 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:20.208429 26539 consensus_queue.cc:1048] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 13:09:20.230746 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:20.231309 23029 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
I20260709 13:09:20.235157 23021 raft_consensus.cc:1240] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:20.240263 22953 raft_consensus.cc:1275] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 13:09:20.254906 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:20.259600 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:20.260205 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:20.260418 23020 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 9.
I20260709 13:09:20.240526 26289 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:20.270903 26212 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:20.272212 26405 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:20.362028 23029 raft_consensus.cc:1275] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:09:20.365793 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:20.400178 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:20.375595 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:20.408825 23022 raft_consensus.cc:2410] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 10 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:09:20.409026 23020 raft_consensus.cc:2393] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 10: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:20.355830 22947 raft_consensus.cc:2410] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:09:20.383132 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:20.325582 26289 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:20.307063 26498 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:20.330749 26381 raft_consensus.cc:2749] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:20.424422 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:20.426644 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:20.347234 26392 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:20.376989 26311 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:20.376935 26488 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:20.430646 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:20.308924 26212 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:20.432801 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 209213us
I20260709 13:09:20.434386 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:20.435128 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:20.350109 26471 consensus_queue.cc:260] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:20.439272 22946 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:20.440348 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:20.443907 22878 raft_consensus.cc:1240] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 4. Current term is 5. Ops: []
I20260709 13:09:20.448947 22872 raft_consensus.cc:1240] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:09:20.448488 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:20.450821 22876 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:09:20.453649 22878 raft_consensus.cc:3055] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:09:20.454013 22878 raft_consensus.cc:740] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:20.455150 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:20.455991 22879 raft_consensus.cc:2410] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 7.
I20260709 13:09:20.431145 26289 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:20.432426 26392 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:20.432325 26311 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:20.445508 26471 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:20.472168 23028 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
W20260709 13:09:20.464406 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:20.488857 26289 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:20.458590 22994 connection.cc:256] RPC call timeout handler was delayed by -1.9663405559999774s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1898s for a response
I20260709 13:09:20.486238 23588 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:20.490814 22994 connection.cc:256] RPC call timeout handler was delayed by -1.8826405559999557s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19910000000000003s for a response
I20260709 13:09:20.491976 26458 raft_consensus.cc:2749] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:09:20.492030 22994 connection.cc:256] RPC call timeout handler was delayed by -1.8529405560000214s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20240000000000002s for a response
I20260709 13:09:20.493781 26458 consensus_queue.cc:1048] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:09:20.495023 22877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 1810) took 2271 ms. Trace:
I20260709 13:09:20.495453 22877 rpcz_store.cc:276] 0709 13:09:18.227362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:18.227518 (+   156us) service_pool.cc:224] Handling call
0709 13:09:18.235389 (+  7871us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:18.241288 (+  5899us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:18.241324 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:18.241832 (+   508us) log.cc:844] Serialized 26 byte log entry
0709 13:09:20.188493 (+1946661us) spinlock_profiling.cc:243] Waited 17.2 ms on lock 0x7b18002e73e0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cdae07 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf7c02 00007f613597f20b 00007f6135984d23 00007f61359850d1 00007f6135da0a0f
0709 13:09:20.389696 (+201203us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:09:20.389767 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:20.389867 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:20.494646 (+104779us) raft_consensus.cc:1685] finished
0709 13:09:20.494677 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:20.494934 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21081,"raft.queue_time_us":6290,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":5816,"spinlock_wait_cycles":39635328,"thread_start_us":200948,"threads_started":1,"wal-append.queue_time_us":2165421}
W20260709 13:09:20.501250 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:20.502764 25707 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:20.503407 26409 consensus_queue.cc:1048] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:09:20.506862 23026 raft_consensus.cc:1240] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 6 LEADER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 5. Current term is 6. Ops: []
I20260709 13:09:20.509745 23026 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 1756) took 2250 ms. Trace:
I20260709 13:09:20.448488 26531 consensus_queue.cc:1048] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 13:09:20.510125 23026 rpcz_store.cc:276] 0709 13:09:18.258846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:18.258965 (+   119us) service_pool.cc:224] Handling call
0709 13:09:18.259207 (+   242us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:20.507285 (+2248078us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:20.509529 (+  2244us) spinlock_profiling.cc:243] Waited 2.25 s on lock 0x7b540010c554. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:20.509685 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5168476544}
I20260709 13:09:20.513623 26551 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:20.514194 26236 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:20.514297 26551 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:20.515934 22880 raft_consensus.cc:1275] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 13:09:20.516263 22880 pending_rounds.cc:85] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 4
I20260709 13:09:20.516479 22880 pending_rounds.cc:107] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 13:09:20.516732 22880 raft_consensus.cc:2889] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:20.516983 26311 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:20.517339 26551 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:20.520339 26392 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:20.515075 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:20.521060 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:20.307919 26440 consensus_queue.cc:1048] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.143s
I20260709 13:09:20.532225 22953 raft_consensus.cc:1240] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:20.468420 22946 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 12.
I20260709 13:09:20.448010 22918 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:20.573686 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:20.577216 22953 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:20.572805 22876 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11.
I20260709 13:09:20.577687 26392 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:20.581957 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 133978us
I20260709 13:09:20.581497 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:20.583501 23616 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:09:20.430064 26498 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:20.581497 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:20.589937 22877 raft_consensus.cc:1240] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 13:09:20.592523 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:20.476423 26385 raft_consensus.cc:2749] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:20.431713 26488 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:20.594408 22872 raft_consensus.cc:2393] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 11: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:20.572836 26482 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:20.603302 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
W20260709 13:09:20.582451 26481 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.006s	sys 0.010s
I20260709 13:09:20.604272 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:20.614459 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:20.618906 22877 raft_consensus.cc:2468] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
I20260709 13:09:20.612155 23620 raft_consensus.cc:2410] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:09:20.458271 26381 consensus_queue.cc:1048] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.143s
I20260709 13:09:20.574213 26549 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:20.621855 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:20.605036 26453 consensus_queue.cc:1048] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260709 13:09:20.486156 26455 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:20.625087 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:20.625442 26509 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:20.593725 26523 raft_consensus.cc:2804] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:20.659041 22955 raft_consensus.cc:1275] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:09:20.659634 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:20.660266 22952 raft_consensus.cc:2376] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 10. Re-sending same reply.
I20260709 13:09:20.625455 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:20.660429 22953 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
W20260709 13:09:20.627148 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 44941.6us
I20260709 13:09:20.628698 26478 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:20.629483 26454 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:20.617076 22873 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:09:20.630513 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:20.634918 26488 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:20.637732 22874 raft_consensus.cc:1240] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:20.639268 22880 raft_consensus.cc:1240] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:09:20.676944 22872 raft_consensus.cc:1275] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:09:20.639550 26236 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:20.640283 26409 raft_consensus.cc:1064] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
I20260709 13:09:20.641582 26311 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:20.679847 23588 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:20.680015 22994 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:09:20.642355 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:20.642433 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 50145.4us
I20260709 13:09:20.590318 26472 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:20.646358 22950 raft_consensus.cc:1275] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 13:09:20.479809 26357 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:20.698206 22880 raft_consensus.cc:1275] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:09:20.682046 22918 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:20.699163 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:20.653170 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:09:20.616997 26481 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.671s	user 0.006s	sys 0.011s
I20260709 13:09:20.655253 22948 raft_consensus.cc:1275] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:09:20.702055 22994 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:20.703331 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:20.703929 22880 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:20.706284 22876 raft_consensus.cc:2393] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 10: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:20.709103 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:20.597393 26392 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:20.662943 26523 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:20.711138 22918 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:20.719242 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:20.652653 26385 consensus_queue.cc:1059] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
I20260709 13:09:20.670632 22873 raft_consensus.cc:1275] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 13:09:20.643195 26482 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:09:20.726972 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:20.728210 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:20.651271 26407 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:20.709754 26515 raft_consensus.cc:2804] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:20.692929 26455 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:20.694995 26425 consensus_queue.cc:1048] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:09:20.722483 23617 raft_consensus.cc:1240] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 8. Current term is 9. Ops: []
W20260709 13:09:20.702538 26409 raft_consensus.cc:1068] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7: Unable to promote non-voter 5b346767a89745fa82928a5dac9cc0b2: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:09:20.721792 22994 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:20.720531 26523 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:20.679340 26478 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:20.724334 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:20.729149 22873 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 5.
I20260709 13:09:20.678812 26454 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:20.746805 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:20.747022 22875 raft_consensus.cc:1275] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 13:09:20.747390 22873 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 9.
I20260709 13:09:20.702411 26236 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:20.709110 26492 raft_consensus.cc:2804] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:20.738425 26515 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:20.667695 26509 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:20.737370 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:20.743669 22994 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:20.654247 26549 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:20.744195 25707 raft_consensus.cc:2804] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:20.763940 25707 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:20.764385 25707 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:20.735392 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:20.696456 26514 consensus_queue.cc:1048] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:09:20.706074 26472 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:20.678978 22949 raft_consensus.cc:1240] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:20.748474 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:20.771471 22873 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
I20260709 13:09:20.663430 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:20.709604 26544 consensus_queue.cc:1048] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.030s
I20260709 13:09:20.702159 26311 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:20.738736 26407 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:20.759912 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:20.726212 26398 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:20.779976 22880 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 9.
I20260709 13:09:20.781433 22953 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:09:20.783713 25707 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:20.761286 22872 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
I20260709 13:09:20.772747 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:20.785548 22873 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:20.727463 26428 raft_consensus.cc:2749] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:20.756196 26487 consensus_queue.cc:1059] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 13:09:20.762692 26515 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:20.798516 22874 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6.
I20260709 13:09:20.765771 22876 raft_consensus.cc:2376] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10. Re-sending same reply.
I20260709 13:09:20.788345 25707 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:20.753922 26492 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:20.791730 22877 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
I20260709 13:09:20.800024 26492 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:20.777112 26401 raft_consensus.cc:2804] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:09:20.814260 22872 raft_consensus.cc:1240] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:20.825116 22873 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11.
W20260709 13:09:20.829362 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 305229us
I20260709 13:09:20.786783 26498 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:20.864812 22876 raft_consensus.cc:1240] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 4. Current term is 5. Ops: []
W20260709 13:09:20.861207 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 181421us
W20260709 13:09:20.819262 26561 log.cc:927] Time spent T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
I20260709 13:09:20.875082 23588 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:09:20.883248 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:20.895300 22951 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 1823) took 1127 ms. Trace:
I20260709 13:09:20.818325 26481 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:20.899544 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:20.902230 22947 raft_consensus.cc:1275] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:09:20.861109 26401 raft_consensus.cc:697] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:09:20.900849 26564 log.cc:927] Time spent T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 13:09:20.882393 26498 raft_consensus.cc:2804] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:09:20.895748 26347 raft_consensus.cc:2749] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:09:20.916390 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 172802us
I20260709 13:09:20.920394 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:20.926676 22879 raft_consensus.cc:1240] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:20.926235 22994 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:20.902966 26357 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:20.933812 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:20.932018 22994 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:20.901783 26481 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:20.937083 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 182212us
I20260709 13:09:20.959012 22992 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:20.937613 22994 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:20.960223 25707 consensus_queue.cc:1048] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 13:09:20.932135 26534 raft_consensus.cc:2804] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:09:20.962502 22994 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:20.935489 26514 raft_consensus.cc:2749] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:20.932760 26498 raft_consensus.cc:697] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 10 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
W20260709 13:09:20.957119 26548 log.cc:927] Time spent T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.779s	user 0.005s	sys 0.000s
I20260709 13:09:20.938246 26508 raft_consensus.cc:2749] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:20.962680 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:20.938848 26482 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:20.971621 26454 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:20.965162 26563 log.cc:927] Time spent T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.219s	user 0.001s	sys 0.000s
I20260709 13:09:20.960038 26477 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:20.995683 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:20.979715 22879 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 5.
I20260709 13:09:21.003322 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:20.997946 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:20.988440 26534 raft_consensus.cc:697] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:20.999684 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:21.001116 22955 raft_consensus.cc:1240] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:20.966583 26437 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:21.005069 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:21.020484 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:20.965378 26478 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:09:21.034035 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:20.936429 26568 consensus_queue.cc:1059] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
W20260709 13:09:21.032464 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:21.033057 22920 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:20.998799 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:21.038273 22946 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 6.
I20260709 13:09:20.972729 26392 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:20.970499 26428 raft_consensus.cc:2749] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:09:21.040809 22949 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 10.
I20260709 13:09:20.978317 26509 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:21.058439 26407 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:21.033993 26477 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:21.032251 26549 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:21.060599 22920 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:21.040115 26236 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:20.961480 26401 consensus_queue.cc:237] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:21.041435 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 166405us
W20260709 13:09:21.105789 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 45307.8us
I20260709 13:09:21.106570 22920 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:21.107638 22920 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:21.040608 26514 consensus_queue.cc:1048] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:09:21.109035 22920 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:21.039824 26579 consensus_queue.cc:1048] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:09:21.105502 26509 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:21.125270 26509 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:21.113821 26578 raft_consensus.cc:2804] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:09:21.072418 26407 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:21.116052 26477 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:21.119151 26539 raft_consensus.cc:2749] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:09:21.119688 26236 raft_consensus.cc:2804] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:21.121145 26575 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:09:21.061507 26311 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:21.063505 26531 raft_consensus.cc:2749] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:21.135453 22920 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:21.148061 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:21.155520 23621 raft_consensus.cc:1240] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 7. Current term is 8. Ops: []
W20260709 13:09:21.188037 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 52685.3us
I20260709 13:09:21.159574 26582 consensus_queue.cc:1048] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:09:21.178675 26495 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:21.191630 26495 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:21.189165 26454 raft_consensus.cc:2749] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:21.194247 26495 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:21.144460 26455 raft_consensus.cc:3055] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:09:21.194970 26455 raft_consensus.cc:740] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:21.179433 26488 raft_consensus.cc:3055] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:09:21.195973 26488 raft_consensus.cc:740] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:21.136492 26407 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:21.197055 26488 consensus_queue.cc:260] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:21.198655 26488 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:21.201547 22947 raft_consensus.cc:1240] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:09:21.192091 26509 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:21.156323 26472 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:21.138027 26236 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:21.203274 26236 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:21.198859 26407 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:21.207656 26572 log.cc:927] Time spent T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
W20260709 13:09:21.209378 26570 log.cc:927] Time spent T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 13:09:21.210244 26576 raft_consensus.cc:2804] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:21.210872 26576 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:21.211393 26576 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:21.214278 22920 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:21.217129 26562 log.cc:927] Time spent T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.483s	user 0.003s	sys 0.002s
W20260709 13:09:21.219296 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 253402us
W20260709 13:09:21.221453 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 172855us
I20260709 13:09:21.220258 22920 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:21.222373 22844 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:21.137015 26568 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:21.136766 26578 raft_consensus.cc:697] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:21.217927 26385 consensus_queue.cc:1059] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 13:09:21.230585 22920 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:21.228255 22843 leader_election.cc:336] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.526s (SENT)
I20260709 13:09:21.231873 22843 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:21.239467 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:21.245627 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:21.246119 23617 raft_consensus.cc:1240] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 7. Current term is 8. Ops: []
I20260709 13:09:21.246241 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:21.238211 22920 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:21.251528 22994 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:21.221590 26551 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:21.255508 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:21.262012 23026 raft_consensus.cc:2410] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:09:21.256230 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:21.256942 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:21.257201 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:21.127063 26498 consensus_queue.cc:237] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:21.298576 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:21.261619 26531 raft_consensus.cc:2749] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:09:21.258037 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:09:21.260087 22843 leader_election.cc:336] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.545s (SENT)
I20260709 13:09:21.260926 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:21.264222 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:21.257844 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:21.300870 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:21.301347 23022 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 5.
I20260709 13:09:21.301592 23029 raft_consensus.cc:2393] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 10: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:21.301864 22875 raft_consensus.cc:1275] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:09:21.302073 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:21.129021 26534 consensus_queue.cc:237] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:21.302366 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:21.302949 23027 raft_consensus.cc:2410] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:09:21.303150 23029 raft_consensus.cc:2368] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 7. Current term is 8.
I20260709 13:09:21.303740 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:21.303911 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:21.305145 26481 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:21.305223 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:21.265307 26392 consensus_queue.cc:1048] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:09:21.305949 23029 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:21.258462 26428 raft_consensus.cc:2749] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:21.307840 23024 raft_consensus.cc:2410] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:09:21.229038 26549 raft_consensus.cc:2749] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:09:21.309144 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:21.265744 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:21.310179 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:21.310510 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:21.311355 23028 raft_consensus.cc:2410] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:09:21.268033 26551 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:21.312089 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:21.227131 26575 raft_consensus.cc:2749] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:09:21.268486 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:21.313230 26551 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:21.313372 23026 raft_consensus.cc:2376] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 9. Re-sending same reply.
I20260709 13:09:21.258570 26482 consensus_queue.cc:1059] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 13:09:21.314265 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:20.895732 22951 rpcz_store.cc:276] 0709 13:09:19.768297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:19.788921 (+ 20624us) service_pool.cc:224] Handling call
0709 13:09:19.801001 (+ 12080us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:19.801227 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:09:19.832313 (+ 31086us) log.cc:844] Serialized 14 byte log entry
0709 13:09:20.876189 (+1043876us) spinlock_profiling.cc:243] Waited 21.7 ms on lock 0x7b18003f1660. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f613598d4bc 00007f6132cdb48b 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf7c02 00007f613597f20b 00007f6135984d23 00007f613598523a
0709 13:09:20.894974 (+ 18785us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:20.895021 (+    47us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:09:20.895069 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:20.895132 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:20.895233 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4774,"prepare.queue_time_us":33034,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":35,"raft.queue_time_us":384,"spinlock_wait_cycles":49903872,"thread_start_us":34768,"threads_started":1,"wal-append.queue_time_us":68822}
I20260709 13:09:21.314839 23026 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 9.
I20260709 13:09:21.315734 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:21.316450 23026 raft_consensus.cc:2410] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:09:21.223824 26582 consensus_queue.cc:1059] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:09:21.269104 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:21.299940 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:21.317093 26582 raft_consensus.cc:3055] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:09:21.317337 23020 raft_consensus.cc:2368] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 8. Current term is 10.
I20260709 13:09:21.317560 26582 raft_consensus.cc:740] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:21.318161 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:21.249985 26539 consensus_queue.cc:1048] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 13:09:21.270110 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:21.270928 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:21.271631 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:21.268590 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:21.304366 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:21.277123 26578 consensus_queue.cc:237] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:21.286113 22993 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:21.286386 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:21.311221 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:21.283280 26544 raft_consensus.cc:2804] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:09:21.288744 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:21.329288 23021 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 7.
I20260709 13:09:21.289116 26477 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:21.335938 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 49947.4us
I20260709 13:09:21.227921 26289 consensus_queue.cc:1048] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 13:09:21.232851 26568 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:21.295934 26455 consensus_queue.cc:260] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:21.233685 26472 raft_consensus.cc:2749] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:21.233651 26347 raft_consensus.cc:2749] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:09:21.300807 22843 leader_election.cc:336] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.548s (SENT)
I20260709 13:09:21.253193 26311 consensus_queue.cc:1059] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
W20260709 13:09:21.286909 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:21.304361 23027 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 10.
I20260709 13:09:21.327824 22876 raft_consensus.cc:2393] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 9: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:21.327768 23023 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 9.
I20260709 13:09:21.317451 22874 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 11.
I20260709 13:09:21.350569 23590 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:21.351416 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:21.352327 22874 raft_consensus.cc:2393] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 9: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:21.353277 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:21.353421 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 52650.3us
I20260709 13:09:21.353878 22874 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 5.
W20260709 13:09:21.354810 22843 leader_election.cc:336] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.610s (SENT)
I20260709 13:09:21.354919 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:21.355348 22843 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:21.356465 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:21.358003 22875 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
I20260709 13:09:21.302129 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:21.322234 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:21.359440 23028 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 7.
I20260709 13:09:21.333776 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:21.360982 23026 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 8.
I20260709 13:09:21.339969 26544 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:21.224085 26409 consensus_queue.cc:1059] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:09:21.379856 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:21.380751 22875 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
W20260709 13:09:21.383883 22843 leader_election.cc:336] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.667s (SENT)
I20260709 13:09:21.384465 22843 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:21.386111 22843 leader_election.cc:336] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.682s (SENT)
I20260709 13:09:21.363795 26549 raft_consensus.cc:3055] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:09:21.373870 26449 raft_consensus.cc:2804] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:09:21.334215 26385 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:21.249574 26389 consensus_queue.cc:1059] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 13:09:21.302428 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:21.417224 23025 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:21.344599 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:21.349393 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 390445us
I20260709 13:09:21.372635 26544 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:21.360351 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:21.420214 23028 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 9.
I20260709 13:09:21.421165 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:21.355687 22874 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:09:21.341009 23029 raft_consensus.cc:2410] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:09:21.341414 26425 raft_consensus.cc:2804] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:21.364321 23590 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:21.365633 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:21.362531 22877 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 9.
I20260709 13:09:21.369655 23023 raft_consensus.cc:1240] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:21.423059 22876 raft_consensus.cc:2410] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 9 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:09:21.367377 23022 raft_consensus.cc:1275] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:09:21.370142 23026 raft_consensus.cc:1240] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 7. Current term is 8. Ops: []
I20260709 13:09:21.342957 26477 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:21.342967 23020 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:09:21.428929 23020 raft_consensus.cc:606] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:21.375692 26472 raft_consensus.cc:2749] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:09:21.314564 26482 raft_consensus.cc:3055] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:09:21.376267 26347 raft_consensus.cc:3055] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:09:21.419627 23021 raft_consensus.cc:2410] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 13:09:21.376339 26455 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:21.425792 26508 consensus_queue.cc:1048] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:09:21.422706 26385 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:21.427016 23023 raft_consensus.cc:1240] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:09:21.427313 22874 raft_consensus.cc:1240] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:21.429160 22843 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:21.414098 26492 consensus_meta.cc:287] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.613s	user 0.013s	sys 0.002s
I20260709 13:09:21.374367 26502 consensus_queue.cc:1059] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 13:09:21.437340 23026 raft_consensus.cc:1240] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 8. Current term is 9. Ops: []
W20260709 13:09:21.439097 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 74839us
I20260709 13:09:21.428496 26425 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:21.440533 26584 log.cc:927] Time spent T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 13:09:21.440630 23025 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 10.
I20260709 13:09:21.318447 26582 consensus_queue.cc:260] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:21.392040 26589 raft_consensus.cc:2749] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:21.409899 26549 raft_consensus.cc:740] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:21.444522 26482 raft_consensus.cc:740] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:21.457152 22843 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:21.461826 22954 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.525s	user 0.011s	sys 0.042s
W20260709 13:09:21.463467 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 225339us
I20260709 13:09:21.463498 23590 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:21.445358 26403 raft_consensus.cc:2749] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:21.443549 26449 raft_consensus.cc:697] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 9 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:21.462059 22872 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11.
I20260709 13:09:21.462148 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:21.456676 22878 consensus_queue.cc:260] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:21.469524 22878 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:21.465869 23021 raft_consensus.cc:1240] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:21.445128 26347 raft_consensus.cc:740] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:21.467307 22843 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:21.466135 22954 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.011s	sys 0.042s
I20260709 13:09:21.466845 22920 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:21.473179 22920 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:21.474694 22920 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:21.443171 26428 consensus_queue.cc:1048] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:09:21.458981 26488 raft_consensus.cc:2749] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:21.460801 26425 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:21.476157 22920 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:21.479578 22843 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:21.514349 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:21.514864 23590 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:21.517877 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:21.519632 23623 raft_consensus.cc:1275] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
I20260709 13:09:21.519980 23623 pending_rounds.cc:85] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2: Aborting all ops after (but not including) 9
I20260709 13:09:21.520205 23623 pending_rounds.cc:107] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2: Aborting uncommitted NO_OP operation due to leader change: 10.10
I20260709 13:09:21.520138 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:21.520552 23623 raft_consensus.cc:2889] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:21.518558 23620 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:09:21.521461 22874 raft_consensus.cc:1275] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 13:09:21.521556 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:21.522487 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:21.515303 22843 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:21.522893 22919 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:21.500017 22994 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:21.516122 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:21.524533 23618 raft_consensus.cc:1275] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 13:09:21.525738 23021 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:21.524058 22843 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:21.458173 26492 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:21.523130 22874 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 11.
I20260709 13:09:21.528580 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:21.521749 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:21.525950 23618 pending_rounds.cc:85] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2: Aborting all ops after (but not including) 8
I20260709 13:09:21.531234 23618 pending_rounds.cc:107] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:09:21.531469 23618 raft_consensus.cc:2889] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:21.527910 22994 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:21.532944 22994 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:21.541364 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:21.542148 22950 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:09:21.544493 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:21.548570 23588 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:21.480541 26409 raft_consensus.cc:2804] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:09:21.550515 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 74439.4us
I20260709 13:09:21.477958 26458 consensus_queue.cc:1059] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.081s
I20260709 13:09:21.486294 26398 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:21.483338 26539 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:21.555078 26514 raft_consensus.cc:2804] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:09:21.567056 26539 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:21.567169 26514 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:21.567574 26539 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:21.560106 26454 raft_consensus.cc:2804] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:09:21.559263 23619 raft_consensus.cc:1275] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:09:21.562470 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:21.565356 26398 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:21.573572 26398 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:21.573845 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:21.575038 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:21.575855 26545 raft_consensus.cc:1064] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
I20260709 13:09:21.576058 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:21.562470 22920 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:21.528601 26403 raft_consensus.cc:2749] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:09:21.473332 26311 raft_consensus.cc:2749] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:09:21.561066 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:21.528704 26575 raft_consensus.cc:2804] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:21.560483 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:21.527145 26481 raft_consensus.cc:2749] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:21.551985 22878 raft_consensus.cc:1275] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:09:21.553977 26437 consensus_meta.cc:287] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.506s	user 0.007s	sys 0.003s
I20260709 13:09:21.567629 26514 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:21.513365 26459 raft_consensus.cc:2804] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 13:09:21.576398 26545 raft_consensus.cc:1068] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7: Unable to promote non-voter 5b346767a89745fa82928a5dac9cc0b2: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:09:21.486780 26472 raft_consensus.cc:2749] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:09:21.578948 23025 raft_consensus.cc:1240] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:21.524871 26589 raft_consensus.cc:2749] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:09:21.587540 23616 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 9.
I20260709 13:09:21.564631 22874 raft_consensus.cc:1240] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 13:09:21.563652 26409 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:21.617143 22878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2337) took 1164 ms. Trace:
I20260709 13:09:21.565412 26495 raft_consensus.cc:2749] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:21.525897 26590 consensus_queue.cc:1048] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.104s
I20260709 13:09:21.605041 26575 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:21.616777 26508 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:21.621905 26437 raft_consensus.cc:2749] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 13:09:21.604140 26454 raft_consensus.cc:697] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:21.617803 26496 raft_consensus.cc:2749] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:09:21.629910 26462 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 13:09:21.568439 26576 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:21.651217 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 127238us
I20260709 13:09:21.638518 22878 rpcz_store.cc:276] 0709 13:09:20.453160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:20.453310 (+   150us) service_pool.cc:224] Handling call
0709 13:09:20.453551 (+   241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:21.456241 (+1002690us) spinlock_profiling.cc:243] Waited 70.1 ms on lock 0x7b4c00377400. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f61340b1a20 00007f61340c2a29 00007f6134088644 00007f61340aff2d 00007f61340afb93 00007f6135de018b 00007f6135de0cb5 00007f6135ddbf30 00007f6135ddd7de 00007f6135de7b49 00007f6135de8416
0709 13:09:21.616860 (+160619us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:21.617087 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":161272576}
I20260709 13:09:21.628533 26311 raft_consensus.cc:2804] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:09:21.669453 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 107062us
W20260709 13:09:21.654137 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 76906.6us
W20260709 13:09:21.658886 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 110411us
I20260709 13:09:21.638326 26574 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:21.632519 26459 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:21.670432 22843 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:21.676462 22843 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:21.640338 26409 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:21.652809 26575 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:21.671747 23020 raft_consensus.cc:2955] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Committing config change with OpId 6.5: config changed from index -1 to 5, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:09:21.677817 22920 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:21.679574 22843 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:21.578521 26549 consensus_queue.cc:260] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:21.688270 22843 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:21.672784 26311 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:21.586694 26482 consensus_queue.cc:260] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:21.679652 26523 consensus_meta.cc:287] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.928s	user 0.006s	sys 0.006s
I20260709 13:09:21.638602 26545 consensus_queue.cc:1048] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:09:21.690877 22843 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:21.639387 26592 consensus_queue.cc:1059] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 13:09:21.586848 26347 consensus_queue.cc:260] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:21.678534 26459 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:21.669013 26498 consensus_queue.cc:1059] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 13:09:21.694504 26472 raft_consensus.cc:2749] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:09:21.681565 26589 raft_consensus.cc:2749] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:21.685864 26481 raft_consensus.cc:2749] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:21.633978 26449 consensus_queue.cc:237] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:21.694170 26311 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:21.694859 26477 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:21.694438 26608 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:09:21.716360 26347 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:09:21.729018 26236 consensus_meta.cc:287] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.525s	user 0.010s	sys 0.005s
I20260709 13:09:21.729836 26481 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:21.666908 26508 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:21.701112 26385 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:21.705282 26482 raft_consensus.cc:3060] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:21.706070 26549 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:21.733228 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:21.720363 26576 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:21.741526 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 181326us
I20260709 13:09:21.733660 26477 raft_consensus.cc:2804] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:09:21.764902 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 63965us
I20260709 13:09:21.732936 26481 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:21.702108 26523 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:21.773130 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:21.679867 26574 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:21.735708 26608 raft_consensus.cc:697] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:21.743520 26492 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:21.735716 26236 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:21.787073 22843 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:21.775498 22919 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:09:21.790354 26597 log.cc:927] Time spent T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 13:09:21.813263 26576 raft_consensus.cc:2804] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:21.813787 26576 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:21.814134 26576 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:21.822001 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:21.822242 26608 consensus_queue.cc:237] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:21.823201 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:21.823112 26459 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:21.824396 23622 raft_consensus.cc:2393] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 6: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:21.828058 22875 raft_consensus.cc:3055] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:09:21.828389 22875 raft_consensus.cc:740] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:21.824404 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:21.830572 23026 raft_consensus.cc:1240] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 13:09:21.831401 26582 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:21.832775 23026 raft_consensus.cc:1240] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
W20260709 13:09:21.834174 26580 log.cc:927] Time spent T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.712s	user 0.000s	sys 0.002s
I20260709 13:09:21.804469 23619 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a"
dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
mode: GRACEFUL
new_leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:50814
I20260709 13:09:21.776638 26481 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:09:21.806681 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 46471.5us
I20260709 13:09:21.740862 26472 consensus_queue.cc:1048] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 13:09:21.809036 26611 consensus_queue.cc:1048] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.111s
I20260709 13:09:21.836563 23619 raft_consensus.cc:606] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Received request to transfer leadership to TS aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:21.859301 23619 raft_consensus.cc:614] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:21.843357 23021 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 12.
I20260709 13:09:21.773857 26477 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:21.844769 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 57687.7us
I20260709 13:09:21.862249 22843 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:21.850911 22946 raft_consensus.cc:1240] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:09:21.861022 22880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 1835) took 1166 ms. Trace:
I20260709 13:09:21.863152 22880 rpcz_store.cc:276] 0709 13:09:20.694109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:20.780637 (+ 86528us) service_pool.cc:224] Handling call
0709 13:09:20.781026 (+   389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:20.783142 (+  2116us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:20.783185 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:20.783929 (+   744us) log.cc:844] Serialized 26 byte log entry
0709 13:09:20.968215 (+184286us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:09:20.968268 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:20.968356 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:21.833419 (+865063us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:09:21.835940 (+  2521us) raft_consensus.cc:1685] finished
0709 13:09:21.835968 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:21.860946 (+ 24978us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1762,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":9031,"spinlock_wait_cycles":26333056,"thread_start_us":103906,"threads_started":1,"wal-append.queue_time_us":303467}
I20260709 13:09:21.867686 26574 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:21.743811 26544 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:21.886395 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:21.887481 22948 raft_consensus.cc:1275] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:09:21.889235 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 211511us
I20260709 13:09:21.894399 26506 raft_consensus.cc:3055] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:09:21.894928 26506 raft_consensus.cc:740] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:21.895376 26581 raft_consensus.cc:3055] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:09:21.895788 26581 raft_consensus.cc:740] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:21.896494 26534 raft_consensus.cc:3055] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:09:21.896955 26534 raft_consensus.cc:740] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:21.899132 26605 consensus_queue.cc:1059] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.109s
I20260709 13:09:21.900005 26605 consensus_queue.cc:1243] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer f3c799b1249e40119d2a5864231c6c75 log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:09:21.901377 23025 raft_consensus.cc:1275] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 13:09:21.903609 26569 log.cc:927] Time spent T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.971s	user 0.000s	sys 0.003s
W20260709 13:09:21.895845 26613 log.cc:927] Time spent T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
I20260709 13:09:21.910333 23021 raft_consensus.cc:1275] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:09:21.742100 26454 consensus_queue.cc:237] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:21.858459 26589 raft_consensus.cc:2749] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:09:21.869557 26477 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:21.863718 26509 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:21.915338 22843 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:21.916007 22879 raft_consensus.cc:1275] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:09:21.921177 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:21.930222 22878 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 11.
I20260709 13:09:21.940622 22954 raft_consensus.cc:1240] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 5. Current term is 6. Ops: []
I20260709 13:09:21.942960 22843 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:21.947989 22874 raft_consensus.cc:1240] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:21.948983 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:21.886957 26574 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:21.949611 22873 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 9.
I20260709 13:09:21.953367 22843 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:21.914445 26462 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:21.950663 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:09:21.967893 26571 log.cc:927] Time spent T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.854s	user 0.004s	sys 0.000s
W20260709 13:09:21.969187 26599 log.cc:927] Time spent T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.256s	user 0.001s	sys 0.000s
I20260709 13:09:21.969570 26212 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:21.970899 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:21.971503 23028 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 9.
I20260709 13:09:21.973826 26236 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:21.975121 26568 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:21.933466 26605 raft_consensus.cc:2749] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:21.978315 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:21.980425 23590 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:21.981871 22948 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 11.
I20260709 13:09:21.982796 23590 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:21.983485 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:21.984248 23588 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:21.981976 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:21.988626 23622 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 9.
I20260709 13:09:21.872485 26614 consensus_queue.cc:1059] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.118s
I20260709 13:09:22.001438 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:22.009011 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:22.020066 23619 raft_consensus.cc:1240] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 7. Current term is 8. Ops: []
I20260709 13:09:22.020206 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:22.024904 22920 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:22.027396 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:22.028263 22871 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:21.954929 26458 consensus_queue.cc:1048] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.041s
I20260709 13:09:21.942198 26449 consensus_queue.cc:1059] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.051s
I20260709 13:09:21.969581 26602 raft_consensus.cc:2749] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:21.958268 26453 consensus_queue.cc:1048] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 13:09:22.003894 26405 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:21.969202 26617 raft_consensus.cc:2804] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:09:21.955245 26509 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:22.044023 22950 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
W20260709 13:09:22.016301 26515 consensus_meta.cc:287] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 1.186s	user 0.008s	sys 0.005s
I20260709 13:09:22.046037 26614 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:22.045727 22920 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:22.001670 26349 raft_consensus.cc:2804] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:21.895915 26506 consensus_queue.cc:260] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:21.976263 26462 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:22.039340 26593 log.cc:927] Time spent T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:09:22.057178 22871 raft_consensus.cc:2468] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:09:21.963433 26574 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:22.041263 26440 raft_consensus.cc:2804] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:22.063793 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:22.064998 22920 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:22.050289 26455 raft_consensus.cc:2804] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:09:22.066241 23619 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a424231ef7b4470bbd575564909a365e"
dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:50814
I20260709 13:09:22.066931 23619 raft_consensus.cc:606] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 8 LEADER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:22.067000 26381 raft_consensus.cc:3055] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:09:22.067440 26381 raft_consensus.cc:740] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:22.060635 26515 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:22.070628 26515 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:22.047919 26405 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:22.071834 26405 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:22.074007 26594 raft_consensus.cc:3055] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:09:22.074394 26594 raft_consensus.cc:740] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:22.074549 26462 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:22.075124 26594 consensus_queue.cc:260] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:22.076296 26594 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:22.077016 26574 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:22.079053 26311 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:22.082221 22953 raft_consensus.cc:1240] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:22.082492 22878 raft_consensus.cc:1240] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:22.066293 26455 raft_consensus.cc:697] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:22.083192 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:22.085676 22871 raft_consensus.cc:1275] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:09:22.086352 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:22.092437 23021 raft_consensus.cc:1275] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:09:22.094201 23025 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:22.080827 26619 consensus_queue.cc:1048] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:09:22.058713 26614 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:22.109158 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:22.060824 26349 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:22.117396 22871 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:22.109112 26289 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:22.119690 26289 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:22.120957 26455 consensus_queue.cc:237] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:22.122229 26289 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:22.122888 26616 log.cc:927] Time spent T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.195s	user 0.001s	sys 0.000s
I20260709 13:09:22.126724 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:22.130615 23025 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
I20260709 13:09:22.063167 26617 raft_consensus.cc:697] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 10 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:22.136775 22871 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6.
I20260709 13:09:22.093135 26592 raft_consensus.cc:2804] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:22.137679 26311 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:22.141477 23025 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:09:22.143999 23025 raft_consensus.cc:606] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Received request to transfer leadership to TS aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:22.144503 23025 raft_consensus.cc:614] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:22.051225 26622 consensus_queue.cc:1059] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 13:09:22.104871 26589 raft_consensus.cc:2749] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:22.101516 26385 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:09:22.149716 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 196454us
I20260709 13:09:22.129953 26614 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:22.117758 26506 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:22.125025 26462 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:22.151013 22843 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:22.130398 26349 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:22.135429 26623 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:22.156924 22843 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:22.143126 26592 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:22.101034 26409 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:22.105567 26236 consensus_queue.cc:1048] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.045s
I20260709 13:09:22.159904 22843 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:22.160367 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:22.162819 22920 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:22.163023 22871 raft_consensus.cc:2393] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 10: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
W20260709 13:09:22.163616 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 179399us
I20260709 13:09:22.165159 22843 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:22.165526 22844 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:22.171066 23025 raft_consensus.cc:1240] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 9. Current term is 10. Ops: [9.7-9.7]
W20260709 13:09:22.155220 26514 consensus_meta.cc:287] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.520s	user 0.009s	sys 0.001s
W20260709 13:09:22.159206 26573 log.cc:927] Time spent T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 1.122s	user 0.007s	sys 0.003s
I20260709 13:09:22.160151 26462 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:22.162344 26385 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:22.168606 26482 raft_consensus.cc:2804] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:22.172327 26212 raft_consensus.cc:2749] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:09:22.176489 26623 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:22.158751 26311 raft_consensus.cc:2804] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:09:22.177384 26568 raft_consensus.cc:2804] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:22.177635 26605 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:22.187814 26477 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:22.172919 26347 raft_consensus.cc:2804] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:09:22.159286 26592 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:22.173884 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:22.175266 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:09:22.178627 26425 consensus_meta.cc:287] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.628s	user 0.010s	sys 0.000s
I20260709 13:09:22.206216 26385 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:22.272965 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:22.203904 26462 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:22.274371 26349 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:22.211338 26568 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:22.277127 26568 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:09:22.279131 26549 consensus_meta.cc:287] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.529s	user 0.003s	sys 0.004s
I20260709 13:09:22.208112 26405 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:22.201190 26617 consensus_queue.cc:237] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:22.188517 26576 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:22.282810 26405 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:22.178966 26618 raft_consensus.cc:2804] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:09:22.179495 26398 consensus_meta.cc:287] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.605s	user 0.011s	sys 0.000s
I20260709 13:09:22.179698 26544 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:22.181142 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:22.185235 23027 raft_consensus.cc:1240] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:22.286772 23027 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 1775) took 1025 ms. Trace:
I20260709 13:09:22.287114 23027 rpcz_store.cc:276] 0709 13:09:21.261162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:21.361231 (+100069us) service_pool.cc:224] Handling call
0709 13:09:21.361576 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:22.185089 (+823513us) spinlock_profiling.cc:243] Waited 3.42 ms on lock 0x7b540003d520. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135d5cdc0 00007f6135de71c0 00007f6135de8231 00007f6135de4d8f 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c
0709 13:09:22.286512 (+101423us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:22.286714 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7861120}
I20260709 13:09:22.287734 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:22.152195 26523 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:22.172350 26472 raft_consensus.cc:2749] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:22.209781 26482 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:22.208896 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:22.214159 22879 raft_consensus.cc:1240] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:09:22.214578 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:22.218591 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:22.200515 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:09:22.285260 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:22.291646 22843 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:22.292420 22871 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:09:22.293063 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 309232us
I20260709 13:09:22.293128 22843 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:22.189927 26409 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:22.284549 26575 consensus_meta.cc:287] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.587s	user 0.007s	sys 0.002s
I20260709 13:09:22.290944 22948 raft_consensus.cc:2393] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 10: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:22.295585 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:22.297022 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:22.301469 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:22.267226 26514 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:22.302838 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:22.303447 22877 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:22.303668 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:22.304224 23025 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:22.304715 26462 raft_consensus.cc:2749] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:09:22.304973 23029 raft_consensus.cc:2393] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 6: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:22.290340 26482 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:22.294996 26398 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:22.311094 26398 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:22.294804 26425 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:22.311749 26575 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:22.178961 26620 log.cc:927] Time spent T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 13:09:22.313879 22873 raft_consensus.cc:1275] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:09:22.322147 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:22.322665 22873 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:22.313267 26425 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:22.309039 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:22.297811 22950 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 5.
I20260709 13:09:22.317839 23025 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
I20260709 13:09:22.318045 23022 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f603724630e44ef78014e7719f096cb3"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:09:22.214570 26477 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:22.318584 26462 consensus_queue.cc:1059] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:09:22.319123 26575 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:22.270689 26481 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:09:22.320240 22952 raft_consensus.cc:1240] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:22.368201 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:22.320611 26472 raft_consensus.cc:2804] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:22.314419 22949 raft_consensus.cc:1240] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
W20260709 13:09:22.192631 26539 consensus_meta.cc:287] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.620s	user 0.008s	sys 0.000s
I20260709 13:09:22.371461 26539 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:22.204614 26604 consensus_queue.cc:1059] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.130s
I20260709 13:09:22.373059 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:22.327935 23022 raft_consensus.cc:606] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 6 LEADER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:22.374393 22946 raft_consensus.cc:1240] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:22.304489 26514 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:22.331902 22871 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
I20260709 13:09:22.333083 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:22.379374 22950 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 9.
I20260709 13:09:22.333810 22877 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6.
I20260709 13:09:22.334954 22873 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 12.
I20260709 13:09:22.366802 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:22.370422 26462 raft_consensus.cc:3055] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:09:22.370812 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:22.325685 26576 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:22.328480 26509 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:22.210788 26311 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:22.211997 26623 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:22.392247 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:22.386382 22950 raft_consensus.cc:1275] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:09:22.365409 26477 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:22.211817 26605 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:22.399075 23025 raft_consensus.cc:1217] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Deduplicated request from leader. Original: 4.6->[6.7-7.8]   Dedup: 6.7->[7.8-7.8]
I20260709 13:09:22.401243 23025 raft_consensus.cc:1240] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 7. Current term is 8. Ops: [6.7-7.8]
I20260709 13:09:22.374105 26472 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:22.386672 22874 raft_consensus.cc:1240] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 11. Current term is 12. Ops: [8.8-11.9]
I20260709 13:09:22.384007 26462 raft_consensus.cc:740] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:22.395792 26311 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:22.412248 26635 raft_consensus.cc:993] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2: : Instructing follower 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:09:22.413085 26403 raft_consensus.cc:3055] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:09:22.400720 26401 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.285s	user 0.016s	sys 0.016s
I20260709 13:09:22.392405 26539 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:22.417521 26403 raft_consensus.cc:740] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:22.419432 26582 raft_consensus.cc:1081] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 8 LEADER]: Signalling peer 0a407504ed1741bf89a276558adf4c0f to start an election
W20260709 13:09:22.417759 26401 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.302s	user 0.016s	sys 0.016s
I20260709 13:09:22.383006 26604 consensus_queue.cc:1059] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:09:22.400247 26349 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:22.408869 26477 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:22.404170 26472 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:22.413182 26502 consensus_queue.cc:1048] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 13:09:22.208963 26534 consensus_queue.cc:260] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:22.444592 26568 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:22.591645 22876 raft_consensus.cc:2393] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 10: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:22.455262 26592 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:22.592586 22992 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:22.456116 26403 consensus_queue.cc:260] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:22.594084 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:22.589154 22949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2466) took 1048 ms. Trace:
I20260709 13:09:22.537312 22953 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 1830) took 1543 ms. Trace:
I20260709 13:09:22.603998 22953 rpcz_store.cc:276] 0709 13:09:20.994272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:20.997061 (+  2789us) service_pool.cc:224] Handling call
0709 13:09:20.997435 (+   374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:22.082598 (+1085163us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:22.536978 (+454380us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b540020ec54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:22.537231 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2494276480}
W20260709 13:09:22.552024 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 264617us
W20260709 13:09:22.600207 23020 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.091s	sys 0.156s
W20260709 13:09:22.609277 23020 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.091s	sys 0.156s
W20260709 13:09:22.600313 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 185880us
I20260709 13:09:22.610332 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:22.611496 23020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 1848) took 1186 ms. Trace:
I20260709 13:09:22.611866 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:22.613072 22843 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:22.614432 22843 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:22.542470 23029 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32794 (request call id 594) took 1056 ms. Trace:
I20260709 13:09:22.611842 23020 rpcz_store.cc:276] 0709 13:09:21.425370 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:21.446487 (+ 21117us) service_pool.cc:224] Handling call
0709 13:09:21.446975 (+   488us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:21.447694 (+   719us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:21.447735 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:21.562003 (+114268us) log.cc:844] Serialized 518 byte log entry
0709 13:09:21.671499 (+109496us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:09:22.026468 (+354969us) log.cc:844] Serialized 14 byte log entry
0709 13:09:22.610906 (+584438us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:22.611281 (+   375us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:22.611320 (+    39us) raft_consensus.cc:1685] finished
0709 13:09:22.611330 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:22.611447 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23094,"raft.queue_time_us":25226,"raft.run_cpu_time_us":240,"raft.run_wall_time_us":3393,"spinlock_wait_cycles":51584,"thread_start_us":634710,"threads_started":2,"wal-append.queue_time_us":129253}
I20260709 13:09:22.615689 23029 rpcz_store.cc:276] 0709 13:09:21.485933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:21.732912 (+246979us) service_pool.cc:224] Handling call
0709 13:09:22.542208 (+809296us) spinlock_profiling.cc:243] Waited 558 ms on lock 0x7b5400135454. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:22.542390 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1284301056}
W20260709 13:09:22.554865 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 336003us
W20260709 13:09:22.503197 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:22.618350 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:22.623222 22880 raft_consensus.cc:1240] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:09:22.625146 22873 raft_consensus.cc:1275] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:09:22.518710 22993 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:22.618660 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:22.632334 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 113690us
I20260709 13:09:22.602072 22949 rpcz_store.cc:276] 0709 13:09:21.540503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:21.540693 (+   190us) service_pool.cc:224] Handling call
0709 13:09:21.541109 (+   416us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:22.369398 (+828289us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:22.588939 (+219541us) spinlock_profiling.cc:243] Waited 773 ms on lock 0x7b54000753d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:22.589100 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1777831296}
I20260709 13:09:22.618854 22920 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:09:22.640322 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:22.642231 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:22.657493 23590 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:22.658373 23590 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:22.661923 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:22.662474 22880 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:09:22.663496 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 44714.6us
I20260709 13:09:22.664430 22872 raft_consensus.cc:1240] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 10 LEADER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:22.658396 22878 raft_consensus.cc:1240] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:09:22.530373 22875 consensus_queue.cc:260] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:22.476228 26637 log.cc:927] Time spent T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.315s	user 0.002s	sys 0.000s
W20260709 13:09:22.587875 26634 log.cc:927] Time spent T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.427s	user 0.003s	sys 0.000s
I20260709 13:09:22.681234 22871 raft_consensus.cc:1240] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 7. Current term is 8. Ops: []
I20260709 13:09:22.505906 26577 raft_consensus.cc:3055] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:09:22.515542 26487 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:22.706352 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:22.695297 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:09:22.678877 26594 consensus_meta.cc:287] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.584s	user 0.013s	sys 0.000s
I20260709 13:09:22.520744 26459 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:22.516399 26605 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:22.654311 26492 raft_consensus.cc:2749] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:22.588560 26602 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:22.692126 22918 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:22.723160 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 68786.9us
I20260709 13:09:22.558902 26508 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:22.465145 26481 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:22.552412 26618 raft_consensus.cc:697] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:22.554750 26347 raft_consensus.cc:697] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:22.477808 26440 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:22.552192 26523 consensus_queue.cc:1059] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.178s
W20260709 13:09:22.724273 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 89768us
I20260709 13:09:22.583947 26477 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:22.731171 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 55936.3us
I20260709 13:09:22.769008 22992 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:22.648200 26549 raft_consensus.cc:2749] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:09:22.538100 26534 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:22.738088 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:22.739197 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:22.788022 23625 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:22.739480 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
W20260709 13:09:22.739009 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:22.740355 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:22.741559 22880 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 12.
I20260709 13:09:22.682976 26539 raft_consensus.cc:2804] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:09:22.654657 26545 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:22.582821 26632 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:22.518461 26506 consensus_queue.cc:1048] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.207s
I20260709 13:09:22.756546 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:22.717411 26454 consensus_queue.cc:1059] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.059s
I20260709 13:09:22.497516 26381 consensus_queue.cc:260] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:22.714035 22920 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:22.767663 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:22.775959 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:22.653968 26619 consensus_queue.cc:1048] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.089s
I20260709 13:09:22.721403 26577 raft_consensus.cc:740] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
W20260709 13:09:22.779786 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 87790.8us
I20260709 13:09:22.798307 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:22.741068 26551 raft_consensus.cc:2804] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:09:22.673941 26604 consensus_queue.cc:1059] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.083s
I20260709 13:09:22.676360 26349 consensus_queue.cc:1048] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.084s
I20260709 13:09:22.712656 26617 raft_consensus.cc:2804] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:09:22.723747 26544 consensus_queue.cc:1059] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.071s
W20260709 13:09:22.956759 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:22.473284 26462 consensus_queue.cc:260] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:22.768395 26440 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:23.075342 26635 raft_consensus.cc:3055] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:09:23.075742 26635 raft_consensus.cc:740] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:23.076804 26551 raft_consensus.cc:697] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 6 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:22.679083 26614 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:23.051900 26618 consensus_queue.cc:237] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:23.077905 26551 consensus_queue.cc:237] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:23.079070 26614 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:22.861734 26632 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:22.864813 23619 raft_consensus.cc:1240] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 11. Current term is 12. Ops: []
I20260709 13:09:22.972240 23625 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
I20260709 13:09:22.845892 26592 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:22.776556 22872 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a424231ef7b4470bbd575564909a365e"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:53724
I20260709 13:09:23.081485 26632 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:23.081737 22872 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:09:23.082154 22872 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:23.083361 26440 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:23.026350 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:23.085088 22873 raft_consensus.cc:2393] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 6: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:23.085549 26440 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:22.723830 26502 consensus_queue.cc:1048] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.087s
W20260709 13:09:23.087968 26640 log.cc:927] Time spent T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.311s	user 0.001s	sys 0.000s
W20260709 13:09:23.090080 26641 log.cc:927] Time spent T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.226s	user 0.000s	sys 0.000s
I20260709 13:09:23.091847 26577 consensus_queue.cc:260] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:23.093189 26577 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:23.095309 26381 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:23.069767 26611 raft_consensus.cc:2749] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:09:23.067446 26606 consensus_queue.cc:1048] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:09:23.064560 26347 consensus_queue.cc:237] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:23.073568 26617 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:23.105587 26617 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:23.107456 26403 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:23.108934 26605 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:23.109670 26605 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:23.113768 26544 raft_consensus.cc:3055] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:09:23.114302 26544 raft_consensus.cc:740] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:23.115348 26544 consensus_queue.cc:260] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:22.894421 26539 raft_consensus.cc:697] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:22.982481 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:23.048928 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 434516us
I20260709 13:09:23.122359 22843 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:22.963565 26425 raft_consensus.cc:2764] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:09:22.855640 26531 raft_consensus.cc:2804] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:09:22.742214 26487 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:22.790594 26455 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:22.976362 26612 consensus_queue.cc:1059] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.272s
I20260709 13:09:22.843228 26575 raft_consensus.cc:2749] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:09:22.523803 26581 consensus_queue.cc:260] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:22.907651 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 138734us
I20260709 13:09:23.130776 22872 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:23.135371 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:09:23.127308 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:22.865432 26545 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:22.908361 26492 consensus_queue.cc:1048] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.119s
I20260709 13:09:23.052482 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:23.144744 22949 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:22.978641 26568 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:23.054921 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 289651us
I20260709 13:09:23.053373 23028 raft_consensus.cc:1240] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 13:09:23.058826 22880 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
W20260709 13:09:23.061798 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 347819us
I20260709 13:09:23.129943 26632 raft_consensus.cc:2804] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:23.127889 26544 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:23.070704 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:23.074760 26631 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:09:23.076633 26635 consensus_queue.cc:260] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:22.908447 26628 raft_consensus.cc:2749] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:09:23.120077 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:23.122854 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 50108.4us
I20260709 13:09:23.202596 22918 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:23.205950 22955 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2469) took 1664 ms. Trace:
I20260709 13:09:23.071444 22948 raft_consensus.cc:1275] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:09:23.123499 22947 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
I20260709 13:09:23.051258 23616 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:23.138600 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:23.209980 23022 raft_consensus.cc:2393] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 6: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:23.140108 23027 raft_consensus.cc:1240] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 13:09:23.142513 23029 raft_consensus.cc:1275] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:09:22.769358 26602 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:23.145490 26549 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:23.149201 22880 raft_consensus.cc:1240] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:23.147730 26385 raft_consensus.cc:3055] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:09:23.132498 26605 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:23.152769 26506 consensus_queue.cc:1059] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:09:23.153405 22872 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:23.153671 26578 consensus_queue.cc:1059] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:09:23.155336 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:23.132794 26455 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:23.158751 26531 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:23.159435 26487 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:23.216172 26631 raft_consensus.cc:697] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:23.256735 26631 consensus_queue.cc:237] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:23.212074 26575 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:23.260190 26575 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:23.262753 26575 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:23.212621 26454 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:23.263625 26454 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:23.165592 26212 consensus_queue.cc:1059] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:09:23.266142 26454 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:23.266275 26212 consensus_queue.cc:1243] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Peer f3c799b1249e40119d2a5864231c6c75 log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:09:23.269174 22950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 569) took 1102 ms. Trace:
I20260709 13:09:23.269863 22774 catalog_manager.cc:5697] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:09:23.219523 26592 raft_consensus.cc:2749] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:09:23.164299 22994 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:09:23.273794 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 109586us
I20260709 13:09:23.274642 22994 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:23.276134 22994 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:23.177428 26539 consensus_queue.cc:237] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:23.183199 22949 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11.
W20260709 13:09:23.049546 26482 consensus_meta.cc:287] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.743s	user 0.011s	sys 0.001s
I20260709 13:09:23.277503 26482 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:23.278770 23028 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 598) took 1429 ms. Trace:
I20260709 13:09:23.279217 23028 rpcz_store.cc:276] 0709 13:09:21.849467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:21.972072 (+122605us) service_pool.cc:224] Handling call
0709 13:09:21.972357 (+   285us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:23.154975 (+1182618us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:23.278437 (+123462us) spinlock_profiling.cc:243] Waited 883 ms on lock 0x7b54001c84d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:23.278697 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2030119168}
I20260709 13:09:23.280156 26482 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:23.195338 22843 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:23.195696 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:23.183851 26632 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:23.282320 22954 raft_consensus.cc:2393] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 10: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:23.282526 26632 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:23.283924 26502 raft_consensus.cc:3055] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:09:23.284397 26502 raft_consensus.cc:740] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:23.285425 26502 consensus_queue.cc:260] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:23.286909 26502 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:09:23.289306 23588 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.042s	sys 0.073s
W20260709 13:09:23.289633 23588 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.042s	sys 0.073s
I20260709 13:09:23.214335 22918 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:23.142514 26425 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:23.149719 22952 raft_consensus.cc:1240] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:23.216979 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:23.291116 22880 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:23.206442 22955 rpcz_store.cc:276] 0709 13:09:21.541787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:21.541933 (+   146us) service_pool.cc:224] Handling call
0709 13:09:21.542311 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:21.555659 (+ 13348us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:21.555689 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:21.556109 (+   420us) log.cc:844] Serialized 26 byte log entry
0709 13:09:22.534854 (+978745us) spinlock_profiling.cc:243] Waited 4.94 ms on lock 0x7b18003ce5c0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cdae07 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf7c02 00007f613597f20b 00007f6135984d23 00007f61359850d1 00007f6135da0a0f
0709 13:09:22.851590 (+316736us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:09:22.851626 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:22.851689 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:23.187696 (+336007us) raft_consensus.cc:1685] finished
0709 13:09:23.187722 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:23.205873 (+ 18151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18488,"raft.queue_time_us":13595,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":6998,"spinlock_wait_cycles":11358976,"thread_start_us":316560,"threads_started":1,"wal-append.queue_time_us":1589336}
I20260709 13:09:23.219974 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:23.269671 22950 rpcz_store.cc:276] 0709 13:09:22.166586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:22.395025 (+228439us) service_pool.cc:224] Handling call
0709 13:09:22.395319 (+   294us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:22.552895 (+157576us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:22.552945 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:22.553573 (+   628us) log.cc:844] Serialized 26 byte log entry
0709 13:09:22.885028 (+331455us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:09:22.885082 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:22.885181 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:23.268921 (+383740us) raft_consensus.cc:1685] finished
0709 13:09:23.268946 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:23.269105 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":157133,"raft.queue_time_us":158643,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":92,"thread_start_us":330871,"threads_started":1,"wal-append.queue_time_us":676277}
I20260709 13:09:23.292969 22951 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 9.
W20260709 13:09:23.220489 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:23.294034 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:23.294205 22946 raft_consensus.cc:1240] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 7. Current term is 8. Ops: []
I20260709 13:09:23.224438 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:23.229349 23616 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11.
I20260709 13:09:23.295231 22948 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 8.
I20260709 13:09:23.231272 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: true dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:23.296257 22947 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:23.203501 26623 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:09:23.229032 26311 consensus_meta.cc:287] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.758s	user 0.010s	sys 0.001s
W20260709 13:09:23.240091 26472 consensus_meta.cc:287] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.771s	user 0.008s	sys 0.003s
I20260709 13:09:23.219125 26385 raft_consensus.cc:740] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:23.193343 26398 consensus_queue.cc:1059] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:09:23.243561 26409 raft_consensus.cc:1064] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
I20260709 13:09:23.244468 26628 raft_consensus.cc:3055] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:09:23.245793 26608 raft_consensus.cc:3055] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:09:23.247598 26602 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:23.250595 26545 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:23.252285 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:23.316877 26592 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:23.254649 26455 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:23.255204 26531 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:23.167387 21756 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:09:23.357187 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:23.332260 26575 raft_consensus.cc:2804] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:09:23.281617 22873 raft_consensus.cc:1240] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:09:23.296437 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:23.361857 23028 raft_consensus.cc:2393] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 6: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:23.300382 22946 raft_consensus.cc:1240] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:23.304569 23588 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:23.306488 22871 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.032s	sys 0.018s
W20260709 13:09:23.362634 22871 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.599s	user 0.032s	sys 0.018s
W20260709 13:09:23.317554 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 103300us
W20260709 13:09:23.362778 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 58293.4us
I20260709 13:09:23.312685 26577 consensus_queue.cc:1048] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:09:23.363389 22918 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:23.364744 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:23.272410 22774 catalog_manager.cc:5697] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 6 to 8, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 8 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
W20260709 13:09:23.325474 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 49439us
I20260709 13:09:23.357493 21756 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:09:23.326623 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:23.327649 22875 raft_consensus.cc:3060] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:23.211054 26589 consensus_queue.cc:1059] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.079s
I20260709 13:09:23.232131 26594 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:23.372411 26407 consensus_queue.cc:1059] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:09:23.358970 26592 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:23.373147 26639 consensus_queue.cc:1048] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:09:23.347129 26487 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:23.312857 26454 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:23.306185 26482 raft_consensus.cc:2749] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:09:23.349882 22872 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:09:23.211644 26612 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:23.384835 26612 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:23.386489 26606 consensus_queue.cc:1059] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:09:23.386834 26612 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:23.387245 26606 consensus_queue.cc:1243] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer aed3f089ec8b49af9c8cf5cdee76e8e7 log is divergent from this leader: its last log entry 8.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:09:23.387703 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:23.382575 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: true dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:23.389935 23621 raft_consensus.cc:3055] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:09:23.304605 26496 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:23.351409 22992 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:23.390209 23621 raft_consensus.cc:740] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:23.390524 26496 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:23.391016 23621 consensus_queue.cc:260] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:23.392058 23621 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:23.229733 26549 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:23.276225 26509 raft_consensus.cc:2804] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:23.276922 26401 raft_consensus.cc:2804] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:23.333241 26492 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:23.359853 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:23.306905 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 111643us
W20260709 13:09:23.353291 26534 consensus_meta.cc:287] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.508s	user 0.004s	sys 0.007s
I20260709 13:09:23.367206 22948 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:23.369484 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:23.406294 22843 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:23.429409 23625 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 9.
I20260709 13:09:23.365530 23616 raft_consensus.cc:2393] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 10: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:23.347306 26628 raft_consensus.cc:740] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:23.324445 26311 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:23.372995 26594 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:23.234302 26635 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:23.349125 26608 raft_consensus.cc:740] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:23.368039 22918 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:23.337630 22880 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
I20260709 13:09:23.440192 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:23.440809 22880 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 9.
I20260709 13:09:23.441530 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:23.442541 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:23.443431 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:23.366130 22774 catalog_manager.cc:5697] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 7 to 8, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 8 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:23.445004 22774 catalog_manager.cc:5697] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:09:23.312939 26425 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:23.378575 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:23.324302 26623 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:23.382541 26592 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:23.452064 26425 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:23.453893 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:23.454485 22874 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 9.
I20260709 13:09:23.454825 26632 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:23.383174 26611 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:23.455804 26611 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:23.457036 26628 consensus_queue.cc:260] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:23.457677 23619 raft_consensus.cc:1275] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:09:23.457837 26611 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:23.383617 26506 raft_consensus.cc:3055] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:09:23.458468 26506 raft_consensus.cc:740] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:09:23.459149 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:23.459227 26506 consensus_queue.cc:260] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:23.384268 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:23.401136 26509 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:23.460953 26509 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:23.460984 22872 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:09:23.463140 22873 raft_consensus.cc:1240] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 5. Current term is 6. Ops: []
I20260709 13:09:23.463251 26617 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:23.394534 23590 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.007s	sys 0.011s
I20260709 13:09:23.392314 22992 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:23.333488 26472 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:23.464735 26612 raft_consensus.cc:3055] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:09:23.465008 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 72758.9us
I20260709 13:09:23.465178 26612 raft_consensus.cc:740] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:23.466074 26612 consensus_queue.cc:260] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:23.466364 26472 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:23.467242 26612 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:23.467677 22954 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:09:23.468178 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:23.417488 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:23.468920 22955 raft_consensus.cc:2393] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 9: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:23.374042 26575 raft_consensus.cc:697] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:23.446384 22774 catalog_manager.cc:5697] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:23.365063 26602 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:23.365154 26498 raft_consensus.cc:2749] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:23.430035 22843 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:23.392531 26496 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:09:23.439551 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 74952.3us
W20260709 13:09:23.488588 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 58653.4us
I20260709 13:09:23.489442 22918 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:23.404593 26401 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:23.449684 26549 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:23.447032 26236 raft_consensus.cc:2749] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:23.428586 26578 raft_consensus.cc:3055] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:09:23.458063 26628 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:23.460706 26608 consensus_queue.cc:260] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:23.464507 23590 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.007s	sys 0.011s
I20260709 13:09:23.466318 26623 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:23.515141 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 758407us
I20260709 13:09:23.516399 23590 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:23.518826 22919 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:23.365001 26648 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:23.470106 22874 raft_consensus.cc:1240] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 13:09:23.464823 22877 raft_consensus.cc:1240] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 5. Current term is 6. Ops: []
I20260709 13:09:23.470255 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:23.526383 22954 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 8.
I20260709 13:09:23.527443 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:23.473968 23621 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 9.
I20260709 13:09:23.476190 22774 catalog_manager.cc:5697] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:09:23.474685 26577 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
W20260709 13:09:23.481580 26409 raft_consensus.cc:1068] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7: Unable to promote non-voter 5b346767a89745fa82928a5dac9cc0b2: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:09:23.482024 26576 raft_consensus.cc:3055] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:09:23.484004 26581 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:09:23.484716 26458 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
W20260709 13:09:23.411111 22777 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33488 (request call id 267) took 20389 ms (client timeout 15000 ms). Trace:
I20260709 13:09:23.487748 22878 raft_consensus.cc:1240] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:09:23.491639 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:23.557307 22843 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:23.494031 22948 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 12.
I20260709 13:09:23.431134 26589 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:23.509081 26549 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:23.508970 26425 consensus_queue.cc:1059] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:09:23.568852 26425 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:23.569310 26425 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:23.569770 26311 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:23.544270 26623 raft_consensus.cc:2749] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:09:23.571353 26425 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:23.433590 26385 consensus_queue.cc:260] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:23.506839 26401 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:23.572145 26385 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:23.572907 26458 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:23.575507 26458 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:23.506853 26472 raft_consensus.cc:2804] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:23.577116 26472 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:23.577486 26472 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:23.577426 26575 consensus_queue.cc:237] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:23.511281 26592 raft_consensus.cc:2749] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 13:09:23.512913 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:23.502967 26403 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:23.535681 26534 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:23.581103 26403 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:23.581584 26534 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:23.583155 26403 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:23.374624 26455 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:23.583487 26534 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:23.585093 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:23.513597 26578 raft_consensus.cc:740] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:23.395471 26487 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:23.401479 26454 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:23.538679 26635 raft_consensus.cc:2804] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:09:23.530800 22774 catalog_manager.cc:5697] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
W20260709 13:09:23.532780 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:23.504892 26492 raft_consensus.cc:2804] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:09:23.510748 26462 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:23.543124 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:23.544119 23625 raft_consensus.cc:1275] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:09:23.617827 23625 pending_rounds.cc:85] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2: Aborting all ops after (but not including) 6
I20260709 13:09:23.618119 23625 pending_rounds.cc:107] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 13:09:23.618331 23625 raft_consensus.cc:2889] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:09:23.618640 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:23.625065 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 194282us
W20260709 13:09:23.627012 22876 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.000s	sys 0.010s
W20260709 13:09:23.630851 22876 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.513s	user 0.000s	sys 0.011s
I20260709 13:09:23.634797 22878 raft_consensus.cc:1240] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 13:09:23.571914 26425 consensus_queue.cc:1059] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:09:23.544147 26464 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:23.564945 26632 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:23.547130 26409 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:23.540452 26576 raft_consensus.cc:740] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:23.536566 26496 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:23.540568 26577 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:23.567086 26589 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:23.567886 26549 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:23.577910 26594 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:23.404448 26639 consensus_queue.cc:1048] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.083s
I20260709 13:09:23.460575 26506 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:23.588317 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:23.535887 26648 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:23.507964 26602 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:23.436000 26647 raft_consensus.cc:2804] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:23.605497 22950 raft_consensus.cc:1275] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260709 13:09:23.606588 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:23.546141 22777 rpcz_store.cc:269] 0709 13:09:02.992038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:02.992273 (+   235us) service_pool.cc:224] Handling call
0709 13:09:23.381618 (+20389345us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:09:16.212876 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:09:16.213556 (+   680us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:09:16.213616 (+    60us) write_op.cc:435] Acquired schema lock
0709 13:09:16.213637 (+    21us) tablet.cc:662] Decoding operations
0709 13:09:16.477128 (+263491us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:09:16.477221 (+    93us) write_op.cc:641] Partition lock acquired for write op
0709 13:09:16.477248 (+    27us) tablet.cc:685] Acquiring locks for 15 operations
0709 13:09:16.478444 (+  1196us) tablet.cc:701] Row locks acquired
0709 13:09:16.478480 (+    36us) write_op.cc:260] PREPARE: finished
0709 13:09:16.478653 (+   173us) write_op.cc:270] Start()
0709 13:09:16.478797 (+   144us) write_op.cc:276] Timestamp: P: 1783602556478589 usec, L: 0
0709 13:09:16.478819 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:09:16.479512 (+   693us) log.cc:844] Serialized 5844 byte log entry
0709 13:09:17.109153 (+629641us) op_driver.cc:464] REPLICATION: finished
0709 13:09:17.950987 (+841834us) write_op.cc:301] APPLY: starting
0709 13:09:18.053019 (+102032us) tablet.cc:1367] starting BulkCheckPresence
0709 13:09:18.121043 (+ 68024us) tablet.cc:1370] finished BulkCheckPresence
0709 13:09:18.121185 (+   142us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:09:18.185666 (+ 64481us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:09:18.186031 (+   365us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=15
0709 13:09:18.186082 (+    51us) write_op.cc:312] APPLY: finished
0709 13:09:18.186546 (+   464us) log.cc:844] Serialized 107 byte log entry
0709 13:09:18.197580 (+ 11034us) spinlock_profiling.cc:243] Waited 3.46 ms on lock 0x7b18002ec8a0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f613598d4bc 00007f6132cdb48b 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf7c02 00007f613981bef0 00007f613981c489 00007f613981e71a
0709 13:09:19.552843 (+1355263us) spinlock_profiling.cc:243] Waited 33.1 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f6132cdb0ac 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf7c02 00007f613597f20b
0709 13:09:19.741158 (+188315us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:19.750925 (+  9767us) write_op.cc:454] Released schema lock
0709 13:09:19.751025 (+   100us) write_op.cc:341] FINISH: Updating metrics
0709 13:09:19.753492 (+  2467us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:19.753526 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":970791,"reported_tablets":42,"child_traces":[["op",{"apply.queue_time_us":753242,"apply.run_cpu_time_us":38269,"apply.run_wall_time_us":1849450,"num_ops":15,"prepare.queue_time_us":227578,"prepare.run_cpu_time_us":93007,"prepare.run_wall_time_us":471591,"raft.queue_time_us":397826,"raft.run_cpu_time_us":25468,"raft.run_wall_time_us":1088954,"replication_time_us":630141,"spinlock_wait_cycles":84021760,"thread_start_us":2762822,"threads_started":5,"wal-append.queue_time_us":1710949,"wal-append.run_cpu_time_us":34098,"wal-append.run_wall_time_us":2380310}]]}
W20260709 13:09:23.726223 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:23.599642 26578 consensus_queue.cc:260] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:23.611903 22774 catalog_manager.cc:5697] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 3 to 5, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 5 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:09:23.646063 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:23.687099 26458 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:23.721095 26508 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.007s	sys 0.006s
I20260709 13:09:23.597534 26311 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:09:23.715060 26425 raft_consensus.cc:3055] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 13:09:23.628731 26618 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.021s	sys 0.040s
I20260709 13:09:23.776935 26425 raft_consensus.cc:740] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:09:23.777132 26618 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.697s	user 0.022s	sys 0.040s
I20260709 13:09:23.671579 22844 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:23.777932 26602 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:23.777860 26425 consensus_queue.cc:260] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:23.778540 22953 raft_consensus.cc:1240] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 7. Current term is 8. Ops: []
I20260709 13:09:23.778689 26602 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:23.619668 26349 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:23.779255 26602 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:23.779574 26349 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:23.664634 26464 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:23.611371 26482 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:23.781554 26482 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:23.782399 26464 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:23.783948 26482 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:23.784565 26639 raft_consensus.cc:3055] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:09:23.784992 26639 raft_consensus.cc:740] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:23.786509 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:23.644999 26635 raft_consensus.cc:697] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:23.723340 26647 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:23.645326 26492 raft_consensus.cc:697] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:23.718353 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:23.661787 26617 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:23.716355 23020 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.002s	sys 0.011s
W20260709 13:09:23.823789 23020 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.002s	sys 0.011s
I20260709 13:09:23.681571 26632 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:23.730312 22920 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:23.731128 23625 raft_consensus.cc:2393] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 9: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:23.636448 22878 raft_consensus.cc:1275] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
W20260709 13:09:23.825130 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 94889.4us
W20260709 13:09:23.719609 23021 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.006s	sys 0.008s
I20260709 13:09:23.682511 26649 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:23.736205 22872 raft_consensus.cc:1275] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 13:09:23.747113 22879 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.010s	sys 0.035s
I20260709 13:09:23.611052 22947 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 9.
W20260709 13:09:23.850515 22879 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.011s	sys 0.035s
W20260709 13:09:23.721946 26551 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.001s	sys 0.013s
I20260709 13:09:23.755962 22873 raft_consensus.cc:1275] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:09:23.563714 23624 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11.
I20260709 13:09:23.758170 22875 raft_consensus.cc:1275] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260709 13:09:23.854455 22875 pending_rounds.cc:85] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 7
I20260709 13:09:23.854816 22875 pending_rounds.cc:107] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 13:09:23.855118 22875 raft_consensus.cc:2889] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:23.855921 22875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2341) took 2567 ms. Trace:
I20260709 13:09:23.664250 26606 consensus_queue.cc:1059] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 13:09:23.766142 26487 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:23.766497 26576 consensus_queue.cc:260] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:09:23.767196 26509 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:23.770277 26455 raft_consensus.cc:2749] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:23.682571 26409 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:23.773984 26648 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:23.648720 26605 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:23.907070 26617 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:23.683584 26496 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:23.735661 26594 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:23.864801 26649 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:23.907385 26605 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:23.908222 26594 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:23.791803 26458 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:23.910734 26594 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:23.910903 26458 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:23.798471 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:23.826128 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:23.910948 26496 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:23.918157 22878 raft_consensus.cc:1240] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 9. Current term is 10. Ops: [7.6-9.7]
I20260709 13:09:23.730063 22774 catalog_manager.cc:5697] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 4 to 8, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 8 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:23.921047 26635 consensus_queue.cc:237] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:23.823235 22877 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:23.923097 22878 raft_consensus.cc:1240] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 8. Current term is 9. Ops: [8.6-8.6]
I20260709 13:09:23.826004 22920 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:09:23.924368 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 98465.8us
I20260709 13:09:23.925221 22920 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:23.925995 22920 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:23.856367 22875 rpcz_store.cc:276] 0709 13:09:21.288507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:21.409593 (+121086us) service_pool.cc:224] Handling call
0709 13:09:21.420323 (+ 10730us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:21.827968 (+407645us) spinlock_profiling.cc:243] Waited 7.5 ms on lock 0x7b54002817a0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135d5cdc0 00007f6135de71c0 00007f6135de8231 00007f6135de4d8f 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c
0709 13:09:22.529942 (+701974us) spinlock_profiling.cc:243] Waited 76.4 ms on lock 0x7b54002817a0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135d6306f 00007f6135ddbf60 00007f6135ddd7de 00007f6135de7b49 00007f6135de8416 00007f6135de4d8f 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62
0709 13:09:23.167086 (+637144us) spinlock_profiling.cc:243] Waited 499 ms on lock 0x7b58003e15d0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135d410aa 00007f6135dc3447 00007f6135ddbfe7 00007f6135ddd7de 00007f6135de7b49 00007f6135de8416 00007f6135de4d8f 00007f6135de483e 00007f613ada9419 00007f6134320bd4
0709 13:09:23.313668 (+146582us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b54002817a0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135d5aa1d 00007f6135d411a4 00007f6135dc3447 00007f6135ddbfe7 00007f6135ddd7de 00007f6135de7b49 00007f6135de8416 00007f6135de4d8f 00007f6135de483e 00007f613ada9419
0709 13:09:23.855651 (+541983us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:23.855860 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1603520384}
I20260709 13:09:23.928364 22877 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 12.
I20260709 13:09:23.865219 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:23.928882 26236 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:23.929461 26236 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:23.866758 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:23.851615 22879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2351) took 1274 ms. Trace:
I20260709 13:09:23.867460 23624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 753) took 1130 ms. Trace:
I20260709 13:09:23.929975 26236 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:23.930351 23624 rpcz_store.cc:276] 0709 13:09:22.737456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:22.739793 (+  2337us) service_pool.cc:224] Handling call
0709 13:09:23.867241 (+1127448us) spinlock_profiling.cc:243] Waited 417 ms on lock 0x7b540048bf54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:23.867392 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":958122368}
I20260709 13:09:23.930086 22879 rpcz_store.cc:276] 0709 13:09:22.577705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:22.594217 (+ 16512us) service_pool.cc:224] Handling call
0709 13:09:22.594513 (+   296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:22.594811 (+   298us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:22.594843 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:22.635909 (+ 41066us) log.cc:844] Serialized 26 byte log entry
0709 13:09:23.201735 (+565826us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:09:23.850996 (+649261us) log.cc:844] Serialized 14 byte log entry
0709 13:09:23.851238 (+   242us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:23.851339 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:23.851393 (+    54us) raft_consensus.cc:1685] finished
0709 13:09:23.851409 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:23.851549 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":374320,"prepare.run_cpu_time_us":23,"prepare.run_wall_time_us":23,"raft.queue_time_us":3477,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":53999,"thread_start_us":1080365,"threads_started":2,"wal-append.queue_time_us":743361}
I20260709 13:09:23.869099 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:23.784590 26482 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:23.813366 26578 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:23.933714 26594 raft_consensus.cc:2804] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:23.934213 26594 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:23.854563 26523 raft_consensus.cc:2804] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:09:23.932396 22873 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
I20260709 13:09:23.935068 26523 raft_consensus.cc:697] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:23.934659 26594 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:23.935984 26523 consensus_queue.cc:237] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:23.827625 26626 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.006s	sys 0.024s
W20260709 13:09:23.936949 26626 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.006s	sys 0.024s
W20260709 13:09:23.854287 26551 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.001s	sys 0.013s
I20260709 13:09:23.810855 26454 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:23.863168 26568 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:23.920769 22774 catalog_manager.cc:5697] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 4 to 6, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 6 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:09:23.938311 26568 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:23.939347 26389 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 13:09:23.906720 26409 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:23.939085 22774 catalog_manager.cc:5697] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: config changed from index -1 to 5, term changed from 4 to 6, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New cstate: current_term: 6 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
W20260709 13:09:23.754280 26508 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.008s	sys 0.006s
I20260709 13:09:23.929597 22872 raft_consensus.cc:2393] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 11: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:23.771720 26472 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:23.932458 26482 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:23.931821 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
W20260709 13:09:23.875280 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:23.928982 26381 raft_consensus.cc:2804] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:23.943987 26381 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:23.944392 26381 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:23.930154 23625 raft_consensus.cc:2468] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 8.
I20260709 13:09:23.945422 23625 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:23.946087 22774 catalog_manager.cc:5697] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 3 to 5, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 5 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:23.947674 22774 catalog_manager.cc:5697] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:09:23.907836 26617 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:23.826730 23021 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.006s	sys 0.008s
I20260709 13:09:23.824224 26311 raft_consensus.cc:2749] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:09:23.945442 22874 raft_consensus.cc:2393] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 10: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:23.952025 22774 catalog_manager.cc:5697] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 3 to 6, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 6 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:23.965835 23625 raft_consensus.cc:1275] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:09:23.968086 22774 catalog_manager.cc:5697] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 7 to 8, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 8 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:23.846982 26647 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:09:23.989411 22871 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.033s	sys 0.000s
W20260709 13:09:23.990904 22871 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.033s	sys 0.000s
W20260709 13:09:23.965528 26347 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.844s	user 0.021s	sys 0.008s
I20260709 13:09:23.972365 22774 catalog_manager.cc:5697] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
W20260709 13:09:23.989616 23025 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.012s	sys 0.017s
I20260709 13:09:23.791240 26401 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:23.949465 26212 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:23.979995 26576 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:23.991739 22871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53724 (request call id 589) took 1272 ms. Trace:
W20260709 13:09:23.995913 23025 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.013s	sys 0.017s
I20260709 13:09:23.863206 26632 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:24.008193 23028 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.012s	sys 0.020s
I20260709 13:09:23.995541 22774 catalog_manager.cc:5697] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 2 to 5, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 5 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
W20260709 13:09:24.003510 26544 consensus_meta.cc:287] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.810s	user 0.010s	sys 0.000s
W20260709 13:09:23.996851 26347 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.892s	user 0.023s	sys 0.009s
I20260709 13:09:23.826725 26639 consensus_queue.cc:260] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:24.019140 23028 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.013s	sys 0.020s
I20260709 13:09:24.025676 26602 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:24.031894 22994 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.022s	sys 0.011s
W20260709 13:09:24.007633 26531 consensus_meta.cc:287] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.639s	user 0.003s	sys 0.006s
W20260709 13:09:24.034972 22994 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.661s	user 0.023s	sys 0.012s
W20260709 13:09:24.035403 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 663906us
I20260709 13:09:24.015095 22871 rpcz_store.cc:276] 0709 13:09:22.719160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:22.732539 (+ 13379us) service_pool.cc:224] Handling call
0709 13:09:22.732960 (+   421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:22.733223 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:09:23.363135 (+629912us) log.cc:844] Serialized 14 byte log entry
0709 13:09:23.991372 (+628237us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:23.991460 (+    88us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:23.991504 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:23.991564 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:23.991684 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":842600,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":93,"raft.run_cpu_time_us":75,"raft.run_wall_time_us":75,"thread_start_us":1226881,"threads_started":2}
W20260709 13:09:24.019045 26631 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.010s	sys 0.020s
I20260709 13:09:24.026361 22774 catalog_manager.cc:5697] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 6 to 8, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 8 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:09:24.025215 22951 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.014s	sys 0.015s
I20260709 13:09:24.035957 23022 raft_consensus.cc:1240] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:09:23.907258 26492 consensus_queue.cc:237] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:24.044013 22952 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.668s	user 0.024s	sys 0.037s
W20260709 13:09:24.044415 22952 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.668s	user 0.024s	sys 0.037s
W20260709 13:09:24.045192 22951 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.014s	sys 0.015s
I20260709 13:09:24.045567 23618 raft_consensus.cc:2410] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
W20260709 13:09:24.052772 22992 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.004s	sys 0.025s
W20260709 13:09:24.053071 22992 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.004s	sys 0.025s
W20260709 13:09:24.053304 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 586366us
W20260709 13:09:24.040874 26631 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.010s	sys 0.020s
W20260709 13:09:24.035917 26539 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.011s	sys 0.017s
I20260709 13:09:24.058003 22774 catalog_manager.cc:5697] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 3 to 7, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 7 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:24.079516 22774 catalog_manager.cc:5697] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 6 to 7, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 7 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:24.082182 23622 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 754) took 1344 ms. Trace:
I20260709 13:09:24.082718 23622 rpcz_store.cc:276] 0709 13:09:22.737736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:22.737855 (+   119us) service_pool.cc:224] Handling call
0709 13:09:24.071053 (+1333198us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54005a23d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:24.082100 (+ 11047us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2645517696}
I20260709 13:09:24.063338 22994 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:24.082233 22774 catalog_manager.cc:5697] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 6 to 10, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 10 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:24.086239 22774 catalog_manager.cc:5697] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
W20260709 13:09:24.088599 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:24.020836 26212 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:24.088025 22774 catalog_manager.cc:5697] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:24.139293 22774 catalog_manager.cc:5697] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:09:24.142895 22774 catalog_manager.cc:5697] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 3 to 8. New cstate: current_term: 8 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:09:24.090956 22994 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:09:24.093109 22949 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.024s	sys 0.008s
W20260709 13:09:24.102818 22993 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.012s	sys 0.017s
W20260709 13:09:24.150183 22993 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.012s	sys 0.017s
I20260709 13:09:24.152729 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:09:24.070494 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:24.153940 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:24.108826 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:24.155671 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:24.156711 23622 raft_consensus.cc:2393] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 8: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
W20260709 13:09:24.149523 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 58636.7us
I20260709 13:09:24.159574 22994 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:09:24.164793 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:24.167161 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:24.153340 22993 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:24.171079 23620 raft_consensus.cc:1240] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:09:24.054488 26619 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:24.078176 26539 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.011s	sys 0.017s
I20260709 13:09:24.041148 26487 consensus_queue.cc:1059] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.099s
W20260709 13:09:24.174154 22946 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.724s	user 0.021s	sys 0.021s
I20260709 13:09:24.079185 26549 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:24.079711 26639 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:24.039675 26531 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:24.090427 26506 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:24.188788 22946 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.739s	user 0.022s	sys 0.021s
I20260709 13:09:24.195612 26349 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:24.204298 26656 log.cc:927] Time spent T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
I20260709 13:09:24.207137 26454 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
W20260709 13:09:24.214082 26502 consensus_meta.cc:287] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.923s	user 0.006s	sys 0.006s
W20260709 13:09:24.182114 26385 consensus_meta.cc:287] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.610s	user 0.010s	sys 0.000s
I20260709 13:09:24.171840 26617 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:24.170732 26611 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
W20260709 13:09:24.216499 23616 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.616s	user 0.006s	sys 0.020s
I20260709 13:09:24.139525 26577 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:24.233263 23623 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.662s	user 0.017s	sys 0.062s
I20260709 13:09:24.236876 22948 raft_consensus.cc:2393] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 7: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
W20260709 13:09:24.248693 23623 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.017s	sys 0.063s
I20260709 13:09:24.186590 26619 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:24.245083 23616 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.006s	sys 0.020s
W20260709 13:09:24.221917 26440 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.017s	sys 0.024s
W20260709 13:09:24.252854 22919 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.717s	user 0.020s	sys 0.017s
W20260709 13:09:24.277209 22919 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.742s	user 0.020s	sys 0.017s
W20260709 13:09:24.277920 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 759112us
I20260709 13:09:24.279129 23588 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:24.280784 23588 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:24.283015 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:24.285091 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:09:24.263816 22876 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.014s	sys 0.009s
I20260709 13:09:24.285203 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:09:24.155939 22949 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.649s	user 0.025s	sys 0.009s
W20260709 13:09:24.268996 22995 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.004s	sys 0.021s
I20260709 13:09:24.245311 26509 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:09:24.303814 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:24.271108 23621 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.013s	sys 0.027s
W20260709 13:09:24.272195 22846 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.012s	sys 0.012s
I20260709 13:09:24.222587 26506 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:24.253985 26575 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.028s	sys 0.056s
I20260709 13:09:24.314626 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:24.287165 22995 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.005s	sys 0.021s
W20260709 13:09:24.316248 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:24.285687 22876 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.015s	sys 0.009s
W20260709 13:09:24.287750 22844 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.007s	sys 0.012s
W20260709 13:09:24.317710 22844 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.008s	sys 0.012s
W20260709 13:09:24.318315 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 646785us
I20260709 13:09:24.319432 22844 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:24.285647 23622 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
W20260709 13:09:24.305457 22846 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.012s	sys 0.012s
W20260709 13:09:24.305773 23621 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.013s	sys 0.027s
W20260709 13:09:24.302971 26440 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.017s	sys 0.024s
I20260709 13:09:24.309592 23020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 600) took 1176 ms. Trace:
I20260709 13:09:24.320551 22844 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:24.242669 26454 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:24.306007 22843 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.010s	sys 0.013s
I20260709 13:09:24.330307 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:24.363111 23020 rpcz_store.cc:276] 0709 13:09:23.133467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:23.137558 (+  4091us) service_pool.cc:224] Handling call
0709 13:09:23.137867 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:23.138110 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:09:23.846919 (+708809us) log.cc:844] Serialized 14 byte log entry
0709 13:09:24.309098 (+462179us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:24.309157 (+    59us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:24.309227 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:24.309323 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:24.309526 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2487,"prepare.queue_time_us":915116,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":59,"thread_start_us":1098534,"threads_started":2}
W20260709 13:09:24.367650 22843 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.011s	sys 0.020s
W20260709 13:09:24.364470 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 43968.9us
I20260709 13:09:24.369432 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:24.370483 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:24.371481 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:24.372538 26311 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:24.372740 26648 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:24.373078 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:24.373168 26311 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:24.373708 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:24.374343 26496 consensus_queue.cc:1059] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:09:24.375682 26496 raft_consensus.cc:3055] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:09:24.376040 26496 raft_consensus.cc:740] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:24.375890 26311 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:24.377110 26496 consensus_queue.cc:260] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:24.377812 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:24.378079 26311 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:24.378422 26496 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:24.373276 26648 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:24.361452 26656 log.cc:927] Time spent T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:09:24.378811 26311 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:24.380143 22876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2352) took 1759 ms. Trace:
I20260709 13:09:24.380719 26648 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:24.381388 26648 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:24.381510 26311 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:24.380533 22876 rpcz_store.cc:276] 0709 13:09:22.621028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:22.621169 (+   141us) service_pool.cc:224] Handling call
0709 13:09:22.621544 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:22.956957 (+335413us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:22.956989 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:23.040822 (+ 83833us) log.cc:844] Serialized 26 byte log entry
0709 13:09:23.633982 (+593160us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:09:23.682460 (+ 48478us) log.cc:844] Serialized 14 byte log entry
0709 13:09:24.317204 (+634744us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:24.317310 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:24.379936 (+ 62626us) raft_consensus.cc:1685] finished
0709 13:09:24.379957 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:24.380085 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":29097,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":165198,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":16092,"thread_start_us":1485092,"threads_started":3,"wal-append.queue_time_us":942064}
I20260709 13:09:24.382097 26648 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:24.372627 26462 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:24.383694 26462 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:24.377214 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:24.384830 23625 raft_consensus.cc:2393] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 11: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:24.385706 26648 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:24.386575 22955 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 11.
I20260709 13:09:24.387468 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:24.378532 23622 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
W20260709 13:09:24.051862 26628 consensus_queue.cc:1175] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "0a407504ed1741bf89a276558adf4c0f" responder_term: 11 status { last_received { term: 9 index: 8 } last_committed_idx: 7 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:09:24.251156 26611 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:24.372438 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:24.394814 22954 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 7.
I20260709 13:09:24.307121 26498 raft_consensus.cc:2804] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:24.392431 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:24.398029 22953 raft_consensus.cc:2468] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 8.
I20260709 13:09:24.309165 26577 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:09:24.324991 26575 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.734s	user 0.028s	sys 0.056s
I20260709 13:09:24.367722 22919 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:24.308730 26398 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:24.405380 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:24.406821 23620 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60044 (request call id 707) took 1667 ms. Trace:
I20260709 13:09:24.417414 23622 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
I20260709 13:09:24.325711 26509 raft_consensus.cc:2804] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:24.327181 26539 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:24.328437 26617 raft_consensus.cc:2804] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:09:24.327983 26632 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:24.361680 26409 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:24.417304 23620 rpcz_store.cc:276] 0709 13:09:22.739436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:22.739647 (+   211us) service_pool.cc:224] Handling call
0709 13:09:22.739947 (+   300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:24.171634 (+1431687us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:24.406563 (+234929us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b54004943d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:24.406755 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3290525440}
I20260709 13:09:24.399338 26602 raft_consensus.cc:2804] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:24.412348 26462 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:24.402554 26498 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:24.330395 26549 consensus_queue.cc:1048] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.095s
I20260709 13:09:24.424022 26509 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:24.427026 26407 raft_consensus.cc:2804] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:09:24.433264 26655 log.cc:927] Time spent T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.354s	user 0.001s	sys 0.000s
I20260709 13:09:24.420360 26577 raft_consensus.cc:697] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:24.406406 26618 consensus_queue.cc:1048] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 13:09:24.423090 26648 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:24.433202 26453 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:24.446668 26602 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:24.447247 26617 raft_consensus.cc:697] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 9 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:24.456983 26498 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:09:24.463968 26635 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.021s	sys 0.013s
I20260709 13:09:24.452144 26502 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:24.456207 26675 raft_consensus.cc:2804] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:09:24.472486 26658 log.cc:927] Time spent T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.301s	user 0.001s	sys 0.000s
I20260709 13:09:24.436206 26575 consensus_queue.cc:1048] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.036s
I20260709 13:09:24.464679 26509 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:24.434623 26539 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:24.476534 26531 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:24.420411 22876 raft_consensus.cc:1275] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:09:24.425207 26398 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:24.452677 26409 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:24.483793 26635 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.022s	sys 0.015s
I20260709 13:09:24.476336 26648 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:24.976892 26664 log.cc:927] Time spent T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.559s	user 0.003s	sys 0.007s
I20260709 13:09:24.877254 26502 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:24.861563 26487 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.629s	user 0.010s	sys 0.015s
W20260709 13:09:24.488087 26523 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.010s	sys 0.017s
I20260709 13:09:24.502249 26626 raft_consensus.cc:2749] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:09:24.981077 26523 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.042s	user 0.010s	sys 0.017s
I20260709 13:09:24.477088 26602 raft_consensus.cc:3060] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:24.469396 26544 consensus_queue.cc:1048] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:09:24.498193 26676 raft_consensus.cc:2804] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:09:24.493815 26659 log.cc:927] Time spent T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.248s	user 0.000s	sys 0.001s
I20260709 13:09:25.011663 26212 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:24.829707 22992 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.003s	sys 0.020s
W20260709 13:09:25.013199 22992 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.858s	user 0.006s	sys 0.022s
I20260709 13:09:25.013639 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:25.014936 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:25.016103 22877 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 9.
I20260709 13:09:24.488477 26672 consensus_queue.cc:1048] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 13:09:24.475903 26453 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:25.017802 26649 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:25.019112 26611 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:25.020077 26453 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:24.485811 26675 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:24.830098 22993 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.017s	sys 0.005s
W20260709 13:09:25.021143 22993 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.842s	user 0.020s	sys 0.007s
I20260709 13:09:25.021224 26675 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:09:25.022974 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 869606us
W20260709 13:09:24.830397 22994 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.670s	user 0.004s	sys 0.018s
W20260709 13:09:24.828236 22949 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.016s	sys 0.024s
W20260709 13:09:24.850741 23616 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.002s	sys 0.000s
W20260709 13:09:25.025038 23616 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.716s	user 0.002s	sys 0.004s
W20260709 13:09:25.025390 26665 log.cc:927] Time spent T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.532s	user 0.003s	sys 0.006s
W20260709 13:09:25.026149 22949 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.740s	user 0.019s	sys 0.028s
W20260709 13:09:25.024572 22994 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.864s	user 0.004s	sys 0.025s
W20260709 13:09:24.552871 26677 log.cc:927] Time spent T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.077s	user 0.002s	sys 0.000s
I20260709 13:09:24.934495 22950 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 8.
I20260709 13:09:25.032114 23623 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 761) took 1489 ms. Trace:
W20260709 13:09:25.033766 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 874210us
W20260709 13:09:24.849326 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:24.522644 26678 log.cc:927] Time spent T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.078s	user 0.002s	sys 0.000s
W20260709 13:09:24.524039 26657 log.cc:927] Time spent T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.198s	user 0.000s	sys 0.001s
W20260709 13:09:24.555169 26673 log.cc:927] Time spent T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 13:09:25.038136 22875 raft_consensus.cc:1275] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 13:09:24.551614 26671 log.cc:927] Time spent T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 13:09:24.465677 26401 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:24.974843 26506 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:24.975533 23621 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.002s	sys 0.002s
I20260709 13:09:25.008283 23618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 759) took 1630 ms. Trace:
W20260709 13:09:25.040714 23621 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.704s	user 0.002s	sys 0.003s
I20260709 13:09:25.040969 23618 rpcz_store.cc:276] 0709 13:09:23.378179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:23.378319 (+   140us) service_pool.cc:224] Handling call
0709 13:09:25.007586 (+1629267us) spinlock_profiling.cc:243] Waited 598 ms on lock 0x7b5400469454. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:25.008020 (+   434us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1374279552}
I20260709 13:09:25.042177 22946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2500) took 1647 ms. Trace:
I20260709 13:09:24.980604 26648 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:25.042775 22946 rpcz_store.cc:276] 0709 13:09:23.394806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:23.394937 (+   131us) service_pool.cc:224] Handling call
0709 13:09:23.395209 (+   272us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:23.395415 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:23.395437 (+    22us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:23.395838 (+   401us) log.cc:844] Serialized 26 byte log entry
0709 13:09:24.216105 (+820267us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:09:24.216166 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:24.216266 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:24.550885 (+334619us) spinlock_profiling.cc:243] Waited 28.4 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:09:25.041486 (+490601us) raft_consensus.cc:1685] finished
0709 13:09:25.041527 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:25.041926 (+   399us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":65223424,"thread_start_us":766296,"threads_started":1,"wal-append.queue_time_us":1037242}
W20260709 13:09:25.046432 26666 log.cc:927] Time spent T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.611s	user 0.004s	sys 0.006s
W20260709 13:09:25.046969 26594 consensus_meta.cc:287] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 1.111s	user 0.013s	sys 0.006s
W20260709 13:09:25.047426 22843 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.871s (SENT) [suppressed 29 similar messages]
I20260709 13:09:25.047605 26594 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:25.050102 26687 log.cc:927] Time spent T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 13:09:24.975334 26492 raft_consensus.cc:2804] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:09:24.881747 26577 consensus_queue.cc:237] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:24.953460 26549 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:24.997897 26440 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.006s	sys 0.005s
I20260709 13:09:24.998333 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:24.999598 26539 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:25.003180 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:25.003036 26398 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:25.005810 26409 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:25.010540 22873 raft_consensus.cc:1240] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:09:25.010244 26617 consensus_queue.cc:237] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:24.989617 26487 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.769s	user 0.012s	sys 0.023s
I20260709 13:09:25.042177 23621 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 762) took 1499 ms. Trace:
I20260709 13:09:25.032647 23623 rpcz_store.cc:276] 0709 13:09:23.542083 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:23.542186 (+   103us) service_pool.cc:224] Handling call
0709 13:09:23.542509 (+   323us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:23.545207 (+  2698us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:23.545261 (+    54us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:23.571332 (+ 26071us) log.cc:844] Serialized 52 byte log entry
0709 13:09:24.249181 (+677849us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:09:24.249241 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:24.249355 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:24.552228 (+302873us) spinlock_profiling.cc:243] Waited 41.8 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:09:25.031369 (+479141us) raft_consensus.cc:1685] finished
0709 13:09:25.031412 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:25.031865 (+   453us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2233,"raft.queue_time_us":2598,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":29883,"spinlock_wait_cycles":96238848,"thread_start_us":677441,"threads_started":1,"wal-append.queue_time_us":870943}
W20260709 13:09:25.096765 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:25.117383 22951 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 578) took 1810 ms. Trace:
W20260709 13:09:25.149482 22920 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.000s	sys 0.004s
W20260709 13:09:25.149699 26236 consensus_meta.cc:287] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 1.171s	user 0.010s	sys 0.017s
I20260709 13:09:25.150380 26236 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:25.153759 26669 log.cc:927] Time spent T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.157s	user 0.000s	sys 0.000s
W20260709 13:09:25.156734 22952 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.003s	sys 0.011s
W20260709 13:09:25.157266 22952 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.757s	user 0.003s	sys 0.011s
I20260709 13:09:25.158360 22952 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2493) took 1803 ms. Trace:
W20260709 13:09:25.149955 22920 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.754s	user 0.000s	sys 0.004s
W20260709 13:09:25.118499 26440 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.704s	user 0.006s	sys 0.005s
W20260709 13:09:25.164901 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 117745us
W20260709 13:09:25.170709 26664 log.cc:927] Time spent T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.189s	user 0.001s	sys 0.001s
I20260709 13:09:25.170249 26549 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:25.159057 22952 rpcz_store.cc:276] 0709 13:09:23.355268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:23.355778 (+   510us) service_pool.cc:224] Handling call
0709 13:09:23.356061 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:23.356261 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:09:24.044751 (+688490us) log.cc:844] Serialized 14 byte log entry
0709 13:09:25.157721 (+1112970us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:25.157824 (+   103us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:09:25.157959 (+   135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:25.158073 (+   114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:25.158289 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4072,"prepare.queue_time_us":1005782,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":89338,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":8285,"thread_start_us":1773070,"threads_started":3,"wal-append.queue_time_us":456395}
W20260709 13:09:25.180152 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:25.183882 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 159593us
W20260709 13:09:25.197184 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 162080us
W20260709 13:09:25.181468 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 785860us
I20260709 13:09:25.117993 23621 rpcz_store.cc:276] 0709 13:09:23.542381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:23.646770 (+104389us) service_pool.cc:224] Handling call
0709 13:09:23.647038 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:23.647245 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:09:24.327971 (+680726us) log.cc:844] Serialized 14 byte log entry
0709 13:09:25.041546 (+713575us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:25.041665 (+   119us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:25.041806 (+   141us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:25.041940 (+   134us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:25.042117 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":828224,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":1834,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":152,"thread_start_us":1381924,"threads_started":2}
W20260709 13:09:25.215657 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:25.160813 26454 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:25.223054 26549 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:25.224361 26492 raft_consensus.cc:697] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:25.225935 26492 consensus_queue.cc:237] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:25.227313 26576 consensus_meta.cc:287] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 1.207s	user 0.002s	sys 0.014s
I20260709 13:09:25.228845 26685 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:25.229151 22993 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:25.229833 26685 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:25.179821 26619 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:25.233167 26401 raft_consensus.cc:2749] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 13:09:25.234805 26381 consensus_meta.cc:287] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 1.254s	user 0.013s	sys 0.008s
W20260709 13:09:25.235312 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:25.235455 26502 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:25.238881 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:25.239936 23020 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 10.
I20260709 13:09:25.240005 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:25.241066 23022 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 9.
W20260709 13:09:25.242095 22919 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.016s	sys 0.028s
I20260709 13:09:25.197316 26398 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:25.242535 22919 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.828s	user 0.017s	sys 0.029s
I20260709 13:09:25.242648 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:25.243008 26398 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:25.243814 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 876091us
I20260709 13:09:25.179821 26531 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:25.247080 22844 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:25.247504 26531 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:25.248729 22995 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.681s	user 0.030s	sys 0.020s
W20260709 13:09:25.249159 22995 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.932s	user 0.030s	sys 0.020s
I20260709 13:09:25.249257 23028 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 609) took 1857 ms. Trace:
I20260709 13:09:25.250612 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:25.249699 23028 rpcz_store.cc:276] 0709 13:09:23.391484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:23.392859 (+  1375us) service_pool.cc:224] Handling call
0709 13:09:23.393244 (+   385us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:24.033749 (+640505us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:24.033817 (+    68us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:24.038447 (+  4630us) log.cc:844] Serialized 26 byte log entry
0709 13:09:24.424107 (+385660us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:09:24.424140 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:24.424207 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:25.248732 (+824525us) raft_consensus.cc:1685] finished
0709 13:09:25.248796 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:25.249043 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":39210,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":10435,"spinlock_wait_cycles":40064,"thread_start_us":925728,"threads_started":2,"wal-append.queue_time_us":1067631}
I20260709 13:09:25.228922 26425 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:25.251760 23027 raft_consensus.cc:2468] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 7.
W20260709 13:09:25.253293 26496 consensus_meta.cc:287] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.867s	user 0.010s	sys 0.005s
I20260709 13:09:25.254388 26695 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:25.255265 26695 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:25.256989 26708 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:25.257264 26236 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:25.248428 22844 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
W20260709 13:09:25.261782 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 79094.8us
I20260709 13:09:25.123209 22951 rpcz_store.cc:276] 0709 13:09:23.300087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:23.300189 (+   102us) service_pool.cc:224] Handling call
0709 13:09:23.328941 (+ 28752us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:23.329254 (+   313us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:23.329285 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:23.335786 (+  6501us) log.cc:844] Serialized 26 byte log entry
0709 13:09:24.045726 (+709940us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:09:24.045781 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:24.046017 (+   236us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:24.553688 (+507671us) spinlock_profiling.cc:243] Waited 90.7 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:09:25.048475 (+494787us) raft_consensus.cc:1685] finished
0709 13:09:25.048511 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:25.110874 (+ 62363us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7350,"raft.queue_time_us":947,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":15458,"spinlock_wait_cycles":208694528,"thread_start_us":675500,"threads_started":1,"wal-append.queue_time_us":975251}
W20260709 13:09:25.218001 23025 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.004s	sys 0.015s
I20260709 13:09:25.257889 26708 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:25.153786 26407 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
W20260709 13:09:25.438321 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:25.415333 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:25.415108 23021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 603) took 2280 ms. Trace:
W20260709 13:09:25.442610 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 99524us
I20260709 13:09:25.307109 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:25.296334 23025 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.934s	user 0.004s	sys 0.017s
I20260709 13:09:25.357786 23616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 758) took 1817 ms. Trace:
W20260709 13:09:25.407320 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 152453us
I20260709 13:09:25.232501 26685 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:25.349800 22948 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58310 (request call id 1854) took 1761 ms (client timeout 2106 ms). Trace:
I20260709 13:09:25.284590 26398 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:25.466991 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 218729us
I20260709 13:09:25.352751 26605 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:25.454813 26612 consensus_meta.cc:287] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 1.659s	user 0.017s	sys 0.006s
I20260709 13:09:25.335142 26401 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:25.454241 26407 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:25.478107 22947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 584) took 1115 ms. Trace:
I20260709 13:09:25.417865 26549 raft_consensus.cc:2804] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:09:25.464571 22948 rpcz_store.cc:269] 0709 13:09:23.587907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:23.588017 (+   110us) service_pool.cc:224] Handling call
0709 13:09:25.349265 (+1761248us) spinlock_profiling.cc:243] Waited 519 ms on lock 0x7b54000a96d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:25.349545 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1194344320}
I20260709 13:09:25.342476 26635 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:25.342132 26708 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:25.444211 26594 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:25.235375 26381 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:25.335467 26392 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:25.355113 26568 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:25.335431 26676 raft_consensus.cc:697] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:25.465618 26425 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:25.520046 26425 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:25.488466 26398 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:25.522959 26688 log.cc:927] Time spent T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.281s	user 0.001s	sys 0.001s
I20260709 13:09:25.522680 26635 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:25.523151 26398 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:25.523636 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:25.339617 26472 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:25.450548 23021 rpcz_store.cc:276] 0709 13:09:23.134371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:23.139325 (+  4954us) service_pool.cc:224] Handling call
0709 13:09:23.160039 (+ 20714us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:23.160375 (+   336us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:23.160403 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:23.160850 (+   447us) log.cc:844] Serialized 26 byte log entry
0709 13:09:23.949224 (+788374us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:09:23.949266 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:23.949328 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:25.359024 (+1409696us) spinlock_profiling.cc:243] Waited 37.3 ms on lock 0x7b240025ad38. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132b7e9c4 00007f6132b7cacb 00007f6132d37dc9 00007f6132d37a55 00007f6132d36fb0 00007f61359ad488 00007f61359adbe8 00007f6135985398 00007f613598076a 00007f613597fbb6
0709 13:09:25.359933 (+   909us) raft_consensus.cc:1685] finished
0709 13:09:25.359962 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:25.414834 (+ 54872us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":598,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":104,"spinlock_wait_cycles":85710080,"thread_start_us":788017,"threads_started":1,"wal-append.queue_time_us":1028053}
I20260709 13:09:25.525811 26398 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:25.522488 23025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 608) took 2223 ms. Trace:
I20260709 13:09:25.526623 22949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2498) took 2164 ms. Trace:
I20260709 13:09:25.516578 26626 raft_consensus.cc:3055] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:09:25.528877 26703 consensus_queue.cc:1059] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:09:25.529165 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:25.529570 26626 raft_consensus.cc:740] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:25.529843 26703 consensus_queue.cc:1243] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer aed3f089ec8b49af9c8cf5cdee76e8e7 log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:09:25.529057 26667 consensus_queue.cc:1048] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 13:09:25.483836 26695 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:25.527026 22949 rpcz_store.cc:276] 0709 13:09:23.361881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:23.362015 (+   134us) service_pool.cc:224] Handling call
0709 13:09:23.362332 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:23.506304 (+143972us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:09:24.286135 (+779831us) log.cc:844] Serialized 14 byte log entry
0709 13:09:25.026929 (+740794us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:25.027081 (+   152us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:09:25.027235 (+   154us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:25.526335 (+499100us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b54000749d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:25.526376 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:25.526555 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":910904,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":23087,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":128,"spinlock_wait_cycles":330594688,"thread_start_us":1519763,"threads_started":2,"wal-append.queue_time_us":959707}
W20260709 13:09:25.544206 26647 consensus_meta.cc:287] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 1.552s	user 0.005s	sys 0.015s
I20260709 13:09:25.544703 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:25.544951 26544 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:25.544898 26647 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:25.545782 26544 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:25.546906 22880 raft_consensus.cc:2393] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 10: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:25.547654 26647 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:25.548640 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 46915.9us
I20260709 13:09:25.549504 22920 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:25.486994 22947 rpcz_store.cc:276] 0709 13:09:24.323297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:24.323684 (+   387us) service_pool.cc:224] Handling call
0709 13:09:24.324025 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:24.421771 (+ 97746us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:24.421811 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:24.421868 (+    57us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:09:24.421912 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:25.438264 (+1016352us) spinlock_profiling.cc:243] Waited 97.5 ms on lock 0x7b540023d7d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:25.438350 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:25.438609 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":224289536}
I20260709 13:09:25.551092 22920 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:25.551733 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:09:25.552812 22880 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35360 (request call id 1872) took 2157 ms (client timeout 2722 ms). Trace:
I20260709 13:09:25.504112 26389 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:25.504357 26454 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:09:25.504443 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 209481us
I20260709 13:09:25.504804 26648 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:25.530045 23027 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 11.
I20260709 13:09:25.560928 26648 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:25.561515 26648 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:25.559734 26389 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:25.505149 26401 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:25.506959 26534 raft_consensus.cc:3055] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:09:25.567180 26534 raft_consensus.cc:740] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:25.567236 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:25.508876 26608 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:25.568182 22874 raft_consensus.cc:2468] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 5.
I20260709 13:09:25.568322 26389 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:25.568816 26401 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:25.559921 26454 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:25.509192 23619 raft_consensus.cc:2410] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:09:25.509337 26589 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:25.509635 26392 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:25.573145 26454 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:25.510567 26531 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:25.512127 26686 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:09:25.517614 26509 consensus_meta.cc:287] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.640s	user 0.008s	sys 0.008s
I20260709 13:09:25.522464 26425 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:09:25.485050 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:25.579054 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 302160us
I20260709 13:09:25.577037 26509 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:25.340729 26482 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:25.584137 26706 raft_consensus.cc:2804] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:25.584743 26706 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:25.585163 26706 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:25.585618 26509 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:25.587635 26531 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:25.588388 26531 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:25.590137 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 53297.9us
I20260709 13:09:25.336782 26670 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:25.530886 26626 consensus_queue.cc:260] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:25.540712 26675 consensus_meta.cc:287] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.518s	user 0.009s	sys 0.005s
I20260709 13:09:25.540902 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:25.549575 22951 raft_consensus.cc:1240] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 6. Current term is 7. Ops: []
I20260709 13:09:25.551914 26347 raft_consensus.cc:2804] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:25.347056 26498 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:25.553725 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:25.551774 26576 consensus_queue.cc:1048] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:09:25.555083 22955 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 588) took 1188 ms. Trace:
I20260709 13:09:25.556874 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:25.558156 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:25.494302 26549 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:25.334211 26683 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:25.454650 23616 rpcz_store.cc:276] 0709 13:09:23.540864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:23.540977 (+   113us) service_pool.cc:224] Handling call
0709 13:09:23.566132 (+ 25155us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:23.587593 (+ 21461us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:09:24.267489 (+679896us) log.cc:844] Serialized 14 byte log entry
0709 13:09:25.295692 (+1028203us) log.cc:844] Serialized 14 byte log entry
0709 13:09:25.332998 (+ 37306us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:25.333657 (+   659us) log.cc:844] Serialized 26 byte log entry
0709 13:09:25.333780 (+   123us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:09:25.333816 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:25.333933 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:25.357540 (+ 23607us) raft_consensus.cc:1685] finished
0709 13:09:25.357563 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:25.357724 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64536,"prepare.queue_time_us":864824,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":60275,"raft.run_cpu_time_us":479,"raft.run_wall_time_us":14526,"thread_start_us":1593550,"threads_started":3,"wal-append.queue_time_us":1077082}
I20260709 13:09:25.574235 26349 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:25.597499 26347 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:25.603211 26349 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:25.603892 26347 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:25.604041 26498 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:25.605684 26626 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:25.606284 26349 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:25.607842 26706 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:25.608937 26236 consensus_queue.cc:1048] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:09:25.610217 22919 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:25.611387 26236 raft_consensus.cc:2804] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:25.611833 26236 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:25.548523 26544 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:25.575556 26392 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:25.576246 26686 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:25.559594 22880 rpcz_store.cc:269] 0709 13:09:23.395146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:23.443242 (+ 48096us) service_pool.cc:224] Handling call
0709 13:09:25.552573 (+2109331us) spinlock_profiling.cc:243] Waited 2.1 s on lock 0x7b540030c0d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:25.552743 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4835354496}
I20260709 13:09:25.597229 26618 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:25.602325 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:25.609123 22879 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 10.
W20260709 13:09:25.573925 23619 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60044 (request call id 708) took 2031 ms (client timeout 1714 ms). Trace:
I20260709 13:09:25.568343 26534 consensus_queue.cc:260] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:25.526852 23025 rpcz_store.cc:276] 0709 13:09:23.298852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:23.298986 (+   134us) service_pool.cc:224] Handling call
0709 13:09:23.299292 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:23.324732 (+ 25440us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:23.324764 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:23.325361 (+   597us) log.cc:844] Serialized 26 byte log entry
0709 13:09:24.017169 (+691808us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:09:24.361520 (+344351us) log.cc:844] Serialized 14 byte log entry
0709 13:09:24.556129 (+194609us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:09:25.501673 (+945544us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:25.522108 (+ 20435us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:25.522168 (+    60us) raft_consensus.cc:1685] finished
0709 13:09:25.522268 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:25.522427 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30501,"prepare.queue_time_us":170331,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":50,"raft.queue_time_us":818583,"raft.run_cpu_time_us":233,"raft.run_wall_time_us":7687,"spinlock_wait_cycles":238474112,"thread_start_us":2068923,"threads_started":3,"wal-append.queue_time_us":952100}
W20260709 13:09:25.626240 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:25.625963 22948 raft_consensus.cc:1240] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:09:25.630344 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:25.598837 26482 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:25.628803 23619 rpcz_store.cc:269] 0709 13:09:23.542658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:23.584744 (+ 42086us) service_pool.cc:224] Handling call
0709 13:09:25.573653 (+1988909us) spinlock_profiling.cc:243] Waited 1.91 s on lock 0x7b54002bc854. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:25.573826 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4395887488}
I20260709 13:09:25.613233 26549 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:25.637527 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:25.641512 22951 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:25.612206 26236 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:25.598990 22955 rpcz_store.cc:276] 0709 13:09:24.366740 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:24.388234 (+ 21494us) service_pool.cc:224] Handling call
0709 13:09:24.388515 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:25.016600 (+628085us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:09:25.207074 (+190474us) log.cc:844] Serialized 14 byte log entry
0709 13:09:25.466811 (+259737us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:25.466872 (+    61us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:09:25.483352 (+ 16480us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:25.554629 (+ 71277us) spinlock_profiling.cc:243] Waited 627 ms on lock 0x7b5400379bd4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:25.554673 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:25.554912 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35400,"prepare.queue_time_us":319516,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":35303,"raft.run_cpu_time_us":457,"raft.run_wall_time_us":21204,"spinlock_wait_cycles":1442384768,"thread_start_us":315367,"threads_started":2}
I20260709 13:09:25.646916 23623 raft_consensus.cc:2410] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:09:25.649667 22947 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
I20260709 13:09:25.650928 23028 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 7.
W20260709 13:09:25.646576 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 223063us
I20260709 13:09:25.612993 26509 consensus_queue.cc:1059] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.032s
I20260709 13:09:25.599741 26675 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:25.529999 26676 consensus_queue.cc:237] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:25.685892 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:25.692111 22872 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 10.
I20260709 13:09:25.697414 26482 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:25.685892 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:25.644052 26683 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:25.726302 22951 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 10.
W20260709 13:09:25.736614 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 76866.4us
W20260709 13:09:25.745097 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 91876.3us
I20260709 13:09:25.750017 22950 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:09:25.754120 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:25.721495 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:25.739396 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 51212us
I20260709 13:09:25.755434 22947 raft_consensus.cc:2393] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 9: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:25.755848 26482 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:25.760978 22874 raft_consensus.cc:2393] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 10: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:25.698567 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:25.761150 23590 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:25.790902 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:09:25.701575 26639 consensus_meta.cc:287] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 1.485s	user 0.011s	sys 0.011s
I20260709 13:09:25.657061 26498 consensus_queue.cc:1059] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.137s
I20260709 13:09:25.802088 26458 raft_consensus.cc:2804] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 13:09:25.802383 26589 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:25.802667 26458 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:25.803200 26458 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:25.803076 26589 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:25.720217 26708 consensus_queue.cc:1048] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.084s
I20260709 13:09:25.805004 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:25.800994 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 46464.1us
I20260709 13:09:25.805847 22955 raft_consensus.cc:2393] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 11: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:25.806663 26506 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:25.704082 26695 raft_consensus.cc:3055] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:09:25.807262 26695 raft_consensus.cc:740] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:25.807299 26506 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:25.640388 26670 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:25.808591 26695 consensus_queue.cc:260] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:25.810137 26695 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:25.810335 26670 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:25.813611 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:25.814445 23028 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:25.817762 26612 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:25.818396 26612 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:25.815212 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:25.815407 23022 raft_consensus.cc:1275] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:09:25.815680 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:25.815966 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:25.806561 23590 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:25.761523 26347 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:25.816382 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:25.817270 26639 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:25.816188 26706 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:25.857563 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 64410.1us
I20260709 13:09:25.784816 26686 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:25.654636 26618 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:25.875537 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:25.736991 26349 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:25.898746 22952 raft_consensus.cc:1240] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:25.702718 26381 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:25.761926 26531 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:25.863699 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:25.789017 26635 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:25.657428 26602 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:25.857939 26667 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:25.872084 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:25.873030 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:25.857146 26506 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:25.873493 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:25.913776 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:09:25.874795 26723 log.cc:927] Time spent T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 13:09:25.805047 26683 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:25.902709 22951 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:25.903569 23029 raft_consensus.cc:2393] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 11: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:25.685984 26551 consensus_queue.cc:1048] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.243s
W20260709 13:09:25.877252 22919 connection.cc:256] RPC call timeout handler was delayed by -1.2467460820000724s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1714s for a response
I20260709 13:09:25.875576 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:25.665273 26425 consensus_queue.cc:1059] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.213s
I20260709 13:09:25.905205 26347 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:25.902640 23020 raft_consensus.cc:2468] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 5.
W20260709 13:09:25.939848 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:25.940440 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:25.906818 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 100342us
I20260709 13:09:25.873950 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:25.915490 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:25.916600 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:25.917481 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:25.917423 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:25.921726 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:25.922434 22953 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
I20260709 13:09:25.925933 26667 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:25.927224 22992 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:25.905325 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:25.875540 26589 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:25.934095 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:25.934945 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:25.943542 22871 raft_consensus.cc:2393] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 9: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:25.945118 22950 raft_consensus.cc:2410] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:09:25.928745 26639 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:25.948063 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:25.943480 23024 raft_consensus.cc:2468] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 7.
I20260709 13:09:25.949615 22946 raft_consensus.cc:2393] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 10: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:25.911937 26577 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:25.827678 26648 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:25.912734 26612 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:25.949930 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:25.956782 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:25.956893 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:25.960140 22948 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
I20260709 13:09:25.971728 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:25.970799 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 43612.6us
I20260709 13:09:25.974092 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:25.976068 23021 raft_consensus.cc:2410] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:09:25.973873 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:25.974754 23025 raft_consensus.cc:2468] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 8.
I20260709 13:09:25.977607 23024 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 10.
I20260709 13:09:25.913904 26349 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:25.949594 26676 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:25.981937 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:25.983158 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:25.957301 26577 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:25.982971 26675 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:25.986591 23026 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 10.
I20260709 13:09:25.978245 23022 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 10.
I20260709 13:09:25.988337 22955 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 8.
I20260709 13:09:25.995520 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:25.990481 22992 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:25.998961 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:25.999663 23625 raft_consensus.cc:2393] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 10: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:25.997030 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:26.005812 23025 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 9.
I20260709 13:09:26.008607 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:26.010779 23020 raft_consensus.cc:2393] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 8: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:26.012857 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:26.014866 26676 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:09:26.015380 26458 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:26.015646 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:26.016615 26602 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:26.016983 26425 raft_consensus.cc:2804] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:26.017460 26425 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:26.017841 26425 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:26.019022 26667 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:26.020164 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:26.023097 23024 raft_consensus.cc:2410] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 10 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:09:26.025449 22951 raft_consensus.cc:2410] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:09:26.027117 23028 raft_consensus.cc:2410] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:09:26.024237 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:26.027829 23021 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 8.
I20260709 13:09:25.996366 26577 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:26.034708 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:26.017478 26676 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:26.032928 26392 raft_consensus.cc:2804] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:09:26.055199 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:26.056483 23625 raft_consensus.cc:1275] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:09:26.062170 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:26.064356 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:26.069262 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:26.070721 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:26.072830 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:26.074167 22876 raft_consensus.cc:1275] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 13:09:26.075598 26407 consensus_meta.cc:287] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.550s	user 0.002s	sys 0.010s
I20260709 13:09:26.076089 26407 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:26.077896 26401 raft_consensus.cc:3055] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 13:09:26.078361 26407 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:26.078734 26236 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:26.079838 22952 raft_consensus.cc:1240] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:09:26.079922 26639 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:26.048008 26347 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:26.085188 26347 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:26.086902 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:26.087513 26347 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:26.047729 26551 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:26.088384 26551 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:26.089425 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:26.090598 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:26.090847 26549 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:26.090773 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:26.091246 22946 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 10.
I20260709 13:09:26.093511 22992 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:26.101940 22992 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:26.103071 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:26.102833 23620 raft_consensus.cc:1275] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260709 13:09:26.104004 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:26.104123 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:26.106779 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:26.104240 23620 pending_rounds.cc:85] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2: Aborting all ops after (but not including) 6
I20260709 13:09:26.078229 26458 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:26.110709 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:26.119583 23021 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 6.
I20260709 13:09:26.120203 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:26.122151 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:26.108706 23620 pending_rounds.cc:107] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 13:09:26.127633 23620 raft_consensus.cc:2889] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:26.128706 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:26.129586 22951 raft_consensus.cc:2393] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 11: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:26.078370 26401 raft_consensus.cc:740] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:26.093083 26392 raft_consensus.cc:697] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:26.133577 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:26.135120 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:26.143657 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:26.144011 22947 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:09:26.136010 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:26.141636 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:26.108644 26667 raft_consensus.cc:2804] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:26.142231 23624 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:09:26.147346 26667 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:26.142951 23024 raft_consensus.cc:2393] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 9: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:26.148522 22946 raft_consensus.cc:2393] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 11: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:26.149528 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:26.150930 22872 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 10.
I20260709 13:09:26.151160 26590 raft_consensus.cc:2804] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:26.151633 26590 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:26.152051 26590 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:26.152760 26549 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:26.153613 26618 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:26.101069 26407 consensus_queue.cc:1059] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.054s
I20260709 13:09:26.156119 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:26.156945 23021 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 6.
I20260709 13:09:26.147818 26667 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:26.159502 23023 raft_consensus.cc:2410] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:09:26.161922 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:26.131258 26699 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:26.163280 23624 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11.
I20260709 13:09:26.172178 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:26.172976 22950 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 10.
I20260709 13:09:26.175616 26407 raft_consensus.cc:3055] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:09:26.176230 26407 raft_consensus.cc:740] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:26.177228 22871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53724 (request call id 602) took 1204 ms. Trace:
I20260709 13:09:26.177640 22871 rpcz_store.cc:276] 0709 13:09:24.979322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:24.996047 (+ 16725us) service_pool.cc:224] Handling call
0709 13:09:26.177013 (+1180966us) spinlock_profiling.cc:243] Waited 850 ms on lock 0x7b54003c64d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:26.177148 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1954155264}
I20260709 13:09:26.186088 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:26.186499 22951 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
I20260709 13:09:26.187702 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:26.188412 23021 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
I20260709 13:09:26.177889 26551 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:26.195721 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:26.198141 23590 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:26.204102 26612 consensus_queue.cc:1048] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 13:09:26.206586 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:26.204537 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:26.210685 23620 raft_consensus.cc:2368] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 9. Current term is 10.
I20260709 13:09:26.213680 22992 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:26.214548 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:26.217120 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:26.163134 26401 consensus_queue.cc:260] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:26.177681 26699 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:09:26.226421 22992 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:26.204694 26458 raft_consensus.cc:2804] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:26.177260 26407 consensus_queue.cc:260] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:26.208751 26236 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:26.252125 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:26.244009 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:26.247501 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:26.253367 22875 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:26.255048 23617 raft_consensus.cc:1275] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:09:26.255560 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:26.256722 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:26.256819 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:26.257280 23028 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 9.
I20260709 13:09:26.257578 23022 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:26.258191 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:26.258991 22872 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 10.
I20260709 13:09:26.246819 26612 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:26.260015 26612 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:26.261456 26730 log.cc:927] Time spent T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:09:26.228545 26626 raft_consensus.cc:2804] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:26.264708 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:26.264997 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:26.262765 26612 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:26.273310 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:26.281514 26401 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:26.213631 26506 consensus_queue.cc:1048] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 13:09:26.294565 26506 consensus_queue.cc:1048] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:09:26.252368 26392 consensus_queue.cc:237] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:26.297699 26647 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:26.298398 26647 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:26.243230 26458 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:26.247092 26708 raft_consensus.cc:2804] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:26.326392 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:26.270905 26626 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:26.264062 26590 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:26.354725 22947 raft_consensus.cc:1275] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:09:26.317996 26458 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:26.356557 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:26.360949 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:26.362089 22948 raft_consensus.cc:2410] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:09:26.319072 26708 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:26.356925 26626 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:26.375022 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:26.375749 23027 raft_consensus.cc:3060] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:09:26.389683 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:26.369910 26708 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:26.392076 22948 raft_consensus.cc:2393] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 9: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:26.398986 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:26.395110 26534 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:09:26.402017 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 130921us
I20260709 13:09:26.402731 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:26.406955 22954 raft_consensus.cc:2410] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:09:26.393913 26236 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:26.414381 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 241983us
I20260709 13:09:26.421006 23022 raft_consensus.cc:2468] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6.
I20260709 13:09:26.422160 26347 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:26.422796 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:26.422808 26347 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:26.425262 26347 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:26.432075 26699 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:26.383124 26577 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:26.444422 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:26.446271 22954 raft_consensus.cc:2393] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 7: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
W20260709 13:09:26.446902 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 192867us
W20260709 13:09:26.449425 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:26.451792 23027 raft_consensus.cc:2468] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 9.
I20260709 13:09:26.457685 23622 raft_consensus.cc:2368] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 11. Current term is 12.
I20260709 13:09:26.458237 22948 raft_consensus.cc:1275] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:09:26.461484 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:26.421608 26236 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:26.472885 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:26.473435 22948 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
W20260709 13:09:26.475194 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:26.477093 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:26.480059 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:26.480111 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:26.480849 22874 raft_consensus.cc:2393] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 7: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:26.482314 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:26.482574 22874 raft_consensus.cc:1275] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:09:26.489691 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:26.490352 23027 raft_consensus.cc:2410] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
W20260709 13:09:26.491966 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:26.494419 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:26.495536 22948 raft_consensus.cc:2393] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 11: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
W20260709 13:09:26.496414 23588 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:26.498654 26577 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:26.509308 22880 raft_consensus.cc:1275] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:09:26.510756 26743 consensus_queue.cc:1048] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:09:26.515807 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:26.518247 23027 raft_consensus.cc:2376] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 9. Re-sending same reply.
I20260709 13:09:26.519512 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:26.521132 23588 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:26.523725 22843 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:26.538041 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:26.536003 22872 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
I20260709 13:09:26.537464 26551 raft_consensus.cc:2749] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:26.516858 26743 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:26.548941 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:26.549965 22880 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 9.
I20260709 13:09:26.552495 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:26.554975 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:26.560019 22872 raft_consensus.cc:2393] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 11: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:26.551471 26626 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:26.550613 26602 raft_consensus.cc:2804] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:26.580603 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:09:26.563068 26425 consensus_meta.cc:287] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.545s	user 0.011s	sys 0.001s
I20260709 13:09:26.585434 22843 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:26.590329 22880 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:26.594117 22877 raft_consensus.cc:1240] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:09:26.596259 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:26.588374 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:26.597994 22872 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:26.571913 26602 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:26.607339 22919 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:26.607990 26236 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:26.608422 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:26.610634 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:26.612850 26347 consensus_queue.cc:1048] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:09:26.614281 26626 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:26.632037 22919 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:26.645867 22919 leader_election.cc:400] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 9. Current term is 10.
I20260709 13:09:26.646200 22919 leader_election.cc:403] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:09:26.645975 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:26.646584 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:26.647120 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:26.647434 23024 raft_consensus.cc:2410] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:09:26.581717 26743 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:26.638186 26602 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:26.643035 26618 raft_consensus.cc:2804] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:26.663466 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:26.593593 26425 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:26.671456 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:26.669389 22879 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
I20260709 13:09:26.674644 23588 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:26.675026 23022 raft_consensus.cc:2393] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 11: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:26.678494 23023 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:26.654475 26236 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:26.680612 22875 raft_consensus.cc:2468] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6.
I20260709 13:09:26.682797 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:26.683670 22879 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11.
I20260709 13:09:26.684420 23025 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 10.
I20260709 13:09:26.668999 26618 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:26.679291 26735 log.cc:927] Time spent T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.485s	user 0.000s	sys 0.005s
I20260709 13:09:26.693991 22875 raft_consensus.cc:1240] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
W20260709 13:09:26.697674 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:26.698225 23027 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 6.
I20260709 13:09:26.724316 22947 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
I20260709 13:09:26.681833 26236 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:26.693888 26551 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:26.694140 26618 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:26.742030 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:26.744580 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:26.739776 23588 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:26.746825 22879 raft_consensus.cc:2393] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 6: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:26.747546 22878 raft_consensus.cc:2393] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 10: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
W20260709 13:09:26.748013 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 239043us
W20260709 13:09:26.748867 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 137826us
I20260709 13:09:26.736248 26236 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:09:26.751955 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 106280us
I20260709 13:09:26.743984 26347 raft_consensus.cc:2749] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 9. Current term is 10.
I20260709 13:09:26.741986 26551 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:09:26.750362 26736 log.cc:927] Time spent T 00000000000000000000000000000000 P 8d175b4ae72f4fc48971c2f31668971c: Append to log took a long time: real 0.313s	user 0.002s	sys 0.002s
I20260709 13:09:26.774151 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
W20260709 13:09:26.774590 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 189273us
I20260709 13:09:26.774013 22992 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:26.775949 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:26.773661 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:26.777746 22875 raft_consensus.cc:2368] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 8. Current term is 9.
I20260709 13:09:26.773242 26551 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:26.791405 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:26.795466 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:26.783847 26667 consensus_meta.cc:287] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.603s	user 0.007s	sys 0.005s
W20260709 13:09:26.801046 26747 log.cc:927] Time spent T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.058s	user 0.001s	sys 0.000s
I20260709 13:09:26.806383 23023 raft_consensus.cc:2410] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:09:26.806952 26756 consensus_queue.cc:1059] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:09:26.806615 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:26.808482 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:26.816406 23027 raft_consensus.cc:2410] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:09:26.809080 23022 raft_consensus.cc:1240] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:09:26.822685 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:26.824779 23028 raft_consensus.cc:1240] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:26.830595 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:26.825829 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:26.810365 26392 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:26.842314 22879 raft_consensus.cc:2393] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 8: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:26.784466 26708 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:26.846257 23024 raft_consensus.cc:1275] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:09:26.845336 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:26.845644 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:26.850728 26746 log.cc:927] Time spent T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:09:26.852157 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:26.852897 22877 raft_consensus.cc:2393] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 6: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:26.854126 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:26.854251 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:26.854449 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:26.855033 23022 raft_consensus.cc:2393] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 11: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:26.854132 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:26.855051 22878 raft_consensus.cc:2393] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 11: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
W20260709 13:09:26.844866 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 66647.8us
I20260709 13:09:26.853642 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:26.858865 22877 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:09:26.856853 26758 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:26.857398 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:26.794893 26458 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:26.844013 26392 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:26.864246 22871 raft_consensus.cc:2368] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 9. Current term is 10.
I20260709 13:09:26.819203 26667 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:26.865132 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:26.865301 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:26.865520 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:26.866226 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 89922.1us
I20260709 13:09:26.870332 22879 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:09:26.872324 26748 log.cc:927] Time spent T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:09:26.856068 26741 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:26.873129 26741 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:26.869755 26392 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:26.874658 26458 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:26.870832 26764 raft_consensus.cc:2804] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:26.876253 26764 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:26.876610 26764 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:26.876832 23619 raft_consensus.cc:2368] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 10. Current term is 11.
I20260709 13:09:26.877532 23621 raft_consensus.cc:2410] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 11 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:09:26.878046 26667 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:26.881918 26590 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:26.882071 26708 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:26.884889 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:26.886267 23028 raft_consensus.cc:1240] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:26.887290 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:26.871377 22992 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:26.887133 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:26.896627 22880 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
I20260709 13:09:26.905620 23622 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 768) took 1087 ms. Trace:
I20260709 13:09:26.906387 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:26.906944 22955 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:26.907020 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:26.908633 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:26.904064 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:26.870416 26743 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:26.910001 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:26.915337 23028 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 9.
I20260709 13:09:26.906008 23622 rpcz_store.cc:276] 0709 13:09:25.822048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:25.873186 (+ 51138us) service_pool.cc:224] Handling call
0709 13:09:26.883928 (+1010742us) spinlock_profiling.cc:243] Waited 538 ms on lock 0x7b540045d654. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:26.898979 (+ 15051us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1238459264}
I20260709 13:09:26.913339 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:26.913360 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:26.918948 23020 raft_consensus.cc:2393] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 10: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:26.917395 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:26.922820 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:26.923595 23022 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:09:26.924759 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:26.925839 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:26.927243 22876 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
I20260709 13:09:26.902789 26708 raft_consensus.cc:2804] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:09:26.932973 22871 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
I20260709 13:09:26.902837 26755 raft_consensus.cc:2804] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:09:26.937716 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:26.945406 22948 raft_consensus.cc:1275] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:09:26.945856 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:26.946871 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 84459.2us
I20260709 13:09:26.947764 22843 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:26.948935 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:26.949523 22880 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:26.954396 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:26.957285 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:26.957538 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:26.958003 22946 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:26.958278 23020 raft_consensus.cc:2393] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 11: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:26.959110 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:26.960080 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:26.961139 26648 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:26.961768 26648 consensus_queue.cc:1048] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:09:26.965416 26612 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:26.965744 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:26.966425 22871 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:26.966102 26612 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:26.867251 26758 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:26.970427 26758 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:26.974468 23590 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:26.975888 23021 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
I20260709 13:09:26.977507 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:26.981383 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:26.983382 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:26.984459 23020 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:26.985426 23022 raft_consensus.cc:1275] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 13:09:26.986231 22878 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 9.
I20260709 13:09:26.990736 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:26.991535 22947 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:26.997035 22874 raft_consensus.cc:2393] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 9: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:27.000077 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:27.003893 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:27.006165 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:27.004763 23623 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:09:26.953649 26755 raft_consensus.cc:697] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 10 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
W20260709 13:09:27.014103 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:27.017342 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 45722.4us
I20260709 13:09:27.012408 22920 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:27.024855 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:27.027882 23021 raft_consensus.cc:2393] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 6: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:27.008520 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:27.025990 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:09:27.035693 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:27.045590 22878 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:09:27.054962 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 150204us
I20260709 13:09:27.056478 23588 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:27.059885 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:27.063791 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:27.016063 26708 raft_consensus.cc:697] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:09:27.068369 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 56065.8us
I20260709 13:09:27.070055 22920 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:27.072643 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:27.057621 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 109945us
I20260709 13:09:27.073746 26753 raft_consensus.cc:2804] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:27.076802 26766 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:27.078377 26708 consensus_queue.cc:237] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:27.022567 26759 consensus_queue.cc:1048] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.063s
I20260709 13:09:27.124907 26759 raft_consensus.cc:2804] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:27.125272 26759 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:27.125574 26759 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:27.126576 22947 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:09:27.018093 26758 consensus_queue.cc:1048] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.050s
W20260709 13:09:27.082351 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:27.065410 26612 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:27.083709 26602 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:27.087159 26764 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:27.088120 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:27.089496 26755 consensus_queue.cc:237] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:27.093954 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:27.095387 26769 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:27.096020 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:27.097126 26770 raft_consensus.cc:2804] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:09:27.110896 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 54509.9us
I20260709 13:09:27.114866 22946 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:09:27.128118 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:27.138908 22879 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
I20260709 13:09:27.139952 22878 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11.
I20260709 13:09:27.140595 22872 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 9.
I20260709 13:09:27.119990 26753 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:27.132334 22843 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:27.147984 22877 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 12.
I20260709 13:09:27.148147 22955 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11.
I20260709 13:09:27.056953 22880 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 11.
W20260709 13:09:27.156569 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:27.150686 23023 raft_consensus.cc:2393] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 11: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:27.142817 26618 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:27.166325 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:27.166275 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:27.167441 23617 raft_consensus.cc:1240] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 7. Current term is 8. Ops: []
I20260709 13:09:27.147011 26612 raft_consensus.cc:2804] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:09:27.173718 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 103779us
I20260709 13:09:27.175719 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:27.176062 22920 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:27.168350 22878 raft_consensus.cc:2393] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 9: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:27.168932 22879 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 11.
I20260709 13:09:27.168108 22873 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 9.
I20260709 13:09:27.130371 26766 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:27.165064 22871 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11.
I20260709 13:09:27.178107 23022 raft_consensus.cc:2410] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 13:09:27.182400 22946 raft_consensus.cc:1240] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 13:09:27.152727 26770 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:27.191486 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:27.192312 22877 raft_consensus.cc:2393] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 11: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:27.192206 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:27.192922 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:27.193081 22874 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
I20260709 13:09:27.193709 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:27.194314 22880 raft_consensus.cc:2376] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11. Re-sending same reply.
I20260709 13:09:27.190778 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:27.194425 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:27.195259 23023 raft_consensus.cc:2393] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 11: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:27.195585 22879 raft_consensus.cc:2410] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:09:27.196455 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:27.150772 26767 raft_consensus.cc:2764] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 13:09:27.166224 26753 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:27.200237 22872 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
I20260709 13:09:27.203279 23020 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
I20260709 13:09:27.203630 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:27.204253 22873 raft_consensus.cc:2393] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 9: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:27.173472 26612 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:27.204855 22843 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:27.208309 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:27.208917 23022 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
I20260709 13:09:27.205873 22992 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:27.179441 26771 raft_consensus.cc:2804] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:27.198383 26770 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:27.212013 22992 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:27.212141 22843 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:27.213901 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:27.217545 23023 raft_consensus.cc:2393] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 11: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:27.217720 22871 raft_consensus.cc:1240] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:09:27.218396 22920 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:27.217326 22843 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:27.173942 26769 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:27.205125 26647 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:27.204739 26762 consensus_queue.cc:1048] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 13:09:27.249962 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:27.280200 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:27.211054 26612 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:27.219660 22920 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:27.253121 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:27.240598 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:27.241428 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:27.234472 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:27.207016 26347 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:27.233968 23588 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:27.235706 22843 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:27.291013 23021 raft_consensus.cc:2393] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 13: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
W20260709 13:09:27.294394 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 74817.6us
I20260709 13:09:27.291154 23022 raft_consensus.cc:2376] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10. Re-sending same reply.
W20260709 13:09:27.288463 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 76526.6us
I20260709 13:09:27.223058 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:27.234561 26602 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:27.216820 26549 raft_consensus.cc:2764] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:09:27.299153 26758 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:27.222472 26781 raft_consensus.cc:2749] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:27.299665 26758 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:27.300133 26758 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:27.215682 26771 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:27.307124 26771 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:27.308744 26766 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:27.310307 26618 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:27.218737 26667 raft_consensus.cc:2804] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:09:27.220276 26768 log.cc:927] Time spent T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 13:09:27.297252 22992 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:27.235957 26760 raft_consensus.cc:2804] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:09:27.317687 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 82063.5us
I20260709 13:09:27.318971 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:27.200892 26392 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
W20260709 13:09:27.311685 26777 log.cc:927] Time spent T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 13:09:27.322932 22871 raft_consensus.cc:2393] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 10: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:27.221189 26764 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:27.321944 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 88077.5us
I20260709 13:09:27.333343 23588 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:27.333925 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:27.335964 23588 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:27.220376 26767 raft_consensus.cc:2804] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:27.293243 26670 raft_consensus.cc:2804] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:09:27.324608 22920 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:27.324532 22843 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:27.327010 26708 raft_consensus.cc:2804] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:09:27.336455 26347 consensus_queue.cc:1059] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.003s
I20260709 13:09:27.339823 26458 raft_consensus.cc:2749] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:09:27.340076 26708 raft_consensus.cc:697] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:27.341087 26708 consensus_queue.cc:237] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:27.323235 26647 raft_consensus.cc:2804] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:27.342006 26779 consensus_queue.cc:1048] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:09:27.327283 26778 raft_consensus.cc:2804] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:27.329960 23027 raft_consensus.cc:1240] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 8. Current term is 10. Ops: []
I20260709 13:09:27.220145 26757 consensus_queue.cc:1059] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.110s
I20260709 13:09:27.330838 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:27.232226 26551 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:27.321123 26760 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:27.311193 26667 raft_consensus.cc:697] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 10 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:09:27.375972 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 51532.7us
I20260709 13:09:27.341884 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:27.342446 26647 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:27.328536 26392 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:27.329077 26492 consensus_queue.cc:1059] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:09:27.370020 22871 raft_consensus.cc:2393] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 10: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:27.374980 22994 leader_election.cc:400] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 8. Current term is 9.
I20260709 13:09:27.381043 22994 leader_election.cc:403] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:09:27.339828 26781 raft_consensus.cc:2749] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:27.377278 22843 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:27.382388 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:27.383333 22952 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:27.383718 22871 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:27.384547 26551 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:27.338721 26767 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:27.385551 26767 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:27.376114 26760 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:27.378021 26764 raft_consensus.cc:2804] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:27.389078 26764 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:27.389243 22843 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:27.389500 26764 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:27.393009 22948 raft_consensus.cc:1240] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:27.345235 26778 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:27.358911 26670 raft_consensus.cc:697] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 10 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:27.397171 22871 raft_consensus.cc:1275] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:09:27.396127 26509 raft_consensus.cc:2749] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 8. Current term is 9.
W20260709 13:09:27.398473 26236 consensus_meta.cc:287] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.626s	user 0.008s	sys 0.004s
I20260709 13:09:27.398799 26602 raft_consensus.cc:2804] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:09:27.399339 22994 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:27.379259 26647 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:27.396210 26549 raft_consensus.cc:2804] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:09:27.393042 26769 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:27.433547 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 44351.9us
I20260709 13:09:27.404726 26778 raft_consensus.cc:3060] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:09:27.440773 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 94729.6us
I20260709 13:09:27.399827 26349 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:27.429603 26602 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:27.445931 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
W20260709 13:09:27.447631 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 48373.4us
I20260709 13:09:27.443885 26784 raft_consensus.cc:2749] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:27.450480 23620 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 777) took 1221 ms. Trace:
I20260709 13:09:27.450894 23620 rpcz_store.cc:276] 0709 13:09:26.229472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:26.229823 (+   351us) service_pool.cc:224] Handling call
0709 13:09:27.445754 (+1215931us) spinlock_profiling.cc:243] Waited 5.85 ms on lock 0x7b58001c4d98. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0a22 00007f613bbd9c09 00007f613ad33ee0 00007f613adfd0ab 00007f613adfd131 00007f613adab0a0 00007f613adaa984 00007f61343256d4 00007f6134325662
0709 13:09:27.450416 (+  4662us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13454080}
W20260709 13:09:27.452972 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 117068us
I20260709 13:09:27.434309 26392 raft_consensus.cc:2749] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:27.439893 26347 consensus_queue.cc:1243] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer f3c799b1249e40119d2a5864231c6c75 log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:09:27.437188 26549 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:27.455969 22843 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:27.429523 26236 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:27.452852 26602 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:27.470086 23026 raft_consensus.cc:1240] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:27.475383 23025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33418 (request call id 1829) took 1456 ms. Trace:
I20260709 13:09:27.475781 23025 rpcz_store.cc:276] 0709 13:09:26.010915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:26.011160 (+   245us) service_pool.cc:224] Handling call
0709 13:09:27.466928 (+1455768us) spinlock_profiling.cc:243] Waited 647 ms on lock 0x7b54001c75d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:27.467061 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1487095168}
I20260709 13:09:27.463176 26549 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:27.483565 22994 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:27.474861 23588 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:27.468888 26769 raft_consensus.cc:2749] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:09:27.492395 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 49446.5us
I20260709 13:09:27.493485 22843 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:27.495823 22878 raft_consensus.cc:1240] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:27.501930 22994 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:27.456221 26667 consensus_queue.cc:237] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:27.506546 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 147335us
I20260709 13:09:27.506791 22994 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:27.507510 22918 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:27.507385 22920 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:27.510504 23588 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:27.510634 22994 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:27.495631 26708 raft_consensus.cc:2749] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 13:09:27.483047 26759 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:27.515801 22920 leader_election.cc:400] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 9. Current term is 10.
I20260709 13:09:27.517164 22994 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:27.517583 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:27.517890 26492 raft_consensus.cc:2749] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:27.517815 22920 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:27.519680 22994 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:27.519526 26767 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:27.520857 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:27.523728 26699 raft_consensus.cc:2749] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:09:27.522887 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:27.530304 23024 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 10.
I20260709 13:09:27.530488 22843 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:27.509017 26458 raft_consensus.cc:2804] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:27.531351 26458 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:27.531906 26458 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:27.524366 26792 consensus_queue.cc:1059] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
W20260709 13:09:27.534641 23590 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.004s	sys 0.012s
W20260709 13:09:27.534946 23590 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.004s	sys 0.012s
W20260709 13:09:27.535395 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 560982us
I20260709 13:09:27.535831 26764 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:27.538067 26753 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:27.540295 26753 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:27.540999 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:27.524377 26791 raft_consensus.cc:2804] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:09:27.523962 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:27.523762 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:27.525346 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:27.468116 26670 consensus_queue.cc:237] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:27.541235 26786 log.cc:927] Time spent T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:09:27.543497 23616 raft_consensus.cc:1275] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:09:27.508957 26392 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:09:27.529430 26349 raft_consensus.cc:2804] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:09:27.554319 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:27.564407 22994 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:27.567673 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:27.567664 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:27.514272 26509 raft_consensus.cc:2749] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:27.570554 23623 raft_consensus.cc:2368] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 11. Current term is 12.
I20260709 13:09:27.572379 23024 raft_consensus.cc:1240] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:27.573450 22994 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:27.573616 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:27.575619 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:27.576236 23588 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:27.577229 23028 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 11.
I20260709 13:09:27.595486 22994 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:27.567780 26695 raft_consensus.cc:2749] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:27.603096 22952 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:09:27.516539 26770 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:27.578965 26788 raft_consensus.cc:2749] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:09:27.608493 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 61822us
I20260709 13:09:27.582607 26708 raft_consensus.cc:2749] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:27.579655 26753 raft_consensus.cc:2764] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:09:27.610345 23590 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:27.579587 26794 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:09:27.598196 26392 raft_consensus.cc:2749] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:09:27.612175 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:27.616901 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 99134.5us
I20260709 13:09:27.618726 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:27.619083 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 88695us
I20260709 13:09:27.610436 26492 raft_consensus.cc:2804] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:09:27.609579 26792 raft_consensus.cc:3055] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:09:27.626292 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:27.635150 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:27.635823 22878 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:27.633369 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:27.637137 23022 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 9
W20260709 13:09:27.634790 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 127298us
I20260709 13:09:27.621881 22920 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:27.641109 22920 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:27.613042 26762 consensus_queue.cc:1059] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 13:09:27.646413 22920 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:27.625677 26492 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:27.622516 26794 raft_consensus.cc:697] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:27.634385 26792 raft_consensus.cc:740] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
W20260709 13:09:27.662947 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:27.656845 22920 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:27.634397 26236 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:27.658288 26759 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:27.658644 26753 raft_consensus.cc:2804] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 13:09:27.659408 26785 log.cc:927] Time spent T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 13:09:27.677073 22843 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:27.659538 26790 log.cc:927] Time spent T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 13:09:27.661309 26770 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:27.650342 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 87717.9us
I20260709 13:09:27.675228 22920 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:27.684465 23022 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 9.
I20260709 13:09:27.649804 26788 raft_consensus.cc:2764] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:09:27.686332 26788 raft_consensus.cc:2804] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:27.686965 26788 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:27.687469 26788 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:09:27.654127 26787 log.cc:927] Time spent T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 13:09:27.689122 22878 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:09:27.682934 22918 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:27.691598 22918 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:27.699935 26349 raft_consensus.cc:697] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 6 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:27.656448 26492 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:27.704960 26458 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:27.705714 26803 raft_consensus.cc:2804] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:09:27.707345 26806 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:27.707700 26647 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:27.717459 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:09:27.725124 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 99825.1us
I20260709 13:09:27.725418 22844 leader_election.cc:400] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 11. Current term is 12.
I20260709 13:09:27.728217 22844 leader_election.cc:403] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:09:27.728802 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:27.730597 23022 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:27.729269 23021 pending_rounds.cc:85] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75: Aborting all ops after (but not including) 8
I20260709 13:09:27.731927 23021 pending_rounds.cc:107] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:09:27.732281 22918 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:27.732319 23021 raft_consensus.cc:2889] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:27.712911 26236 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 5 FOLLOWER]: Advancing to term 6
I20260709 13:09:27.713148 26753 raft_consensus.cc:697] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:27.750430 26753 consensus_queue.cc:237] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:27.714915 26759 raft_consensus.cc:2749] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:27.722205 26816 raft_consensus.cc:2804] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:09:27.740557 26799 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:09:27.753468 26816 raft_consensus.cc:697] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:27.754299 26602 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:27.754577 26816 consensus_queue.cc:237] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:27.741053 26808 raft_consensus.cc:2804] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:09:27.742057 26796 log.cc:927] Time spent T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:09:27.742810 26425 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:27.712841 26764 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:27.749032 26793 consensus_queue.cc:1048] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.087s
I20260709 13:09:27.746387 26801 raft_consensus.cc:2749] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:27.722832 26769 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
W20260709 13:09:27.759006 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:27.757386 26798 raft_consensus.cc:2749] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:09:27.735692 26803 raft_consensus.cc:697] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:29.419720 26803 consensus_queue.cc:237] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:29.421432 26781 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:27.761966 26808 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:29.422057 26781 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:29.422816 26808 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:29.423945 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:29.421460 26509 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:29.425287 22946 raft_consensus.cc:1240] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:29.425994 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:29.425609 26509 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:27.779249 26767 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:29.426662 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:29.427408 22947 raft_consensus.cc:2393] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 10: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:27.776453 26791 raft_consensus.cc:697] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 9 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:29.427839 22953 raft_consensus.cc:1275] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:09:29.428586 26791 consensus_queue.cc:237] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:29.424490 26781 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:27.729616 26806 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:29.433535 26806 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:27.762193 26758 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:29.432454 26808 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:29.435693 26758 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:27.868124 26602 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:27.886428 26813 consensus_queue.cc:1048] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.013s
I20260709 13:09:29.436296 26808 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:27.864516 26769 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:27.778604 26794 consensus_queue.cc:237] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:27.804945 26349 consensus_queue.cc:237] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:27.839808 26792 consensus_queue.cc:260] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:27.928190 26819 raft_consensus.cc:2749] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 11. Current term is 12.
I20260709 13:09:27.936870 26458 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:27.943612 26760 consensus_meta.cc:287] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.557s	user 0.010s	sys 0.000s
W20260709 13:09:27.946774 26612 consensus_meta.cc:287] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.634s	user 0.009s	sys 0.001s
I20260709 13:09:27.946552 26492 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:27.925069 26778 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:27.944101 26788 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:27.938582 26549 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:27.952363 26236 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 13:09:27.956827 26771 consensus_meta.cc:287] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.649s	user 0.008s	sys 0.005s
W20260709 13:09:27.957907 26782 log.cc:927] Time spent T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.701s	user 0.006s	sys 0.014s
I20260709 13:09:27.990011 26741 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:09:28.001271 26774 log.cc:927] Time spent T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.766s	user 0.004s	sys 0.000s
W20260709 13:09:28.002180 26739 log.cc:927] Time spent T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 1.504s	user 0.004s	sys 0.000s
I20260709 13:09:28.030563 26626 raft_consensus.cc:3055] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 13:09:29.197397 26772 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 1.977s	user 0.000s	sys 0.015s
W20260709 13:09:29.207770 26815 log.cc:927] Time spent T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 1.338s	user 0.006s	sys 0.013s
I20260709 13:09:29.220574 23027 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 1874) took 2406 ms. Trace:
I20260709 13:09:29.230330 22873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53724 (request call id 624) took 2377 ms. Trace:
W20260709 13:09:29.235478 22920 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.550s	user 0.027s	sys 0.011s
W20260709 13:09:29.243984 26818 log.cc:927] Time spent T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 1.327s	user 0.008s	sys 0.011s
W20260709 13:09:29.256354 23619 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60044 (request call id 726) took 2502 ms (client timeout 1714 ms). Trace:
I20260709 13:09:27.919374 26647 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:29.263062 26425 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:29.263600 26742 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:29.267778 26590 raft_consensus.cc:3055] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:09:29.271157 26407 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:09:29.283911 26810 log.cc:927] Time spent T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 1.386s	user 0.000s	sys 0.014s
W20260709 13:09:29.303440 22843 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.591s	user 0.012s	sys 0.041s
I20260709 13:09:29.334801 23622 raft_consensus.cc:1240] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 10. Current term is 11. Ops: [8.7-10.8]
W20260709 13:09:29.335371 23021 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.586s	user 0.016s	sys 0.035s
I20260709 13:09:29.365749 26347 raft_consensus.cc:3055] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:09:29.367053 23621 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 766) took 3814 ms (client timeout 2506 ms). Trace:
I20260709 13:09:29.364307 26780 raft_consensus.cc:3055] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:09:29.363466 23618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 782) took 2957 ms. Trace:
I20260709 13:09:29.385012 23026 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 1864) took 2575 ms. Trace:
W20260709 13:09:29.389565 23625 consensus_meta.cc:287] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 1.823s	user 0.017s	sys 0.006s
I20260709 13:09:29.392408 26699 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:29.392839 26506 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:29.394788 26743 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:29.395154 26765 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:29.395295 26667 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
W20260709 13:09:29.396350 23022 consensus_meta.cc:287] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 1.665s	user 0.002s	sys 0.020s
I20260709 13:09:29.398082 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:29.398435 22918 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.642s	user 0.011s	sys 0.019s
I20260709 13:09:29.398722 26755 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:29.399992 22877 raft_consensus.cc:1275] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:09:29.401051 22872 raft_consensus.cc:1275] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:09:29.403659 26648 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:29.263716 26762 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
W20260709 13:09:29.431195 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:29.411069 26789 raft_consensus.cc:2804] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:29.466713 22952 raft_consensus.cc:1240] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:29.527480 26775 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:29.452023 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:29.542291 26775 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:29.542636 23619 rpcz_store.cc:269] 0709 13:09:26.753529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:26.753665 (+   136us) service_pool.cc:224] Handling call
0709 13:09:29.256051 (+2502386us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b54002bc854. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:29.256230 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":234406272}
W20260709 13:09:29.494860 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:29.548905 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:09:29.505720 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 1.77709e+06us
I20260709 13:09:29.511766 23617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 786) took 2538 ms. Trace:
I20260709 13:09:29.467214 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:29.559120 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:29.486106 22949 raft_consensus.cc:1275] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:09:29.563654 22843 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.851s	user 0.013s	sys 0.046s
I20260709 13:09:29.559625 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:29.564350 23625 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
W20260709 13:09:29.565065 23021 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.816s	user 0.018s	sys 0.037s
I20260709 13:09:29.565912 23021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 638) took 1850 ms. Trace:
I20260709 13:09:29.566085 22872 raft_consensus.cc:1240] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:09:29.566288 23021 rpcz_store.cc:276] 0709 13:09:27.715945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:27.716465 (+   520us) service_pool.cc:224] Handling call
0709 13:09:27.729054 (+ 12589us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:27.739463 (+ 10409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:27.739509 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:27.740086 (+   577us) log.cc:844] Serialized 26 byte log entry
0709 13:09:29.565503 (+1825417us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:09:29.565538 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:29.565601 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:29.565641 (+    40us) raft_consensus.cc:1685] finished
0709 13:09:29.565751 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:29.565853 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":1816352,"threads_started":1,"wal-append.queue_time_us":189304,"wal-append.run_cpu_time_us":1492,"wal-append.run_wall_time_us":1159162}
W20260709 13:09:29.567474 22918 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.811s	user 0.013s	sys 0.021s
W20260709 13:09:29.470039 22993 connection.cc:256] RPC call timeout handler was delayed by -1.9809559020000052s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2506s for a response
W20260709 13:09:29.568188 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 1.83577e+06us
I20260709 13:09:29.440829 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:29.506731 23623 raft_consensus.cc:1240] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:29.568969 22953 raft_consensus.cc:2393] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 8: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:29.561643 23616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 788) took 2015 ms. Trace:
I20260709 13:09:29.486102 22951 raft_consensus.cc:1275] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:09:29.540814 23022 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 11.
I20260709 13:09:29.570355 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:29.570608 23022 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32794 (request call id 637) took 1855 ms (client timeout 2089 ms). Trace:
I20260709 13:09:29.479749 22955 raft_consensus.cc:2393] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 12: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:29.569882 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:09:29.565078 23625 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 775) took 2052 ms (client timeout 1848 ms). Trace:
I20260709 13:09:29.569602 23616 rpcz_store.cc:276] 0709 13:09:27.545822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:27.574894 (+ 29072us) service_pool.cc:224] Handling call
0709 13:09:27.575617 (+   723us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:27.626662 (+ 51045us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:27.626708 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:27.639118 (+ 12410us) log.cc:844] Serialized 26 byte log entry
0709 13:09:27.694416 (+ 55298us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:27.694461 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:27.694532 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:29.207012 (+1512480us) spinlock_profiling.cc:243] Waited 3.84 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:09:29.410846 (+203834us) spinlock_profiling.cc:243] Waited 6.15 ms on lock 0x7b1800416c80. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132c72c0a 00007f613c3b2219 00007f6139cb958f 00007f6132cf8571 00007f6132cfa7c1 00007f6132cfa779 00007f6132cfa709 00007f6132cfa6d1 00007f6132cf99cd
0709 13:09:29.561448 (+150602us) raft_consensus.cc:1685] finished
0709 13:09:29.561469 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:29.561592 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1955,"raft.queue_time_us":135029,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":8662,"spinlock_wait_cycles":22968832,"thread_start_us":52918,"threads_started":2,"wal-append.queue_time_us":74094}
I20260709 13:09:29.451463 22874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 1908) took 2477 ms. Trace:
I20260709 13:09:29.457779 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:29.541241 22920 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.856s	user 0.033s	sys 0.012s
I20260709 13:09:29.558202 26762 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:29.549468 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:29.573916 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 1.89854e+06us
I20260709 13:09:29.574517 22946 raft_consensus.cc:2410] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 13:09:29.571949 23026 rpcz_store.cc:276] 0709 13:09:26.809775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:26.809922 (+   147us) service_pool.cc:224] Handling call
0709 13:09:27.469896 (+659974us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:27.470696 (+   800us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:29.384741 (+1914045us) spinlock_profiling.cc:243] Waited 637 ms on lock 0x7b54001c75d0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:29.384935 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1465468672}
W20260709 13:09:29.572606 23625 rpcz_store.cc:269] 0709 13:09:27.512490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:27.512595 (+   105us) service_pool.cc:224] Handling call
0709 13:09:29.565016 (+2052421us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:09:29.450217 23028 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32794 (request call id 633) took 1923 ms (client timeout 1846 ms). Trace:
W20260709 13:09:29.575491 23028 rpcz_store.cc:269] 0709 13:09:27.526981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:27.527122 (+   141us) service_pool.cc:224] Handling call
0709 13:09:29.449977 (+1922855us) spinlock_profiling.cc:243] Waited 7.76 ms on lock 0x7b54000bc2d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:29.450125 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17910656}
W20260709 13:09:29.542104 23621 rpcz_store.cc:269] 0709 13:09:25.553211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:25.553408 (+   197us) service_pool.cc:224] Handling call
0709 13:09:29.366788 (+3813380us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b54001108d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:29.366976 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2942575488}
I20260709 13:09:29.543345 23027 rpcz_store.cc:276] 0709 13:09:26.813807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:26.851470 (+ 37663us) service_pool.cc:224] Handling call
0709 13:09:26.851727 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:27.345651 (+493924us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:29.220249 (+1874598us) spinlock_profiling.cc:243] Waited 478 ms on lock 0x7b54001f72d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:29.220497 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1099242240}
I20260709 13:09:29.569604 22918 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:29.559944 22950 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:29.560331 22873 rpcz_store.cc:276] 0709 13:09:26.852536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:27.221763 (+369227us) service_pool.cc:224] Handling call
0709 13:09:27.234075 (+ 12312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:27.311569 (+ 77494us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:27.311615 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:27.311679 (+    64us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:27.311731 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:29.230004 (+1918273us) spinlock_profiling.cc:243] Waited 77.2 ms on lock 0x7b540022a6d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:29.230058 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:29.230230 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":177645952}
I20260709 13:09:29.560608 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:29.559922 23618 rpcz_store.cc:276] 0709 13:09:26.406331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:26.406505 (+   174us) service_pool.cc:224] Handling call
0709 13:09:26.417591 (+ 11086us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:26.438934 (+ 21343us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:09:26.453368 (+ 14434us) log.cc:844] Serialized 14 byte log entry
0709 13:09:26.493236 (+ 39868us) spinlock_profiling.cc:243] Waited 1.33 ms on lock 0x7b1800452860. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cf83f3 00007f6132cfa7c1 00007f6132cfa779 00007f6132cfa709 00007f6132cfa6d1 00007f6132cf99cd 000000000038f354 000000000038f199 00007f6132cdb83a
0709 13:09:26.720814 (+227578us) log.cc:844] Serialized 14 byte log entry
0709 13:09:28.084491 (+1363677us) spinlock_profiling.cc:243] Waited 2.04 ms on lock 0x7b1800452860. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cf8522 00007f6132cfa7c1 00007f6132cfa779 00007f6132cfa709 00007f6132cfa6d1 00007f6132cf99cd 000000000038f354 000000000038f199 00007f6132cdb83a
0709 13:09:29.319839 (+1235348us) spinlock_profiling.cc:243] Waited 8.31 ms on lock 0x7b1800452860. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cdae07 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf5455 00007f6135ddd060 00007f6135def716 00007f6135e00287 00007f6135e00208
0709 13:09:29.336336 (+ 16497us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:29.337049 (+   713us) log.cc:844] Serialized 26 byte log entry
0709 13:09:29.358305 (+ 21256us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:09:29.358345 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:29.358428 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:29.363160 (+  4732us) raft_consensus.cc:1685] finished
0709 13:09:29.363187 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:29.363348 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1344992,"prepare.queue_time_us":13495,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":1715,"raft.queue_time_us":16616,"raft.run_cpu_time_us":393,"raft.run_wall_time_us":12075,"spinlock_wait_cycles":26848768,"thread_start_us":283434,"threads_started":3,"wal-append.queue_time_us":305335,"wal-append.run_cpu_time_us":2341,"wal-append.run_wall_time_us":1129574}
I20260709 13:09:29.572518 22876 raft_consensus.cc:2410] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:09:29.579221 22952 raft_consensus.cc:2468] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 8.
I20260709 13:09:29.558864 23617 rpcz_store.cc:276] 0709 13:09:26.973807 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:26.973923 (+   116us) service_pool.cc:224] Handling call
0709 13:09:26.974200 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:27.173245 (+199045us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:29.511427 (+2338182us) spinlock_profiling.cc:243] Waited 193 ms on lock 0x7b5400453654. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:29.511686 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":443825024}
I20260709 13:09:29.560384 26789 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:29.581297 26789 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:29.573387 22880 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11.
I20260709 13:09:29.569053 22948 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 11.
I20260709 13:09:29.585991 26670 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:29.586745 26670 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:29.587970 22950 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 12.
I20260709 13:09:29.509325 22878 raft_consensus.cc:1240] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:29.573138 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:29.559265 23590 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
W20260709 13:09:29.564352 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 1.89829e+06us
I20260709 13:09:29.571108 22951 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 11.
I20260709 13:09:29.597072 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:29.592760 23590 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
W20260709 13:09:29.571206 23022 rpcz_store.cc:269] 0709 13:09:27.715521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:27.715652 (+   131us) service_pool.cc:224] Handling call
0709 13:09:29.570552 (+1854900us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:29.599208 22843 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:29.573016 22874 rpcz_store.cc:276] 0709 13:09:26.973735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:27.463709 (+489974us) service_pool.cc:224] Handling call
0709 13:09:27.481554 (+ 17845us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:27.481856 (+   302us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:27.481897 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:27.490713 (+  8816us) log.cc:844] Serialized 26 byte log entry
0709 13:09:27.577489 (+ 86776us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:09:27.577755 (+   266us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:27.577829 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:29.451169 (+1873340us) raft_consensus.cc:1685] finished
0709 13:09:29.451196 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:29.451397 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":86445,"threads_started":1,"wal-append.queue_time_us":129860,"wal-append.run_cpu_time_us":3275,"wal-append.run_wall_time_us":1466745}
W20260709 13:09:29.616639 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:29.622493 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 182563us
I20260709 13:09:29.626307 26647 raft_consensus.cc:2804] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader election won for term 6
I20260709 13:09:29.626978 26647 raft_consensus.cc:697] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 6 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:29.627974 26647 consensus_queue.cc:237] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:29.628397 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:29.629205 22949 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
I20260709 13:09:29.633035 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:29.634044 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:29.634676 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:29.635300 23625 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:29.588513 22947 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:09:29.548221 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:29.642266 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:29.634900 23617 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 9.
I20260709 13:09:29.637193 22954 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 9.
I20260709 13:09:29.597414 26670 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:29.613340 26551 raft_consensus.cc:2804] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:09:29.598242 26775 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:29.652701 23617 raft_consensus.cc:1240] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 11. Current term is 12. Ops: []
I20260709 13:09:29.653170 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:29.630553 26577 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:09:29.652796 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 69880.8us
W20260709 13:09:29.653764 22993 connection.cc:256] RPC call timeout handler was delayed by -1.4083364499999789s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2202s for a response
I20260709 13:09:29.657585 23625 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 12.
I20260709 13:09:29.598006 26425 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:29.659603 26842 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:29.660199 26842 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:29.661377 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:29.661947 22874 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:29.645021 26812 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:29.662997 22993 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
W20260709 13:09:29.664486 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 65343.1us
W20260709 13:09:29.665413 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 253347us
I20260709 13:09:29.665886 22843 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:29.678678 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:29.681157 22843 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:29.458393 26618 raft_consensus.cc:3055] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:09:29.683952 26618 raft_consensus.cc:740] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:29.684485 22843 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:29.685254 26602 raft_consensus.cc:2804] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:29.685671 22843 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:29.685698 26602 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:29.686205 23588 connection.cc:256] RPC call timeout handler was delayed by -1.5031968160000133s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2013s for a response
I20260709 13:09:29.686457 26602 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:09:29.687304 23588 connection.cc:256] RPC call timeout handler was delayed by -1.3312968159999627s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22040000000000004s for a response
I20260709 13:09:29.687529 22843 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:29.688828 22843 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:29.689177 26780 raft_consensus.cc:740] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:29.689668 26798 raft_consensus.cc:2749] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:09:29.690111 22843 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:29.690009 26767 raft_consensus.cc:2749] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:09:29.690327 26798 raft_consensus.cc:2749] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:29.691358 26767 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:29.691493 26798 raft_consensus.cc:2749] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:29.691695 22843 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:29.692780 26798 raft_consensus.cc:2749] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:29.692979 26762 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:29.693188 22843 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:29.693531 26762 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:29.694151 26798 raft_consensus.cc:2749] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:09:29.694061 26762 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:29.694660 26767 consensus_queue.cc:1048] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:09:29.695379 22843 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:29.695930 26762 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:29.696138 26767 raft_consensus.cc:2749] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:29.696419 26762 raft_consensus.cc:2764] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:09:29.697471 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:09:29.699060 26772 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 2.479s	user 0.000s	sys 0.020s
I20260709 13:09:29.701496 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:29.702282 22873 raft_consensus.cc:2393] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 12: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:29.703095 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:29.704411 22873 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:29.666326 26804 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:29.705931 26804 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:29.702283 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 132719us
I20260709 13:09:29.707834 22918 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:29.653731 26551 raft_consensus.cc:697] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 11 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:29.685945 26765 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:29.711896 26765 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:29.717068 22874 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 11.
I20260709 13:09:29.718358 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:29.718456 23590 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:29.696521 22843 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:29.690059 26780 consensus_queue.cc:260] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:29.724885 26767 raft_consensus.cc:2749] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:09:29.729514 26551 consensus_queue.cc:237] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:29.730811 26756 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:29.731370 26756 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:29.724134 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:29.665318 26577 raft_consensus.cc:697] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:29.732297 26817 raft_consensus.cc:2804] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 13:09:29.736680 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 104186us
W20260709 13:09:29.737504 23588 leader_election.cc:336] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Timed out: RequestConsensusVote RPC to 127.21.63.2:33309 timed out after 1.690s (SENT)
I20260709 13:09:29.737854 26765 consensus_queue.cc:1059] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:09:29.738132 23588 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:29.739315 26811 raft_consensus.cc:2749] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:29.742589 26811 consensus_queue.cc:1048] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:09:29.743177 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:29.735149 22873 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 9.
I20260709 13:09:29.746716 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:29.747275 22874 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:29.749156 26577 consensus_queue.cc:237] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:29.754917 23619 raft_consensus.cc:1275] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:09:29.755074 26760 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:29.758107 26804 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:29.759595 26236 raft_consensus.cc:2749] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:09:29.760301 26236 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:29.761595 22879 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:29.765733 22992 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:29.722332 26819 raft_consensus.cc:2749] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:09:29.780157 23620 raft_consensus.cc:2393] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 9: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:29.787385 26849 raft_consensus.cc:2749] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:29.791170 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:29.791908 22947 raft_consensus.cc:2368] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 9. Current term is 10.
I20260709 13:09:29.793047 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:29.793763 22947 raft_consensus.cc:2368] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 11. Current term is 12.
I20260709 13:09:29.793630 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:29.794685 22879 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
W20260709 13:09:29.796654 22843 connection.cc:256] RPC call timeout handler was delayed by -1.9481454599999779s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1892s for a response
W20260709 13:09:29.801011 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 121786us
I20260709 13:09:29.732589 26425 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:09:29.801374 22919 connection.cc:256] RPC call timeout handler was delayed by -1.3315489660000139s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1633s for a response
W20260709 13:09:29.804793 22919 connection.cc:256] RPC call timeout handler was delayed by -1.2586489660000098s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1714s for a response
I20260709 13:09:29.802960 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:29.805423 22993 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:29.684937 26618 consensus_queue.cc:260] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:29.796056 22920 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:09:29.806064 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 98289.2us
I20260709 13:09:29.808415 22843 leader_election.cc:400] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 9. Current term is 10.
I20260709 13:09:29.809330 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:29.809883 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:29.758132 26778 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:29.809382 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:29.810161 23619 raft_consensus.cc:2368] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 11. Current term is 12.
I20260709 13:09:29.812348 22874 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 12.
I20260709 13:09:29.812742 22843 leader_election.cc:400] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 11. Current term is 12.
I20260709 13:09:29.812960 22919 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:29.813714 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:29.814097 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:29.814383 22874 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
I20260709 13:09:29.814806 22947 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 9.
I20260709 13:09:29.815698 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:29.776235 26817 raft_consensus.cc:697] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 11 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:29.801755 26780 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:29.676245 26812 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:29.825299 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:29.819527 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:29.826547 22844 leader_election.cc:400] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 11. Current term is 12.
I20260709 13:09:29.826857 22844 leader_election.cc:403] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:09:29.829483 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:29.829663 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:29.823961 22918 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:29.831960 22919 leader_election.cc:400] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 election: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 10. Current term is 11.
I20260709 13:09:29.779165 26236 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:29.828722 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 51956.1us
I20260709 13:09:29.820109 22920 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:29.837528 22919 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:29.804201 26765 consensus_queue.cc:1059] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:09:29.816804 26852 raft_consensus.cc:2804] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:09:29.848292 23618 raft_consensus.cc:1275] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:09:29.816762 26788 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:29.820153 26849 raft_consensus.cc:2749] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:09:29.806921 26811 consensus_queue.cc:1059] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 13:09:29.848932 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:29.816474 26846 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:09:29.858415 22951 raft_consensus.cc:2368] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 9. Current term is 10.
I20260709 13:09:29.844313 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:29.825608 26762 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:29.859237 22873 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:09:29.859186 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:29.828336 26760 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:29.833630 26798 raft_consensus.cc:2749] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 11. Current term is 12.
I20260709 13:09:29.861877 22946 raft_consensus.cc:2393] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 10: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:29.835772 26839 raft_consensus.cc:2749] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:09:29.864542 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:29.865741 22946 raft_consensus.cc:1275] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:09:29.818601 26767 consensus_queue.cc:1048] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:09:29.870591 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:09:29.892866 22994 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:29.858590 26852 raft_consensus.cc:697] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
W20260709 13:09:29.896106 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 72251.1us
I20260709 13:09:29.866490 26788 raft_consensus.cc:2749] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:09:29.901696 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:29.872732 26849 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:09:29.910430 22918 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:29.863848 26762 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:29.912878 22843 leader_election.cc:336] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.892s (SENT)
I20260709 13:09:29.914439 22918 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:29.915560 22843 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:29.916810 26788 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:29.917608 22918 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:29.918339 26842 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:29.918892 22918 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:29.920370 22918 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:29.919076 26602 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:29.927745 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 71475.2us
I20260709 13:09:29.926991 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:29.928216 26870 raft_consensus.cc:2749] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:29.928758 22873 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
I20260709 13:09:29.929028 26870 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:29.929350 26602 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:29.929901 22873 raft_consensus.cc:1240] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:09:29.930675 26861 raft_consensus.cc:2749] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:29.931119 22873 raft_consensus.cc:1240] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:09:29.931370 26861 raft_consensus.cc:2804] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:09:29.931835 26861 raft_consensus.cc:697] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:29.933687 26852 consensus_queue.cc:237] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:29.937543 26236 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:29.865760 26817 consensus_queue.cc:237] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:29.933454 22873 raft_consensus.cc:1240] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 10. Current term is 11. Ops: [7.6-10.8]
W20260709 13:09:29.967973 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 147904us
I20260709 13:09:29.919718 26849 raft_consensus.cc:697] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
W20260709 13:09:29.973285 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 135839us
I20260709 13:09:29.975845 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:29.977152 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:29.977804 23022 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
I20260709 13:09:29.979700 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:29.980283 23028 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 9.
I20260709 13:09:29.981820 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:29.982651 26612 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:29.983732 23621 raft_consensus.cc:2393] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 9: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:29.981550 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:29.927323 26760 raft_consensus.cc:2749] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:29.981566 23588 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:29.987215 26807 raft_consensus.cc:2749] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:29.985092 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:29.988960 23027 raft_consensus.cc:2393] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 8: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
W20260709 13:09:29.988633 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 75801.2us
I20260709 13:09:29.990684 22843 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:29.988878 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 60500.4us
I20260709 13:09:29.995903 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:29.996742 23021 raft_consensus.cc:2393] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 10: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:29.999660 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:30.001458 23027 raft_consensus.cc:2393] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 11: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:29.999678 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:30.003612 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:30.004695 23026 raft_consensus.cc:2393] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 8: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:30.011405 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:09:30.016570 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 159995us
I20260709 13:09:29.984632 26612 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:29.962811 26765 consensus_queue.cc:1059] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 13:09:30.026737 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:30.027601 22951 raft_consensus.cc:2393] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 9: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:30.028079 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:29.997328 26868 raft_consensus.cc:3055] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:09:30.029985 22880 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:09:30.025496 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:30.026111 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:30.035411 22843 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:29.974861 26789 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:30.009933 26788 raft_consensus.cc:2749] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:30.012593 26602 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:29.976492 26861 consensus_queue.cc:237] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:30.040923 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:09:30.933074 23646 debug-util.cc:398] Leaking SignalData structure 0x7b0800258d00 after lost signal to thread 21759
W20260709 13:09:30.934401 23646 debug-util.cc:398] Leaking SignalData structure 0x7b080029ace0 after lost signal to thread 22812
I20260709 13:09:30.934851 22946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 1898) took 1115 ms. Trace:
W20260709 13:09:30.664722 22920 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.011s	sys 0.020s
I20260709 13:09:30.935195 22946 rpcz_store.cc:276] 0709 13:09:29.819569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:29.867928 (+ 48359us) service_pool.cc:224] Handling call
0709 13:09:29.889224 (+ 21296us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:29.898974 (+  9750us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:29.899023 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:29.904120 (+  5097us) log.cc:844] Serialized 26 byte log entry
0709 13:09:30.133443 (+229323us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:09:30.133502 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:30.133667 (+   165us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:30.934652 (+800985us) raft_consensus.cc:1685] finished
0709 13:09:30.934670 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:30.934790 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17304,"raft.run_cpu_time_us":66,"raft.run_wall_time_us":66,"thread_start_us":213570,"threads_started":1,"wal-append.queue_time_us":1013822}
I20260709 13:09:30.082094 26873 consensus_queue.cc:1059] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.928s
I20260709 13:09:30.937101 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:30.082026 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:30.937836 22946 raft_consensus.cc:2393] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 8: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:30.937912 23025 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:30.938985 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:30.939337 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:30.975694 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:30.551276 26781 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:30.979390 26781 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:30.980619 22955 raft_consensus.cc:2376] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 12. Re-sending same reply.
I20260709 13:09:30.981897 26781 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:30.940342 22871 raft_consensus.cc:1275] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:09:30.114965 26798 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:30.912760 22873 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.938s	user 0.046s	sys 0.050s
W20260709 13:09:31.021136 22873 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.048s	user 0.047s	sys 0.050s
I20260709 13:09:31.022157 22873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 1913) took 1356 ms. Trace:
W20260709 13:09:30.122516 26878 log.cc:927] Time spent T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:09:30.033324 26868 raft_consensus.cc:740] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:30.949467 23027 raft_consensus.cc:2393] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 7: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:30.136834 26853 raft_consensus.cc:2804] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:30.949939 26812 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:09:30.084486 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 57120.4us
I20260709 13:09:30.044409 22843 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:30.950881 26492 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:30.084913 26765 consensus_queue.cc:1059] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.867s
I20260709 13:09:30.138326 26851 consensus_queue.cc:1059] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.814s
I20260709 13:09:30.024904 26875 consensus_queue.cc:1059] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.955s
W20260709 13:09:30.083388 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 91616.8us
I20260709 13:09:31.032781 22992 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:30.953181 26789 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:30.936218 22920 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.929s	user 0.017s	sys 0.020s
I20260709 13:09:30.164806 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:30.958557 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:31.036392 22953 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 11.
I20260709 13:09:31.036850 22953 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52222 (request call id 612) took 1041 ms. Trace:
I20260709 13:09:31.037854 22951 raft_consensus.cc:2393] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 12: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:31.037142 22953 rpcz_store.cc:276] 0709 13:09:29.995082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:30.164482 (+169400us) service_pool.cc:224] Handling call
0709 13:09:31.036798 (+872316us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:09:30.907980 26780 consensus_meta.cc:287] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 1.082s	user 0.008s	sys 0.003s
W20260709 13:09:30.846436 26870 consensus_meta.cc:287] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.917s	user 0.009s	sys 0.004s
I20260709 13:09:30.978638 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:30.154981 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:30.145797 23622 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60044 (request call id 732) took 3374 ms. Trace:
I20260709 13:09:30.981986 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:30.984865 23619 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 785) took 1020 ms. Trace:
I20260709 13:09:30.986964 26648 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:30.988382 26884 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:31.040647 22955 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52222 (request call id 620) took 1032 ms. Trace:
I20260709 13:09:31.041210 22955 rpcz_store.cc:276] 0709 13:09:30.007771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:30.981741 (+973970us) service_pool.cc:224] Handling call
0709 13:09:31.040597 (+ 58856us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:30.993153 26506 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:30.125123 26767 raft_consensus.cc:2804] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:30.995882 26647 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:30.998947 26883 log.cc:927] Time spent T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:09:30.999064 23624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60044 (request call id 723) took 4829 ms. Trace:
I20260709 13:09:31.083441 23028 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 10.
I20260709 13:09:31.082578 23624 rpcz_store.cc:276] 0709 13:09:26.169605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:26.169749 (+   144us) service_pool.cc:224] Handling call
0709 13:09:26.174964 (+  5215us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:26.175413 (+   449us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:26.175442 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:26.175906 (+   464us) log.cc:844] Serialized 26 byte log entry
0709 13:09:26.377475 (+201569us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:09:26.377516 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:26.377579 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:28.001588 (+1624009us) spinlock_profiling.cc:243] Waited 17.1 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:09:30.998768 (+2997180us) raft_consensus.cc:1685] finished
0709 13:09:30.998789 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:30.998988 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2184,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":3730,"spinlock_wait_cycles":39314176,"thread_start_us":201054,"threads_started":1,"wal-append.queue_time_us":321345}
I20260709 13:09:31.084313 26618 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:30.157392 23618 raft_consensus.cc:1240] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:09:31.084617 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:31.085088 23618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60044 (request call id 738) took 1089 ms. Trace:
I20260709 13:09:31.084805 26647 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:09:31.085305 23624 raft_consensus.cc:2468] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
I20260709 13:09:31.085441 23618 rpcz_store.cc:276] 0709 13:09:29.995930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:30.156829 (+160899us) service_pool.cc:224] Handling call
0709 13:09:30.157166 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:31.084862 (+927696us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:31.085041 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:31.002629 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:31.084692 23028 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33418 (request call id 1858) took 1103 ms. Trace:
I20260709 13:09:31.086491 23020 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:09:31.086721 23028 rpcz_store.cc:276] 0709 13:09:29.981196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:29.981296 (+   100us) service_pool.cc:224] Handling call
0709 13:09:31.084523 (+1103227us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b540018ae54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:31.084622 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2524871040}
I20260709 13:09:31.087078 26647 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:31.062572 26870 raft_consensus.cc:2749] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:31.087500 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:31.088157 23020 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:31.003804 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:31.089344 23023 raft_consensus.cc:2393] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 11: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:31.090101 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:31.091285 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:31.092140 26870 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:31.092280 26758 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:31.087870 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:31.092206 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:31.008880 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:09:30.935644 23646 debug-util.cc:398] Leaking SignalData structure 0x7b08002e3040 after lost signal to thread 22904
I20260709 13:09:31.020076 22871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 1918) took 1339 ms. Trace:
W20260709 13:09:31.094862 23646 debug-util.cc:398] Leaking SignalData structure 0x7b08002e3120 after lost signal to thread 22905
I20260709 13:09:31.094957 22871 rpcz_store.cc:276] 0709 13:09:29.680689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:30.086435 (+405746us) service_pool.cc:224] Handling call
0709 13:09:30.940039 (+853604us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:31.019730 (+ 79691us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:31.020004 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:09:31.096242 23646 debug-util.cc:398] Leaking SignalData structure 0x7b080038b4e0 after lost signal to thread 22979
I20260709 13:09:30.573483 26788 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:31.022778 22873 rpcz_store.cc:276] 0709 13:09:29.665695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:29.968907 (+303212us) service_pool.cc:224] Handling call
0709 13:09:29.969288 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:29.969540 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:09:29.972932 (+  3392us) log.cc:844] Serialized 14 byte log entry
0709 13:09:31.021738 (+1048806us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:31.021806 (+    68us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:09:31.021877 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:31.021959 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:31.022094 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":4574,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"thread_start_us":1048417,"threads_started":1,"wal-append.queue_time_us":989649}
I20260709 13:09:31.024776 26781 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:30.139078 26762 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:31.038270 22951 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52222 (request call id 614) took 1042 ms. Trace:
I20260709 13:09:31.103883 22951 rpcz_store.cc:276] 0709 13:09:29.997464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:30.032461 (+ 34997us) service_pool.cc:224] Handling call
0709 13:09:31.038222 (+1005761us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:30.120277 26849 consensus_queue.cc:237] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:31.042120 22992 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:31.040232 23622 rpcz_store.cc:276] 0709 13:09:26.771800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:26.931954 (+160154us) service_pool.cc:224] Handling call
0709 13:09:29.334481 (+2402527us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:29.557840 (+223359us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:30.145599 (+587759us) spinlock_profiling.cc:243] Waited 2.4 s on lock 0x7b54002bc850. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:30.145737 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5523740928}
I20260709 13:09:31.040257 22950 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6.
I20260709 13:09:31.048127 23025 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
I20260709 13:09:31.107121 22950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52222 (request call id 623) took 1066 ms. Trace:
I20260709 13:09:31.107312 23025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33418 (request call id 1867) took 1067 ms. Trace:
I20260709 13:09:31.107955 23025 rpcz_store.cc:276] 0709 13:09:30.039790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:30.039925 (+   135us) service_pool.cc:224] Handling call
0709 13:09:31.107252 (+1067327us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:31.051764 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:31.034938 26839 consensus_queue.cc:1048] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260709 13:09:31.107517 22950 rpcz_store.cc:276] 0709 13:09:30.040725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:30.154715 (+113990us) service_pool.cc:224] Handling call
0709 13:09:31.107069 (+952354us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:31.064654 26868 consensus_queue.cc:260] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:31.113359 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:31.067113 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:31.069132 26506 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:09:31.070545 23620 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 773) took 3558 ms (client timeout 1628 ms). Trace:
I20260709 13:09:31.040207 22946 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 11.
W20260709 13:09:31.072571 26803 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.649s	user 0.000s	sys 0.026s
I20260709 13:09:31.048949 26884 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:31.073943 26742 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:31.142755 23618 raft_consensus.cc:1240] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 11. Current term is 12. Ops: []
I20260709 13:09:31.143747 26742 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:31.077049 22875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 1900) took 4261 ms. Trace:
I20260709 13:09:31.063934 26767 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:31.144500 26767 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:31.041311 26879 raft_consensus.cc:3055] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:09:31.028244 26798 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:31.144379 22875 rpcz_store.cc:276] 0709 13:09:26.816117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:26.822163 (+  6046us) service_pool.cc:224] Handling call
0709 13:09:26.846224 (+ 24061us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:26.927883 (+ 81659us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:26.927914 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:26.947060 (+ 19146us) log.cc:844] Serialized 26 byte log entry
0709 13:09:27.154947 (+207887us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:09:27.154998 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:27.155090 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:28.000609 (+845519us) spinlock_profiling.cc:243] Waited 25 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:09:31.076710 (+3076101us) raft_consensus.cc:1685] finished
0709 13:09:31.076733 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:31.076958 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":98396,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":23714,"spinlock_wait_cycles":57560448,"thread_start_us":220861,"threads_started":2,"wal-append.queue_time_us":247845}
I20260709 13:09:31.079450 26819 raft_consensus.cc:2749] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:09:31.080062 26699 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:31.082001 26602 raft_consensus.cc:2749] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:31.093706 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:09:31.008118 22994 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.63.1:33929 timed out after 2.492s (SENT) [suppressed 66 similar messages]
I20260709 13:09:30.945165 26590 raft_consensus.cc:740] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:31.096431 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:31.092983 26870 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:09:31.098742 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:31.151655 22873 raft_consensus.cc:2393] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 10: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
W20260709 13:09:31.097918 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 1.05342e+06us
I20260709 13:09:31.033161 26741 consensus_queue.cc:1048] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
W20260709 13:09:31.103506 23646 debug-util.cc:398] Leaking SignalData structure 0x7b0800370040 after lost signal to thread 23053
I20260709 13:09:31.104983 23020 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 9.
I20260709 13:09:31.062762 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:31.156085 22843 leader_election.cc:400] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 9. Current term is 10.
I20260709 13:09:31.156152 23616 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
I20260709 13:09:31.156425 22843 leader_election.cc:403] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 13:09:31.109249 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:31.110437 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 68392.6us
I20260709 13:09:31.049727 26765 raft_consensus.cc:2804] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:31.049397 26875 consensus_queue.cc:1243] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer 0a407504ed1741bf89a276558adf4c0f log is divergent from this leader: its last log entry 9.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:09:31.040822 23619 rpcz_store.cc:276] 0709 13:09:29.964099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:29.982907 (+ 18808us) service_pool.cc:224] Handling call
0709 13:09:29.983190 (+   283us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:29.984087 (+   897us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:29.984114 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:29.984554 (+   440us) log.cc:844] Serialized 26 byte log entry
0709 13:09:30.339001 (+354447us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:09:30.339039 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:30.339103 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:30.984576 (+645473us) raft_consensus.cc:1685] finished
0709 13:09:30.984597 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:30.984788 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1464,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":2101,"thread_start_us":354070,"threads_started":1,"wal-append.queue_time_us":998120}
I20260709 13:09:31.066466 26648 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:31.116366 22946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52222 (request call id 619) took 1114 ms. Trace:
I20260709 13:09:31.173580 22946 rpcz_store.cc:276] 0709 13:09:30.002114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:30.978316 (+976202us) service_pool.cc:224] Handling call
0709 13:09:31.116314 (+137998us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:31.101043 26780 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:31.116147 23620 rpcz_store.cc:269] 0709 13:09:27.511840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:27.511954 (+   114us) service_pool.cc:224] Handling call
0709 13:09:31.070318 (+3558364us) spinlock_profiling.cc:243] Waited 2.22 s on lock 0x7b54003d9ad4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:31.070466 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5114699008}
I20260709 13:09:31.117671 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:31.174496 26780 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:31.175534 22950 raft_consensus.cc:2393] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 12: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:31.176119 22950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52222 (request call id 622) took 1146 ms. Trace:
I20260709 13:09:31.176628 26780 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:31.176517 22950 rpcz_store.cc:276] 0709 13:09:30.029855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:31.117382 (+1087527us) service_pool.cc:224] Handling call
0709 13:09:31.176044 (+ 58662us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:31.119961 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:31.177860 22955 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:31.178735 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:31.146580 26879 raft_consensus.cc:740] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:09:31.120963 26803 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.697s	user 0.001s	sys 0.027s
I20260709 13:09:31.121409 26887 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:31.123713 26868 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:31.098394 26667 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:31.112066 26762 raft_consensus.cc:2764] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election decision vote started in defunct term 10: won
I20260709 13:09:31.166703 26765 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:31.116439 26769 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:31.093060 26758 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:31.126513 23021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 1859) took 1128 ms. Trace:
I20260709 13:09:31.106889 26788 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:31.130795 26802 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.711s	user 0.021s	sys 0.015s
I20260709 13:09:31.098307 26695 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:31.148296 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:31.148651 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:31.150252 22871 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11.
I20260709 13:09:31.094441 23026 raft_consensus.cc:2376] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7. Re-sending same reply.
I20260709 13:09:31.162909 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:31.167847 23024 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 6.
W20260709 13:09:31.204448 26802 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.784s	user 0.023s	sys 0.015s
W20260709 13:09:31.169792 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 161683us
I20260709 13:09:31.172914 26647 consensus_queue.cc:1048] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
W20260709 13:09:31.214666 22994 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:31.195931 26590 consensus_queue.cc:260] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:31.157671 26846 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.240s	user 0.031s	sys 0.041s
W20260709 13:09:31.215711 26846 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.298s	user 0.031s	sys 0.041s
I20260709 13:09:31.215965 26590 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:31.218170 26879 consensus_queue.cc:260] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:31.219620 26879 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:31.222767 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:31.224581 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:31.225323 26648 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:31.225843 26648 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:31.225991 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:31.226962 22946 raft_consensus.cc:2393] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 10: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:31.216001 26846 raft_consensus.cc:697] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 10 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:31.227962 26648 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:31.228473 26648 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:31.171501 26771 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:31.229475 26648 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:31.230499 26798 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:31.231326 26648 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:31.108322 26781 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:31.201962 26699 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:31.228332 26846 consensus_queue.cc:237] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:09:31.233218 22953 raft_consensus.cc:2393] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 10: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
W20260709 13:09:31.184631 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:31.235524 22953 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52222 (request call id 621) took 1224 ms. Trace:
I20260709 13:09:31.166723 26894 raft_consensus.cc:2749] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 9. Current term is 10.
W20260709 13:09:31.169354 26791 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.720s	user 0.016s	sys 0.013s
I20260709 13:09:31.197657 23021 rpcz_store.cc:276] 0709 13:09:29.998603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:29.998759 (+   156us) service_pool.cc:224] Handling call
0709 13:09:29.999128 (+   369us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:09:30.083846 (+ 84718us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:09:30.947532 (+863686us) log.cc:844] Serialized 14 byte log entry
0709 13:09:31.040999 (+ 93467us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:09:31.107735 (+ 66736us) log.cc:844] Serialized 78 byte log entry
0709 13:09:31.107956 (+   221us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:31.108205 (+   249us) log.cc:844] Serialized 14 byte log entry
0709 13:09:31.114425 (+  6220us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:31.114525 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:31.126153 (+ 11628us) raft_consensus.cc:1685] finished
0709 13:09:31.126182 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:31.126432 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":99156,"prepare.queue_time_us":877858,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":2802,"raft.queue_time_us":92377,"raft.run_cpu_time_us":371,"raft.run_wall_time_us":14870,"thread_start_us":87939,"threads_started":1,"wal-append.queue_time_us":129405}
I20260709 13:09:31.206561 22951 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52222 (request call id 624) took 1159 ms. Trace:
I20260709 13:09:31.203466 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:31.245581 22951 rpcz_store.cc:276] 0709 13:09:30.047129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:31.118722 (+1071593us) service_pool.cc:224] Handling call
0709 13:09:31.206493 (+ 87771us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:31.208272 22875 raft_consensus.cc:2393] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 7: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
W20260709 13:09:31.209662 23646 debug-util.cc:398] Leaking SignalData structure 0x7b0800310d60 after lost signal to thread 23649
I20260709 13:09:31.210071 22873 raft_consensus.cc:1240] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:09:31.247069 22873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 1921) took 1458 ms. Trace:
I20260709 13:09:31.247524 22873 rpcz_store.cc:276] 0709 13:09:29.788224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:31.209339 (+1421115us) service_pool.cc:224] Handling call
0709 13:09:31.209756 (+   417us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:31.246735 (+ 36979us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:31.247003 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:31.249274 22873 raft_consensus.cc:1240] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:09:31.249989 22873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2416) took 1440 ms. Trace:
I20260709 13:09:31.250401 22873 rpcz_store.cc:276] 0709 13:09:29.809615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:31.248552 (+1438937us) service_pool.cc:224] Handling call
0709 13:09:31.249000 (+   448us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:31.249687 (+   687us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:31.249928 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:31.210623 22871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35370 (request call id 2418) took 1121 ms. Trace:
I20260709 13:09:31.251425 22871 rpcz_store.cc:276] 0709 13:09:30.088886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:31.096174 (+1007288us) service_pool.cc:224] Handling call
0709 13:09:31.210546 (+114372us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:31.231268 26771 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:31.235849 22953 rpcz_store.cc:276] 0709 13:09:30.011586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:31.049554 (+1037968us) service_pool.cc:224] Handling call
0709 13:09:31.235358 (+185804us) spinlock_profiling.cc:243] Waited 119 ms on lock 0x7b540037aad4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:31.235472 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":273666304}
I20260709 13:09:31.193408 22955 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
I20260709 13:09:31.196938 26765 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:31.191457 26819 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
W20260709 13:09:31.298926 26791 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.850s	user 0.016s	sys 0.014s
I20260709 13:09:31.246649 26742 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:31.316941 22843 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:31.319936 26742 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:31.320482 26900 log.cc:927] Time spent T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:09:31.320683 26864 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:31.321085 26667 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:31.321215 26864 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:31.322072 26742 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:31.303241 26648 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:31.322746 26648 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:31.323247 26788 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:31.323544 26905 log.cc:927] Time spent T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:09:31.324069 26870 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:31.324911 26648 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:31.318413 26647 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:31.327361 26647 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:31.329044 26626 raft_consensus.cc:740] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:31.330161 26626 consensus_queue.cc:260] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:31.191975 26887 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:31.334107 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:31.322654 23028 raft_consensus.cc:1275] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:09:31.352177 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:31.356704 26647 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:31.357153 26758 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:31.314492 26771 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:31.357425 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:31.359066 22951 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:09:29.427855 26509 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:31.244994 26695 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:31.361249 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:31.362016 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:09:31.320494 26896 log.cc:927] Time spent T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.168s	user 0.000s	sys 0.000s
I20260709 13:09:31.363036 26695 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:31.363160 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:31.363771 22952 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
I20260709 13:09:31.314381 26895 consensus_queue.cc:1048] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:09:31.357942 26771 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:31.311750 26819 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:31.366385 26771 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:31.367825 26819 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:31.368158 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:31.369335 22951 raft_consensus.cc:2410] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 11 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:09:31.372836 23620 raft_consensus.cc:3055] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:09:31.370366 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:31.373202 23620 raft_consensus.cc:740] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:31.356248 22875 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
W20260709 13:09:31.379767 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 62951.5us
I20260709 13:09:31.381646 22843 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:09:31.387574 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 123907us
I20260709 13:09:31.323349 26864 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:31.378880 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:31.390448 23622 raft_consensus.cc:2393] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 12: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
W20260709 13:09:31.393577 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 146681us
I20260709 13:09:31.391765 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:31.372067 26626 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:31.396634 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:31.375823 23620 consensus_queue.cc:260] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:31.354176 26870 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:31.398059 22843 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:31.401986 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:31.402901 23619 raft_consensus.cc:2410] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:09:31.377496 26648 raft_consensus.cc:2804] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:31.401672 22951 pending_rounds.cc:85] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting all ops after (but not including) 8
I20260709 13:09:31.405619 22951 pending_rounds.cc:107] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:09:31.405840 22951 raft_consensus.cc:2889] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:31.408299 23620 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:31.413839 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:31.414717 22994 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:31.394632 26771 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:31.417918 22952 raft_consensus.cc:1240] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:31.418712 22952 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 611) took 1423 ms. Trace:
I20260709 13:09:31.419171 22952 rpcz_store.cc:276] 0709 13:09:29.994852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:31.404323 (+1409471us) service_pool.cc:224] Handling call
0709 13:09:31.417688 (+ 13365us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:31.418424 (+   736us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:31.418648 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:31.401922 26819 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:09:31.430212 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:31.431442 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:31.421072 26902 raft_consensus.cc:2749] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:31.426090 26819 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:31.433499 26819 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:31.405345 26648 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:31.421233 26695 consensus_queue.cc:1048] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260709 13:09:31.436972 26648 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:31.437029 26647 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:31.437994 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:31.446988 26868 raft_consensus.cc:2749] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:31.447999 26577 raft_consensus.cc:3055] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:09:31.406849 26870 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:31.499872 26577 raft_consensus.cc:740] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
W20260709 13:09:31.499969 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 85305.1us
I20260709 13:09:31.444828 23028 raft_consensus.cc:2393] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 9: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:31.451844 26647 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:31.500727 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 102734us
I20260709 13:09:31.501767 22994 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:31.507757 26908 raft_consensus.cc:2804] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:31.508258 26908 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:31.508713 26908 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:31.511234 22843 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:31.499122 26903 log.cc:927] Time spent T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.202s	user 0.001s	sys 0.000s
W20260709 13:09:31.501135 26805 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 2.065s	user 0.011s	sys 0.030s
W20260709 13:09:31.482438 26904 log.cc:927] Time spent T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 13:09:31.513305 22994 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:31.516024 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:31.517891 22843 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:31.518652 26799 raft_consensus.cc:2749] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:31.515125 26758 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:31.526110 26758 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:31.519026 26805 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.083s	user 0.011s	sys 0.031s
I20260709 13:09:31.509469 26647 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:31.481190 26618 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:31.518230 26870 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:31.514031 22992 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:31.501559 26887 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:31.423930 26771 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:31.527586 26895 raft_consensus.cc:2804] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:31.530252 22843 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:31.421351 26864 consensus_queue.cc:1048] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.090s
I20260709 13:09:31.529886 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:31.534639 22992 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:31.539934 23020 raft_consensus.cc:2393] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 9: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:31.534983 22871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 1925) took 1527 ms. Trace:
I20260709 13:09:31.540437 22871 rpcz_store.cc:276] 0709 13:09:30.007438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:31.464652 (+1457214us) service_pool.cc:224] Handling call
0709 13:09:31.465573 (+   921us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:31.465765 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:09:31.466032 (+   267us) log.cc:844] Serialized 14 byte log entry
0709 13:09:31.534547 (+ 68515us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:31.534607 (+    60us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:31.534672 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:31.534755 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:31.534914 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":881,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":2407,"raft.queue_time_us":74,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":88,"thread_start_us":67913,"threads_started":1}
I20260709 13:09:31.543340 22871 raft_consensus.cc:1275] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:09:31.546433 26846 raft_consensus.cc:2804] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:31.546973 26846 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:31.547418 26846 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:31.548002 26758 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:31.548521 26771 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:31.539785 26895 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:31.550091 26895 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:31.548583 26647 raft_consensus.cc:2749] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:09:31.553277 22871 raft_consensus.cc:1275] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:09:31.556543 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:31.557659 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:31.558182 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:31.558768 23028 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 11.
I20260709 13:09:31.533408 26799 raft_consensus.cc:2749] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:09:31.564345 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:31.555706 26577 consensus_queue.cc:260] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:31.568272 26577 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:31.570269 26767 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:31.573009 26767 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:31.562620 26695 raft_consensus.cc:2804] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:09:31.578500 26695 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:31.578852 26695 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:31.515594 26765 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:31.580027 23020 raft_consensus.cc:2368] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 9. Current term is 10.
I20260709 13:09:31.582237 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:31.582839 23622 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:31.583276 26789 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:31.583791 26789 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:31.585615 22871 raft_consensus.cc:1275] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 13:09:31.585762 26789 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:31.589841 23622 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:09:31.537858 26618 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:31.594107 26765 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:31.572263 26648 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:31.598817 22992 leader_election.cc:336] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Timed out: RequestConsensusVote RPC to 127.21.63.2:33309 timed out after 1.741s (SENT)
I20260709 13:09:31.602240 23620 raft_consensus.cc:1275] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:09:31.595183 26618 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:31.611480 22878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2402) took 3945 ms. Trace:
I20260709 13:09:31.611825 22878 rpcz_store.cc:276] 0709 13:09:27.664172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:27.689788 (+ 25616us) service_pool.cc:224] Handling call
0709 13:09:27.690108 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:29.589342 (+1899234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:31.608921 (+2019579us) spinlock_profiling.cc:243] Waited 1.82 s on lock 0x7b5400368154. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:31.609106 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4183152512}
I20260709 13:09:31.611074 26648 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:31.622519 26908 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:31.623508 22951 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 610) took 1628 ms. Trace:
W20260709 13:09:31.626837 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:31.625118 22951 rpcz_store.cc:276] 0709 13:09:29.994625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:31.392217 (+1397592us) service_pool.cc:224] Handling call
0709 13:09:31.401543 (+  9326us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:31.406266 (+  4723us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:31.406284 (+    18us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:31.406571 (+   287us) log.cc:844] Serialized 26 byte log entry
0709 13:09:31.602722 (+196151us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:09:31.602760 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:31.602828 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:31.623298 (+ 20470us) raft_consensus.cc:1685] finished
0709 13:09:31.623316 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:31.623450 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":195867,"threads_started":1,"wal-append.queue_time_us":211443}
I20260709 13:09:31.628651 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:31.630730 26908 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:31.630511 26895 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:31.629774 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:31.632743 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:31.611080 26618 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:31.635881 26819 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:31.639346 26695 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:31.640628 23588 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:31.647723 23025 raft_consensus.cc:1240] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:09:31.648418 23027 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33418 (request call id 1863) took 1645 ms. Trace:
I20260709 13:09:31.648752 23027 rpcz_store.cc:276] 0709 13:09:30.003094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:30.003369 (+   275us) service_pool.cc:224] Handling call
0709 13:09:31.641931 (+1638562us) spinlock_profiling.cc:243] Waited 945 ms on lock 0x7b54001798d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:31.648376 (+  6445us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2173273856}
I20260709 13:09:31.649698 23027 raft_consensus.cc:1240] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:09:31.650799 23588 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:31.652472 23588 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:31.654214 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:31.656960 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:31.653066 26811 raft_consensus.cc:2749] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:09:31.647097 26618 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:31.657728 22951 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:31.663386 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:31.664975 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:31.666407 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:31.664531 26792 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:31.667236 22946 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
I20260709 13:09:31.668326 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:31.668645 22952 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 615) took 1671 ms. Trace:
I20260709 13:09:31.669356 22946 raft_consensus.cc:2410] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:09:31.670514 26851 raft_consensus.cc:3055] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:09:31.669090 22952 rpcz_store.cc:276] 0709 13:09:29.997716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:31.420163 (+1422447us) service_pool.cc:224] Handling call
0709 13:09:31.420553 (+   390us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:31.510034 (+ 89481us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:31.510054 (+    20us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:31.512168 (+  2114us) log.cc:844] Serialized 52 byte log entry
0709 13:09:31.555049 (+ 42881us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:09:31.555103 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:31.668256 (+113153us) spinlock_profiling.cc:243] Waited 89 ms on lock 0x7b540037b4d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:31.668304 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:31.668354 (+    50us) raft_consensus.cc:1685] finished
0709 13:09:31.668366 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:31.668591 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2395,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":4204,"spinlock_wait_cycles":204794112,"thread_start_us":36257,"threads_started":1,"wal-append.queue_time_us":25531}
I20260709 13:09:31.674046 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:31.675065 22952 raft_consensus.cc:2410] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:09:31.670358 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:31.676291 22946 raft_consensus.cc:2410] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 12 because replica has last-logged OpId of term: 10 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:09:31.657335 26819 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:31.677735 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:31.680477 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:31.673120 26851 raft_consensus.cc:740] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:31.685702 26851 consensus_queue.cc:260] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:31.686743 26851 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:31.687060 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:31.682060 26618 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:31.688881 26811 raft_consensus.cc:2804] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:31.689421 26811 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:31.689695 23026 raft_consensus.cc:1240] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 10 LEADER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:31.689750 22993 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:31.691447 22993 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:31.680119 22952 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
I20260709 13:09:31.692850 26908 consensus_queue.cc:1059] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:09:31.696626 26775 raft_consensus.cc:2749] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:31.697068 26908 raft_consensus.cc:3055] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:09:31.697093 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:31.697628 26908 raft_consensus.cc:740] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:31.695026 26879 consensus_queue.cc:1059] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:09:31.698781 26879 raft_consensus.cc:3055] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:09:31.698660 26549 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:31.704389 26843 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:31.684701 22946 raft_consensus.cc:2410] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 12 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:09:31.694545 26670 raft_consensus.cc:2804] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:31.699138 26879 raft_consensus.cc:740] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:31.695837 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:31.698510 26908 consensus_queue.cc:260] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:31.689785 26811 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:31.703262 22874 raft_consensus.cc:2393] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 7: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:31.707417 22951 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:09:31.713289 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:31.713907 22878 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:31.716464 22993 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:31.681129 23588 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:31.721300 22992 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:31.704924 26843 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:31.660684 26846 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:31.705453 26618 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:31.725322 23588 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:31.725003 22951 raft_consensus.cc:1240] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:31.734489 26509 raft_consensus.cc:2804] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:09:31.735348 26626 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:31.711133 26670 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:31.750123 26670 raft_consensus.cc:3060] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:31.749742 26743 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:31.736676 23028 raft_consensus.cc:1240] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:31.743748 26886 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:09:31.751147 22949 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
I20260709 13:09:31.750919 23022 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:31.725054 26808 raft_consensus.cc:2749] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:09:31.748015 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:31.748747 23623 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 779) took 4032 ms. Trace:
I20260709 13:09:31.749193 26695 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:31.769825 23020 raft_consensus.cc:1275] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 13:09:31.769579 23623 rpcz_store.cc:276] 0709 13:09:27.716229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:27.716379 (+   150us) service_pool.cc:224] Handling call
0709 13:09:27.735420 (+ 19041us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:29.569216 (+1833796us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:31.748429 (+2179213us) spinlock_profiling.cc:243] Waited 1.77 s on lock 0x7b5400572454. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:31.748653 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4072893696}
I20260709 13:09:31.770447 22951 raft_consensus.cc:1240] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:31.771808 22951 raft_consensus.cc:1275] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 13:09:31.775259 23590 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:31.775604 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:31.776186 22871 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 10.
I20260709 13:09:31.776432 22920 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:31.777292 22876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53724 (request call id 631) took 2116 ms. Trace:
I20260709 13:09:31.778025 26794 consensus_queue.cc:1048] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:09:31.778518 22920 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:31.749559 26908 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:31.782392 26764 raft_consensus.cc:2749] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:09:31.783370 26755 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:31.777683 22876 rpcz_store.cc:276] 0709 13:09:29.661005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:29.663227 (+  2222us) service_pool.cc:224] Handling call
0709 13:09:29.678177 (+ 14950us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:31.093080 (+1414903us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:09:31.163848 (+ 70768us) log.cc:844] Serialized 14 byte log entry
0709 13:09:31.305041 (+141193us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:31.305113 (+    72us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:09:31.305183 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:31.686122 (+380939us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b1800408c40. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cf83f3 00007f6132cfa7c1 00007f6132cfa779 00007f6132cfa709 00007f6132cfa6d1 00007f6132cf99cd 000000000038f354 000000000038f199 00007f6132cdb83a
0709 13:09:31.777100 (+ 90978us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b540022b0d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:31.777142 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:31.777242 (+   100us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41839,"prepare.queue_time_us":75989,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":49,"raft.queue_time_us":4259,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":3042,"spinlock_wait_cycles":3287934080,"thread_start_us":211304,"threads_started":2,"wal-append.queue_time_us":155644}
I20260709 13:09:31.755623 23028 raft_consensus.cc:1275] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:09:31.785342 22951 raft_consensus.cc:1275] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:09:31.756219 26886 raft_consensus.cc:697] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:31.789719 26755 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:31.790217 26755 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:31.789947 26886 consensus_queue.cc:237] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:31.790841 26755 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:31.793720 26852 raft_consensus.cc:3055] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:09:31.794114 26852 raft_consensus.cc:740] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:31.758783 26853 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:31.794831 26853 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:31.795014 26852 consensus_queue.cc:260] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:31.796027 26852 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:31.797551 26843 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:31.759045 22879 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35370 (request call id 2398) took 4466 ms (client timeout 2207 ms). Trace:
W20260709 13:09:31.759750 23024 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33418 (request call id 1865) took 1756 ms (client timeout 1600 ms). Trace:
I20260709 13:09:31.755646 26509 raft_consensus.cc:697] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 10 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:31.801147 26781 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:31.801337 23024 rpcz_store.cc:269] 0709 13:09:30.003739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:30.003867 (+   128us) service_pool.cc:224] Handling call
0709 13:09:31.759566 (+1755699us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b5400135454. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:31.759678 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2604687488}
I20260709 13:09:31.801959 26509 consensus_queue.cc:237] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:31.802677 26861 consensus_queue.cc:1048] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:09:31.747026 26895 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:31.804647 26861 consensus_queue.cc:1048] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:09:31.766422 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:31.805477 22947 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:09:31.800866 22879 rpcz_store.cc:269] 0709 13:09:27.292237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:27.341627 (+ 49390us) service_pool.cc:224] Handling call
0709 13:09:31.758778 (+4417151us) spinlock_profiling.cc:243] Waited 2.38 s on lock 0x7b54002882d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:31.758929 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5483381120}
I20260709 13:09:31.807053 22879 raft_consensus.cc:1240] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:31.743770 26879 consensus_queue.cc:260] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:31.779889 26889 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:31.787433 26756 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:31.787783 22951 raft_consensus.cc:1240] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:09:31.788347 26846 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:31.788388 26849 consensus_queue.cc:1059] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:09:31.801744 26781 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:31.750926 26618 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:31.787323 23029 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 10.
I20260709 13:09:31.809460 22871 raft_consensus.cc:1240] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:09:31.811443 23590 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:31.812891 22879 raft_consensus.cc:1275] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:09:31.818998 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:31.805059 26741 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:31.815539 23022 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
I20260709 13:09:31.815963 22872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2407) took 2406 ms. Trace:
I20260709 13:09:31.825599 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:31.813035 22878 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 12.
I20260709 13:09:31.822327 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:31.816967 26781 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:31.825417 22872 rpcz_store.cc:276] 0709 13:09:29.409687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:29.566845 (+157158us) service_pool.cc:224] Handling call
0709 13:09:29.567086 (+   241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:31.068865 (+1501779us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:31.068895 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:31.068935 (+    40us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:09:31.068978 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:31.815769 (+746791us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b540030d4d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:31.815810 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:31.815913 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3453240064}
I20260709 13:09:31.832119 22947 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:09:31.808440 26879 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:31.826939 22876 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11.
I20260709 13:09:31.836759 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:31.841288 22878 raft_consensus.cc:1275] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:09:31.841837 23590 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:31.837313 22876 raft_consensus.cc:2468] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 6.
I20260709 13:09:31.851136 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:31.851944 22871 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
W20260709 13:09:31.852031 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:31.853806 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 128540us
I20260709 13:09:31.854627 23588 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:31.857192 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:09:31.858625 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 146535us
I20260709 13:09:31.858944 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:31.859673 22878 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 9.
I20260709 13:09:31.859817 22993 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:09:31.860110 22874 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35370 (request call id 2417) took 2042 ms (client timeout 1896 ms). Trace:
I20260709 13:09:31.860224 22875 raft_consensus.cc:1240] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 6. Current term is 7. Ops: []
W20260709 13:09:31.860646 22874 rpcz_store.cc:269] 0709 13:09:29.817943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:29.819092 (+  1149us) service_pool.cc:224] Handling call
0709 13:09:31.859905 (+2040813us) spinlock_profiling.cc:243] Waited 1.88 s on lock 0x7b54001bba54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:31.860049 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4316307328}
I20260709 13:09:31.860831 22878 raft_consensus.cc:1240] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 13:09:31.863602 22878 raft_consensus.cc:1275] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:09:31.817744 26675 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:31.869616 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:31.866745 22955 raft_consensus.cc:1240] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 11. Current term is 12. Ops: []
I20260709 13:09:31.870579 22992 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:31.827303 26808 raft_consensus.cc:2804] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:31.871672 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:31.872295 23622 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
I20260709 13:09:31.872475 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:31.869308 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:31.867533 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:31.874281 22872 raft_consensus.cc:1240] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:09:31.827746 26781 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:31.867995 22878 raft_consensus.cc:1275] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:09:31.869849 22920 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:31.873716 26347 raft_consensus.cc:740] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:31.840857 26807 consensus_queue.cc:1059] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:09:31.870240 23620 raft_consensus.cc:2468] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:09:31.886396 22875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 1924) took 1895 ms. Trace:
I20260709 13:09:31.887035 22875 rpcz_store.cc:276] 0709 13:09:29.991266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:31.376953 (+1385687us) service_pool.cc:224] Handling call
0709 13:09:31.860040 (+483087us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:31.860850 (+   810us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:31.886125 (+ 25275us) spinlock_profiling.cc:243] Waited 466 ms on lock 0x7b54001bba50. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:31.886318 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1072078720}
I20260709 13:09:31.889128 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:31.891868 22947 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:09:31.889170 23022 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33418 (request call id 1862) took 1889 ms (client timeout 1504 ms). Trace:
W20260709 13:09:31.893312 23022 rpcz_store.cc:269] 0709 13:09:29.999577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:30.002824 (+  3247us) service_pool.cc:224] Handling call
0709 13:09:31.888920 (+1886096us) spinlock_profiling.cc:243] Waited 1.72 s on lock 0x7b540010ca54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:31.889076 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3959892992}
I20260709 13:09:31.898770 23021 raft_consensus.cc:1240] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 13:09:31.899541 22877 raft_consensus.cc:1275] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:09:31.817408 26889 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:31.900300 22918 leader_election.cc:336] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.959s (SENT)
I20260709 13:09:31.909806 22918 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:31.902216 22920 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:31.903836 22871 raft_consensus.cc:1240] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 10. Current term is 11. Ops: [7.6-10.8]
I20260709 13:09:31.909335 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:09:31.912117 23029 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33418 (request call id 1842) took 5497 ms (client timeout 1892 ms). Trace:
I20260709 13:09:31.914129 26762 consensus_queue.cc:1048] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:09:31.916514 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:31.921006 22949 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58320 (request call id 2562) took 2279 ms (client timeout 1741 ms). Trace:
W20260709 13:09:31.921607 22949 rpcz_store.cc:269] 0709 13:09:29.632557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:29.632742 (+   185us) service_pool.cc:224] Handling call
0709 13:09:31.911289 (+2278547us) spinlock_profiling.cc:243] Waited 2.11 s on lock 0x7b5400149954. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:31.911411 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4863423360}
W20260709 13:09:31.912643 23029 rpcz_store.cc:269] 0709 13:09:26.414570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:26.414718 (+   148us) service_pool.cc:224] Handling call
0709 13:09:31.911927 (+5497209us) spinlock_profiling.cc:243] Waited 5.36 s on lock 0x7b54002c9054. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:31.912037 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12335006976}
I20260709 13:09:31.875576 26808 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:31.841810 26741 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:31.865681 26789 consensus_queue.cc:1048] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:09:31.901155 26695 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:31.928098 22872 raft_consensus.cc:1240] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 8. Current term is 9. Ops: []
W20260709 13:09:31.929210 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:31.930181 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:31.898425 26895 raft_consensus.cc:2804] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:09:31.897843 26802 raft_consensus.cc:2804] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:09:31.858418 26805 consensus_queue.cc:1059] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
W20260709 13:09:31.932608 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:31.882458 26347 consensus_queue.cc:260] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:31.874634 26675 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:31.954754 22918 leader_election.cc:336] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 2.011s (SENT)
I20260709 13:09:31.896981 26788 consensus_queue.cc:1048] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 13:09:31.874786 26766 consensus_queue.cc:1059] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 13:09:31.875322 26407 consensus_queue.cc:1048] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:09:31.898622 26612 consensus_queue.cc:1048] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:09:31.930414 26778 raft_consensus.cc:2804] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:31.946300 26755 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:31.930137 26808 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:09:31.969262 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 413376us
I20260709 13:09:31.926743 22876 raft_consensus.cc:1275] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 13:09:31.972195 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:31.952046 26803 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:09:31.997161 23621 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60044 (request call id 735) took 2363 ms (client timeout 1755 ms). Trace:
W20260709 13:09:31.997694 23621 rpcz_store.cc:269] 0709 13:09:29.633978 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:29.643719 (+  9741us) service_pool.cc:224] Handling call
0709 13:09:31.996912 (+2353193us) spinlock_profiling.cc:243] Waited 330 ms on lock 0x7b540046a854. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:31.997087 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":758486272}
W20260709 13:09:31.977957 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:31.936762 22920 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:32.001298 23622 raft_consensus.cc:1275] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:09:31.956406 26895 raft_consensus.cc:697] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:31.961162 26743 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:09:32.003116 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 66473us
W20260709 13:09:32.002306 22843 leader_election.cc:336] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.958s (SENT)
I20260709 13:09:32.014009 22843 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:31.957681 26843 consensus_queue.cc:1048] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.046s
I20260709 13:09:32.014170 22920 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:31.966820 26802 raft_consensus.cc:697] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:31.976089 26778 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:32.020402 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:32.021065 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 68009.9us
I20260709 13:09:31.997318 26803 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:32.000790 26813 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:31.963387 26849 consensus_queue.cc:1059] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 13:09:32.001235 26755 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:32.017513 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 162947us
I20260709 13:09:32.004289 26799 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:32.028834 23588 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:32.017038 26887 raft_consensus.cc:2804] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:32.058297 22947 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
I20260709 13:09:32.061757 22954 raft_consensus.cc:2410] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:09:32.062433 22992 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:09:32.063805 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:31.966506 26236 consensus_queue.cc:1175] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "5b346767a89745fa82928a5dac9cc0b2" responder_term: 11 status { last_received { term: 11 index: 10 } last_committed_idx: 9 last_received_current_leader { term: 11 index: 10 } }
I20260709 13:09:32.023232 26778 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:32.083658 26792 raft_consensus.cc:2804] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:32.084340 26792 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:32.086210 26888 raft_consensus.cc:3055] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:09:32.086273 26792 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:09:32.070513 26933 log.cc:927] Time spent T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:09:32.087584 26577 raft_consensus.cc:3055] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:09:32.088068 26577 raft_consensus.cc:740] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:32.088939 26577 consensus_queue.cc:260] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:32.089607 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:32.090291 26577 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:32.071322 26938 raft_consensus.cc:2749] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:09:32.071326 26930 log.cc:927] Time spent T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:09:32.093189 26895 consensus_queue.cc:237] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:32.095193 26788 consensus_queue.cc:1059] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:09:32.072296 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 43531.2us
I20260709 13:09:32.028725 26803 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:32.075991 26764 raft_consensus.cc:3055] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:09:32.086769 26888 raft_consensus.cc:740] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
W20260709 13:09:32.072395 26928 log.cc:927] Time spent T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 13:09:32.097116 22918 leader_election.cc:336] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 2.072s (SENT)
W20260709 13:09:32.121189 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:32.097993 23588 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:32.064029 26799 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:32.064446 26794 raft_consensus.cc:2749] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:09:32.068178 26889 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:32.117486 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:32.085522 26929 log.cc:927] Time spent T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:09:32.124782 23588 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:32.137307 23588 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:32.074000 26407 raft_consensus.cc:2804] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:09:32.084492 26675 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:09:32.127974 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 113859us
W20260709 13:09:32.132302 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 110483us
I20260709 13:09:32.061304 26813 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:32.096653 26851 raft_consensus.cc:2804] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:09:32.142396 26612 raft_consensus.cc:2749] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:09:32.156414 26851 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:32.157213 22919 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:32.161506 26813 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:32.164515 22872 raft_consensus.cc:1240] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 9. Current term is 10. Ops: []
W20260709 13:09:32.164502 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 644000us
I20260709 13:09:32.165681 22994 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:32.167248 22994 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:32.104311 26811 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:32.111196 26764 raft_consensus.cc:740] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:32.173671 26764 consensus_queue.cc:260] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:32.174675 26894 raft_consensus.cc:2749] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:32.176908 26670 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:32.179422 26670 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:32.066556 26853 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:32.160151 26851 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:32.174826 26764 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:32.104204 26618 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:32.174700 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 172396us
W20260709 13:09:32.183470 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 46264.2us
I20260709 13:09:32.206106 23588 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:32.060840 26755 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:32.158085 26407 raft_consensus.cc:697] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:32.158766 26888 consensus_queue.cc:260] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:32.144429 26945 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
W20260709 13:09:32.153473 26931 log.cc:927] Time spent T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 13:09:32.209107 22843 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:09:32.210040 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 52905.8us
I20260709 13:09:32.241577 22919 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:32.243052 22919 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:32.179960 26741 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:32.244081 22919 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:09:32.154287 26936 log.cc:927] Time spent T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 13:09:32.135661 26938 raft_consensus.cc:2804] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:09:32.249536 26938 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:32.249954 26938 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:32.213847 26618 consensus_queue.cc:1048] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 13:09:32.219455 26755 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:32.220187 26407 consensus_queue.cc:237] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:32.221961 26870 raft_consensus.cc:2749] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:09:32.222398 26236 consensus_queue.cc:1048] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:09:32.225587 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:32.172472 26811 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:32.227342 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:32.152770 26758 raft_consensus.cc:2804] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:32.215809 26492 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:32.205462 26756 raft_consensus.cc:2749] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:32.230561 26458 raft_consensus.cc:2749] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:09:32.230850 26861 consensus_queue.cc:1059] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:09:32.231868 26888 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:32.154439 26884 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:32.235281 26803 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:32.236756 26945 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:32.238078 26799 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:32.244820 22946 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:49044
I20260709 13:09:32.245106 26943 raft_consensus.cc:2804] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:32.244869 22949 raft_consensus.cc:1240] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:32.243924 26792 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:32.245869 26853 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:32.245826 26819 consensus_queue.cc:1059] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:09:32.226856 26935 raft_consensus.cc:2749] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:09:32.247423 22843 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:32.189766 22994 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:32.265942 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:32.320101 22874 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:32.266415 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:32.275955 26492 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:32.269075 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:32.272989 22879 raft_consensus.cc:1275] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:09:32.271291 22953 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
I20260709 13:09:32.274600 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:32.330808 23029 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:32.192044 26626 raft_consensus.cc:2749] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:32.278263 23625 raft_consensus.cc:1275] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 13:09:32.282851 22948 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58310 (request call id 1892) took 2465 ms (client timeout 1598 ms). Trace:
W20260709 13:09:32.331908 22948 rpcz_store.cc:269] 0709 13:09:29.817054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:29.817173 (+   119us) service_pool.cc:224] Handling call
0709 13:09:32.282622 (+2465449us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54003c7154. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:32.282787 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2758832000}
I20260709 13:09:32.281329 23617 raft_consensus.cc:1275] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:09:32.281978 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:32.333494 23621 raft_consensus.cc:3060] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:32.308866 26908 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:32.273723 26843 raft_consensus.cc:2749] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:32.268198 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:09:32.325582 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 135877us
W20260709 13:09:32.326591 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 79223.2us
I20260709 13:09:32.326848 22874 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 12.
I20260709 13:09:32.342438 22994 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:32.279925 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:32.343245 23618 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:32.135192 26802 consensus_queue.cc:237] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:32.279644 26947 raft_consensus.cc:2804] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:32.314980 26943 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:32.337002 22950 raft_consensus.cc:1240] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:32.271523 26778 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:32.363227 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:32.372162 22950 raft_consensus.cc:1275] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:09:32.354717 22994 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:32.373917 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:32.333536 26492 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:32.299526 26851 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:32.327050 26756 raft_consensus.cc:2749] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:09:32.341331 26894 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:32.393322 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:32.379422 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:32.374462 22953 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:32.394215 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:32.378479 22918 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:32.395756 23026 raft_consensus.cc:1240] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 6. Current term is 7. Ops: []
I20260709 13:09:32.384985 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:32.334597 26755 raft_consensus.cc:2804] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:09:32.389257 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 178248us
I20260709 13:09:32.390022 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:32.391593 23022 raft_consensus.cc:1240] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:32.420360 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:32.420632 22920 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:32.396536 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:32.348515 26947 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:32.422864 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:32.373620 26743 raft_consensus.cc:2804] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:32.384341 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:32.426026 22872 raft_consensus.cc:2376] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10. Re-sending same reply.
I20260709 13:09:32.421141 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:32.421542 23020 raft_consensus.cc:2410] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 11 because replica has last-logged OpId of term: 9 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:09:32.426754 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:32.421607 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:32.427327 23026 raft_consensus.cc:2376] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11. Re-sending same reply.
I20260709 13:09:32.396466 22873 raft_consensus.cc:2468] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
I20260709 13:09:32.388525 26626 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:32.345158 26935 consensus_queue.cc:1048] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.078s
I20260709 13:09:32.431326 26803 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:32.431906 26852 consensus_meta.cc:287] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.621s	user 0.011s	sys 0.000s
I20260709 13:09:32.377775 26868 consensus_queue.cc:1048] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:09:32.417826 26755 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:32.432922 26755 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:32.379338 26849 consensus_queue.cc:1048] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 13:09:32.425596 26947 raft_consensus.cc:3060] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:32.440377 26799 raft_consensus.cc:3055] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:09:32.425823 26743 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:32.440814 26799 raft_consensus.cc:740] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:32.441110 26743 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:32.422063 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:32.442682 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:32.443397 23023 raft_consensus.cc:2393] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 12: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:32.444114 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:32.445806 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:32.448642 23023 raft_consensus.cc:2410] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:09:32.457593 26792 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:32.457602 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:32.458964 23023 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 9.
I20260709 13:09:32.459975 26778 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:32.460033 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:32.460723 26870 consensus_queue.cc:1059] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:09:32.428694 26949 raft_consensus.cc:2749] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:32.445986 26945 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:32.447896 22947 raft_consensus.cc:1240] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:32.463404 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:32.429596 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:32.464650 23026 raft_consensus.cc:2376] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11. Re-sending same reply.
I20260709 13:09:32.448938 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:32.449987 23621 raft_consensus.cc:2468] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
W20260709 13:09:32.451647 26781 consensus_meta.cc:287] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.541s	user 0.006s	sys 0.004s
I20260709 13:09:32.466661 26781 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:09:32.467800 22954 raft_consensus.cc:1240] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:32.374049 26618 consensus_queue.cc:1048] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260709 13:09:32.429061 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:32.452239 26941 log.cc:927] Time spent T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.380s	user 0.004s	sys 0.000s
I20260709 13:09:32.461517 26870 raft_consensus.cc:3055] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:09:32.468811 22946 raft_consensus.cc:606] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Received request to transfer leadership
I20260709 13:09:32.470424 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:32.483446 22946 raft_consensus.cc:614] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:32.430589 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:32.473837 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:32.376083 26943 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:32.474305 23618 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
I20260709 13:09:32.479097 23029 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 12.
I20260709 13:09:32.430039 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:32.480793 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 85198.7us
I20260709 13:09:32.347239 26908 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:32.348789 26819 consensus_queue.cc:1048] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.081s
I20260709 13:09:32.486286 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:32.486248 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:32.515111 22873 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:09:32.516429 23021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 1925) took 1510 ms. Trace:
I20260709 13:09:32.506111 23028 raft_consensus.cc:2393] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 11: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
W20260709 13:09:32.469736 26950 log.cc:927] Time spent T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 13:09:32.418139 26236 raft_consensus.cc:2749] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:32.516865 23021 rpcz_store.cc:276] 0709 13:09:31.005746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:31.245410 (+239664us) service_pool.cc:224] Handling call
0709 13:09:31.889262 (+643852us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:31.902132 (+ 12870us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:32.516247 (+614115us) spinlock_profiling.cc:243] Waited 585 ms on lock 0x7b540010ca50. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:32.516346 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1346159360}
I20260709 13:09:32.482259 26849 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:32.468640 26938 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:32.511279 26492 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:32.522933 26851 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:32.524931 22954 raft_consensus.cc:1240] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 6. Current term is 7. Ops: []
I20260709 13:09:32.525370 26803 consensus_queue.cc:1048] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:09:32.528980 23023 raft_consensus.cc:2410] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 12 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 9.
I20260709 13:09:32.530391 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:32.469178 26884 consensus_queue.cc:1048] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:09:32.542454 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:32.543344 23021 raft_consensus.cc:2393] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 7: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:32.483223 26870 raft_consensus.cc:740] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:32.551102 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:32.554639 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:32.573109 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:32.573345 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:32.574544 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:32.574601 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:32.575798 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:32.576565 22948 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
I20260709 13:09:32.577582 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:32.569968 23021 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
W20260709 13:09:32.579927 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 322389us
I20260709 13:09:32.583709 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:32.583756 23618 raft_consensus.cc:1240] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:32.583802 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:32.578214 22948 raft_consensus.cc:2376] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11. Re-sending same reply.
I20260709 13:09:32.581990 22843 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:32.588330 22843 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:32.593278 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:32.594683 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:32.594986 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:32.595244 22878 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:32.595458 23021 raft_consensus.cc:1275] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:09:32.596880 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:32.589702 22843 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:32.598428 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:32.599231 22874 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:32.600075 23625 raft_consensus.cc:1240] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:32.536986 26849 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:32.571174 26781 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:32.607035 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:32.611486 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:32.615825 26849 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:32.615682 26743 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:32.618045 26743 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:32.618141 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:32.618940 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:32.621567 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:32.633538 26954 log.cc:927] Time spent T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:09:32.634752 26908 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:32.635694 26947 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:32.636281 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:32.637055 22876 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:32.599915 26760 raft_consensus.cc:2804] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:09:32.640249 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:32.650210 22948 raft_consensus.cc:1240] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:32.600538 26889 raft_consensus.cc:2749] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:09:32.681593 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:09:32.621733 26808 consensus_meta.cc:287] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.613s	user 0.006s	sys 0.005s
I20260709 13:09:32.873632 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:32.778630 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:32.880137 22879 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:09:32.735278 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:32.871311 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:32.875813 22953 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 12.
I20260709 13:09:32.876952 22874 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 9.
I20260709 13:09:32.891933 22920 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:32.796033 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:32.691655 22878 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11.
I20260709 13:09:32.878481 23617 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
I20260709 13:09:32.903522 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:32.912463 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:32.913517 23618 raft_consensus.cc:2410] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:09:32.919344 22879 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
W20260709 13:09:32.637224 26879 consensus_meta.cc:287] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.787s	user 0.011s	sys 0.000s
I20260709 13:09:32.921602 22920 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:32.961097 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:32.962680 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:32.963503 22879 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
W20260709 13:09:32.965042 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 43551.5us
I20260709 13:09:32.966472 22920 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:32.964610 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:32.968336 22879 raft_consensus.cc:2393] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 11: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:32.976225 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:32.976110 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:32.978698 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:32.979383 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:32.853799 26846 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:32.729605 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 62808.5us
I20260709 13:09:32.986274 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:32.989537 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:32.609033 26819 raft_consensus.cc:2804] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:09:32.989605 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:32.995229 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:32.998423 22879 raft_consensus.cc:2393] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 10: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:32.964951 22767 maintenance_manager.cc:419] P 8d175b4ae72f4fc48971c2f31668971c: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033374
I20260709 13:09:33.001992 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:32.999753 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:33.008769 22880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 1917) took 3328 ms. Trace:
I20260709 13:09:33.012470 22879 raft_consensus.cc:2393] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 12: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:32.960137 26949 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:09:32.960788 26852 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:33.009392 22880 rpcz_store.cc:276] 0709 13:09:29.680386 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:30.034236 (+353850us) service_pool.cc:224] Handling call
0709 13:09:30.082654 (+ 48418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:31.785586 (+1702932us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:09:31.794038 (+  8452us) log.cc:844] Serialized 14 byte log entry
0709 13:09:31.925859 (+131821us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:31.925911 (+    52us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:09:31.925962 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:32.438794 (+512832us) spinlock_profiling.cc:243] Waited 2.07 ms on lock 0x7b4800002958. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613ad0eaa1 00007f613ad05b37 00007f613ad059ca 00007f613adfd26f 00007f613adfcf37 00007f613ae1155c 00007f613ae11508 00007f613ae11488 00007f613ae11443 00007f613ae106ec 00007f6139d3aa96
0709 13:09:33.008482 (+569688us) spinlock_profiling.cc:243] Waited 1.7 s on lock 0x7b5400369554. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:33.008545 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:33.008705 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56108,"prepare.queue_time_us":8691,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":23572,"raft.run_cpu_time_us":3370,"raft.run_wall_time_us":505096,"spinlock_wait_cycles":3920455296,"thread_start_us":108397,"threads_started":1,"wal-append.queue_time_us":159406}
I20260709 13:09:32.666944 26870 consensus_queue.cc:260] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:32.666956 26549 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:32.667835 26760 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:32.983374 26955 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:09:33.027705 22954 raft_consensus.cc:1240] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 13:09:33.034598 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:32.989215 26908 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:32.996295 26803 raft_consensus.cc:2804] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:09:33.037902 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:32.986147 26492 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:33.039637 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:33.048650 22878 raft_consensus.cc:2393] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 12: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:33.042598 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:33.049662 22879 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
I20260709 13:09:33.051180 22876 raft_consensus.cc:2468] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
I20260709 13:09:32.955433 26808 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:33.058077 26870 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:33.040740 26949 raft_consensus.cc:697] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
W20260709 13:09:33.041249 26953 log.cc:927] Time spent T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.486s	user 0.000s	sys 0.002s
I20260709 13:09:33.019069 26819 raft_consensus.cc:697] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 11 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:33.065979 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:33.084144 23029 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11.
I20260709 13:09:33.040992 26760 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:09:33.069980 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:33.003891 26846 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:33.062508 26626 consensus_meta.cc:287] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.598s	user 0.008s	sys 0.001s
I20260709 13:09:33.077740 23021 raft_consensus.cc:1275] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:09:33.074580 22878 raft_consensus.cc:1240] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:09:33.058928 26803 raft_consensus.cc:697] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:32.997615 26853 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:33.038748 26852 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:33.052657 26938 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:33.054047 26908 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:33.071046 26955 raft_consensus.cc:697] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:33.069309 26886 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:33.105099 23624 raft_consensus.cc:1240] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 10. Current term is 11. Ops: [8.8-10.9]
I20260709 13:09:33.044564 26549 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:33.113299 26852 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:33.114292 26549 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:33.116271 26908 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:33.103228 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:33.108121 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
W20260709 13:09:33.107546 26958 log.cc:927] Time spent T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.440s	user 0.001s	sys 0.000s
I20260709 13:09:33.111963 26886 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:33.119853 22874 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:33.094838 26626 raft_consensus.cc:2749] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:09:33.117759 26949 consensus_queue.cc:237] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:33.102241 26853 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:33.293275 23623 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 10.
W20260709 13:09:33.212229 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:33.659121 23623 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 796) took 1041 ms. Trace:
I20260709 13:09:33.137602 26780 raft_consensus.cc:3055] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:09:33.341984 22874 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
W20260709 13:09:33.655288 26943 consensus_meta.cc:287] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 1.137s	user 0.011s	sys 0.000s
W20260709 13:09:33.302898 26956 log.cc:927] Time spent T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.179s	user 0.001s	sys 0.000s
I20260709 13:09:33.661715 26943 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:33.358094 26755 consensus_meta.cc:287] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.925s	user 0.008s	sys 0.004s
I20260709 13:09:33.662967 26755 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:33.664347 22878 raft_consensus.cc:1240] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:09:33.665831 26755 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:33.577519 26886 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:33.340145 26549 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:33.660373 26780 raft_consensus.cc:740] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:33.307577 26955 consensus_queue.cc:237] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:09:33.137002 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:33.679207 23021 raft_consensus.cc:2393] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 10: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:33.242590 26803 consensus_queue.cc:237] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:33.138150 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:33.219419 22871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53724 (request call id 645) took 1057 ms. Trace:
W20260709 13:09:33.657420 26888 consensus_meta.cc:287] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 1.345s	user 0.010s	sys 0.003s
I20260709 13:09:33.681952 22947 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:09:33.671914 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:33.672495 26626 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:33.125859 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:33.682140 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:33.270761 23028 raft_consensus.cc:1275] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:09:33.109231 26947 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:33.138338 22877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2408) took 3708 ms. Trace:
W20260709 13:09:33.542696 23588 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.003s	sys 0.015s
I20260709 13:09:33.680394 22871 rpcz_store.cc:276] 0709 13:09:32.161720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:32.162832 (+  1112us) service_pool.cc:224] Handling call
0709 13:09:32.163170 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:32.176158 (+ 12988us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:32.176186 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:32.176700 (+   514us) log.cc:844] Serialized 26 byte log entry
0709 13:09:32.337536 (+160836us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:32.337582 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:32.337700 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:33.219177 (+881477us) raft_consensus.cc:1685] finished
0709 13:09:33.219204 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:33.219359 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12600,"raft.queue_time_us":13926,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":94,"thread_start_us":151428,"threads_started":1,"wal-append.queue_time_us":309197}
I20260709 13:09:33.683492 23622 raft_consensus.cc:2410] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 13:09:33.659591 23623 rpcz_store.cc:276] 0709 13:09:32.622656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:32.639519 (+ 16863us) service_pool.cc:224] Handling call
0709 13:09:33.659063 (+1019544us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:33.673800 26762 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:33.684751 22950 raft_consensus.cc:1275] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 13:09:33.127552 26846 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:33.138787 26940 raft_consensus.cc:3055] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:09:33.362293 26808 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:33.684985 26626 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:33.692713 22951 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 627) took 1434 ms. Trace:
I20260709 13:09:33.693081 22951 rpcz_store.cc:276] 0709 13:09:32.258030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:32.258173 (+   143us) service_pool.cc:224] Handling call
0709 13:09:32.258443 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:32.333512 (+ 75069us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:32.333558 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:32.333618 (+    60us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:32.333671 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:33.692476 (+1358805us) spinlock_profiling.cc:243] Waited 74.8 ms on lock 0x7b54003af054. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:33.692520 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:33.692640 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":172059008}
I20260709 13:09:33.694344 26626 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:33.686918 26780 consensus_queue.cc:260] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:33.696134 26870 consensus_meta.cc:287] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.612s	user 0.006s	sys 0.001s
I20260709 13:09:33.586580 26760 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:33.697027 26780 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:33.698287 26760 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:33.685139 22877 rpcz_store.cc:276] 0709 13:09:29.429699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:29.568100 (+138401us) service_pool.cc:224] Handling call
0709 13:09:31.759516 (+2191416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:31.899955 (+140439us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:32.176981 (+277026us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b54002882d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:33.138133 (+961152us) spinlock_profiling.cc:243] Waited 2.19 s on lock 0x7b54002882d0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:33.138279 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5327248896}
W20260709 13:09:33.685686 23588 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.004s	sys 0.016s
I20260709 13:09:33.688442 26853 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:33.664297 26943 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:33.690395 22950 pending_rounds.cc:85] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting all ops after (but not including) 6
I20260709 13:09:33.705981 22950 pending_rounds.cc:107] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 13:09:33.706333 22950 raft_consensus.cc:2889] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:33.706717 22950 pending_rounds.cc:107] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting uncommitted NO_OP operation due to leader change: 10.8
W20260709 13:09:33.701282 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 695575us
I20260709 13:09:33.701683 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:33.681799 26781 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:33.706846 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:33.707310 22950 raft_consensus.cc:2889] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:33.709345 26781 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:33.705205 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:33.710492 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:33.710827 23028 raft_consensus.cc:2393] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 10: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:33.711057 23622 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:33.711616 26781 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:33.710497 22951 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11.
I20260709 13:09:33.713588 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:33.715600 22877 raft_consensus.cc:1240] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 6. Current term is 7. Ops: []
I20260709 13:09:33.716101 22877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2436) took 1329 ms. Trace:
I20260709 13:09:33.716339 22877 rpcz_store.cc:276] 0709 13:09:32.386860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:33.699671 (+1312811us) service_pool.cc:224] Handling call
0709 13:09:33.715484 (+ 15813us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:33.715940 (+   456us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:33.716061 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:33.718092 23588 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:33.142115 26819 consensus_queue.cc:237] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:33.726321 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:33.726670 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:33.700937 26940 raft_consensus.cc:740] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:33.719015 26887 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:33.679917 26549 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:33.727481 23622 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11.
I20260709 13:09:33.728559 22877 raft_consensus.cc:1240] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:33.729120 22877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2439) took 1333 ms. Trace:
I20260709 13:09:33.729393 22877 rpcz_store.cc:276] 0709 13:09:32.396094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:33.716967 (+1320873us) service_pool.cc:224] Handling call
0709 13:09:33.728414 (+ 11447us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:33.728960 (+   546us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:33.729082 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:33.730911 22877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2440) took 1332 ms. Trace:
I20260709 13:09:33.731271 22877 rpcz_store.cc:276] 0709 13:09:32.398565 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:33.730169 (+1331604us) service_pool.cc:224] Handling call
0709 13:09:33.730421 (+   252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:33.730568 (+   147us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:33.730591 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:33.730629 (+    38us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:09:33.730716 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:33.730774 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:33.730862 (+    88us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:33.732395 22877 raft_consensus.cc:1240] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:33.732864 22877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2442) took 1315 ms. Trace:
I20260709 13:09:33.733459 22877 rpcz_store.cc:276] 0709 13:09:32.417883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:33.732079 (+1314196us) service_pool.cc:224] Handling call
0709 13:09:33.732294 (+   215us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:33.732648 (+   354us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:33.732821 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:09:33.734102 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:33.734747 22919 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:33.733129 26886 raft_consensus.cc:2804] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:09:33.736192 26886 raft_consensus.cc:697] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:33.718981 26966 consensus_queue.cc:1048] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.018s
I20260709 13:09:33.737102 26886 consensus_queue.cc:237] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:33.739845 22919 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:33.744936 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:33.745515 23021 raft_consensus.cc:1275] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:09:33.698936 26762 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:33.748739 22919 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:33.743850 26626 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:33.743680 26969 raft_consensus.cc:2749] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:33.720046 26808 consensus_queue.cc:1048] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.027s
I20260709 13:09:33.729715 26887 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:33.758098 26778 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:33.758607 22954 raft_consensus.cc:1275] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:09:33.749605 26966 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:33.763057 23025 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 12.
I20260709 13:09:33.758522 26894 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:33.758374 26765 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:33.765108 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:33.762653 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:33.767656 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:33.768164 22876 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11.
I20260709 13:09:33.769111 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:33.753610 26943 raft_consensus.cc:2804] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:09:33.769578 22876 raft_consensus.cc:2393] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 10: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:33.750144 26970 raft_consensus.cc:2804] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:09:33.766531 26894 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:33.772957 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:33.774148 22876 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:09:33.774850 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:33.775355 22876 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
W20260709 13:09:33.770979 26926 log.cc:927] Time spent T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 1.810s	user 0.018s	sys 0.022s
I20260709 13:09:33.776072 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:33.766811 26966 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:33.764237 26626 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:33.775909 26943 raft_consensus.cc:697] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:33.770527 26765 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:33.763998 26778 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:33.776192 26970 raft_consensus.cc:697] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:33.788281 26966 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:33.796713 22843 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:33.797199 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:33.799777 22953 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:33.840315 22994 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:33.765933 26940 consensus_queue.cc:260] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:33.792788 26943 consensus_queue.cc:237] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:33.899462 26975 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:33.900810 26778 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:33.902108 26626 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:33.902335 26765 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:33.902710 26626 raft_consensus.cc:2749] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:09:33.894292 26967 log.cc:927] Time spent T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.174s	user 0.000s	sys 0.000s
I20260709 13:09:33.905215 22953 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 12.
I20260709 13:09:33.802536 26755 consensus_queue.cc:1059] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.104s
I20260709 13:09:33.802030 26760 consensus_queue.cc:1059] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.105s
I20260709 13:09:33.900134 26975 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:33.926107 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:33.927181 22953 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:09:33.919307 26819 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:33.927964 26755 consensus_queue.cc:1243] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer f3c799b1249e40119d2a5864231c6c75 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 13:09:33.927124 26549 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:33.924942 26778 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:33.934993 26843 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:33.934660 26778 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:33.935606 26843 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:33.937572 26778 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:33.937664 26843 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:33.938699 26755 raft_consensus.cc:3055] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:09:33.939257 26755 raft_consensus.cc:740] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:33.938719 26970 consensus_queue.cc:237] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:33.941108 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:33.936506 22946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:49044 (request call id 0) took 1692 ms. Trace:
I20260709 13:09:33.948802 22946 rpcz_store.cc:276] 0709 13:09:32.244522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:32.244683 (+   161us) service_pool.cc:224] Handling call
0709 13:09:33.936318 (+1691635us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b5400283f54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135ddf853 00007f613adad60a 00007f61343337d4 00007f6134333762 00007f6134333691 00007f613433360c 00007f61343328c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:33.936431 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":355944192}
I20260709 13:09:33.940380 23588 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:33.926649 26760 raft_consensus.cc:3055] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:09:33.953212 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:33.955950 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:33.956799 22953 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
I20260709 13:09:33.951663 26843 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:33.960894 22876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 1951) took 1533 ms. Trace:
I20260709 13:09:33.976328 22876 rpcz_store.cc:276] 0709 13:09:32.427612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:33.782351 (+1354739us) service_pool.cc:224] Handling call
0709 13:09:33.782696 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:33.785250 (+  2554us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:33.785280 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:33.799057 (+ 13777us) log.cc:844] Serialized 26 byte log entry
0709 13:09:33.842259 (+ 43202us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:09:33.842312 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:33.842406 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:33.960638 (+118232us) raft_consensus.cc:1685] finished
0709 13:09:33.960657 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:33.960826 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":81,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":2036,"thread_start_us":2609,"threads_started":1,"wal-append.queue_time_us":94311}
I20260709 13:09:33.979334 22876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 1954) took 1403 ms. Trace:
W20260709 13:09:33.966322 26972 log.cc:927] Time spent T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:09:33.950295 26755 consensus_queue.cc:260] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:33.927668 26811 consensus_queue.cc:1048] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 13:09:33.955204 26760 raft_consensus.cc:740] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:33.979797 22876 rpcz_store.cc:276] 0709 13:09:32.576116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:33.978304 (+1402188us) service_pool.cc:224] Handling call
0709 13:09:33.978701 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:33.978915 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:33.978953 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:33.979011 (+    58us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:33.979066 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:33.979138 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:33.979269 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:33.987047 26762 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:33.988900 26975 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:33.989933 26944 raft_consensus.cc:3055] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:09:33.990347 26944 raft_consensus.cc:740] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:33.991125 26944 consensus_queue.cc:260] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:33.997006 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:33.985127 26843 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:33.998474 23024 raft_consensus.cc:1275] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:09:34.000046 26843 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:33.972611 26894 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:34.001253 22950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 631) took 1742 ms. Trace:
I20260709 13:09:34.001622 22950 rpcz_store.cc:276] 0709 13:09:32.259051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:32.373119 (+114068us) service_pool.cc:224] Handling call
0709 13:09:32.395859 (+ 22740us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:33.709580 (+1313721us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:34.001053 (+291473us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b5400148554. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:34.001199 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2963297024}
I20260709 13:09:34.002399 26894 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:34.003108 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:33.984581 26976 raft_consensus.cc:2749] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:34.016791 26755 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 7
W20260709 13:09:34.018047 26979 log.cc:927] Time spent T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 13:09:34.018218 26960 log.cc:927] Time spent T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.971s	user 0.000s	sys 0.017s
I20260709 13:09:34.019199 22877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 1949) took 1592 ms. Trace:
I20260709 13:09:34.019399 23616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60044 (request call id 751) took 1574 ms. Trace:
I20260709 13:09:34.019623 22877 rpcz_store.cc:276] 0709 13:09:32.426999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:33.734226 (+1307227us) service_pool.cc:224] Handling call
0709 13:09:33.734484 (+   258us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:33.734892 (+   408us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:33.734919 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:33.735393 (+   474us) log.cc:844] Serialized 26 byte log entry
0709 13:09:33.920099 (+184706us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:09:33.920149 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:33.920228 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:34.018960 (+ 98732us) raft_consensus.cc:1685] finished
0709 13:09:34.018985 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:34.019132 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6280,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":4011,"thread_start_us":162678,"threads_started":1,"wal-append.queue_time_us":201061}
I20260709 13:09:34.019860 23616 rpcz_store.cc:276] 0709 13:09:32.445375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:32.449370 (+  3995us) service_pool.cc:224] Handling call
0709 13:09:32.449654 (+   284us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:32.450006 (+   352us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:32.450030 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:32.486876 (+ 36846us) log.cc:844] Serialized 26 byte log entry
0709 13:09:32.679408 (+192532us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:09:32.679465 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:32.679566 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:33.139222 (+459656us) spinlock_profiling.cc:243] Waited 2.47 ms on lock 0x7b48001be658. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613ad0eaa1 00007f613ad05b37 00007f613ad059ca 00007f613adfd26f 00007f613adfcf37 00007f613ae1155c 00007f613ae11508 00007f613ae11488 00007f613ae11443 00007f613ae106ec 00007f6139d3aa96
0709 13:09:34.017716 (+878494us) spinlock_profiling.cc:243] Waited 1.41 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:09:34.019147 (+  1431us) raft_consensus.cc:1685] finished
0709 13:09:34.019165 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:34.019302 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":21458,"raft.run_cpu_time_us":46841,"raft.run_wall_time_us":657916,"spinlock_wait_cycles":8936448,"thread_start_us":143821,"threads_started":1,"wal-append.queue_time_us":526936}
I20260709 13:09:34.023234 22877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 1960) took 1422 ms. Trace:
I20260709 13:09:34.023705 22877 rpcz_store.cc:276] 0709 13:09:32.600645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:34.021557 (+1420912us) service_pool.cc:224] Handling call
0709 13:09:34.021825 (+   268us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:34.021995 (+   170us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:34.022886 (+   891us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:34.022934 (+    48us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:09:34.022979 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:34.023044 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:34.023162 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:34.025445 22877 raft_consensus.cc:1240] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:34.026877 26887 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:34.027770 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 187547us
I20260709 13:09:34.028375 23588 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:34.029945 22994 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:09:34.032104 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 235490us
I20260709 13:09:34.033118 22843 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:34.033416 26626 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:34.017179 26760 consensus_queue.cc:260] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:34.034893 26626 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:34.035496 26760 raft_consensus.cc:3060] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:34.035633 26347 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:34.036988 26626 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:34.022753 26675 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:34.038972 26675 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:34.039535 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:34.040161 22950 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
I20260709 13:09:34.025876 26780 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:34.042375 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:34.042092 22843 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:34.044108 22953 raft_consensus.cc:1275] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 13:09:34.044718 22879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 1952) took 1616 ms. Trace:
I20260709 13:09:34.045483 22871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2431) took 1659 ms. Trace:
I20260709 13:09:34.046882 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:34.047389 22994 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:34.041194 26780 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:34.053491 23619 raft_consensus.cc:1240] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:34.053783 22843 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:34.054742 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:34.055154 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:34.056440 22950 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 13.
I20260709 13:09:34.046501 26811 raft_consensus.cc:2804] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:09:34.056397 22843 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:34.049994 26985 raft_consensus.cc:2804] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 13:09:34.045367 26989 raft_consensus.cc:2749] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:09:34.047575 26549 raft_consensus.cc:2804] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:09:34.087998 26675 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:34.089025 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:34.094672 22953 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 8.
I20260709 13:09:34.095193 26780 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:34.096766 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:34.057623 26819 raft_consensus.cc:2749] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:09:34.088274 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:34.033452 26976 consensus_queue.cc:1048] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:09:34.053417 26975 raft_consensus.cc:2749] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:34.091485 26985 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:34.088617 23588 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:34.048995 26938 consensus_queue.cc:1048] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 13:09:34.097489 22953 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 13.
W20260709 13:09:34.104319 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 48034.9us
I20260709 13:09:34.105038 22843 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:34.105706 23588 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:34.106410 22843 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:34.050386 22879 rpcz_store.cc:276] 0709 13:09:32.427911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:33.937052 (+1509141us) service_pool.cc:224] Handling call
0709 13:09:33.937365 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:33.937684 (+   319us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:33.937714 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:33.938120 (+   406us) log.cc:844] Serialized 26 byte log entry
0709 13:09:34.030973 (+ 92853us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:09:34.031026 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:34.031163 (+   137us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:34.044540 (+ 13377us) raft_consensus.cc:1685] finished
0709 13:09:34.044557 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:34.044653 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":154,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":108,"thread_start_us":92417,"threads_started":1,"wal-append.queue_time_us":91417}
I20260709 13:09:34.051074 22871 rpcz_store.cc:276] 0709 13:09:32.385546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:33.690854 (+1305308us) service_pool.cc:224] Handling call
0709 13:09:33.699159 (+  8305us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:33.699779 (+   620us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:33.699799 (+    20us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:33.700300 (+   501us) log.cc:844] Serialized 52 byte log entry
0709 13:09:33.839667 (+139367us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:09:34.027318 (+187651us) log.cc:844] Serialized 14 byte log entry
0709 13:09:34.043587 (+ 16269us) log.cc:844] Serialized 14 byte log entry
0709 13:09:34.045177 (+  1590us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:34.045267 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:34.045314 (+    47us) raft_consensus.cc:1685] finished
0709 13:09:34.045324 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:34.045435 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15320,"prepare.queue_time_us":213453,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":66,"raft.queue_time_us":5994,"raft.run_cpu_time_us":300,"raft.run_wall_time_us":2868,"thread_start_us":249140,"threads_started":2,"wal-append.queue_time_us":218292}
I20260709 13:09:34.107563 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:34.108137 22879 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:09:34.108817 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:34.097501 26887 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:34.100502 26811 raft_consensus.cc:697] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:34.109611 23028 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
I20260709 13:09:34.103852 26985 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:34.103190 26989 raft_consensus.cc:2749] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:34.110744 26778 raft_consensus.cc:2749] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:09:34.109654 26886 raft_consensus.cc:2749] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:09:34.118964 26966 consensus_queue.cc:1048] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 13:09:34.112337 26799 consensus_queue.cc:260] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:34.119694 26938 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:34.141084 26938 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:34.146871 26799 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:34.157081 26811 consensus_queue.cc:237] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:34.166349 26988 log.cc:927] Time spent T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:09:34.129705 26989 raft_consensus.cc:2749] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:34.143811 26938 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:34.181214 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:34.193437 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:34.200723 22949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 1909) took 2655 ms. Trace:
I20260709 13:09:34.201287 22949 rpcz_store.cc:276] 0709 13:09:31.545618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:31.924332 (+378714us) service_pool.cc:224] Handling call
0709 13:09:31.925312 (+   980us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:32.314447 (+389135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:34.200393 (+1885946us) spinlock_profiling.cc:243] Waited 319 ms on lock 0x7b540035d4d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:34.200628 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":734425344}
I20260709 13:09:34.209901 22994 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:34.211652 22994 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:34.216373 22994 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:34.219964 22994 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:34.222168 22955 raft_consensus.cc:2410] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:09:34.223672 26675 raft_consensus.cc:2749] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:09:34.177184 26549 raft_consensus.cc:697] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 11 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:34.222493 22994 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:34.218680 26780 raft_consensus.cc:2749] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:34.221132 26975 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:09:34.232931 26960 log.cc:927] Time spent T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:09:34.245107 22876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 1957) took 1664 ms. Trace:
I20260709 13:09:34.247486 22994 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:34.238001 26626 raft_consensus.cc:2804] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:34.249338 26765 raft_consensus.cc:2749] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:09:34.249586 26990 log.cc:927] Time spent T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 13:09:34.248399 26819 raft_consensus.cc:2804] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:34.248723 26762 raft_consensus.cc:2749] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:09:34.242941 26675 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:34.252939 22994 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:09:34.270076 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 54977.9us
I20260709 13:09:34.248493 26975 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:34.270440 22948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2604) took 1680 ms. Trace:
W20260709 13:09:34.271095 22918 connection.cc:256] RPC call timeout handler was delayed by -1.163984948999996s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1895s for a response
I20260709 13:09:34.270895 22948 rpcz_store.cc:276] 0709 13:09:32.590573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:32.613164 (+ 22591us) service_pool.cc:224] Handling call
0709 13:09:32.613509 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:32.914623 (+301114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:34.270176 (+1355553us) spinlock_profiling.cc:243] Waited 36.4 ms on lock 0x7b54003d1b54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:34.270378 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":83791104}
W20260709 13:09:34.271648 22918 connection.cc:256] RPC call timeout handler was delayed by -1.0863151590000371s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1716s for a response
W20260709 13:09:34.272034 22918 connection.cc:256] RPC call timeout handler was delayed by -1.0332151590000649s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1775s for a response
I20260709 13:09:34.272921 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:34.273497 23029 raft_consensus.cc:2376] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 9. Re-sending same reply.
I20260709 13:09:34.273937 23029 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33420 (request call id 1943) took 1051 ms. Trace:
I20260709 13:09:34.274201 23029 rpcz_store.cc:276] 0709 13:09:33.222798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:33.659260 (+436462us) service_pool.cc:224] Handling call
0709 13:09:34.272778 (+613518us) spinlock_profiling.cc:243] Waited 1.82 ms on lock 0x7b58001ca198. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0a22 00007f613bbd9c09 00007f613ad33ee0 00007f613adfd0ab 00007f613adfd131 00007f613adab0a0 00007f613adaa984 00007f61343256d4 00007f6134325662
0709 13:09:34.273885 (+  1107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4185472}
I20260709 13:09:34.275460 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:34.276204 23026 raft_consensus.cc:2410] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:09:34.282239 23616 raft_consensus.cc:1240] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 11. Current term is 12. Ops: []
I20260709 13:09:34.283762 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:34.285315 23620 raft_consensus.cc:1240] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:09:34.254328 26626 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:34.272365 22918 connection.cc:256] RPC call timeout handler was delayed by -1.0104916029999913s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15400000000000003s for a response
W20260709 13:09:34.287339 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:34.289426 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:34.290132 23623 raft_consensus.cc:2393] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 10: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:34.288496 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:34.264881 26780 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:34.295636 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:34.296844 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:34.265022 26765 raft_consensus.cc:2749] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:09:34.298494 23620 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 LEARNER]: Advancing to term 10
W20260709 13:09:34.298727 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 81070.4us
I20260709 13:09:34.300367 22918 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:34.289287 26884 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:34.286377 26975 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:09:34.301996 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 195606us
I20260709 13:09:34.304572 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:34.286605 26758 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:09:34.305413 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:34.292376 26626 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:34.306586 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:34.307961 26884 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:34.309546 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:34.317229 23623 raft_consensus.cc:2393] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 11: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:34.308615 26794 consensus_queue.cc:1048] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:09:34.304039 26549 consensus_queue.cc:237] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:34.270076 26675 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:34.289228 26762 consensus_queue.cc:1048] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 13:09:34.322570 22843 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:34.324409 23620 raft_consensus.cc:1275] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 LEARNER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 13:09:34.313676 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:34.315163 23026 raft_consensus.cc:2393] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 11: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:34.301568 26509 consensus_queue.cc:1059] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260709 13:09:34.304611 26780 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:34.308271 26996 log.cc:927] Time spent T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 13:09:34.306756 26347 raft_consensus.cc:2804] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:34.330459 26347 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:34.330886 26347 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:34.331059 26780 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:34.335359 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:34.308255 26765 consensus_queue.cc:1059] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 13:09:34.325196 23616 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
I20260709 13:09:34.301986 26970 consensus_queue.cc:1048] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 13:09:34.336202 23620 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 13.
I20260709 13:09:34.339195 23029 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:34.336884 22843 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:34.327854 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:34.340819 23023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33418 (request call id 1888) took 1916 ms. Trace:
I20260709 13:09:34.326550 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:34.337839 27005 raft_consensus.cc:2749] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:09:34.341800 23023 rpcz_store.cc:276] 0709 13:09:32.424105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:32.459596 (+ 35491us) service_pool.cc:224] Handling call
0709 13:09:34.340541 (+1880945us) spinlock_profiling.cc:243] Waited 67.8 ms on lock 0x7b540010c054. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:34.340749 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":155852672}
I20260709 13:09:34.344810 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:09:34.345521 22826 auto_rebalancer.cc:893] Removing replace marker failed: Leader has not yet committed an operation in its own term
W20260709 13:09:34.345942 22826 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 13:09:34.346225 22826 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:09:34.345952 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:34.349586 23023 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 13.
I20260709 13:09:34.352311 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:34.314141 26887 consensus_queue.cc:1048] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 13:09:34.353613 26675 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:34.353720 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:34.354763 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:34.355278 23023 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 8.
I20260709 13:09:34.356004 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:34.356496 23023 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
I20260709 13:09:34.357226 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:34.358251 26884 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:34.359900 23029 raft_consensus.cc:2468] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 9.
I20260709 13:09:34.365360 26762 raft_consensus.cc:3055] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:09:34.365850 26762 raft_consensus.cc:740] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:34.366626 26762 consensus_queue.cc:260] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:34.367030 26347 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:34.316618 26758 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:34.315924 26236 consensus_queue.cc:1048] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
W20260709 13:09:34.379235 26951 log.cc:927] Time spent T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 1.827s	user 0.005s	sys 0.024s
W20260709 13:09:34.381145 22918 leader_election.cc:336] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.864s (SENT)
I20260709 13:09:34.368880 26894 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:34.381780 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:34.381852 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 45084.5us
I20260709 13:09:34.384116 22843 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:34.384963 23026 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 10.
I20260709 13:09:34.385481 26894 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:34.390496 22877 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11.
I20260709 13:09:34.391810 22843 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:34.395783 26755 raft_consensus.cc:2749] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 13:09:34.396519 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:34.397979 22918 leader_election.cc:336] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 2.017s (SENT)
I20260709 13:09:34.373994 26985 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:34.384557 26799 raft_consensus.cc:2749] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 13:09:34.393573 26937 log.cc:927] Time spent T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 2.323s	user 0.019s	sys 0.055s
I20260709 13:09:34.392710 27005 raft_consensus.cc:2749] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:09:34.409878 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:34.411221 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:34.394730 26347 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:34.404273 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 117393us
W20260709 13:09:34.405920 22918 leader_election.cc:336] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 2.116s (SENT)
I20260709 13:09:34.413288 23023 raft_consensus.cc:1240] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 6. Current term is 7. Ops: [4.7-6.8]
I20260709 13:09:34.414181 22918 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:34.417245 22918 leader_election.cc:336] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 2.155s (SENT)
I20260709 13:09:34.417786 22918 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:34.422079 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 106808us
I20260709 13:09:34.246289 22876 rpcz_store.cc:276] 0709 13:09:32.580563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:33.985492 (+1404929us) service_pool.cc:224] Handling call
0709 13:09:33.985891 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:33.986112 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:09:34.175135 (+189023us) log.cc:844] Serialized 14 byte log entry
0709 13:09:34.244223 (+ 69088us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:34.244281 (+    58us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:09:34.244688 (+   407us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:34.244793 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:34.245034 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":48093,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":27,"raft.queue_time_us":1552,"thread_start_us":209857,"threads_started":2}
W20260709 13:09:34.431207 26962 log.cc:927] Time spent T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 1.384s	user 0.005s	sys 0.016s
W20260709 13:09:34.432317 22955 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58310 (request call id 1919) took 1858 ms (client timeout 1670 ms). Trace:
W20260709 13:09:34.432898 22955 rpcz_store.cc:269] 0709 13:09:32.573851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:32.574277 (+   426us) service_pool.cc:224] Handling call
0709 13:09:34.432039 (+1857762us) spinlock_profiling.cc:243] Waited 1.65 s on lock 0x7b54000a91d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:34.432223 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3787026048}
I20260709 13:09:34.411528 26626 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:34.423242 26236 raft_consensus.cc:2749] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:09:34.424458 26347 raft_consensus.cc:2749] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:09:34.443747 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:34.444893 22920 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:34.452219 22875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 1947) took 2577 ms. Trace:
I20260709 13:09:34.461045 26549 consensus_queue.cc:1059] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:09:34.462291 26549 raft_consensus.cc:3055] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:09:34.462723 26549 raft_consensus.cc:740] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:34.463750 26549 consensus_queue.cc:260] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:34.452992 22875 rpcz_store.cc:276] 0709 13:09:31.874993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:31.917967 (+ 42974us) service_pool.cc:224] Handling call
0709 13:09:31.933249 (+ 15282us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:31.934351 (+  1102us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:31.934395 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:31.935047 (+   652us) log.cc:844] Serialized 52 byte log entry
0709 13:09:32.012615 (+ 77568us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:32.138056 (+125441us) log.cc:844] Serialized 14 byte log entry
0709 13:09:32.175259 (+ 37203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:32.175332 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:34.388766 (+2213434us) spinlock_profiling.cc:243] Waited 4.33 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:09:34.436526 (+ 47760us) raft_consensus.cc:1685] finished
0709 13:09:34.451906 (+ 15380us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:34.452114 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37675,"prepare.queue_time_us":43359,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":41305,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":6222,"spinlock_wait_cycles":9968896,"thread_start_us":185897,"threads_started":2,"wal-append.queue_time_us":135165}
W20260709 13:09:34.470431 22874 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35370 (request call id 2435) took 2083 ms (client timeout 1839 ms). Trace:
W20260709 13:09:34.470851 22874 rpcz_store.cc:269] 0709 13:09:32.386556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:32.891056 (+504500us) service_pool.cc:224] Handling call
0709 13:09:34.470264 (+1579208us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b540022b5d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:34.470370 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":328443648}
I20260709 13:09:34.038405 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:34.472975 22993 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:34.480314 22994 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:09:34.484023 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 44052.2us
I20260709 13:09:34.467129 26884 raft_consensus.cc:2749] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:09:34.469250 26887 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:34.468736 26236 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:34.486917 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:34.487497 23022 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
W20260709 13:09:34.486930 22844 connection.cc:256] RPC call timeout handler was delayed by -1.234950294999976s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1643s for a response
W20260709 13:09:34.488606 22844 connection.cc:256] RPC call timeout handler was delayed by -1.0954502949999778s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17980000000000002s for a response
I20260709 13:09:34.489796 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:34.490689 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:34.491653 26984 log.cc:927] Time spent T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.404s	user 0.000s	sys 0.002s
I20260709 13:09:34.493489 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:34.494166 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:34.494290 22876 raft_consensus.cc:2410] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:09:34.495460 22876 raft_consensus.cc:1240] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 6. Current term is 7. Ops: []
I20260709 13:09:34.495659 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:34.493938 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:34.497550 22880 raft_consensus.cc:2393] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 10: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:34.497576 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:34.498828 22951 raft_consensus.cc:2393] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 12: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:34.499580 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:34.499862 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:34.500272 22948 raft_consensus.cc:2368] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 11. Current term is 12.
I20260709 13:09:34.500507 22946 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 6.
I20260709 13:09:34.501745 23588 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:34.502449 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:34.502583 26675 raft_consensus.cc:3055] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:09:34.502638 23588 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:34.503058 26675 raft_consensus.cc:740] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:34.503541 26938 raft_consensus.cc:2804] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:34.503927 26938 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:34.504279 26938 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:34.505028 23028 raft_consensus.cc:1240] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 11. Current term is 12. Ops: [11.5-11.5]
I20260709 13:09:34.505075 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:34.505829 23620 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11.
I20260709 13:09:34.503943 26675 consensus_queue.cc:260] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:34.507339 26935 raft_consensus.cc:2804] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:34.507764 26935 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:34.508133 26935 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:34.511417 26940 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:34.512805 26938 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:34.513733 23026 raft_consensus.cc:1240] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:34.515056 26938 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:34.515609 23026 raft_consensus.cc:1240] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:09:34.516695 22947 raft_consensus.cc:1275] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:09:34.518417 22843 leader_election.cc:400] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 11. Current term is 12.
I20260709 13:09:34.518711 22843 leader_election.cc:403] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:09:34.519066 22920 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:34.524483 26935 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:34.527590 26935 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:34.530434 26819 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:34.531096 26819 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:34.533093 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:34.534215 23020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33420 (request call id 1947) took 1024 ms. Trace:
I20260709 13:09:34.534613 23020 rpcz_store.cc:276] 0709 13:09:33.510422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:33.510596 (+   174us) service_pool.cc:224] Handling call
0709 13:09:34.532901 (+1022305us) spinlock_profiling.cc:243] Waited 4.36 ms on lock 0x7b58001ca198. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0a22 00007f613bbd9c09 00007f613ad33ee0 00007f613adfd0ab 00007f613adfd131 00007f613adab0a0 00007f613adaa984 00007f61343256d4 00007f6134325662
0709 13:09:34.534137 (+  1236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10029952}
I20260709 13:09:34.547343 26935 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:34.548738 26935 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:34.554499 26935 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:34.566432 26819 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:34.573520 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:34.586838 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 67841.1us
I20260709 13:09:34.587270 22880 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 12.
I20260709 13:09:34.588109 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:34.486063 26549 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:34.582407 26970 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:34.577571 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:34.478929 26347 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:34.584826 22919 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:34.579072 22844 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:34.586481 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:34.588955 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:34.587600 22920 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:34.573958 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:34.589921 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:34.593315 26758 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:34.594676 27019 consensus_queue.cc:1059] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:09:34.595953 26626 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:34.597153 26843 raft_consensus.cc:2804] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:09:34.597605 26975 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:34.575894 22879 raft_consensus.cc:1275] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260709 13:09:34.600214 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 81900.9us
W20260709 13:09:34.601081 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 109341us
I20260709 13:09:34.607455 22871 raft_consensus.cc:2393] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 12: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:34.596575 26985 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:34.609913 22844 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:34.584235 26236 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:34.659821 22843 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:09:34.624500 27009 log.cc:927] Time spent T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.189s	user 0.001s	sys 0.000s
I20260709 13:09:34.613368 26347 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:34.614244 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:34.615371 26675 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:34.616074 27014 raft_consensus.cc:2804] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:09:34.619922 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:34.680042 22952 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:34.622380 23622 raft_consensus.cc:1275] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:09:34.622509 23620 raft_consensus.cc:1240] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:09:34.585108 27005 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:34.653891 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:34.652693 26969 raft_consensus.cc:2804] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:34.687222 22954 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:09:34.656402 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 68873.8us
I20260709 13:09:34.653090 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:34.668859 22843 leader_election.cc:336] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.643s (SENT)
I20260709 13:09:34.606190 26755 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:34.691072 22948 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 11.
I20260709 13:09:34.669176 22879 raft_consensus.cc:1275] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 13:09:34.671173 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:34.691129 22843 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:34.607859 26940 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:34.583066 26887 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:34.672262 23624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60044 (request call id 749) took 2403 ms. Trace:
I20260709 13:09:34.620126 26799 raft_consensus.cc:2749] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:34.656754 26945 raft_consensus.cc:2804] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:34.668087 26807 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:34.660725 22992 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:09:34.710598 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 237699us
I20260709 13:09:34.712488 22878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2430) took 2327 ms. Trace:
W20260709 13:09:34.753965 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:34.715493 23588 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:34.716651 23624 rpcz_store.cc:276] 0709 13:09:32.269038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:32.269155 (+   117us) service_pool.cc:224] Handling call
0709 13:09:32.269445 (+   290us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:33.110285 (+840840us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:34.672022 (+1561737us) spinlock_profiling.cc:243] Waited 835 ms on lock 0x7b540045bfd4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:34.672207 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1920832256}
I20260709 13:09:34.718750 26819 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:34.714812 27018 consensus_queue.cc:1059] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:09:34.618837 26970 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:34.721230 26975 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:34.723068 27014 raft_consensus.cc:697] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 10 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:34.724026 27006 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:34.724838 26870 consensus_queue.cc:1048] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:09:34.673115 26997 raft_consensus.cc:2804] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:09:34.775771 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 115113us
I20260709 13:09:34.688445 26969 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:34.776808 26969 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:34.753316 22878 rpcz_store.cc:276] 0709 13:09:32.385377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:33.094173 (+708796us) service_pool.cc:224] Handling call
0709 13:09:33.109172 (+ 14999us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:33.664931 (+555759us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:34.712224 (+1047293us) spinlock_profiling.cc:243] Waited 555 ms on lock 0x7b54001ba154. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:34.712431 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1276084992}
I20260709 13:09:34.778620 22948 raft_consensus.cc:1275] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:09:34.671063 26509 raft_consensus.cc:2804] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:09:34.758872 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:34.766132 22952 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 14.
I20260709 13:09:34.767117 22955 raft_consensus.cc:2393] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 8: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
W20260709 13:09:34.769240 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 159394us
I20260709 13:09:34.782878 22844 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:34.783736 23588 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:34.784929 22844 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:34.673643 26989 consensus_queue.cc:1048] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.090s
I20260709 13:09:34.786931 26989 raft_consensus.cc:2749] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:34.751860 26945 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:34.779749 22992 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:34.677981 26843 raft_consensus.cc:697] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:34.787997 26945 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:34.786237 26758 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:34.788789 26843 consensus_queue.cc:237] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:34.793968 22844 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:34.796167 22844 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:34.794760 26989 raft_consensus.cc:2749] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:34.802176 26989 raft_consensus.cc:2804] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:09:34.802716 26989 raft_consensus.cc:697] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 12 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:34.803785 26989 consensus_queue.cc:237] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:34.809216 22844 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:34.797731 26969 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:34.800809 26945 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:34.765345 26799 raft_consensus.cc:2749] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:34.765933 26819 raft_consensus.cc:2749] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:34.726641 26940 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:34.816160 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 335865us
I20260709 13:09:34.817611 27029 consensus_queue.cc:1048] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:09:34.780319 26997 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:34.811838 27015 raft_consensus.cc:3055] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:09:34.827731 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 51120.1us
I20260709 13:09:34.768831 26861 consensus_queue.cc:1059] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 13:09:34.829919 22994 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:34.783037 26778 raft_consensus.cc:2804] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:34.826743 26846 consensus_queue.cc:1059] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:09:34.855118 26778 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:34.855580 26846 raft_consensus.cc:3055] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:09:34.713497 27005 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:34.856204 26846 raft_consensus.cc:740] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:34.856779 22994 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:34.857355 26985 raft_consensus.cc:2804] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:34.857246 26846 consensus_queue.cc:260] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:34.712879 27007 consensus_queue.cc:1059] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.116s
I20260709 13:09:34.775916 27006 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:34.858683 27016 log.cc:927] Time spent T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.189s	user 0.001s	sys 0.000s
W20260709 13:09:34.859866 27021 log.cc:927] Time spent T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 13:09:34.860057 27006 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:34.860371 27019 consensus_queue.cc:1059] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:09:34.861327 27019 consensus_queue.cc:1059] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:09:34.862341 27019 raft_consensus.cc:3055] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:09:34.862701 27019 raft_consensus.cc:740] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:34.863333 26894 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:34.863682 27019 consensus_queue.cc:260] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:34.860008 27007 raft_consensus.cc:2749] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:09:34.866067 26846 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:34.866212 26940 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:34.868145 26969 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:34.869871 23025 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33420 (request call id 1945) took 1571 ms (client timeout 1655 ms). Trace:
W20260709 13:09:34.870285 23025 rpcz_store.cc:269] 0709 13:09:33.298168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:33.298345 (+   177us) service_pool.cc:224] Handling call
0709 13:09:34.869688 (+1571343us) spinlock_profiling.cc:243] Waited 79.8 ms on lock 0x7b5400134554. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:34.869801 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":183508736}
I20260709 13:09:34.872772 27019 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:34.873092 23024 raft_consensus.cc:1275] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:09:34.854164 26935 raft_consensus.cc:2804] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:09:34.838320 22954 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 9.
I20260709 13:09:34.765215 26807 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:34.781148 26509 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:34.850054 26755 raft_consensus.cc:2749] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 11. Current term is 12.
I20260709 13:09:34.829180 27023 raft_consensus.cc:2804] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:34.855639 26778 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:34.857895 26985 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:34.857964 27005 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:34.875661 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:34.908205 23619 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
W20260709 13:09:34.814412 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 48727.7us
I20260709 13:09:34.827241 27027 raft_consensus.cc:2749] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:34.849340 22992 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:09:34.873142 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:34.876487 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:34.913663 23624 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Advancing to term 13
W20260709 13:09:34.897228 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 171091us
I20260709 13:09:34.899415 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:34.902927 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:34.836802 27017 raft_consensus.cc:2804] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:34.917177 22879 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:34.850577 26997 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:34.918777 22872 raft_consensus.cc:2393] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 9: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:34.926064 22993 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:09:34.919626 22994 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:34.848743 27015 raft_consensus.cc:740] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:34.932720 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:34.812175 27020 consensus_queue.cc:1059] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.128s
W20260709 13:09:34.938102 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 88883.6us
I20260709 13:09:34.939904 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:09:34.960427 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 291540us
I20260709 13:09:34.906059 26509 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:34.992784 23024 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "5b346767a89745fa82928a5dac9cc0b2"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:09:35.001155 23024 raft_consensus.cc:606] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Received request to transfer leadership to TS 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:35.001616 23024 raft_consensus.cc:614] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:34.995821 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:34.996455 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:35.002455 23020 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11.
I20260709 13:09:34.997100 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:34.907423 27023 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:35.002964 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:35.003489 23023 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11.
I20260709 13:09:35.003548 27023 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:35.005653 22872 raft_consensus.cc:1275] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 13:09:35.007754 22872 pending_rounds.cc:85] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 4
I20260709 13:09:35.008088 22872 pending_rounds.cc:107] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 13:09:35.008399 22872 raft_consensus.cc:2889] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:35.014140 22876 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 12.
I20260709 13:09:34.915899 26985 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:35.018391 22992 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:35.019312 22875 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:34.998143 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:35.003757 23024 raft_consensus.cc:2368] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 10. Current term is 11.
I20260709 13:09:35.006327 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:34.998993 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:35.020128 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 94148.6us
I20260709 13:09:35.007308 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:35.022116 23023 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:35.024657 22879 raft_consensus.cc:2468] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
W20260709 13:09:35.026100 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:35.026834 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:35.033324 22992 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:34.918606 27017 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:35.034672 22992 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:35.021039 23020 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 12.
I20260709 13:09:35.035509 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:35.036370 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:35.037662 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:35.041829 22843 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:35.049328 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:35.053791 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:35.055063 26989 raft_consensus.cc:2749] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:09:35.056835 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:35.074231 23023 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 12.
W20260709 13:09:35.086619 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 44679.9us
W20260709 13:09:35.087076 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 277941us
I20260709 13:09:35.076370 22875 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11.
W20260709 13:09:35.093464 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 58834.1us
W20260709 13:09:35.090945 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 261070us
I20260709 13:09:35.094880 22992 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
W20260709 13:09:35.095863 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 73191.5us
I20260709 13:09:35.096051 23588 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:35.090378 22844 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:35.069306 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 132504us
I20260709 13:09:35.068939 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:35.099524 23024 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:35.100107 22843 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:35.098222 22844 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:35.057160 26975 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:35.052264 27017 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:35.057099 27006 raft_consensus.cc:2749] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:09:35.067744 27024 raft_consensus.cc:2804] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:35.109583 22994 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:35.061918 27018 raft_consensus.cc:2749] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:35.078274 26799 raft_consensus.cc:2749] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:35.107683 26887 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:35.102742 27032 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:35.069013 27020 consensus_queue.cc:1243] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer 5b346767a89745fa82928a5dac9cc0b2 log is divergent from this leader: its last log entry 12.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:09:35.091595 26236 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:35.129431 27024 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:34.905100 27014 consensus_queue.cc:237] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:35.144021 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:35.079327 27011 consensus_queue.cc:1048] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.068s
I20260709 13:09:35.104422 26549 raft_consensus.cc:2749] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:09:35.162206 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 58325.8us
W20260709 13:09:35.147805 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:35.115932 26755 raft_consensus.cc:2804] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:35.055395 27023 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:35.163229 26755 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:35.163655 26755 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:35.164098 22993 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:35.108646 27008 raft_consensus.cc:2804] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:35.114127 27007 consensus_queue.cc:1059] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:09:35.146245 26765 raft_consensus.cc:2804] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:35.152616 26970 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:35.157626 27020 raft_consensus.cc:2804] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 13:09:35.160012 26870 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:35.165617 27023 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:35.159060 27024 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:35.165417 27010 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:35.159655 26945 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:35.189873 26236 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:35.193347 27008 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:35.222563 27036 log.cc:927] Time spent T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 13:09:35.195351 26765 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:35.225082 26675 consensus_meta.cc:287] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.544s	user 0.003s	sys 0.006s
I20260709 13:09:35.233588 26765 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:09:35.227792 27030 log.cc:927] Time spent T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.182s	user 0.000s	sys 0.001s
I20260709 13:09:35.217432 26870 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:35.236289 26886 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:35.211376 26846 raft_consensus.cc:2749] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:35.127878 27015 consensus_queue.cc:260] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:09:35.236995 26886 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:35.238006 26870 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:35.238270 27015 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:35.238817 26347 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:35.239344 26886 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:35.241174 27034 log.cc:927] Time spent T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.159s	user 0.003s	sys 0.000s
W20260709 13:09:35.240867 27006 consensus_queue.cc:1175] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "5b346767a89745fa82928a5dac9cc0b2" responder_term: 10 status { last_received { term: 8 index: 6 } last_committed_idx: 5 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:09:35.235911 26966 raft_consensus.cc:2804] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:09:35.243326 26966 raft_consensus.cc:697] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 9 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:35.244220 26966 consensus_queue.cc:237] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:35.245085 26762 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:35.247201 27008 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:35.249832 26626 raft_consensus.cc:3055] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:09:35.250200 26626 raft_consensus.cc:740] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:35.251016 26626 consensus_queue.cc:260] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:35.252290 26626 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:35.155385 27032 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:35.253484 27022 raft_consensus.cc:3055] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:09:35.253883 27022 raft_consensus.cc:740] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:35.254627 27022 consensus_queue.cc:260] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:35.255786 27022 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:35.228914 26807 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:35.201367 27018 consensus_queue.cc:1048] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 13:09:35.218060 27020 raft_consensus.cc:697] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:35.257833 26975 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:35.278908 26940 raft_consensus.cc:3055] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:09:35.279381 26940 raft_consensus.cc:740] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:35.279646 27008 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:35.280251 26940 consensus_queue.cc:260] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:35.281603 26940 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:35.281868 27008 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:35.282076 26944 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:35.283656 27024 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:35.285610 27055 raft_consensus.cc:3055] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:09:35.286165 27055 raft_consensus.cc:740] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:35.287077 27055 consensus_queue.cc:260] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:35.277280 26975 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:35.231797 27010 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:35.291244 27055 raft_consensus.cc:3060] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:35.306298 27024 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:35.306753 27010 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:35.291427 27017 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:35.308881 27017 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:35.311528 26997 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:35.316394 27032 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:35.302991 27020 consensus_queue.cc:237] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:35.360874 22871 raft_consensus.cc:1240] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 6. Current term is 7. Ops: []
W20260709 13:09:35.363878 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 265523us
I20260709 13:09:35.105479 22843 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:35.389804 22877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35370 (request call id 2453) took 1330 ms. Trace:
I20260709 13:09:35.390522 22877 rpcz_store.cc:276] 0709 13:09:34.059610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:34.060070 (+   460us) service_pool.cc:224] Handling call
0709 13:09:35.389237 (+1329167us) spinlock_profiling.cc:243] Waited 289 ms on lock 0x7b540040a954. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:35.389522 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":664491264}
I20260709 13:09:35.391384 22844 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:09:35.389863 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 284379us
I20260709 13:09:35.401528 23025 raft_consensus.cc:2393] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 9: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:35.187022 23027 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 641) took 3810 ms. Trace:
I20260709 13:09:35.404693 23027 rpcz_store.cc:276] 0709 13:09:31.377146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:31.650285 (+273139us) service_pool.cc:224] Handling call
0709 13:09:31.764738 (+114453us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:31.764947 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:31.764970 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:31.765393 (+   423us) log.cc:844] Serialized 26 byte log entry
0709 13:09:31.867326 (+101933us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:09:31.867721 (+   395us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:31.867849 (+   128us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:33.692024 (+1824175us) spinlock_profiling.cc:243] Waited 2.7 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:09:33.911396 (+219372us) raft_consensus.cc:1685] finished
0709 13:09:33.911420 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:35.168233 (+1256813us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b5400135450. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:35.186937 (+ 18704us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":268907520,"thread_start_us":101579,"threads_started":1,"wal-append.queue_time_us":137149,"wal-append.run_cpu_time_us":42341,"wal-append.run_wall_time_us":3010866}
I20260709 13:09:35.409614 23028 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 1957) took 1210 ms. Trace:
I20260709 13:09:35.410286 23028 rpcz_store.cc:276] 0709 13:09:34.198913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:34.505885 (+306972us) service_pool.cc:224] Handling call
0709 13:09:34.506153 (+   268us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:34.506349 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:34.506380 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:34.693493 (+187113us) spinlock_profiling.cc:243] Waited 81.7 ms on lock 0x7b2c000bc418. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f61346dea11 00007f61346ee372 00007f6135e20a01 00007f6135de543a 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2
0709 13:09:34.693960 (+   467us) log.cc:844] Serialized 26 byte log entry
0709 13:09:34.999450 (+305490us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:34.999484 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:34.999550 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:35.408890 (+409340us) raft_consensus.cc:1685] finished
0709 13:09:35.408941 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:35.409361 (+   420us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":187937536,"thread_start_us":305176,"threads_started":1,"wal-append.queue_time_us":594846}
W20260709 13:09:35.366766 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:35.414927 23021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 661) took 1362 ms. Trace:
W20260709 13:09:35.375388 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:35.376775 22873 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
mode: GRACEFUL
new_leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:40244
I20260709 13:09:35.419765 23029 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11.
I20260709 13:09:35.420164 22873 raft_consensus.cc:606] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Received request to transfer leadership to TS aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:35.420849 22873 raft_consensus.cc:614] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:35.415511 23021 rpcz_store.cc:276] 0709 13:09:34.052440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:34.052807 (+   367us) service_pool.cc:224] Handling call
0709 13:09:34.341501 (+288694us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:34.341839 (+   338us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:09:34.341866 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:34.380976 (+ 39110us) log.cc:844] Serialized 26 byte log entry
0709 13:09:34.438301 (+ 57325us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:09:34.438363 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:34.438465 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:34.615332 (+176867us) raft_consensus.cc:1685] finished
0709 13:09:34.615359 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:35.414291 (+798932us) spinlock_profiling.cc:243] Waited 288 ms on lock 0x7b540010c050. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:35.414671 (+   380us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":73449,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":5093,"spinlock_wait_cycles":663247104,"thread_start_us":56936,"threads_started":1,"wal-append.queue_time_us":107075}
W20260709 13:09:35.373111 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 278186us
I20260709 13:09:35.431524 22947 raft_consensus.cc:1275] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:09:35.431348 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:35.431738 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:35.432701 22954 raft_consensus.cc:2410] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 13:09:35.433079 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:35.435674 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:35.436642 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:35.437583 23625 raft_consensus.cc:1240] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 6. Current term is 7. Ops: []
I20260709 13:09:35.433130 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:35.442342 22946 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
I20260709 13:09:35.408593 23027 raft_consensus.cc:1240] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
W20260709 13:09:35.504806 26935 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.009s	sys 0.009s
W20260709 13:09:35.508767 26935 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.010s	sys 0.009s
I20260709 13:09:35.509251 26935 raft_consensus.cc:697] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
W20260709 13:09:35.506978 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 397341us
W20260709 13:09:35.518891 22994 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:35.519891 22994 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:09:35.544130 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:35.296243 26938 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.698s	user 0.007s	sys 0.003s
I20260709 13:09:35.585566 27027 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:35.586517 27027 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:35.588999 22880 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
mode: GRACEFUL
new_leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:40244
I20260709 13:09:35.589793 23020 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 12.
I20260709 13:09:35.589356 26509 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:35.590066 22880 raft_consensus.cc:606] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Received request to transfer leadership to TS aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:35.590801 22880 raft_consensus.cc:614] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:35.592217 26989 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:35.592496 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:35.593151 26989 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:35.593816 22871 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:35.595963 26989 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:35.596585 26236 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:35.599558 26236 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:35.600533 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:35.600919 26819 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:35.601876 22877 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:09:35.258831 26778 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:35.602557 26549 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:09:35.603229 26549 raft_consensus.cc:697] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 10 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:35.603343 26894 raft_consensus.cc:2749] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:09:35.604781 26778 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:35.604552 26549 consensus_queue.cc:237] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:35.606007 22955 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
I20260709 13:09:35.609323 27048 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:35.609949 27048 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:35.610618 23024 consensus_meta.cc:287] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.509s	user 0.009s	sys 0.009s
I20260709 13:09:35.611330 23024 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 10.
I20260709 13:09:35.613020 27078 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:35.613843 27078 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:35.617141 23025 raft_consensus.cc:1240] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:09:35.256587 26755 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:35.620795 26938 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.983s	user 0.007s	sys 0.003s
I20260709 13:09:35.589226 27063 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:35.622265 27063 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:35.623723 27058 consensus_queue.cc:1059] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:09:35.625908 27058 raft_consensus.cc:3055] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:09:35.626295 27058 raft_consensus.cc:740] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:09:35.627210 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 107566us
I20260709 13:09:35.628079 26755 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:35.628502 22994 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:35.601759 26819 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:35.590476 27061 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:35.644084 27020 raft_consensus.cc:2804] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:35.644172 27022 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:35.649727 27031 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:35.627240 27063 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:35.653250 23617 raft_consensus.cc:1240] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 11. Current term is 12. Ops: []
W20260709 13:09:35.653721 22918 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.004s	sys 0.007s
I20260709 13:09:35.654316 27058 consensus_queue.cc:260] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:35.672926 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:35.612488 27048 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:35.661840 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:35.662598 26808 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:35.662670 27060 raft_consensus.cc:2804] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:09:35.592229 26509 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:35.654475 26935 consensus_queue.cc:237] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:35.680944 22843 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:09:35.677227 22918 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.004s	sys 0.007s
I20260709 13:09:35.679917 26799 raft_consensus.cc:2804] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:09:35.684690 26799 raft_consensus.cc:697] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 9 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:35.685875 26799 consensus_queue.cc:237] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:35.686816 22918 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:35.688383 27043 raft_consensus.cc:2749] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:09:35.689390 27005 raft_consensus.cc:2804] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:09:35.690033 27005 raft_consensus.cc:697] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 10 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:35.691190 27005 consensus_queue.cc:237] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:35.692546 27060 raft_consensus.cc:697] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:09:35.692934 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 50098.1us
I20260709 13:09:35.699855 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:35.696142 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:35.701982 23027 raft_consensus.cc:2393] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 12: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:35.702966 27022 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:35.663901 27020 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:35.704954 27020 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:35.704865 26808 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:35.705541 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:35.706359 22876 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:35.707247 26808 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:35.708164 27027 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:35.708976 22954 raft_consensus.cc:2468] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
I20260709 13:09:35.708854 27031 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:35.710014 26997 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:35.713384 23029 raft_consensus.cc:2393] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 10: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:35.697096 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:35.715029 22947 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:09:35.720685 23590 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:35.722147 27048 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:35.723004 27048 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:35.723941 23590 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:35.724776 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:35.669265 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:35.726143 23622 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
I20260709 13:09:35.726964 22952 raft_consensus.cc:1240] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:35.726114 27048 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:35.734457 22876 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11.
I20260709 13:09:35.727878 27048 consensus_queue.cc:1048] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:09:35.742194 27043 consensus_queue.cc:1059] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
W20260709 13:09:35.761396 26626 consensus_meta.cc:287] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.502s	user 0.005s	sys 0.005s
I20260709 13:09:35.762804 23022 raft_consensus.cc:1275] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:09:35.696421 27078 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:35.670322 26989 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:35.712544 27031 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:35.783191 23024 raft_consensus.cc:1275] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:09:35.786100 22946 raft_consensus.cc:1275] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:09:35.787006 22952 raft_consensus.cc:1240] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 11. Current term is 12. Ops: []
I20260709 13:09:35.793665 22994 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:35.801074 27038 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:35.776508 27063 raft_consensus.cc:2749] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:09:35.808039 23029 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ec334666b7964c969125af95d9c797ed"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "5b346767a89745fa82928a5dac9cc0b2"
 from {username='slave'} at 127.0.0.1:60044
W20260709 13:09:35.784175 26943 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.622s	user 0.003s	sys 0.014s
I20260709 13:09:35.814124 23029 raft_consensus.cc:606] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Received request to transfer leadership to TS 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:35.814558 23029 raft_consensus.cc:614] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:09:35.813071 22874 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.007s	sys 0.007s
I20260709 13:09:35.817903 22876 raft_consensus.cc:1275] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 13:09:35.819207 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 48171.3us
I20260709 13:09:35.820437 22947 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11.
I20260709 13:09:35.782148 26997 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:35.778492 26989 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:35.825011 23624 consensus_meta.cc:287] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.911s	user 0.013s	sys 0.004s
I20260709 13:09:35.825094 26843 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:35.825028 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:35.835057 22880 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:09:35.826694 26985 consensus_meta.cc:287] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.765s	user 0.004s	sys 0.011s
W20260709 13:09:35.838093 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 114343us
I20260709 13:09:35.839250 26819 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:35.840248 22877 raft_consensus.cc:1275] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:09:35.841005 27079 consensus_queue.cc:1048] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 13:09:35.841800 26765 consensus_meta.cc:287] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.607s	user 0.006s	sys 0.006s
I20260709 13:09:35.841688 27043 raft_consensus.cc:3055] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:09:35.842280 27043 raft_consensus.cc:740] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:35.842309 26765 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:35.843478 27043 consensus_queue.cc:260] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:35.844775 26626 consensus_queue.cc:1059] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:09:35.844889 26997 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:35.846141 26626 raft_consensus.cc:3055] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:09:35.846560 26626 raft_consensus.cc:740] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:35.847354 26997 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:35.847630 26626 consensus_queue.cc:260] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:35.848748 26765 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:35.849907 23624 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 13.
I20260709 13:09:35.852949 26802 raft_consensus.cc:2749] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:35.856027 27038 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:35.856884 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:35.817020 26943 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.858s	user 0.003s	sys 0.015s
I20260709 13:09:35.739707 27061 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:35.944216 27058 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:35.954814 27061 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:35.898881 27043 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:35.857296 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:09:35.835196 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 576831us
I20260709 13:09:35.958757 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:35.888720 22993 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:09:35.961277 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 90425.8us
I20260709 13:09:35.889447 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:35.769456 27060 consensus_queue.cc:237] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:35.931537 22919 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:09:35.967752 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 175193us
I20260709 13:09:35.969339 27022 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:35.857909 27014 raft_consensus.cc:2804] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:35.974257 27014 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:35.974659 27014 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:35.857549 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:35.893726 27081 log.cc:927] Time spent T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:09:35.977802 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:35.978394 22952 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:35.894959 22871 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11.
I20260709 13:09:35.897392 22946 raft_consensus.cc:1240] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 11. Current term is 12. Ops: []
I20260709 13:09:35.900964 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:35.982051 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:35.982106 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:35.983006 27066 log.cc:927] Time spent T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 13:09:35.983227 22946 raft_consensus.cc:2393] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 14: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:35.983419 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:35.984078 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:35.915447 27048 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:35.962891 27055 raft_consensus.cc:2804] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:35.984786 27048 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:35.891108 26975 consensus_meta.cc:287] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.592s	user 0.010s	sys 0.000s
I20260709 13:09:35.986967 27048 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:35.919634 27032 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:35.987586 27032 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:35.989639 27032 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:35.933305 26236 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:35.946699 27038 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:35.947446 22920 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:35.990361 26236 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:35.991252 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 44093.6us
I20260709 13:09:35.944160 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:35.992986 22876 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:35.995378 22920 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:35.969357 26794 raft_consensus.cc:2804] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:35.996506 27077 raft_consensus.cc:2749] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:35.996848 26794 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:35.997367 26794 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:35.997597 27077 raft_consensus.cc:2804] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:09:35.998225 27077 raft_consensus.cc:697] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:35.940316 23624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60044 (request call id 765) took 1004 ms. Trace:
I20260709 13:09:35.905057 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:35.985176 27055 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:36.003317 22950 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
I20260709 13:09:36.003578 27055 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:36.003886 26236 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:36.004367 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:36.005909 22876 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 13.
W20260709 13:09:35.944693 27009 log.cc:927] Time spent T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.438s	user 0.004s	sys 0.002s
I20260709 13:09:35.984277 22949 raft_consensus.cc:2393] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 9: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:36.010174 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:36.011204 26626 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:35.949728 22775 catalog_manager.cc:5697] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 4 to 8, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 8 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:36.016966 22946 raft_consensus.cc:2393] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 10: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:35.929456 26985 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:36.017390 27058 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:35.986658 26975 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:36.007477 22950 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
I20260709 13:09:36.000481 23624 rpcz_store.cc:276] 0709 13:09:34.875247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:34.876214 (+   967us) service_pool.cc:224] Handling call
0709 13:09:35.850476 (+974262us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:35.999413 27077 consensus_queue.cc:237] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:35.945175 26843 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:36.020471 27058 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:36.022145 26975 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:36.022224 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:36.026050 22880 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 9.
I20260709 13:09:36.028700 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:36.031486 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:36.034370 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:36.036365 22775 catalog_manager.cc:5697] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 6 to 8, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 8 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:09:36.038100 27058 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:36.038611 22775 catalog_manager.cc:5697] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:09:36.032063 23622 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 12.
I20260709 13:09:36.041301 23621 raft_consensus.cc:2393] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 7: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:36.041831 26989 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:36.043979 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:36.044911 23027 raft_consensus.cc:2393] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 14: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:36.045894 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:36.046725 23022 raft_consensus.cc:2393] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 9: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:36.047961 23020 raft_consensus.cc:1275] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:09:36.049139 26843 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:36.049947 22948 raft_consensus.cc:2410] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 13:09:36.051153 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:36.051699 23025 raft_consensus.cc:1240] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:09:36.053187 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:36.053843 23028 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 10.
I20260709 13:09:36.040017 22775 catalog_manager.cc:5697] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:09:36.055858 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:36.056280 23022 raft_consensus.cc:1275] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:09:36.057073 23027 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 12.
I20260709 13:09:36.076706 22947 raft_consensus.cc:2393] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 11: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:36.089967 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:36.091636 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:36.093696 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:36.094180 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 70185.3us
I20260709 13:09:36.093731 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:36.096138 22876 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:09:36.024161 27020 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:36.108245 23619 raft_consensus.cc:2393] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 11: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:36.110499 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:09:36.111542 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:36.116391 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:36.119220 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:36.120215 22876 raft_consensus.cc:2393] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 11: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:36.093083 26975 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:36.115111 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:36.124899 23622 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
W20260709 13:09:36.125726 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:36.103183 26989 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:36.132369 22955 raft_consensus.cc:1240] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 LEADER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:36.132279 23621 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 LEARNER]: Advancing to term 11
I20260709 13:09:36.143791 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:36.149555 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:36.150821 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:36.151114 23622 raft_consensus.cc:1275] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:09:36.151541 22879 raft_consensus.cc:2393] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 13: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:36.144593 22877 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:09:36.124150 26755 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:36.158303 22952 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11.
I20260709 13:09:36.161291 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:36.163946 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:36.172981 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:36.168325 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:36.155793 27048 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:36.188966 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:36.184783 22877 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 12.
I20260709 13:09:36.193419 22918 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:36.198743 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:36.200522 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:36.169919 26985 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:36.143306 26975 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:36.201304 22879 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
I20260709 13:09:36.216765 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:36.217965 23616 raft_consensus.cc:3055] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 13:09:36.218393 26762 consensus_meta.cc:287] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.920s	user 0.015s	sys 0.001s
I20260709 13:09:36.218398 23616 raft_consensus.cc:740] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:36.165648 26989 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:36.235459 22994 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:36.241946 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:36.240753 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:36.246109 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 63495.7us
I20260709 13:09:36.254748 22992 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:09:36.257377 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:36.229518 26943 raft_consensus.cc:2804] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:36.261567 22879 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 12.
I20260709 13:09:36.266558 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:36.199517 27048 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:36.269995 22843 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.012s	sys 0.020s
W20260709 13:09:36.270360 22843 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.012s	sys 0.020s
W20260709 13:09:36.278101 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 159397us
I20260709 13:09:36.288161 27048 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:36.289752 26989 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:36.293898 27022 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:36.294697 27022 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:36.297417 27022 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:36.297600 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:36.299006 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:36.299742 22950 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 12.
I20260709 13:09:36.299944 22919 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:36.300786 22872 raft_consensus.cc:1275] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:09:36.301167 26843 raft_consensus.cc:2804] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:36.301615 26843 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:36.302003 26843 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:36.302438 22950 raft_consensus.cc:1275] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:09:36.302377 23026 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
I20260709 13:09:36.228343 26755 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:36.301867 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:36.305305 22918 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:36.265067 26943 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:36.228011 26780 consensus_queue.cc:1048] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.055s
I20260709 13:09:36.312067 22948 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:36.307569 23028 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:09:36.312775 23028 raft_consensus.cc:606] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Received request to transfer leadership to TS aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:36.313136 23028 raft_consensus.cc:614] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:36.304524 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:36.302399 23588 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:36.316874 22994 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:36.320513 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:36.320755 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:36.321280 23625 raft_consensus.cc:1275] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:09:36.321897 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 100724us
I20260709 13:09:36.323786 22992 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:36.324817 26762 raft_consensus.cc:2764] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:09:36.324905 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:36.325629 27020 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:36.325858 27019 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:36.306679 27058 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:36.326427 27019 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:36.330542 26843 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:36.284049 23616 consensus_queue.cc:260] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:36.333236 23616 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:36.339016 26755 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:36.338924 26758 consensus_queue.cc:1048] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:09:36.339576 27032 consensus_queue.cc:1048] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:09:36.340081 27055 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:36.345774 22920 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:36.347602 22879 raft_consensus.cc:1275] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:09:36.349709 23022 raft_consensus.cc:2393] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 13: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:36.320504 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:36.350498 22946 raft_consensus.cc:2393] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 9: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:36.352676 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:36.355245 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:36.312239 26943 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:36.308218 26985 raft_consensus.cc:2749] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:36.333187 26843 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:36.361557 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 81991.2us
W20260709 13:09:36.362428 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 57196.5us
I20260709 13:09:36.328549 27019 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:36.361080 22872 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 12.
W20260709 13:09:36.388165 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 46440.3us
W20260709 13:09:36.368774 26549 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.701s	user 0.007s	sys 0.040s
I20260709 13:09:36.355515 26755 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:36.347159 27043 raft_consensus.cc:2804] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:36.387764 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:09:36.392467 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 68783.2us
I20260709 13:09:36.368425 26989 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:36.395601 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:36.395601 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 93257.1us
I20260709 13:09:36.326431 27058 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:36.392721 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:36.394976 26549 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.738s	user 0.008s	sys 0.040s
I20260709 13:09:36.394791 26764 raft_consensus.cc:2749] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:36.342517 27014 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:36.395146 26762 raft_consensus.cc:2804] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:09:36.415699 22877 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:36.416045 23616 raft_consensus.cc:1275] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 13:09:36.356547 27020 consensus_queue.cc:1048] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 13:09:36.395692 26758 raft_consensus.cc:2804] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:09:36.430203 23616 pending_rounds.cc:85] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2: Aborting all ops after (but not including) 6
I20260709 13:09:36.400141 27043 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:36.456421 22994 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:36.434787 22775 catalog_manager.cc:5697] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 5 to 7, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 7 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:36.433265 23621 raft_consensus.cc:1275] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 LEARNER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:09:36.453902 23616 pending_rounds.cc:107] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260709 13:09:36.490240 23616 raft_consensus.cc:2889] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:36.397372 27048 consensus_queue.cc:1059] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
W20260709 13:09:36.487798 22954 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.024s	sys 0.072s
W20260709 13:09:36.496589 22954 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.024s	sys 0.072s
I20260709 13:09:36.499956 23616 pending_rounds.cc:107] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260709 13:09:36.500564 23616 raft_consensus.cc:2889] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:36.502570 22920 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:36.467845 27043 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:36.503665 23026 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33420 (request call id 1966) took 1506 ms. Trace:
I20260709 13:09:36.505748 22918 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:36.512243 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:36.504101 23026 rpcz_store.cc:276] 0709 13:09:34.996736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:34.997184 (+   448us) service_pool.cc:224] Handling call
0709 13:09:36.469282 (+1472098us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b540018a954. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:36.503549 (+ 34267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2804070784}
I20260709 13:09:36.514140 22946 raft_consensus.cc:1275] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:09:36.486375 26758 raft_consensus.cc:697] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:36.522034 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:36.534092 22920 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:36.438360 26755 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:36.440862 26794 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:36.536948 27032 raft_consensus.cc:2804] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:09:36.538106 27091 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:36.538782 27057 raft_consensus.cc:2749] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:09:36.539674 27038 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:36.503741 26762 raft_consensus.cc:697] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:36.516418 26975 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:36.545306 23027 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 12.
I20260709 13:09:36.509022 26549 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:36.500171 26989 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:36.575757 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:36.575613 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 62997.1us
I20260709 13:09:36.576503 22878 raft_consensus.cc:1275] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:09:36.576782 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 78500us
I20260709 13:09:36.581012 22879 raft_consensus.cc:2410] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 13:09:36.528860 26808 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:36.585879 27092 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:36.586313 27014 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:36.586022 26762 consensus_queue.cc:237] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:36.586449 27092 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:09:36.579568 27077 raft_consensus.cc:2804] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:36.585021 22994 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:36.518806 26626 consensus_queue.cc:1059] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:09:36.593379 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:09:36.594751 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 53512.3us
W20260709 13:09:36.595366 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 59201.2us
I20260709 13:09:36.596249 22994 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:36.598806 22872 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 10.
W20260709 13:09:36.603153 26944 consensus_meta.cc:287] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 1.321s	user 0.010s	sys 0.006s
I20260709 13:09:36.603757 26861 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:36.603847 26944 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:36.604282 26861 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:36.604410 26944 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:36.606621 26944 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:36.607385 27014 raft_consensus.cc:2804] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:09:36.607406 22950 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:09:36.608549 26794 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:36.609644 26989 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:36.610368 26989 consensus_queue.cc:1048] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:09:36.614459 26764 raft_consensus.cc:3055] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:09:36.614869 26764 raft_consensus.cc:740] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:36.615758 26764 consensus_queue.cc:260] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:36.583611 27055 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:36.590720 27048 consensus_queue.cc:1048] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:09:36.581857 27032 raft_consensus.cc:697] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:36.625686 27032 consensus_queue.cc:237] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:36.636909 22877 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 13.
I20260709 13:09:36.638399 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:36.641420 26975 raft_consensus.cc:3055] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:09:36.641875 26975 raft_consensus.cc:740] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:36.642992 26975 consensus_queue.cc:260] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:36.643805 22878 raft_consensus.cc:1240] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:36.644172 26975 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:36.644906 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:36.647053 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 48450.6us
I20260709 13:09:36.651152 22948 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 12.
I20260709 13:09:36.591806 27077 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:36.644785 26626 raft_consensus.cc:2804] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:36.639531 22992 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:36.632682 22775 catalog_manager.cc:5697] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 5 to 9, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 9 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:36.579483 26549 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:36.666800 22874 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.608s	user 0.017s	sys 0.055s
W20260709 13:09:36.669368 22874 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.017s	sys 0.055s
W20260709 13:09:36.671909 22880 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.009s	sys 0.036s
W20260709 13:09:36.674110 22880 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.009s	sys 0.036s
W20260709 13:09:36.681583 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 86129us
I20260709 13:09:36.670241 22920 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:36.654642 27077 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:36.654682 26626 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:36.645027 27014 raft_consensus.cc:697] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:36.598169 26808 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:36.687566 22992 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:36.598335 27038 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:36.696177 22920 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:36.677335 27055 raft_consensus.cc:2749] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:09:36.601006 27091 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:36.704310 22918 leader_election.cc:400] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 10. Current term is 11.
I20260709 13:09:36.704795 22918 leader_election.cc:403] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:09:36.675403 27092 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:36.687429 26764 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:36.696725 26794 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:36.715978 26794 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:36.716473 26794 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:36.685629 26758 consensus_queue.cc:237] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:36.689285 26626 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:36.752543 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:36.754307 22878 raft_consensus.cc:2468] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
W20260709 13:09:36.759580 22873 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.007s	sys 0.031s
W20260709 13:09:36.761947 22873 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.007s	sys 0.031s
I20260709 13:09:36.762082 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:36.762655 22878 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:36.759989 27022 raft_consensus.cc:2749] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:36.752506 26944 raft_consensus.cc:2804] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 13:09:36.770321 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:36.771026 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 88475.8us
W20260709 13:09:36.764876 27076 log.cc:927] Time spent T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 13:09:36.773643 22992 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:36.783217 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:36.786303 26794 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:36.787812 27043 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:36.791399 26943 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:36.801946 27024 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:36.802471 27024 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:36.803781 26943 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:36.804399 26943 consensus_queue.cc:1048] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:09:36.806313 26943 consensus_queue.cc:1048] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:09:36.809054 27043 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:36.809675 27043 raft_consensus.cc:2749] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 10. Current term is 11.
I20260709 13:09:36.774210 26944 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:36.834196 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:36.797362 22992 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:09:36.801527 22951 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.701s	user 0.010s	sys 0.050s
I20260709 13:09:36.792805 21756 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 13:09:36.837452 22951 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.860s	user 0.010s	sys 0.051s
W20260709 13:09:36.783286 27095 log.cc:927] Time spent T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 13:09:36.844985 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:36.795246 26509 raft_consensus.cc:2749] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:09:36.856316 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 59068.5us
I20260709 13:09:36.858459 22877 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11.
I20260709 13:09:36.862989 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:36.863525 23026 raft_consensus.cc:1275] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:09:36.841725 26944 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:36.804410 27024 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:36.843626 26755 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:36.913270 27093 log.cc:927] Time spent T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.216s	user 0.001s	sys 0.001s
I20260709 13:09:36.998816 23022 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 10.
I20260709 13:09:36.995249 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:37.004324 22877 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
I20260709 13:09:36.865118 27022 raft_consensus.cc:2749] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 13:09:36.838764 22947 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.017s	sys 0.022s
W20260709 13:09:37.032850 22947 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.740s	user 0.018s	sys 0.022s
I20260709 13:09:36.848912 27014 consensus_queue.cc:237] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:37.035840 22877 raft_consensus.cc:1240] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 8. Current term is 9. Ops: []
W20260709 13:09:37.037382 22875 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.784s	user 0.014s	sys 0.030s
W20260709 13:09:37.037796 22875 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.822s	user 0.014s	sys 0.030s
I20260709 13:09:37.037607 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:37.038504 23028 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 10.
W20260709 13:09:37.038968 27094 log.cc:927] Time spent T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
I20260709 13:09:37.055800 22877 raft_consensus.cc:1275] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 13:09:37.074043 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 74910.7us
I20260709 13:09:37.076721 22876 raft_consensus.cc:1275] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:09:37.079048 23028 raft_consensus.cc:1240] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 11. Current term is 12. Ops: []
I20260709 13:09:37.081516 22877 raft_consensus.cc:1275] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 13:09:37.085561 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:37.086333 23026 raft_consensus.cc:2468] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 7.
I20260709 13:09:37.066880 26236 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:37.101708 22876 raft_consensus.cc:1240] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 12. Current term is 13. Ops: [11.10-12.11]
I20260709 13:09:37.113148 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:37.110460 22954 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 652) took 1366 ms. Trace:
I20260709 13:09:37.115499 22876 raft_consensus.cc:1240] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 6. Current term is 7. Ops: [4.7-6.8]
I20260709 13:09:37.115983 22994 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:37.089983 26808 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:37.094830 27038 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:37.095299 26549 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:37.124593 21756 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:09:37.129022 22994 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:37.144961 27055 raft_consensus.cc:2804] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:37.145545 27055 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:37.146008 27055 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:37.114524 26794 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:37.163987 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:37.164968 22876 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
W20260709 13:09:37.167157 22774 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33466 (request call id 291) took 17908 ms (client timeout 15000 ms). Trace:
I20260709 13:09:37.173326 26985 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:37.174038 22874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 1962) took 2684 ms. Trace:
I20260709 13:09:37.173951 26985 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:37.174410 22874 rpcz_store.cc:276] 0709 13:09:34.489655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:34.490015 (+   360us) service_pool.cc:224] Handling call
0709 13:09:34.589647 (+ 99632us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:34.864000 (+274353us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:09:35.088511 (+224511us) log.cc:844] Serialized 14 byte log entry
0709 13:09:36.669768 (+1581257us) log.cc:844] Serialized 14 byte log entry
0709 13:09:36.687152 (+ 17384us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:36.693473 (+  6321us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:09:36.693542 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:37.173763 (+480221us) spinlock_profiling.cc:243] Waited 274 ms on lock 0x7b540030c5d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:37.173802 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:37.173970 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3779,"prepare.queue_time_us":1441309,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":66,"raft.queue_time_us":61768,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":6119,"spinlock_wait_cycles":630415104,"thread_start_us":1572747,"threads_started":3,"wal-append.queue_time_us":858524}
I20260709 13:09:37.175997 26985 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:09:37.177848 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 690582us
I20260709 13:09:37.178634 22919 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:37.180231 27024 raft_consensus.cc:2804] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:09:37.180855 27043 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:37.181021 27024 raft_consensus.cc:697] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:37.182298 27024 consensus_queue.cc:237] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:37.184980 22775 catalog_manager.cc:5697] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
W20260709 13:09:37.185595 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 59630us
I20260709 13:09:37.186676 22872 raft_consensus.cc:1240] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 10 LEADER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:37.186362 22775 catalog_manager.cc:5697] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:37.115309 22954 rpcz_store.cc:276] 0709 13:09:35.741008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:35.744441 (+  3433us) service_pool.cc:224] Handling call
0709 13:09:35.764957 (+ 20516us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:35.959749 (+194792us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:09:35.959777 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:35.960323 (+   546us) log.cc:844] Serialized 26 byte log entry
0709 13:09:36.497987 (+537664us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:09:36.498047 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:37.094210 (+596163us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b54004123d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:37.094274 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:37.094335 (+    61us) raft_consensus.cc:1685] finished
0709 13:09:37.094355 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:37.103505 (+  9150us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4346,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":120,"spinlock_wait_cycles":438005504,"thread_start_us":537229,"threads_started":1,"wal-append.queue_time_us":818344}
I20260709 13:09:37.142611 22994 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:09:37.206673 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:37.214366 22878 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
I20260709 13:09:37.218583 22878 raft_consensus.cc:1275] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:09:37.132229 27022 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:37.220913 22874 raft_consensus.cc:1275] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 13:09:37.222011 22874 pending_rounds.cc:85] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 8
I20260709 13:09:37.222241 22874 pending_rounds.cc:107] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 13:09:37.222596 22874 raft_consensus.cc:2889] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:37.141376 26509 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:37.224138 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 81624.1us
W20260709 13:09:37.221099 22843 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.001s	sys 0.018s
W20260709 13:09:37.236500 22843 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.695s	user 0.001s	sys 0.019s
W20260709 13:09:37.236922 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 695787us
I20260709 13:09:37.227715 22994 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:09:37.220984 23590 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.695s	user 0.000s	sys 0.019s
I20260709 13:09:37.219646 26626 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:37.230451 26780 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:37.231118 27057 raft_consensus.cc:2804] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:09:37.212301 27019 consensus_queue.cc:1048] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 13:09:37.275291 27019 consensus_queue.cc:1048] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:09:37.215691 27092 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:37.276965 27092 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:37.215662 27038 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:37.240772 22953 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 1933) took 1232 ms. Trace:
W20260709 13:09:37.242125 23588 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.019s	sys 0.027s
W20260709 13:09:37.278149 23588 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.019s	sys 0.028s
I20260709 13:09:37.133939 27077 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:37.252548 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:37.272998 26944 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:37.278716 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 598994us
W20260709 13:09:37.243201 23590 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.717s	user 0.000s	sys 0.020s
I20260709 13:09:37.282289 22994 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:09:37.282796 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 789512us
I20260709 13:09:37.212047 26794 consensus_queue.cc:1048] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.035s
W20260709 13:09:37.288128 22871 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.013s	sys 0.045s
I20260709 13:09:37.286903 22994 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:09:37.290064 22880 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.015s	sys 0.001s
W20260709 13:09:37.290575 22880 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.015s	sys 0.001s
W20260709 13:09:37.291487 22873 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.008s	sys 0.050s
I20260709 13:09:37.292244 23028 raft_consensus.cc:2393] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 8: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
W20260709 13:09:37.288967 22871 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.013s	sys 0.045s
W20260709 13:09:37.280781 26758 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.010s	sys 0.036s
I20260709 13:09:37.278394 27020 raft_consensus.cc:2804] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 13:09:37.291786 22873 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.008s	sys 0.050s
I20260709 13:09:37.277987 22953 rpcz_store.cc:276] 0709 13:09:35.900829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:35.901085 (+   256us) service_pool.cc:224] Handling call
0709 13:09:35.901478 (+   393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:35.901990 (+   512us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:35.902024 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:35.902653 (+   629us) log.cc:844] Serialized 26 byte log entry
0709 13:09:36.163082 (+260429us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:09:36.163158 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:36.163543 (+   385us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:37.132923 (+969380us) raft_consensus.cc:1685] finished
0709 13:09:37.132949 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:37.133117 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1441,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":179,"thread_start_us":259965,"threads_started":1,"wal-append.queue_time_us":630896}
W20260709 13:09:37.295336 22920 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.006s	sys 0.011s
W20260709 13:09:37.298097 22920 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.006s	sys 0.012s
W20260709 13:09:37.298482 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 542567us
W20260709 13:09:37.289979 26843 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.753s	user 0.009s	sys 0.016s
I20260709 13:09:37.297470 23590 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:37.286846 27043 raft_consensus.cc:2749] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:37.335732 22880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 1977) took 1239 ms. Trace:
I20260709 13:09:37.288484 26985 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:37.340380 22871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2469) took 1212 ms. Trace:
I20260709 13:09:37.343423 22873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 1981) took 1219 ms. Trace:
W20260709 13:09:37.331058 22844 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.63.4:44043 timed out after 1.679s (SENT) [suppressed 21 similar messages]
W20260709 13:09:37.298676 26762 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.004s	sys 0.059s
W20260709 13:09:37.296912 26758 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.010s	sys 0.037s
I20260709 13:09:37.363405 26758 consensus_queue.cc:1048] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:09:37.388278 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:37.389040 23621 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:37.280766 27022 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:37.365818 27077 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:37.296200 27020 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:37.354915 22843 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:37.344691 27048 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:37.344431 26985 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:37.371248 26626 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:09:37.372418 26975 consensus_meta.cc:287] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.728s	user 0.013s	sys 0.000s
W20260709 13:09:37.340054 26843 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.803s	user 0.009s	sys 0.016s
I20260709 13:09:37.388475 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:37.286847 27019 consensus_queue.cc:1048] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 13:09:37.433667 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:37.344843 22873 rpcz_store.cc:276] 0709 13:09:36.123842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:36.128871 (+  5029us) service_pool.cc:224] Handling call
0709 13:09:36.140689 (+ 11818us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:36.153317 (+ 12628us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:09:36.762367 (+609050us) log.cc:844] Serialized 14 byte log entry
0709 13:09:37.342995 (+580628us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:37.343063 (+    68us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:09:37.343121 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:37.343206 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:37.343358 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50885,"prepare.queue_time_us":884105,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":27,"raft.queue_time_us":45749,"thread_start_us":1127579,"threads_started":2}
I20260709 13:09:37.429592 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:37.286115 26780 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:37.167676 22774 rpcz_store.cc:269] 0709 13:09:19.263086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:19.292898 (+ 29812us) service_pool.cc:224] Handling call
0709 13:09:37.166924 (+17874026us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:09:25.510999 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:09:25.597874 (+ 86875us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:09:25.612463 (+ 14589us) write_op.cc:435] Acquired schema lock
0709 13:09:25.627088 (+ 14625us) tablet.cc:662] Decoding operations
0709 13:09:25.742754 (+115666us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:09:25.742835 (+    81us) write_op.cc:641] Partition lock acquired for write op
0709 13:09:25.742863 (+    28us) tablet.cc:685] Acquiring locks for 25 operations
0709 13:09:25.862388 (+119525us) tablet.cc:701] Row locks acquired
0709 13:09:25.862444 (+    56us) write_op.cc:260] PREPARE: finished
0709 13:09:25.862752 (+   308us) write_op.cc:270] Start()
0709 13:09:25.862946 (+   194us) write_op.cc:276] Timestamp: P: 1783602565862573 usec, L: 0
0709 13:09:25.862970 (+    24us) op_driver.cc:348] REPLICATION: starting
0709 13:09:25.973703 (+110733us) log.cc:844] Serialized 9716 byte log entry
0709 13:09:27.180515 (+1206812us) op_driver.cc:464] REPLICATION: finished
0709 13:09:29.195302 (+2014787us) spinlock_profiling.cc:243] Waited 67.9 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f6132cdb0ac 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf7c02 00007f613981bef0
0709 13:09:29.198114 (+  2812us) write_op.cc:301] APPLY: starting
0709 13:09:29.198216 (+   102us) tablet.cc:1367] starting BulkCheckPresence
0709 13:09:29.198600 (+   384us) tablet.cc:1370] finished BulkCheckPresence
0709 13:09:29.198629 (+    29us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:09:29.202119 (+  3490us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:09:29.202592 (+   473us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=25
0709 13:09:29.202644 (+    52us) write_op.cc:312] APPLY: finished
0709 13:09:29.203156 (+   512us) log.cc:844] Serialized 170 byte log entry
0709 13:09:29.204099 (+   943us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:29.204300 (+   201us) write_op.cc:454] Released schema lock
0709 13:09:29.204379 (+    79us) write_op.cc:341] FINISH: Updating metrics
0709 13:09:29.699593 (+495214us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:29.699624 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":113435,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":1999318,"apply.run_cpu_time_us":6252,"apply.run_wall_time_us":6747,"num_ops":25,"prepare.queue_time_us":465468,"prepare.run_cpu_time_us":27312,"prepare.run_wall_time_us":638151,"raft.queue_time_us":258273,"raft.run_cpu_time_us":23470,"raft.run_wall_time_us":2570206,"replication_time_us":1287317,"spinlock_wait_cycles":156126080,"thread_start_us":2879020,"threads_started":5,"wal-append.queue_time_us":392958,"wal-append.run_cpu_time_us":22274,"wal-append.run_wall_time_us":2677814}]]}
I20260709 13:09:37.436249 23027 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 9.
I20260709 13:09:37.400799 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
W20260709 13:09:37.403146 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 115116us
W20260709 13:09:37.360008 26762 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.005s	sys 0.059s
I20260709 13:09:37.287977 26509 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:09:37.438658 22875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 1983) took 1306 ms. Trace:
W20260709 13:09:37.369131 22921 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.043s	sys 0.023s
W20260709 13:09:37.440235 22921 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.857s	user 0.044s	sys 0.023s
I20260709 13:09:37.348417 27038 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:37.416620 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:37.418458 27048 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:37.434646 23617 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:37.416421 26989 raft_consensus.cc:2804] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:37.417505 27020 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:37.438702 23619 raft_consensus.cc:2393] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 11: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
W20260709 13:09:37.406083 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 70826.6us
I20260709 13:09:37.426735 26985 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:37.469871 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:37.470889 22947 raft_consensus.cc:2393] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 8: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:37.472471 26236 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:37.473111 26236 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:37.341192 22880 rpcz_store.cc:276] 0709 13:09:36.059035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:36.070581 (+ 11546us) service_pool.cc:224] Handling call
0709 13:09:36.070981 (+   400us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:36.127406 (+ 56425us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:09:36.679895 (+552489us) log.cc:844] Serialized 14 byte log entry
0709 13:09:37.297496 (+617601us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:37.297562 (+    66us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:37.297627 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:37.297707 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:37.297847 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":906427,"prepare.run_cpu_time_us":28,"prepare.run_wall_time_us":29,"raft.queue_time_us":47950,"thread_start_us":1103946,"threads_started":2}
I20260709 13:09:37.477231 26861 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:37.477331 26762 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:37.478044 26762 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:37.478924 27091 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:37.482139 26780 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:37.483043 26944 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:37.416900 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:37.403772 26764 consensus_queue.cc:1059] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 13:09:37.487187 26764 raft_consensus.cc:3055] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:09:37.487565 26764 raft_consensus.cc:740] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:37.462911 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:37.463557 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:37.488796 22950 raft_consensus.cc:2393] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 13: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:37.405992 26794 consensus_queue.cc:1048] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:09:37.463274 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:37.465737 22843 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:37.439908 26626 raft_consensus.cc:2804] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:09:37.508937 22843 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:37.467001 27048 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:37.524458 23621 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 13.
I20260709 13:09:37.485049 26989 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:37.480495 26762 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:37.526628 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:37.526729 22951 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2625) took 1724 ms. Trace:
W20260709 13:09:37.516450 22918 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.750s	user 0.006s	sys 0.035s
I20260709 13:09:37.529309 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:37.531144 22918 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.764s	user 0.007s	sys 0.035s
I20260709 13:09:37.530843 23022 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
I20260709 13:09:37.531798 23020 raft_consensus.cc:2410] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:09:37.533653 23022 raft_consensus.cc:1275] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:09:37.445273 26975 consensus_queue.cc:1048] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.066s
I20260709 13:09:37.538347 23025 raft_consensus.cc:1275] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:09:37.546378 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:37.510650 27112 raft_consensus.cc:2749] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:09:37.552316 23588 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:37.553390 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:37.549273 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:37.551550 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:37.547041 23026 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 10.
I20260709 13:09:37.518484 26626 raft_consensus.cc:697] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:37.527061 27091 raft_consensus.cc:2804] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:37.530779 22951 rpcz_store.cc:276] 0709 13:09:35.802778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:35.823639 (+ 20861us) service_pool.cc:224] Handling call
0709 13:09:35.824097 (+   458us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:35.824535 (+   438us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:09:36.842110 (+1017575us) log.cc:844] Serialized 14 byte log entry
0709 13:09:37.506416 (+664306us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:37.506469 (+    53us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:37.526486 (+ 20017us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:37.526561 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:37.526677 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":152072,"prepare.queue_time_us":152700,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":25,"raft.queue_time_us":16366,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":8958,"thread_start_us":1313744,"threads_started":2}
I20260709 13:09:37.536808 26989 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:37.565270 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:37.533289 26509 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:37.568003 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:37.569607 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:37.527297 26802 consensus_queue.cc:1048] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.032s
W20260709 13:09:37.572700 27116 log.cc:927] Time spent T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.168s	user 0.001s	sys 0.000s
W20260709 13:09:37.594719 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 263678us
I20260709 13:09:37.577917 22874 raft_consensus.cc:1275] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:09:37.596036 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:37.610016 22880 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:37.573691 26808 raft_consensus.cc:2749] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:37.600975 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:37.611038 22873 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11.
I20260709 13:09:37.585462 27092 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:37.602257 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 925227us
I20260709 13:09:37.604645 23588 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:37.556644 27014 consensus_queue.cc:1048] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:09:37.565788 27091 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:37.536947 26764 consensus_queue.cc:260] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:37.613365 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 307282us
I20260709 13:09:37.574009 26762 consensus_queue.cc:1048] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:09:37.649053 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:37.645885 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:37.613039 27057 raft_consensus.cc:697] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:37.658680 22918 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:37.637893 27091 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:37.638193 27058 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:37.675370 22874 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:09:37.682224 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:37.648944 27019 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:37.691157 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:37.691953 22948 raft_consensus.cc:2368] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 8. Current term is 9.
I20260709 13:09:37.660696 26808 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:37.714380 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:37.715037 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:37.721676 22873 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:09:37.720481 22954 raft_consensus.cc:1240] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:37.664844 27024 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:37.702796 23617 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11.
I20260709 13:09:37.730350 22880 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
I20260709 13:09:37.690241 26764 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:37.680605 27112 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:37.741185 22952 raft_consensus.cc:1240] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 11. Current term is 12. Ops: []
W20260709 13:09:37.740202 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 91676.1us
I20260709 13:09:37.691383 26626 consensus_queue.cc:237] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:37.749447 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 201848us
I20260709 13:09:37.702600 27092 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:37.723493 26509 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:37.703577 27032 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:37.722999 26802 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:37.752846 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:37.780241 22952 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 7.
I20260709 13:09:37.774250 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:37.719894 26755 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:37.703444 27019 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:37.754005 27014 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:37.794461 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 44137.2us
I20260709 13:09:37.797083 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:37.798771 22994 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:37.779629 26802 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:37.839068 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:37.807914 22952 raft_consensus.cc:3055] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:09:37.806190 22994 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:37.844473 22952 raft_consensus.cc:740] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:37.846056 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:09:37.852494 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 247927us
I20260709 13:09:37.748216 27024 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:37.854585 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 48746us
I20260709 13:09:37.855208 22994 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:37.749818 26808 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:37.778664 27092 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:37.855829 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:37.865675 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:37.861465 22947 raft_consensus.cc:1275] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:09:37.864192 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:37.866912 22949 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:37.844944 26802 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:09:37.876214 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 87756.4us
I20260709 13:09:37.786633 26509 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:37.856544 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:37.878152 22954 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 9.
I20260709 13:09:37.847702 27133 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:37.440115 22875 rpcz_store.cc:276] 0709 13:09:36.132227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:36.165203 (+ 32976us) service_pool.cc:224] Handling call
0709 13:09:36.185194 (+ 19991us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:36.203810 (+ 18616us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:09:37.038311 (+834501us) log.cc:844] Serialized 14 byte log entry
0709 13:09:37.437979 (+399668us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:37.438041 (+    62us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:09:37.438088 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:37.438484 (+   396us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:37.438603 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15241,"prepare.queue_time_us":834201,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":35,"raft.queue_time_us":15381,"thread_start_us":1208056,"threads_started":2}
I20260709 13:09:37.882673 22948 raft_consensus.cc:1240] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 13:09:37.845491 27119 raft_consensus.cc:2749] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:37.856601 22952 consensus_queue.cc:260] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:37.886593 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:37.785516 27032 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:37.889945 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:37.795596 26755 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:37.890944 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:09:37.868680 27114 log.cc:927] Time spent T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.270s	user 0.001s	sys 0.000s
I20260709 13:09:37.867992 26762 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:09:37.894104 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 92987.2us
I20260709 13:09:37.802145 27110 consensus_queue.cc:1059] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.280s
I20260709 13:09:37.897670 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:37.851867 26780 consensus_queue.cc:1048] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.035s
W20260709 13:09:37.886962 27121 log.cc:927] Time spent T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.309s	user 0.000s	sys 0.001s
I20260709 13:09:37.803488 27014 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:37.898525 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:37.900880 22873 raft_consensus.cc:2393] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 8: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:37.900604 22952 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:37.902263 22947 raft_consensus.cc:1275] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 13:09:37.887346 27122 log.cc:927] Time spent T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.281s	user 0.000s	sys 0.001s
W20260709 13:09:37.910483 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:37.926389 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:37.899012 27117 log.cc:927] Time spent T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.297s	user 0.000s	sys 0.001s
I20260709 13:09:37.344713 22871 rpcz_store.cc:276] 0709 13:09:36.122150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:36.127164 (+  5014us) service_pool.cc:224] Handling call
0709 13:09:36.141943 (+ 14779us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:36.177542 (+ 35599us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:09:36.662762 (+485220us) log.cc:844] Serialized 14 byte log entry
0709 13:09:37.333683 (+670921us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:37.333740 (+    57us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:37.333782 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:37.333857 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:37.334001 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5401,"prepare.queue_time_us":848878,"prepare.run_cpu_time_us":22,"prepare.run_wall_time_us":22,"raft.queue_time_us":44927,"spinlock_wait_cycles":878592,"thread_start_us":1064761,"threads_started":2}
I20260709 13:09:37.919080 27022 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:37.927095 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:37.790961 27112 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:37.933796 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 78641.1us
I20260709 13:09:37.898190 26762 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:37.809559 27057 consensus_queue.cc:237] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:37.936506 22994 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:37.948764 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:37.949407 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:37.950618 22875 raft_consensus.cc:2393] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 8: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:37.942046 22918 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:37.961992 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:37.966502 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:37.921166 27145 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:37.967167 22954 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
I20260709 13:09:37.949535 22946 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
W20260709 13:09:37.989027 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 134086us
I20260709 13:09:38.021610 23588 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:37.989118 22992 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:38.022948 22955 raft_consensus.cc:1275] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 13:09:37.949864 27124 log.cc:927] Time spent T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.274s	user 0.001s	sys 0.000s
I20260709 13:09:37.945783 27022 raft_consensus.cc:2804] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:38.025534 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:37.946343 26762 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:38.020946 22843 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:38.036175 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 47212.8us
I20260709 13:09:38.038082 22992 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:37.893034 27133 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:37.948261 27077 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:37.946640 27146 raft_consensus.cc:2804] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:09:38.119910 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 98391.9us
W20260709 13:09:38.117801 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 127932us
I20260709 13:09:37.970513 27134 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:38.124377 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 220669us
I20260709 13:09:38.126705 22843 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:38.127135 23588 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:38.130165 22843 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:38.029022 27022 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:38.116113 27125 log.cc:927] Time spent T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.305s	user 0.000s	sys 0.001s
I20260709 13:09:38.129187 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:37.967746 27058 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:38.143234 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:38.174117 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:38.175028 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:37.966857 27115 consensus_queue.cc:1048] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.198s
I20260709 13:09:38.178843 22952 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
I20260709 13:09:38.183050 22946 raft_consensus.cc:2393] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 11: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:38.052716 27148 raft_consensus.cc:2804] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:09:38.184250 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 146255us
I20260709 13:09:38.028498 27130 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:38.190421 22992 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:38.195042 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:38.128957 27146 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:38.146760 27022 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:09:38.196365 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 66157.7us
W20260709 13:09:38.201179 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 301325us
I20260709 13:09:38.205111 22843 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:38.205222 23590 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:38.202685 22955 raft_consensus.cc:1275] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:09:38.204077 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:38.202512 27146 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:38.207479 22954 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6.
I20260709 13:09:38.210196 22843 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:38.211217 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:38.212004 22954 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 7.
I20260709 13:09:38.133733 27110 consensus_queue.cc:1048] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 13:09:38.181778 26549 raft_consensus.cc:2804] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:38.024842 27145 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:38.213398 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:38.134882 27077 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:38.213147 26236 consensus_meta.cc:287] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.739s	user 0.007s	sys 0.007s
I20260709 13:09:38.198359 27148 raft_consensus.cc:697] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:38.195801 27136 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:38.150134 27144 consensus_queue.cc:1059] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.099s
I20260709 13:09:38.193470 27019 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:38.147588 27134 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:38.187476 27058 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:38.203871 27130 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:38.229370 22951 raft_consensus.cc:2393] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 9: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
W20260709 13:09:38.430426 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 240102us
W20260709 13:09:38.452342 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:38.485533 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 287257us
W20260709 13:09:38.228582 27127 log.cc:927] Time spent T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.360s	user 0.000s	sys 0.001s
W20260709 13:09:38.504594 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:38.423403 27135 log.cc:927] Time spent T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.364s	user 0.001s	sys 0.001s
I20260709 13:09:38.423054 27153 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:38.245791 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:38.221607 26549 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
W20260709 13:09:38.392422 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 51576.4us
W20260709 13:09:38.451011 27129 log.cc:927] Time spent T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.666s	user 0.000s	sys 0.002s
I20260709 13:09:38.404636 27168 raft_consensus.cc:2804] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:38.423130 27150 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:38.422873 27151 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:38.506174 27123 log.cc:927] Time spent T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.376s	user 0.001s	sys 0.000s
I20260709 13:09:38.357115 26755 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:38.422192 27152 raft_consensus.cc:2804] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:38.356774 26808 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:38.228180 27024 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:38.422968 27156 raft_consensus.cc:2749] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:09:38.262001 27126 log.cc:927] Time spent T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.157s	user 0.000s	sys 0.001s
I20260709 13:09:38.449635 27077 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:38.396724 27032 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:38.449275 27145 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:38.529170 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:38.423259 27165 raft_consensus.cc:2749] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:09:38.515162 26236 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:38.392167 27060 raft_consensus.cc:2749] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:09:38.533033 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 322882us
W20260709 13:09:38.548532 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 343348us
I20260709 13:09:38.452015 27038 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:38.273957 27055 consensus_meta.cc:287] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 1.127s	user 0.014s	sys 0.000s
I20260709 13:09:38.357193 26509 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:38.390195 27163 raft_consensus.cc:2804] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:09:38.444541 27143 log.cc:927] Time spent T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.478s	user 0.000s	sys 0.001s
I20260709 13:09:38.504652 27112 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:38.303890 27115 raft_consensus.cc:2804] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:38.282632 27092 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:38.516193 26549 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:38.636071 27115 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:38.636523 27115 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:38.528313 27168 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:38.568349 22874 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.665s	user 0.026s	sys 0.022s
I20260709 13:09:38.422359 27157 consensus_queue.cc:1048] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.362s
I20260709 13:09:38.640782 27169 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:38.640976 27168 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 8
W20260709 13:09:38.641211 22874 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.738s	user 0.026s	sys 0.022s
I20260709 13:09:38.641443 27169 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
W20260709 13:09:38.555842 27147 log.cc:927] Time spent T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.434s	user 0.000s	sys 0.001s
I20260709 13:09:38.641903 27169 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:38.574806 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:38.642031 22874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2492) took 1051 ms. Trace:
I20260709 13:09:38.642594 22871 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
I20260709 13:09:38.642464 22874 rpcz_store.cc:276] 0709 13:09:37.590415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:37.682778 (+ 92363us) service_pool.cc:224] Handling call
0709 13:09:37.714988 (+ 32210us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:37.858816 (+143828us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:37.858877 (+    61us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:37.868948 (+ 10071us) log.cc:844] Serialized 26 byte log entry
0709 13:09:38.641716 (+772768us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:09:38.641754 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:38.641820 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:38.641861 (+    41us) raft_consensus.cc:1685] finished
0709 13:09:38.641872 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:38.641979 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":202140,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":14670,"thread_start_us":856753,"threads_started":2,"wal-append.queue_time_us":740146}
I20260709 13:09:38.643612 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:38.644182 22871 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
I20260709 13:09:38.409616 27014 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:38.627229 27055 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:38.550920 27150 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:38.647698 22994 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:09:38.595677 22949 consensus_meta.cc:287] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.728s	user 0.010s	sys 0.000s
I20260709 13:09:38.555817 27152 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:38.622064 22879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 1995) took 2166 ms. Trace:
I20260709 13:09:38.649111 22949 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
I20260709 13:09:38.630507 22843 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:38.632397 22953 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2643) took 1124 ms. Trace:
I20260709 13:09:38.635248 27166 raft_consensus.cc:2749] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:09:38.635696 27142 log.cc:927] Time spent T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.489s	user 0.000s	sys 0.001s
I20260709 13:09:38.524449 27153 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:38.531316 27136 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:38.621448 26985 consensus_meta.cc:287] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 1.151s	user 0.011s	sys 0.001s
I20260709 13:09:38.555614 27151 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:38.698556 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 50929.5us
W20260709 13:09:38.700541 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 267407us
I20260709 13:09:38.698408 22876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 1996) took 2195 ms. Trace:
I20260709 13:09:38.734167 22992 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:09:38.735740 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 166187us
I20260709 13:09:38.648866 27152 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:09:38.736364 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 115575us
I20260709 13:09:38.736181 22992 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:38.734030 22876 rpcz_store.cc:276] 0709 13:09:36.503041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:37.165961 (+662920us) service_pool.cc:224] Handling call
0709 13:09:37.166421 (+   460us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:37.185905 (+ 19484us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:37.185934 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:37.213052 (+ 27118us) log.cc:844] Serialized 26 byte log entry
0709 13:09:37.559197 (+346145us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:09:37.559243 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:37.559311 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:38.450316 (+891005us) spinlock_profiling.cc:243] Waited 12.6 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:09:38.650804 (+200488us) raft_consensus.cc:1685] finished
0709 13:09:38.650826 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:38.698333 (+ 47507us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":34633,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":8527,"spinlock_wait_cycles":28888832,"thread_start_us":328186,"threads_started":1,"wal-append.queue_time_us":463498}
W20260709 13:09:38.740202 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 109753us
W20260709 13:09:38.709909 27167 log.cc:927] Time spent T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
W20260709 13:09:38.709514 27175 log.cc:927] Time spent T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:09:38.783507 22843 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:38.649386 22879 rpcz_store.cc:276] 0709 13:09:36.455840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:37.161597 (+705757us) service_pool.cc:224] Handling call
0709 13:09:37.162053 (+   456us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:37.162504 (+   451us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:37.162543 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:37.163110 (+   567us) log.cc:844] Serialized 26 byte log entry
0709 13:09:37.549632 (+386522us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:37.549670 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:37.549738 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:38.621852 (+1072114us) raft_consensus.cc:1685] finished
0709 13:09:38.621874 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:38.622002 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":23411,"raft.run_cpu_time_us":180,"raft.run_wall_time_us":20827,"thread_start_us":386133,"threads_started":1,"wal-append.queue_time_us":622322}
I20260709 13:09:38.731509 22953 rpcz_store.cc:276] 0709 13:09:37.508494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:37.508939 (+   445us) service_pool.cc:224] Handling call
0709 13:09:37.528977 (+ 20038us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:37.564563 (+ 35586us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:37.564593 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:37.574873 (+ 10280us) log.cc:844] Serialized 52 byte log entry
0709 13:09:37.808416 (+233543us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:09:37.933181 (+124765us) log.cc:844] Serialized 14 byte log entry
0709 13:09:37.945380 (+ 12199us) log.cc:844] Serialized 14 byte log entry
0709 13:09:37.967887 (+ 22507us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:37.967983 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:38.632179 (+664196us) raft_consensus.cc:1685] finished
0709 13:09:38.632197 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:38.632338 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47621,"prepare.queue_time_us":1319001,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":75,"raft.queue_time_us":56516,"raft.run_cpu_time_us":451,"raft.run_wall_time_us":81127,"spinlock_wait_cycles":25389824,"thread_start_us":271741,"threads_started":2,"wal-append.queue_time_us":302064}
I20260709 13:09:38.757480 23588 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:38.707517 27170 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:38.732093 27048 consensus_meta.cc:287] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 1.194s	user 0.005s	sys 0.008s
I20260709 13:09:38.787164 22875 raft_consensus.cc:2410] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 13 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:09:38.716887 27019 raft_consensus.cc:2804] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:38.793936 27019 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:38.794320 27019 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:38.796819 27155 raft_consensus.cc:2749] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:09:38.798530 27134 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:38.803664 27019 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:38.808575 23029 raft_consensus.cc:1275] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 13:09:38.784317 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 48212.7us
I20260709 13:09:38.703581 27161 raft_consensus.cc:2804] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:38.811714 27161 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:38.812124 27161 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:38.815970 27115 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:38.786412 27112 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:38.738513 27092 raft_consensus.cc:2804] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:38.793150 27048 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:38.804931 27172 raft_consensus.cc:2764] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 13:09:38.806155 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:38.807559 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:38.649504 27038 consensus_queue.cc:1048] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.159s
I20260709 13:09:38.713333 27130 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:38.818204 27151 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:38.819268 27133 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:38.791263 26509 raft_consensus.cc:2749] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:38.708643 27178 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:38.819723 26808 consensus_queue.cc:1048] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:09:38.709093 27177 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:38.821329 27139 log.cc:927] Time spent T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.689s	user 0.000s	sys 0.008s
I20260709 13:09:38.648882 27055 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:38.707418 27165 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:38.667495 27163 raft_consensus.cc:697] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:38.619244 27145 consensus_queue.cc:1048] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.167s
I20260709 13:09:38.672966 27157 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:38.825918 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:38.825531 22992 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:38.918943 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:38.708864 27150 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:38.846402 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:38.919350 23020 raft_consensus.cc:2393] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 13: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:38.827188 27112 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:38.819468 27161 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:38.898061 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:38.923023 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 97559.9us
I20260709 13:09:38.898687 22877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53724 (request call id 673) took 2562 ms. Trace:
I20260709 13:09:38.898722 22872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2483) took 1798 ms. Trace:
I20260709 13:09:38.900904 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:38.901050 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 68191.7us
I20260709 13:09:38.925698 23588 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:38.926187 27161 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:09:38.926811 27131 log.cc:927] Time spent T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 1.035s	user 0.005s	sys 0.006s
I20260709 13:09:38.928798 27157 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:38.931347 27157 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:38.932267 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:38.902235 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:38.933182 26509 consensus_queue.cc:1048] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:09:38.933580 27020 consensus_meta.cc:287] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 1.409s	user 0.012s	sys 0.000s
I20260709 13:09:38.934053 27020 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:38.824726 27171 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:38.707672 27160 consensus_queue.cc:1059] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.319s
I20260709 13:09:38.939553 27160 raft_consensus.cc:3055] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:09:38.939980 27160 raft_consensus.cc:740] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:38.940151 23616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 847) took 1507 ms. Trace:
I20260709 13:09:38.827504 27170 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:38.941045 27181 raft_consensus.cc:2749] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:38.941030 27160 consensus_queue.cc:260] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:38.942112 27020 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:38.942415 27160 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:38.942958 23022 raft_consensus.cc:2393] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 7: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
W20260709 13:09:38.943802 26989 consensus_meta.cc:287] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 1.353s	user 0.007s	sys 0.004s
I20260709 13:09:38.738474 26985 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:38.945834 26985 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:38.831254 27092 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:38.909201 23617 raft_consensus.cc:3055] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:09:38.911381 22874 raft_consensus.cc:1275] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:09:38.946419 27092 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:38.822753 26549 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:38.948173 27115 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:38.949002 23025 raft_consensus.cc:2393] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 10: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:38.946487 23617 raft_consensus.cc:740] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:38.916319 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:38.950204 23617 consensus_queue.cc:260] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:38.951171 23617 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:38.951411 26989 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:38.909396 27022 consensus_meta.cc:287] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.692s	user 0.008s	sys 0.002s
I20260709 13:09:38.940631 23616 rpcz_store.cc:276] 0709 13:09:37.433149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:37.433280 (+   131us) service_pool.cc:224] Handling call
0709 13:09:37.440597 (+  7317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:37.484887 (+ 44290us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:37.484917 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:37.510005 (+ 25088us) log.cc:844] Serialized 26 byte log entry
0709 13:09:37.786738 (+276733us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:09:37.786783 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:37.786849 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:38.820569 (+1033720us) spinlock_profiling.cc:243] Waited 2.66 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:09:38.939826 (+119257us) raft_consensus.cc:1685] finished
0709 13:09:38.939852 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:38.940062 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37732,"raft.queue_time_us":39888,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":122,"spinlock_wait_cycles":6112000,"thread_start_us":229033,"threads_started":1,"wal-append.queue_time_us":425357}
I20260709 13:09:38.963660 22992 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:38.905458 27130 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:38.920333 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:38.919566 23624 raft_consensus.cc:2468] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
I20260709 13:09:38.921247 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:38.921478 27176 raft_consensus.cc:2764] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:09:38.936946 23588 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:38.924088 22872 rpcz_store.cc:276] 0709 13:09:37.100655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:37.216691 (+116036us) service_pool.cc:224] Handling call
0709 13:09:37.235226 (+ 18535us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:37.286253 (+ 51027us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:09:37.294328 (+  8075us) log.cc:844] Serialized 14 byte log entry
0709 13:09:37.656720 (+362392us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:37.656790 (+    70us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:09:37.656849 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:38.898560 (+1241711us) spinlock_profiling.cc:243] Waited 50.8 ms on lock 0x7b54002d6254. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:38.898595 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:38.898681 (+    86us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":7925,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":1407,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":14787,"spinlock_wait_cycles":116873088,"thread_start_us":361982,"threads_started":1,"wal-append.queue_time_us":554065}
I20260709 13:09:38.971493 23617 raft_consensus.cc:1275] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 13:09:38.916412 27019 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:38.667171 27148 consensus_queue.cc:237] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:38.976151 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:38.668251 27156 consensus_queue.cc:1048] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.163s
W20260709 13:09:38.906134 26802 consensus_meta.cc:287] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 1.008s	user 0.004s	sys 0.006s
I20260709 13:09:38.976927 22875 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:38.912020 27151 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
W20260709 13:09:38.922299 27091 consensus_meta.cc:287] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 1.221s	user 0.009s	sys 0.002s
I20260709 13:09:38.968753 23619 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
I20260709 13:09:38.971920 23617 pending_rounds.cc:85] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2: Aborting all ops after (but not including) 10
I20260709 13:09:38.938325 27153 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:38.972802 22878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2489) took 1419 ms. Trace:
W20260709 13:09:38.966313 26764 consensus_meta.cc:287] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 1.215s	user 0.012s	sys 0.000s
W20260709 13:09:38.962944 27146 consensus_meta.cc:287] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.755s	user 0.011s	sys 0.000s
I20260709 13:09:38.948881 26549 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:38.966711 27181 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:38.967262 26762 consensus_meta.cc:287] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.835s	user 0.011s	sys 0.000s
I20260709 13:09:39.016041 27163 consensus_queue.cc:237] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:39.016068 26762 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:09:38.918676 27172 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:38.924042 22877 rpcz_store.cc:276] 0709 13:09:36.336074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:37.130566 (+794492us) service_pool.cc:224] Handling call
0709 13:09:37.220818 (+ 90252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:37.489080 (+268262us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:09:37.533666 (+ 44586us) log.cc:844] Serialized 14 byte log entry
0709 13:09:38.133855 (+600189us) log.cc:844] Serialized 14 byte log entry
0709 13:09:38.203148 (+ 69293us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:38.203230 (+    82us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:09:38.203302 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:38.898454 (+695152us) spinlock_profiling.cc:243] Waited 268 ms on lock 0x7b5400265554. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:38.898502 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:38.898633 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20595,"prepare.queue_time_us":39617,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":76,"raft.queue_time_us":339601,"raft.run_cpu_time_us":437,"raft.run_wall_time_us":144107,"spinlock_wait_cycles":616477440,"thread_start_us":627247,"threads_started":3,"wal-append.queue_time_us":344769}
I20260709 13:09:38.935719 27055 raft_consensus.cc:2749] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:38.942960 27170 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:38.997934 23617 pending_rounds.cc:107] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 13:09:38.935110 27171 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:39.025005 23617 raft_consensus.cc:2889] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:38.992559 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:38.993250 23621 raft_consensus.cc:1240] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:38.996892 22993 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:38.962076 26236 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:38.906769 27168 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:38.999526 22878 rpcz_store.cc:276] 0709 13:09:37.559615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:37.568554 (+  8939us) service_pool.cc:224] Handling call
0709 13:09:37.569010 (+   456us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:37.597132 (+ 28122us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:37.597183 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:37.613983 (+ 16800us) log.cc:844] Serialized 52 byte log entry
0709 13:09:37.802797 (+188814us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:09:37.853185 (+ 50388us) log.cc:844] Serialized 14 byte log entry
0709 13:09:37.989328 (+136143us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:37.989445 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:38.972562 (+983117us) raft_consensus.cc:1685] finished
0709 13:09:38.972583 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:38.972748 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22600,"prepare.queue_time_us":10816,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":36,"raft.queue_time_us":208664,"raft.run_cpu_time_us":254,"raft.run_wall_time_us":16500,"thread_start_us":243080,"threads_started":2,"wal-append.queue_time_us":283387}
I20260709 13:09:39.005501 27161 raft_consensus.cc:2804] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:39.004222 27091 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:39.007253 27048 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:39.012771 26985 raft_consensus.cc:2749] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:39.039588 27171 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:39.039279 27169 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:39.040446 22873 raft_consensus.cc:1240] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:09:38.992581 27110 consensus_queue.cc:1048] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:09:39.040933 27168 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:39.042014 27112 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:39.025401 27172 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:39.011312 27152 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:39.013497 23618 raft_consensus.cc:1240] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:38.907644 27178 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:38.967211 27162 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:38.993739 27157 consensus_queue.cc:1048] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 13:09:39.012920 26975 consensus_queue.cc:1048] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260709 13:09:39.063552 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:39.000135 26802 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:39.064926 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:39.065294 27169 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:39.026933 27019 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:39.066001 27158 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 13 current term: 13 is higher.
I20260709 13:09:39.066136 27019 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:38.913151 27177 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:39.068898 23027 raft_consensus.cc:1240] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:09:39.066139 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:39.070472 23618 raft_consensus.cc:2393] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 11: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:39.070436 27177 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:39.027343 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:39.070931 27091 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:39.028342 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:39.071457 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:38.975255 27130 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:39.033504 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:38.972999 27022 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:39.074569 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:39.075208 23621 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
W20260709 13:09:39.035037 22843 leader_election.cc:336] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Timed out: RequestConsensusVote RPC to 127.21.63.2:33309 timed out after 2.288s (SENT)
I20260709 13:09:39.035825 22993 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:39.075892 22843 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:39.008224 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:38.975492 27020 consensus_queue.cc:1059] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.048s
I20260709 13:09:39.019690 27146 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:39.053555 23026 raft_consensus.cc:2393] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 12: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:39.045509 27057 raft_consensus.cc:2804] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:39.045169 26989 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:39.056618 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:39.057042 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:39.030138 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:39.071866 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:39.072486 23020 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 7.
I20260709 13:09:39.020404 27151 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:38.922739 27165 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:39.036720 27170 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:39.121019 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:39.121742 23617 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11.
I20260709 13:09:39.122035 26794 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:39.122547 26794 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:39.122778 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:39.123872 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:39.022567 27181 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:39.125065 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:39.125872 22954 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
I20260709 13:09:39.128908 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:39.130005 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:39.082180 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:39.082218 27020 raft_consensus.cc:3055] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:09:39.053567 27172 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:39.089705 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 879800us
I20260709 13:09:39.053364 26762 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:39.151559 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:39.090399 23029 raft_consensus.cc:2393] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 11: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:39.034492 27180 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:39.068498 27019 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:39.133414 23624 raft_consensus.cc:1275] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:09:39.168358 23024 raft_consensus.cc:1275] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:09:39.133311 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:39.132750 23617 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
I20260709 13:09:39.141050 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:39.089506 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 903033us
I20260709 13:09:39.078289 27057 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:39.079275 27091 raft_consensus.cc:2749] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:09:39.152267 22949 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
I20260709 13:09:39.154842 27022 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:39.155437 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:39.124533 26794 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:39.142681 27020 raft_consensus.cc:740] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:39.181713 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:39.157176 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:39.182899 22952 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 6.
I20260709 13:09:39.157979 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:39.183681 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:39.183957 22947 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:39.158327 23621 raft_consensus.cc:1240] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:39.077607 27162 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:39.187674 27151 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:39.187436 27020 consensus_queue.cc:260] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:39.188871 27020 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:39.189082 27162 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:39.189306 27178 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:39.178597 27180 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:39.133885 27170 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:39.114948 27177 raft_consensus.cc:2804] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:39.160974 22880 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
I20260709 13:09:39.162563 27130 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:39.163895 26802 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:39.089618 27060 consensus_queue.cc:1048] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.079s
I20260709 13:09:39.090252 27119 raft_consensus.cc:2749] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:39.164858 27152 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:39.165392 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:39.174226 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 139209us
I20260709 13:09:39.175078 22946 raft_consensus.cc:2410] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
W20260709 13:09:39.175571 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 242615us
I20260709 13:09:39.091609 27169 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:39.184756 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:39.185568 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:39.185767 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:39.158890 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:39.191805 23622 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
I20260709 13:09:39.192692 22950 raft_consensus.cc:1275] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:09:39.193396 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:39.166940 27165 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:39.197010 22992 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:09:39.226907 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:39.126777 27181 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:39.304836 23625 raft_consensus.cc:1240] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:39.320986 26626 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:39.378969 27180 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:39.178557 27057 raft_consensus.cc:3060] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:39.383069 22872 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 11.
W20260709 13:09:39.380453 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 183557us
W20260709 13:09:39.357648 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 173396us
I20260709 13:09:39.379765 22919 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:39.305774 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:39.382570 22951 raft_consensus.cc:2468] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
I20260709 13:09:39.381765 22955 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
I20260709 13:09:39.390771 22919 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:09:39.378053 27190 log.cc:927] Time spent T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.270s	user 0.001s	sys 0.000s
I20260709 13:09:39.391974 22919 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:39.394461 26626 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:39.405045 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:39.405489 26236 raft_consensus.cc:2804] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:09:39.405853 22950 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:09:39.405233 27120 raft_consensus.cc:2804] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:09:39.390136 22992 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:39.406155 26861 consensus_queue.cc:1059] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:09:39.393532 22920 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:39.364140 27170 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:39.384198 27022 raft_consensus.cc:2804] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:39.418586 22872 raft_consensus.cc:1275] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:09:39.421480 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:39.419135 22872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2498) took 1471 ms. Trace:
I20260709 13:09:39.433202 22872 rpcz_store.cc:276] 0709 13:09:37.947930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:39.389809 (+1441879us) service_pool.cc:224] Handling call
0709 13:09:39.415468 (+ 25659us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:39.418954 (+  3486us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:39.419094 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:39.442711 22947 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 13.
I20260709 13:09:39.429065 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:39.447522 22950 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11.
I20260709 13:09:39.448702 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:39.450237 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:39.450446 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 48079.8us
I20260709 13:09:39.451207 22950 raft_consensus.cc:2468] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
I20260709 13:09:39.453137 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:39.453972 22947 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
I20260709 13:09:39.364784 27092 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:39.448546 22992 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:09:39.461642 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 70492.3us
I20260709 13:09:39.388556 27169 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:39.463083 22992 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:39.429673 27022 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:39.429255 27180 raft_consensus.cc:2749] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:39.447099 27146 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:39.450204 26236 raft_consensus.cc:697] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:39.464071 26943 raft_consensus.cc:2749] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:09:39.460522 27152 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:39.470674 27171 raft_consensus.cc:2804] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:39.461793 27134 raft_consensus.cc:2804] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:39.446930 27130 consensus_queue.cc:1048] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 13:09:39.457489 27120 raft_consensus.cc:697] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:39.472551 27022 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:39.464013 26794 consensus_queue.cc:1059] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.019s
I20260709 13:09:39.476910 27165 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:39.498979 22875 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 13.
I20260709 13:09:39.488883 27157 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:39.503269 26236 consensus_queue.cc:237] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:39.503466 27157 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:39.506081 27157 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:39.510241 27092 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:39.511468 26794 raft_consensus.cc:3055] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:09:39.511938 26794 raft_consensus.cc:740] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
W20260709 13:09:39.513155 27138 log.cc:927] Time spent T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 1.585s	user 0.000s	sys 0.011s
I20260709 13:09:39.512873 26794 consensus_queue.cc:260] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:39.514451 23620 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60044 (request call id 784) took 2123 ms. Trace:
I20260709 13:09:39.514554 26794 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:39.515544 26626 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:39.517385 22879 raft_consensus.cc:1275] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:09:39.518191 22879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2501) took 1030 ms. Trace:
I20260709 13:09:39.487797 27171 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:39.520546 22879 rpcz_store.cc:276] 0709 13:09:38.487423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:39.491648 (+1004225us) service_pool.cc:224] Handling call
0709 13:09:39.517058 (+ 25410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:39.517884 (+   826us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:39.518103 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:39.514849 23620 rpcz_store.cc:276] 0709 13:09:37.390723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:37.390876 (+   153us) service_pool.cc:224] Handling call
0709 13:09:37.396985 (+  6109us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:37.399451 (+  2466us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:09:37.399493 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:37.432474 (+ 32981us) log.cc:844] Serialized 26 byte log entry
0709 13:09:37.752194 (+319720us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:09:37.752252 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:37.752336 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:39.412027 (+1659691us) spinlock_profiling.cc:243] Waited 7.88 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:09:39.514127 (+102100us) raft_consensus.cc:1685] finished
0709 13:09:39.514150 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:39.514373 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1950,"raft.queue_time_us":7606,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":130,"spinlock_wait_cycles":18131200,"thread_start_us":284840,"threads_started":1,"wal-append.queue_time_us":427620}
I20260709 13:09:39.491421 27134 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:39.528970 27134 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:09:39.533583 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 65067.2us
I20260709 13:09:39.534381 22872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2499) took 1047 ms. Trace:
I20260709 13:09:39.547230 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:39.547703 22918 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:39.489837 27152 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:39.552091 22872 rpcz_store.cc:276] 0709 13:09:38.486831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:39.471919 (+985088us) service_pool.cc:224] Handling call
0709 13:09:39.483233 (+ 11314us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:39.487623 (+  4390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:39.487782 (+   159us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:39.498300 (+ 10518us) log.cc:844] Serialized 52 byte log entry
0709 13:09:39.526005 (+ 27705us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:09:39.526049 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:39.526134 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:39.527548 (+  1414us) raft_consensus.cc:1685] finished
0709 13:09:39.527569 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:39.527727 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3594,"raft.queue_time_us":5390,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":7970,"thread_start_us":19067,"threads_started":1,"wal-append.queue_time_us":19305}
I20260709 13:09:39.534052 27171 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:39.596279 22872 raft_consensus.cc:1275] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 13:09:39.534623 27091 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:39.592434 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:39.598989 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:39.599999 27120 consensus_queue.cc:237] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:39.592830 27119 raft_consensus.cc:2764] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader election decision vote started in defunct term 12: won
I20260709 13:09:39.600701 27091 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:39.616160 27191 log.cc:927] Time spent T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
I20260709 13:09:39.618083 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:39.618736 22949 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:39.620086 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:39.630816 23588 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:39.648258 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:39.649221 22954 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 14.
W20260709 13:09:39.651528 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 103899us
I20260709 13:09:39.656387 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:39.659143 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:39.660334 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:39.663695 22949 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11.
I20260709 13:09:39.666708 23623 raft_consensus.cc:2393] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 11: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:39.644970 27145 raft_consensus.cc:2749] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:09:39.669927 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:39.671188 23625 raft_consensus.cc:2393] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 13: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:39.673586 27136 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:39.673760 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:39.653962 27092 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:39.674813 27092 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:39.675384 22873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 2000) took 1956 ms. Trace:
I20260709 13:09:39.675773 22873 rpcz_store.cc:276] 0709 13:09:37.721761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:38.374702 (+652941us) service_pool.cc:224] Handling call
0709 13:09:38.404813 (+ 30111us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:39.040857 (+636044us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:39.675109 (+634252us) spinlock_profiling.cc:243] Waited 635 ms on lock 0x7b54001bbf54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:39.675311 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1461135744}
I20260709 13:09:39.676910 27092 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:39.678892 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:39.679448 22950 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6.
I20260709 13:09:39.679818 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:39.667960 27169 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:39.683295 22844 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:39.681125 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:39.618104 27022 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:39.685259 22873 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 7.
I20260709 13:09:39.686777 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:39.688155 22876 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 13.
I20260709 13:09:39.669021 27149 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:39.652418 27179 consensus_queue.cc:1059] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:09:39.669255 27160 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:39.693859 23588 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:39.696267 22918 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:39.678059 27057 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:39.706558 22844 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:39.694478 27092 raft_consensus.cc:2804] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:39.724785 22844 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:39.726472 27058 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:39.716302 26861 consensus_queue.cc:1048] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 13:09:39.721025 27151 raft_consensus.cc:2749] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:09:39.716090 27170 raft_consensus.cc:2804] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:39.730085 27170 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:39.730439 27170 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:39.733136 27136 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:39.733685 27136 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:39.738891 22844 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:39.686205 27157 consensus_queue.cc:1059] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:09:39.745911 23028 raft_consensus.cc:2368] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 9. Current term is 10.
I20260709 13:09:39.745935 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:39.749156 22878 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:39.699831 27149 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:39.772637 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:39.722798 27092 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:39.801769 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:39.744880 26861 raft_consensus.cc:2764] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:09:39.745653 27169 raft_consensus.cc:2749] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:39.703872 27160 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:39.814373 22878 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:09:39.830816 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:39.831601 22877 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:39.812678 27092 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:39.753194 27171 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:39.839071 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 240982us
I20260709 13:09:39.839375 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:39.843632 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:39.823891 27136 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:39.847994 22874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 2003) took 2075 ms. Trace:
I20260709 13:09:39.848392 22874 rpcz_store.cc:276] 0709 13:09:37.772515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:38.644052 (+871537us) service_pool.cc:224] Handling call
0709 13:09:38.911106 (+267054us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:38.946756 (+ 35650us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:39.842649 (+895893us) spinlock_profiling.cc:243] Waited 267 ms on lock 0x7b5400369550. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:39.847937 (+  5288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":613527808}
I20260709 13:09:39.859748 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:39.861030 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:39.861681 22874 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11.
W20260709 13:09:39.937289 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:39.877436 22876 raft_consensus.cc:2393] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 11: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:39.942428 22877 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
W20260709 13:09:39.947463 27206 log.cc:927] Time spent T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 13:09:39.949930 27134 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:39.957924 22879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53724 (request call id 679) took 1002 ms. Trace:
I20260709 13:09:39.958267 22879 rpcz_store.cc:276] 0709 13:09:38.955070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:39.606401 (+651331us) service_pool.cc:224] Handling call
0709 13:09:39.606734 (+   333us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:39.607183 (+   449us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:39.607208 (+    25us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:39.607767 (+   559us) log.cc:844] Serialized 52 byte log entry
0709 13:09:39.665026 (+ 57259us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:09:39.665064 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:39.665186 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:39.957713 (+292527us) raft_consensus.cc:1685] finished
0709 13:09:39.957738 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:39.957868 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1566,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":89,"thread_start_us":56702,"threads_started":1,"wal-append.queue_time_us":185205}
I20260709 13:09:39.959443 27134 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:39.959946 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:39.960907 22879 raft_consensus.cc:2410] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:09:39.954622 27160 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:39.978837 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:39.979712 22874 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:09:40.022562 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 283719us
I20260709 13:09:40.022866 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:40.004434 27207 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:40.053507 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:09:40.066179 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 217161us
I20260709 13:09:40.067185 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:40.068445 23616 raft_consensus.cc:2410] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 14 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:09:40.027761 27160 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:40.077278 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:40.078449 23624 raft_consensus.cc:2410] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 14 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:09:40.081375 27171 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:40.083341 27193 log.cc:927] Time spent T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.530s	user 0.003s	sys 0.009s
I20260709 13:09:40.033522 27057 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:40.084726 23023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 1926) took 1253 ms. Trace:
I20260709 13:09:40.085760 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:40.085114 23023 rpcz_store.cc:276] 0709 13:09:38.831578 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:38.831738 (+   160us) service_pool.cc:224] Handling call
0709 13:09:38.900249 (+ 68511us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:38.912652 (+ 12403us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:38.912706 (+    54us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:38.920216 (+  7510us) log.cc:844] Serialized 52 byte log entry
0709 13:09:39.071614 (+151398us) spinlock_profiling.cc:243] Waited 12.1 ms on lock 0x7b1800442de0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cdae07 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf7c02 00007f613597f20b 00007f6135984d23 00007f61359850d1 00007f6135da0a0f
0709 13:09:39.314092 (+242478us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:09:39.314147 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:39.314249 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:40.082697 (+768448us) spinlock_profiling.cc:243] Waited 4.91 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:09:40.084384 (+  1687us) raft_consensus.cc:1685] finished
0709 13:09:40.084406 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:40.084543 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28725,"raft.queue_time_us":9175,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":20402,"spinlock_wait_cycles":39227136,"thread_start_us":242211,"threads_started":1,"wal-append.queue_time_us":547458}
I20260709 13:09:40.087721 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:40.066902 27157 raft_consensus.cc:3055] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:09:40.101941 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:40.079070 27160 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:40.104422 27151 consensus_queue.cc:1059] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:09:40.108103 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:40.108974 22950 raft_consensus.cc:2410] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 13 because replica has last-logged OpId of term: 11 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:09:40.112589 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:40.112612 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:40.113544 22952 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
I20260709 13:09:40.114153 23023 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 9.
I20260709 13:09:40.061705 27207 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:40.118731 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:40.120934 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:40.121682 22871 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11.
I20260709 13:09:40.122915 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:40.095324 27157 raft_consensus.cc:740] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:40.132180 27221 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:40.132377 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:40.132768 27221 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:40.134164 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:40.134930 27221 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:40.135066 23023 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 7.
I20260709 13:09:40.137504 27219 consensus_queue.cc:1059] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.010s
I20260709 13:09:40.138411 27219 consensus_queue.cc:1243] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer aed3f089ec8b49af9c8cf5cdee76e8e7 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:09:40.138953 27219 raft_consensus.cc:3055] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:09:40.139281 27219 raft_consensus.cc:740] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:40.141314 23024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 681) took 1108 ms. Trace:
I20260709 13:09:40.141712 23024 rpcz_store.cc:276] 0709 13:09:39.032758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:39.032872 (+   114us) service_pool.cc:224] Handling call
0709 13:09:39.039297 (+  6425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:39.168931 (+129634us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:40.141058 (+972127us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b54001c84d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:40.141239 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":295811968}
I20260709 13:09:40.142923 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:40.143543 23024 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 11.
I20260709 13:09:40.109876 27160 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:40.147691 27091 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:40.108490 27218 consensus_queue.cc:1059] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 13:09:40.136549 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:40.152055 22872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:43122 (request call id 0) took 1178 ms. Trace:
I20260709 13:09:40.152576 23023 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:40.155879 27177 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:40.156355 27177 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:40.160889 27170 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:40.106796 26755 consensus_queue.cc:1059] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:09:40.162595 26755 raft_consensus.cc:3055] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:09:40.162961 26755 raft_consensus.cc:740] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:40.163851 26755 consensus_queue.cc:260] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:40.165210 23026 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33418 (request call id 1921) took 1358 ms (client timeout 1747 ms). Trace:
W20260709 13:09:40.165798 23026 rpcz_store.cc:269] 0709 13:09:38.807183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:38.807310 (+   127us) service_pool.cc:224] Handling call
0709 13:09:40.164977 (+1357667us) spinlock_profiling.cc:243] Waited 214 ms on lock 0x7b540010c054. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:40.165152 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":491143936}
I20260709 13:09:40.166935 23020 raft_consensus.cc:1240] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:09:40.167260 23023 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 11.
I20260709 13:09:40.164587 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:40.169991 22949 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
I20260709 13:09:40.171269 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:40.129376 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:40.173048 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:40.173673 22871 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:40.144435 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:40.177353 23024 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:40.152534 22872 rpcz_store.cc:276] 0709 13:09:38.974162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:39.614601 (+640439us) service_pool.cc:224] Handling call
0709 13:09:40.151735 (+537134us) spinlock_profiling.cc:243] Waited 50.3 ms on lock 0x7b540040b354. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135deb735 00007f6135de39fa 00007f613adab4c9 00007f6134327c54 00007f6134327be2 00007f6134327b11 00007f6134327a8c 00007f6134326d42 00007f613412048c 00007f613411f8c6
0709 13:09:40.151963 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":115582720}
W20260709 13:09:40.180536 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 220477us
I20260709 13:09:40.181280 23025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33418 (request call id 1918) took 2619 ms. Trace:
I20260709 13:09:40.182826 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:40.183817 23023 raft_consensus.cc:2410] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 13:09:40.184623 22876 raft_consensus.cc:2393] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 7: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:40.185678 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:40.186592 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:40.187332 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:40.187799 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:40.188027 23026 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:40.185803 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:40.189436 26755 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:40.181936 23025 rpcz_store.cc:276] 0709 13:09:37.549196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:37.549339 (+   143us) service_pool.cc:224] Handling call
0709 13:09:40.167862 (+2618523us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b54000bc7d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:40.168033 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3174713088}
I20260709 13:09:40.111719 27149 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:40.193686 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:40.195966 22920 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:40.198271 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:40.199425 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:40.130991 27151 consensus_queue.cc:1048] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 13:09:40.200263 27161 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:40.200712 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:40.200918 27161 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:40.201427 23025 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 14.
I20260709 13:09:40.202317 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:40.203207 23025 raft_consensus.cc:2393] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 13: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:40.199244 26794 consensus_queue.cc:1048] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:09:40.204803 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 136654us
I20260709 13:09:40.205515 22843 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:40.180909 27170 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:40.204279 22952 raft_consensus.cc:2393] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 11: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:40.209066 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:40.209355 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:40.207435 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:40.210112 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:40.210363 23023 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
I20260709 13:09:40.210696 22954 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 10.
I20260709 13:09:40.213366 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:40.212371 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:40.209039 22843 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:40.218694 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:40.219198 23025 raft_consensus.cc:2393] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 13: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:40.221269 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:40.221551 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:40.219633 22843 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:40.140115 27219 consensus_queue.cc:260] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:40.223793 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:40.219660 22952 raft_consensus.cc:2393] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 13: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:40.230427 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:40.224515 22946 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
I20260709 13:09:40.231127 23023 raft_consensus.cc:2393] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 13: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:40.231848 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:40.232189 22871 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 12.
I20260709 13:09:40.232757 22946 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:09:40.208357 26236 raft_consensus.cc:2749] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:40.233850 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:40.241331 22950 raft_consensus.cc:2368] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 11. Current term is 12.
I20260709 13:09:40.232004 22843 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:40.235077 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:40.235664 22876 raft_consensus.cc:1275] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:09:40.245028 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:40.245702 23025 raft_consensus.cc:2393] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 13: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:40.243764 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:40.250368 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 44750.3us
I20260709 13:09:40.238265 23024 raft_consensus.cc:2410] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:09:40.222967 27221 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:40.221148 27220 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:40.255595 22876 raft_consensus.cc:1275] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:09:40.256245 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:40.261752 23023 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 10.
I20260709 13:09:40.206193 27223 consensus_queue.cc:1059] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 13:09:40.264379 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:40.265111 23025 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:40.240032 22871 raft_consensus.cc:1240] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:09:40.275259 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:40.276274 23024 raft_consensus.cc:2393] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 12: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
W20260709 13:09:40.278323 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:40.278230 23026 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
I20260709 13:09:40.281068 22876 raft_consensus.cc:1275] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:09:40.280712 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:40.281934 23024 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:40.283553 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:40.286684 22872 raft_consensus.cc:1240] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 11. Current term is 12. Ops: [8.6-11.7]
I20260709 13:09:40.287278 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:40.258327 27221 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:40.291606 27221 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:40.294664 27160 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:40.289628 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:40.298996 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:40.290721 27225 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:40.300267 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:40.289189 26794 raft_consensus.cc:3055] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:09:40.300906 23023 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
I20260709 13:09:40.301829 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:40.302753 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:40.300421 27225 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:40.303778 27225 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:40.300961 26794 raft_consensus.cc:740] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:40.305861 26794 consensus_queue.cc:260] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:40.305522 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:40.308007 23026 raft_consensus.cc:2393] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 11: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:40.310416 27149 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:40.310947 27149 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:40.313252 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:40.316614 23026 raft_consensus.cc:2393] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 12: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:39.845486 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:40.313436 27207 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:40.319187 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:40.328008 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:40.329036 23023 raft_consensus.cc:2393] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 11: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:40.330001 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:40.321763 22874 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11.
I20260709 13:09:40.332355 22874 raft_consensus.cc:1240] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 10. Current term is 11. Ops: []
W20260709 13:09:40.324092 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 92165us
I20260709 13:09:40.334460 22843 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:40.335855 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:40.341214 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:40.336318 22994 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:40.344796 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:40.345947 22843 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:40.346086 22994 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:40.349138 22874 raft_consensus.cc:1240] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:09:40.324990 26794 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:40.348553 23024 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
I20260709 13:09:40.351755 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:40.353505 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:40.354261 23026 raft_consensus.cc:2410] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
W20260709 13:09:40.354200 27092 consensus_meta.cc:287] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.513s	user 0.010s	sys 0.000s
I20260709 13:09:40.354807 27092 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:40.355547 23026 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 687) took 1318 ms. Trace:
I20260709 13:09:40.355295 27161 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:40.355883 23026 rpcz_store.cc:276] 0709 13:09:39.037542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:40.354858 (+1317316us) service_pool.cc:224] Handling call
0709 13:09:40.355124 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:40.355272 (+   148us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:40.355296 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:40.355343 (+    47us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:09:40.355373 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:40.355418 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:40.355502 (+    84us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:40.356868 27092 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:40.357827 27234 raft_consensus.cc:2804] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:40.357657 22954 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
I20260709 13:09:40.358500 27234 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:40.359030 27234 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:40.359042 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:40.350631 27058 raft_consensus.cc:2749] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:09:40.360627 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:40.350670 27057 raft_consensus.cc:2749] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:40.361377 22954 raft_consensus.cc:2393] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 8: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:40.362310 27022 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:40.362985 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:40.365296 23622 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60044 (request call id 789) took 1394 ms (client timeout 1508 ms). Trace:
W20260709 13:09:40.365882 23622 rpcz_store.cc:269] 0709 13:09:38.970895 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:38.971032 (+   137us) service_pool.cc:224] Handling call
0709 13:09:40.365068 (+1394036us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b54002bb6d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:40.365196 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":381574400}
I20260709 13:09:40.366608 22954 raft_consensus.cc:1240] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:40.368155 22955 raft_consensus.cc:1240] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:40.368566 22953 raft_consensus.cc:1275] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 4. (term mismatch)
I20260709 13:09:40.368932 22953 pending_rounds.cc:85] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting all ops after (but not including) 3
I20260709 13:09:40.364511 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:40.370574 22951 raft_consensus.cc:2393] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 13: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:40.369251 22953 pending_rounds.cc:107] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 13:09:40.371271 22953 raft_consensus.cc:2889] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:40.369240 22994 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:40.374708 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:40.381960 27092 raft_consensus.cc:2749] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:40.384096 27221 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:40.385679 27032 raft_consensus.cc:3055] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:09:40.386104 27032 raft_consensus.cc:740] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:40.386801 27221 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:40.386987 27032 consensus_queue.cc:260] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:40.388425 27032 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:09:40.391559 27195 log.cc:927] Time spent T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.797s	user 0.001s	sys 0.001s
I20260709 13:09:40.392901 23027 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 686) took 1358 ms. Trace:
I20260709 13:09:40.393288 23027 rpcz_store.cc:276] 0709 13:09:39.034060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:39.165121 (+131061us) service_pool.cc:224] Handling call
0709 13:09:39.165517 (+   396us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:39.165953 (+   436us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:39.165983 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:39.166668 (+   685us) log.cc:844] Serialized 52 byte log entry
0709 13:09:39.475299 (+308631us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:09:39.475455 (+   156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:39.475536 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:40.391035 (+915499us) spinlock_profiling.cc:243] Waited 23.8 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:09:40.392556 (+  1521us) raft_consensus.cc:1685] finished
0709 13:09:40.392584 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:40.392841 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1563,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":5367,"spinlock_wait_cycles":54789376,"thread_start_us":308245,"threads_started":1,"wal-append.queue_time_us":334172}
I20260709 13:09:40.396133 27225 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:40.399072 27225 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:40.400146 27157 consensus_queue.cc:260] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:40.350018 27179 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:09:40.413103 23022 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33418 (request call id 1917) took 2874 ms (client timeout 2143 ms). Trace:
I20260709 13:09:40.415601 23021 raft_consensus.cc:1240] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 6. Current term is 7. Ops: []
W20260709 13:09:40.413650 23022 rpcz_store.cc:269] 0709 13:09:37.538488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:37.538823 (+   335us) service_pool.cc:224] Handling call
0709 13:09:40.412564 (+2873741us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b5400135454. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:40.413031 (+   467us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3192966528}
I20260709 13:09:40.417359 23022 raft_consensus.cc:1240] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:09:40.418432 23026 raft_consensus.cc:1240] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:09:40.406649 27157 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:40.407991 27057 raft_consensus.cc:2804] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:09:40.429935 23029 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33418 (request call id 1923) took 1608 ms (client timeout 1557 ms). Trace:
I20260709 13:09:40.406988 27136 consensus_queue.cc:1048] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 13:09:40.433794 23028 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32794 (request call id 680) took 1401 ms. Trace:
I20260709 13:09:40.467558 23028 rpcz_store.cc:276] 0709 13:09:39.032440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:39.032591 (+   151us) service_pool.cc:224] Handling call
0709 13:09:40.433608 (+1401017us) spinlock_profiling.cc:243] Waited 652 ms on lock 0x7b5400135954. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:40.433739 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1498505088}
I20260709 13:09:40.435473 23020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 685) took 1401 ms. Trace:
I20260709 13:09:40.440261 27144 raft_consensus.cc:3055] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:09:40.441799 27229 log.cc:927] Time spent T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:09:40.441890 27149 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:40.443035 22946 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 8.
I20260709 13:09:40.443662 23025 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
W20260709 13:09:40.482427 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 108260us
I20260709 13:09:40.416548 27179 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:40.430815 27161 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:40.414573 27092 raft_consensus.cc:3055] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:09:40.466470 23029 rpcz_store.cc:269] 0709 13:09:38.821887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:38.827735 (+  5848us) service_pool.cc:224] Handling call
0709 13:09:40.429663 (+1601928us) spinlock_profiling.cc:243] Waited 166 ms on lock 0x7b54002c9054. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:40.429843 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":381652352}
I20260709 13:09:40.469023 23020 rpcz_store.cc:276] 0709 13:09:39.033891 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:39.115006 (+ 81115us) service_pool.cc:224] Handling call
0709 13:09:40.166760 (+1051754us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:40.167313 (+   553us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:40.435290 (+267977us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b540010c050. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:40.435421 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2417983744}
I20260709 13:09:40.488019 23020 raft_consensus.cc:1240] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:09:40.489621 23020 raft_consensus.cc:1240] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:40.477618 23028 raft_consensus.cc:1240] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:40.378533 27022 consensus_queue.cc:1048] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 13:09:40.483012 23024 raft_consensus.cc:1275] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:09:40.483059 23025 raft_consensus.cc:1275] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:09:40.484365 22880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2509) took 1673 ms. Trace:
I20260709 13:09:40.475178 27144 raft_consensus.cc:740] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:40.493173 22880 rpcz_store.cc:276] 0709 13:09:38.811395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:39.596614 (+785219us) service_pool.cc:224] Handling call
0709 13:09:39.621186 (+ 24572us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:39.621387 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:09:39.702039 (+ 80652us) log.cc:844] Serialized 14 byte log entry
0709 13:09:40.475611 (+773572us) spinlock_profiling.cc:243] Waited 2.38 ms on lock 0x7b1800243100. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f613598d4bc 00007f6132cdb48b 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf7c02 00007f613597f20b 00007f6135984d23 00007f613598523a
0709 13:09:40.484048 (+  8437us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:40.484101 (+    53us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:40.484145 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:40.484210 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:40.484314 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8180,"prepare.queue_time_us":198367,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":1781,"raft.queue_time_us":8505,"spinlock_wait_cycles":5466880,"thread_start_us":208667,"threads_started":2,"wal-append.queue_time_us":317342}
W20260709 13:09:40.495637 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 149780us
I20260709 13:09:40.498221 22843 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:40.500509 23029 raft_consensus.cc:1275] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:09:40.502754 27225 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:09:40.505434 22843 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:40.506247 27225 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:40.515553 27225 raft_consensus.cc:2749] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:40.524642 23023 raft_consensus.cc:2393] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 8: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:40.526041 23028 raft_consensus.cc:1240] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:09:40.483327 22994 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:09:40.533438 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 50184.1us
W20260709 13:09:40.535000 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:40.537611 22994 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:40.556141 23026 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:32782 (request call id 13) took 1499 ms. Trace:
I20260709 13:09:40.556883 23026 rpcz_store.cc:276] 0709 13:09:39.062563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:40.554685 (+1492122us) service_pool.cc:224] Handling call
0709 13:09:40.556024 (+  1339us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:40.486646 27092 raft_consensus.cc:740] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:40.494447 27170 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:40.567775 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:40.568400 23622 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:40.562548 23028 raft_consensus.cc:1240] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:09:40.570178 27170 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:40.566799 27241 raft_consensus.cc:2804] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:40.574031 27241 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:40.574530 27241 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:40.575214 27170 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:40.577276 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:40.578081 23624 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
I20260709 13:09:40.578346 27092 consensus_queue.cc:260] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:40.579706 27092 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:40.585630 27144 consensus_queue.cc:260] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:40.587177 27144 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:40.591368 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:40.594492 23625 raft_consensus.cc:1275] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:09:40.600881 27245 consensus_queue.cc:1048] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:09:40.602876 27245 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:09:40.603371 27245 raft_consensus.cc:697] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 10 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:40.604228 27245 consensus_queue.cc:237] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:40.618499 27241 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:40.606191 27179 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:40.611488 22876 raft_consensus.cc:1275] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:09:40.623569 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:40.632114 23619 raft_consensus.cc:2393] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 11: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:40.633682 23622 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 12.
I20260709 13:09:40.642261 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:40.643878 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:40.645649 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:40.646450 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:40.648363 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:40.649016 22946 raft_consensus.cc:3060] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:40.650069 23588 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:40.651417 23588 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:40.653076 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:40.653352 23588 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:40.654095 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:40.654861 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:40.659759 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:40.662478 23588 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:40.663058 27057 raft_consensus.cc:697] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:40.631765 27022 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:40.672155 22946 raft_consensus.cc:2468] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:09:40.683379 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:40.675856 22992 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:40.684311 22948 raft_consensus.cc:2410] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:09:40.685376 22992 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:40.647351 27217 raft_consensus.cc:2749] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:40.702948 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:40.716913 23025 raft_consensus.cc:2368] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 9. Current term is 10.
I20260709 13:09:40.660946 27170 raft_consensus.cc:2804] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:40.718659 22992 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:40.712888 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:40.703320 23624 raft_consensus.cc:1275] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:09:40.656901 27179 raft_consensus.cc:2749] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:40.665844 27165 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:40.720409 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:09:40.728574 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:40.683215 27230 raft_consensus.cc:2804] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:09:40.741241 22826 auto_rebalancer.cc:629] Failed to schedule move for tablet f11454b577ab42d7bfe7c09154b00817: Illegal state: Replica f3c799b1249e40119d2a5864231c6c75 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 8 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:09:40.741060 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:40.742967 23590 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:40.721058 27170 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
W20260709 13:09:40.744261 27205 log.cc:927] Time spent T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.909s	user 0.000s	sys 0.002s
I20260709 13:09:40.744764 23022 raft_consensus.cc:1240] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 7. Current term is 8. Ops: []
I20260709 13:09:40.744706 27242 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:09:40.683609 27234 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:40.713395 27171 raft_consensus.cc:2804] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:09:40.687841 27022 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:40.712524 27157 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:40.767762 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:40.703209 27247 consensus_queue.cc:1048] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.070s
W20260709 13:09:40.752486 26755 consensus_meta.cc:287] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.562s	user 0.011s	sys 0.002s
I20260709 13:09:40.753254 22949 raft_consensus.cc:2410] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:09:40.766621 22920 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:40.749155 27165 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:40.768414 23624 raft_consensus.cc:3060] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:09:40.768857 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:40.798026 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:40.793906 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:40.767463 27170 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:40.804427 23028 raft_consensus.cc:1240] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:09:40.753607 27230 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:40.819510 27165 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:40.804096 27119 raft_consensus.cc:2804] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:09:40.824784 27057 consensus_queue.cc:237] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:40.825459 27230 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:40.801669 27246 raft_consensus.cc:2804] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:09:40.823266 27177 consensus_meta.cc:287] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.627s	user 0.009s	sys 0.002s
I20260709 13:09:40.828555 23624 raft_consensus.cc:2468] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:09:40.802699 27136 raft_consensus.cc:2804] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 13:09:40.832712 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 295162us
I20260709 13:09:40.798658 27171 raft_consensus.cc:697] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:40.837249 22994 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:40.829671 27246 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:40.830765 27119 raft_consensus.cc:697] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:40.832953 27241 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:40.843713 27136 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:40.806798 27157 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:40.830544 27210 consensus_queue.cc:1048] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:09:40.849400 26755 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:40.857666 27207 consensus_queue.cc:1059] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:09:40.863842 27136 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:40.836879 27177 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:40.859506 27246 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:09:40.897796 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:40.902071 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 159197us
I20260709 13:09:40.904866 23590 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:40.869926 27241 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:40.917313 23590 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:40.919574 27247 raft_consensus.cc:2749] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:09:40.920205 27247 raft_consensus.cc:2749] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:09:40.920846 27247 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:40.921365 27247 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:40.923655 27247 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:40.870323 27077 consensus_queue.cc:1048] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 13:09:40.925045 27241 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:40.927228 27241 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:40.928192 23590 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:40.929605 23590 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:40.930878 27251 log.cc:927] Time spent T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:09:40.930166 27247 raft_consensus.cc:2749] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:09:40.932785 27247 raft_consensus.cc:2804] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:40.933189 27247 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:40.933545 27247 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:40.938200 23590 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:40.942655 23026 raft_consensus.cc:2393] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 8: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
W20260709 13:09:40.940567 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 216442us
I20260709 13:09:40.944280 22992 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:40.950495 22992 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:09:40.973604 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:40.993153 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 155279us
I20260709 13:09:40.918073 27250 raft_consensus.cc:2749] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:40.875000 26755 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:40.944140 27239 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:40.998169 27249 raft_consensus.cc:2804] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:09:40.998311 27262 raft_consensus.cc:2804] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:09:40.998544 22826 auto_rebalancer.cc:629] Failed to schedule move for tablet 612dc49c81064f4faa79f65fcf1cfc02: Illegal state: Replica aed3f089ec8b49af9c8cf5cdee76e8e7 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 11 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:09:40.974470 27241 raft_consensus.cc:2804] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:40.999522 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:40.942492 27218 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:40.995702 27022 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:41.042958 22952 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 6 FOLLOWER]: Advancing to term 8
W20260709 13:09:41.047170 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 105078us
W20260709 13:09:41.082501 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 401083us
I20260709 13:09:41.084506 27259 consensus_queue.cc:1048] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 13:09:41.085640 22766 tablet_replica.cc:1411] Time spent flushing the Log queue took a long time: real 0.471s	user 0.007s	sys 0.010s
I20260709 13:09:41.038399 27032 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
W20260709 13:09:41.086068 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 248898us
I20260709 13:09:41.040558 27262 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:40.946909 27171 consensus_queue.cc:237] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:41.043490 27241 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:41.103112 22952 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 8.
I20260709 13:09:41.085841 27249 raft_consensus.cc:697] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:41.090372 27234 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:41.017081 27119 consensus_queue.cc:237] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:41.090921 27144 raft_consensus.cc:2749] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:41.096763 27262 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:41.038728 27239 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:41.046079 27218 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:41.095391 22994 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:41.105049 27241 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:09:41.129796 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:41.095171 27032 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:41.143911 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 49075.1us
I20260709 13:09:41.146874 22994 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:41.119311 27269 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:09:41.155942 22826 auto_rebalancer.cc:629] Failed to schedule move for tablet b41a09964104486c8e50b539ec2b16f6: Illegal state: Replica 0a407504ed1741bf89a276558adf4c0f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 11 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:09:41.158463 27218 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:41.160974 22826 auto_rebalancer.cc:629] Failed to schedule move for tablet ced80ef275e04ef8bf4c071883b8284e: Illegal state: Replica 0a407504ed1741bf89a276558adf4c0f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 11 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:09:41.161083 27271 consensus_queue.cc:1059] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:09:41.162190 27271 consensus_queue.cc:1059] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:09:41.166689 23027 raft_consensus.cc:1240] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:09:41.167240 26755 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:41.167608 23027 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 692) took 1617 ms. Trace:
I20260709 13:09:41.168125 23027 rpcz_store.cc:276] 0709 13:09:39.550317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:40.394792 (+844475us) service_pool.cc:224] Handling call
0709 13:09:40.395130 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:41.166374 (+771244us) spinlock_profiling.cc:243] Waited 80.8 ms on lock 0x7b540003bc20. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135d5cdc0 00007f6135de71c0 00007f6135de8231 00007f6135de4d8f 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c
0709 13:09:41.167256 (+   882us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:41.167540 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":185933184}
I20260709 13:09:41.141101 27157 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:41.175614 22994 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:41.133214 27134 consensus_queue.cc:1048] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 13:09:41.175076 27022 raft_consensus.cc:2804] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:41.196182 27274 raft_consensus.cc:1064] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
I20260709 13:09:41.186255 27277 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:41.196547 27022 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:41.197515 27207 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:41.197335 27277 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:41.198091 27207 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:41.200240 27207 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:41.195205 27258 raft_consensus.cc:3055] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:09:41.201568 27258 raft_consensus.cc:740] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:41.201875 27032 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:41.202778 22843 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.011s	sys 0.019s
W20260709 13:09:41.203123 22843 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.011s	sys 0.019s
I20260709 13:09:41.202469 27258 consensus_queue.cc:260] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:41.203385 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 530768us
I20260709 13:09:41.175927 27177 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:41.203323 27274 raft_consensus.cc:1068] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7: Unable to promote non-voter 5b346767a89745fa82928a5dac9cc0b2: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:09:41.214489 22843 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:41.194861 27265 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:09:41.197365 27022 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:09:41.227886 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 243425us
I20260709 13:09:41.235102 22992 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:41.195422 27250 raft_consensus.cc:2749] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:41.231415 22843 leader_election.cc:400] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 11. Current term is 12.
I20260709 13:09:41.240190 22843 leader_election.cc:403] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
W20260709 13:09:41.233647 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:41.240191 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:41.241699 22955 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
I20260709 13:09:41.242182 22843 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:41.239490 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:41.243824 22952 raft_consensus.cc:2410] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:09:41.243662 22843 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:41.249217 22843 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:41.249249 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 159500us
I20260709 13:09:41.254161 22875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2503) took 2606 ms. Trace:
I20260709 13:09:41.249647 22992 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:41.254788 22875 rpcz_store.cc:276] 0709 13:09:38.647373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:39.500383 (+853010us) service_pool.cc:224] Handling call
0709 13:09:39.509051 (+  8668us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:39.645002 (+135951us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:39.645034 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:39.645073 (+    39us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:09:39.645102 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:41.253849 (+1608747us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b54002d6c54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:41.253915 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:41.254085 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":312151424}
I20260709 13:09:41.271847 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:41.265558 23588 leader_election.cc:400] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 9. Current term is 10.
I20260709 13:09:41.274914 23588 leader_election.cc:403] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:09:41.275209 22954 raft_consensus.cc:2393] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 8: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:41.240486 27157 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:41.172839 27269 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:41.283906 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 91413.7us
I20260709 13:09:41.250944 27218 raft_consensus.cc:2749] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:41.248200 26755 raft_consensus.cc:2749] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 11. Current term is 12.
I20260709 13:09:41.289348 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:41.291227 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:41.258085 27234 raft_consensus.cc:2804] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:09:41.299702 22879 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
I20260709 13:09:41.260409 27277 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:41.301187 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:41.193557 27249 consensus_queue.cc:237] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:41.283622 27253 log.cc:927] Time spent T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.196s	user 0.001s	sys 0.000s
I20260709 13:09:41.308615 22879 raft_consensus.cc:2410] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:09:41.280275 27250 raft_consensus.cc:2804] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:09:41.312584 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:41.318449 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 68888.3us
I20260709 13:09:41.248643 27273 consensus_queue.cc:1048] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.079s
I20260709 13:09:41.328227 23020 raft_consensus.cc:1240] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:41.332391 22873 raft_consensus.cc:1240] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:09:41.327513 26755 raft_consensus.cc:2749] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:09:41.319398 27272 raft_consensus.cc:2804] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:41.339694 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:41.344218 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:41.344856 22874 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:41.347318 23023 raft_consensus.cc:1275] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:09:41.344770 22873 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 13.
I20260709 13:09:41.348556 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:41.349140 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:41.349278 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 100102us
I20260709 13:09:41.351056 22843 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:41.351837 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:41.349802 22873 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
I20260709 13:09:41.371515 22994 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:41.373195 22994 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:41.374366 27234 raft_consensus.cc:697] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 12 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:41.375943 27264 raft_consensus.cc:2804] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:41.375532 27234 consensus_queue.cc:237] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:41.376933 27289 raft_consensus.cc:2749] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:41.377765 27289 consensus_queue.cc:1059] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:09:41.380288 23029 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:09:41.381062 23029 raft_consensus.cc:606] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 10 LEADER]: Received request to transfer leadership to TS aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:41.297011 27282 consensus_queue.cc:1048] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 13:09:41.347361 27238 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:41.385928 27238 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:41.387585 27270 log.cc:927] Time spent T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 13:09:41.388149 27238 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:41.378839 22843 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:41.320731 27283 raft_consensus.cc:2749] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 9. Current term is 10.
I20260709 13:09:41.392490 22843 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:41.400063 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:41.400648 22874 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 13.
I20260709 13:09:41.402256 22843 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:41.376488 27264 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:41.347615 27272 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:41.307535 27230 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:41.384124 27250 raft_consensus.cc:697] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 10 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:09:41.402845 27170 consensus_meta.cc:287] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.577s	user 0.003s	sys 0.007s
I20260709 13:09:41.409205 22843 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:41.391505 27265 raft_consensus.cc:697] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 12 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:09:41.425228 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 50787.4us
I20260709 13:09:41.425994 22994 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:09:41.418884 27136 consensus_meta.cc:287] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.521s	user 0.010s	sys 0.000s
I20260709 13:09:41.412170 27264 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:41.416113 27272 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:41.410068 26755 raft_consensus.cc:2749] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:41.410066 27134 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:41.445428 27134 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:41.410853 27287 raft_consensus.cc:3055] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:09:41.447845 27287 raft_consensus.cc:740] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:41.450680 22955 raft_consensus.cc:1275] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 13:09:41.449141 27287 consensus_queue.cc:260] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:41.452998 27298 raft_consensus.cc:2749] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:09:41.453256 27287 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:09:41.427870 27266 log.cc:927] Time spent T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.221s	user 0.000s	sys 0.001s
I20260709 13:09:41.454916 27262 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:41.427160 27170 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:41.444814 26755 raft_consensus.cc:2764] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 13:09:41.456369 27250 consensus_queue.cc:237] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:41.441426 22843 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:09:41.509857 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 68518.5us
I20260709 13:09:41.441250 27218 raft_consensus.cc:2804] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:41.466610 27160 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:41.479821 27247 consensus_meta.cc:287] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.546s	user 0.003s	sys 0.006s
I20260709 13:09:41.481799 22872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 2007) took 1446 ms. Trace:
I20260709 13:09:41.443388 27136 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:41.488850 27241 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:41.495880 27246 consensus_meta.cc:287] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.591s	user 0.009s	sys 0.000s
I20260709 13:09:41.423695 27239 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:41.424113 27296 raft_consensus.cc:2749] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:09:41.496372 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:41.503815 27032 consensus_queue.cc:1048] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 13:09:41.520448 27170 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:41.506637 27119 consensus_queue.cc:1059] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 13:09:41.524165 27119 raft_consensus.cc:3055] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:09:41.524493 27119 raft_consensus.cc:740] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:41.525287 27119 consensus_queue.cc:260] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:41.507655 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 188980us
I20260709 13:09:41.526391 27119 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:41.526268 27265 consensus_queue.cc:237] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:09:41.527752 23588 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:41.507090 27274 consensus_queue.cc:1059] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:09:41.528728 23588 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:41.528852 27145 consensus_queue.cc:1059] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:09:41.529415 27274 raft_consensus.cc:3055] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:09:41.529748 27274 raft_consensus.cc:740] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:41.529929 27296 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:41.530375 27296 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:41.530535 27274 consensus_queue.cc:260] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:41.531193 26802 raft_consensus.cc:2804] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:09:41.531711 26802 raft_consensus.cc:697] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:41.531919 27158 raft_consensus.cc:2804] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:09:41.506006 27269 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:41.532469 27158 raft_consensus.cc:697] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:41.510596 22843 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:41.532898 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:41.533077 27282 raft_consensus.cc:3055] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:09:41.533463 23620 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:41.533355 27158 consensus_queue.cc:237] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:41.534148 22843 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:41.535652 27292 raft_consensus.cc:2749] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:41.536204 27151 raft_consensus.cc:2749] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:09:41.513180 22872 rpcz_store.cc:276] 0709 13:09:40.035374 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:40.223215 (+187841us) service_pool.cc:224] Handling call
0709 13:09:40.244983 (+ 21768us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:40.290531 (+ 45548us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:41.481530 (+1190999us) spinlock_profiling.cc:243] Waited 40.6 ms on lock 0x7b540040a454. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:41.481738 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":93270400}
I20260709 13:09:41.516229 27219 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:09:41.525761 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:41.538393 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:41.447736 27230 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:41.513283 26755 raft_consensus.cc:2804] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:09:41.542147 23027 raft_consensus.cc:1240] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:09:41.542917 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:41.544472 23616 raft_consensus.cc:1240] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:09:41.510459 22766 maintenance_manager.cc:643] P 8d175b4ae72f4fc48971c2f31668971c: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 8.399s	user 0.188s	sys 0.075s Metrics: {"bytes_written":29693,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":338704,"dirs.run_cpu_time_us":2553,"dirs.run_wall_time_us":50716,"drs_written":1,"lbm_read_time_us":33575,"lbm_reads_1-10_ms":1,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":2,"lbm_write_time_us":423773,"lbm_writes_1-10_ms":1,"lbm_writes_10-100_ms":9,"lbm_writes_gt_100_ms":1,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"reinsert_count":0,"rows_written":63,"thread_start_us":405443,"threads_started":2,"update_count":8,"wal-append.queue_time_us":469962}
I20260709 13:09:41.544874 27292 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:41.545310 27179 consensus_queue.cc:1059] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:09:41.547513 22874 raft_consensus.cc:1240] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:41.548121 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:41.511425 27218 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:41.597518 22873 raft_consensus.cc:2393] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 14: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:41.600643 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:41.602669 22951 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 13.
W20260709 13:09:41.588315 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:41.606235 23588 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:41.531725 27274 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:41.604624 27032 consensus_queue.cc:1059] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:09:41.607507 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:41.607826 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:41.608035 27032 raft_consensus.cc:3055] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:09:41.608492 27032 raft_consensus.cc:740] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:41.609166 22767 maintenance_manager.cc:419] P 8d175b4ae72f4fc48971c2f31668971c: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 10169 bytes on disk
I20260709 13:09:41.608116 23621 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
I20260709 13:09:41.532547 26802 consensus_queue.cc:237] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:41.611507 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:41.612057 22953 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
I20260709 13:09:41.612854 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:41.613360 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:41.550940 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:41.616834 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:41.533478 27282 raft_consensus.cc:740] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:41.614413 22844 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:41.627039 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:41.627421 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:41.629372 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:41.629724 27277 consensus_queue.cc:1059] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:09:41.630386 27136 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:41.630508 27277 raft_consensus.cc:3055] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:09:41.629364 23021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 684) took 2589 ms. Trace:
I20260709 13:09:41.633989 23021 rpcz_store.cc:276] 0709 13:09:39.033621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:39.033724 (+   103us) service_pool.cc:224] Handling call
0709 13:09:40.415438 (+1381714us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:40.415970 (+   532us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:41.623109 (+1207139us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b5400135450. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:41.623252 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3120025472}
I20260709 13:09:41.521817 27022 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:41.532516 27296 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:41.665789 22766 maintenance_manager.cc:643] P 8d175b4ae72f4fc48971c2f31668971c: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.036s	user 0.001s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":340,"lbm_reads_lt_1ms":4}
I20260709 13:09:41.666189 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:41.606411 26755 raft_consensus.cc:697] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 11 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:41.605736 27218 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:41.631719 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:41.537271 27246 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:41.661932 23620 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
I20260709 13:09:41.662542 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:41.521464 27247 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:41.680137 22951 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:09:41.613107 22764 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2959101199a44a4583a4ee456ecf044d with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 0a407504ed1741bf89a276558adf4c0f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 12 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:09:41.664018 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:41.548566 27272 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:41.671334 22946 raft_consensus.cc:2393] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 9: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:41.672206 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:41.631273 27217 raft_consensus.cc:2804] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:09:41.682005 22764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 69103.4us
I20260709 13:09:41.633919 27277 raft_consensus.cc:740] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
W20260709 13:09:41.681367 27242 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.860s	user 0.013s	sys 0.041s
I20260709 13:09:41.706749 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:41.708828 23624 raft_consensus.cc:2393] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 12: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:41.682019 27296 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:41.689563 27157 raft_consensus.cc:2749] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:41.688921 27179 raft_consensus.cc:2764] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 13:09:41.693004 27217 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:41.609383 27032 consensus_queue.cc:260] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:09:41.748404 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:41.749387 22874 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11.
I20260709 13:09:41.750888 22920 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:41.751981 27119 raft_consensus.cc:2804] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:41.752755 27119 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:41.753242 27119 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:41.630864 27282 consensus_queue.cc:260] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:41.735787 26755 consensus_queue.cc:237] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:41.631191 27292 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:41.703514 27242 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.882s	user 0.013s	sys 0.042s
I20260709 13:09:41.757447 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:41.743212 27296 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:41.775105 27246 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:41.776064 27163 raft_consensus.cc:3055] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:09:41.776594 27163 raft_consensus.cc:740] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:41.777701 27163 consensus_queue.cc:260] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:41.761977 27242 raft_consensus.cc:697] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 11 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:41.779551 27242 consensus_queue.cc:237] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:41.784478 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:41.759281 27217 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:41.761746 27309 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:41.816365 22951 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
I20260709 13:09:41.818142 22955 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
I20260709 13:09:41.818595 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:41.822726 27022 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:41.794832 27262 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:41.826968 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:41.829030 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:41.808876 27282 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:41.831585 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:41.832329 22874 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:41.833189 27233 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:41.833864 27233 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:41.836983 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:41.843026 27233 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:41.846699 22874 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 8.
I20260709 13:09:41.849902 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:41.853247 22871 raft_consensus.cc:2468] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 8.
I20260709 13:09:41.855701 22871 raft_consensus.cc:1240] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:09:41.816859 27241 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:41.856961 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 58805.1us
I20260709 13:09:41.860833 22871 raft_consensus.cc:1240] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:41.826177 27032 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:41.868162 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:41.874329 22918 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:41.883003 23590 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:41.884766 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:41.876168 27311 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:41.885802 23623 raft_consensus.cc:1240] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:09:41.886767 26861 consensus_queue.cc:1059] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 13:09:41.946645 27308 log.cc:927] Time spent T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:09:41.946944 26861 raft_consensus.cc:2764] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 13:09:41.947702 26861 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:41.948280 26861 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:41.949319 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:41.949978 22878 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:41.888994 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:41.952184 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
W20260709 13:09:41.899919 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:41.792835 27277 consensus_queue.cc:260] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:41.932374 22843 leader_election.cc:336] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Timed out: RequestConsensusVote RPC to 127.21.63.2:33309 timed out after 2.447s (SENT)
I20260709 13:09:41.938513 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:41.885566 23618 raft_consensus.cc:1275] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:09:41.882184 27269 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:41.944576 23622 raft_consensus.cc:2393] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 14: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:41.889853 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:41.953634 27032 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:41.985167 23620 raft_consensus.cc:1240] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 9. Current term is 10. Ops: []
W20260709 13:09:41.962869 27264 consensus_meta.cc:287] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.502s	user 0.007s	sys 0.001s
I20260709 13:09:41.991649 27247 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:41.963441 22949 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58310 (request call id 1971) took 1937 ms (client timeout 2447 ms). Trace:
W20260709 13:09:41.992379 22949 rpcz_store.cc:269] 0709 13:09:40.026255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:40.299333 (+273078us) service_pool.cc:224] Handling call
0709 13:09:41.963202 (+1663869us) spinlock_profiling.cc:243] Waited 416 ms on lock 0x7b540037b9d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:41.963353 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":957584640}
I20260709 13:09:41.991438 27264 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:41.972512 26861 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:41.970849 27311 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:41.999853 22844 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:42.000423 22878 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 14.
W20260709 13:09:41.969182 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 86390.7us
I20260709 13:09:41.971685 23616 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 13.
I20260709 13:09:41.979148 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:41.821413 27309 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:41.982683 22872 raft_consensus.cc:2393] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 14: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:41.955103 27268 raft_consensus.cc:3055] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:09:41.987486 27032 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:41.971951 27158 raft_consensus.cc:2749] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:42.003717 22879 raft_consensus.cc:2393] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 12: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:42.003822 23590 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:42.010799 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:09:42.017745 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 143487us
I20260709 13:09:41.985018 27269 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:42.014209 27169 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:09:42.029821 22876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 2020) took 1862 ms. Trace:
I20260709 13:09:42.030903 22878 raft_consensus.cc:1240] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:42.021631 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:42.024135 23617 raft_consensus.cc:1240] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:42.032102 22843 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:42.031109 23590 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:42.047616 27169 raft_consensus.cc:697] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 14 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:42.048707 27169 consensus_queue.cc:237] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:42.049325 27311 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:42.039449 27247 raft_consensus.cc:2749] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:42.050034 22954 raft_consensus.cc:1275] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:09:42.025573 27292 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:42.051945 27247 consensus_queue.cc:1048] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:09:42.052392 22843 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:42.055465 22955 raft_consensus.cc:1275] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:09:42.020478 27309 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:42.006795 27268 raft_consensus.cc:740] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
W20260709 13:09:42.058498 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:42.069274 23590 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:42.069494 22843 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:42.073390 22843 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:42.078276 22919 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:42.022310 27296 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:42.040622 22876 rpcz_store.cc:276] 0709 13:09:40.167153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:40.332705 (+165552us) service_pool.cc:224] Handling call
0709 13:09:40.332996 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:40.622584 (+289588us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:42.029614 (+1407030us) spinlock_profiling.cc:243] Waited 278 ms on lock 0x7b5400368654. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:42.029767 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":639424384}
I20260709 13:09:42.029970 27239 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:42.057618 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:42.026293 27134 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:42.053306 27292 raft_consensus.cc:2804] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:42.025117 27152 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:42.026695 27269 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:42.018750 22918 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:42.026124 27032 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:42.071549 27264 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:42.087069 27217 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:42.113338 27217 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:42.070964 27262 raft_consensus.cc:2804] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:42.115710 27262 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:42.081382 27157 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:09:42.117589 26794 raft_consensus.cc:2749] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:09:42.118003 27157 raft_consensus.cc:697] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 11 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:42.118243 27134 consensus_queue.cc:1048] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:09:42.119112 27157 consensus_queue.cc:237] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:42.090886 27218 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:42.121384 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 102712us
I20260709 13:09:42.121635 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:42.088253 27268 consensus_queue.cc:260] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:42.122325 23621 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:42.122645 27032 raft_consensus.cc:2749] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:09:42.123090 27268 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:09:42.123793 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:42.093274 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:42.124475 27275 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.094075 27309 consensus_queue.cc:1059] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:09:42.125011 27275 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:42.125880 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:42.125942 27309 raft_consensus.cc:3055] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:09:42.126456 27309 raft_consensus.cc:740] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:42.127254 27275 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:42.127381 27309 consensus_queue.cc:260] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:42.094187 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:42.071732 27219 raft_consensus.cc:2749] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:09:42.077816 27243 log.cc:927] Time spent T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 1.257s	user 0.003s	sys 0.001s
I20260709 13:09:42.083309 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:42.083247 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:42.129479 22876 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 13.
I20260709 13:09:42.130312 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:42.131100 22878 raft_consensus.cc:2468] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:09:42.132115 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:42.082345 27281 raft_consensus.cc:2804] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:09:42.094326 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:42.122236 22918 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:42.130412 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:42.133333 22874 raft_consensus.cc:1240] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:42.027096 27119 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:42.116339 27262 raft_consensus.cc:3060] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:42.134510 22878 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11.
I20260709 13:09:42.136631 22919 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:42.136974 22918 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:42.139551 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:42.139825 26861 raft_consensus.cc:2749] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:42.140365 22876 raft_consensus.cc:2393] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 13: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:42.141381 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:42.141522 27275 consensus_queue.cc:1059] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:09:42.142041 22876 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 13.
I20260709 13:09:42.142717 27032 raft_consensus.cc:3055] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:09:42.143190 27032 raft_consensus.cc:740] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:42.143491 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:42.144150 22872 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 10.
I20260709 13:09:42.144855 22919 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:42.145341 23616 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 13.
I20260709 13:09:42.145750 26861 consensus_queue.cc:1048] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:09:42.147291 27269 raft_consensus.cc:2804] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:09:42.147399 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 721396us
I20260709 13:09:42.147863 27269 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:42.132584 22920 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:42.107811 27292 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:42.120426 27258 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:42.153218 22918 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:42.148404 27269 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:42.149082 22994 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:42.149856 22920 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:42.150908 23621 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
I20260709 13:09:42.155280 22918 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:42.156443 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:42.164299 22844 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
W20260709 13:09:42.162237 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 85931.2us
I20260709 13:09:42.168423 22918 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:42.168699 22920 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:42.097178 27296 consensus_queue.cc:1048] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 13:09:42.170979 23022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 1986) took 2345 ms. Trace:
I20260709 13:09:42.138659 27272 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:42.171716 22876 raft_consensus.cc:1240] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:42.122395 27218 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:42.171466 27329 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:42.177258 27275 raft_consensus.cc:2749] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:09:42.171518 23022 rpcz_store.cc:276] 0709 13:09:39.825283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:40.418327 (+593044us) service_pool.cc:224] Handling call
0709 13:09:40.744579 (+326252us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:40.758478 (+ 13899us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:42.167379 (+1408901us) spinlock_profiling.cc:243] Waited 326 ms on lock 0x7b54001f8bd0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:42.170914 (+  3535us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":749346048}
I20260709 13:09:42.150384 27273 raft_consensus.cc:2804] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:09:42.172902 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:42.179128 27273 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:42.179673 27273 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:42.180915 27242 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.108418 27239 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:42.181572 27242 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:42.151429 27292 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:42.150519 27207 raft_consensus.cc:2749] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:09:42.184785 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:42.185603 27301 consensus_queue.cc:1048] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:09:42.138218 27309 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:42.169507 23616 raft_consensus.cc:1240] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:09:42.177625 27274 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:42.188015 27274 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:42.188416 27274 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:09:42.176637 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 679555us
I20260709 13:09:42.196041 27274 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:42.200124 27274 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:42.200904 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:42.201589 23621 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:42.203661 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:42.205524 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:42.207960 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:42.208338 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:42.144163 27032 consensus_queue.cc:260] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:42.208968 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:42.209726 27032 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:42.210033 23619 raft_consensus.cc:2410] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:09:42.210989 23621 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 14.
I20260709 13:09:42.211321 27217 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.211833 27217 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
W20260709 13:09:42.198269 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 48685us
I20260709 13:09:42.213909 27217 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:42.218537 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:42.220194 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:42.221705 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:42.222299 27119 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:42.155913 26861 raft_consensus.cc:2804] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:42.223575 26861 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:42.224103 26861 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:42.226065 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:42.227100 22919 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:42.228291 27179 consensus_queue.cc:1059] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:09:42.229147 27179 raft_consensus.cc:3055] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:09:42.229537 27179 raft_consensus.cc:740] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:42.230822 27161 raft_consensus.cc:3055] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:09:42.231345 27161 raft_consensus.cc:740] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:42.149137 27281 raft_consensus.cc:697] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 14 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:42.233512 27281 consensus_queue.cc:237] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:42.223193 27282 consensus_queue.cc:1059] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:09:42.238363 27282 raft_consensus.cc:3055] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 13:09:42.238900 27282 raft_consensus.cc:740] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:42.151559 27330 consensus_queue.cc:1059] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.093s
I20260709 13:09:42.167397 27134 raft_consensus.cc:2749] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:09:42.174646 26794 raft_consensus.cc:2804] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:42.242926 22952 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 13.
I20260709 13:09:42.243186 26794 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:42.243573 26794 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:42.244072 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:09:42.247165 27326 log.cc:927] Time spent T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.301s	user 0.003s	sys 0.000s
I20260709 13:09:42.249624 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:42.250365 22876 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:42.230541 27179 consensus_queue.cc:260] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:42.254963 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:42.255753 22952 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:42.256880 27275 raft_consensus.cc:2749] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:09:42.258591 27119 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:09:42.259106 27119 raft_consensus.cc:697] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:42.260015 27119 consensus_queue.cc:237] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:42.173429 26755 raft_consensus.cc:2804] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:09:42.266049 23588 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:42.267061 22875 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:42.272413 22947 raft_consensus.cc:1240] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 11. Current term is 12. Ops: []
I20260709 13:09:42.277326 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:42.260772 27296 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:09:42.284359 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:42.184542 27242 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:42.289441 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:42.296077 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 82843.6us
I20260709 13:09:42.303709 22876 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 14.
I20260709 13:09:42.306629 22876 raft_consensus.cc:1240] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 13. Current term is 14. Ops: [11.10-13.11]
W20260709 13:09:42.307294 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:42.307628 22952 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
W20260709 13:09:42.297382 22993 connection.cc:256] RPC call timeout handler was delayed by -1.05927192799993s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1628s for a response
W20260709 13:09:42.310169 22993 connection.cc:256] RPC call timeout handler was delayed by -1.0241719279998733s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16670000000000001s for a response
I20260709 13:09:42.280946 27259 raft_consensus.cc:2804] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:09:42.232410 27161 consensus_queue.cc:260] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:42.297529 27145 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:42.369314 23625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 860) took 1751 ms. Trace:
I20260709 13:09:42.326344 22875 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 12.
I20260709 13:09:42.297226 26755 raft_consensus.cc:697] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 12 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:42.369766 22879 raft_consensus.cc:1275] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 13:09:42.373509 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 75447.4us
I20260709 13:09:42.369812 23625 rpcz_store.cc:276] 0709 13:09:40.618419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:40.625800 (+  7381us) service_pool.cc:224] Handling call
0709 13:09:40.656871 (+ 31071us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:40.801733 (+144862us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:40.801768 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:40.801806 (+    38us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:09:40.801852 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:42.369037 (+1567185us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b5400616a54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:42.369088 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:42.369252 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":332654848}
I20260709 13:09:42.378376 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:42.381659 23022 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:42.373737 27145 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:42.240177 27282 consensus_queue.cc:260] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:42.383538 22993 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:42.314836 27179 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:42.401538 23021 raft_consensus.cc:1240] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:09:42.414731 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:42.395853 22994 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:42.389854 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:42.301708 27296 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:42.417825 23029 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 13.
I20260709 13:09:42.370316 27289 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:42.419376 23024 raft_consensus.cc:1240] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
W20260709 13:09:42.432578 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 49181.5us
I20260709 13:09:42.434412 22993 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:09:42.395359 27332 log.cc:927] Time spent T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 13:09:42.389536 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:42.322085 27233 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:42.382763 27145 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:42.436597 23028 raft_consensus.cc:2410] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:09:42.382948 27264 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:42.448247 22993 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:09:42.445376 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 49598.4us
W20260709 13:09:42.453866 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:42.457751 22879 raft_consensus.cc:1275] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 13:09:42.451251 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 282861us
I20260709 13:09:42.473994 22947 raft_consensus.cc:1275] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:09:42.473582 23588 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:42.463445 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:42.393354 26755 consensus_queue.cc:237] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:42.482904 22947 raft_consensus.cc:1275] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:09:42.460582 22874 raft_consensus.cc:1240] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 11. Current term is 12. Ops: []
I20260709 13:09:42.483793 22955 raft_consensus.cc:1275] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:09:42.429361 27329 raft_consensus.cc:2749] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:42.486388 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:42.491139 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:42.491858 23619 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:42.493671 23621 raft_consensus.cc:1240] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 14 LEADER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:09:42.453045 27249 raft_consensus.cc:2749] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:09:42.487782 23588 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:42.389990 27259 raft_consensus.cc:697] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:42.501475 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:42.456851 27250 raft_consensus.cc:2749] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:42.486410 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:42.504433 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 56263.8us
W20260709 13:09:42.463492 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:42.491380 23021 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
I20260709 13:09:42.485843 22918 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:42.436558 27272 raft_consensus.cc:2804] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:42.479247 27022 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:42.503082 23588 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:42.503118 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:42.503896 22947 raft_consensus.cc:1275] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:09:42.534032 23029 raft_consensus.cc:2410] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:09:42.453812 27282 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 6 FOLLOWER]: Advancing to term 7
I20260709 13:09:42.504063 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:42.539283 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:42.539783 22874 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:09:42.507654 23026 raft_consensus.cc:2393] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 11: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
W20260709 13:09:42.439736 27320 log.cc:927] Time spent T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:09:42.450224 27149 consensus_queue.cc:1059] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 13:09:42.544595 27057 consensus_queue.cc:1059] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:09:42.508440 22918 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:42.511466 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:42.505972 27165 raft_consensus.cc:2749] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 13:09:42.548538 23616 raft_consensus.cc:1275] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:09:42.545454 26802 raft_consensus.cc:2749] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:09:42.550647 27312 log.cc:927] Time spent T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.607s	user 0.003s	sys 0.000s
I20260709 13:09:42.538236 27329 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.551805 27329 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:42.552071 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:09:42.558175 27322 log.cc:927] Time spent T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.505s	user 0.002s	sys 0.000s
I20260709 13:09:42.545732 27022 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:42.563092 22880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2527) took 1254 ms. Trace:
I20260709 13:09:42.565932 22874 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:09:42.521512 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:42.521839 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:42.528993 22843 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:42.567466 22843 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:42.532585 27336 log.cc:927] Time spent T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:09:42.517189 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:42.517069 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:42.450469 27289 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:42.573755 23028 raft_consensus.cc:2393] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 11: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:42.492869 27158 raft_consensus.cc:2749] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:09:42.544806 27271 consensus_queue.cc:1048] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:09:42.546173 22918 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:42.520200 27272 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:42.547735 22994 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:42.474351 27264 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:42.565954 23590 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:42.568821 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:42.571250 22871 raft_consensus.cc:1275] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:09:42.572755 22843 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:42.574319 23021 raft_consensus.cc:2368] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 9. Current term is 10.
I20260709 13:09:42.578277 22918 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:42.574977 22920 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:42.563565 22880 rpcz_store.cc:276] 0709 13:09:41.308260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:41.308538 (+   278us) service_pool.cc:224] Handling call
0709 13:09:41.308931 (+   393us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:41.403212 (+ 94281us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:41.403238 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:41.410392 (+  7154us) log.cc:844] Serialized 52 byte log entry
0709 13:09:41.850793 (+440401us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:09:41.881408 (+ 30615us) log.cc:844] Serialized 14 byte log entry
0709 13:09:41.881647 (+   239us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:41.881745 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:42.241262 (+359517us) spinlock_profiling.cc:243] Waited 3.63 ms on lock 0x7b18002f0740. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cf83f3 00007f6132cfa7c1 00007f6132cfa779 00007f6132cfa709 00007f6132cfa6d1 00007f6132cf99cd 000000000038f354 000000000038f199 00007f6132cdb83a
0709 13:09:42.557559 (+316297us) spinlock_profiling.cc:243] Waited 1.66 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:09:42.562831 (+  5272us) raft_consensus.cc:1685] finished
0709 13:09:42.562859 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:42.563023 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":33840,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":57,"raft.queue_time_us":25735,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":2672,"spinlock_wait_cycles":12180096,"thread_start_us":438075,"threads_started":2,"wal-append.queue_time_us":615737}
I20260709 13:09:42.510686 22993 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:42.582165 22994 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:42.588400 22994 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:42.527244 27269 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:42.497587 27170 consensus_queue.cc:1059] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.078s
W20260709 13:09:42.591171 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 80569.9us
I20260709 13:09:42.591495 22871 raft_consensus.cc:1240] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:42.595516 22948 raft_consensus.cc:1275] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:09:42.599206 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:42.600606 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:09:42.617427 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:42.576817 27217 raft_consensus.cc:2749] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:09:42.653057 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 55106.3us
I20260709 13:09:42.678604 23022 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 12.
I20260709 13:09:42.589406 27218 raft_consensus.cc:2749] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:42.594933 26802 raft_consensus.cc:2804] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:42.583653 27275 raft_consensus.cc:2749] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:09:42.595785 26861 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:42.655722 27330 consensus_queue.cc:1059] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:09:42.657472 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 79256us
I20260709 13:09:42.598558 22993 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:42.709990 22918 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:42.711660 22918 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:42.658077 27268 raft_consensus.cc:2804] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:42.659160 27340 raft_consensus.cc:2749] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:42.584271 27329 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:42.660938 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:42.662168 22880 raft_consensus.cc:1275] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:09:42.655752 27296 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:42.665236 27292 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:42.563503 27249 consensus_queue.cc:1048] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 13:09:42.584540 27293 raft_consensus.cc:2749] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:09:42.583379 27272 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:42.671741 27274 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:42.754587 27032 raft_consensus.cc:3055] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:09:42.672065 27264 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:42.755216 27032 raft_consensus.cc:740] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:42.675779 22950 raft_consensus.cc:1240] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:09:42.761582 27277 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:42.676581 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:42.582170 27220 raft_consensus.cc:2749] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:42.764495 23619 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 13.
I20260709 13:09:42.679207 23617 raft_consensus.cc:2468] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
I20260709 13:09:42.683197 23588 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:42.683414 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:09:42.680858 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:42.766168 23022 raft_consensus.cc:2376] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11. Re-sending same reply.
W20260709 13:09:42.766422 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 83296.3us
I20260709 13:09:42.679565 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:42.767136 22871 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
I20260709 13:09:42.767059 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:42.684932 27145 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:42.685051 27077 consensus_queue.cc:1059] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:09:42.679658 27158 consensus_queue.cc:1048] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 13:09:42.687022 23624 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 13.
I20260709 13:09:42.768850 27305 raft_consensus.cc:2804] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:09:42.769362 27161 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:42.769675 27305 raft_consensus.cc:697] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:42.770102 27145 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:42.770598 27305 consensus_queue.cc:237] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:42.771374 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:42.772033 23022 raft_consensus.cc:2368] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 12. Current term is 13.
I20260709 13:09:42.772600 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:42.772749 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:42.773809 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:42.773420 27032 consensus_queue.cc:260] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:42.774351 23022 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 9.
I20260709 13:09:42.774749 27032 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:42.568742 26794 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:42.775122 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:42.776472 23022 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
I20260709 13:09:42.776988 26794 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:42.596560 27149 raft_consensus.cc:2749] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:09:42.777370 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:42.704841 27259 consensus_queue.cc:237] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:42.777954 23022 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:42.675166 27295 raft_consensus.cc:2749] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:09:42.652385 27287 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:42.659833 27262 consensus_meta.cc:287] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.508s	user 0.009s	sys 0.000s
I20260709 13:09:42.654340 22843 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:42.717569 23021 raft_consensus.cc:2376] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12. Re-sending same reply.
W20260709 13:09:42.817028 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 48917.7us
I20260709 13:09:42.675259 27245 raft_consensus.cc:2804] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:09:42.664775 27136 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.820876 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:42.596932 27258 consensus_queue.cc:1059] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.110s
I20260709 13:09:42.682751 27302 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.696966 26802 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:42.722905 26758 raft_consensus.cc:2749] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:42.722905 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:42.731415 27239 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:42.733852 27022 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:42.735877 26861 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:42.715128 27268 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:42.741362 27273 consensus_meta.cc:287] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.561s	user 0.010s	sys 0.000s
I20260709 13:09:42.718258 27134 raft_consensus.cc:2749] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:42.742739 27058 consensus_queue.cc:1059] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:09:42.745396 22918 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:42.747011 27289 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:42.749025 27292 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:42.751569 27249 raft_consensus.cc:2749] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:09:42.673703 27269 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:42.678912 22948 pending_rounds.cc:85] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting all ops after (but not including) 5
I20260709 13:09:42.765017 23616 raft_consensus.cc:2468] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 7.
I20260709 13:09:42.853174 22948 pending_rounds.cc:107] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 13:09:42.682476 22920 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:42.853436 22948 raft_consensus.cc:2889] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:09:42.853824 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 171435us
I20260709 13:09:42.854239 22844 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:42.774372 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:42.855456 23020 raft_consensus.cc:2393] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 10: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:42.773427 23024 raft_consensus.cc:2393] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 10: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:42.856420 27058 raft_consensus.cc:3055] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:09:42.856570 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:42.856894 27058 raft_consensus.cc:740] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:42.857761 23024 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "5b346767a89745fa82928a5dac9cc0b2"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:09:42.839036 27136 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:42.707646 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:42.858995 22954 raft_consensus.cc:2393] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 12: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:42.847582 27145 consensus_queue.cc:1059] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 13:09:42.851584 26758 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:09:42.821602 22947 raft_consensus.cc:1240] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 11. Current term is 12. Ops: []
W20260709 13:09:42.822557 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 168514us
W20260709 13:09:42.860973 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 115659us
I20260709 13:09:42.840802 27302 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:42.820591 27262 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:42.863684 22843 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:42.801381 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:42.864813 27207 raft_consensus.cc:3055] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:09:42.865168 27207 raft_consensus.cc:740] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:42.828204 22877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2528) took 1519 ms. Trace:
I20260709 13:09:42.865933 27207 consensus_queue.cc:260] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:42.867070 27207 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:42.866281 22877 rpcz_store.cc:276] 0709 13:09:41.308914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:41.309350 (+   436us) service_pool.cc:224] Handling call
0709 13:09:41.309655 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:42.565191 (+1255536us) spinlock_profiling.cc:243] Waited 1.93 ms on lock 0x7b180036eec0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f613598d4bc 00007f6132cdb48b 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf5455 00007f6135ddd060 00007f6135dddbfb 00007f6135de864b
0709 13:09:42.578026 (+ 12835us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:42.578055 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:42.584953 (+  6898us) log.cc:844] Serialized 26 byte log entry
0709 13:09:42.807802 (+222849us) raft_consensus.cc:1645] Marking committed up to 3
0709 13:09:42.807865 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:42.809689 (+  1824us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:42.827761 (+ 18072us) raft_consensus.cc:1685] finished
0709 13:09:42.827793 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:42.828126 (+   333us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":37746,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":23176,"spinlock_wait_cycles":4441344,"thread_start_us":207343,"threads_started":2,"wal-append.queue_time_us":227587}
I20260709 13:09:42.864555 22950 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:42.821595 27287 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:42.869728 27334 log.cc:927] Time spent T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.369s	user 0.000s	sys 0.001s
I20260709 13:09:42.871055 27287 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:09:42.833626 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 235116us
I20260709 13:09:42.850594 27273 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:42.874446 22993 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:42.874644 27273 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:42.875663 22993 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:42.877516 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:42.878211 22947 raft_consensus.cc:2393] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 9: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:42.878834 23027 raft_consensus.cc:1240] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 12. Current term is 14. Ops: []
I20260709 13:09:42.834080 23022 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 14.
I20260709 13:09:42.839119 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:42.890853 23588 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:42.841775 27245 raft_consensus.cc:697] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 10 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:42.893052 22993 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:42.839644 23029 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
I20260709 13:09:42.846192 27268 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:42.841146 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:42.894883 23021 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:42.763948 27272 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:42.848945 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 261365us
I20260709 13:09:42.839689 26802 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:09:42.839680 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 158145us
I20260709 13:09:42.893483 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:42.896868 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:42.898396 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:42.901480 23020 raft_consensus.cc:1240] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:42.906908 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:42.910944 22950 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 14.
W20260709 13:09:42.912030 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 48509us
I20260709 13:09:42.910254 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:42.917482 22993 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:42.918607 22877 raft_consensus.cc:2393] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 11: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:42.918494 27269 consensus_queue.cc:1048] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:09:42.920538 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:42.921200 22946 raft_consensus.cc:1275] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
I20260709 13:09:42.921624 22946 pending_rounds.cc:85] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting all ops after (but not including) 10
I20260709 13:09:42.921861 22946 pending_rounds.cc:107] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:09:42.922075 22946 raft_consensus.cc:2889] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:42.922081 22918 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:42.922960 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:42.923548 22880 raft_consensus.cc:2468] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
I20260709 13:09:42.859902 27136 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:42.929574 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:42.893702 27262 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:42.912014 27289 raft_consensus.cc:2749] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:09:42.899852 27077 raft_consensus.cc:2804] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:09:42.900451 27170 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:09:42.905112 27273 raft_consensus.cc:2749] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:09:42.931826 27358 raft_consensus.cc:2749] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:42.932874 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:42.950147 22873 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:09:42.933805 27324 log.cc:927] Time spent T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.981s	user 0.001s	sys 0.001s
I20260709 13:09:42.939900 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:42.952018 22918 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:42.941071 27359 log.cc:927] Time spent T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 13:09:42.938985 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:42.961978 23624 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:42.943763 23021 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
I20260709 13:09:42.945292 23020 raft_consensus.cc:1275] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 13:09:42.930558 27165 raft_consensus.cc:2749] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:09:42.963263 22950 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:09:42.900115 26861 consensus_queue.cc:1048] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 13:09:42.946455 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:42.970131 23616 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 LEARNER]: Advancing to term 12
I20260709 13:09:42.971951 23624 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 14.
I20260709 13:09:42.949303 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:42.952968 22918 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:42.932235 22877 raft_consensus.cc:1275] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:09:42.945823 22875 raft_consensus.cc:1240] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 12 LEADER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 13:09:42.964031 23020 raft_consensus.cc:1240] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 12. Current term is 13. Ops: [12.12-12.12]
I20260709 13:09:42.973915 22873 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 14.
I20260709 13:09:42.964059 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:42.965916 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:42.857788 27058 consensus_queue.cc:260] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:42.947180 26794 consensus_queue.cc:1048] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.019s
I20260709 13:09:42.956013 27077 raft_consensus.cc:697] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:42.938733 27272 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:42.946576 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:42.974900 22950 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:42.975508 23021 raft_consensus.cc:1275] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:09:42.995208 23020 raft_consensus.cc:1240] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:09:42.995586 22951 raft_consensus.cc:2410] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:09:42.980849 27274 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.982151 27349 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:43.003191 22950 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
I20260709 13:09:42.990182 23025 raft_consensus.cc:1240] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:42.940477 27329 consensus_queue.cc:1048] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 13:09:42.953234 26758 raft_consensus.cc:2749] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:42.993856 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:43.004922 23024 raft_consensus.cc:606] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Received request to transfer leadership to TS 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:42.978448 23619 raft_consensus.cc:2393] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 10: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:42.988734 23616 raft_consensus.cc:1275] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEARNER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:09:43.005178 22954 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
I20260709 13:09:43.012256 27262 consensus_queue.cc:1059] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:09:42.985289 27277 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:42.991662 26794 consensus_queue.cc:1243] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer f3c799b1249e40119d2a5864231c6c75 log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 13:09:42.975441 27170 raft_consensus.cc:697] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 14 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:43.019608 27077 consensus_queue.cc:237] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:42.936993 27245 consensus_queue.cc:237] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:43.005295 27210 raft_consensus.cc:2749] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 13:09:43.029449 23616 raft_consensus.cc:1275] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:09:43.040208 27218 raft_consensus.cc:2749] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:09:43.043625 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 189477us
I20260709 13:09:43.044564 23026 raft_consensus.cc:1275] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:09:43.009829 27274 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:43.045570 27233 raft_consensus.cc:2804] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:43.045975 27233 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:43.046355 27233 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:43.047134 27274 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:43.048636 27292 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:43.049028 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:43.049166 27292 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:43.049600 22875 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:43.050879 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:43.051458 22874 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:43.052831 27296 raft_consensus.cc:2749] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:09:43.055372 27119 consensus_queue.cc:1059] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:09:43.056263 27119 consensus_queue.cc:1243] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer 0a407504ed1741bf89a276558adf4c0f log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:09:43.056813 27119 raft_consensus.cc:3055] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:09:43.057308 27119 raft_consensus.cc:740] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:43.059984 27119 consensus_queue.cc:260] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:43.061777 27119 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:43.021358 27277 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:43.014909 27349 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:43.092406 27277 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:43.113329 27170 consensus_queue.cc:237] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:43.086560 27161 consensus_queue.cc:1048] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:09:43.109351 27292 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:43.086280 27283 consensus_queue.cc:1048] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 13:09:43.141803 26802 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:43.142938 27268 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:43.144987 27349 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:43.147536 26802 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:43.148110 27268 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:43.159802 27233 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:43.172932 27343 log.cc:927] Time spent T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.652s	user 0.004s	sys 0.000s
I20260709 13:09:43.176261 27233 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:43.189891 22947 raft_consensus.cc:1275] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:09:43.190527 27207 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:43.200917 27207 raft_consensus.cc:2749] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 13:09:44.030051 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 1.16632e+06us
I20260709 13:09:44.030995 22843 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:44.033164 22843 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:44.034874 22843 leader_election.cc:400] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 9. Current term is 10.
I20260709 13:09:44.035169 22843 leader_election.cc:403] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:09:44.036137 22843 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:44.037217 22843 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:44.039377 22843 leader_election.cc:400] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 12. Current term is 13.
I20260709 13:09:44.039654 22843 leader_election.cc:403] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:09:44.040653 22843 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:44.042440 22843 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:44.044091 27379 raft_consensus.cc:2749] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:44.044243 22843 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:44.044917 27379 raft_consensus.cc:2749] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:44.045567 27379 raft_consensus.cc:2749] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 9. Current term is 10.
I20260709 13:09:44.045699 22843 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:44.046227 27379 raft_consensus.cc:2749] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:09:44.046831 27379 raft_consensus.cc:2804] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:09:44.047405 27379 raft_consensus.cc:697] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 11 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:44.048218 27379 consensus_queue.cc:237] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:44.051337 27379 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:09:44.046860 22843 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:09:44.054184 22764 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2959101199a44a4583a4ee456ecf044d with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 0a407504ed1741bf89a276558adf4c0f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 12 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
W20260709 13:09:44.058610 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:44.059091 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:44.060001 27379 raft_consensus.cc:2749] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 12. Current term is 13.
I20260709 13:09:44.060782 27379 raft_consensus.cc:2804] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:44.061745 27379 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:44.062141 27379 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:44.065227 27382 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:44.065762 27382 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:44.065795 22843 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:44.066275 27382 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:44.068490 27381 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:09:44.073943 27382 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:44.076249 27382 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:44.077970 27382 raft_consensus.cc:2804] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:44.078421 27382 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:44.078843 27382 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:44.080505 27384 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.081059 27384 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:44.077175 27169 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:44.077419 27383 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.088936 27383 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:44.091400 27383 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:44.091979 27383 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:44.092747 27382 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:44.095136 27382 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:44.081935 27381 raft_consensus.cc:697] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 12 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:44.103519 27381 consensus_queue.cc:237] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:44.106046 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:44.106832 27379 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:44.108631 22993 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:44.111588 23022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 1960) took 1588 ms. Trace:
I20260709 13:09:44.113756 22993 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:44.112965 23022 rpcz_store.cc:276] 0709 13:09:42.523485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:42.880554 (+357069us) service_pool.cc:224] Handling call
0709 13:09:42.881057 (+   503us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:42.881380 (+   323us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:09:42.881822 (+   442us) log.cc:844] Serialized 14 byte log entry
0709 13:09:44.079363 (+1197541us) spinlock_profiling.cc:243] Waited 11 ms on lock 0x7b18003af6c0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cdae07 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf7c02 00007f613597f20b 00007f6135984d23 00007f613598523a 00007f6135defa2a
0709 13:09:44.111209 (+ 31846us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:44.111261 (+    52us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:44.111316 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:44.111410 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:44.111530 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":148,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":1075,"spinlock_wait_cycles":25292544,"thread_start_us":31351,"threads_started":1,"wal-append.queue_time_us":1230699}
I20260709 13:09:44.114682 27169 raft_consensus.cc:2749] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:44.119899 27379 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:44.121186 27282 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:44.121677 27282 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:44.123361 27282 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:44.123819 27282 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:44.124370 27282 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:44.125094 27384 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:44.126116 27271 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:44.126811 27282 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:44.126749 27271 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:44.129736 27271 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:44.130316 27207 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.130870 27207 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:44.133133 27207 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:44.133664 27207 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.134330 27207 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:09:44.136885 27207 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:44.139003 27207 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:44.139564 27207 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:44.141681 27207 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:44.142226 27207 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.142664 27207 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:44.144603 27207 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:44.145116 27207 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:44.145576 27207 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:44.147567 27207 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:44.148083 27207 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:44.148542 27207 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:44.150876 27207 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:44.152776 27281 raft_consensus.cc:3055] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:09:44.153152 27382 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:44.153985 22872 raft_consensus.cc:1240] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:09:44.153769 27382 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:44.152820 27381 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.155707 27381 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:44.153234 27281 raft_consensus.cc:740] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:44.156032 27382 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:44.158639 27281 consensus_queue.cc:260] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:44.161761 27281 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:44.162739 27381 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:44.165720 27273 raft_consensus.cc:2804] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:09:44.166407 27273 raft_consensus.cc:697] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:44.167312 27273 consensus_queue.cc:237] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:44.179471 23025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 706) took 1804 ms. Trace:
I20260709 13:09:44.180003 23025 rpcz_store.cc:276] 0709 13:09:42.375246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:42.384158 (+  8912us) service_pool.cc:224] Handling call
0709 13:09:42.426173 (+ 42015us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:43.004675 (+578502us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:44.179135 (+1174460us) spinlock_profiling.cc:243] Waited 564 ms on lock 0x7b54002c9554. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:44.179391 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1296478592}
I20260709 13:09:44.197742 27281 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.199095 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:44.198727 27281 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:44.199970 22877 raft_consensus.cc:3060] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:44.202629 27057 raft_consensus.cc:2804] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:09:44.203199 27057 raft_consensus.cc:697] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 14 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:44.204138 27057 consensus_queue.cc:237] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:44.205405 26802 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:44.205957 26802 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:44.211222 26802 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:44.213454 22877 raft_consensus.cc:2468] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 9.
W20260709 13:09:44.218369 23050 debug-util.cc:398] Leaking SignalData structure 0x7b080016fe80 after lost signal to thread 21756
W20260709 13:09:44.218797 23050 debug-util.cc:398] Leaking SignalData structure 0x7b080007fc20 after lost signal to thread 21759
I20260709 13:09:44.221266 27220 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:44.223225 27220 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:44.223717 27057 consensus_queue.cc:1048] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:09:44.221920 27249 consensus_queue.cc:1048] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 13:09:44.230952 27220 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:44.231763 27249 consensus_queue.cc:1243] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer 5b346767a89745fa82928a5dac9cc0b2 log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:09:44.235734 27249 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.236285 27249 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:44.239666 27249 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:44.229295 27273 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.240593 27273 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:44.242667 27273 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:44.232631 27281 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:44.254522 22874 consensus_meta.cc:287] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 1.203s	user 0.014s	sys 0.006s
I20260709 13:09:44.255833 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:44.256765 22880 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 13.
I20260709 13:09:44.257972 22874 raft_consensus.cc:2468] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 9.
I20260709 13:09:44.258560 22874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35360 (request call id 2047) took 1213 ms. Trace:
I20260709 13:09:44.258930 22874 rpcz_store.cc:276] 0709 13:09:43.045273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:43.050633 (+  5360us) service_pool.cc:224] Handling call
0709 13:09:44.258500 (+1207867us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:09:44.274236 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:44.279835 27163 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:44.280844 26794 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.281490 26794 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:44.283917 26794 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:44.286648 22949 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.277s	user 0.013s	sys 0.013s
W20260709 13:09:44.287204 22949 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.279s	user 0.014s	sys 0.014s
I20260709 13:09:44.287969 27032 raft_consensus.cc:2749] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:44.291138 22949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 1997) took 1346 ms. Trace:
I20260709 13:09:44.292483 27268 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:44.295650 22949 rpcz_store.cc:276] 0709 13:09:42.945001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:42.945638 (+   637us) service_pool.cc:224] Handling call
0709 13:09:42.945920 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:42.946096 (+   176us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:09:42.975463 (+ 29367us) log.cc:844] Serialized 14 byte log entry
0709 13:09:44.287635 (+1312172us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:44.287699 (+    64us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:44.287750 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:44.290930 (+  3180us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:44.291076 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33903,"prepare.queue_time_us":36527,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":27,"raft.queue_time_us":9821,"raft.run_cpu_time_us":100,"raft.run_wall_time_us":6303,"thread_start_us":1300893,"threads_started":2,"wal-append.queue_time_us":36739,"wal-append.run_cpu_time_us":1500,"wal-append.run_wall_time_us":1001495}
I20260709 13:09:44.297911 27268 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:44.284442 26794 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.301209 26794 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:44.301904 27268 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:44.303151 26794 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:44.306258 27275 consensus_queue.cc:1048] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:09:44.313845 27305 raft_consensus.cc:3055] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:09:44.314293 27305 raft_consensus.cc:740] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:44.315119 27305 consensus_queue.cc:260] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:44.316113 27305 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:09:44.327384 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:44.328305 27233 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:44.328941 27233 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:44.331949 27220 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:44.332551 26758 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:44.332814 27281 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.332515 27220 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:44.333355 27281 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:44.334465 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:44.334776 27220 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:44.335065 23616 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 13.
I20260709 13:09:44.331423 27233 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:44.335498 27281 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:44.336115 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:44.333386 26758 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:44.339159 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:44.339938 26758 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:44.340413 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:44.336481 27349 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.341430 27349 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:44.343473 27349 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:44.344882 27247 consensus_queue.cc:1059] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:09:44.345621 27247 consensus_queue.cc:1243] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer 0a407504ed1741bf89a276558adf4c0f log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:09:44.345767 23027 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 1943) took 1883 ms. Trace:
I20260709 13:09:44.346305 27145 raft_consensus.cc:3055] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:09:44.346730 27145 raft_consensus.cc:740] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:44.347684 27145 consensus_queue.cc:260] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:44.349973 23590 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:44.352283 23621 raft_consensus.cc:1275] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:09:44.355755 22826 auto_rebalancer.cc:629] Failed to schedule move for tablet ec7dd25342084943aacb4b9830f5f1ea: Illegal state: Replica 0a407504ed1741bf89a276558adf4c0f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 10 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:09:44.359941 26758 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.360486 26758 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:44.362124 27274 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:44.362625 26758 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:44.362661 27274 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:44.363684 27301 consensus_queue.cc:1048] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:09:44.365097 27274 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:44.365190 22843 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:44.366106 27218 raft_consensus.cc:2804] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:44.366236 22843 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:44.366528 27218 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:44.366917 27218 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:44.367085 27233 raft_consensus.cc:2804] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:09:44.367774 27233 raft_consensus.cc:697] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 13 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:44.368620 27233 consensus_queue.cc:237] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:44.369158 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:44.370294 26802 raft_consensus.cc:2804] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:09:44.370841 26802 raft_consensus.cc:697] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:44.372224 27358 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.373133 22880 raft_consensus.cc:1240] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:44.373869 23590 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:44.374462 22952 raft_consensus.cc:1240] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14. Ops: []
W20260709 13:09:44.374666 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 74408.4us
I20260709 13:09:44.377090 22919 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:44.346158 23027 rpcz_store.cc:276] 0709 13:09:42.462009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:42.462139 (+   130us) service_pool.cc:224] Handling call
0709 13:09:42.490817 (+ 28678us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:42.879272 (+388455us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:44.345551 (+1466279us) spinlock_profiling.cc:243] Waited 388 ms on lock 0x7b5400248e54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:44.345708 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":891854208}
I20260709 13:09:44.379061 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:44.381093 22880 raft_consensus.cc:1275] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:09:44.389515 23616 raft_consensus.cc:1217] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEARNER]: Deduplicated request from leader. Original: 6.5->[8.6-12.8]   Dedup: 8.6->[10.7-12.8]
I20260709 13:09:44.390046 23616 pending_rounds.cc:85] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2: Aborting all ops after (but not including) 6
I20260709 13:09:44.390282 23616 pending_rounds.cc:107] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:09:44.390488 23616 raft_consensus.cc:2889] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEARNER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:44.371711 26802 consensus_queue.cc:237] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:44.391145 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:44.392630 22919 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:44.393860 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:44.394439 22872 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 13.
I20260709 13:09:44.395659 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:44.391775 22953 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11.
I20260709 13:09:44.397204 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:44.397764 22872 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 14.
I20260709 13:09:44.395581 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:44.398175 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:44.398670 22880 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:44.396325 22947 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 13.
I20260709 13:09:44.402664 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:44.403419 22951 raft_consensus.cc:2410] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 12 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:09:44.405083 22992 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:44.406455 22919 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:44.409646 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:44.410305 22877 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 12.
I20260709 13:09:44.411320 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:44.408427 22920 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:44.399693 27349 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:44.391598 27022 consensus_queue.cc:1048] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 13:09:44.419549 27349 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:44.419906 27022 consensus_queue.cc:1243] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer aed3f089ec8b49af9c8cf5cdee76e8e7 log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:09:44.419970 27349 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:44.396363 27358 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:44.421175 26861 consensus_queue.cc:1048] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:09:44.422544 27382 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:44.423002 27358 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:44.423038 27382 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:44.423822 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:44.425407 27382 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:44.423332 27379 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:44.427065 27379 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:44.427340 27218 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:44.404040 27329 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:44.428997 27329 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:44.429394 27329 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:09:44.431535 22826 auto_rebalancer.cc:629] Failed to schedule move for tablet 612dc49c81064f4faa79f65fcf1cfc02: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:09:44.432211 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:44.432830 22872 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11.
I20260709 13:09:44.434624 22880 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 12.
I20260709 13:09:44.399411 27145 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:44.439502 22872 raft_consensus.cc:1240] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:44.438592 22992 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:44.399609 27134 raft_consensus.cc:2804] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:44.442885 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:44.396543 27274 consensus_queue.cc:1059] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:09:44.418740 26758 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:44.399622 27335 consensus_queue.cc:1048] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:09:44.431334 27354 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.446710 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:44.447212 22872 raft_consensus.cc:1240] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:44.453462 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:44.454094 22872 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 14.
I20260709 13:09:44.451134 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:44.450939 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:44.448169 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:44.399643 27301 consensus_queue.cc:1048] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:09:44.454957 22871 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 6.
I20260709 13:09:44.440812 27032 raft_consensus.cc:2804] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:09:44.455065 23625 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 13.
I20260709 13:09:44.456117 23590 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:44.462744 22992 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:44.444414 27134 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:44.475977 22920 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:44.447382 26758 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:44.481539 27274 consensus_queue.cc:1059] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 13:09:44.483314 27207 consensus_queue.cc:1048] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 13:09:44.485190 27379 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:44.485632 27032 raft_consensus.cc:697] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:44.447893 27218 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:44.484234 23590 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:44.492373 22953 raft_consensus.cc:1240] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 13:09:44.492306 22920 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:44.496479 22949 raft_consensus.cc:1275] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:09:44.488471 27134 raft_consensus.cc:3060] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:44.452723 27022 raft_consensus.cc:2749] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:44.494206 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:44.497800 22947 raft_consensus.cc:2410] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:09:44.496973 22919 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:44.452111 27273 consensus_queue.cc:1059] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:09:44.497010 22920 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:44.513113 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:44.490903 26758 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:44.501482 22951 raft_consensus.cc:1217] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Deduplicated request from leader. Original: 6.5->[8.6-12.8]   Dedup: 8.6->[10.7-12.8]
I20260709 13:09:44.514815 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:44.514889 22951 pending_rounds.cc:85] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting all ops after (but not including) 6
I20260709 13:09:44.502537 27258 raft_consensus.cc:2804] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:44.502946 27274 raft_consensus.cc:3055] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:09:44.491822 27136 raft_consensus.cc:2804] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:09:44.494731 27268 raft_consensus.cc:2804] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:09:44.509778 22826 auto_rebalancer.cc:629] Failed to schedule move for tablet 7cd8492f63fb4193a47cf9fbc574ef58: Illegal state: Replica 0a407504ed1741bf89a276558adf4c0f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 10 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:09:44.519212 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:44.519990 23020 raft_consensus.cc:1275] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:09:44.456535 27354 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:44.516767 22951 pending_rounds.cc:107] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 13:09:44.516811 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:44.517616 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:44.515970 22920 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:44.519796 23025 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
I20260709 13:09:44.511348 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:44.521816 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:44.522171 27258 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:44.526429 27136 raft_consensus.cc:697] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 10 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:44.526878 27268 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:44.501112 27239 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.530375 22994 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:44.504997 27022 raft_consensus.cc:2804] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:44.544034 22951 raft_consensus.cc:2889] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:44.543432 23022 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:44.507290 27358 consensus_queue.cc:1059] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 13:09:44.547461 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:44.549516 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:44.550441 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:44.552421 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:44.553449 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:44.515017 27301 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:44.557502 23022 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
I20260709 13:09:44.520409 27305 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:44.549996 27022 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:44.559226 27022 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:44.529115 27274 raft_consensus.cc:740] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:44.559291 22949 raft_consensus.cc:1275] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
W20260709 13:09:44.623858 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:44.624517 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:44.622967 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:44.570132 22875 consensus_meta.cc:287] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 1.520s	user 0.015s	sys 0.003s
I20260709 13:09:44.618621 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:09:44.562120 22918 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.587s (SENT) [suppressed 23 similar messages]
I20260709 13:09:44.626519 23028 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 13
W20260709 13:09:44.626711 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:44.626457 22875 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 14.
I20260709 13:09:44.608608 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:44.570566 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:44.630287 23025 raft_consensus.cc:2393] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 7: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:44.633116 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:44.634063 23023 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:09:44.633635 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:44.635131 22875 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35360 (request call id 2046) took 1590 ms (client timeout 1916 ms). Trace:
I20260709 13:09:44.635408 27402 raft_consensus.cc:2804] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:09:44.635560 22875 rpcz_store.cc:269] 0709 13:09:43.044970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:43.048779 (+  3809us) service_pool.cc:224] Handling call
0709 13:09:44.635064 (+1586285us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:44.636121 27402 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:44.547039 27268 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:44.547266 27258 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:44.637063 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:44.628141 23020 consensus_queue.cc:237] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:09:44.645423 22949 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 7.
I20260709 13:09:44.647099 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:44.637677 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:44.548475 27273 consensus_queue.cc:1059] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.090s
I20260709 13:09:44.630481 23022 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
W20260709 13:09:44.639413 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 113359us
I20260709 13:09:44.640758 23621 raft_consensus.cc:1240] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 LEADER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:44.647982 22955 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
I20260709 13:09:44.622925 27136 consensus_queue.cc:237] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:44.648933 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:44.649459 22955 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:44.649829 27408 raft_consensus.cc:2804] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:44.647540 22949 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
I20260709 13:09:44.655309 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:44.551705 27239 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:44.644461 27402 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:44.653995 27249 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:44.650408 27408 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:44.665915 27404 consensus_queue.cc:1048] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:09:44.667943 27404 consensus_queue.cc:1048] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:09:44.584800 27032 consensus_queue.cc:237] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:44.586586 27301 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:44.660384 26802 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:44.698618 27210 consensus_queue.cc:1059] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:09:44.699757 27210 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.702016 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:44.703280 27329 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:44.658910 27274 consensus_queue.cc:260] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:44.704819 27119 consensus_queue.cc:1059] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:09:44.705328 27274 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:09:44.707041 27397 log.cc:927] Time spent T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.075s	user 0.002s	sys 0.000s
I20260709 13:09:44.709375 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:44.700237 27210 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:44.711228 23028 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
I20260709 13:09:44.712152 27210 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:44.713392 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:44.714892 27149 raft_consensus.cc:3055] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:09:44.715288 27149 raft_consensus.cc:740] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:44.685608 27405 raft_consensus.cc:2804] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:09:44.718966 22955 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
I20260709 13:09:44.720093 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:44.647805 23023 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 10.
I20260709 13:09:44.723876 22949 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
I20260709 13:09:44.726166 22955 raft_consensus.cc:1275] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:09:44.725565 22919 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:44.726593 22955 pending_rounds.cc:85] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting all ops after (but not including) 9
I20260709 13:09:44.726948 22955 pending_rounds.cc:107] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:09:44.727205 22955 raft_consensus.cc:2889] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:44.680959 27408 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:44.725548 22920 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:44.742134 27410 raft_consensus.cc:2804] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:09:44.743002 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:44.729341 27404 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.746236 27239 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:44.747464 27301 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:44.681028 27249 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:44.750802 22880 raft_consensus.cc:1275] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:09:44.764739 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 202624us
I20260709 13:09:44.760990 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:09:44.761077 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 136172us
I20260709 13:09:44.733434 27207 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:44.716142 27149 consensus_queue.cc:260] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:44.738328 27405 raft_consensus.cc:697] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:44.766144 27409 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:44.775506 27301 raft_consensus.cc:3055] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:09:44.776113 27301 raft_consensus.cc:740] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:44.775571 27405 consensus_queue.cc:237] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:44.773572 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:44.776925 27301 consensus_queue.cc:260] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:44.778458 27301 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:44.780539 22871 raft_consensus.cc:2393] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 13: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:44.686090 26802 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:44.716326 27268 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:44.781494 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 111762us
W20260709 13:09:44.772344 22918 leader_election.cc:336] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.634s (SENT)
I20260709 13:09:44.795255 22918 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:44.795689 22994 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:44.796176 23618 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 13.
I20260709 13:09:44.735144 27210 raft_consensus.cc:2749] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:44.797711 22919 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:09:44.769340 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:44.751996 27329 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:44.807057 22994 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:44.811478 22994 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:44.774158 27149 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:44.775506 27413 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.817217 22877 raft_consensus.cc:1275] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:09:44.794591 27145 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.750758 27349 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:44.819695 22877 raft_consensus.cc:1240] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:44.749660 27134 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:44.835171 22877 raft_consensus.cc:1240] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 7. Current term is 8. Ops: []
I20260709 13:09:44.765713 27136 raft_consensus.cc:2764] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 13:09:44.794410 27410 raft_consensus.cc:697] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:44.839177 27410 consensus_queue.cc:237] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:44.840816 26802 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:44.820601 26758 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:44.842700 22877 raft_consensus.cc:1275] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:09:44.843344 26758 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:44.843640 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:44.748607 27258 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:44.845666 27414 raft_consensus.cc:2749] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:44.845788 27416 raft_consensus.cc:2749] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:09:44.846181 27395 log.cc:927] Time spent T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.325s	user 0.002s	sys 0.000s
I20260709 13:09:44.846503 27414 raft_consensus.cc:2804] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:44.847474 27258 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:44.822750 22880 raft_consensus.cc:1240] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 11. Current term is 12. Ops: []
I20260709 13:09:44.846902 27414 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:44.849699 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:44.774643 27022 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:44.766144 27404 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:44.856541 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:44.860594 27022 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:44.775444 27409 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:44.836751 27145 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:44.864967 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:09:44.869004 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 73092.5us
I20260709 13:09:44.870350 23021 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:44.821146 27268 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:44.864459 27404 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:44.865639 22953 raft_consensus.cc:1240] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 13:09:44.877024 27416 consensus_queue.cc:1048] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:09:44.877462 27302 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:44.879228 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:44.879848 22880 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 6 FOLLOWER]: Advancing to term 8
I20260709 13:09:44.862150 27414 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:44.879958 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:44.882151 23617 raft_consensus.cc:2410] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 14 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 13:09:44.887616 22954 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 10.
I20260709 13:09:44.891422 27353 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.891995 27353 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:44.892462 27145 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:44.894855 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:44.895141 27349 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:44.895567 27353 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:44.872306 27418 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.888933 27022 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.899808 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:44.900090 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:44.897607 22993 consensus_peers.cc:597] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 -> Peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Couldn't send request to peer 5b346767a89745fa82928a5dac9cc0b2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ac5cc94926a742e6a0c1340ec33c0ff0. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:44.900565 23020 raft_consensus.cc:2368] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 12. Current term is 13.
I20260709 13:09:44.898358 27022 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:44.901064 23023 raft_consensus.cc:2410] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:09:44.897223 27404 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.833813 27413 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:44.903499 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:44.903805 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:44.909139 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:44.903972 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:44.907101 27134 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:44.909677 22872 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:44.916129 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:44.916940 23022 raft_consensus.cc:2393] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 15: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:44.916241 22880 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 8.
I20260709 13:09:44.912472 23021 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
I20260709 13:09:44.917640 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:44.917681 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:44.918145 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:44.918769 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:44.919538 23023 raft_consensus.cc:2410] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:09:44.896548 27136 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.923777 23023 raft_consensus.cc:1240] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:09:44.924768 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:44.928112 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:44.928627 23021 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 12.
I20260709 13:09:44.934942 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:44.935647 23025 raft_consensus.cc:2393] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 12: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:44.923749 27136 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:44.903187 27418 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:44.919564 27404 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:44.933848 27402 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:44.970327 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:44.976745 23020 raft_consensus.cc:2410] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 13:09:45.031678 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:45.040094 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:09:45.038491 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:44.984270 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:45.049686 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:45.046787 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:45.052551 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:45.033406 27404 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:45.053305 23027 raft_consensus.cc:2393] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 10: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:45.051146 23025 raft_consensus.cc:1240] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:45.057332 22872 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 14.
I20260709 13:09:45.047983 23617 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
W20260709 13:09:45.057915 23588 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:45.049719 23623 raft_consensus.cc:1275] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:09:45.058449 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:45.058761 22993 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:45.060046 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:45.060748 22872 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
I20260709 13:09:45.061684 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:45.062259 22872 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:45.067811 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:45.066839 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:45.077625 23021 raft_consensus.cc:2393] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 13: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:45.057912 23026 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:09:45.072508 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:45.080360 23027 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 12.
I20260709 13:09:45.077988 22880 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 13.
I20260709 13:09:45.079731 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:45.031847 27022 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:45.084460 22872 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 15.
I20260709 13:09:45.083284 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:45.087185 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:45.046097 27207 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:45.082715 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:45.009298 27301 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:44.985387 27418 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:45.103032 27145 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.103557 27145 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:45.103961 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:45.104737 22871 raft_consensus.cc:2410] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 13:09:45.105288 27402 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:45.105516 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:45.087565 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:45.094662 23021 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 13.
W20260709 13:09:45.095476 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:45.106590 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:45.107369 23590 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:45.107429 22872 raft_consensus.cc:1275] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:09:45.097110 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:45.108042 22871 raft_consensus.cc:1240] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 11. Current term is 12. Ops: []
I20260709 13:09:45.108450 22880 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 12.
I20260709 13:09:45.108929 22872 raft_consensus.cc:1240] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14. Ops: [11.6-13.7]
I20260709 13:09:45.078923 27404 raft_consensus.cc:2804] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:45.098902 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:45.032287 27408 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:45.107432 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:45.111389 23021 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:45.112573 23590 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:45.113144 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:45.113415 23590 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:45.097783 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:45.110111 22872 raft_consensus.cc:1275] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:09:45.115623 22994 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:45.089299 27409 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:45.099041 27207 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:45.119144 22994 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:45.119359 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:45.110850 27413 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:09:45.122505 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 56565.6us
I20260709 13:09:45.084802 27421 consensus_queue.cc:1048] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:09:45.110725 27404 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:45.129747 22880 raft_consensus.cc:1275] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:09:45.129642 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:45.099428 27022 consensus_queue.cc:1048] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 13:09:45.100104 27416 consensus_queue.cc:1059] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:09:45.105844 27301 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:45.141254 23590 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:45.153158 23021 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 8.
I20260709 13:09:45.154086 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:45.154662 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:45.155755 23021 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11.
I20260709 13:09:45.157217 23021 raft_consensus.cc:1217] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Deduplicated request from leader. Original: 5.3->[7.4-12.6]   Dedup: 7.4->[10.5-12.6]
I20260709 13:09:45.158672 23021 raft_consensus.cc:1240] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 12. Current term is 13. Ops: [7.4-12.6]
I20260709 13:09:45.160854 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:45.161995 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:45.128353 27404 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:45.162618 23027 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11.
I20260709 13:09:45.160655 23021 raft_consensus.cc:1275] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:09:45.129115 27136 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:45.126888 27241 consensus_queue.cc:1059] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 13:09:45.131413 27302 consensus_queue.cc:1048] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
W20260709 13:09:45.184821 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 44844.3us
I20260709 13:09:45.185994 27440 raft_consensus.cc:2749] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:45.186893 27265 consensus_queue.cc:1059] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:09:45.186846 27440 consensus_queue.cc:1048] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 13:09:45.178167 27136 raft_consensus.cc:2749] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:45.176502 27436 raft_consensus.cc:2804] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 13:09:45.205150 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 52589.9us
I20260709 13:09:45.194221 27438 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:09:45.201915 27434 raft_consensus.cc:2804] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:45.223165 27434 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:45.223628 27301 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:45.223848 27145 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:45.224465 22946 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.141s	user 0.004s	sys 0.016s
I20260709 13:09:45.225446 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:45.225832 22948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2704) took 2769 ms. Trace:
I20260709 13:09:45.226302 23620 raft_consensus.cc:2410] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:09:45.226234 22948 rpcz_store.cc:276] 0709 13:09:42.456260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:42.456763 (+   503us) service_pool.cc:224] Handling call
0709 13:09:42.457028 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:42.853895 (+396867us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:45.225524 (+2371629us) spinlock_profiling.cc:243] Waited 133 ms on lock 0x7b540037afd4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:45.225705 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":306034944}
I20260709 13:09:45.223642 27434 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:09:45.233227 27435 log.cc:927] Time spent T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:09:45.205732 27241 raft_consensus.cc:3055] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:09:45.234802 27241 raft_consensus.cc:740] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:45.235637 27241 consensus_queue.cc:260] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:45.236749 23029 raft_consensus.cc:1275] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:09:45.236959 27241 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:45.237284 27434 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:45.237867 27408 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:45.238175 23024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:60044 (request call id 32) took 2711 ms. Trace:
I20260709 13:09:45.238575 23024 rpcz_store.cc:276] 0709 13:09:42.526669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:42.857616 (+330947us) service_pool.cc:224] Handling call
0709 13:09:45.238000 (+2380384us) spinlock_profiling.cc:243] Waited 147 ms on lock 0x7b54002c9554. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135ddf853 00007f613adad60a 00007f61343337d4 00007f6134333762 00007f6134333691 00007f613433360c 00007f61343328c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:45.238089 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":336995584}
I20260709 13:09:45.238535 27436 raft_consensus.cc:697] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 8 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
W20260709 13:09:45.214447 27433 log.cc:927] Time spent T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 13:09:45.243338 27145 raft_consensus.cc:2804] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:45.243757 27177 raft_consensus.cc:3055] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:09:45.244233 27177 raft_consensus.cc:740] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:45.244415 27434 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:45.245155 27177 consensus_queue.cc:260] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:45.246799 27177 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:45.249181 27436 consensus_queue.cc:237] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:45.250103 26794 raft_consensus.cc:3055] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:09:45.250586 26794 raft_consensus.cc:740] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:45.257580 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:45.260008 22946 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.176s	user 0.005s	sys 0.016s
I20260709 13:09:45.243786 27145 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:45.288630 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:45.206248 27265 consensus_queue.cc:1048] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.050s
I20260709 13:09:45.231652 27414 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:45.303001 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:45.303460 27329 raft_consensus.cc:2749] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:09:45.304332 27329 consensus_queue.cc:1048] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:09:45.307543 27077 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:09:45.308180 27077 raft_consensus.cc:697] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:45.309325 27077 consensus_queue.cc:237] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:45.316838 27032 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.317446 27032 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:45.319859 27032 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:45.322439 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:45.324776 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:45.325584 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:45.327368 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:45.329106 27311 raft_consensus.cc:3055] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:09:45.329663 27311 raft_consensus.cc:740] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:45.330600 27311 consensus_queue.cc:260] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:45.333295 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:45.331789 27311 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:45.334066 23619 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
W20260709 13:09:45.339258 27391 log.cc:927] Time spent T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.837s	user 0.001s	sys 0.000s
I20260709 13:09:45.345937 27402 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.346479 27402 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:45.348742 23618 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 13.
W20260709 13:09:45.350854 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:45.352777 22920 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:45.354735 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:45.355458 22875 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:45.356256 27408 raft_consensus.cc:2804] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:09:45.356809 27408 raft_consensus.cc:697] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:45.350339 27402 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:45.357718 27408 consensus_queue.cc:237] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:45.359113 27272 consensus_queue.cc:1048] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:09:45.359508 27305 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:45.359947 27305 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:45.363404 27272 consensus_queue.cc:1059] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:09:45.366048 27396 log.cc:927] Time spent T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.850s	user 0.001s	sys 0.000s
W20260709 13:09:45.367609 27394 log.cc:927] Time spent T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.864s	user 0.002s	sys 0.000s
I20260709 13:09:45.370008 27340 raft_consensus.cc:3055] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:09:45.370407 27340 raft_consensus.cc:740] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:45.371263 27340 consensus_queue.cc:260] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:45.372267 27340 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:45.364437 27272 raft_consensus.cc:3055] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:09:45.372991 27272 raft_consensus.cc:740] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:45.373821 22873 raft_consensus.cc:1240] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:09:45.374441 22873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53724 (request call id 712) took 1168 ms. Trace:
I20260709 13:09:45.374114 27272 consensus_queue.cc:260] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:45.376303 27272 raft_consensus.cc:3060] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:45.374783 22873 rpcz_store.cc:276] 0709 13:09:44.205623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:45.373364 (+1167741us) service_pool.cc:224] Handling call
0709 13:09:45.373663 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:45.374202 (+   539us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:45.374390 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:45.388193 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:45.390071 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:45.382886 22876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2548) took 2716 ms. Trace:
I20260709 13:09:45.390641 22876 rpcz_store.cc:276] 0709 13:09:42.666310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:42.735966 (+ 69656us) service_pool.cc:224] Handling call
0709 13:09:42.736257 (+   291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:42.781453 (+ 45196us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:09:42.781783 (+   330us) log.cc:844] Serialized 14 byte log entry
0709 13:09:43.036512 (+254729us) spinlock_profiling.cc:243] Waited 17.1 ms on lock 0x7b18003355c0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132c72c0a 00007f613c3b2219 00007f6132cfaa40 00007f6132cf54e4 00007f6132cf5455 00007f6135ddd060 00007f6135def716 00007f6135e00287 00007f6135e00208
0709 13:09:43.036658 (+   146us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:43.036721 (+    63us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:09:43.036775 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:45.382607 (+2345832us) spinlock_profiling.cc:243] Waited 44.9 ms on lock 0x7b54003bc754. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:45.382663 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:45.382824 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49976,"prepare.queue_time_us":129,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":255854,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":89,"spinlock_wait_cycles":142629376}
I20260709 13:09:45.393278 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:45.394955 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:45.395965 22876 raft_consensus.cc:2393] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 11: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:45.400283 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:45.401012 27408 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.400300 22873 raft_consensus.cc:1240] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:45.401558 27408 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:45.402845 27410 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:45.403810 27408 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:45.403668 27410 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:45.405278 22952 raft_consensus.cc:1240] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 11. Current term is 12. Ops: []
I20260709 13:09:45.254977 27438 raft_consensus.cc:697] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:45.406507 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:45.411309 22952 raft_consensus.cc:2410] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:09:45.411191 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:45.407740 22949 raft_consensus.cc:1275] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:09:45.401211 22948 raft_consensus.cc:2410] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 14 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 11.
I20260709 13:09:45.413417 22955 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:45.416333 22879 raft_consensus.cc:1240] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:45.416860 23029 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 713) took 2878 ms. Trace:
I20260709 13:09:45.416967 22879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53724 (request call id 713) took 1211 ms. Trace:
I20260709 13:09:45.417536 22879 rpcz_store.cc:276] 0709 13:09:44.205882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:45.410620 (+1204738us) service_pool.cc:224] Handling call
0709 13:09:45.410857 (+   237us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:45.416753 (+  5896us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:45.416919 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:45.418087 22954 raft_consensus.cc:1275] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 13:09:45.417742 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:45.290253 27145 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:09:45.431864 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 304931us
I20260709 13:09:45.447053 22953 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:09:45.447618 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:45.447902 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:45.448701 23618 raft_consensus.cc:2410] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:09:45.418686 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:45.450901 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:45.451231 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:45.458192 23621 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
I20260709 13:09:45.458789 22879 raft_consensus.cc:1240] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:45.460424 23619 raft_consensus.cc:1275] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:09:45.419914 23028 raft_consensus.cc:2393] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 9: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:45.402690 22873 raft_consensus.cc:1240] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:45.465479 22949 raft_consensus.cc:1275] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:09:45.464041 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:45.466463 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:45.457170 23029 rpcz_store.cc:276] 0709 13:09:42.538051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:45.235963 (+2697912us) service_pool.cc:224] Handling call
0709 13:09:45.236336 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:45.237219 (+   883us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:45.416797 (+179578us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:45.467873 22948 raft_consensus.cc:2468] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:09:45.466445 22873 raft_consensus.cc:1275] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:09:45.469925 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:45.471069 22955 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
I20260709 13:09:45.461046 22879 raft_consensus.cc:1275] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:09:45.473376 26861 consensus_queue.cc:1048] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:09:45.476930 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:45.444797 22875 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 14.
I20260709 13:09:45.486622 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:45.494400 22992 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:45.497932 22844 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:45.495728 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:45.488631 22955 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
I20260709 13:09:45.494855 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:45.491070 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:45.502504 23618 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
I20260709 13:09:45.501922 23617 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
I20260709 13:09:45.513226 27301 consensus_queue.cc:1059] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:09:45.519662 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:45.520761 22953 raft_consensus.cc:2410] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:09:45.402220 27305 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:45.527078 27439 log.cc:927] Time spent T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.229s	user 0.000s	sys 0.000s
I20260709 13:09:45.528260 27305 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:45.423708 27406 consensus_queue.cc:1059] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.092s
I20260709 13:09:45.529120 27406 raft_consensus.cc:3055] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:09:45.507511 26755 consensus_queue.cc:1048] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 13:09:45.529472 27406 raft_consensus.cc:740] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:45.528406 27414 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:45.530280 27406 consensus_queue.cc:260] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:45.524493 27245 raft_consensus.cc:2749] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:45.531445 27406 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:45.530663 27410 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:45.534652 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:45.541486 23029 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 10.
I20260709 13:09:45.536315 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:45.509703 27292 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:45.550827 27207 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:09:45.491719 27329 consensus_queue.cc:1048] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:09:45.551383 27447 consensus_queue.cc:1048] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:09:45.554795 22954 raft_consensus.cc:1240] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:45.559296 27438 consensus_queue.cc:237] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:45.561635 22992 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:45.547102 27292 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:45.562162 27442 consensus_queue.cc:1048] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 13:09:45.565894 27207 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:45.570627 27165 consensus_queue.cc:1048] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:09:45.576750 27249 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:45.568013 27404 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:45.586824 27292 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:45.597775 22992 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:45.603891 22992 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:45.627942 23026 raft_consensus.cc:2393] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 13: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
W20260709 13:09:45.640815 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:45.596689 27249 raft_consensus.cc:2749] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:45.672441 27445 raft_consensus.cc:2804] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:45.673000 27445 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:45.679704 27356 raft_consensus.cc:2804] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:45.680316 27356 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:45.680761 27356 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:09:45.732672 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:45.751878 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 148090us
I20260709 13:09:45.710934 27445 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:45.628880 27289 consensus_queue.cc:1048] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.130s
I20260709 13:09:45.757896 27145 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:45.765865 27145 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:45.790688 22992 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:45.804347 27165 raft_consensus.cc:2804] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:09:45.812188 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:45.818271 27177 consensus_meta.cc:287] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.571s	user 0.013s	sys 0.000s
I20260709 13:09:45.819962 27404 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:45.866158 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 75562.1us
I20260709 13:09:45.867560 22992 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:45.868831 27404 raft_consensus.cc:2749] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:45.812107 27165 raft_consensus.cc:697] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 12 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:45.875291 27165 consensus_queue.cc:237] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:45.840656 27241 consensus_meta.cc:287] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.598s	user 0.010s	sys 0.001s
I20260709 13:09:46.017613 27292 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:46.150537 27292 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:46.063987 27445 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:46.153560 27445 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:46.144371 27356 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:46.189050 27356 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:09:46.020833 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 153402us
W20260709 13:09:46.673513 23050 debug-util.cc:398] Leaking SignalData structure 0x7b0800193160 after lost signal to thread 22812
I20260709 13:09:46.806255 22992 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
W20260709 13:09:46.807123 23050 debug-util.cc:398] Leaking SignalData structure 0x7b08000b0700 after lost signal to thread 22904
W20260709 13:09:46.675767 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:46.087424 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:46.810423 23050 debug-util.cc:398] Leaking SignalData structure 0x7b08003ff6c0 after lost signal to thread 22979
W20260709 13:09:46.810775 23050 debug-util.cc:398] Leaking SignalData structure 0x7b08002a42c0 after lost signal to thread 22980
I20260709 13:09:46.748795 27301 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
W20260709 13:09:46.671147 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:46.812781 27301 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:46.834743 27287 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:46.830000 23625 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.335s	user 0.048s	sys 0.050s
I20260709 13:09:46.839165 22992 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:09:46.845023 23625 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.350s	user 0.049s	sys 0.051s
I20260709 13:09:46.847936 27407 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:46.849370 27407 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:46.848183 27287 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:46.857921 27407 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:09:46.893464 23050 debug-util.cc:398] Leaking SignalData structure 0x7b08003684a0 after lost signal to thread 23053
W20260709 13:09:46.894146 23050 debug-util.cc:398] Leaking SignalData structure 0x7b080031a320 after lost signal to thread 23054
I20260709 13:09:46.892529 27445 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:46.903450 23590 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.523s	user 0.010s	sys 0.047s
W20260709 13:09:46.934357 27077 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.315s	user 0.009s	sys 0.030s
W20260709 13:09:46.921901 23590 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.599s	user 0.011s	sys 0.047s
W20260709 13:09:46.912850 23589 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.885s	user 0.015s	sys 0.079s
W20260709 13:09:46.938254 27077 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.350s	user 0.009s	sys 0.031s
W20260709 13:09:46.938812 23589 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.911s	user 0.015s	sys 0.079s
W20260709 13:09:46.944212 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 1.62114e+06us
W20260709 13:09:46.981223 27438 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.142s	user 0.010s	sys 0.030s
W20260709 13:09:46.981877 27438 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.166s	user 0.010s	sys 0.030s
I20260709 13:09:46.984395 27287 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:46.984580 27454 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:46.975056 27445 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:46.985962 23625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 844) took 1535 ms. Trace:
I20260709 13:09:46.986094 27454 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:46.986673 23625 rpcz_store.cc:276] 0709 13:09:45.450596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:45.450702 (+   106us) service_pool.cc:224] Handling call
0709 13:09:45.494197 (+ 43495us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:45.494482 (+   285us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:45.494518 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:45.495038 (+   520us) log.cc:844] Serialized 26 byte log entry
0709 13:09:46.847889 (+1352851us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:46.847991 (+   102us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:46.848137 (+   146us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:46.985188 (+137051us) raft_consensus.cc:1685] finished
0709 13:09:46.985208 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:46.985804 (+   596us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":1350206,"threads_started":1,"wal-append.queue_time_us":1486260}
I20260709 13:09:46.988013 27445 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:46.990257 27454 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:46.991240 27165 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.922s	user 0.017s	sys 0.036s
I20260709 13:09:47.000025 27445 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:46.983863 27425 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.295s	user 0.001s	sys 0.045s
W20260709 13:09:47.000512 27393 log.cc:927] Time spent T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 2.497s	user 0.020s	sys 0.018s
W20260709 13:09:47.001499 27392 log.cc:927] Time spent T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 2.502s	user 0.005s	sys 0.029s
W20260709 13:09:47.006330 23050 debug-util.cc:398] Leaking SignalData structure 0x7b080036fbc0 after lost signal to thread 23649
W20260709 13:09:47.006695 23050 debug-util.cc:398] Leaking SignalData structure 0x7b0800397e00 after lost signal to thread 23650
I20260709 13:09:47.008258 23616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 889) took 2619 ms. Trace:
I20260709 13:09:47.009021 27473 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:47.008878 23616 rpcz_store.cc:276] 0709 13:09:44.388733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:44.388886 (+   153us) service_pool.cc:224] Handling call
0709 13:09:44.389304 (+   418us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:09:44.396647 (+  7343us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:09:44.396730 (+    83us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:44.397406 (+   676us) log.cc:844] Serialized 52 byte log entry
0709 13:09:44.444132 (+ 46726us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:09:44.444180 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:44.444241 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:46.999888 (+2555647us) spinlock_profiling.cc:243] Waited 35.2 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:09:47.007584 (+  7696us) raft_consensus.cc:1685] finished
0709 13:09:47.007605 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:47.008042 (+   437us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5521,"raft.queue_time_us":37686,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":779,"spinlock_wait_cycles":81019648,"thread_start_us":46379,"threads_started":1,"wal-append.queue_time_us":90542}
I20260709 13:09:47.002249 27471 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:47.012020 27471 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:47.004859 27301 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:47.010236 27473 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:47.001154 27165 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.962s	user 0.018s	sys 0.037s
I20260709 13:09:47.017004 27471 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:46.996464 27057 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:47.015911 27425 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.353s	user 0.001s	sys 0.046s
I20260709 13:09:47.005085 27287 raft_consensus.cc:2749] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:47.030378 27471 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:47.030735 23616 raft_consensus.cc:1275] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:09:47.030619 27472 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:47.030187 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:47.031270 27471 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:47.031594 27472 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:47.032331 23623 raft_consensus.cc:2393] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 13: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:47.036696 27472 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:47.037307 27444 consensus_queue.cc:1059] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:09:47.037868 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:47.038550 27444 raft_consensus.cc:3055] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 13:09:47.038815 22876 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.592s	user 0.033s	sys 0.053s
I20260709 13:09:47.038956 27444 raft_consensus.cc:740] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
W20260709 13:09:47.039163 22876 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.592s	user 0.033s	sys 0.053s
I20260709 13:09:47.040431 22876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 2073) took 1699 ms. Trace:
I20260709 13:09:47.040830 22876 rpcz_store.cc:276] 0709 13:09:45.341377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:45.398326 (+ 56949us) service_pool.cc:224] Handling call
0709 13:09:45.398631 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:45.418652 (+ 20021us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 13:09:45.418679 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:45.446787 (+ 28108us) log.cc:844] Serialized 26 byte log entry
0709 13:09:47.039772 (+1592985us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:09:47.039849 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:47.039971 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:47.040022 (+    51us) raft_consensus.cc:1685] finished
0709 13:09:47.040035 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:47.040260 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":21060,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":40004,"thread_start_us":1592406,"threads_started":1,"wal-append.queue_time_us":1527911}
I20260709 13:09:47.039981 27057 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:47.043407 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:47.044296 22876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35360 (request call id 2085) took 1480 ms. Trace:
I20260709 13:09:47.044639 22876 rpcz_store.cc:276] 0709 13:09:45.563433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:47.042823 (+1479390us) service_pool.cc:224] Handling call
0709 13:09:47.044196 (+  1373us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:09:47.046064 23590 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:47.044876 27471 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:47.046425 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:47.047266 27472 raft_consensus.cc:2804] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:09:47.047359 22876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35370 (request call id 2586) took 1517 ms. Trace:
I20260709 13:09:47.048269 27472 raft_consensus.cc:697] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 14 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:47.048091 22876 rpcz_store.cc:276] 0709 13:09:45.529973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:47.046068 (+1516095us) service_pool.cc:224] Handling call
0709 13:09:47.047293 (+  1225us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:09:47.049317 27458 log.cc:927] Time spent T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.214s	user 0.000s	sys 0.003s
I20260709 13:09:47.049865 27057 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:47.050653 27473 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:47.040575 27444 consensus_queue.cc:260] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:47.073547 27444 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:09:47.083289 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 244261us
I20260709 13:09:47.050799 27472 consensus_queue.cc:237] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:47.028630 27445 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:47.101416 27407 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:47.102035 27445 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:47.104285 27407 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:09:47.112352 23620 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60044 (request call id 824) took 1790 ms (client timeout 1734 ms). Trace:
W20260709 13:09:47.112871 23620 rpcz_store.cc:269] 0709 13:09:45.321562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:45.324487 (+  2925us) service_pool.cc:224] Handling call
0709 13:09:47.111975 (+1787488us) spinlock_profiling.cc:243] Waited 1.66 s on lock 0x7b54001d4a54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:47.112144 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3812764544}
I20260709 13:09:47.118321 26802 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:47.119097 26802 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:47.123287 26802 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:47.123919 27057 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:47.127130 27471 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:47.127874 27471 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:47.129081 27057 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:47.130311 27416 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:47.131456 27416 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:47.133806 27057 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:47.120096 26794 consensus_queue.cc:260] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:47.136096 27416 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:47.136121 27354 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:47.116421 27455 log.cc:927] Time spent T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.174s	user 0.001s	sys 0.001s
I20260709 13:09:47.137493 27440 raft_consensus.cc:3055] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:09:47.138098 27440 raft_consensus.cc:740] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:47.138494 26794 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:47.139707 27440 consensus_queue.cc:260] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:47.147197 27440 raft_consensus.cc:3060] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:47.137521 27471 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:47.151391 22874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 2049) took 2783 ms. Trace:
I20260709 13:09:47.151786 22874 rpcz_store.cc:276] 0709 13:09:44.367965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:44.372971 (+  5006us) service_pool.cc:224] Handling call
0709 13:09:44.389754 (+ 16783us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:44.392402 (+  2648us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:44.392434 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:44.399858 (+  7424us) log.cc:844] Serialized 26 byte log entry
0709 13:09:44.440796 (+ 40938us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:09:44.440867 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:44.440958 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:46.999409 (+2558451us) spinlock_profiling.cc:243] Waited 31.9 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:09:47.150875 (+151466us) raft_consensus.cc:1685] finished
0709 13:09:47.150895 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:47.151191 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2330,"raft.queue_time_us":2781,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":6086,"spinlock_wait_cycles":73437568,"thread_start_us":40350,"threads_started":1,"wal-append.queue_time_us":97370}
I20260709 13:09:47.153090 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:47.153563 27471 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:47.153834 23620 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
I20260709 13:09:47.154455 27471 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:47.157699 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:47.158587 23623 raft_consensus.cc:2468] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:09:47.160046 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:47.161019 23625 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 13.
W20260709 13:09:47.162214 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:47.162617 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:47.163931 27462 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:47.164654 27462 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:47.167321 27464 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:47.165030 27471 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:47.168152 27464 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:47.171489 22876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2587) took 1615 ms. Trace:
I20260709 13:09:47.171906 22876 rpcz_store.cc:276] 0709 13:09:45.556557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:47.049652 (+1493095us) service_pool.cc:224] Handling call
0709 13:09:47.050465 (+   813us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:47.051043 (+   578us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:47.051072 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:47.052504 (+  1432us) log.cc:844] Serialized 492 byte log entry
0709 13:09:47.165686 (+113182us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:09:47.165741 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:47.165830 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:47.171245 (+  5415us) raft_consensus.cc:1685] finished
0709 13:09:47.171264 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:47.171407 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":112648,"threads_started":1,"wal-append.queue_time_us":102440}
I20260709 13:09:47.175083 27462 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:47.175735 27462 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:47.176525 27462 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:47.178395 27165 consensus_queue.cc:1048] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:09:47.179183 27462 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:47.179832 27464 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:47.191166 22993 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:47.193908 27440 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:47.194164 23023 raft_consensus.cc:1275] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:09:47.194446 27440 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:47.194799 23024 raft_consensus.cc:1275] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:09:47.195000 27440 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:47.196007 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:47.198029 27471 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:47.198858 27471 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:47.202553 27471 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:47.205145 22993 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:47.206112 22993 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:09:47.206650 22951 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.863s	user 0.000s	sys 0.041s
I20260709 13:09:47.206959 27478 raft_consensus.cc:2804] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:09:47.207127 22951 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.864s	user 0.000s	sys 0.041s
I20260709 13:09:47.207438 27478 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:47.207923 27478 raft_consensus.cc:3060] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:47.212428 27165 raft_consensus.cc:2804] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:47.212903 27165 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:47.213240 27165 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:09:47.214515 22879 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.681s	user 0.013s	sys 0.064s
W20260709 13:09:47.214987 22879 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.682s	user 0.013s	sys 0.064s
I20260709 13:09:47.219272 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:47.220207 23616 raft_consensus.cc:2410] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:09:47.220711 23620 raft_consensus.cc:1240] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 14 LEADER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14. Ops: []
I20260709 13:09:47.225283 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:47.204020 27440 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:47.227272 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:47.227945 23029 raft_consensus.cc:2468] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 9.
I20260709 13:09:47.227880 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:47.229058 27440 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:47.229414 23024 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 10.
I20260709 13:09:47.229952 27478 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:47.230660 27407 consensus_queue.cc:1059] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:09:47.231487 27407 consensus_queue.cc:1243] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer 5b346767a89745fa82928a5dac9cc0b2 log is divergent from this leader: its last log entry 14.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
W20260709 13:09:47.230916 23026 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32794 (request call id 731) took 2118 ms (client timeout 2054 ms). Trace:
I20260709 13:09:47.236994 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:47.216463 22879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 2084) took 1740 ms. Trace:
W20260709 13:09:47.232595 23026 rpcz_store.cc:269] 0709 13:09:45.112589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:45.112944 (+   355us) service_pool.cc:224] Handling call
0709 13:09:47.230402 (+2117458us) spinlock_profiling.cc:243] Waited 512 ms on lock 0x7b5400134a54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:47.230642 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1178252032}
I20260709 13:09:47.249588 27445 raft_consensus.cc:3055] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:09:47.250365 27445 raft_consensus.cc:740] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:09:47.252777 22843 connection.cc:256] RPC call timeout handler was delayed by -2.6480105709999862s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1867s for a response
W20260709 13:09:47.254063 22843 connection.cc:256] RPC call timeout handler was delayed by -2.6250935770000297s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21930000000000002s for a response
I20260709 13:09:47.239831 22879 rpcz_store.cc:276] 0709 13:09:45.475727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:45.480901 (+  5174us) service_pool.cc:224] Handling call
0709 13:09:45.481312 (+   411us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:45.483588 (+  2276us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:45.483629 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:45.505920 (+ 22291us) log.cc:844] Serialized 52 byte log entry
0709 13:09:47.215829 (+1709909us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:09:47.215922 (+    93us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:47.216052 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:47.216108 (+    56us) raft_consensus.cc:1685] finished
0709 13:09:47.216124 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:47.216387 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9297,"raft.queue_time_us":10937,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":19553,"thread_start_us":1682398,"threads_started":1,"wal-append.queue_time_us":1479504}
I20260709 13:09:47.251816 27445 consensus_queue.cc:260] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:47.265415 23623 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
I20260709 13:09:47.271430 22993 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:47.272389 27407 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:09:47.272979 27407 raft_consensus.cc:697] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 12 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:47.274235 27407 consensus_queue.cc:237] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:47.272396 27482 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:47.278412 27482 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:47.243769 27165 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:47.285178 27462 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:47.367249 27482 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:47.366655 27471 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:47.372835 27478 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:47.373886 27471 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:47.385154 27398 log.cc:927] Time spent T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 2.862s	user 0.010s	sys 0.031s
I20260709 13:09:47.390637 27471 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:47.392453 27472 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:47.426381 22873 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.904s	user 0.012s	sys 0.019s
I20260709 13:09:47.426143 27472 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:47.400105 22951 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2729) took 2668 ms. Trace:
I20260709 13:09:47.396849 23622 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60044 (request call id 820) took 2955 ms. Trace:
I20260709 13:09:47.427001 22951 rpcz_store.cc:276] 0709 13:09:44.731426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:44.767982 (+ 36556us) service_pool.cc:224] Handling call
0709 13:09:44.768434 (+   452us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:44.779729 (+ 11295us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:09:44.780011 (+   282us) log.cc:844] Serialized 14 byte log entry
0709 13:09:45.303206 (+523195us) spinlock_profiling.cc:243] Waited 2.68 ms on lock 0x7b18004372c0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132c72c0a 00007f613c3b2219 00007f6132cfaa40 00007f6132cf54e4 00007f6132cf5455 00007f6135ddd060 00007f6135def716 00007f6135e00287 00007f6135e00208
0709 13:09:45.346856 (+ 43650us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b18004372c0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cf8522 00007f6132cfa7c1 00007f6132cfa779 00007f6132cfa709 00007f6132cfa6d1 00007f6132cf99cd 000000000038f354 000000000038f199 00007f6132cdb83a
0709 13:09:47.207805 (+1860949us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:47.231933 (+ 24128us) log.cc:844] Serialized 52 byte log entry
0709 13:09:47.270135 (+ 38202us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:09:47.270167 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:47.320937 (+ 50770us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:47.399591 (+ 78654us) raft_consensus.cc:1685] finished
0709 13:09:47.399623 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:47.399906 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27686,"prepare.queue_time_us":746,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":9349,"raft.queue_time_us":529756,"raft.run_cpu_time_us":287,"raft.run_wall_time_us":16160,"spinlock_wait_cycles":36324992,"thread_start_us":1920113,"threads_started":3,"wal-append.queue_time_us":169356,"wal-append.run_cpu_time_us":1690,"wal-append.run_wall_time_us":1001708}
W20260709 13:09:47.410259 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:47.429646 22872 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.091s	user 0.021s	sys 0.023s
W20260709 13:09:47.429986 22872 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.092s	user 0.021s	sys 0.023s
I20260709 13:09:47.412564 22879 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
mode: GRACEFUL
new_leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:40244
I20260709 13:09:47.430867 22872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2582) took 2213 ms. Trace:
I20260709 13:09:47.431057 22879 raft_consensus.cc:606] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Received request to transfer leadership to TS aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:47.431499 22879 raft_consensus.cc:614] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:09:47.432930 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:09:47.434118 22844 connection.cc:256] RPC call timeout handler was delayed by -1.7453122819999862s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17090000000000002s for a response
I20260709 13:09:47.414253 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:47.419147 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:47.424953 22874 raft_consensus.cc:1240] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:47.435796 22876 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
I20260709 13:09:47.437124 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:47.437331 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:47.438091 22876 raft_consensus.cc:2410] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:09:47.438496 22874 raft_consensus.cc:2393] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 12: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:47.438948 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:47.439704 22876 raft_consensus.cc:2393] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 12: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:47.439554 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:47.426694 22873 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.904s	user 0.013s	sys 0.019s
I20260709 13:09:47.434535 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:47.440651 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:47.443390 22876 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
I20260709 13:09:47.444403 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:47.445226 22876 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
I20260709 13:09:47.446331 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:47.447314 22876 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 14.
I20260709 13:09:47.428900 23622 rpcz_store.cc:276] 0709 13:09:44.441512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:44.441991 (+   479us) service_pool.cc:224] Handling call
0709 13:09:44.448466 (+  6475us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:44.451781 (+  3315us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:44.451808 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:44.483672 (+ 31864us) log.cc:844] Serialized 26 byte log entry
0709 13:09:44.516967 (+ 33295us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:09:44.517002 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:44.517063 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:47.380135 (+2863072us) spinlock_profiling.cc:243] Waited 6.48 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:09:47.396368 (+ 16233us) raft_consensus.cc:1685] finished
0709 13:09:47.396388 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:47.396648 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2049,"raft.queue_time_us":2223,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":25804,"spinlock_wait_cycles":14901120,"thread_start_us":19395,"threads_started":1,"wal-append.queue_time_us":31321}
W20260709 13:09:47.435015 22844 connection.cc:256] RPC call timeout handler was delayed by -1.2549964529999897s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1585s for a response
I20260709 13:09:47.401446 27445 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:09:47.452601 23020 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33420 (request call id 2016) took 2551 ms (client timeout 1817 ms). Trace:
W20260709 13:09:47.453092 23020 rpcz_store.cc:269] 0709 13:09:44.901564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:44.903768 (+  2204us) service_pool.cc:224] Handling call
0709 13:09:47.452188 (+2548420us) spinlock_profiling.cc:243] Waited 66.2 ms on lock 0x7b540018a954. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:47.452420 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":152344320}
I20260709 13:09:47.456638 27275 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
W20260709 13:09:47.456924 22871 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.184s	user 0.036s	sys 0.040s
W20260709 13:09:47.457299 22871 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.217s	user 0.038s	sys 0.040s
I20260709 13:09:47.457406 27275 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:47.431233 22872 rpcz_store.cc:276] 0709 13:09:45.217086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:45.317462 (+100376us) service_pool.cc:224] Handling call
0709 13:09:45.317993 (+   531us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:45.337245 (+ 19252us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:09:45.337304 (+    59us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:45.338014 (+   710us) log.cc:844] Serialized 26 byte log entry
0709 13:09:47.115259 (+1777245us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:09:47.430341 (+315082us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:09:47.430412 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:47.430521 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:47.430575 (+    54us) raft_consensus.cc:1685] finished
0709 13:09:47.430586 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:47.430795 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":18305,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":126,"spinlock_wait_cycles":29317888,"thread_start_us":2091924,"threads_started":1,"wal-append.queue_time_us":1535754}
W20260709 13:09:47.458385 22919 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.144s	user 0.008s	sys 0.015s
I20260709 13:09:47.458822 26794 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:47.441468 22879 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:09:47.435873 22877 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:47.459226 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:47.459448 26794 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:47.459887 22872 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:47.459960 27275 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:47.461575 26794 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:47.441623 27472 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:09:47.463485 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:47.440454 22874 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 13.
I20260709 13:09:47.466761 27269 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:47.467941 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:47.466133 22875 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.943s	user 0.002s	sys 0.017s
W20260709 13:09:47.471359 22875 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.949s	user 0.003s	sys 0.018s
W20260709 13:09:47.469741 23028 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33420 (request call id 2017) took 2565 ms (client timeout 1612 ms). Trace:
I20260709 13:09:47.477591 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:47.474611 23028 rpcz_store.cc:269] 0709 13:09:44.903141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:44.903269 (+   128us) service_pool.cc:224] Handling call
0709 13:09:47.468520 (+2565251us) spinlock_profiling.cc:243] Waited 511 ms on lock 0x7b54001f8bd4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:47.468778 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1174156672}
I20260709 13:09:47.477324 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:09:47.481730 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:47.484853 22919 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.170s	user 0.009s	sys 0.016s
I20260709 13:09:47.488035 27482 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:47.488850 27482 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:47.491298 27482 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:47.491375 22918 connection.cc:256] RPC call timeout handler was delayed by -2.5372663829999738s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2162s for a response
W20260709 13:09:47.492961 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 2.17835e+06us
W20260709 13:09:47.494520 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 128264us
I20260709 13:09:47.495772 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:09:47.493161 27485 log.cc:927] Time spent T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 13:09:47.473104 27462 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:47.505143 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:47.505923 22879 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 12.
I20260709 13:09:47.506973 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:47.507746 22879 raft_consensus.cc:2393] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 8: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:47.493187 27482 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:47.485100 27269 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:47.511010 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:47.511915 27269 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:47.512199 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:47.512904 23621 raft_consensus.cc:3060] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:09:47.513082 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:47.513684 23622 raft_consensus.cc:2468] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 7.
W20260709 13:09:47.514199 22948 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.001s	user 0.024s	sys 0.044s
W20260709 13:09:47.514559 22948 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.001s	user 0.024s	sys 0.044s
I20260709 13:09:47.509321 27482 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:47.500571 22877 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 13.
I20260709 13:09:47.508538 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:47.525285 22872 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 15.
I20260709 13:09:47.498571 27165 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:47.528961 23590 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:47.527462 22876 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11.
I20260709 13:09:47.530082 27077 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:47.530588 27425 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:47.543870 27425 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:47.545255 22874 raft_consensus.cc:1275] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 13:09:47.535530 27472 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:47.530239 27275 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:47.530977 27269 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:47.550614 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:47.552918 27484 ts_tablet_manager.cc:933] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2: Initiating tablet copy from peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:47.553279 22948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2746) took 2056 ms. Trace:
I20260709 13:09:47.553489 27456 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:47.553699 22948 rpcz_store.cc:276] 0709 13:09:45.496929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:45.498936 (+  2007us) service_pool.cc:224] Handling call
0709 13:09:45.499305 (+   369us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:45.499704 (+   399us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:45.499739 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:45.500232 (+   493us) log.cc:844] Serialized 26 byte log entry
0709 13:09:47.514990 (+2014758us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:09:47.538658 (+ 23668us) log.cc:844] Serialized 14 byte log entry
0709 13:09:47.550543 (+ 11885us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:47.552966 (+  2423us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:47.553043 (+    77us) raft_consensus.cc:1685] finished
0709 13:09:47.553056 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:47.553205 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4302,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":10007,"thread_start_us":2024186,"threads_started":2,"wal-append.queue_time_us":1467589}
I20260709 13:09:47.554565 27456 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:47.551031 27489 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:47.556628 27489 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:47.559185 27489 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:47.559715 27489 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:47.564819 23590 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:47.566192 27434 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:09:47.566764 27434 raft_consensus.cc:697] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:47.566690 27269 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:47.543833 27077 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:47.568222 22879 raft_consensus.cc:2393] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 14: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:47.568053 27434 consensus_queue.cc:237] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:47.560168 27489 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:47.549985 27472 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:47.588985 23621 raft_consensus.cc:2468] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
I20260709 13:09:47.579459 27472 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:47.593448 22874 raft_consensus.cc:1275] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:09:47.594000 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 97009.6us
I20260709 13:09:47.584132 27438 consensus_queue.cc:1048] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 13:09:47.633124 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:47.644982 27456 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:47.645433 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 301573us
W20260709 13:09:47.646530 22918 connection.cc:256] RPC call timeout handler was delayed by -1.8981053399999155s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1554s for a response
W20260709 13:09:47.646962 22918 connection.cc:256] RPC call timeout handler was delayed by -1.8270053399999142s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1633s for a response
W20260709 13:09:47.647274 22918 connection.cc:256] RPC call timeout handler was delayed by -1.702450970999962s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15400000000000003s for a response
W20260709 13:09:47.647564 22918 connection.cc:256] RPC call timeout handler was delayed by -1.6611239379999461s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18500000000000003s for a response
W20260709 13:09:47.647938 22918 connection.cc:256] RPC call timeout handler was delayed by -1.6108901979998791s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1674s for a response
W20260709 13:09:47.648240 22918 connection.cc:256] RPC call timeout handler was delayed by -1.5423239379998677s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19820000000000002s for a response
I20260709 13:09:47.648314 27165 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:47.649262 27165 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:47.650192 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:47.650803 22874 raft_consensus.cc:2468] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 8.
W20260709 13:09:47.650192 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:47.652117 27456 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:47.652778 27456 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:47.654579 22875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2584) took 2147 ms. Trace:
I20260709 13:09:47.655090 22875 rpcz_store.cc:276] 0709 13:09:45.507205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:45.512334 (+  5129us) service_pool.cc:224] Handling call
0709 13:09:45.512760 (+   426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:47.512756 (+1999996us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:47.512814 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:47.513498 (+   684us) log.cc:844] Serialized 26 byte log entry
0709 13:09:47.539265 (+ 25767us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:09:47.539316 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:47.541961 (+  2645us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:47.654302 (+112341us) raft_consensus.cc:1685] finished
0709 13:09:47.654326 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:47.654514 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10848,"raft.queue_time_us":21313,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":5220,"thread_start_us":1974323,"threads_started":2,"wal-append.queue_time_us":78614}
W20260709 13:09:47.656947 22994 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:47.658154 22875 raft_consensus.cc:3055] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:09:47.658562 22875 raft_consensus.cc:740] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:47.659514 22875 consensus_queue.cc:260] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:47.659922 22873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2585) took 2152 ms. Trace:
I20260709 13:09:47.660362 22873 rpcz_store.cc:276] 0709 13:09:45.507875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:45.511488 (+  3613us) service_pool.cc:224] Handling call
0709 13:09:45.511774 (+   286us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:47.441126 (+1929352us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:47.441192 (+    66us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:47.441923 (+   731us) log.cc:844] Serialized 26 byte log entry
0709 13:09:47.520954 (+ 79031us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:09:47.521001 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:47.521085 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:47.659688 (+138603us) raft_consensus.cc:1685] finished
0709 13:09:47.659711 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:47.659865 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6452,"raft.queue_time_us":12266,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":9681,"thread_start_us":1997020,"threads_started":2,"wal-append.queue_time_us":135997}
I20260709 13:09:47.660576 22875 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:47.663005 27503 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:47.663889 27503 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:47.665144 27499 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:47.665783 27499 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:47.655062 27456 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:47.668643 27462 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:47.669425 27462 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:47.648574 22918 connection.cc:256] RPC call timeout handler was delayed by -1.5215698829999837s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1998s for a response
W20260709 13:09:47.670501 22918 connection.cc:256] RPC call timeout handler was delayed by -1.3907726619999039s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19610000000000002s for a response
W20260709 13:09:47.670810 22918 connection.cc:256] RPC call timeout handler was delayed by -1.2809234459999743s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2102s for a response
I20260709 13:09:47.670894 27482 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:47.671118 22918 connection.cc:256] RPC call timeout handler was delayed by -1.2094880920000151s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21600000000000003s for a response
W20260709 13:09:47.671443 22918 connection.cc:256] RPC call timeout handler was delayed by -1.0841509709999855s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2227s for a response
I20260709 13:09:47.671530 27482 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:47.670132 27462 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:47.672176 27482 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:09:47.674187 27462 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:47.675091 27489 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:47.676337 22918 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:47.679566 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:47.680507 22952 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 10.
I20260709 13:09:47.681087 27456 raft_consensus.cc:2804] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:47.681679 27456 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:47.682056 27456 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:47.682255 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:47.683475 22952 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 9.
I20260709 13:09:47.675143 27499 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:47.685731 22954 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2740) took 2258 ms. Trace:
I20260709 13:09:47.686137 22954 rpcz_store.cc:276] 0709 13:09:45.426863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:45.426990 (+   127us) service_pool.cc:224] Handling call
0709 13:09:45.449973 (+ 22983us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:45.555318 (+105345us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:47.685288 (+2129970us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b5400283f54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:47.685559 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":240636288}
I20260709 13:09:47.689185 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:47.689656 22871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2578) took 2632 ms. Trace:
I20260709 13:09:47.690362 22947 raft_consensus.cc:2393] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 14: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:47.596462 27275 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:47.662027 27077 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:47.695634 22875 raft_consensus.cc:1275] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 13:09:47.696126 22875 pending_rounds.cc:85] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 8
I20260709 13:09:47.699717 22874 raft_consensus.cc:1240] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14. Ops: []
I20260709 13:09:47.666729 27503 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:47.702142 22951 raft_consensus.cc:1275] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:09:47.703382 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:09:47.703979 22919 connection.cc:256] RPC call timeout handler was delayed by -1.9321900639999967s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1633s for a response
I20260709 13:09:47.704193 23026 raft_consensus.cc:2368] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 11. Current term is 12.
I20260709 13:09:47.690104 22871 rpcz_store.cc:276] 0709 13:09:45.056779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:45.203626 (+146847us) service_pool.cc:224] Handling call
0709 13:09:45.203990 (+   364us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:09:45.205629 (+  1639us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:45.205659 (+    30us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:09:45.213425 (+  7766us) log.cc:844] Serialized 78 byte log entry
0709 13:09:45.377170 (+163745us) spinlock_profiling.cc:243] Waited 5.77 ms on lock 0x7b4800002958. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613ad0eaa1 00007f613ad05b37 00007f613ad059ca 00007f613adfd26f 00007f613adfcf37 00007f613ae1155c 00007f613ae11508 00007f613ae11488 00007f613ae11443 00007f613ae106ec 00007f6139d3aa96
0709 13:09:47.457760 (+2080590us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:09:47.498190 (+ 40430us) log.cc:844] Serialized 14 byte log entry
0709 13:09:47.590067 (+ 91877us) log.cc:844] Serialized 14 byte log entry
0709 13:09:47.617336 (+ 27269us) log.cc:844] Serialized 14 byte log entry
0709 13:09:47.642468 (+ 25132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:47.642675 (+   207us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:47.642720 (+    45us) raft_consensus.cc:1685] finished
0709 13:09:47.643075 (+   355us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:47.689600 (+ 46525us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2515,"prepare.queue_time_us":100231,"prepare.run_cpu_time_us":117,"prepare.run_wall_time_us":115,"raft.queue_time_us":223432,"raft.run_cpu_time_us":1995,"raft.run_wall_time_us":205019,"spinlock_wait_cycles":16139264,"thread_start_us":2386018,"threads_started":5,"wal-append.queue_time_us":1597406}
I20260709 13:09:47.696470 22875 pending_rounds.cc:107] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:09:47.706557 22875 raft_consensus.cc:2889] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:47.674191 27482 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:47.707998 26794 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:47.707860 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:47.709441 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:47.714854 23020 raft_consensus.cc:2393] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 15: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:47.714682 23028 raft_consensus.cc:1275] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:09:47.720551 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:47.722528 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:47.722970 22871 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:09:47.711709 27077 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:47.726914 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:47.728051 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:47.728879 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:47.718812 27165 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:47.712939 27484 tablet_copy_client.cc:323] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.63.3:42499
I20260709 13:09:47.735411 27275 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:47.740231 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:47.741626 23024 raft_consensus.cc:2393] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 13: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
W20260709 13:09:47.742514 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 62376.1us
I20260709 13:09:47.745608 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:47.747393 22872 raft_consensus.cc:1240] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 13. Current term is 14. Ops: []
I20260709 13:09:47.747632 23020 raft_consensus.cc:2393] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 13: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:47.750262 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:47.729043 26794 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:47.754129 23028 raft_consensus.cc:1275] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 13:09:47.754089 22946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 2002) took 4807 ms. Trace:
I20260709 13:09:47.740463 27425 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:47.728121 27077 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:47.741676 27269 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:47.769989 26794 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:47.758994 22946 rpcz_store.cc:276] 0709 13:09:42.946405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:42.982220 (+ 35815us) service_pool.cc:224] Handling call
0709 13:09:42.982652 (+   432us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:43.082955 (+100303us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:43.083000 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:43.083469 (+   469us) log.cc:844] Serialized 26 byte log entry
0709 13:09:45.260520 (+2177051us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:09:45.260554 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:47.751288 (+2490734us) spinlock_profiling.cc:243] Waited 99.9 ms on lock 0x7b5400284954. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:47.753533 (+  2245us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:47.753614 (+    81us) raft_consensus.cc:1685] finished
0709 13:09:47.753626 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:47.753886 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":98,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":108,"spinlock_wait_cycles":229656320,"thread_start_us":2176755,"threads_started":1,"wal-append.queue_time_us":41215,"wal-append.run_cpu_time_us":1587,"wal-append.run_wall_time_us":1003353}
I20260709 13:09:47.775341 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:47.776328 22953 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 13.
I20260709 13:09:47.765601 27077 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:47.781605 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:47.782546 27244 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:47.783571 27246 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:47.783272 27244 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:47.784206 27246 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:47.785940 27244 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:47.786654 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:47.783159 23028 raft_consensus.cc:1275] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 13:09:47.787607 23028 pending_rounds.cc:85] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75: Aborting all ops after (but not including) 6
I20260709 13:09:47.789204 23028 pending_rounds.cc:107] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 13:09:47.789491 23028 raft_consensus.cc:2889] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:47.791968 27171 raft_consensus.cc:1064] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
I20260709 13:09:47.792299 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:47.795930 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:47.796737 23024 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 13.
I20260709 13:09:47.797829 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:47.797173 27249 raft_consensus.cc:2955] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Committing config change with OpId 7.6: config changed from index -1 to 6, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:09:47.799371 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:47.790473 22993 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:47.801783 23023 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 12.
I20260709 13:09:47.802575 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:47.803337 23021 raft_consensus.cc:2393] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 9: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:47.807605 23020 raft_consensus.cc:2393] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 11: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:47.807476 27246 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:47.784260 27058 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:47.809933 22918 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:47.811447 27077 raft_consensus.cc:2804] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:47.811859 27077 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:47.812242 27077 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:47.794677 27171 consensus_queue.cc:237] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 7, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:47.816964 27250 raft_consensus.cc:2804] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:47.817487 27250 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:47.817862 27250 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:09:47.821424 23588 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:47.822237 23588 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:47.825558 22878 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53724 (request call id 715) took 3619 ms (client timeout 3271 ms). Trace:
W20260709 13:09:47.826020 22878 rpcz_store.cc:269] 0709 13:09:44.206388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:47.825167 (+3618779us) service_pool.cc:211] Skipping call since client already timed out
0709 13:09:47.825331 (+   164us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 13:09:47.828411 27022 raft_consensus.cc:3055] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:09:47.828867 27022 raft_consensus.cc:740] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:47.829663 23588 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:47.830546 22993 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:47.801345 22994 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:47.836359 22994 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:47.839748 27179 raft_consensus.cc:2804] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:47.840197 27179 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:47.840569 27179 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:47.840916 27445 raft_consensus.cc:2804] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:47.841306 27445 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:47.841638 27445 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:47.845593 22918 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:47.835795 27264 raft_consensus.cc:2804] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:47.837126 27246 raft_consensus.cc:2804] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:09:47.845029 27022 consensus_queue.cc:260] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:47.853097 27022 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:47.849555 27264 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:47.857316 27264 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:47.861609 22918 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:47.865664 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:47.867293 27425 raft_consensus.cc:2804] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:47.867831 27425 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:47.868227 27425 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:47.865463 27246 raft_consensus.cc:697] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 12 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:47.873867 23588 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:47.865885 22920 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:47.897996 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:47.904533 22918 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:47.905019 22919 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:47.905663 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 54089.1us
I20260709 13:09:47.906548 22994 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:47.909298 22948 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 10.
W20260709 13:09:47.913740 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 62900.8us
I20260709 13:09:47.918751 27442 raft_consensus.cc:2804] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:47.919234 27442 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:47.919622 27442 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:47.921728 23588 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:47.913354 27447 raft_consensus.cc:2749] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:47.923130 27445 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:47.866313 27472 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:47.861912 27250 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:47.866564 27077 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:47.904336 27246 consensus_queue.cc:237] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:47.924762 26794 raft_consensus.cc:2749] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:09:47.931478 22843 connection.cc:256] RPC call timeout handler was delayed by -1.7432593990000669s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1547s for a response
W20260709 13:09:47.931942 22843 connection.cc:256] RPC call timeout handler was delayed by -1.7432593990000669s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1547s for a response
W20260709 13:09:47.932363 22843 connection.cc:256] RPC call timeout handler was delayed by -1.6730593990000671s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1625s for a response
W20260709 13:09:47.932760 22843 connection.cc:256] RPC call timeout handler was delayed by -1.6380546950000507s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1549s for a response
W20260709 13:09:47.933135 22843 connection.cc:256] RPC call timeout handler was delayed by -1.5836794680000139s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16090000000000002s for a response
W20260709 13:09:47.933521 22843 connection.cc:256] RPC call timeout handler was delayed by -1.5785011830000713s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16090000000000002s for a response
W20260709 13:09:47.933863 22843 connection.cc:256] RPC call timeout handler was delayed by -1.5556758910000781s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17090000000000002s for a response
W20260709 13:09:47.934254 22843 connection.cc:256] RPC call timeout handler was delayed by -1.5098906690000149s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17800000000000002s for a response
W20260709 13:09:47.934628 22843 connection.cc:256] RPC call timeout handler was delayed by -1.5010520929999984s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15080000000000002s for a response
W20260709 13:09:47.935015 22843 connection.cc:256] RPC call timeout handler was delayed by -1.5006632839999838s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15080000000000002s for a response
I20260709 13:09:47.912063 27508 raft_consensus.cc:2804] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 13:09:47.935639 22843 connection.cc:256] RPC call timeout handler was delayed by -1.3738758910000115s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19110000000000002s for a response
W20260709 13:09:47.936937 22843 connection.cc:256] RPC call timeout handler was delayed by -1.3738758910000115s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19110000000000002s for a response
W20260709 13:09:47.937554 22843 connection.cc:256] RPC call timeout handler was delayed by -1.3569046479999542s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1898s for a response
W20260709 13:09:47.938131 22843 connection.cc:256] RPC call timeout handler was delayed by -1.3351468199999772s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1587s for a response
W20260709 13:09:47.938508 22843 connection.cc:256] RPC call timeout handler was delayed by -1.3351468199999772s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1587s for a response
W20260709 13:09:47.938905 22843 connection.cc:256] RPC call timeout handler was delayed by -1.2939634050001132s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16080000000000003s for a response
W20260709 13:09:47.939262 22843 connection.cc:256] RPC call timeout handler was delayed by -1.2939634050001132s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16080000000000003s for a response
W20260709 13:09:47.939581 22843 connection.cc:256] RPC call timeout handler was delayed by -1.1636537130000306s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2118s for a response
W20260709 13:09:47.939960 22843 connection.cc:256] RPC call timeout handler was delayed by -1.1632208409999976s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2118s for a response
I20260709 13:09:47.926941 27275 raft_consensus.cc:2804] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:47.929981 27282 raft_consensus.cc:2749] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:47.946985 27032 raft_consensus.cc:2749] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:47.947785 27032 consensus_queue.cc:1059] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:09:47.948777 23588 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:47.948920 27163 consensus_queue.cc:1048] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:09:47.950630 27163 consensus_queue.cc:1048] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:09:47.952093 27436 raft_consensus.cc:2804] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:47.952558 27436 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:47.952953 27436 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:09:47.952986 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 48571.7us
I20260709 13:09:47.953958 22918 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
W20260709 13:09:47.965726 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 48225.2us
I20260709 13:09:47.967396 27249 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:32782 (request call id 14) took 3437 ms. Trace:
I20260709 13:09:47.968344 22948 raft_consensus.cc:1240] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:09:47.967886 27249 rpcz_store.cc:276] 0709 13:09:44.529815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:44.585579 (+ 55764us) service_pool.cc:224] Handling call
0709 13:09:44.723140 (+137561us) log.cc:844] Serialized 492 byte log entry
0709 13:09:47.967258 (+3244118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30385,"raft.queue_time_us":108471,"raft.run_cpu_time_us":2058,"raft.run_wall_time_us":3097,"spinlock_wait_cycles":64384,"thread_start_us":81639,"threads_started":1,"wal-append.queue_time_us":79042,"wal-append.run_cpu_time_us":2611,"wal-append.run_wall_time_us":1086813}
I20260709 13:09:47.969290 22953 raft_consensus.cc:1275] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:09:47.970141 22950 raft_consensus.cc:1275] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:09:47.943630 27275 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:47.972995 27275 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:47.973667 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:47.941607 27508 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:47.977440 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:47.978119 22874 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 12
W20260709 13:09:47.978209 27507 log.cc:927] Time spent T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
W20260709 13:09:47.980098 27510 log.cc:927] Time spent T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 13:09:47.981757 27250 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:47.981696 27512 consensus_queue.cc:1059] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 13:09:47.982450 22880 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.684s	user 0.007s	sys 0.012s
W20260709 13:09:47.982795 22880 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.685s	user 0.007s	sys 0.012s
I20260709 13:09:47.983141 27493 raft_consensus.cc:3055] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:09:47.983705 22880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2581) took 2769 ms. Trace:
I20260709 13:09:47.989820 22919 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:09:47.991050 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:47.983628 27493 raft_consensus.cc:740] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:47.993371 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:47.993870 27149 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:47.993634 27493 consensus_queue.cc:260] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:47.994393 27149 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:47.995316 27493 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:47.996801 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:47.996625 27264 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:47.997435 23618 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:47.997481 22878 raft_consensus.cc:1275] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:09:47.960273 27456 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:48.014551 23621 raft_consensus.cc:1275] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:09:47.984838 27508 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:47.988014 27445 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:48.018083 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:47.984731 27464 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:48.001549 27264 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:48.034104 22843 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:48.034529 27456 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:48.045524 23622 raft_consensus.cc:1275] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEARNER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:09:48.036927 27512 raft_consensus.cc:2804] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:09:48.049530 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:09:48.053171 27506 log.cc:927] Time spent T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.164s	user 0.001s	sys 0.001s
I20260709 13:09:48.025928 27504 consensus_queue.cc:1048] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 13:09:48.056350 27464 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:48.060429 22843 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:48.060839 27515 raft_consensus.cc:2804] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:09:48.061736 27515 raft_consensus.cc:697] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:48.062837 27504 raft_consensus.cc:2749] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:09:48.063262 27503 consensus_queue.cc:1048] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:09:48.062983 27515 consensus_queue.cc:237] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:48.064396 27512 raft_consensus.cc:697] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 11 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:48.064935 27514 raft_consensus.cc:3055] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:09:48.065392 27514 raft_consensus.cc:740] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:48.065531 27512 consensus_queue.cc:237] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:48.066358 27514 consensus_queue.cc:260] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:48.067691 27514 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:48.068933 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:48.069602 22950 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 10.
I20260709 13:09:48.070475 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:48.071055 22953 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
W20260709 13:09:48.072413 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:48.073285 22878 raft_consensus.cc:1275] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 13:09:48.079370 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:48.074569 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:48.079799 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:48.080062 22875 raft_consensus.cc:2410] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 14 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:09:48.075292 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:48.078596 22843 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:48.075958 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:48.034566 27472 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:48.080605 22874 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 12.
I20260709 13:09:48.081265 22876 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:09:48.077083 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:48.073865 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:48.083691 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:48.091907 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:48.093780 22843 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:48.094647 27503 raft_consensus.cc:2749] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:48.095420 22843 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:48.103186 27503 raft_consensus.cc:2749] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:48.105167 27502 raft_consensus.cc:2749] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 13:09:48.131042 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 177195us
W20260709 13:09:48.135006 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 169078us
I20260709 13:09:48.134934 27500 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:48.145076 22919 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:48.146427 27500 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:48.150431 27500 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:09:48.157655 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 62372.9us
I20260709 13:09:48.158987 22918 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:48.162201 27508 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:48.165760 22843 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:48.168380 22843 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:48.169354 27493 raft_consensus.cc:2749] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 13:09:48.169629 22843 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:48.170133 27493 raft_consensus.cc:2804] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:09:48.170673 27493 raft_consensus.cc:697] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 13 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:48.170814 22843 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:48.171398 27503 raft_consensus.cc:2804] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:09:48.171638 27493 consensus_queue.cc:237] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:48.180423 23618 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 14.
I20260709 13:09:48.189868 23026 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11.
I20260709 13:09:48.190816 27503 raft_consensus.cc:697] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 10 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:47.984159 22880 rpcz_store.cc:276] 0709 13:09:45.214795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:45.261859 (+ 47064us) service_pool.cc:224] Handling call
0709 13:09:45.262149 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:45.262331 (+   182us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:09:45.297763 (+ 35432us) log.cc:844] Serialized 14 byte log entry
0709 13:09:47.983226 (+2685463us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:47.983312 (+    86us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:09:47.983401 (+    89us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:47.983509 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:47.983650 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":144119,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":31,"raft.queue_time_us":3852,"thread_start_us":2685097,"threads_started":1,"wal-append.queue_time_us":1718017}
I20260709 13:09:48.191956 27503 consensus_queue.cc:237] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:48.192705 27508 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:48.199545 27500 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:48.200963 27462 raft_consensus.cc:2804] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:09:48.201110 27489 consensus_queue.cc:1048] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:09:48.201532 27462 raft_consensus.cc:697] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 13 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:48.200214 27179 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:48.202733 27462 consensus_queue.cc:237] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:48.204068 23623 raft_consensus.cc:2393] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 14: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:48.208734 22843 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:48.210014 22843 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:48.211314 27502 raft_consensus.cc:2804] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:48.225324 27503 consensus_queue.cc:1048] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:09:48.229382 27502 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:48.230527 27502 raft_consensus.cc:3060] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:48.231966 27179 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:48.219074 27504 raft_consensus.cc:2804] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:48.215620 27512 consensus_queue.cc:1048] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.023s
I20260709 13:09:48.238759 27504 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:48.252878 27504 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:48.261904 27500 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:48.265640 27502 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:48.270298 27502 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:48.271410 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 49954.3us
I20260709 13:09:48.272418 22843 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:48.273712 22843 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:48.274523 27512 raft_consensus.cc:2749] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:48.274976 22843 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:48.275293 27512 raft_consensus.cc:2804] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 13:09:48.275815 27512 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:48.276255 27462 raft_consensus.cc:2804] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:48.276261 27512 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:48.276793 27462 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:48.277303 27462 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:48.278609 22843 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:48.280273 27502 raft_consensus.cc:2804] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:48.281263 27502 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:48.282140 27502 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:48.286619 27482 consensus_queue.cc:1048] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:09:48.303161 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:48.307451 27504 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:48.309235 27482 consensus_queue.cc:1048] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:09:48.310392 27425 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:48.310647 27482 consensus_queue.cc:1048] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:09:48.315490 27442 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:48.314117 27149 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:48.318044 27462 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:48.320293 27462 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:48.328590 22764 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2959101199a44a4583a4ee456ecf044d with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 0a407504ed1741bf89a276558adf4c0f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 12 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:09:48.344833 27179 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.345634 27179 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:48.347716 27425 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:48.348484 27179 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:09:48.349686 23588 connection.cc:256] RPC call timeout handler was delayed by -1.4866183259999843s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1562s for a response
W20260709 13:09:48.350244 23588 connection.cc:256] RPC call timeout handler was delayed by -1.2994183260000227s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17700000000000002s for a response
W20260709 13:09:48.350633 23588 connection.cc:256] RPC call timeout handler was delayed by -1.1293183259999751s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19590000000000002s for a response
I20260709 13:09:48.350764 27163 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:48.351063 23588 connection.cc:256] RPC call timeout handler was delayed by -1.1050183259999358s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1986s for a response
W20260709 13:09:48.351455 23588 connection.cc:256] RPC call timeout handler was delayed by -1.0195183260000249s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2081s for a response
I20260709 13:09:48.351329 27163 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:48.352739 23588 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:48.353739 27163 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:48.354331 27163 consensus_queue.cc:1048] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:09:48.355676 27163 consensus_queue.cc:1048] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:09:48.330824 27512 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:09:48.356879 27472 raft_consensus.cc:2749] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:48.359699 27512 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:48.330178 27504 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:48.334790 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:48.360692 23618 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 7 FOLLOWER]: Advancing to term 9
I20260709 13:09:48.364753 22777 catalog_manager.cc:5697] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 4 to 9, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 9 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:09:48.334976 27077 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:48.335955 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:48.368762 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:48.369297 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:48.369633 23622 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
I20260709 13:09:48.369899 23625 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:48.370700 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:48.372033 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:48.372608 23622 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
I20260709 13:09:48.320523 27502 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:48.373610 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:48.374749 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:48.374949 27502 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:48.379469 23618 raft_consensus.cc:2468] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
I20260709 13:09:48.380095 22777 catalog_manager.cc:5697] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 9 to 11, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 11 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:09:48.389420 23625 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
I20260709 13:09:48.390300 27442 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:48.390407 27464 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:48.401131 23590 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:48.401321 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:48.417892 22955 raft_consensus.cc:2410] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
W20260709 13:09:48.404232 27511 log.cc:927] Time spent T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.374s	user 0.000s	sys 0.002s
I20260709 13:09:48.415694 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:48.431663 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:48.434273 27436 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:48.441613 27179 raft_consensus.cc:2804] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:09:48.442209 27179 raft_consensus.cc:697] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:48.443156 27179 consensus_queue.cc:237] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:48.445094 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:09:48.455724 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 54710.4us
I20260709 13:09:48.461668 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:48.462441 23620 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 13.
W20260709 13:09:48.463369 27518 log.cc:927] Time spent T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.430s	user 0.000s	sys 0.001s
I20260709 13:09:48.467231 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:48.467991 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:48.469594 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:48.471253 23620 raft_consensus.cc:1275] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
W20260709 13:09:48.478471 27022 consensus_meta.cc:287] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.625s	user 0.013s	sys 0.000s
I20260709 13:09:48.479387 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:48.481264 27436 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:48.483989 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:09:48.520102 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 436940us
W20260709 13:09:48.522915 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 384557us
I20260709 13:09:48.523942 22919 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:48.525873 22919 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:48.526808 27515 raft_consensus.cc:2749] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:09:48.529143 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 370235us
I20260709 13:09:48.532974 27500 raft_consensus.cc:2749] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:48.533236 27425 raft_consensus.cc:2804] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:48.532863 27514 consensus_queue.cc:1048] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:09:48.533691 27425 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:48.537279 22843 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:48.542204 27515 raft_consensus.cc:2804] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:48.542549 26794 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.551748 26794 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:48.551990 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:48.552281 23624 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.006s	sys 0.003s
I20260709 13:09:48.552886 22952 raft_consensus.cc:2393] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 13: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:48.553049 22953 raft_consensus.cc:1240] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:48.549772 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:48.553848 22955 raft_consensus.cc:2393] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 9: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:48.554064 22950 raft_consensus.cc:1275] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:09:48.554142 26794 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:48.555452 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:48.556382 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:48.557104 27275 consensus_meta.cc:287] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.583s	user 0.012s	sys 0.000s
I20260709 13:09:48.557183 22947 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 10.
I20260709 13:09:48.557634 27275 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:48.559685 27275 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:48.550590 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:48.555346 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:48.552068 27503 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:48.548207 27425 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:48.550837 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:48.561038 23624 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.006s	sys 0.003s
I20260709 13:09:48.560384 22948 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
I20260709 13:09:48.560982 22949 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
I20260709 13:09:48.565393 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:48.560920 27515 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:48.567596 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:48.562505 27503 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:48.567443 22951 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 10.
W20260709 13:09:48.579875 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 54083.8us
I20260709 13:09:48.583876 22919 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:48.572552 27515 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:48.566406 27520 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.572865 27275 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:48.580618 27503 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:48.594085 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:48.594832 22950 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:48.592545 27520 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:48.591265 22951 raft_consensus.cc:1240] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:48.598654 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:48.599336 22955 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:48.601013 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:48.601732 22954 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:48.603117 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:48.604313 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:48.606531 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:48.608103 22951 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 8.
I20260709 13:09:47.910964 22994 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:48.610670 22920 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:48.619016 27269 raft_consensus.cc:2804] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:48.619426 27269 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:48.619784 27269 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:48.621158 27464 consensus_queue.cc:1048] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:09:48.625401 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:48.625916 22877 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:48.626219 22843 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:48.627220 27502 raft_consensus.cc:2804] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:48.627703 27502 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:48.628131 27502 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:48.629037 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:48.630211 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:48.632421 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:48.633370 22947 raft_consensus.cc:1240] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:09:48.610809 27520 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:48.634397 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:48.611013 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:48.611339 27500 consensus_queue.cc:1048] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:09:48.635433 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:48.635967 22947 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:09:48.638616 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:48.639065 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:48.645612 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:48.646463 22951 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
I20260709 13:09:48.645119 22843 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:48.648911 22843 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:48.649905 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:48.650791 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:48.652287 27479 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:48.652837 27479 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:48.653856 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 187240us
I20260709 13:09:48.654907 27479 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:48.653514 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:48.658110 22872 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:48.658406 22877 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 9.
I20260709 13:09:48.669107 22955 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
I20260709 13:09:48.669382 22843 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:09:48.672639 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 761712us
I20260709 13:09:48.673303 22994 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:48.674579 22994 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:48.675729 22994 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:48.675485 27269 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:48.676993 27249 raft_consensus.cc:2804] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:48.677196 27149 raft_consensus.cc:2804] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:09:48.677449 27249 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:48.677726 27149 raft_consensus.cc:697] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:48.677836 27249 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:48.678987 23020 raft_consensus.cc:1240] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:48.678736 27149 consensus_queue.cc:237] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:48.680097 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:48.680838 23022 raft_consensus.cc:2368] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 8. Current term is 9.
I20260709 13:09:48.681255 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:48.681599 22993 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:48.682149 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:48.682042 23024 raft_consensus.cc:1275] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:09:48.682511 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:48.683296 23021 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 15.
I20260709 13:09:48.683948 27289 consensus_queue.cc:1048] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:09:48.684309 22876 raft_consensus.cc:1240] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:48.677012 27250 raft_consensus.cc:2804] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:48.685688 27250 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:48.685982 22873 raft_consensus.cc:1275] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:09:48.684118 27356 consensus_queue.cc:1048] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:09:48.665022 27464 raft_consensus.cc:2764] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:09:48.645069 27275 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:48.687865 22952 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:48.686079 27250 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:48.682571 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:48.687688 27462 raft_consensus.cc:2804] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:48.682391 27177 raft_consensus.cc:2804] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:48.691049 23020 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 13.
I20260709 13:09:48.692919 23623 raft_consensus.cc:1240] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 10. Current term is 12. Ops: []
I20260709 13:09:48.697713 27536 raft_consensus.cc:2804] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:48.700726 27536 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:48.701134 27536 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:48.680652 23039 tablet_copy_service.cc:140] P f3c799b1249e40119d2a5864231c6c75: Received BeginTabletCopySession request for tablet ac5cc94926a742e6a0c1340ec33c0ff0 from peer 5b346767a89745fa82928a5dac9cc0b2 ({username='slave'} at 127.0.0.1:32794)
I20260709 13:09:48.702476 23039 tablet_copy_service.cc:161] P f3c799b1249e40119d2a5864231c6c75: Beginning new tablet copy session on tablet ac5cc94926a742e6a0c1340ec33c0ff0 from peer 5b346767a89745fa82928a5dac9cc0b2 at {username='slave'} at 127.0.0.1:32794: session id = 5b346767a89745fa82928a5dac9cc0b2-ac5cc94926a742e6a0c1340ec33c0ff0
I20260709 13:09:48.698345 22843 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:48.706938 27482 raft_consensus.cc:2804] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:48.707727 22993 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:48.708117 22843 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:09:48.705245 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 122539us
I20260709 13:09:48.693089 27356 raft_consensus.cc:2804] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:09:48.692598 27462 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:48.692862 27177 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:48.714753 22875 raft_consensus.cc:1240] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:48.723464 22844 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:48.730414 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:48.691824 27275 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:48.716715 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:48.718813 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:48.737054 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:48.737150 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:48.690177 27515 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:48.707389 27482 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:48.749963 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:48.752651 23618 raft_consensus.cc:2368] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 10. Current term is 11.
I20260709 13:09:48.711750 27493 raft_consensus.cc:2804] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:09:48.751261 23619 raft_consensus.cc:1240] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 LEADER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 14. Current term is 15. Ops: []
I20260709 13:09:48.705871 27464 consensus_queue.cc:1059] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 13:09:48.721330 27462 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:48.761009 27537 consensus_queue.cc:1059] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:09:48.725347 27022 raft_consensus.cc:2804] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:09:48.727413 27177 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:48.782261 23620 raft_consensus.cc:1240] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 13. Current term is 14. Ops: []
I20260709 13:09:48.777259 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:48.777936 22954 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
I20260709 13:09:48.744411 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:48.756567 27179 consensus_queue.cc:1048] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 13:09:48.747877 22950 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
W20260709 13:09:48.789198 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 65844.1us
I20260709 13:09:48.790447 22844 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:48.732656 27356 raft_consensus.cc:697] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 14 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:48.770571 27482 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:48.792802 22946 raft_consensus.cc:1240] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 13:09:48.795238 23029 raft_consensus.cc:1275] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 12. (term mismatch)
I20260709 13:09:48.796988 23029 pending_rounds.cc:85] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75: Aborting all ops after (but not including) 11
I20260709 13:09:48.797235 23029 pending_rounds.cc:107] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 13:09:48.797541 23029 raft_consensus.cc:2889] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:48.798760 22844 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:48.798938 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:48.800161 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:48.800987 22872 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 13.
I20260709 13:09:48.792007 27022 raft_consensus.cc:697] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:48.787181 27508 raft_consensus.cc:2749] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:48.821367 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:48.823616 23029 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:48.835469 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:48.836330 23021 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:48.839217 27250 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:48.847818 22952 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 8.
I20260709 13:09:48.796115 27356 consensus_queue.cc:237] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:48.849267 27553 log.cc:927] Time spent T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
W20260709 13:09:48.865656 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 66962us
I20260709 13:09:48.867338 22844 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:48.791217 27249 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:48.802084 27489 raft_consensus.cc:2804] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:48.873054 23588 leader_election.cc:400] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 8. Current term is 9.
I20260709 13:09:48.873471 23588 leader_election.cc:403] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:09:48.874734 23028 raft_consensus.cc:1275] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:09:48.872323 27282 consensus_queue.cc:1059] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 13:09:48.877460 27163 raft_consensus.cc:2749] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 8. Current term is 9.
I20260709 13:09:48.878281 27282 raft_consensus.cc:3055] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:09:48.878719 27282 raft_consensus.cc:740] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:48.882948 23588 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:09:48.901476 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:48.902518 23588 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:09:48.907224 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 198299us
W20260709 13:09:48.911782 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:48.914727 23588 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:48.808108 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:48.922091 22950 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:09:48.801177 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:48.872717 27489 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:48.931829 22947 raft_consensus.cc:2468] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11.
I20260709 13:09:48.938474 27281 raft_consensus.cc:2804] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:48.832070 27503 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:48.946120 22954 raft_consensus.cc:3060] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:48.872804 27425 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:48.801669 27464 raft_consensus.cc:3055] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:09:48.908874 27269 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:48.944401 23021 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 11.
I20260709 13:09:48.781466 27502 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:48.938102 27489 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:48.934125 27472 raft_consensus.cc:2804] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:48.879714 27282 consensus_queue.cc:260] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:48.885604 27536 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:48.906383 27163 consensus_queue.cc:1048] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 13:09:48.984409 22877 raft_consensus.cc:1240] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:09:48.985836 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:48.944981 27281 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:48.990314 22953 raft_consensus.cc:1240] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:09:48.957216 27515 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:48.980034 27464 raft_consensus.cc:740] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:48.937166 27447 consensus_queue.cc:1048] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.051s
W20260709 13:09:49.007309 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 143326us
W20260709 13:09:48.996793 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 263731us
I20260709 13:09:49.000799 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:48.981252 27493 raft_consensus.cc:697] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 11 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:49.002948 22950 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 13.
I20260709 13:09:48.989912 27472 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:49.009996 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:49.001593 27281 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:49.000416 27282 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:49.003687 27537 raft_consensus.cc:2749] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:49.021546 27503 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:49.021703 23039 tablet_copy_source_session.cc:215] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:09:49.022225 27503 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:09:49.022701 27503 raft_consensus.cc:697] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 14 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:48.988155 27022 consensus_queue.cc:237] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:49.023643 27503 consensus_queue.cc:237] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:49.027954 27482 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:48.997617 27250 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:49.030467 27249 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:49.030646 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:49.031172 27464 consensus_queue.cc:260] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:49.006016 27557 raft_consensus.cc:2804] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:49.034723 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:49.035842 23029 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 14.
I20260709 13:09:49.035953 27425 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:49.020491 27177 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:49.013669 27472 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:49.037832 27502 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:49.055469 22777 catalog_manager.cc:5697] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 9 to 12, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 12 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:09:49.038453 27462 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:49.039250 27536 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:49.009953 27493 consensus_queue.cc:237] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:49.041164 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 126536us
I20260709 13:09:49.062353 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:49.062660 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:49.063239 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:49.041749 27158 raft_consensus.cc:2749] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:09:49.042634 22954 raft_consensus.cc:2468] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
I20260709 13:09:49.036022 23621 raft_consensus.cc:1275] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:09:49.064109 22946 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:09:49.046868 22918 leader_election.cc:336] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.530s (SENT)
I20260709 13:09:49.059836 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:49.063805 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:49.064896 22976 debug-util.cc:398] Leaking SignalData structure 0x7b080039dce0 after lost signal to thread 21759
I20260709 13:09:49.067415 22918 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:49.068068 22976 debug-util.cc:398] Leaking SignalData structure 0x7b08003e1be0 after lost signal to thread 22812
I20260709 13:09:49.053382 27557 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:49.058234 27464 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:49.072252 22871 raft_consensus.cc:1240] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:09:49.072319 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:49.073928 22954 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11.
I20260709 13:09:49.073915 22843 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:49.075440 22948 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
W20260709 13:09:49.074539 22976 debug-util.cc:398] Leaking SignalData structure 0x7b08002e87c0 after lost signal to thread 22904
W20260709 13:09:49.078223 22976 debug-util.cc:398] Leaking SignalData structure 0x7b08003d57c0 after lost signal to thread 22979
I20260709 13:09:49.078188 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:49.078693 22976 debug-util.cc:398] Leaking SignalData structure 0x7b08003bc960 after lost signal to thread 22980
I20260709 13:09:49.079466 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:49.079845 22976 debug-util.cc:398] Leaking SignalData structure 0x7b08003d9860 after lost signal to thread 23053
W20260709 13:09:49.080277 22976 debug-util.cc:398] Leaking SignalData structure 0x7b08003e8100 after lost signal to thread 23054
I20260709 13:09:49.081712 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:49.082563 22947 raft_consensus.cc:2410] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 15 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:09:49.080473 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:49.083307 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:49.084151 22948 raft_consensus.cc:2410] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 14 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:09:49.084527 22947 raft_consensus.cc:1240] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 8. Current term is 9. Ops: []
W20260709 13:09:49.074709 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 207222us
I20260709 13:09:49.088636 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:49.089460 22844 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:09:49.079770 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 63422.8us
I20260709 13:09:49.093012 22843 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:09:49.094794 22976 debug-util.cc:398] Leaking SignalData structure 0x7b080029c2a0 after lost signal to thread 23649
W20260709 13:09:49.095170 22976 debug-util.cc:398] Leaking SignalData structure 0x7b0800224e20 after lost signal to thread 23650
I20260709 13:09:49.105717 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:49.105767 22976 debug-util.cc:398] Leaking SignalData structure 0x7b08003fd600 after lost signal to thread 27459
I20260709 13:09:49.108588 22949 raft_consensus.cc:2393] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 13: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:49.108301 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:49.150044 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:49.070556 27249 consensus_queue.cc:1048] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:09:49.111263 22992 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:49.155251 22876 raft_consensus.cc:1240] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:49.111847 22947 raft_consensus.cc:1240] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 11. Current term is 12. Ops: []
I20260709 13:09:49.110476 27536 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:49.072932 27149 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:49.114035 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:49.163990 22953 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:49.112562 27479 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:49.086035 27177 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:49.114645 27356 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:49.115244 27407 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:49.115621 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:49.116489 22843 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:49.116600 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:49.104370 27502 raft_consensus.cc:2804] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:09:49.117215 27505 raft_consensus.cc:3055] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:09:49.117830 23020 raft_consensus.cc:1240] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:49.117908 22951 raft_consensus.cc:1240] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 13:09:49.119027 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:49.120018 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:49.185707 23024 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
I20260709 13:09:49.121390 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:49.186393 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:49.186973 22880 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:49.121677 22875 raft_consensus.cc:1240] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 7. Current term is 8. Ops: []
I20260709 13:09:49.121929 22844 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:49.122448 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:49.187687 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:49.129096 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:49.132838 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 87594.2us
I20260709 13:09:49.073298 27557 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:49.110163 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:49.162606 23621 raft_consensus.cc:1240] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:49.155066 27158 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:49.176831 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:49.176964 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 65786.6us
I20260709 13:09:49.177151 22871 raft_consensus.cc:1240] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:49.165582 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:49.177664 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:49.177806 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:49.178355 22877 raft_consensus.cc:1275] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:09:49.179153 22952 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:49.195705 22878 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:49.196412 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:49.192926 27022 raft_consensus.cc:2749] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:09:49.179337 23029 raft_consensus.cc:2376] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13. Re-sending same reply.
I20260709 13:09:49.091645 27250 consensus_queue.cc:1048] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 13:09:49.180764 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:49.201686 22879 raft_consensus.cc:2393] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 12: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:49.181077 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:49.183743 27563 raft_consensus.cc:2804] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:09:49.139077 27425 raft_consensus.cc:2749] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:49.128474 27537 consensus_queue.cc:1059] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
W20260709 13:09:49.189579 22918 leader_election.cc:336] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.587s (SENT)
W20260709 13:09:49.190237 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 73824.5us
I20260709 13:09:49.213670 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:49.214428 22950 raft_consensus.cc:2410] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:09:49.209338 23028 raft_consensus.cc:2468] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 9.
I20260709 13:09:49.191105 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:49.202316 27575 raft_consensus.cc:2749] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:49.188154 23027 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
W20260709 13:09:49.192102 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 70246.3us
I20260709 13:09:49.216449 27563 raft_consensus.cc:697] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 12 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:49.217151 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:49.178398 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:49.180634 27149 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:49.208235 22952 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
I20260709 13:09:49.221251 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:49.184491 27505 raft_consensus.cc:740] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:49.222185 22918 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:49.209096 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:49.216194 22992 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:49.193028 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:49.218354 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:49.219120 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:49.220534 23027 raft_consensus.cc:2368] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 14. Current term is 15.
I20260709 13:09:49.191635 27502 raft_consensus.cc:697] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 12 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:49.230993 23617 raft_consensus.cc:1275] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:09:49.229351 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:49.226231 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:49.233377 22949 raft_consensus.cc:2393] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 14: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:49.230988 23025 raft_consensus.cc:2368] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 12. Current term is 13.
I20260709 13:09:49.231052 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:49.234743 22950 raft_consensus.cc:2410] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:09:49.193763 27482 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:49.237557 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:09:49.239691 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:49.209550 27562 raft_consensus.cc:2804] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader election won for term 8
I20260709 13:09:49.252171 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:49.186646 27356 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:49.252911 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:49.260582 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:49.260519 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:49.252333 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:49.261627 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:49.268563 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:49.269912 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:49.270603 22950 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 10.
I20260709 13:09:49.257673 22920 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:49.265007 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:49.263317 23590 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:49.273427 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:49.273804 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:49.273660 23020 raft_consensus.cc:2393] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 14: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:49.274521 23029 raft_consensus.cc:2393] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 12: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:49.184553 27407 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:49.281832 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 65733.9us
I20260709 13:09:49.283721 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:49.285205 22843 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:49.299034 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:49.285823 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:49.297351 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:49.279377 27269 raft_consensus.cc:2804] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:09:49.306624 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:49.307353 27269 raft_consensus.cc:697] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:49.307659 27502 consensus_queue.cc:237] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:49.308240 27269 consensus_queue.cc:237] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:49.306571 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:49.213033 27479 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:49.310992 23022 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:49.311707 23027 raft_consensus.cc:3060] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:49.312422 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:49.301607 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:49.314561 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:49.261652 27537 raft_consensus.cc:3055] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:09:49.318210 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:49.318907 23020 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:49.323051 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:49.323628 22948 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:49.214592 27158 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:49.284332 27508 raft_consensus.cc:2749] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:09:49.276842 27482 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:49.324635 23023 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
I20260709 13:09:49.275486 27447 consensus_queue.cc:1059] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 13:09:49.324898 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:49.279843 27562 raft_consensus.cc:697] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 8 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:49.326848 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:49.299574 27454 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:09:49.331014 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 141467us
W20260709 13:09:49.347566 22918 leader_election.cc:336] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.588s (SENT)
I20260709 13:09:49.331387 23029 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:49.331777 22878 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 13.
I20260709 13:09:49.324716 22876 raft_consensus.cc:1275] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:09:49.335598 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:49.350040 22947 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:09:49.351657 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:49.352356 22949 raft_consensus.cc:2393] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 9: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:49.315639 27289 raft_consensus.cc:2804] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:09:49.251220 27281 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:49.344821 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:49.354915 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:49.355908 23028 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:09:49.356362 23028 raft_consensus.cc:606] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:49.356892 27271 consensus_queue.cc:1059] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 13:09:49.357928 27271 raft_consensus.cc:3055] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:09:49.358271 27271 raft_consensus.cc:740] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:49.359102 22919 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:49.310086 27179 consensus_queue.cc:1059] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:09:49.360085 22919 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:49.341468 23025 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 13.
I20260709 13:09:49.279757 27282 consensus_queue.cc:1059] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 13:09:49.346612 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:49.322319 27537 raft_consensus.cc:740] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:49.331532 27462 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:49.314680 27249 consensus_queue.cc:1048] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 13:09:49.374711 22919 leader_election.cc:400] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 election: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 10. Current term is 11.
I20260709 13:09:49.375141 22919 leader_election.cc:403] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 13:09:49.376647 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:49.325783 27136 consensus_queue.cc:1059] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.136s
I20260709 13:09:49.371974 22947 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
I20260709 13:09:49.370005 23029 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 13.
I20260709 13:09:49.392971 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:49.396870 23021 raft_consensus.cc:2393] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 11: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:49.396724 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:49.393386 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 45851.9us
I20260709 13:09:49.398116 22918 leader_election.cc:400] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 11. Current term is 12.
I20260709 13:09:49.398417 22918 leader_election.cc:403] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:09:49.401041 23024 raft_consensus.cc:1240] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 12. Current term is 13. Ops: [11.5-12.6]
I20260709 13:09:49.370728 27503 raft_consensus.cc:2804] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:09:49.370230 27289 raft_consensus.cc:697] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:49.404747 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:49.408020 23029 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 14.
I20260709 13:09:49.370137 27500 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:49.345427 27482 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:49.369027 27536 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:49.397135 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:49.416353 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:49.418596 23021 raft_consensus.cc:2368] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 10. Current term is 11.
I20260709 13:09:49.409385 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:49.409576 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:49.420051 22992 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:49.420805 22951 raft_consensus.cc:1275] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:09:49.420838 27246 consensus_queue.cc:1059] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:09:49.376832 27519 raft_consensus.cc:2749] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:49.422415 27246 raft_consensus.cc:3055] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:09:49.423046 27246 raft_consensus.cc:740] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:49.422375 22951 raft_consensus.cc:1240] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 11. Current term is 12. Ops: []
I20260709 13:09:49.425787 22951 raft_consensus.cc:1240] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:49.425984 22992 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:49.427019 27022 raft_consensus.cc:2749] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:49.427335 22951 raft_consensus.cc:1240] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:49.428820 22951 raft_consensus.cc:1240] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:49.428716 22992 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:09:49.416097 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 131001us
I20260709 13:09:49.401067 27179 raft_consensus.cc:3055] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:09:49.434341 22992 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:49.342818 27505 consensus_queue.cc:260] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:49.439862 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:49.377954 27508 consensus_queue.cc:1059] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:09:49.437114 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:49.444099 22878 raft_consensus.cc:2393] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 9: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:49.217455 27563 consensus_queue.cc:237] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:49.383134 27489 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:49.413170 27550 raft_consensus.cc:2749] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 11. Current term is 12.
I20260709 13:09:49.455211 22918 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:49.446185 22777 catalog_manager.cc:5697] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 5 to 10, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 10 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:09:49.401314 27275 consensus_queue.cc:1059] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 13:09:49.413175 27558 raft_consensus.cc:3055] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:09:49.414178 27503 raft_consensus.cc:697] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 11 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:49.465312 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:49.359104 27271 consensus_queue.cc:260] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:49.464882 22953 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15.
I20260709 13:09:49.464304 22946 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11.
I20260709 13:09:49.447201 22951 raft_consensus.cc:1275] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:09:49.443126 27264 raft_consensus.cc:2749] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:49.481665 23021 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:49.438650 27179 raft_consensus.cc:740] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:49.487255 22947 raft_consensus.cc:1240] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:49.434142 27022 raft_consensus.cc:2749] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:49.492973 22872 raft_consensus.cc:1240] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 14. Current term is 15. Ops: []
I20260709 13:09:49.452682 27504 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:49.502816 22875 raft_consensus.cc:1240] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:09:49.429563 27349 consensus_queue.cc:1048] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.193s
I20260709 13:09:49.438758 27536 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:49.483711 27508 consensus_queue.cc:1243] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer 5b346767a89745fa82928a5dac9cc0b2 log is divergent from this leader: its last log entry 15.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:09:49.482386 27558 raft_consensus.cc:740] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:49.522691 22946 raft_consensus.cc:1240] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 14. Current term is 15. Ops: []
W20260709 13:09:49.523550 27580 log.cc:927] Time spent T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
W20260709 13:09:49.523758 27574 log.cc:927] Time spent T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.230s	user 0.000s	sys 0.000s
W20260709 13:09:49.524428 27576 log.cc:927] Time spent T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.184s	user 0.000s	sys 0.001s
I20260709 13:09:49.560077 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:49.560942 27407 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:49.561362 27581 log.cc:927] Time spent T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:09:49.562613 27489 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:49.563233 27489 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:49.563383 27493 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:49.563766 27489 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:49.538554 27356 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:49.443559 27537 consensus_queue.cc:260] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:49.565544 27557 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:49.566522 27438 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:49.554167 27179 consensus_queue.cc:260] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:49.567421 27179 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:49.567107 27438 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:49.567934 27557 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:49.568958 22879 raft_consensus.cc:1240] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:49.554157 27503 consensus_queue.cc:237] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:49.569416 22948 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 11.
I20260709 13:09:49.569561 27482 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:49.570796 22948 raft_consensus.cc:1240] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:09:49.525760 27271 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:09:49.525291 27577 log.cc:927] Time spent T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:09:49.500574 27505 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:09:49.573719 27472 consensus_meta.cc:287] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.506s	user 0.009s	sys 0.000s
I20260709 13:09:49.573805 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:49.574290 27472 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:49.575932 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:49.522756 27572 log.cc:927] Time spent T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:09:49.531528 27575 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:49.576932 22875 raft_consensus.cc:2393] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 10: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:49.577253 27575 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:49.578774 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:49.579631 27575 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:49.580204 23026 raft_consensus.cc:1240] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 11. Current term is 12. Ops: []
I20260709 13:09:49.580320 27515 raft_consensus.cc:2749] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 10. Current term is 11.
I20260709 13:09:49.506067 27585 raft_consensus.cc:3055] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:09:49.581096 27585 raft_consensus.cc:740] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:49.581825 27479 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:49.400604 27562 consensus_queue.cc:237] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:09:49.582132 27585 consensus_queue.cc:260] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:49.583528 27585 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:49.550873 22951 raft_consensus.cc:1275] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:09:49.585109 27281 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:49.585783 22951 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:49.554165 27596 raft_consensus.cc:2749] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:09:49.554551 27158 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:49.555040 27289 consensus_queue.cc:237] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:49.565959 27537 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:49.572129 22948 raft_consensus.cc:1240] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 8. Current term is 9. Ops: []
W20260709 13:09:49.539106 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 123010us
I20260709 13:09:49.563975 27493 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:49.570711 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:49.502410 27149 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:49.588654 22843 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:49.591579 23020 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
I20260709 13:09:49.594234 23022 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
I20260709 13:09:49.597424 23027 raft_consensus.cc:2468] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 10.
I20260709 13:09:49.467727 27442 consensus_queue.cc:1059] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.065s
I20260709 13:09:49.613128 22877 raft_consensus.cc:1240] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 12 LEADER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 11. Current term is 12. Ops: []
I20260709 13:09:49.622406 23026 raft_consensus.cc:1240] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:49.550725 27508 raft_consensus.cc:3055] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:09:49.630357 23020 raft_consensus.cc:1240] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:49.576560 27472 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:49.631378 23022 raft_consensus.cc:1240] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 14 LEADER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 13. Current term is 14. Ops: []
W20260709 13:09:49.631631 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 67023.2us
I20260709 13:09:49.518128 27504 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:49.633024 22880 raft_consensus.cc:2468] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
I20260709 13:09:49.631881 22994 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:49.641129 22875 raft_consensus.cc:1240] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 14. Current term is 15. Ops: []
W20260709 13:09:49.641258 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 201359us
I20260709 13:09:49.643760 23021 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 14.
I20260709 13:09:49.644032 22992 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:09:49.636262 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:49.649746 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:49.651597 23026 raft_consensus.cc:1275] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:09:49.627207 27596 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:49.627014 27136 raft_consensus.cc:2764] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
W20260709 13:09:49.663458 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 224990us
I20260709 13:09:49.626978 27454 raft_consensus.cc:697] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 13 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:49.639119 27575 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:49.634883 27508 raft_consensus.cc:740] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:49.671763 27249 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:49.674540 22879 raft_consensus.cc:1275] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:09:49.675256 27264 raft_consensus.cc:2749] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:09:49.676481 22992 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:49.700147 22951 raft_consensus.cc:1275] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:09:49.708608 22992 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:49.546634 27246 consensus_queue.cc:260] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:49.670305 27596 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:49.663934 27558 consensus_queue.cc:260] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:49.690380 27504 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:49.712054 27246 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:49.714560 27596 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:49.715435 27407 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:49.690722 27438 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:49.692094 27489 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:49.705283 27599 consensus_queue.cc:1048] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:09:49.706086 22879 raft_consensus.cc:1275] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:09:49.708846 22947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 2050) took 1048 ms. Trace:
I20260709 13:09:49.710633 27594 consensus_queue.cc:1059] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.004s
I20260709 13:09:49.678478 22843 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:49.717861 22994 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:49.722244 22947 rpcz_store.cc:276] 0709 13:09:48.660726 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:49.404460 (+743734us) service_pool.cc:224] Handling call
0709 13:09:49.487093 (+ 82633us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:49.495060 (+  7967us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:49.708639 (+213579us) spinlock_profiling.cc:243] Waited 71.2 ms on lock 0x7b54003c7150. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:49.708789 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":163775744}
I20260709 13:09:49.733145 22953 raft_consensus.cc:1240] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 LEADER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 8. Current term is 9. Ops: []
W20260709 13:09:49.734448 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 56073.4us
I20260709 13:09:49.735970 22876 raft_consensus.cc:2410] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 12 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:09:49.737270 22843 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:49.761096 22918 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:49.763155 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 45380.7us
I20260709 13:09:49.767794 22994 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:49.669140 27588 consensus_queue.cc:1059] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.093s
I20260709 13:09:49.771548 22874 raft_consensus.cc:1240] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 12 LEADER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 11. Current term is 12. Ops: []
I20260709 13:09:49.772835 22843 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:49.769675 22994 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:49.777522 27595 raft_consensus.cc:3055] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:09:49.777833 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 64722.1us
I20260709 13:09:49.778376 27592 raft_consensus.cc:2749] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:09:49.778760 27269 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:09:49.784792 27464 consensus_meta.cc:287] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.696s	user 0.005s	sys 0.005s
I20260709 13:09:49.786381 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:49.787075 27579 log.cc:927] Time spent T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.420s	user 0.000s	sys 0.003s
I20260709 13:09:49.777971 27595 raft_consensus.cc:740] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:49.789276 27149 raft_consensus.cc:3055] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:09:49.789764 27149 raft_consensus.cc:740] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:49.789263 27595 consensus_queue.cc:260] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:49.791823 27595 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:09:49.792798 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:49.793852 27601 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:49.794371 27591 raft_consensus.cc:3055] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:09:49.758186 27504 raft_consensus.cc:2804] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:49.794942 27504 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:49.795444 27504 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:49.794729 27601 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:49.797745 22918 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:49.799821 22918 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:49.721629 27022 consensus_queue.cc:1048] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 13:09:49.800736 27275 consensus_queue.cc:1048] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:09:49.761677 27249 raft_consensus.cc:2749] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:49.802296 27275 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:09:49.802997 27275 raft_consensus.cc:2749] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:49.761989 27558 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:49.804361 23617 raft_consensus.cc:1240] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 13. Current term is 14. Ops: []
I20260709 13:09:49.804826 27163 consensus_queue.cc:1048] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:09:49.718969 27356 consensus_queue.cc:1059] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 13:09:49.805629 27588 raft_consensus.cc:3055] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:09:49.805845 27356 raft_consensus.cc:3055] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 13:09:49.806017 27588 raft_consensus.cc:740] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:49.806358 27356 raft_consensus.cc:740] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:49.806679 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:49.807523 23624 raft_consensus.cc:2393] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 14: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:49.806967 27588 consensus_queue.cc:260] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:49.809032 22952 raft_consensus.cc:1240] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 13:09:49.765776 27508 consensus_queue.cc:260] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:49.809985 27508 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:49.811677 27596 consensus_queue.cc:1059] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:09:49.812552 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:49.771808 27479 raft_consensus.cc:2749] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:49.812942 27596 raft_consensus.cc:3055] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:09:49.813416 27596 raft_consensus.cc:740] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:49.761433 27537 raft_consensus.cc:2804] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:49.814368 27537 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:49.815008 27537 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:49.814322 27596 consensus_queue.cc:260] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:49.815888 27596 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:49.789234 27493 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:49.793807 27454 consensus_queue.cc:237] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:49.817873 22952 raft_consensus.cc:1275] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:09:49.819308 22777 catalog_manager.cc:5697] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:49.821031 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:49.821403 22777 catalog_manager.cc:5697] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
W20260709 13:09:49.823449 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 50667.7us
I20260709 13:09:49.823175 22777 catalog_manager.cc:5697] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:49.824919 22843 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:49.790674 27149 consensus_queue.cc:260] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:49.827472 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 57872.7us
I20260709 13:09:49.826622 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:49.829177 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:49.823300 27575 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:49.808504 27588 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:49.835773 22843 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:49.836468 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:49.837167 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:49.839520 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:49.855093 22947 raft_consensus.cc:2393] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 11: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:49.835090 27505 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:09:49.842374 27275 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:49.825160 27591 raft_consensus.cc:740] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:49.832038 27503 raft_consensus.cc:2804] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:09:49.836869 27520 consensus_queue.cc:1059] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 13:09:49.834841 27489 consensus_queue.cc:1048] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 13:09:49.836265 27482 consensus_queue.cc:1048] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 13:09:49.840317 27594 consensus_queue.cc:1048] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.011s
W20260709 13:09:49.879732 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:49.856281 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:49.885694 22949 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:09:49.863137 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 63394.4us
W20260709 13:09:49.863070 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 153089us
I20260709 13:09:49.862501 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:49.832582 27519 consensus_queue.cc:1048] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:09:49.887362 22952 raft_consensus.cc:2468] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 12.
I20260709 13:09:49.835353 27022 raft_consensus.cc:2804] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:09:49.875340 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:49.891518 22948 raft_consensus.cc:2393] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 14: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:49.834873 27599 consensus_queue.cc:1059] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
W20260709 13:09:49.894526 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:49.894034 23588 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:49.851740 27356 consensus_queue.cc:260] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:09:49.898829 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:49.900942 22918 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:49.900563 27503 raft_consensus.cc:697] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 12 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:49.904958 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:49.906137 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:49.907585 23588 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:49.911499 27438 raft_consensus.cc:2749] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:49.913437 23588 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:49.914080 27601 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:49.873911 27136 consensus_queue.cc:1059] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 13:09:49.899508 27275 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:49.916869 27604 raft_consensus.cc:2749] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:49.917814 27275 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:49.918460 22951 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 10.
I20260709 13:09:49.919224 23588 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:49.920109 27604 raft_consensus.cc:2749] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:49.921506 23617 raft_consensus.cc:1275] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:09:49.922026 27599 raft_consensus.cc:3055] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:09:49.922417 27599 raft_consensus.cc:740] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:49.923265 27599 consensus_queue.cc:260] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:49.905138 27586 raft_consensus.cc:2749] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:09:49.925000 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:49.925606 22952 raft_consensus.cc:1275] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 11. (term mismatch)
W20260709 13:09:49.926036 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:49.926054 22952 pending_rounds.cc:85] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting all ops after (but not including) 10
I20260709 13:09:49.927544 22952 pending_rounds.cc:107] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting uncommitted NO_OP operation due to leader change: 13.11
I20260709 13:09:49.927819 22952 raft_consensus.cc:2889] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:49.905615 27503 consensus_queue.cc:237] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:49.931193 22843 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:49.932753 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:49.924501 27599 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:49.868340 27493 consensus_queue.cc:1059] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 13:09:49.981094 23622 raft_consensus.cc:1275] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:09:49.981323 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 62172.5us
I20260709 13:09:49.971724 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:49.960681 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:49.969966 23625 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 14.
I20260709 13:09:49.969113 22993 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:49.900880 27271 raft_consensus.cc:2749] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:49.871659 27505 raft_consensus.cc:697] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 15 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:49.982318 23588 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:49.991415 23588 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:09:49.992331 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 97674.5us
I20260709 13:09:49.981552 27583 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:49.993204 27583 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:49.993561 27583 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:49.993582 22952 raft_consensus.cc:1275] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:09:49.994817 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:49.995896 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:49.996400 22952 raft_consensus.cc:2376] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11. Re-sending same reply.
I20260709 13:09:49.952601 27591 consensus_queue.cc:260] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:49.998085 23588 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:49.999495 27136 raft_consensus.cc:2749] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:50.000631 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:09:50.037703 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:50.037843 22951 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:49.992682 27585 raft_consensus.cc:2749] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:09:50.005679 27578 log.cc:927] Time spent T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.607s	user 0.000s	sys 0.007s
I20260709 13:09:50.005038 22777 catalog_manager.cc:5697] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:50.006357 27502 raft_consensus.cc:3055] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:09:50.009143 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:50.071050 22948 raft_consensus.cc:2376] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12. Re-sending same reply.
W20260709 13:09:50.013350 22775 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33478 (request call id 277) took 43017 ms (client timeout 15000 ms). Trace:
I20260709 13:09:49.952384 27250 consensus_queue.cc:1048] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 13:09:50.014751 27505 consensus_queue.cc:237] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:50.015764 27550 raft_consensus.cc:3055] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:09:49.952406 27515 raft_consensus.cc:2764] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 13:09:50.017508 27269 raft_consensus.cc:2749] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:50.076449 27271 consensus_queue.cc:1059] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.010s
I20260709 13:09:50.018332 27149 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:50.083714 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:50.084100 23588 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:50.084830 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:50.085407 22948 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
I20260709 13:09:50.085788 27585 consensus_queue.cc:1059] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:09:50.086771 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:50.086225 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:50.088605 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:50.089161 22948 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:50.089377 22951 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 14.
I20260709 13:09:50.070899 27447 raft_consensus.cc:2804] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:09:50.017660 27566 consensus_queue.cc:1048] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.097s
I20260709 13:09:50.099349 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:49.971858 27179 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:49.895519 27512 consensus_queue.cc:1048] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 13:09:50.077419 27550 raft_consensus.cc:740] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:50.019652 27604 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:50.021328 27583 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:50.025722 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:50.129914 22993 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:50.027252 27594 raft_consensus.cc:3055] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:09:50.028206 22918 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:50.027050 27407 raft_consensus.cc:2749] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:09:50.039295 22843 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.649s (SENT) [suppressed 27 similar messages]
I20260709 13:09:50.071792 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:50.135330 22952 raft_consensus.cc:2376] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 14. Re-sending same reply.
I20260709 13:09:50.032248 27504 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:50.070715 27502 raft_consensus.cc:740] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:50.080360 22874 raft_consensus.cc:1275] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:09:50.017482 27537 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:49.944157 27289 consensus_queue.cc:1059] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.075s
I20260709 13:09:50.115221 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:50.077207 27349 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:50.116401 22777 catalog_manager.cc:5697] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 4 to 6, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 6 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:09:50.119225 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:50.165335 22951 raft_consensus.cc:2410] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 14 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 13:09:49.895273 27520 consensus_queue.cc:1048] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 13:09:50.067823 27462 consensus_queue.cc:1048] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 13:09:50.121712 22950 raft_consensus.cc:1275] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:09:50.086369 27590 consensus_queue.cc:1048] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 13:09:50.091951 27591 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:50.156764 27482 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
W20260709 13:09:50.115593 27282 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.717s	user 0.008s	sys 0.010s
I20260709 13:09:50.130632 22948 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
W20260709 13:09:50.133213 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 105085us
I20260709 13:09:50.122210 27512 consensus_queue.cc:1243] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer aed3f089ec8b49af9c8cf5cdee76e8e7 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:09:50.150108 22955 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2750) took 1533 ms. Trace:
I20260709 13:09:50.169231 27281 raft_consensus.cc:2804] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:50.151144 22876 raft_consensus.cc:2393] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 11: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:50.154656 22844 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:50.271234 22955 rpcz_store.cc:276] 0709 13:09:48.616722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:48.690047 (+ 73325us) service_pool.cc:224] Handling call
0709 13:09:48.705044 (+ 14997us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:49.122942 (+417898us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:49.122971 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:49.123438 (+   467us) log.cc:844] Serialized 26 byte log entry
0709 13:09:49.238992 (+115554us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:09:49.239049 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:49.239140 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:50.005085 (+765945us) spinlock_profiling.cc:243] Waited 3.83 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:09:50.136270 (+131185us) raft_consensus.cc:1685] finished
0709 13:09:50.136299 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:50.150041 (+ 13742us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":413732,"raft.queue_time_us":420501,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":80,"spinlock_wait_cycles":8816000,"thread_start_us":115187,"threads_started":1,"wal-append.queue_time_us":219506}
W20260709 13:09:50.160183 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:50.161571 23619 raft_consensus.cc:1275] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 13:09:50.162971 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 288507us
I20260709 13:09:50.280520 27022 raft_consensus.cc:697] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:50.141582 27454 raft_consensus.cc:3055] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:09:50.203547 22871 raft_consensus.cc:1240] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:50.131222 27447 raft_consensus.cc:697] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 11 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:50.134559 27472 raft_consensus.cc:3055] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:09:50.132634 27594 raft_consensus.cc:740] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:50.201974 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:50.123579 27179 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:50.168779 22947 raft_consensus.cc:1275] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
W20260709 13:09:50.240955 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:50.241293 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 87658.3us
I20260709 13:09:50.225996 27550 consensus_queue.cc:260] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:50.243055 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:50.149194 27508 raft_consensus.cc:2749] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:50.141686 27250 raft_consensus.cc:2749] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:50.240486 27586 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:50.266597 22879 raft_consensus.cc:1240] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:50.081579 27438 consensus_queue.cc:1048] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
W20260709 13:09:50.266827 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 227510us
I20260709 13:09:50.270187 22948 raft_consensus.cc:1275] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 13:09:50.279605 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 125065us
I20260709 13:09:50.156811 27515 consensus_queue.cc:1048] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.112s
I20260709 13:09:50.132140 27604 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:50.321187 22777 catalog_manager.cc:5697] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 7 to 8, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 8 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:50.275611 27482 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:50.292593 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:50.262457 27289 consensus_queue.cc:1243] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer 0a407504ed1741bf89a276558adf4c0f log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 13:09:50.271076 27282 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.877s	user 0.009s	sys 0.010s
I20260709 13:09:50.405827 27482 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:50.406061 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:50.407047 22955 raft_consensus.cc:1275] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:09:50.407411 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:50.407596 27271 consensus_queue.cc:1048] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:09:50.408015 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:50.302685 22918 leader_election.cc:400] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 14. Current term is 15.
W20260709 13:09:50.409168 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 106606us
I20260709 13:09:50.409921 22918 leader_election.cc:400] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 12. Current term is 13.
I20260709 13:09:50.411391 22918 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:50.411238 27594 consensus_queue.cc:260] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:50.249307 27349 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:50.412818 27630 raft_consensus.cc:2749] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:50.413359 22946 raft_consensus.cc:1275] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:09:50.414439 27349 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:50.339689 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:50.203701 27613 consensus_queue.cc:1048] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.161s
I20260709 13:09:50.340984 23025 raft_consensus.cc:1240] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 13. Current term is 14. Ops: []
I20260709 13:09:50.416911 23025 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "5b346767a89745fa82928a5dac9cc0b2"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:09:50.417420 23025 raft_consensus.cc:606] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Received request to transfer leadership to TS 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:09:50.072554 22775 rpcz_store.cc:269] 0709 13:09:07.031723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:07.042578 (+ 10855us) service_pool.cc:224] Handling call
0709 13:09:50.013085 (+42970507us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:09:37.845786 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:09:38.186184 (+340398us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:09:38.198401 (+ 12217us) write_op.cc:435] Acquired schema lock
0709 13:09:38.203652 (+  5251us) tablet.cc:662] Decoding operations
0709 13:09:38.578798 (+375146us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:09:38.625719 (+ 46921us) write_op.cc:641] Partition lock acquired for write op
0709 13:09:38.627326 (+  1607us) tablet.cc:685] Acquiring locks for 8 operations
0709 13:09:38.827255 (+199929us) tablet.cc:701] Row locks acquired
0709 13:09:38.881794 (+ 54539us) write_op.cc:260] PREPARE: finished
0709 13:09:38.934301 (+ 52507us) write_op.cc:270] Start()
0709 13:09:38.934454 (+   153us) write_op.cc:276] Timestamp: P: 1783602578910733 usec, L: 0
0709 13:09:38.934475 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:09:38.938116 (+  3641us) log.cc:844] Serialized 3181 byte log entry
0709 13:09:39.500648 (+562532us) op_driver.cc:464] REPLICATION: finished
0709 13:09:39.529493 (+ 28845us) spinlock_profiling.cc:243] Waited 3.59 ms on lock 0x7b1800114b20. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cf83f3 00007f6132cfa7c1 00007f6132cfa779 00007f6132cfa709 00007f6132cfa6d1 00007f6132cf99cd 000000000038f354 000000000038f199 00007f6132cdb83a
0709 13:09:39.529673 (+   180us) write_op.cc:301] APPLY: starting
0709 13:09:39.546584 (+ 16911us) tablet.cc:1367] starting BulkCheckPresence
0709 13:09:39.551112 (+  4528us) tablet.cc:1370] finished BulkCheckPresence
0709 13:09:39.551153 (+    41us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:09:39.609994 (+ 58841us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:09:39.610303 (+   309us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=16
0709 13:09:39.610363 (+    60us) write_op.cc:312] APPLY: finished
0709 13:09:39.610727 (+   364us) log.cc:844] Serialized 65 byte log entry
0709 13:09:39.610922 (+   195us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:39.611053 (+   131us) write_op.cc:454] Released schema lock
0709 13:09:39.611124 (+    71us) write_op.cc:341] FINISH: Updating metrics
0709 13:09:39.611552 (+   428us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:39.611588 (+    36us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3008343,"reported_tablets":46,"child_traces":[["op",{"apply.queue_time_us":17315,"apply.run_cpu_time_us":6064,"apply.run_wall_time_us":81851,"num_ops":8,"prepare.queue_time_us":502203,"prepare.run_cpu_time_us":57338,"prepare.run_wall_time_us":1273278,"raft.queue_time_us":271503,"raft.run_cpu_time_us":5917,"raft.run_wall_time_us":58580,"replication_time_us":545336,"spinlock_wait_cycles":8261504,"thread_start_us":623856,"threads_started":4,"wal-append.queue_time_us":186080,"wal-append.run_cpu_time_us":21292,"wal-append.run_wall_time_us":1487144}]]}
I20260709 13:09:50.282500 22919 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f
W20260709 13:09:50.419386 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 137103us
I20260709 13:09:50.419461 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:50.280874 27562 raft_consensus.cc:2749] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:09:50.343353 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:50.421792 23026 raft_consensus.cc:1275] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:09:50.353058 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:50.423156 27281 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:09:50.423465 27596 consensus_meta.cc:287] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.607s	user 0.006s	sys 0.002s
I20260709 13:09:50.423797 27281 raft_consensus.cc:3060] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:50.353926 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:50.353710 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:50.427055 27179 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:50.356645 23029 raft_consensus.cc:1240] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:50.361493 22874 raft_consensus.cc:1240] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:09:50.367053 22949 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 16.
I20260709 13:09:50.420552 22950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 744) took 1094 ms. Trace:
W20260709 13:09:50.369827 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:50.373116 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:50.374554 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:50.375481 23588 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:50.341697 27536 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:50.379815 22873 raft_consensus.cc:1275] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 13:09:50.280920 27502 consensus_queue.cc:260] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:50.349002 27454 raft_consensus.cc:740] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:50.350584 27583 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:50.390165 22764 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2959101199a44a4583a4ee456ecf044d with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 0a407504ed1741bf89a276558adf4c0f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 12 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
W20260709 13:09:50.488075 22764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 97950.5us
I20260709 13:09:50.392098 22948 raft_consensus.cc:1240] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:09:50.393333 22947 raft_consensus.cc:1275] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:09:50.357448 27472 raft_consensus.cc:740] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
W20260709 13:09:50.400760 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:50.402554 22777 catalog_manager.cc:5697] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 5 to 10, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 10 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:09:50.337560 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:50.412629 27594 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:09:50.502084 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 64493.8us
I20260709 13:09:50.502614 23028 raft_consensus.cc:1240] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 13:09:50.415828 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:50.353137 23022 raft_consensus.cc:1275] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:09:50.428004 22918 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:50.503966 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 78903.5us
I20260709 13:09:50.429153 22950 rpcz_store.cc:276] 0709 13:09:49.330754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:49.561131 (+230377us) service_pool.cc:224] Handling call
0709 13:09:49.561430 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:50.260683 (+699253us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:50.420353 (+159670us) spinlock_profiling.cc:243] Waited 560 ms on lock 0x7b5400379bd4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:50.420499 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1287787904}
W20260709 13:09:50.441237 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 65888.2us
W20260709 13:09:50.444566 27620 log.cc:927] Time spent T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
W20260709 13:09:50.446745 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:50.509596 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:09:50.486253 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:50.487172 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:50.491187 27614 log.cc:927] Time spent T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.220s	user 0.000s	sys 0.001s
I20260709 13:09:50.513214 22777 catalog_manager.cc:5697] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:50.505273 22918 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:50.510097 22950 raft_consensus.cc:1240] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 13. Current term is 14. Ops: [9.7-13.9]
I20260709 13:09:50.518667 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:50.514364 22844 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:50.522666 22918 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:50.489676 27504 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:50.502928 27588 consensus_meta.cc:287] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.639s	user 0.005s	sys 0.013s
I20260709 13:09:50.521409 23588 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:50.499706 27349 raft_consensus.cc:2804] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:09:50.504878 27596 raft_consensus.cc:2749] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:50.505030 27536 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:50.533717 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:50.513020 27537 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:50.505232 27598 raft_consensus.cc:2749] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:50.450991 27630 consensus_queue.cc:1059] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.080s
I20260709 13:09:50.545720 22777 catalog_manager.cc:5697] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 8 to 9, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 9 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:50.532833 27500 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:50.516574 27249 consensus_queue.cc:1059] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 13:09:50.515261 27583 consensus_queue.cc:1048] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 13:09:50.569334 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:50.541561 27271 consensus_queue.cc:1048] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 13:09:50.515980 27562 consensus_queue.cc:1059] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 13:09:50.569478 22777 catalog_manager.cc:5697] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 3 to 6, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 6 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:09:50.573047 23028 raft_consensus.cc:2393] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 14: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:50.537273 27590 raft_consensus.cc:2804] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:09:50.545744 23027 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 9.
I20260709 13:09:50.579664 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:50.582592 23023 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:09:50.589911 27622 log.cc:927] Time spent T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 13:09:50.591751 22872 raft_consensus.cc:1240] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 8. Current term is 9. Ops: []
W20260709 13:09:50.593073 27615 log.cc:927] Time spent T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.314s	user 0.001s	sys 0.000s
I20260709 13:09:50.592994 27562 raft_consensus.cc:3055] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:09:50.594054 27617 log.cc:927] Time spent T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 13:09:50.594312 27562 raft_consensus.cc:740] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:50.594429 27604 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:50.545848 27349 raft_consensus.cc:697] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 14 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:50.595286 27625 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:50.595175 27562 consensus_queue.cc:260] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:50.595615 27349 consensus_queue.cc:237] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:50.524842 27022 consensus_queue.cc:237] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:50.590009 27484 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac5cc94926a742e6a0c1340ec33c0ff0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:09:50.595194 22880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2634) took 1117 ms. Trace:
I20260709 13:09:50.567628 27563 raft_consensus.cc:2749] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:50.568763 27596 raft_consensus.cc:2749] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:50.394428 27586 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:50.660936 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 160196us
I20260709 13:09:50.662503 22880 rpcz_store.cc:276] 0709 13:09:49.477703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:49.734941 (+257238us) service_pool.cc:224] Handling call
0709 13:09:49.765109 (+ 30168us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:49.773929 (+  8820us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:49.773961 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:49.774416 (+   455us) log.cc:844] Serialized 26 byte log entry
0709 13:09:50.081142 (+306726us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:50.081179 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:50.081241 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:50.595006 (+513765us) raft_consensus.cc:1685] finished
0709 13:09:50.595026 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:50.595138 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11866,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":93,"thread_start_us":263755,"threads_started":1,"wal-append.queue_time_us":351828}
I20260709 13:09:50.595824 27625 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:50.598595 27635 log.cc:927] Time spent T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.209s	user 0.001s	sys 0.000s
I20260709 13:09:50.667881 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:50.686012 23023 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 16.
I20260709 13:09:50.581992 27583 consensus_queue.cc:1243] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer f3c799b1249e40119d2a5864231c6c75 log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
W20260709 13:09:50.597258 27634 log.cc:927] Time spent T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:09:50.546461 27504 consensus_queue.cc:1048] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:09:50.667155 22777 catalog_manager.cc:5697] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 8 to 9, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 9 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:50.672641 22843 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:50.543418 27536 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:50.596398 27599 consensus_meta.cc:287] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.605s	user 0.004s	sys 0.004s
I20260709 13:09:50.596370 27562 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:50.584218 27621 raft_consensus.cc:3055] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:09:50.593005 22872 raft_consensus.cc:1275] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:09:50.597957 27631 log.cc:927] Time spent T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
W20260709 13:09:50.598860 27616 log.cc:927] Time spent T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.212s	user 0.001s	sys 0.000s
W20260709 13:09:50.676626 27628 log.cc:927] Time spent T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.341s	user 0.001s	sys 0.000s
I20260709 13:09:50.703389 22871 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 8.
I20260709 13:09:50.597285 27590 raft_consensus.cc:697] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:50.707556 22843 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:50.708400 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:50.712064 23023 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 9.
I20260709 13:09:50.566495 27472 consensus_queue.cc:260] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:50.717201 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 194596us
I20260709 13:09:50.718958 22777 catalog_manager.cc:5697] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 5 to 8, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 8 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:50.516191 27447 consensus_queue.cc:237] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:50.720216 22918 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:50.715337 27632 log.cc:927] Time spent T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.320s	user 0.001s	sys 0.000s
I20260709 13:09:50.704921 27621 raft_consensus.cc:740] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:50.729854 22777 catalog_manager.cc:5697] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 4 to 10. New cstate: current_term: 10 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:09:50.726311 22918 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:50.733433 27637 log.cc:927] Time spent T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.202s	user 0.000s	sys 0.001s
W20260709 13:09:50.782069 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:50.735208 27536 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:50.737906 27625 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:50.738591 23024 raft_consensus.cc:2410] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:09:50.738994 27641 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:50.721407 27520 raft_consensus.cc:2749] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:50.715056 27537 raft_consensus.cc:2804] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:09:50.747104 22918 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
W20260709 13:09:50.795408 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:50.860569 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:50.980999 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 234032us
I20260709 13:09:50.745239 23023 raft_consensus.cc:1275] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 13:09:50.981788 23023 pending_rounds.cc:85] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75: Aborting all ops after (but not including) 9
I20260709 13:09:50.982048 23023 pending_rounds.cc:107] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 13:09:50.982278 23023 raft_consensus.cc:2889] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:50.982100 22918 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:50.728127 27599 raft_consensus.cc:2749] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:09:50.972429 22777 catalog_manager.cc:5697] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 5 to 8, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 8 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:50.996800 22777 catalog_manager.cc:5697] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 9 to 11, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 11 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:09:50.717576 27566 consensus_queue.cc:1059] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.227s
W20260709 13:09:50.942996 27616 log.cc:927] Time spent T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.223s	user 0.000s	sys 0.000s
W20260709 13:09:51.039345 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 331827us
W20260709 13:09:50.983650 27634 log.cc:927] Time spent T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.242s	user 0.000s	sys 0.001s
I20260709 13:09:50.755690 27596 raft_consensus.cc:2749] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:09:50.945216 27629 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:51.040140 22843 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:50.676329 27454 consensus_queue.cc:260] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:50.988960 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:51.051862 23023 raft_consensus.cc:1240] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:51.042982 27425 raft_consensus.cc:2804] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:51.065577 27425 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
W20260709 13:09:51.062989 22844 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.053s	sys 0.033s
I20260709 13:09:51.066025 27425 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:09:51.066367 22844 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.054s	sys 0.033s
W20260709 13:09:51.066653 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 533080us
I20260709 13:09:51.067342 22844 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:51.074476 22879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 2122) took 1035 ms. Trace:
I20260709 13:09:51.074924 22879 rpcz_store.cc:276] 0709 13:09:50.038825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:50.038933 (+   108us) service_pool.cc:224] Handling call
0709 13:09:50.075280 (+ 36347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:50.386140 (+310860us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:51.074245 (+688105us) spinlock_profiling.cc:243] Waited 191 ms on lock 0x7b54001bbf54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:51.074416 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":439387520}
I20260709 13:09:51.077004 27590 consensus_queue.cc:237] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:51.078307 27601 raft_consensus.cc:2749] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:09:51.061997 23624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 941) took 1141 ms. Trace:
W20260709 13:09:51.082813 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:51.089936 27425 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:51.090862 27588 raft_consensus.cc:2749] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:50.991266 27604 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:51.092298 27425 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:51.036717 27503 raft_consensus.cc:2749] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:51.099556 23623 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 944) took 1178 ms. Trace:
I20260709 13:09:51.038246 27630 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:51.081035 23624 rpcz_store.cc:276] 0709 13:09:49.920319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:49.923866 (+  3547us) service_pool.cc:224] Handling call
0709 13:09:49.924148 (+   282us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:49.924477 (+   329us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:49.924510 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:49.925089 (+   579us) log.cc:844] Serialized 26 byte log entry
0709 13:09:50.158464 (+233375us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:09:50.158517 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:50.158604 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:51.061748 (+903144us) raft_consensus.cc:1685] finished
0709 13:09:51.061771 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:51.061929 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":19311,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":17634,"thread_start_us":155715,"threads_started":1,"wal-append.queue_time_us":325690}
I20260709 13:09:51.114501 22951 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 2075) took 1078 ms. Trace:
I20260709 13:09:50.987242 27536 consensus_queue.cc:1048] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.098s
I20260709 13:09:51.044997 27599 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:51.045730 27537 raft_consensus.cc:697] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 10 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:51.115060 22951 rpcz_store.cc:276] 0709 13:09:50.036339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:50.265509 (+229170us) service_pool.cc:224] Handling call
0709 13:09:50.281240 (+ 15731us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:50.300549 (+ 19309us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:50.300579 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:50.301089 (+   510us) log.cc:844] Serialized 26 byte log entry
0709 13:09:50.323599 (+ 22510us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:09:50.323640 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:50.323711 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:51.114211 (+790500us) raft_consensus.cc:1685] finished
0709 13:09:51.114242 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:51.114424 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":89,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":116,"thread_start_us":22127,"threads_started":1,"wal-append.queue_time_us":88204}
I20260709 13:09:50.983964 27641 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:51.133708 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:51.135469 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 121094us
I20260709 13:09:51.145059 22946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 2074) took 1109 ms. Trace:
I20260709 13:09:51.145704 22946 rpcz_store.cc:276] 0709 13:09:50.036045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:50.239340 (+203295us) service_pool.cc:224] Handling call
0709 13:09:50.239694 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:50.413712 (+174018us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:51.144695 (+730983us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b54000aa0d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:51.144974 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":398724480}
I20260709 13:09:51.063709 22777 catalog_manager.cc:5697] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 6 to 10, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 10 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:51.103427 27586 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:51.187708 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:51.094153 27629 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:51.223362 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 183125us
I20260709 13:09:51.234673 22876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35360 (request call id 2118) took 1304 ms. Trace:
I20260709 13:09:51.234841 22874 raft_consensus.cc:1240] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:51.235574 22876 rpcz_store.cc:276] 0709 13:09:49.930661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:49.930846 (+   185us) service_pool.cc:224] Handling call
0709 13:09:51.234421 (+1303575us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b54001bb554. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:51.234593 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":383331584}
I20260709 13:09:51.236316 22777 catalog_manager.cc:5697] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:09:51.241125 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 104685us
I20260709 13:09:51.113051 27604 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:51.249639 22992 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:51.251143 22992 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:51.125236 27630 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:51.286388 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 43900.9us
I20260709 13:09:51.235931 27425 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:51.227722 27583 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:51.093794 27621 consensus_queue.cc:260] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:51.305876 27479 consensus_queue.cc:1059] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 13:09:51.308933 22843 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:51.227623 27601 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:51.136956 27599 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:51.320456 27489 raft_consensus.cc:2749] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:09:51.320830 27557 consensus_queue.cc:1048] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:09:51.321472 27566 consensus_queue.cc:1048] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:09:51.249748 27536 consensus_queue.cc:1059] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.085s
W20260709 13:09:51.339283 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 253278us
I20260709 13:09:51.310622 27289 raft_consensus.cc:2804] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:09:51.326090 27629 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:51.326473 27604 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:51.326934 27623 log.cc:927] Time spent T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.557s	user 0.001s	sys 0.000s
I20260709 13:09:51.328512 27641 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:09:51.308477 27644 log.cc:927] Time spent T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
W20260709 13:09:51.330575 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:51.332625 22777 catalog_manager.cc:5697] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:09:51.324321 27281 consensus_meta.cc:287] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.871s	user 0.006s	sys 0.002s
I20260709 13:09:51.341156 22844 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:51.357230 27479 raft_consensus.cc:2749] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:09:51.330243 27596 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:51.316324 27650 consensus_queue.cc:1048] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:09:51.372627 27281 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:51.372977 27596 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:51.374025 27503 consensus_queue.cc:1048] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:09:51.374967 27596 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:51.378149 27281 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:51.338825 27601 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:51.381035 22777 catalog_manager.cc:5697] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 7 to 9, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 9 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:09:51.383097 27601 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:51.383723 27591 consensus_meta.cc:287] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 1.113s	user 0.011s	sys 0.000s
I20260709 13:09:51.330001 27537 consensus_queue.cc:237] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:51.384477 22954 raft_consensus.cc:1275] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:09:51.378149 27599 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:51.317032 27639 consensus_queue.cc:1059] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:09:51.315114 27425 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:51.347848 27489 consensus_queue.cc:1059] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260709 13:09:51.388827 27425 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:51.342625 27646 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:51.389671 27646 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:51.390086 23025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:60044 (request call id 34) took 1021 ms. Trace:
I20260709 13:09:51.390465 23025 rpcz_store.cc:276] 0709 13:09:50.369061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:50.416801 (+ 47740us) service_pool.cc:224] Handling call
0709 13:09:50.492506 (+ 75705us) spinlock_profiling.cc:243] Waited 7.53 ms on lock 0x7b540003bc20. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135d62520 00007f6135ddf652 00007f613adad60a 00007f61343337d4 00007f6134333762 00007f6134333691 00007f613433360c 00007f61343328c2 00007f613412048c 00007f613411f8c6
0709 13:09:51.389872 (+897366us) spinlock_profiling.cc:243] Waited 15.9 ms on lock 0x7b4c00499500. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f61340b1a20 00007f61340c2a29 00007f6134088644 00007f61340aff2d 00007f61340afb93 00007f6135ddfc92 00007f6135ddf84a 00007f613adad60a 00007f61343337d4 00007f6134333762 00007f6134333691
0709 13:09:51.390000 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":53945600}
I20260709 13:09:51.391443 27289 raft_consensus.cc:697] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:09:51.345528 27643 log.cc:927] Time spent T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.407s	user 0.000s	sys 0.001s
I20260709 13:09:51.392753 27289 consensus_queue.cc:237] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:51.393555 22953 raft_consensus.cc:1275] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:09:51.391572 27646 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:51.395903 22875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53724 (request call id 755) took 1577 ms. Trace:
I20260709 13:09:51.401234 22875 rpcz_store.cc:276] 0709 13:09:49.818720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:49.822151 (+  3431us) service_pool.cc:224] Handling call
0709 13:09:49.822472 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:49.822800 (+   328us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:49.822834 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:49.823326 (+   492us) log.cc:844] Serialized 26 byte log entry
0709 13:09:50.120234 (+296908us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:09:50.120283 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:50.120352 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:51.395659 (+1275307us) raft_consensus.cc:1685] finished
0709 13:09:51.395687 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:51.395838 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":996,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":6541,"thread_start_us":296552,"threads_started":1,"wal-append.queue_time_us":560995}
W20260709 13:09:51.407450 27562 consensus_meta.cc:287] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.705s	user 0.009s	sys 0.000s
I20260709 13:09:51.408044 27596 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:51.408593 27596 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:51.409324 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 68258.5us
I20260709 13:09:51.410017 27505 raft_consensus.cc:2749] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:51.410593 27596 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:51.411118 27596 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:51.411530 27596 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:51.412819 27641 consensus_queue.cc:1048] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:09:51.413403 27641 consensus_queue.cc:1243] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer 0a407504ed1741bf89a276558adf4c0f log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:09:51.363008 27629 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:51.413978 27629 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:51.415915 27629 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:51.416056 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:51.417328 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:51.110121 23623 rpcz_store.cc:276] 0709 13:09:49.921372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:49.922313 (+   941us) service_pool.cc:224] Handling call
0709 13:09:49.922679 (+   366us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:49.923116 (+   437us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:09:50.122590 (+199474us) log.cc:844] Serialized 14 byte log entry
0709 13:09:51.071306 (+948716us) spinlock_profiling.cc:243] Waited 2.55 ms on lock 0x7b1800347da0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f613598d4bc 00007f6132cdb48b 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf7c02 00007f613597f20b 00007f6135984d23 00007f613598523a
0709 13:09:51.071499 (+   193us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:51.072114 (+   615us) log.cc:844] Serialized 52 byte log entry
0709 13:09:51.072265 (+   151us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:09:51.072302 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:51.072369 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:51.088938 (+ 16569us) raft_consensus.cc:1685] finished
0709 13:09:51.099323 (+ 10385us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:51.099489 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":87178,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":25,"raft.queue_time_us":26914,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":18560,"spinlock_wait_cycles":5854976,"thread_start_us":370530,"threads_started":2,"wal-append.queue_time_us":185995}
I20260709 13:09:51.418430 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:51.433425 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:51.433166 21756 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:09:51.426417 27585 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:51.435223 27425 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:51.437505 27583 raft_consensus.cc:2749] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:51.429713 27489 consensus_queue.cc:1243] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer aed3f089ec8b49af9c8cf5cdee76e8e7 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:09:51.433090 27271 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:51.439296 27562 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:51.429073 27505 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:51.426169 27650 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:51.451758 27596 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:51.486086 27271 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:51.504968 27505 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:51.482113 27585 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:51.433252 27163 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:51.437332 27591 consensus_queue.cc:1059] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.043s
I20260709 13:09:51.474435 27281 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:51.484694 27425 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:51.566699 27271 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:51.567296 27425 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:51.561635 27596 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:51.568078 27596 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:51.569269 27505 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:51.571308 21756 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:09:51.574429 27636 log.cc:927] Time spent T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 1.047s	user 0.007s	sys 0.002s
I20260709 13:09:51.484417 27557 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:51.552793 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:51.578312 22918 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.018s	sys 0.008s
W20260709 13:09:51.579118 22918 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.018s	sys 0.008s
W20260709 13:09:51.579747 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 597694us
I20260709 13:09:51.580008 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:51.581087 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:51.580565 22918 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:51.596419 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:51.608536 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:51.611580 22918 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:51.613009 22918 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:51.614400 22918 leader_election.cc:400] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 10. Current term is 11.
W20260709 13:09:51.616046 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 364930us
I20260709 13:09:51.570617 27596 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:51.506388 27562 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:51.510391 27650 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:51.622678 22992 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:51.625334 22992 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:51.610000 27271 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:51.616021 27591 raft_consensus.cc:2749] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:51.577589 27163 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:51.578964 27281 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:51.580981 27557 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:51.624431 27479 raft_consensus.cc:2804] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:51.618759 27656 raft_consensus.cc:2804] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:09:51.626662 22992 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
W20260709 13:09:51.657107 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:51.627990 27630 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:51.632135 27629 raft_consensus.cc:2749] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:09:51.725955 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 403661us
W20260709 13:09:51.631661 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:51.634507 27479 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:51.746951 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:51.747550 22953 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 8.
W20260709 13:09:51.697633 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 71060us
I20260709 13:09:51.740907 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:51.741504 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:51.744124 27536 consensus_queue.cc:1048] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:09:51.752139 23028 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:51.752948 27536 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:51.734320 22918 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:51.629545 27271 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:51.744359 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:51.774576 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:51.774027 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:51.720067 27022 raft_consensus.cc:2804] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:09:51.814949 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:51.817459 22992 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:09:51.634668 27590 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.035s	sys 0.038s
I20260709 13:09:51.814960 23025 raft_consensus.cc:2393] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 15: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:51.818174 23027 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:51.722851 27656 raft_consensus.cc:697] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:51.739552 27425 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:09:51.822227 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 88028us
I20260709 13:09:51.850143 22918 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:51.826030 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:51.851023 23029 raft_consensus.cc:1275] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 13:09:51.851697 23023 raft_consensus.cc:2393] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 9: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:51.825213 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:51.852914 23021 raft_consensus.cc:2410] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 14 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:09:51.748255 27479 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:09:51.819690 27590 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.740s	user 0.037s	sys 0.038s
I20260709 13:09:51.773600 27630 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:51.819135 27646 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:51.814038 27536 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:51.876593 23021 raft_consensus.cc:1217] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Deduplicated request from leader. Original: 5.3->[7.4-13.7]   Dedup: 7.4->[10.5-13.7]
I20260709 13:09:51.882913 23021 pending_rounds.cc:85] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75: Aborting all ops after (but not including) 4
I20260709 13:09:51.883363 23021 pending_rounds.cc:107] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 13:09:51.883909 23021 raft_consensus.cc:2889] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:51.863196 27585 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:51.869845 27596 raft_consensus.cc:2749] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:09:51.893781 27666 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:51.901786 27289 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:51.881007 27630 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:51.885962 27646 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:51.914497 27666 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:51.902107 27661 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:51.912649 27562 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:51.923483 27163 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:51.930078 27646 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:51.934597 27562 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:51.935236 22951 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 752) took 1423 ms. Trace:
I20260709 13:09:51.935196 27562 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:09:51.935806 22951 rpcz_store.cc:276] 0709 13:09:50.511490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:51.130381 (+618891us) service_pool.cc:224] Handling call
0709 13:09:51.149564 (+ 19183us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:51.150063 (+   499us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:51.150112 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:51.150748 (+   636us) log.cc:844] Serialized 26 byte log entry
0709 13:09:51.724935 (+574187us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:09:51.724978 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:51.726486 (+  1508us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:51.934981 (+208495us) raft_consensus.cc:1685] finished
0709 13:09:51.935007 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:51.935167 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4852,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":103,"thread_start_us":418051,"threads_started":1,"wal-append.queue_time_us":622317}
I20260709 13:09:51.939044 22874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2642) took 1575 ms. Trace:
I20260709 13:09:51.939568 22874 rpcz_store.cc:276] 0709 13:09:50.363909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:50.436808 (+ 72899us) service_pool.cc:224] Handling call
0709 13:09:51.234625 (+797817us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:51.235302 (+   677us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:51.938820 (+703518us) spinlock_profiling.cc:243] Waited 797 ms on lock 0x7b54001bb550. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:51.938978 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1833878912}
I20260709 13:09:51.923012 27281 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:51.961360 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:09:51.962400 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 112367us
I20260709 13:09:51.963163 22918 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:51.966526 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:51.967075 23621 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
I20260709 13:09:51.968474 22919 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:51.971097 22876 raft_consensus.cc:1240] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 10 LEADER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:51.971095 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:51.971777 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:51.916864 27289 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:51.972362 22874 raft_consensus.cc:2393] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 15: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:51.971387 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:51.973754 22879 raft_consensus.cc:2468] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 9.
I20260709 13:09:51.926386 27557 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:51.919190 27650 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:51.972364 22875 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 13.
I20260709 13:09:51.980165 22843 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:51.996002 22918 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:51.997184 22874 pending_rounds.cc:85] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 8
I20260709 13:09:51.956149 23027 raft_consensus.cc:2468] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 10.
W20260709 13:09:51.994838 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:51.998610 22843 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:51.997623 22874 pending_rounds.cc:107] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 13:09:52.000787 22874 raft_consensus.cc:2889] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:52.002178 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:52.008718 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:51.969313 27639 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:51.920481 27661 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:51.974888 27520 raft_consensus.cc:2749] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:09:51.970535 27681 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
W20260709 13:09:52.063402 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 62159.1us
I20260709 13:09:51.911319 27656 consensus_queue.cc:237] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:52.000195 27646 raft_consensus.cc:2804] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:09:52.089568 22764 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2959101199a44a4583a4ee456ecf044d with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 0a407504ed1741bf89a276558adf4c0f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 12 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:09:52.094871 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:52.097455 22877 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 13.
I20260709 13:09:52.003685 27557 raft_consensus.cc:2804] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:09:51.997342 27022 raft_consensus.cc:697] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 14 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:09:52.015223 27659 log.cc:927] Time spent T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.165s	user 0.001s	sys 0.001s
I20260709 13:09:52.012466 27676 raft_consensus.cc:2749] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:52.002925 27596 raft_consensus.cc:2804] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:09:52.104516 22879 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 9.
W20260709 13:09:52.104503 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 104927us
W20260709 13:09:52.121868 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:52.122292 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:52.110430 27520 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:52.096338 27646 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:52.016180 27271 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:09:52.114171 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 296789us
I20260709 13:09:52.120077 23028 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
I20260709 13:09:52.126403 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:52.139618 22952 raft_consensus.cc:2393] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 12: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:52.129703 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:52.143302 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:52.144865 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:52.145612 22952 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
I20260709 13:09:52.147125 22952 raft_consensus.cc:1240] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 13. Current term is 14. Ops: [9.7-13.9]
W20260709 13:09:52.148303 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:52.102706 27591 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 13:09:52.109320 27677 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:52.073190 27650 consensus_queue.cc:1048] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 13:09:52.153501 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:52.165585 22872 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 13.
I20260709 13:09:52.161751 27022 consensus_queue.cc:237] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:52.149025 27249 raft_consensus.cc:2749] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:52.122108 27557 raft_consensus.cc:697] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 9 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:52.070448 27639 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:52.135428 27646 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:52.182493 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:52.137567 27596 raft_consensus.cc:697] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:52.150840 27536 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:52.102006 27681 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:52.199276 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:09:52.199641 27667 log.cc:927] Time spent T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 13:09:52.200459 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:52.201288 27685 log.cc:927] Time spent T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.127s	user 0.000s	sys 0.000s
I20260709 13:09:52.201558 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:52.202651 27668 log.cc:927] Time spent T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.275s	user 0.000s	sys 0.001s
I20260709 13:09:52.202960 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:52.204174 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:52.205315 22872 raft_consensus.cc:2410] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 13 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
I20260709 13:09:52.168591 27677 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:52.135905 27520 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:52.186234 27682 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:52.213811 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:52.214747 22947 raft_consensus.cc:2393] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 12: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:52.191823 27562 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:52.224735 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:52.226181 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:52.225453 22872 raft_consensus.cc:2468] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11.
I20260709 13:09:52.231267 23590 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:52.232798 23590 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:52.233779 27289 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:52.234114 27585 consensus_queue.cc:1059] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:09:52.225600 27665 log.cc:927] Time spent T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.355s	user 0.001s	sys 0.000s
I20260709 13:09:52.234970 27585 consensus_queue.cc:1243] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer 0a407504ed1741bf89a276558adf4c0f log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 13:09:52.235579 27585 raft_consensus.cc:2804] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:52.235940 27585 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:52.236001 27596 consensus_queue.cc:237] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:52.237025 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:52.237334 27641 raft_consensus.cc:2804] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:52.237702 27641 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:52.238040 27641 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:52.238109 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:52.238739 22880 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:09:52.227316 27682 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:52.241242 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:52.242517 22953 raft_consensus.cc:1240] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 11. Current term is 12. Ops: []
I20260709 13:09:52.232847 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:52.236418 27585 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:52.251562 27639 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:52.253422 27666 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:52.253340 23590 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:52.255146 27271 raft_consensus.cc:2804] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:52.255668 27271 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:52.256029 27680 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:52.255611 27479 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:52.256276 27271 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:52.257736 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:52.256515 27680 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:52.262419 27589 raft_consensus.cc:3055] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:09:52.275997 23621 raft_consensus.cc:2410] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:09:52.262050 27641 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:52.263154 27484 consensus_meta.cc:287] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.804s	user 0.011s	sys 0.000s
I20260709 13:09:52.255183 27520 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:52.265208 22947 raft_consensus.cc:2393] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 16: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
W20260709 13:09:52.266986 22871 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53724 (request call id 761) took 1849 ms (client timeout 1856 ms). Trace:
W20260709 13:09:52.284742 22871 rpcz_store.cc:269] 0709 13:09:50.417421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:50.419229 (+  1808us) service_pool.cc:224] Handling call
0709 13:09:52.266824 (+1847595us) spinlock_profiling.cc:243] Waited 283 ms on lock 0x7b5400412654. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:52.266935 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":651280768}
I20260709 13:09:52.266848 22920 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:52.286899 22920 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:52.208850 27630 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:52.234665 27674 log.cc:927] Time spent T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.296s	user 0.000s	sys 0.001s
I20260709 13:09:52.233178 27661 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:52.296953 23028 raft_consensus.cc:1240] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 13. Current term is 14. Ops: []
I20260709 13:09:52.298055 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:52.301594 27703 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:52.302157 27703 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:52.302624 27356 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 7 FOLLOWER]: Advancing to term 8
I20260709 13:09:52.290206 27663 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:52.305209 27663 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:52.307291 27663 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:52.307777 27663 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:52.310603 22920 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:52.304405 27703 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:52.319087 23028 raft_consensus.cc:1240] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:09:52.325798 22918 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:52.329365 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:52.343495 27596 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:52.283320 27589 raft_consensus.cc:740] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:52.345705 22920 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:52.351536 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:52.352509 23028 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 14.
I20260709 13:09:52.353513 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:52.354627 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:52.288957 27697 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:52.308209 27663 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:52.381524 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:52.382337 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:52.390812 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:52.391530 23028 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
I20260709 13:09:52.309796 27630 raft_consensus.cc:2749] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:52.394878 22872 raft_consensus.cc:1240] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:52.395292 22880 raft_consensus.cc:1240] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:52.327860 27520 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:52.381543 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:52.392453 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:52.406342 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 132696us
I20260709 13:09:52.406966 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:52.378773 27289 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:52.426450 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:52.427397 23020 raft_consensus.cc:3055] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:09:52.427839 23020 raft_consensus.cc:740] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:52.395437 27639 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:52.395603 27703 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:52.344153 27596 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:52.403668 27697 raft_consensus.cc:2749] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:52.421023 27484 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:60032 (request call id 910) took 5203 ms. Trace:
W20260709 13:09:52.416709 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:52.431937 27696 raft_consensus.cc:2804] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:52.407872 23028 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
I20260709 13:09:52.434760 27663 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:52.446506 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:52.396880 27585 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:52.446440 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:52.449095 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:52.442333 23020 consensus_queue.cc:260] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:52.451128 22875 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
I20260709 13:09:52.451340 23020 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:52.453598 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:52.454243 22871 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
I20260709 13:09:52.409643 27479 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:52.446565 27698 consensus_queue.cc:1048] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.009s
I20260709 13:09:52.458103 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:52.420023 27681 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:52.376518 27557 consensus_queue.cc:237] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:52.460685 23028 raft_consensus.cc:2368] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 12. Current term is 13.
I20260709 13:09:52.419363 27682 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:52.438338 27630 raft_consensus.cc:2749] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:52.463373 22875 raft_consensus.cc:1275] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:09:52.443498 27696 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:52.408950 27520 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:52.479506 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 153815us
I20260709 13:09:52.487774 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:52.491873 22880 raft_consensus.cc:2410] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
W20260709 13:09:52.488482 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 107303us
W20260709 13:09:52.489609 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 143975us
I20260709 13:09:52.498471 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:52.499553 22843 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:52.502210 23020 raft_consensus.cc:1275] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:09:52.499225 23028 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11.
I20260709 13:09:52.501742 22920 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:52.516387 23020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 2023) took 2163 ms. Trace:
W20260709 13:09:52.517011 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:52.516882 23020 rpcz_store.cc:276] 0709 13:09:50.339617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:50.343788 (+  4171us) service_pool.cc:224] Handling call
0709 13:09:50.370572 (+ 26784us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:52.427270 (+2056698us) spinlock_profiling.cc:243] Waited 27.7 ms on lock 0x7b5400039920. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135d5cdc0 00007f6135de71c0 00007f6135de8231 00007f6135de4d8f 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c
0709 13:09:52.502725 (+ 75455us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:52.502965 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":63630208}
I20260709 13:09:52.515496 22920 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:52.437832 27703 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:52.521808 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 146057us
I20260709 13:09:52.436483 27289 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:52.437973 27639 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:52.416960 27589 consensus_queue.cc:260] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:52.528457 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:52.487843 27479 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:52.530133 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:52.491425 27696 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:52.530938 22871 raft_consensus.cc:1240] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 12 LEADER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 11. Current term is 12. Ops: []
I20260709 13:09:52.530916 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:52.547353 27550 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:52.555867 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:52.518402 27706 log.cc:927] Time spent T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 13:09:52.523666 27680 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:52.545657 27611 log.cc:927] Time spent T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 2.422s	user 0.004s	sys 0.016s
W20260709 13:09:52.540598 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:52.533859 27716 raft_consensus.cc:2804] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:52.562932 23028 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 12.
I20260709 13:09:52.576754 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:52.550923 27638 log.cc:927] Time spent T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 1.961s	user 0.013s	sys 0.023s
I20260709 13:09:52.533349 27163 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:52.553110 27515 raft_consensus.cc:3055] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:09:52.545972 27663 consensus_queue.cc:1059] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:09:52.572504 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:52.578317 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:52.577507 23028 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11.
W20260709 13:09:52.583340 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:52.583859 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:52.583958 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:52.584190 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:52.565177 27589 raft_consensus.cc:3060] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:52.585793 23020 raft_consensus.cc:1240] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 9. Current term is 10. Ops: [7.6-9.7]
W20260709 13:09:52.586300 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 62687us
I20260709 13:09:52.585160 22992 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:52.587193 22994 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:52.570684 27681 raft_consensus.cc:2749] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:52.438325 27484 rpcz_store.cc:276] 0709 13:09:47.218094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:47.221780 (+  3686us) service_pool.cc:224] Handling call
0709 13:09:47.553312 (+331532us) ts_tablet_manager.cc:934] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2: Initiating tablet copy from peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
0709 13:09:50.376307 (+2822995us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:09:51.342183 (+965876us) tablet_metadata.cc:665] Metadata flushed
0709 13:09:51.357859 (+ 15676us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:09:52.329756 (+971897us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:09:52.420435 (+ 90679us) tablet_replica.cc:206] Creating consensus instance
0709 13:09:52.420839 (+   404us) ts_tablet_manager.cc:989] Replica registered.
0709 13:09:52.420922 (+    83us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":19589,"tablet-copy.queue_time_us":164721,"thread_start_us":175254,"threads_started":2}
W20260709 13:09:52.591333 27649 log.cc:927] Time spent T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 1.338s	user 0.000s	sys 0.006s
I20260709 13:09:52.592105 27641 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:52.586090 27163 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:52.593209 27585 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:52.582763 27515 raft_consensus.cc:740] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:52.602737 27163 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:52.602483 27515 consensus_queue.cc:260] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:52.538766 27479 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:52.579653 27716 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:52.600940 27484 tablet_copy_client.cc:806] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 0 data blocks...
W20260709 13:09:52.594131 27407 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 13:09:52.612025 22994 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:09:52.604565 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:52.607813 27479 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:52.607842 27621 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:52.609397 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:52.588794 27677 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:52.581846 27682 consensus_queue.cc:1059] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
W20260709 13:09:52.613819 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 81659.5us
I20260709 13:09:52.615842 22871 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:52.616819 27407 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:52.617060 27484 tablet_copy_client.cc:670] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:09:52.614997 27515 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:52.621831 27520 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:52.622288 23621 raft_consensus.cc:1275] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 13:09:52.626494 27662 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:52.626763 27271 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:52.628376 23026 raft_consensus.cc:1275] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 13:09:52.629640 27703 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:52.610652 27681 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:52.610477 27716 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:09:52.659271 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 47107.2us
I20260709 13:09:52.665421 27719 consensus_queue.cc:1048] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:09:52.665946 22878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 2119) took 2635 ms. Trace:
I20260709 13:09:52.666335 22878 rpcz_store.cc:276] 0709 13:09:50.030369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:50.030578 (+   209us) service_pool.cc:224] Handling call
0709 13:09:50.031193 (+   615us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:09:50.031569 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:50.031597 (+    28us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:09:50.032245 (+   648us) log.cc:844] Serialized 78 byte log entry
0709 13:09:50.492113 (+459868us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:09:50.533428 (+ 41315us) log.cc:844] Serialized 14 byte log entry
0709 13:09:50.535719 (+  2291us) log.cc:844] Serialized 14 byte log entry
0709 13:09:50.566875 (+ 31156us) log.cc:844] Serialized 14 byte log entry
0709 13:09:50.567090 (+   215us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:50.567188 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:51.069205 (+502017us) spinlock_profiling.cc:243] Waited 9.22 ms on lock 0x7b18002aca60. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cf83f3 00007f6132cfa7c1 00007f6132cfa779 00007f6132cfa709 00007f6132cfa6d1 00007f6132cf99cd 000000000038f354 000000000038f199 00007f6132cdb83a
0709 13:09:51.409717 (+340512us) spinlock_profiling.cc:243] Waited 2.26 ms on lock 0x7b18002aca60. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cf83f3 00007f6132cfa7c1 00007f6132cfa779 00007f6132cfa709 00007f6132cfa6d1 00007f6132cf99cd 000000000038f354 000000000038f199 00007f6132cdb83a
0709 13:09:52.550219 (+1140502us) spinlock_profiling.cc:243] Waited 150 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:09:52.665746 (+115527us) raft_consensus.cc:1685] finished
0709 13:09:52.665767 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:52.665893 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34797,"prepare.queue_time_us":335568,"prepare.run_cpu_time_us":6076,"prepare.run_wall_time_us":32586,"raft.queue_time_us":4097,"raft.run_cpu_time_us":431,"raft.run_wall_time_us":35702,"spinlock_wait_cycles":371579136,"thread_start_us":443319,"threads_started":1,"wal-append.queue_time_us":557050}
I20260709 13:09:52.671283 27696 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:52.662427 23588 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:52.663177 23024 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33418 (request call id 2018) took 2325 ms (client timeout 1743 ms). Trace:
I20260709 13:09:52.663177 22918 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:52.674377 22918 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:52.668541 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:52.642714 27596 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:52.673626 23024 rpcz_store.cc:269] 0709 13:09:50.338052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:50.338175 (+   123us) service_pool.cc:224] Handling call
0709 13:09:52.662994 (+2324819us) spinlock_profiling.cc:243] Waited 311 ms on lock 0x7b5400248e54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:52.663108 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":715843456}
I20260709 13:09:52.659366 27594 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:52.686856 22992 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:52.687986 22871 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 14.
I20260709 13:09:52.688746 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:52.677870 27723 raft_consensus.cc:2804] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:52.678136 27520 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:52.679504 27163 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:52.681102 27281 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:52.681507 23588 connection.cc:256] RPC call timeout handler was delayed by -1.4021905349999315s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15780000000000002s for a response
W20260709 13:09:52.681842 27648 log.cc:927] Time spent T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 1.440s	user 0.004s	sys 0.000s
I20260709 13:09:52.657164 27407 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:52.683552 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:52.684175 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:52.695633 27407 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:52.695834 22875 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:09:52.674433 27681 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:52.657684 27722 raft_consensus.cc:2804] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:52.686815 22918 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:52.661134 27662 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:52.689728 22871 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 8.
I20260709 13:09:52.691164 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:52.694489 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:52.696377 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:52.702512 22950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 751) took 2191 ms. Trace:
I20260709 13:09:52.706071 22918 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:52.707413 22918 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:52.706743 22950 rpcz_store.cc:276] 0709 13:09:50.511189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:50.524070 (+ 12881us) service_pool.cc:224] Handling call
0709 13:09:50.524383 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:50.539878 (+ 15495us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:50.539923 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:50.540493 (+   570us) log.cc:844] Serialized 26 byte log entry
0709 13:09:50.972398 (+431905us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:09:50.972451 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:50.972530 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:52.681191 (+1708661us) spinlock_profiling.cc:243] Waited 2.61 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:09:52.696623 (+ 15432us) raft_consensus.cc:1685] finished
0709 13:09:52.696646 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:52.702450 (+  5804us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9834,"raft.queue_time_us":17577,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":20859,"spinlock_wait_cycles":6007680,"thread_start_us":387670,"threads_started":1,"wal-append.queue_time_us":552427}
I20260709 13:09:52.708966 22918 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:09:52.716773 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:52.721299 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:52.722131 22950 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 10.
I20260709 13:09:52.723035 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:52.694815 27723 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:52.701599 27520 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:09:52.726950 27663 consensus_queue.cc:1048] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:09:52.728708 27520 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:52.728873 27703 consensus_queue.cc:1048] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:09:52.729316 27717 raft_consensus.cc:2749] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:52.729671 22949 raft_consensus.cc:2393] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 13: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:52.701290 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:52.727245 27723 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:52.725292 22953 raft_consensus.cc:3060] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:52.733249 27425 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:52.734426 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:52.735107 27464 raft_consensus.cc:3055] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:09:52.738850 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:52.739367 23022 raft_consensus.cc:1275] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 13:09:52.704382 27479 raft_consensus.cc:2804] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:09:52.742614 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:52.743790 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:52.703083 27722 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:52.719385 27269 raft_consensus.cc:2749] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:09:52.719177 27550 raft_consensus.cc:2804] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:52.736395 27601 raft_consensus.cc:3055] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:09:52.747696 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:52.719882 27682 raft_consensus.cc:2749] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:52.729285 27520 raft_consensus.cc:2804] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:52.750015 22950 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 15.
I20260709 13:09:52.751616 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:52.749640 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:52.701339 27281 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:52.694401 27596 consensus_queue.cc:1048] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 13:09:52.750443 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:52.730800 27639 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:52.701400 27163 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:52.741283 27663 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:52.697669 27594 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:09:52.736984 27696 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:52.753487 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:52.768278 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:52.746137 27425 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:52.767222 22953 raft_consensus.cc:2468] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 13.
I20260709 13:09:52.745857 27464 raft_consensus.cc:740] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:52.750537 27722 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:09:52.758406 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:52.758158 22875 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 13.
I20260709 13:09:52.760070 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:52.762609 27163 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:52.763147 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:52.753785 27438 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:52.750541 27601 raft_consensus.cc:740] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:52.769344 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:52.751703 27520 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:52.752640 22871 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:52.753808 27479 raft_consensus.cc:697] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 12 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:52.750689 27550 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:52.756601 27289 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:52.851408 22952 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:52.866359 27520 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:52.785404 23027 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 2038) took 1036 ms. Trace:
I20260709 13:09:52.867527 23027 rpcz_store.cc:276] 0709 13:09:51.749150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:52.003132 (+253982us) service_pool.cc:224] Handling call
0709 13:09:52.004439 (+  1307us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:52.165046 (+160607us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:52.165077 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:52.195934 (+ 30857us) log.cc:844] Serialized 52 byte log entry
0709 13:09:52.612309 (+416375us) spinlock_profiling.cc:243] Waited 28.9 ms on lock 0x7b180014dca0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f613598d4bc 00007f6132cdb48b 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf7c02 00007f613597f20b 00007f6135984d23 00007f61359850d1
0709 13:09:52.612473 (+   164us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:09:52.655650 (+ 43177us) log.cc:844] Serialized 14 byte log entry
0709 13:09:52.655871 (+   221us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:52.655964 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:52.775681 (+119717us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b180014dca0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cf83f3 00007f6132cfa7c1 00007f6132cfa779 00007f6132cfa709 00007f6132cfa6d1 00007f6132cf99cd 000000000038f354 000000000038f199 00007f6132cdb83a
0709 13:09:52.785110 (+  9429us) raft_consensus.cc:1685] finished
0709 13:09:52.785141 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:52.785331 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51844,"prepare.queue_time_us":8752,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":223243,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":25642,"spinlock_wait_cycles":93763840,"thread_start_us":207877,"threads_started":3,"wal-append.queue_time_us":126207}
I20260709 13:09:52.811487 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:52.870311 22948 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
I20260709 13:09:52.871649 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:52.866498 27550 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:52.874648 23020 raft_consensus.cc:1275] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:09:52.875912 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:52.876180 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 167265us
I20260709 13:09:52.923280 27407 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:52.935694 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:52.936550 23624 raft_consensus.cc:2393] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 13: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:52.936955 27407 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:52.927409 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:52.780119 27646 consensus_meta.cc:287] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.585s	user 0.005s	sys 0.005s
I20260709 13:09:52.915680 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:52.814429 27500 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
W20260709 13:09:52.908025 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:52.774139 27281 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:52.778643 27681 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:52.766014 27730 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:52.899194 23023 raft_consensus.cc:1275] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:09:52.878973 22992 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:52.946018 27500 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:52.947113 27425 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:52.924847 27472 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:09:52.950186 27640 log.cc:927] Time spent T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 2.379s	user 0.013s	sys 0.010s
I20260709 13:09:52.950958 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:09:52.945104 23621 raft_consensus.cc:1275] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:09:52.945062 27646 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:52.815177 27502 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:52.898269 27464 consensus_queue.cc:260] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:52.959501 27698 raft_consensus.cc:2749] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:52.926375 22952 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 13.
I20260709 13:09:52.768555 27663 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:52.955425 23021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 2036) took 1212 ms. Trace:
I20260709 13:09:52.903064 27716 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:52.948534 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:52.959137 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 80244.7us
I20260709 13:09:52.962688 22871 raft_consensus.cc:2468] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 10.
I20260709 13:09:52.980796 22950 raft_consensus.cc:2393] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 12: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:52.981465 22872 raft_consensus.cc:1240] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:09:52.981832 22992 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:52.983047 22992 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:52.988040 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:52.872924 27438 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:52.965350 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 205154us
I20260709 13:09:52.939015 27479 consensus_queue.cc:237] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:52.877202 22948 raft_consensus.cc:2393] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 9: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:52.960421 27407 raft_consensus.cc:2804] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:52.908291 27601 consensus_queue.cc:260] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:53.002389 27407 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:53.002828 27407 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:53.003284 27722 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:52.939611 22918 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:52.966454 27737 log.cc:927] Time spent T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 13:09:52.978368 23021 rpcz_store.cc:276] 0709 13:09:51.742928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:51.872571 (+129643us) service_pool.cc:224] Handling call
0709 13:09:51.876285 (+  3714us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:09:51.898961 (+ 22676us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 13:09:51.989129 (+ 90168us) log.cc:844] Serialized 14 byte log entry
0709 13:09:52.279315 (+290186us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:09:52.280104 (+   789us) log.cc:844] Serialized 78 byte log entry
0709 13:09:52.280283 (+   179us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:09:52.280527 (+   244us) log.cc:844] Serialized 14 byte log entry
0709 13:09:52.954374 (+673847us) spinlock_profiling.cc:243] Waited 1.41 ms on lock 0x7b18003c7420. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132c72c0a 00007f613c3b2219 00007f613599bd70 00007f613598f293 00007f6135984cea 00007f613598523a 00007f6135defa2a 00007f6135dff0bf 00007f6135dff038
0709 13:09:52.954824 (+   450us) log.cc:844] Serialized 14 byte log entry
0709 13:09:52.954995 (+   171us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:52.955154 (+   159us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:52.955197 (+    43us) raft_consensus.cc:1685] finished
0709 13:09:52.955212 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:52.955349 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":745675,"prepare.queue_time_us":28094,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":3013,"raft.queue_time_us":281972,"raft.run_cpu_time_us":522,"raft.run_wall_time_us":35778,"spinlock_wait_cycles":9547008,"thread_start_us":319381,"threads_started":3,"wal-append.queue_time_us":18138}
I20260709 13:09:53.000106 27680 raft_consensus.cc:2804] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:09:53.004886 27719 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:53.005488 27639 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:52.907532 27550 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:53.007776 23022 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 15.
I20260709 13:09:53.001207 23588 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:53.008973 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:09:53.008947 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 69416.5us
I20260709 13:09:53.010910 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:53.010655 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:52.951823 27723 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:52.970718 27520 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:53.023303 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:53.023844 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:53.024401 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:53.025919 23619 raft_consensus.cc:1240] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14. Ops: []
I20260709 13:09:53.031260 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:53.030905 22918 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:53.027281 23621 raft_consensus.cc:1240] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:09:53.028311 22948 raft_consensus.cc:2393] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 12: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:53.028267 23588 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:53.029940 27438 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:09:53.034569 27642 log.cc:927] Time spent T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 2.374s	user 0.003s	sys 0.018s
I20260709 13:09:53.035037 27425 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:53.030125 27550 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:53.031656 23625 raft_consensus.cc:2410] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:09:53.020378 27681 raft_consensus.cc:2749] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:53.036774 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:53.033830 23588 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:53.031657 23618 raft_consensus.cc:2410] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 15 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:09:53.032430 22952 raft_consensus.cc:2410] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:09:53.006882 27464 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:53.036047 23617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 951) took 2950 ms. Trace:
I20260709 13:09:52.957499 27730 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:53.042584 22951 raft_consensus.cc:1240] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:09:53.038334 23617 rpcz_store.cc:276] 0709 13:09:50.085490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:50.085614 (+   124us) service_pool.cc:224] Handling call
0709 13:09:50.085928 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:50.388422 (+302494us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:50.388450 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:50.396835 (+  8385us) log.cc:844] Serialized 26 byte log entry
0709 13:09:50.506609 (+109774us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:09:50.506649 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:50.591637 (+ 84988us) spinlock_profiling.cc:243] Waited 302 ms on lock 0x7b540049a2d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:50.591677 (+    40us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:52.947256 (+2355579us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:09:53.035831 (+ 88575us) raft_consensus.cc:1685] finished
0709 13:09:53.035853 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:53.035992 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":455,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":30506,"spinlock_wait_cycles":1024685824,"thread_start_us":98957,"threads_started":1,"wal-append.queue_time_us":140761}
I20260709 13:09:53.045334 22950 raft_consensus.cc:2393] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 12: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:53.046101 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:53.046310 27663 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:53.046399 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:53.046710 23618 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 9.
I20260709 13:09:53.048303 27407 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:53.027544 27680 raft_consensus.cc:697] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:09:53.053090 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:53.053642 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:53.030148 27722 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:53.039554 27598 raft_consensus.cc:2749] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:53.027835 27719 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:53.061549 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:53.063409 23024 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 13.
I20260709 13:09:53.063907 27739 raft_consensus.cc:2804] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:53.064177 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:53.064368 27739 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:53.042254 27550 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:53.042326 27438 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:53.065063 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:53.065583 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:53.030977 27639 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:53.069406 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:53.070380 22946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 754) took 2534 ms. Trace:
I20260709 13:09:53.072122 27646 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:53.073963 27719 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:53.068881 27407 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:53.090149 22946 rpcz_store.cc:276] 0709 13:09:50.535901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:51.147050 (+611149us) service_pool.cc:224] Handling call
0709 13:09:51.147473 (+   423us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:51.345728 (+198255us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:51.345760 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:51.345797 (+    37us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:09:51.345827 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:53.070160 (+1724333us) spinlock_profiling.cc:243] Waited 198 ms on lock 0x7b54004123d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:53.070204 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:53.070328 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":455401344}
I20260709 13:09:53.094213 22946 raft_consensus.cc:1240] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 13:09:53.095392 22946 raft_consensus.cc:1240] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:53.096072 27520 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:53.088645 23619 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
I20260709 13:09:53.096513 22946 raft_consensus.cc:1275] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:09:53.097195 23617 raft_consensus.cc:1275] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 13:09:53.100931 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:53.102527 22947 raft_consensus.cc:2393] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 12: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:53.104036 22947 raft_consensus.cc:1275] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:09:53.113106 27709 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:53.115554 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:53.113951 27709 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:53.100332 27438 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:53.076800 27739 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:53.123056 27646 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:53.129335 27520 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:53.129489 27651 raft_consensus.cc:3055] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:09:53.147461 27651 raft_consensus.cc:740] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:53.147557 22874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53724 (request call id 769) took 1151 ms. Trace:
I20260709 13:09:53.129873 27681 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:53.130322 27591 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:53.152063 27591 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:53.152297 27681 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:53.154364 27681 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:53.154430 27591 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:53.130885 27425 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:53.131031 27716 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:53.134199 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:53.137478 27720 raft_consensus.cc:3055] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:09:53.139591 27680 consensus_queue.cc:237] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:53.141845 27723 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:53.142392 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:53.143101 27646 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:53.167201 27646 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:53.146757 27520 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:53.148001 22874 rpcz_store.cc:276] 0709 13:09:51.995767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:51.996619 (+   852us) service_pool.cc:224] Handling call
0709 13:09:51.996996 (+   377us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:52.070699 (+ 73703us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:52.070729 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:52.071251 (+   522us) log.cc:844] Serialized 52 byte log entry
0709 13:09:52.816048 (+744797us) spinlock_profiling.cc:243] Waited 6.99 ms on lock 0x7b180005d8e0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cdae07 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf7c02 00007f613597f20b 00007f6135984d23 00007f61359850d1 00007f6135da0a0f
0709 13:09:52.875164 (+ 59116us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:09:52.875220 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:52.875323 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:53.147243 (+271920us) raft_consensus.cc:1685] finished
0709 13:09:53.147263 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:53.147478 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1760,"raft.queue_time_us":4238,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":10839,"spinlock_wait_cycles":16087296,"thread_start_us":58900,"threads_started":1,"wal-append.queue_time_us":781179}
I20260709 13:09:53.157442 23617 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 14.
I20260709 13:09:53.155236 27425 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:53.160799 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:53.155354 27651 consensus_queue.cc:260] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:53.162822 27720 raft_consensus.cc:740] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:53.143260 27438 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:53.169334 23619 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:53.171020 27520 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:53.171618 27425 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:53.148118 27407 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:53.176332 22953 raft_consensus.cc:1240] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:09:53.175397 27407 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:53.181324 22947 raft_consensus.cc:1240] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 12. Current term is 13. Ops: [10.9-12.10]
I20260709 13:09:53.185621 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:53.183866 27407 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:53.206487 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:53.207412 23625 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 12.
I20260709 13:09:53.222080 27639 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:53.223747 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:53.241298 27730 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:53.193264 27651 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:53.243515 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:53.180418 27720 consensus_queue.cc:260] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:53.258754 27709 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:53.271191 23625 raft_consensus.cc:2410] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:09:53.276149 23026 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 2026) took 2886 ms. Trace:
I20260709 13:09:53.240036 27500 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:53.276691 23026 rpcz_store.cc:276] 0709 13:09:50.389933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:50.422477 (+ 32544us) service_pool.cc:224] Handling call
0709 13:09:50.422699 (+   222us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:52.659193 (+2236494us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:53.275902 (+616709us) spinlock_profiling.cc:243] Waited 2.21 s on lock 0x7b54000bbdd4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:53.276085 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5071783680}
I20260709 13:09:53.282552 27720 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:53.277380 27678 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:53.287911 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:53.276535 27589 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:53.297708 27589 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:53.303884 23025 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33418 (request call id 2028) took 1563 ms (client timeout 2079 ms). Trace:
W20260709 13:09:53.304421 23025 rpcz_store.cc:269] 0709 13:09:51.740502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:51.740633 (+   131us) service_pool.cc:224] Handling call
0709 13:09:53.303587 (+1562954us) spinlock_profiling.cc:243] Waited 24 ms on lock 0x7b540010bb54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:53.303777 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":55140736}
I20260709 13:09:53.305966 22954 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2793) took 3338 ms. Trace:
I20260709 13:09:53.306489 22954 rpcz_store.cc:276] 0709 13:09:49.967725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:50.097578 (+129853us) service_pool.cc:224] Handling call
0709 13:09:50.097921 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:51.384853 (+1286932us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:53.305653 (+1920800us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b5400149454. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:53.305881 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2957988736}
I20260709 13:09:53.308029 23028 raft_consensus.cc:1240] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 14. Current term is 15. Ops: [14.7-14.7]
I20260709 13:09:53.280570 27550 raft_consensus.cc:2749] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:53.288398 27678 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:53.367808 22946 raft_consensus.cc:1240] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:53.394032 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:09:53.398834 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 900660us
I20260709 13:09:53.400933 27512 raft_consensus.cc:2804] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:53.401523 27512 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:53.402030 27512 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:53.407122 27462 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:53.408035 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:53.407827 27462 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:53.410322 27462 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:53.414929 23618 raft_consensus.cc:1240] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 12. Current term is 13. Ops: []
W20260709 13:09:53.408074 22844 consensus_peers.cc:597] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f -> Peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Couldn't send request to peer 5b346767a89745fa82928a5dac9cc0b2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ac5cc94926a742e6a0c1340ec33c0ff0. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:09:53.415771 23619 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11.
I20260709 13:09:53.414741 23590 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:53.419070 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:53.418982 23590 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:53.423797 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:53.419778 22874 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 13.
I20260709 13:09:53.414851 22843 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:53.428790 27158 raft_consensus.cc:2804] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:53.429024 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:53.429831 22871 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:53.431617 27594 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:53.428403 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:53.429272 27158 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:53.437868 27158 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:53.438005 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:53.439006 22876 raft_consensus.cc:2393] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 13: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
W20260709 13:09:53.439158 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:53.440101 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:53.440428 27588 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:53.440845 27588 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:53.441179 27588 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:53.435328 23590 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:53.441656 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:09:53.442392 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:53.443742 22876 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 13.
I20260709 13:09:53.443760 23590 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:53.434584 22873 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 13.
I20260709 13:09:53.448590 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:53.449383 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:53.450450 22879 raft_consensus.cc:2410] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:09:53.452136 27271 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:53.449615 22875 raft_consensus.cc:2393] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 12: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:53.458207 23590 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:53.463011 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:53.433872 27472 raft_consensus.cc:2804] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:09:53.489324 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:53.473173 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:53.491047 22954 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
I20260709 13:09:53.496945 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:53.497810 22949 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
I20260709 13:09:53.506345 22873 raft_consensus.cc:1240] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:53.508473 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:53.511010 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:53.512044 22875 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 12.
I20260709 13:09:53.519076 22871 raft_consensus.cc:2468] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 12.
I20260709 13:09:53.519963 22949 raft_consensus.cc:1275] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:09:53.534029 27703 consensus_queue.cc:1059] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 13:09:53.525183 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:53.527915 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:53.564067 22951 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
I20260709 13:09:53.529518 22993 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:53.489297 27271 raft_consensus.cc:2749] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:53.567194 22954 raft_consensus.cc:1240] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 13. Current term is 14. Ops: []
W20260709 13:09:53.572230 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:53.532297 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:53.533823 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:53.533221 22919 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:09:53.536164 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:53.521139 27627 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:53.577523 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 44391.9us
I20260709 13:09:53.565229 27664 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:53.545933 27739 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:53.530700 27472 raft_consensus.cc:697] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 14 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:53.520923 27520 consensus_queue.cc:1048] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 13:09:53.532408 27407 consensus_queue.cc:1059] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 13:09:53.447496 27179 raft_consensus.cc:2804] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:53.562323 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:53.582217 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:53.584010 22993 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:53.579679 22919 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:53.538149 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:53.583985 27723 raft_consensus.cc:2804] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:53.564997 27504 raft_consensus.cc:2955] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 8 LEADER]: Committing config change with OpId 7.6: config changed from index -1 to 6, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:09:53.581638 27629 raft_consensus.cc:2749] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:53.584851 27627 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:53.583158 27500 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:53.585630 22843 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:53.588366 27472 consensus_queue.cc:237] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:53.589937 22949 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
I20260709 13:09:53.599436 22993 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:09:53.631199 23588 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.022s	sys 0.021s
I20260709 13:09:53.649922 22949 raft_consensus.cc:1275] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 13:09:53.650286 23588 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.022s	sys 0.021s
W20260709 13:09:53.650554 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 585869us
W20260709 13:09:53.646039 27673 log.cc:927] Time spent T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 1.728s	user 0.004s	sys 0.000s
I20260709 13:09:53.597674 27407 raft_consensus.cc:3055] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:09:53.651746 23588 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:53.649780 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 50460.1us
I20260709 13:09:53.657905 23588 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:53.591538 22875 raft_consensus.cc:2393] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 13: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:53.658166 22993 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:09:53.614358 27744 log.cc:927] Time spent T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 13:09:53.669243 23588 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:53.622748 22954 raft_consensus.cc:1275] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:09:53.651620 27594 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:09:53.651870 27724 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.021s	sys 0.042s
I20260709 13:09:53.597033 27736 raft_consensus.cc:2804] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 13:09:53.697757 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:53.613590 27723 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:53.598485 27663 raft_consensus.cc:2749] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:53.635486 27454 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:53.589569 27179 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:53.591712 27520 raft_consensus.cc:2804] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:09:53.656489 27289 raft_consensus.cc:2749] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:53.658799 27651 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:53.656849 27407 raft_consensus.cc:740] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:53.587678 27739 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:53.628440 27664 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:53.718374 27724 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.021s	sys 0.042s
W20260709 13:09:53.773761 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 188198us
W20260709 13:09:53.735361 27479 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.635s	user 0.020s	sys 0.102s
I20260709 13:09:53.673837 27729 raft_consensus.cc:2749] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:09:53.775228 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
W20260709 13:09:53.719069 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 48052.3us
I20260709 13:09:53.725773 27249 raft_consensus.cc:2804] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:53.783983 22843 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:53.720096 27500 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:53.741775 27743 log.cc:927] Time spent T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.174s	user 0.001s	sys 0.001s
I20260709 13:09:53.722070 27271 raft_consensus.cc:2749] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:53.738997 27700 raft_consensus.cc:2804] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:09:53.721954 27625 raft_consensus.cc:2749] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:53.813191 22843 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:53.773098 27179 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:53.815290 27677 raft_consensus.cc:2804] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:09:53.786788 22955 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2799) took 2771 ms. Trace:
I20260709 13:09:53.896212 22955 rpcz_store.cc:276] 0709 13:09:51.015586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:51.376581 (+360995us) service_pool.cc:224] Handling call
0709 13:09:51.376935 (+   354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:51.399975 (+ 23040us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:51.400017 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:51.400540 (+   523us) log.cc:844] Serialized 26 byte log entry
0709 13:09:51.750297 (+349757us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:51.750353 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:52.421717 (+671364us) spinlock_profiling.cc:243] Waited 22.8 ms on lock 0x7b54003c78d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:52.421775 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:53.645409 (+1223634us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:09:53.786577 (+141168us) raft_consensus.cc:1685] finished
0709 13:09:53.786599 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:53.786727 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":77480192,"thread_start_us":297511,"threads_started":1,"wal-append.queue_time_us":447240}
W20260709 13:09:53.817322 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 202413us
W20260709 13:09:53.897974 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 84862.7us
I20260709 13:09:53.811489 27249 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:53.785714 27479 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.686s	user 0.021s	sys 0.102s
I20260709 13:09:53.778633 27520 raft_consensus.cc:697] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:53.728735 27703 consensus_queue.cc:1059] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.087s
I20260709 13:09:53.824950 23622 raft_consensus.cc:1240] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:53.899536 22843 leader_election.cc:400] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 12. Current term is 13.
I20260709 13:09:53.905663 22843 leader_election.cc:403] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:09:53.783190 27736 raft_consensus.cc:697] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 10 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:53.729015 27723 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:53.721882 27745 consensus_queue.cc:1048] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.098s
W20260709 13:09:53.904332 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 83778.2us
I20260709 13:09:53.819156 27670 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:53.903980 27505 raft_consensus.cc:2804] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:53.821348 27502 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:53.778517 27651 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:53.934119 27658 log.cc:927] Time spent T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 2.036s	user 0.001s	sys 0.002s
I20260709 13:09:53.935791 27736 consensus_queue.cc:237] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:53.906864 27512 consensus_meta.cc:287] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.504s	user 0.009s	sys 0.012s
I20260709 13:09:53.905900 27249 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:53.820402 27500 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:53.906592 27700 raft_consensus.cc:697] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 10 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
W20260709 13:09:53.941464 27656 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.018s	sys 0.013s
W20260709 13:09:53.992079 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 79126.1us
I20260709 13:09:53.920856 27677 raft_consensus.cc:697] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 13 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:53.927687 27505 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:53.952289 27438 consensus_meta.cc:287] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.768s	user 0.006s	sys 0.005s
I20260709 13:09:54.000646 23588 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:53.941943 27583 raft_consensus.cc:2749] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 12. Current term is 13.
W20260709 13:09:54.008137 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 83336.7us
I20260709 13:09:53.940155 27520 consensus_queue.cc:237] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:54.010571 27505 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:09:54.017364 27750 log.cc:927] Time spent T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 13:09:53.942615 27729 consensus_queue.cc:1048] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.061s
W20260709 13:09:53.999032 27656 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.729s	user 0.018s	sys 0.027s
I20260709 13:09:53.927073 27670 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:53.942389 27271 consensus_queue.cc:1048] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.070s
I20260709 13:09:53.943581 27502 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:53.953269 27557 consensus_queue.cc:1048] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 13:09:53.909860 27407 consensus_queue.cc:260] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:53.981824 27512 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:54.019047 27438 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:54.011996 27586 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:54.010802 27627 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:54.029192 27720 consensus_meta.cc:287] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.746s	user 0.011s	sys 0.000s
W20260709 13:09:54.047291 27688 log.cc:927] Time spent T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 1.881s	user 0.006s	sys 0.000s
I20260709 13:09:54.048043 27598 consensus_queue.cc:1048] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:09:54.054837 23621 raft_consensus.cc:1240] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:54.048942 27586 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:54.066774 27662 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:54.068230 27407 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:54.062865 27598 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:54.055696 27766 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:54.073997 27767 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:54.078692 27766 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:54.079082 27767 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:54.081326 27767 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:54.074265 27762 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.082818 27762 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:54.077060 27598 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:54.086406 27598 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:54.072717 27664 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:09:54.088501 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:54.073788 27502 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:54.075022 27512 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:54.077346 27438 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:54.091732 27766 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:54.103840 27762 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:54.098692 27158 consensus_meta.cc:287] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.658s	user 0.005s	sys 0.006s
W20260709 13:09:54.113294 27757 log.cc:927] Time spent T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 13:09:54.113108 27158 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:54.072796 27700 consensus_queue.cc:237] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:54.115298 27680 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.947s	user 0.023s	sys 0.026s
W20260709 13:09:54.115670 27680 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.948s	user 0.023s	sys 0.026s
I20260709 13:09:54.117488 27500 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:54.071969 27677 consensus_queue.cc:237] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:54.099243 27588 consensus_meta.cc:287] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.645s	user 0.011s	sys 0.001s
I20260709 13:09:54.124056 27588 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:54.125175 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:54.125686 27500 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.125805 22876 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 13.
I20260709 13:09:54.126220 27500 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:54.126632 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:54.126777 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:54.127415 22873 raft_consensus.cc:2410] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:09:54.128021 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:54.128785 27500 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:54.122225 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:54.129822 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:54.128809 22878 raft_consensus.cc:2410] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
W20260709 13:09:54.132616 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 132059us
I20260709 13:09:54.129220 27588 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:54.129525 27664 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.121167 27586 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:54.133626 27356 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.141896 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:54.145372 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:54.134433 27723 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:54.161618 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:54.162273 22955 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 12.
I20260709 13:09:54.164184 27601 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:54.145421 27664 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:54.150580 27356 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:54.167419 27179 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:54.169399 27179 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:54.169516 27723 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:54.171271 27249 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:54.171904 27505 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:54.173568 27249 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:54.178260 27604 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.178823 27604 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:54.179484 27670 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:54.180095 23623 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
I20260709 13:09:54.180251 27356 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:54.182382 23623 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 962) took 1156 ms. Trace:
I20260709 13:09:54.182705 23623 rpcz_store.cc:276] 0709 13:09:53.026275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:53.026404 (+   129us) service_pool.cc:224] Handling call
0709 13:09:54.182231 (+1155827us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54005901d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:54.182329 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2627812736}
I20260709 13:09:54.185146 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:54.185920 22878 raft_consensus.cc:2393] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 12: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:54.186573 27589 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:54.190306 27752 log.cc:927] Time spent T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.249s	user 0.002s	sys 0.000s
I20260709 13:09:54.192301 23029 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 2037) took 2449 ms. Trace:
I20260709 13:09:54.192749 23029 rpcz_store.cc:276] 0709 13:09:51.748758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:51.978151 (+229393us) service_pool.cc:224] Handling call
0709 13:09:52.000154 (+ 22003us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:52.064600 (+ 64446us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:52.064650 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:52.065269 (+   619us) log.cc:844] Serialized 26 byte log entry
0709 13:09:53.655690 (+1590421us) spinlock_profiling.cc:243] Waited 37.4 ms on lock 0x7b180027ff40. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f613598d4bc 00007f6132cdb48b 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf7c02 00007f613597f20b 00007f6135984d23 00007f61359850d1
0709 13:09:53.655846 (+   156us) raft_consensus.cc:1645] Marking committed up to 5
0709 13:09:53.655880 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:54.191992 (+536112us) spinlock_profiling.cc:243] Waited 13.4 ms on lock 0x7b5400359654. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:54.192046 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:54.192092 (+    46us) raft_consensus.cc:1685] finished
0709 13:09:54.192105 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:54.192246 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":537,"raft.queue_time_us":250143,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":43255,"spinlock_wait_cycles":116816128,"thread_start_us":246230,"threads_started":1,"wal-append.queue_time_us":355875,"wal-append.run_cpu_time_us":2478,"wal-append.run_wall_time_us":1172719}
I20260709 13:09:54.156745 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:54.196772 22948 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 12.
I20260709 13:09:54.201076 22948 raft_consensus.cc:1240] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:09:54.203903 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:54.206871 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:54.207469 22949 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:09:54.207870 23588 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:54.198872 22844 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:54.210364 22844 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:54.223249 23588 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:54.229943 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:54.231436 22875 raft_consensus.cc:1240] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 13. Current term is 14. Ops: []
I20260709 13:09:54.233168 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:54.233539 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:54.233796 22955 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 13.
I20260709 13:09:54.235994 27769 raft_consensus.cc:2804] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:09:54.236559 27769 raft_consensus.cc:697] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:54.237710 27769 consensus_queue.cc:237] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:54.231937 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:54.241034 23020 raft_consensus.cc:2468] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 10.
I20260709 13:09:54.223093 27586 raft_consensus.cc:2804] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:54.245267 23588 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:54.249311 23617 raft_consensus.cc:1240] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 14 LEADER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14. Ops: []
I20260709 13:09:54.248822 27179 raft_consensus.cc:2804] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:54.250506 27179 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:54.250878 27179 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:54.247268 23588 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:54.232191 27588 consensus_queue.cc:1048] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 13:09:54.257772 27670 raft_consensus.cc:2749] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:54.247795 27586 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:54.258697 23588 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:54.260113 22949 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 16.
I20260709 13:09:54.262375 27586 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:54.264626 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:54.296533 22955 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 13.
I20260709 13:09:54.258145 27773 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.298712 23029 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 14.
I20260709 13:09:54.258846 27590 raft_consensus.cc:2804] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:54.295882 27356 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.305739 27356 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:54.308188 27356 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:54.261704 27746 consensus_queue.cc:1059] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:09:54.302386 27588 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:54.310465 27588 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:54.313179 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:54.268994 27589 raft_consensus.cc:2804] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:54.316442 27588 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:54.317063 27589 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:54.317538 27589 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:54.317977 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:54.299898 27158 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:54.319720 23029 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 13.
I20260709 13:09:54.304257 27590 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:54.322553 27590 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:54.304154 27773 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:54.324313 27601 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:54.324947 27601 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:09:54.325577 27773 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:54.327137 27601 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:54.328649 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:54.328856 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:54.248237 27768 consensus_queue.cc:1059] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 13:09:54.351701 22946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2810) took 1623 ms. Trace:
I20260709 13:09:54.352177 22946 rpcz_store.cc:276] 0709 13:09:52.728028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:53.097921 (+369893us) service_pool.cc:224] Handling call
0709 13:09:53.098163 (+   242us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:53.368393 (+270230us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:54.348991 (+980598us) spinlock_profiling.cc:243] Waited 269 ms on lock 0x7b540037afd4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:54.351641 (+  2650us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":619503488}
I20260709 13:09:54.329865 27500 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:54.359138 27771 log.cc:927] Time spent T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 13:09:54.355783 27768 consensus_queue.cc:1243] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer aed3f089ec8b49af9c8cf5cdee76e8e7 log is divergent from this leader: its last log entry 13.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:09:54.368975 27505 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:54.357383 27500 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:54.353621 27598 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:54.374568 22877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 2123) took 2183 ms. Trace:
I20260709 13:09:54.379106 27651 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:54.379412 27356 consensus_queue.cc:1059] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:09:54.380414 27719 raft_consensus.cc:3055] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:09:54.380841 27719 raft_consensus.cc:740] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:54.381726 27719 consensus_queue.cc:260] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:54.382625 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:54.383634 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:54.383766 27639 raft_consensus.cc:3055] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:09:54.384225 27639 raft_consensus.cc:740] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:54.385151 27678 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:54.385262 22948 raft_consensus.cc:1240] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:09:54.385185 22950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58320 (request call id 2814) took 1656 ms. Trace:
I20260709 13:09:54.386186 22950 rpcz_store.cc:276] 0709 13:09:52.729062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:52.987678 (+258616us) service_pool.cc:224] Handling call
0709 13:09:54.384965 (+1397287us) spinlock_profiling.cc:243] Waited 56.3 ms on lock 0x7b540035d754. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:54.385099 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":129522944}
I20260709 13:09:54.383651 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:54.387795 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:54.388438 22873 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 10
I20260709 13:09:54.392741 22879 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 13.
I20260709 13:09:54.393046 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:54.394905 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:09:54.395123 23022 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32794 (request call id 765) took 1653 ms (client timeout 1579 ms). Trace:
W20260709 13:09:54.395558 23022 rpcz_store.cc:269] 0709 13:09:52.742127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:52.742246 (+   119us) service_pool.cc:224] Handling call
0709 13:09:54.394964 (+1652718us) spinlock_profiling.cc:243] Waited 260 ms on lock 0x7b5400248e54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:54.395061 (+    97us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":598521600}
I20260709 13:09:54.396777 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:54.397569 23617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 967) took 1111 ms. Trace:
I20260709 13:09:54.397975 23617 rpcz_store.cc:276] 0709 13:09:53.286529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:53.287193 (+   664us) service_pool.cc:224] Handling call
0709 13:09:53.466022 (+178829us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:54.249975 (+783953us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:54.397358 (+147383us) spinlock_profiling.cc:243] Waited 783 ms on lock 0x7b5400572454. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:54.397511 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1800817408}
I20260709 13:09:54.398541 23624 raft_consensus.cc:1240] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 10 LEADER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 9. Current term is 10. Ops: [9.8-9.8]
I20260709 13:09:54.399277 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:54.399807 23617 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:09:54.401633 27664 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:54.402446 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:54.403604 22947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2816) took 1532 ms. Trace:
I20260709 13:09:54.404002 22947 rpcz_store.cc:276] 0709 13:09:52.871563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:53.108063 (+236500us) service_pool.cc:224] Handling call
0709 13:09:53.178329 (+ 70266us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:53.182324 (+  3995us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:54.403399 (+1221075us) spinlock_profiling.cc:243] Waited 69.8 ms on lock 0x7b54003d1b50. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:54.403547 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":160596224}
I20260709 13:09:54.374352 27179 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:54.375110 22877 rpcz_store.cc:276] 0709 13:09:52.191560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:52.195250 (+  3690us) service_pool.cc:224] Handling call
0709 13:09:52.957364 (+762114us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:53.410015 (+452651us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:53.410058 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:53.410567 (+   509us) log.cc:844] Serialized 26 byte log entry
0709 13:09:53.531316 (+120749us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:53.531359 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:53.531424 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:53.545946 (+ 14522us) raft_consensus.cc:1685] finished
0709 13:09:53.545967 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:54.374252 (+828285us) spinlock_profiling.cc:243] Waited 746 ms on lock 0x7b540022a6d0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:54.374496 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":453465,"raft.queue_time_us":453331,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":15795,"spinlock_wait_cycles":1715531648,"thread_start_us":119466,"threads_started":1,"wal-append.queue_time_us":129396}
I20260709 13:09:54.397603 23618 raft_consensus.cc:1240] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14. Ops: []
I20260709 13:09:54.408651 22993 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:54.411587 22873 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 10.
I20260709 13:09:54.412995 22920 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:54.372311 27598 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:54.414969 23590 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:54.416164 22920 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:09:54.420986 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:54.421255 22920 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:54.445032 23028 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 2085) took 2129 ms. Trace:
I20260709 13:09:54.445644 23028 rpcz_store.cc:276] 0709 13:09:52.315689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:52.586721 (+271032us) service_pool.cc:224] Handling call
0709 13:09:53.307661 (+720940us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:53.308514 (+   853us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:54.444755 (+1136241us) spinlock_profiling.cc:243] Waited 721 ms on lock 0x7b540010bb50. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:54.444961 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1657200384}
I20260709 13:09:54.447459 23625 raft_consensus.cc:1240] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:54.436185 22920 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:54.425637 27664 raft_consensus.cc:2804] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:54.424471 27601 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.424791 27697 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:54.454378 22920 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:54.463004 22920 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:54.463977 22920 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:54.427954 27773 raft_consensus.cc:2804] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:54.444706 27249 consensus_queue.cc:1059] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 13:09:54.425195 27733 raft_consensus.cc:2749] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:54.435901 27520 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:09:54.505221 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:54.505352 27520 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:54.505617 27179 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:54.500592 23624 raft_consensus.cc:1240] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 LEADER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 12. Current term is 13. Ops: []
W20260709 13:09:54.479928 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:54.466328 27601 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:54.424073 27590 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:54.490468 27550 raft_consensus.cc:2749] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:54.504302 27773 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:54.507563 27773 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:54.507735 27550 consensus_queue.cc:1059] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:09:54.509196 23622 raft_consensus.cc:2368] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 12. Current term is 13.
I20260709 13:09:54.510007 22872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2660) took 1836 ms. Trace:
I20260709 13:09:54.508731 27601 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:54.455875 27586 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:54.510391 22872 rpcz_store.cc:276] 0709 13:09:52.673941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:52.756282 (+ 82341us) service_pool.cc:224] Handling call
0709 13:09:52.981139 (+224857us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:52.982043 (+   904us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:54.509831 (+1527788us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b54002d6750. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:54.509953 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":497092864}
I20260709 13:09:54.512146 27627 raft_consensus.cc:3055] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:09:54.512501 27627 raft_consensus.cc:740] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:54.505831 27520 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:54.458583 27770 raft_consensus.cc:2749] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:09:54.512971 23621 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 879) took 1468 ms. Trace:
I20260709 13:09:54.513319 27627 consensus_queue.cc:260] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:54.458757 27664 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:54.514909 27664 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:54.517045 27586 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:54.517078 27770 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.517802 27770 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:54.457211 27639 consensus_queue.cc:260] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:54.519826 27639 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:54.520079 27770 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:54.489190 27723 raft_consensus.cc:2804] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:54.521322 27723 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:54.521673 27723 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:54.521751 27770 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:54.522356 27770 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:54.524382 27520 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:54.504726 27249 consensus_queue.cc:1243] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer 5b346767a89745fa82928a5dac9cc0b2 log is divergent from this leader: its last log entry 14.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:09:54.525656 27770 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:54.526201 27770 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:54.489811 27662 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:54.513374 23621 rpcz_store.cc:276] 0709 13:09:53.044973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:53.045081 (+   108us) service_pool.cc:224] Handling call
0709 13:09:53.046197 (+  1116us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:54.055266 (+1009069us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:54.512752 (+457486us) spinlock_profiling.cc:243] Waited 1 s on lock 0x7b54001d4a54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:54.512921 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2308034944}
I20260709 13:09:54.527557 22877 raft_consensus.cc:1240] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14. Ops: []
W20260709 13:09:54.528363 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:54.462329 27697 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:54.536857 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:54.537808 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:54.537972 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:54.535084 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:54.568567 22877 raft_consensus.cc:2368] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 12. Current term is 13.
I20260709 13:09:54.538429 22875 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 15.
I20260709 13:09:54.538976 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:54.586084 22872 raft_consensus.cc:1240] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:09:54.587826 23028 raft_consensus.cc:1275] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 13:09:54.588990 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:54.589600 23028 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
I20260709 13:09:54.589795 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:54.590647 23029 raft_consensus.cc:2368] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 15. Current term is 16.
I20260709 13:09:54.526612 27770 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:54.500315 27646 raft_consensus.cc:2804] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:09:54.533696 27504 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.589200 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:54.603158 23022 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
I20260709 13:09:54.605947 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:54.606920 23022 raft_consensus.cc:2393] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 11: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:54.539244 27697 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:54.571245 27407 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.610545 23029 raft_consensus.cc:1275] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:09:54.609753 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:54.615628 23024 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 13.
I20260709 13:09:54.593045 27709 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:54.613267 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:54.612910 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:54.624244 23021 raft_consensus.cc:2410] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:09:54.554035 27589 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:54.621033 22879 raft_consensus.cc:1240] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:54.571527 27720 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.539367 27586 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:54.566699 27662 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:54.610378 27646 raft_consensus.cc:697] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:54.656688 22994 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:54.657910 27520 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:54.659125 22994 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:54.660362 23623 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 13.
I20260709 13:09:54.661420 27684 raft_consensus.cc:2749] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:09:54.662019 27782 log.cc:927] Time spent T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
I20260709 13:09:54.618101 27590 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:54.629655 27586 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:54.630162 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:54.628917 27720 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:54.617975 27508 consensus_queue.cc:1059] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.011s
I20260709 13:09:54.640955 22876 raft_consensus.cc:1240] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:54.610320 27504 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:54.641237 23024 raft_consensus.cc:1275] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:09:54.641747 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:54.673592 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:54.642355 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:54.643173 22874 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
I20260709 13:09:54.643944 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:54.645589 22879 raft_consensus.cc:1275] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:09:54.653520 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:54.661458 27589 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:54.676357 22994 leader_election.cc:400] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 election: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 12. Current term is 13.
I20260709 13:09:54.652830 27662 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:54.657290 27646 consensus_queue.cc:237] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:54.680792 27508 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:54.681368 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:54.681294 27508 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:54.682029 22876 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:09:54.527720 27356 consensus_queue.cc:1175] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "5b346767a89745fa82928a5dac9cc0b2" responder_term: 10 status { last_received { term: 10 index: 8 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 9. Current term is 10. Ops: [9.8-9.8]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:09:54.669286 23029 raft_consensus.cc:2410] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:09:54.677011 22994 leader_election.cc:403] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
W20260709 13:09:54.684818 27788 log.cc:927] Time spent T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 13:09:54.685501 27684 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:54.686452 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:54.687345 27508 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:54.688936 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:54.689133 27773 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:54.690490 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:54.691344 27773 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:54.691768 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:54.680034 23617 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 16.
I20260709 13:09:54.675467 27705 consensus_queue.cc:1059] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 13:09:54.692978 27684 raft_consensus.cc:2749] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 12. Current term is 13.
I20260709 13:09:54.693396 27705 consensus_queue.cc:1059] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:09:54.694569 27705 raft_consensus.cc:3055] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:09:54.694999 27705 raft_consensus.cc:740] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:54.704624 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:54.705291 22947 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:54.707049 27770 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:54.707563 27723 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:54.678126 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:54.709286 27504 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:54.709059 27705 consensus_queue.cc:260] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:54.709604 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:54.618913 27407 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:54.710683 27705 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:54.710698 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:54.711514 23022 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
I20260709 13:09:54.712230 27407 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:54.662812 27590 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:54.713114 27590 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:54.714073 27723 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:54.715371 27590 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:54.716472 22947 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:09:54.712289 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:54.701805 22843 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:54.702111 23024 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 12.
I20260709 13:09:54.703670 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:54.696643 22918 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:54.719624 22950 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
I20260709 13:09:54.721146 22918 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:54.723469 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:54.723276 22918 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:54.724328 22950 raft_consensus.cc:2393] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 15: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:54.718421 22880 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11.
W20260709 13:09:54.731256 27765 log.cc:927] Time spent T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.588s	user 0.001s	sys 0.000s
I20260709 13:09:54.731313 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:54.731684 27662 raft_consensus.cc:2749] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:54.726599 22918 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:54.732714 23619 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60044 (request call id 876) took 1009 ms. Trace:
I20260709 13:09:54.735167 23619 rpcz_store.cc:276] 0709 13:09:53.723314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:53.723437 (+   123us) service_pool.cc:224] Handling call
0709 13:09:53.723761 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:53.894514 (+170753us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:53.894568 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:53.901986 (+  7418us) log.cc:844] Serialized 26 byte log entry
0709 13:09:54.011357 (+109371us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:09:54.011407 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:54.194419 (+183012us) spinlock_profiling.cc:243] Waited 97.3 ms on lock 0x7b54005aebd4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:54.194463 (+    44us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:54.730734 (+536271us) spinlock_profiling.cc:243] Waited 5.84 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:09:54.732496 (+  1762us) raft_consensus.cc:1685] finished
0709 13:09:54.732518 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:54.732644 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":81220,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":11430,"spinlock_wait_cycles":237298432,"thread_start_us":28241,"threads_started":1,"wal-append.queue_time_us":173801}
I20260709 13:09:54.734428 22876 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 16.
I20260709 13:09:54.736375 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:54.737072 22950 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 13.
I20260709 13:09:54.737637 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:54.679527 27520 raft_consensus.cc:3055] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:09:54.736016 27505 consensus_queue.cc:1048] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:09:54.740594 22918 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:54.740931 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:54.693384 27720 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:54.743112 22918 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:54.738931 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:54.626590 27709 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:54.739699 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:54.725317 27770 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:54.733501 27707 raft_consensus.cc:3055] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:09:54.736097 22843 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:54.745030 22950 raft_consensus.cc:2410] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:09:54.745154 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:54.732700 27670 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:54.740617 27520 raft_consensus.cc:740] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:54.695973 27249 consensus_queue.cc:1059] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 13:09:54.749185 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:54.749528 22918 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
W20260709 13:09:54.746018 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 244417us
I20260709 13:09:54.750057 22951 raft_consensus.cc:2393] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 14: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:54.741164 27662 raft_consensus.cc:2749] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:09:54.732555 27723 raft_consensus.cc:2749] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:54.749889 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:54.751149 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:54.751775 22843 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:54.744257 27508 raft_consensus.cc:2749] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:09:54.725620 27664 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:54.769060 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:54.748824 27275 raft_consensus.cc:2749] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 13:09:54.770870 27596 raft_consensus.cc:2804] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:54.771041 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:54.773169 22954 raft_consensus.cc:1240] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 11. Current term is 12. Ops: []
I20260709 13:09:54.752267 27504 raft_consensus.cc:2749] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:54.789333 22918 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:54.775467 22955 raft_consensus.cc:1240] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 14. Current term is 15. Ops: []
I20260709 13:09:54.776079 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:54.777871 22953 raft_consensus.cc:1240] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 LEADER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:54.790578 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:54.794219 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:54.746518 27707 raft_consensus.cc:740] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:09:54.788409 22992 leader_election.cc:336] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Timed out: RequestConsensusVote RPC to 127.21.63.2:33309 timed out after 2.346s (SENT)
I20260709 13:09:54.748117 27670 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:54.792802 23023 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:54.755442 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:54.742193 27773 consensus_queue.cc:1048] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:09:54.795624 22953 raft_consensus.cc:2393] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 9: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:54.795727 22992 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:54.799748 22951 raft_consensus.cc:2410] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 14 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:09:54.799067 22955 raft_consensus.cc:1275] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:09:54.801075 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:54.802158 23021 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 9.
I20260709 13:09:54.801880 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:54.800253 22843 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:54.803973 22955 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:54.803606 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:54.806357 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:54.806499 22947 raft_consensus.cc:1240] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:54.806510 22992 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:54.807286 22950 raft_consensus.cc:2410] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 13:09:54.793356 27596 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:54.807736 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:54.795142 27683 raft_consensus.cc:2749] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:54.808429 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:54.797614 27505 raft_consensus.cc:2804] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:54.798933 27670 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:54.798638 27662 raft_consensus.cc:2749] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:09:54.818535 22955 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
I20260709 13:09:54.820567 27629 consensus_queue.cc:1048] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:09:54.821676 27627 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:54.784413 27520 consensus_queue.cc:260] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:54.822548 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:54.823588 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:54.824107 22955 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
I20260709 13:09:54.824864 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:54.805572 27684 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:54.803596 27249 raft_consensus.cc:2749] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:54.807534 27698 raft_consensus.cc:2749] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:54.839515 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:54.842097 22953 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
I20260709 13:09:54.807945 27504 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:54.812207 27596 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:54.849128 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:54.851691 22950 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:54.848351 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:54.814874 27505 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:54.805038 27566 consensus_queue.cc:1059] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:09:54.835953 27520 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:54.865759 23023 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
I20260709 13:09:54.812346 27588 consensus_queue.cc:1048] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 13:09:54.859053 27505 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:09:54.869041 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 59820.7us
I20260709 13:09:54.882586 27588 consensus_queue.cc:1048] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:09:54.887118 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:54.847810 27719 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:54.905447 27289 consensus_queue.cc:1059] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:09:54.906358 27289 consensus_queue.cc:1048] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:09:54.904170 27505 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:54.911266 27505 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:54.921185 27586 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:54.924774 27698 consensus_queue.cc:1048] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:09:54.925721 27698 consensus_queue.cc:1243] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer aed3f089ec8b49af9c8cf5cdee76e8e7 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:09:54.928241 27664 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:54.937681 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:54.941533 22876 raft_consensus.cc:1275] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 8. (term mismatch)
I20260709 13:09:54.942004 22876 pending_rounds.cc:85] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 7
I20260709 13:09:54.942260 22876 pending_rounds.cc:107] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 13:09:54.942521 22876 raft_consensus.cc:2889] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:54.887797 23025 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 13.
I20260709 13:09:54.887988 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:54.844841 27504 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:54.953912 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:54.955318 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:54.889604 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:54.889659 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:54.842365 27684 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:54.901224 22955 raft_consensus.cc:2410] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
I20260709 13:09:54.957820 27504 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:09:54.961999 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:54.963799 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:54.962990 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:54.966598 22949 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:54.966495 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:09:54.971638 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 138650us
I20260709 13:09:54.972421 23588 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:54.971756 22950 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 13.
I20260709 13:09:54.973814 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:54.974484 22950 raft_consensus.cc:2393] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 14: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:54.979005 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:54.981864 23020 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
I20260709 13:09:54.983165 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:54.983922 23020 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 13.
I20260709 13:09:54.985562 23588 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:54.988070 23588 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:54.915835 27697 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:09:54.993177 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:54.993368 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:54.994292 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:54.991958 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 183366us
I20260709 13:09:54.998447 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:54.999506 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:55.000108 23619 raft_consensus.cc:2468] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 10.
I20260709 13:09:54.847944 27707 consensus_queue.cc:260] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:54.985131 27773 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:54.992365 27768 raft_consensus.cc:2804] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:55.001107 22954 raft_consensus.cc:2368] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 13. Current term is 14.
I20260709 13:09:55.002987 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:55.001190 23025 raft_consensus.cc:2410] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 13:09:55.004827 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:55.011089 22950 raft_consensus.cc:2368] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 13. Current term is 14.
I20260709 13:09:55.010921 23023 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:09:55.011548 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 106531us
I20260709 13:09:55.005334 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:55.006173 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:55.012916 22992 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:55.006598 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:55.001157 23620 raft_consensus.cc:2368] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 9. Current term is 10.
I20260709 13:09:54.992436 27590 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:09:54.995258 27730 raft_consensus.cc:2749] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 13:09:55.014220 23619 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15.
I20260709 13:09:55.018051 23020 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 8.
I20260709 13:09:55.019872 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:55.019217 22992 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:55.020625 22950 raft_consensus.cc:2393] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 13: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:55.022094 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:55.023288 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:55.000113 27179 consensus_queue.cc:1059] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 13:09:55.024706 27684 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:55.021220 22949 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 13.
I20260709 13:09:55.030936 22949 raft_consensus.cc:1240] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 14. Current term is 15. Ops: [14.13-14.13]
I20260709 13:09:55.012864 27768 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:55.023599 22992 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:55.036324 22949 raft_consensus.cc:1240] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:09:55.038311 22992 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:55.016602 27709 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:55.042702 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:55.044507 22918 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.792s (SENT) [suppressed 6 similar messages]
I20260709 13:09:55.045702 22992 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:55.022006 27407 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:55.051332 27697 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:55.058287 27639 consensus_meta.cc:287] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.538s	user 0.011s	sys 0.000s
I20260709 13:09:55.082885 23625 raft_consensus.cc:1217] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 14 LEADER]: Deduplicated request from leader. Original: 10.9->[11.10-13.11]   Dedup: 13.11->[]
I20260709 13:09:55.083444 23625 raft_consensus.cc:1240] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 14 LEADER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14. Ops: [11.10-13.11]
I20260709 13:09:55.015285 27773 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:55.026216 27705 raft_consensus.cc:2804] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:09:55.037050 27802 consensus_queue.cc:1059] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:09:55.041927 27768 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:55.090197 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:55.088620 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:55.094012 23023 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 14.
I20260709 13:09:55.094602 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:55.095515 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:55.096231 23616 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 14.
I20260709 13:09:55.097671 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:55.098259 23621 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 9.
I20260709 13:09:55.066869 23024 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 9.
I20260709 13:09:55.070551 27770 raft_consensus.cc:2749] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:55.101370 23616 raft_consensus.cc:1240] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:55.073231 22919 leader_election.cc:400] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 election: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 12. Current term is 13.
I20260709 13:09:55.102448 22919 leader_election.cc:403] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 13:09:55.075105 27698 raft_consensus.cc:2804] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:55.075726 27500 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:55.079211 27670 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:09:55.029378 27707 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:09:55.081269 22994 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:55.125592 23622 raft_consensus.cc:1275] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:09:55.128000 22952 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11.
I20260709 13:09:55.044097 27684 raft_consensus.cc:2804] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:55.084950 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:55.058645 27588 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:55.048593 27664 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:55.100910 23625 raft_consensus.cc:2393] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 11: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:55.107731 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
W20260709 13:09:55.106107 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 60490.9us
I20260709 13:09:55.104873 23588 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:55.110652 22919 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:55.134976 22919 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:55.101445 27705 raft_consensus.cc:697] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 10 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:09:55.130528 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:55.104765 27484 tablet_copy_client.cc:538] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:09:55.056584 27596 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:55.134128 23617 raft_consensus.cc:1240] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:55.105386 27698 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:55.146170 27698 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:55.105625 27697 raft_consensus.cc:2749] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 12. Current term is 13.
I20260709 13:09:55.150395 27596 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:55.151983 27768 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:55.140939 27770 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:55.153101 27770 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:55.153558 27770 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:09:55.154327 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:55.154806 27768 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:55.144639 27500 consensus_queue.cc:1059] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.012s
I20260709 13:09:55.140883 27793 raft_consensus.cc:2804] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:09:55.159011 27793 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:55.159373 27793 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:55.159735 27746 raft_consensus.cc:2749] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:09:55.164801 22919 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:55.167636 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:55.168834 27627 consensus_queue.cc:1048] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:09:55.170286 27627 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:55.171141 27705 consensus_queue.cc:237] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:55.122737 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 125652us
I20260709 13:09:55.136478 23588 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:55.174300 27773 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:55.174718 23588 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:55.176141 27739 raft_consensus.cc:2804] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:55.176461 27724 raft_consensus.cc:3055] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:09:55.177003 27724 raft_consensus.cc:740] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:55.177538 27641 raft_consensus.cc:2804] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:09:55.177876 27724 consensus_queue.cc:260] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:55.178735 27746 raft_consensus.cc:3055] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:09:55.178942 27724 raft_consensus.cc:3060] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:55.179225 27746 raft_consensus.cc:740] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:55.180416 27746 consensus_queue.cc:260] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:55.184289 27720 raft_consensus.cc:3055] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:09:55.184747 27720 raft_consensus.cc:740] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:55.184619 27807 consensus_queue.cc:1059] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:09:55.185745 27720 consensus_queue.cc:260] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:55.186115 27807 raft_consensus.cc:3055] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:09:55.186829 27807 raft_consensus.cc:740] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:55.187592 27500 raft_consensus.cc:3055] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:09:55.188076 27500 raft_consensus.cc:740] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:55.187950 27807 consensus_queue.cc:260] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:55.189182 27500 consensus_queue.cc:260] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:55.189857 27807 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:55.190496 27500 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:55.176530 27739 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:55.192440 27739 raft_consensus.cc:3060] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:55.192520 27736 raft_consensus.cc:3055] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:09:55.193027 27736 raft_consensus.cc:740] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:55.194111 27736 consensus_queue.cc:260] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.13, Last appended by leader: 13, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:55.195565 27736 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:55.131420 27684 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:55.207144 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:55.208112 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 64012.4us
I20260709 13:09:55.220868 22949 raft_consensus.cc:1240] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14. Ops: [11.7-13.8]
I20260709 13:09:55.205379 22843 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:55.226653 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:55.226922 22993 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:55.227278 22949 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11.
I20260709 13:09:55.228196 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:55.225535 22843 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:55.242841 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:55.176141 27670 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:55.224102 27684 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:09:55.254756 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:55.263850 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:55.264539 22948 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:55.256915 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:55.266440 22947 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
I20260709 13:09:55.267619 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:55.259140 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:55.136530 27588 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:55.241852 27683 raft_consensus.cc:2804] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:09:55.256184 23029 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11.
I20260709 13:09:55.232818 27720 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:55.271989 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:55.273885 23022 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:55.228756 27596 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
W20260709 13:09:55.278005 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 233613us
I20260709 13:09:55.278095 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:55.270767 23590 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:55.282459 22948 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
I20260709 13:09:55.226377 27022 consensus_queue.cc:1048] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.044s
W20260709 13:09:55.283975 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:55.269904 27639 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:55.270015 27590 raft_consensus.cc:697] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:55.273054 27670 consensus_queue.cc:1059] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 13:09:55.300491 27670 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:09:55.300990 27670 raft_consensus.cc:697] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 13 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:55.301872 27670 consensus_queue.cc:237] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:55.303154 27407 raft_consensus.cc:2749] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:55.303305 23026 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 781) took 1091 ms. Trace:
I20260709 13:09:55.284842 27022 consensus_queue.cc:1243] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer aed3f089ec8b49af9c8cf5cdee76e8e7 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:09:55.296438 23022 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
I20260709 13:09:55.331449 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:55.287071 27179 raft_consensus.cc:2804] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:09:55.332726 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:55.297096 22953 raft_consensus.cc:1240] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 LEADER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:09:55.333437 23022 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
I20260709 13:09:55.303663 23026 rpcz_store.cc:276] 0709 13:09:54.212311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:54.212441 (+   130us) service_pool.cc:224] Handling call
0709 13:09:54.212854 (+   413us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:54.213309 (+   455us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:54.213349 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:54.229951 (+ 16602us) log.cc:844] Serialized 52 byte log entry
0709 13:09:55.062226 (+832275us) spinlock_profiling.cc:243] Waited 3.6 ms on lock 0x7b18003c2860. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cdae07 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf7c02 00007f613597f20b 00007f6135984d23 00007f61359850d1 00007f6135da0a0f
0709 13:09:55.136692 (+ 74466us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:09:55.136745 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:55.136835 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:55.302992 (+166157us) raft_consensus.cc:1685] finished
0709 13:09:55.303011 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:55.303247 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8548,"raft.queue_time_us":653,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":10594,"spinlock_wait_cycles":8274176,"thread_start_us":73203,"threads_started":1,"wal-append.queue_time_us":1065907}
I20260709 13:09:55.291220 22918 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:55.245178 27698 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:55.291527 27683 raft_consensus.cc:697] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 16 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:55.342264 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:55.343783 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:55.283496 27596 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:55.343087 23026 raft_consensus.cc:2468] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
I20260709 13:09:55.348829 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:55.351272 22951 raft_consensus.cc:2410] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 16 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:09:55.336449 27462 raft_consensus.cc:2804] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:09:55.367967 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 76817.4us
I20260709 13:09:55.373404 22918 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:55.347183 27179 raft_consensus.cc:697] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 10 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:55.374100 27504 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:55.378440 27179 consensus_queue.cc:237] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:55.305889 27639 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:55.424628 23025 raft_consensus.cc:1275] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:09:55.398088 22952 raft_consensus.cc:1240] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 15. Current term is 16. Ops: [15.10-15.10]
W20260709 13:09:55.396895 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:55.386775 23024 raft_consensus.cc:1275] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:09:55.406160 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:55.436992 22952 raft_consensus.cc:1240] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:09:55.437500 23617 raft_consensus.cc:2393] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 14: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:55.383565 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:55.399641 22951 raft_consensus.cc:1240] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:09:55.426159 23623 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60044 (request call id 877) took 1031 ms. Trace:
I20260709 13:09:55.438589 23623 rpcz_store.cc:276] 0709 13:09:54.394425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:54.394640 (+   215us) service_pool.cc:224] Handling call
0709 13:09:55.425990 (+1031350us) spinlock_profiling.cc:243] Waited 265 ms on lock 0x7b540045bfd4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:55.426101 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":608981888}
I20260709 13:09:55.441622 23620 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 9.
W20260709 13:09:55.442764 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:55.443241 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 69887.8us
I20260709 13:09:55.394029 27484 tablet_bootstrap.cc:492] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap starting.
I20260709 13:09:55.370899 27462 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:55.404816 27816 log.cc:927] Time spent T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
W20260709 13:09:55.414577 27801 log.cc:927] Time spent T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.357s	user 0.002s	sys 0.000s
I20260709 13:09:55.395599 27588 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:55.403052 27604 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:55.380568 27504 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:55.427112 27596 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:55.373373 27646 raft_consensus.cc:2804] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:55.479511 23026 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:55.411557 27505 raft_consensus.cc:3055] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:09:55.412151 27730 raft_consensus.cc:3055] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:09:55.449542 27697 raft_consensus.cc:2749] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:55.398588 27698 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:55.389279 27809 log.cc:927] Time spent T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 13:09:55.428917 27793 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:55.459108 27627 raft_consensus.cc:2749] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:55.480223 22918 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:55.450726 27664 raft_consensus.cc:2749] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:55.407120 27289 consensus_queue.cc:1059] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.081s
I20260709 13:09:55.484197 27462 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:55.399974 27683 consensus_queue.cc:237] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:55.542932 22948 raft_consensus.cc:1240] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 10. Current term is 11. Ops: []
W20260709 13:09:55.544246 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 64137.2us
I20260709 13:09:55.545320 22918 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:55.547973 22918 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:55.549309 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:55.550212 22949 raft_consensus.cc:2368] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 9. Current term is 10.
I20260709 13:09:55.550890 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:55.553352 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:55.554371 22871 raft_consensus.cc:2410] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 16 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 13:09:55.555606 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:55.531154 27464 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.033s	sys 0.054s
I20260709 13:09:55.513073 27505 raft_consensus.cc:740] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:55.561995 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:55.511880 27770 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:55.589552 27770 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:55.557307 27627 raft_consensus.cc:2749] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:55.563426 27803 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:55.591501 27803 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:55.523311 27730 raft_consensus.cc:740] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:55.594026 22993 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:55.593779 27730 consensus_queue.cc:260] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:55.514582 27814 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:55.595788 27805 raft_consensus.cc:2804] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:09:55.595410 27814 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:55.595292 27730 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:55.598062 27814 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:55.596457 27805 raft_consensus.cc:697] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:55.600081 27805 consensus_queue.cc:237] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:55.610121 27803 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:55.560434 22918 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:55.410221 27590 consensus_queue.cc:237] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:55.613129 23029 raft_consensus.cc:1275] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 13:09:55.613607 23029 pending_rounds.cc:85] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75: Aborting all ops after (but not including) 7
I20260709 13:09:55.613871 23029 pending_rounds.cc:107] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 13:09:55.614161 23029 raft_consensus.cc:2889] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:55.564342 22994 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:09:55.617650 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 53436.5us
I20260709 13:09:55.619434 22994 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:55.565685 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:55.566828 27684 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:55.573228 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:55.577189 27505 consensus_queue.cc:260] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:55.561882 27464 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.034s	sys 0.054s
I20260709 13:09:55.515787 27646 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:55.584962 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:55.633376 22872 raft_consensus.cc:2393] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 13: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:55.585826 23026 raft_consensus.cc:2410] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:09:55.587215 27793 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:55.587612 27747 log.cc:927] Time spent T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 1.647s	user 0.002s	sys 0.003s
I20260709 13:09:55.620468 22879 raft_consensus.cc:2393] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 13: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:55.640056 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:55.640900 23623 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 FOLLOWER]: Advancing to term 10
W20260709 13:09:55.627156 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 66830.2us
I20260709 13:09:55.642355 22918 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:55.536785 27641 raft_consensus.cc:697] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 14 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:55.631985 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:55.638470 22994 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:55.651480 23625 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:55.652201 22994 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:55.563237 27596 raft_consensus.cc:2749] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:09:55.650456 22918 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:55.656404 23022 raft_consensus.cc:2393] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 13: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:55.625064 27520 raft_consensus.cc:2804] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:55.662479 22994 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:55.633188 27646 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:55.659677 22918 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:55.664286 22918 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:55.634161 27356 raft_consensus.cc:2764] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election decision vote started in defunct term 12: won
I20260709 13:09:55.669198 22994 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:55.677014 22918 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:55.679539 22918 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:09:55.681397 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:55.682523 23625 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 15.
I20260709 13:09:55.685691 22918 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:55.656986 27662 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:09:55.715682 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 448756us
I20260709 13:09:55.682337 23623 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
W20260709 13:09:55.694643 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:55.649147 27664 raft_consensus.cc:2749] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:55.658815 27627 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:09:55.683102 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 505766us
W20260709 13:09:55.708139 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 152062us
I20260709 13:09:55.672817 27500 raft_consensus.cc:2804] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 13:09:55.661067 27724 raft_consensus.cc:2749] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:55.666004 27770 raft_consensus.cc:2749] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:55.661190 27707 raft_consensus.cc:2749] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:55.752425 27356 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:55.672919 27022 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:09:55.661523 27520 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:55.649734 27720 consensus_queue.cc:1048] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.059s
I20260709 13:09:55.661401 27505 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:55.739599 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:55.741074 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:55.768821 22877 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:55.740317 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:55.694054 27596 raft_consensus.cc:2749] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:55.749286 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:55.777570 22874 raft_consensus.cc:2393] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 14: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:55.751616 22994 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:55.687451 27550 raft_consensus.cc:2749] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:55.687647 27407 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:55.657033 27739 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:55.706513 27766 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:09:55.735296 27803 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:55.758561 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:55.757882 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:55.760569 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:55.787251 22873 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:55.763706 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:55.756279 27793 raft_consensus.cc:2804] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:09:55.765856 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:55.767592 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:55.743788 27500 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:55.758303 27356 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:55.758746 27810 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:55.774796 27736 consensus_meta.cc:287] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.551s	user 0.008s	sys 0.004s
I20260709 13:09:55.774971 22875 raft_consensus.cc:2393] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 14: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:55.706909 27703 consensus_queue.cc:1048] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 13:09:55.780961 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:55.740828 27629 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:55.799376 22948 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:55.790138 22954 raft_consensus.cc:1275] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:09:55.724074 27662 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:55.796241 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:55.764968 22844 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:55.801450 22994 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:55.803258 22950 raft_consensus.cc:2393] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 10: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:55.773497 27520 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:55.812877 27826 consensus_queue.cc:1048] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:09:55.835970 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:55.836292 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:55.813278 27810 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:55.813760 27629 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:55.803082 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:55.816401 27684 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:55.818388 27356 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:55.822026 27739 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:55.797219 27766 raft_consensus.cc:697] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:55.829427 22873 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 14.
I20260709 13:09:55.791316 27709 raft_consensus.cc:2804] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:09:55.838779 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:55.839455 22877 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 14.
I20260709 13:09:55.791313 27793 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:55.755716 27818 consensus_queue.cc:1059] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.058s
I20260709 13:09:55.842087 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:55.864449 22876 raft_consensus.cc:2393] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 14: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:55.842427 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:55.841511 27664 raft_consensus.cc:2749] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:09:55.843699 22948 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 14.
W20260709 13:09:55.845191 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 43800.5us
I20260709 13:09:55.843506 27814 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:55.798911 27500 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:55.871984 27818 consensus_queue.cc:1243] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer 5b346767a89745fa82928a5dac9cc0b2 log is divergent from this leader: its last log entry 14.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:09:55.872560 27818 raft_consensus.cc:3055] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:09:55.872915 27818 raft_consensus.cc:740] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:55.873713 27818 consensus_queue.cc:260] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:55.874891 27818 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:55.879324 27500 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
W20260709 13:09:55.792757 27764 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.718s	user 0.011s	sys 0.010s
W20260709 13:09:55.880887 27764 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.807s	user 0.012s	sys 0.010s
I20260709 13:09:55.882083 27500 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:55.886394 27586 consensus_queue.cc:1048] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:09:55.887248 22843 leader_election.cc:400] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 15. Current term is 16.
I20260709 13:09:55.887569 22843 leader_election.cc:403] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
W20260709 13:09:55.890103 27828 log.cc:927] Time spent T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:09:55.892143 27678 raft_consensus.cc:2749] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 15. Current term is 16.
I20260709 13:09:55.777456 27641 consensus_queue.cc:237] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:55.897909 27807 consensus_queue.cc:1048] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:09:55.900339 22843 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:55.858762 22994 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:55.862087 22873 raft_consensus.cc:1275] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 13:09:55.903998 22994 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:55.904448 22994 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:55.805374 23588 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:09:55.906746 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 101449us
I20260709 13:09:55.907979 23588 leader_election.cc:400] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 election: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 13. Current term is 14.
I20260709 13:09:55.908332 23588 leader_election.cc:403] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:09:55.909031 27719 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:55.909919 27684 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:55.909737 27719 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:55.910481 27684 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:55.912163 27719 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:55.931766 27639 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:55.844646 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 79754.1us
I20260709 13:09:55.957643 27806 raft_consensus.cc:2749] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:55.963200 22844 leader_election.cc:400] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 9. Current term is 10.
I20260709 13:09:55.963675 22844 leader_election.cc:403] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:09:55.903028 27629 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:55.866398 22875 pending_rounds.cc:85] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 7
I20260709 13:09:55.967592 22875 pending_rounds.cc:107] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 13:09:55.967883 22875 raft_consensus.cc:2889] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:55.857024 22879 raft_consensus.cc:1275] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 13:09:55.793612 27803 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:55.851399 27462 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:55.915925 27810 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:55.855139 27761 raft_consensus.cc:2804] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:55.853996 27709 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:55.926462 27814 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:55.864413 27793 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:55.922036 27746 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:09:55.958268 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 57997.6us
I20260709 13:09:56.017194 22877 raft_consensus.cc:1275] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:09:55.915251 23588 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:56.018057 27814 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:56.018194 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 103033us
I20260709 13:09:55.993924 27709 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:56.019595 27746 raft_consensus.cc:2804] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:09:56.020190 23588 leader_election.cc:400] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 election: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 13. Current term is 14.
I20260709 13:09:55.948120 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:56.021801 22874 pending_rounds.cc:85] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 8
I20260709 13:09:56.022058 22874 pending_rounds.cc:107] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 13:09:56.022287 22874 raft_consensus.cc:2889] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:56.023008 27746 raft_consensus.cc:697] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 13 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:55.912843 27739 raft_consensus.cc:2804] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:09:56.023943 27739 raft_consensus.cc:697] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 13 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:55.984125 27761 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:56.025023 27761 raft_consensus.cc:3060] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:56.024793 27739 consensus_queue.cc:237] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:55.912508 27684 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:55.867085 27823 log.cc:927] Time spent T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 13:09:55.915984 27179 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:55.919009 27662 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:55.965252 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:56.029227 22872 raft_consensus.cc:2468] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:09:55.970067 22844 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:09:56.033604 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 63625.3us
I20260709 13:09:56.033917 22994 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:56.034678 27684 raft_consensus.cc:2804] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:56.035068 27684 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:56.035393 27684 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:56.034672 27818 raft_consensus.cc:2804] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:56.037089 27818 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:56.037441 27818 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:56.017197 22843 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:56.039469 27810 consensus_queue.cc:1059] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:09:56.042052 27504 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:56.042421 27504 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:56.042637 27810 raft_consensus.cc:3055] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:09:56.043462 27810 raft_consensus.cc:740] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:56.044529 27810 consensus_queue.cc:260] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:56.050698 27810 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:09:55.959539 27705 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.715s	user 0.017s	sys 0.026s
W20260709 13:09:56.051436 27705 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.823s	user 0.017s	sys 0.027s
I20260709 13:09:56.051537 27698 raft_consensus.cc:3055] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:09:55.994714 22879 raft_consensus.cc:1240] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14. Ops: [11.7-13.8]
I20260709 13:09:56.052006 27698 raft_consensus.cc:740] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:56.052188 23023 raft_consensus.cc:1240] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 10 LEADER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:55.973239 27830 raft_consensus.cc:2749] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 9. Current term is 10.
I20260709 13:09:56.001045 22876 raft_consensus.cc:1240] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:09:56.053012 27698 consensus_queue.cc:260] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:56.020533 23588 leader_election.cc:403] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:09:56.054400 27698 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:56.054726 27705 consensus_queue.cc:1059] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:09:56.055497 27730 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:56.055998 27705 raft_consensus.cc:3055] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:09:56.056380 27705 raft_consensus.cc:740] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:56.057044 27464 raft_consensus.cc:3055] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:09:56.057482 27464 raft_consensus.cc:740] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:56.057178 27705 consensus_queue.cc:260] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:55.960223 22992 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.740s	user 0.030s	sys 0.089s
W20260709 13:09:56.058456 22992 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.839s	user 0.031s	sys 0.089s
I20260709 13:09:56.058320 27464 consensus_queue.cc:260] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:56.058813 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 839332us
I20260709 13:09:56.059372 27464 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:56.059429 27705 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:56.060277 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:56.060982 23026 raft_consensus.cc:2368] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 13. Current term is 14.
I20260709 13:09:56.061205 23028 raft_consensus.cc:1275] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:09:56.062299 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:56.063031 23025 raft_consensus.cc:2393] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 14: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:56.024302 27746 consensus_queue.cc:237] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:56.057817 27625 consensus_queue.cc:1048] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:09:56.063382 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:56.063957 23022 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:56.066860 23028 raft_consensus.cc:1275] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:09:56.067675 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:56.068296 23625 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 15.
I20260709 13:09:56.058800 23588 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:56.071413 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:56.050289 27766 consensus_queue.cc:237] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:56.082860 23590 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:56.073545 23625 raft_consensus.cc:1240] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:56.074080 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:56.074097 22843 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:56.075891 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:56.076143 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:56.084506 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:56.085445 22948 raft_consensus.cc:2393] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 14: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:56.076985 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:56.084549 22843 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:56.087759 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:56.088307 22843 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:56.077666 23022 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 10.
I20260709 13:09:56.079983 22920 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:55.982668 27697 raft_consensus.cc:2749] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:56.094213 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:56.081352 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:56.077960 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:56.095742 23616 raft_consensus.cc:1240] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 14. Current term is 15. Ops: []
I20260709 13:09:56.096284 22952 raft_consensus.cc:2393] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 14: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:56.090472 22920 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:55.913921 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:56.091639 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:56.092088 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:56.097960 22949 raft_consensus.cc:2393] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 13: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:56.089187 22919 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:56.093214 22954 pending_rounds.cc:85] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting all ops after (but not including) 6
I20260709 13:09:56.098965 22954 pending_rounds.cc:107] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 13:09:56.099368 22954 raft_consensus.cc:2889] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:56.093640 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:56.099152 22919 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:56.100483 22955 raft_consensus.cc:2393] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 15: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:56.104070 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:56.094275 22843 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:56.108031 22950 raft_consensus.cc:1240] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 12. Current term is 13. Ops: [10.9-12.10]
I20260709 13:09:56.042744 27504 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:56.106477 22843 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:56.112493 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:56.106325 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:56.032274 27520 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:56.117192 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:56.135278 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:56.082233 27736 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:56.094878 27815 raft_consensus.cc:2804] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:09:56.099716 27814 raft_consensus.cc:2749] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:56.105909 27500 raft_consensus.cc:2749] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:56.115258 22920 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:56.116452 27651 raft_consensus.cc:2804] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:09:56.141984 27709 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:56.143107 27407 raft_consensus.cc:2749] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:09:56.146648 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:56.146898 27684 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:56.148461 27811 consensus_queue.cc:1048] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:09:56.112269 27550 raft_consensus.cc:2804] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:09:56.151603 22876 raft_consensus.cc:1275] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:09:56.116362 27596 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:09:56.153121 27806 raft_consensus.cc:2804] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:09:56.185186 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 499599us
W20260709 13:09:56.153481 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 47097.1us
I20260709 13:09:56.095094 27768 raft_consensus.cc:2749] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:56.154796 22873 raft_consensus.cc:1275] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:09:56.154507 22993 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:56.140125 22992 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:56.105494 27830 raft_consensus.cc:2804] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:09:56.162367 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 47332.1us
I20260709 13:09:56.135317 27627 raft_consensus.cc:2804] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:09:56.180650 23023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 2058) took 1481 ms. Trace:
I20260709 13:09:56.156162 27356 consensus_queue.cc:1059] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.065s
I20260709 13:09:56.156853 27736 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:56.223209 23029 raft_consensus.cc:1240] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:56.246289 27821 consensus_queue.cc:1048] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:09:56.247007 27832 log.cc:927] Time spent T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 13:09:56.247452 27770 consensus_queue.cc:1048] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:09:56.248210 22953 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2846) took 1028 ms. Trace:
I20260709 13:09:56.245260 23023 rpcz_store.cc:276] 0709 13:09:54.699488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:55.094909 (+395421us) service_pool.cc:224] Handling call
0709 13:09:55.095245 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:56.052632 (+957387us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:56.180333 (+127701us) spinlock_profiling.cc:243] Waited 957 ms on lock 0x7b54001798d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:56.180581 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2200285952}
I20260709 13:09:56.161334 27815 raft_consensus.cc:697] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 14 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:56.197108 22843 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:09:56.250917 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 78795.1us
I20260709 13:09:56.248745 22953 rpcz_store.cc:276] 0709 13:09:55.220093 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:55.340880 (+120787us) service_pool.cc:224] Handling call
0709 13:09:55.341287 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:55.341563 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:55.341599 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:55.347063 (+  5464us) log.cc:844] Serialized 26 byte log entry
0709 13:09:55.865122 (+518059us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:55.865185 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:55.995323 (+130138us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:56.248009 (+252686us) raft_consensus.cc:1685] finished
0709 13:09:56.248028 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:56.248159 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":454232,"threads_started":1,"wal-append.queue_time_us":722767}
I20260709 13:09:56.158139 27179 raft_consensus.cc:2749] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 13. Current term is 14.
W20260709 13:09:56.235566 27837 log.cc:927] Time spent T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
W20260709 13:09:56.266099 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 111654us
W20260709 13:09:56.267565 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 70584.4us
I20260709 13:09:56.200699 27500 raft_consensus.cc:2804] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:09:56.301266 22953 raft_consensus.cc:1240] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 8. Current term is 9. Ops: []
W20260709 13:09:56.302474 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 162451us
I20260709 13:09:56.286070 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:56.303524 22992 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:56.303869 22952 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
I20260709 13:09:56.284376 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:56.172011 27651 raft_consensus.cc:697] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 13 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:56.310256 22993 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:09:56.318425 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:56.320284 22955 raft_consensus.cc:1240] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 13. Current term is 14. Ops: []
I20260709 13:09:56.321230 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:56.322175 22992 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:56.182017 27806 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:56.295830 22843 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:56.266114 27830 raft_consensus.cc:697] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 14 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:56.278772 27678 raft_consensus.cc:2804] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:09:56.234356 27662 consensus_queue.cc:1048] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.080s
I20260709 13:09:56.268608 27627 raft_consensus.cc:697] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
W20260709 13:09:56.308838 27820 log.cc:927] Time spent T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.484s	user 0.002s	sys 0.002s
W20260709 13:09:56.342394 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 46677us
I20260709 13:09:56.344781 22843 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:56.346338 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:56.354885 22843 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:56.355072 22992 leader_election.cc:400] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 election: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 9. Current term is 10.
I20260709 13:09:56.350091 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:56.347764 22955 raft_consensus.cc:1240] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 11. Current term is 12. Ops: []
I20260709 13:09:56.319329 22949 raft_consensus.cc:1240] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:56.370898 22949 raft_consensus.cc:1275] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:09:56.269125 27811 consensus_queue.cc:1048] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.071s
W20260709 13:09:56.388049 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 312641us
I20260709 13:09:56.298092 27807 consensus_queue.cc:1048] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.068s
I20260709 13:09:56.309446 27724 raft_consensus.cc:2804] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:56.390697 23588 leader_election.cc:400] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 13. Current term is 14.
I20260709 13:09:56.319813 27289 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:56.308457 27840 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:56.307816 27505 raft_consensus.cc:2749] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:56.399448 23621 raft_consensus.cc:1240] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:56.347059 27814 raft_consensus.cc:2749] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:56.325294 27664 raft_consensus.cc:2749] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:56.296281 27590 consensus_queue.cc:1048] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.107s
I20260709 13:09:56.322819 27356 raft_consensus.cc:2804] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:56.338958 27703 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:56.368642 27826 raft_consensus.cc:2804] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:56.346239 27407 raft_consensus.cc:2749] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:09:56.383517 27806 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:56.324072 27462 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:56.369244 27803 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:56.449394 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 260010us
I20260709 13:09:56.398653 27724 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:56.400208 27566 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:09:56.461341 22844 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:09:56.452909 27670 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.794s	user 0.017s	sys 0.008s
I20260709 13:09:56.465806 22918 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:56.442334 27590 consensus_queue.cc:1243] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer 0a407504ed1741bf89a276558adf4c0f log is divergent from this leader: its last log entry 15.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:09:56.441516 27356 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:56.452534 27826 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:56.480243 22955 raft_consensus.cc:1275] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:09:56.434001 27641 consensus_queue.cc:1048] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 13:09:56.393676 27839 consensus_queue.cc:1048] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.085s
I20260709 13:09:56.441382 27818 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:56.467744 27550 raft_consensus.cc:697] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:56.432610 27289 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:56.614449 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 259632us
I20260709 13:09:56.470125 27724 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:56.472733 27596 raft_consensus.cc:697] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:56.479023 27709 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:56.478737 27803 raft_consensus.cc:2749] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:09:56.401134 27815 consensus_queue.cc:237] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:56.432876 27840 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:56.644934 27770 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:56.645457 27770 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:56.646421 27840 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:56.491460 27356 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:56.650425 27818 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:56.493703 27826 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:09:56.652009 27464 consensus_meta.cc:287] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.592s	user 0.006s	sys 0.002s
W20260709 13:09:56.625998 27842 log.cc:927] Time spent T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.345s	user 0.000s	sys 0.001s
W20260709 13:09:56.626698 27841 log.cc:927] Time spent T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.174s	user 0.000s	sys 0.000s
W20260709 13:09:56.627449 27846 log.cc:927] Time spent T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
I20260709 13:09:56.627904 27853 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:56.654946 27853 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:56.655891 27646 consensus_meta.cc:287] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.967s	user 0.008s	sys 0.000s
I20260709 13:09:56.478456 27830 consensus_queue.cc:237] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:56.631238 27764 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:56.633152 27839 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:09:56.473776 27627 consensus_queue.cc:237] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:56.635417 27289 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:56.661113 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 265998us
W20260709 13:09:56.656852 27831 log.cc:927] Time spent T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.429s	user 0.000s	sys 0.001s
I20260709 13:09:56.469319 27586 consensus_queue.cc:1059] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.152s
I20260709 13:09:56.665736 22843 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:56.452327 27651 consensus_queue.cc:237] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:56.613750 27484 tablet_bootstrap.cc:700] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2: Time spent opening tablet: real 0.373s	user 0.001s	sys 0.001s
I20260709 13:09:56.609277 27683 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:56.682679 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 216950us
I20260709 13:09:56.688311 22918 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
W20260709 13:09:56.642198 27761 consensus_meta.cc:287] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.617s	user 0.010s	sys 0.000s
I20260709 13:09:56.666826 22992 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:56.618024 27500 raft_consensus.cc:697] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
W20260709 13:09:56.477125 27670 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.821s	user 0.017s	sys 0.008s
W20260709 13:09:56.666481 27838 log.cc:927] Time spent T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.424s	user 0.001s	sys 0.000s
I20260709 13:09:56.708803 22876 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:56.647441 27770 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:56.744181 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:56.744324 23616 raft_consensus.cc:1240] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:56.749907 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:56.751102 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:56.656275 27646 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:56.744954 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 78211.2us
I20260709 13:09:56.757541 22992 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:56.666283 27566 consensus_queue.cc:1059] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.086s
I20260709 13:09:56.743001 27818 raft_consensus.cc:2804] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:56.706152 27709 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:56.676414 27462 consensus_queue.cc:1059] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.092s
W20260709 13:09:56.765394 27810 consensus_meta.cc:287] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.703s	user 0.009s	sys 0.001s
I20260709 13:09:56.759652 27289 raft_consensus.cc:2749] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:09:56.666102 27764 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:56.841064 22952 raft_consensus.cc:1240] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 13. Current term is 14. Ops: []
I20260709 13:09:56.737959 27683 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:56.840423 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 82909.7us
I20260709 13:09:56.844574 23620 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60044 (request call id 889) took 1215 ms. Trace:
I20260709 13:09:56.847550 23620 rpcz_store.cc:276] 0709 13:09:55.629496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:55.629635 (+   139us) service_pool.cc:224] Handling call
0709 13:09:55.630037 (+   402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:55.635487 (+  5450us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:55.635531 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:55.636525 (+   994us) log.cc:844] Serialized 26 byte log entry
0709 13:09:55.970635 (+334110us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:09:55.970690 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:55.970786 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:56.844301 (+873515us) raft_consensus.cc:1685] finished
0709 13:09:56.844323 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:56.844505 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4968,"raft.queue_time_us":5726,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":13132,"thread_start_us":283962,"threads_started":1,"wal-append.queue_time_us":506196}
I20260709 13:09:56.850981 22992 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:56.741568 27724 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:56.796039 27818 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:56.865671 22876 raft_consensus.cc:1275] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:09:56.858583 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 472133us
I20260709 13:09:57.044545 27874 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:56.915217 27550 consensus_queue.cc:237] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:56.858541 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 377712us
I20260709 13:09:56.807638 27662 raft_consensus.cc:2749] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:09:57.095502 27705 consensus_meta.cc:287] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 1.035s	user 0.005s	sys 0.005s
W20260709 13:09:57.096975 27848 log.cc:927] Time spent T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.148s	user 0.002s	sys 0.000s
W20260709 13:09:57.101325 27861 log.cc:927] Time spent T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 13:09:57.106321 27730 consensus_meta.cc:287] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 1.050s	user 0.010s	sys 0.000s
I20260709 13:09:57.106805 27730 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 13. Current term is 14.
I20260709 13:09:57.107456 23625 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 14.
I20260709 13:09:56.768169 27588 consensus_queue.cc:1059] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.078s
I20260709 13:09:57.113193 27588 raft_consensus.cc:3055] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:09:57.113608 27588 raft_consensus.cc:740] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:57.113859 27764 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:56.770533 27586 consensus_queue.cc:1048] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.087s
I20260709 13:09:57.115236 27683 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:56.845729 27840 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:56.743544 27464 consensus_queue.cc:1059] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.100s
I20260709 13:09:56.915513 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:56.916121 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:09:56.920455 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 210255us
I20260709 13:09:57.126745 22951 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11.
I20260709 13:09:57.127701 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:57.129984 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:57.130277 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:57.130971 23622 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 15.
I20260709 13:09:56.738536 27761 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:56.923666 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 215250us
W20260709 13:09:57.133301 22764 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2959101199a44a4583a4ee456ecf044d with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 0a407504ed1741bf89a276558adf4c0f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 12 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:09:56.856953 27839 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:56.926525 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:57.135878 23027 raft_consensus.cc:3060] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:56.859328 27805 raft_consensus.cc:2749] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:09:56.859864 27566 raft_consensus.cc:3055] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:09:56.932435 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:56.866724 27678 raft_consensus.cc:697] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 9 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:57.147473 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:56.859642 27770 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:56.807530 27709 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:56.885603 27803 raft_consensus.cc:3055] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:09:56.885850 27739 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:57.035439 23023 raft_consensus.cc:1275] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:09:57.064834 27356 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:57.121814 27818 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:56.934844 27859 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:56.941646 27793 consensus_meta.cc:287] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.929s	user 0.008s	sys 0.003s
I20260709 13:09:56.947160 27766 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:56.847064 27596 consensus_queue.cc:237] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:57.035430 27462 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:56.860721 27500 consensus_queue.cc:237] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:56.936924 27826 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:57.081493 27865 consensus_queue.cc:1059] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:09:57.083565 27850 log.cc:927] Time spent T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.340s	user 0.000s	sys 0.000s
I20260709 13:09:57.094399 27871 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:57.077239 27289 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:57.062472 27845 log.cc:927] Time spent T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.405s	user 0.001s	sys 0.001s
I20260709 13:09:57.118079 23625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 989) took 1043 ms. Trace:
I20260709 13:09:57.140618 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:57.153131 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:57.190630 23625 rpcz_store.cc:276] 0709 13:09:56.074903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:56.084243 (+  9340us) service_pool.cc:224] Handling call
0709 13:09:57.117877 (+1033634us) spinlock_profiling.cc:243] Waited 1.02 s on lock 0x7b540048dd54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:09:57.118019 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2350217984}
I20260709 13:09:57.114486 27588 consensus_queue.cc:260] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:57.141300 27504 consensus_meta.cc:287] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.985s	user 0.006s	sys 0.006s
I20260709 13:09:56.920470 27858 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:57.139072 27670 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:57.143817 27853 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:57.143888 27566 raft_consensus.cc:740] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:57.157771 27803 raft_consensus.cc:740] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:57.157845 27770 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:57.117300 27874 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:57.218202 27720 raft_consensus.cc:3055] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:09:57.222074 27720 raft_consensus.cc:740] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:57.220209 27818 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:57.223199 27874 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:57.164317 27739 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:57.181816 27766 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:57.224205 27818 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:57.222262 27566 consensus_queue.cc:260] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:57.201524 27876 log.cc:927] Time spent T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 13:09:57.201745 27709 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:57.192076 27871 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:57.226332 27770 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:57.226353 27566 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:57.225390 27739 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:57.227623 27871 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:57.200839 27858 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:57.202199 27670 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:57.228152 27867 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:57.216392 27857 log.cc:927] Time spent T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.302s	user 0.001s	sys 0.000s
I20260709 13:09:57.229708 27858 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:57.229498 27867 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:57.230014 27670 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:57.203562 27678 consensus_queue.cc:237] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:57.231251 27867 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:57.219548 27826 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:57.214928 27847 log.cc:927] Time spent T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.582s	user 0.002s	sys 0.000s
W20260709 13:09:57.221302 27806 consensus_meta.cc:287] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.753s	user 0.005s	sys 0.005s
I20260709 13:09:57.205533 27761 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:57.232838 27806 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:57.142599 27839 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:57.233454 27588 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:57.234252 27814 raft_consensus.cc:3055] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:09:57.234664 27814 raft_consensus.cc:740] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:57.234913 27806 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:57.180193 27793 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:57.235317 27839 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:57.235570 27814 consensus_queue.cc:260] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:57.237440 27814 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:57.203267 27879 consensus_queue.cc:1048] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 13:09:57.180001 27859 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:57.237463 27793 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:57.238665 27684 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:57.239712 27854 log.cc:927] Time spent T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.606s	user 0.001s	sys 0.014s
I20260709 13:09:57.239826 27859 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:57.222873 27720 consensus_queue.cc:260] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:57.225317 27766 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:09:57.241945 27843 log.cc:927] Time spent T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.904s	user 0.002s	sys 0.000s
I20260709 13:09:57.244493 27805 raft_consensus.cc:3055] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:09:57.245013 27805 raft_consensus.cc:740] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:57.220180 27803 consensus_queue.cc:260] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:57.245823 27803 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:57.245893 27805 consensus_queue.cc:260] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:57.247282 27805 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:09:57.247637 27724 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:57.252305 27736 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:57.120975 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:57.291464 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:57.292130 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:57.087047 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:57.158686 22873 raft_consensus.cc:1240] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:57.293815 22950 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 14.
I20260709 13:09:57.295706 22874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2701) took 1348 ms. Trace:
I20260709 13:09:57.297967 22948 raft_consensus.cc:1240] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:57.152289 22919 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:09:57.299888 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 147550us
I20260709 13:09:57.139024 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:57.305249 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:57.307878 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:57.309808 22954 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2850) took 1236 ms. Trace:
I20260709 13:09:57.152282 23027 raft_consensus.cc:2468] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 11.
I20260709 13:09:57.154995 23029 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 789) took 1094 ms. Trace:
I20260709 13:09:57.312943 22878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2697) took 1367 ms. Trace:
I20260709 13:09:57.313078 23029 rpcz_store.cc:276] 0709 13:09:56.060211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:56.061716 (+  1505us) service_pool.cc:224] Handling call
0709 13:09:56.061965 (+   249us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:56.242819 (+180854us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:57.154754 (+911935us) spinlock_profiling.cc:243] Waited 161 ms on lock 0x7b54002c9f54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:57.154937 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":370252160}
I20260709 13:09:57.296303 22874 rpcz_store.cc:276] 0709 13:09:55.947303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:56.021286 (+ 73983us) service_pool.cc:224] Handling call
0709 13:09:56.021600 (+   314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:56.022766 (+  1166us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 13:09:56.022792 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:56.023188 (+   396us) log.cc:844] Serialized 26 byte log entry
0709 13:09:56.340435 (+317247us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:09:56.340472 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:56.340536 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:57.295062 (+954526us) raft_consensus.cc:1685] finished
0709 13:09:57.295100 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:57.295410 (+   310us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":316930,"threads_started":1,"wal-append.queue_time_us":462201}
I20260709 13:09:57.313393 22878 rpcz_store.cc:276] 0709 13:09:55.945759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:55.947480 (+  1721us) service_pool.cc:224] Handling call
0709 13:09:55.966301 (+ 18821us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:55.966575 (+   274us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:55.966612 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:55.970807 (+  4195us) log.cc:844] Serialized 26 byte log entry
0709 13:09:56.430010 (+459203us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:09:56.430063 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:56.430155 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:57.082985 (+652830us) spinlock_profiling.cc:243] Waited 8.82 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:09:57.296939 (+213954us) raft_consensus.cc:1685] finished
0709 13:09:57.296979 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:57.312762 (+ 15783us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":20286848,"thread_start_us":346256,"threads_started":1,"wal-append.queue_time_us":625046}
I20260709 13:09:57.319888 22949 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:57.310794 22950 raft_consensus.cc:1240] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 13:09:57.310310 22954 rpcz_store.cc:276] 0709 13:09:56.073461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:56.087374 (+ 13913us) service_pool.cc:224] Handling call
0709 13:09:56.092994 (+  5620us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:09:56.139554 (+ 46560us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:56.139588 (+    34us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:09:56.141127 (+  1539us) log.cc:844] Serialized 78 byte log entry
0709 13:09:56.372651 (+231524us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:09:56.372728 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:56.372818 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:57.239110 (+866292us) spinlock_profiling.cc:243] Waited 23.3 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:09:57.308908 (+ 69798us) raft_consensus.cc:1685] finished
0709 13:09:57.308959 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:57.309575 (+   616us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5500,"raft.queue_time_us":17619,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":5666,"spinlock_wait_cycles":55063040,"thread_start_us":192696,"threads_started":1,"wal-append.queue_time_us":471780}
I20260709 13:09:57.330291 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:57.330523 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:57.331360 22950 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:57.331588 22954 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 10.
W20260709 13:09:57.337965 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 172331us
I20260709 13:09:57.340103 23619 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 14.
I20260709 13:09:57.194319 27289 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:57.344830 27289 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:57.185638 27462 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:57.348338 27356 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:57.350775 27462 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:57.351794 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:57.352840 22954 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
I20260709 13:09:57.353272 27462 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:57.354208 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:57.355300 23588 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:57.355511 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:57.356499 23618 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
I20260709 13:09:57.356884 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:57.357743 23588 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:57.354405 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
W20260709 13:09:57.359767 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:57.354270 27462 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:57.361861 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:57.362931 27462 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:57.362802 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:57.200109 27504 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:57.220175 27520 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:57.365738 27520 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:57.366381 27520 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:57.367496 27504 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:57.369405 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:57.369741 27520 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:57.370522 27520 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:57.371155 27520 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:57.368566 27504 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:57.373694 27504 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:57.379251 22993 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:57.364213 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:57.380864 27356 raft_consensus.cc:2804] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:57.381448 27356 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:57.381932 27356 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:09:57.402370 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 44857.8us
I20260709 13:09:57.420694 27793 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:57.422175 27793 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:57.423878 27855 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:57.424134 27839 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:57.427314 22954 raft_consensus.cc:1240] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 8. Current term is 9. Ops: []
I20260709 13:09:57.426839 22952 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 2140) took 1138 ms. Trace:
I20260709 13:09:57.428457 22952 rpcz_store.cc:276] 0709 13:09:56.288324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:56.305156 (+ 16832us) service_pool.cc:224] Handling call
0709 13:09:56.305444 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:56.843792 (+538348us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:57.424355 (+580563us) spinlock_profiling.cc:243] Waited 535 ms on lock 0x7b5400284e54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:57.426652 (+  2297us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1231164160}
I20260709 13:09:57.429505 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:57.430009 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:57.430392 22954 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
I20260709 13:09:57.430823 22952 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:57.425735 27839 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:57.433202 27855 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:57.433307 27764 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:57.437026 27764 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:57.437479 27764 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:57.421682 23625 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 15.
I20260709 13:09:57.456281 22949 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
W20260709 13:09:57.453316 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 69313.3us
I20260709 13:09:57.461901 22952 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
I20260709 13:09:57.463439 27504 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:57.464144 27883 raft_consensus.cc:2749] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:57.473398 22953 raft_consensus.cc:1240] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 LEADER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 14. Current term is 15. Ops: []
I20260709 13:09:57.478928 22874 pending_rounds.cc:85] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 9
I20260709 13:09:57.482476 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:57.482987 27803 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:57.483322 22876 raft_consensus.cc:1240] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:09:57.483623 27824 consensus_queue.cc:1048] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:09:57.484252 27462 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:57.485350 27882 raft_consensus.cc:2749] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:09:57.507447 27462 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:57.510674 27462 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:57.511636 22955 raft_consensus.cc:1240] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 LEADER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 9. Current term is 10. Ops: []
I20260709 13:09:57.486631 22880 raft_consensus.cc:1275] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:09:57.521577 27793 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:57.522996 23588 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:09:57.525002 27730 raft_consensus.cc:2804] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:09:57.525686 27730 raft_consensus.cc:697] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:57.528019 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:57.487145 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:57.487499 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:57.488747 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:57.489087 27867 consensus_queue.cc:1048] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:09:57.491387 22951 raft_consensus.cc:1275] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 13:09:57.492623 27878 consensus_queue.cc:1048] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:09:57.452646 27855 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:57.495021 22874 pending_rounds.cc:107] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 13:09:57.535813 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:57.536729 22874 raft_consensus.cc:2889] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:57.545578 22950 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
I20260709 13:09:57.550745 22993 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:57.527066 27730 consensus_queue.cc:237] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:57.552402 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:57.594772 27867 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:57.595554 27867 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:57.598417 27867 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:57.598683 27520 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:57.552754 22952 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
I20260709 13:09:57.601178 27862 consensus_queue.cc:1059] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:09:57.602684 27807 raft_consensus.cc:3055] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:09:57.591219 22947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2848) took 1518 ms. Trace:
I20260709 13:09:57.603300 27807 raft_consensus.cc:740] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:57.604560 27807 consensus_queue.cc:260] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:57.608747 27839 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:57.560853 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:57.561539 27883 consensus_queue.cc:1059] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:09:57.561931 27805 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:57.603449 22947 rpcz_store.cc:276] 0709 13:09:56.072780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:56.075135 (+  2355us) service_pool.cc:224] Handling call
0709 13:09:56.075466 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:56.075697 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:09:56.093152 (+ 17455us) log.cc:844] Serialized 14 byte log entry
0709 13:09:57.215695 (+1122543us) spinlock_profiling.cc:243] Waited 6.55 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:09:57.590108 (+374413us) spinlock_profiling.cc:243] Waited 1.01 ms on lock 0x7b18001fd9e0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f613598d4bc 00007f6132cdb48b 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf5455 00007f6135ddd060 00007f6135def716 00007f6135e00287
0709 13:09:57.590373 (+   265us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:57.590501 (+   128us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:09:57.590659 (+   158us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:57.590774 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:57.590970 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":216734,"prepare.queue_time_us":29645,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":189998,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":36751,"spinlock_wait_cycles":17391744,"thread_start_us":624627,"threads_started":2,"wal-append.queue_time_us":714266}
I20260709 13:09:57.535401 27289 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:57.565025 22954 raft_consensus.cc:2393] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 15: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:57.565459 22879 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 9.
I20260709 13:09:57.570772 27810 raft_consensus.cc:3055] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 8 LEADER]: Stepping down as leader of term 8
I20260709 13:09:57.572857 27803 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:57.574072 22919 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:09:57.574947 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 47061.6us
I20260709 13:09:57.575187 27724 raft_consensus.cc:2804] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:09:57.577509 22946 raft_consensus.cc:1275] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 13:09:57.587040 22873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2704) took 1336 ms. Trace:
I20260709 13:09:57.588043 22992 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:57.589931 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:57.591179 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:57.603380 22953 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 2143) took 1310 ms. Trace:
I20260709 13:09:57.616343 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:57.619474 22953 rpcz_store.cc:276] 0709 13:09:56.292521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:56.309418 (+ 16897us) service_pool.cc:224] Handling call
0709 13:09:56.309672 (+   254us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:57.535610 (+1225938us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:57.603103 (+ 67493us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54000758d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:57.603325 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2674920832}
I20260709 13:09:57.625581 27839 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:57.618788 22873 rpcz_store.cc:276] 0709 13:09:56.250528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:56.250686 (+   158us) service_pool.cc:224] Handling call
0709 13:09:56.251064 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:57.294492 (+1043428us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:57.586482 (+291990us) spinlock_profiling.cc:243] Waited 907 ms on lock 0x7b54003c64d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:57.586810 (+   328us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2086870272}
I20260709 13:09:57.626639 27803 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:57.627208 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:57.627369 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:57.626068 27839 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:57.622264 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 89719.2us
I20260709 13:09:57.622539 22871 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 15.
I20260709 13:09:57.632375 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:57.632947 27698 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:57.633201 23624 raft_consensus.cc:3060] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:57.633622 27698 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:57.636181 27698 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:09:57.640738 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:57.641396 22948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 2139) took 1354 ms. Trace:
I20260709 13:09:57.641681 23621 raft_consensus.cc:2393] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 10: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:09:57.641856 22948 rpcz_store.cc:276] 0709 13:09:56.287443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:56.287551 (+   108us) service_pool.cc:224] Handling call
0709 13:09:56.287796 (+   245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:57.298550 (+1010754us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:57.641093 (+342543us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b540037b9d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:57.641324 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2321748352}
W20260709 13:09:57.641744 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:57.643555 27805 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:57.645255 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:57.646046 27805 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:57.646700 27724 raft_consensus.cc:697] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:57.647893 27724 consensus_queue.cc:237] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:57.615556 27810 raft_consensus.cc:740] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:57.653299 27810 consensus_queue.cc:260] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:09:57.654214 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:57.654654 27810 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 8 FOLLOWER]: Advancing to term 9
I20260709 13:09:57.655234 22877 raft_consensus.cc:2393] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 12: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:57.659500 27646 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:57.628321 27289 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:57.663277 27289 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:57.666186 22876 raft_consensus.cc:1240] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:09:57.667196 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:57.668164 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:57.668948 22872 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 14.
I20260709 13:09:57.669075 22879 raft_consensus.cc:1275] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:09:57.612887 27883 consensus_queue.cc:1243] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer f3c799b1249e40119d2a5864231c6c75 log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:09:57.669652 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:57.670356 23621 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11.
I20260709 13:09:57.670390 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:57.671427 22875 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 14.
I20260709 13:09:57.671681 27720 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:57.672308 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:57.673583 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:57.672814 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:57.674387 23623 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
I20260709 13:09:57.674456 23619 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 15.
I20260709 13:09:57.676201 27883 consensus_queue.cc:1048] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:09:57.679391 27839 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:57.679209 27883 consensus_queue.cc:1059] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:09:57.681756 27883 raft_consensus.cc:3055] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:09:57.682150 27883 raft_consensus.cc:740] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:57.683218 27883 consensus_queue.cc:260] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:57.684108 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:57.685285 27883 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:57.686378 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:57.690295 27807 raft_consensus.cc:3060] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:57.694121 22871 raft_consensus.cc:1275] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 5. (term mismatch)
W20260709 13:09:57.713378 27815 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.007s	sys 0.014s
W20260709 13:09:57.713855 27815 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.007s	sys 0.014s
I20260709 13:09:57.711843 22871 pending_rounds.cc:85] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 4
I20260709 13:09:57.719515 22871 pending_rounds.cc:107] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 13:09:57.719844 22871 raft_consensus.cc:2889] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:09:57.725870 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:57.727885 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:57.729110 23588 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:57.729280 23590 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:57.729458 27719 raft_consensus.cc:2749] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:57.730872 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:57.731498 23616 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
I20260709 13:09:57.732100 27625 raft_consensus.cc:2804] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:57.732563 27625 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:57.732996 27625 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:57.732751 22773 catalog_manager.cc:5697] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 6 to 11, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 11 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:09:57.734978 22773 catalog_manager.cc:5697] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 7 to 9, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 9 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:09:57.736238 23590 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
W20260709 13:09:57.737052 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 149411us
W20260709 13:09:57.737268 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:57.741029 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:57.736975 27736 raft_consensus.cc:2804] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:57.754215 22875 raft_consensus.cc:2393] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 11: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:57.757611 27356 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:57.760128 27356 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:57.767910 23624 raft_consensus.cc:2468] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
I20260709 13:09:57.769507 22879 raft_consensus.cc:1275] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:09:57.751696 27862 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:57.754541 27736 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:57.747558 27289 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:57.788388 23617 raft_consensus.cc:1240] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 10. Current term is 11. Ops: []
I20260709 13:09:57.789047 27746 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:57.789179 27697 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:57.789664 27746 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:57.790153 27746 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:57.789901 27697 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:57.790928 27862 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:57.792991 27179 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:57.793275 27862 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:57.793661 27179 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:57.803200 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 53188.5us
W20260709 13:09:57.805248 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 49637.9us
I20260709 13:09:57.795524 27736 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:57.792543 27697 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:57.817865 22993 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:57.822641 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:57.824477 23623 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:57.800701 27356 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:57.798664 27289 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:57.827102 27629 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:57.836138 22949 raft_consensus.cc:1275] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:09:57.836594 27709 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:57.823930 27805 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:57.837311 27709 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:57.838066 27805 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:57.838656 27805 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:57.822705 27724 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:57.840530 22992 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:57.841154 27724 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:57.843025 23619 raft_consensus.cc:1275] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:09:57.843292 27724 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:57.843782 27724 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:57.844733 27588 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:57.845605 27588 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:57.845891 27629 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:57.848142 27629 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:57.848656 27629 raft_consensus.cc:2749] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:57.842559 27356 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:09:57.815250 27698 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:57.840342 27709 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:57.844249 27724 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:57.868418 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:57.868574 22950 raft_consensus.cc:1240] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 12. Current term is 13. Ops: []
W20260709 13:09:57.869033 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:57.856739 27764 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:57.867988 27709 consensus_queue.cc:1059] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 13:09:57.885347 27588 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:57.898326 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:57.901209 27651 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:57.901700 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:57.902400 27651 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:57.901201 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:57.904168 23029 raft_consensus.cc:2376] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11. Re-sending same reply.
I20260709 13:09:57.904625 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:57.905436 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:57.902834 23022 raft_consensus.cc:2410] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:09:57.906454 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:57.907385 23023 raft_consensus.cc:2393] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 12: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:57.898069 27719 consensus_queue.cc:1048] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.011s
I20260709 13:09:57.908716 22844 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
W20260709 13:09:57.909482 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 78121.7us
I20260709 13:09:57.905352 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:57.905308 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:57.923210 23020 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 8 FOLLOWER]: Advancing to term 9
W20260709 13:09:57.923600 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:57.889529 27698 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:57.926373 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:57.933509 22949 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 10.
I20260709 13:09:57.928923 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:57.929468 27356 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:57.916987 27179 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:57.919240 27588 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:57.940124 23027 raft_consensus.cc:2410] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:09:57.940485 23623 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 13.
I20260709 13:09:57.942065 22993 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:57.909929 27719 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:57.943020 23588 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:57.941928 22844 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
W20260709 13:09:57.947552 27861 log.cc:927] Time spent T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 13:09:57.952126 22949 raft_consensus.cc:1275] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:09:57.952309 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:57.954653 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:57.955163 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:57.944018 27588 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:57.966075 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:57.964252 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:09:57.974556 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 134249us
I20260709 13:09:57.978478 27862 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:57.980151 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:57.980890 23026 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 14.
I20260709 13:09:57.982547 27698 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:57.953578 27651 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:57.952275 27724 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:57.958957 27588 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:57.964326 27720 raft_consensus.cc:2804] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:09:57.985566 23027 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:09:57.992717 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:57.979086 27862 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:58.002528 23023 raft_consensus.cc:1275] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:09:58.058337 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:58.060655 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:57.971118 27730 consensus_queue.cc:1048] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.088s
I20260709 13:09:58.063980 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:58.064457 27909 log.cc:927] Time spent T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:09:58.064842 22874 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 15.
W20260709 13:09:58.065104 27484 log.cc:927] Time spent T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 13:09:58.065634 27830 raft_consensus.cc:2804] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:09:58.066586 22871 raft_consensus.cc:1240] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 15. Current term is 16. Ops: []
I20260709 13:09:58.066836 27484 tablet_bootstrap.cc:492] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=6 overwritten=0 applied=6 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:09:58.067850 27484 tablet_bootstrap.cc:492] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap complete.
I20260709 13:09:58.067754 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:09:58.068285 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:58.068917 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:58.069693 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:58.071892 23590 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:58.072304 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:58.073436 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:58.073717 27604 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:58.074256 27604 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:58.074694 27604 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:58.075552 27821 raft_consensus.cc:3055] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:09:58.075949 27821 raft_consensus.cc:740] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:09:58.076112 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:58.076653 23623 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:58.076939 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:58.081346 27590 consensus_queue.cc:1048] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:09:58.084515 27910 log.cc:927] Time spent T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 13:09:58.085939 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:58.086617 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:58.087308 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:58.088027 23021 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 12.
I20260709 13:09:58.087607 22773 catalog_manager.cc:5697] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 12 to 13, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 13 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:09:58.087312 23022 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 9.
I20260709 13:09:58.090039 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:58.089587 22773 catalog_manager.cc:5697] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 9 to 12, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 12 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:09:58.090615 23623 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 16.
I20260709 13:09:58.090917 23022 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 15.
I20260709 13:09:58.091943 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:58.091574 22773 catalog_manager.cc:5697] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:09:58.050426 27719 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:58.069006 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:58.074852 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:58.062369 23026 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11.
I20260709 13:09:58.093951 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:58.090147 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:58.092521 23022 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:09:58.120900 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 49019us
I20260709 13:09:58.126583 23020 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 9.
I20260709 13:09:58.126207 22919 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:58.125135 22876 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 12.
I20260709 13:09:57.971392 27289 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:09:58.148692 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:58.101864 22777 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33488 (request call id 268) took 21896 ms (client timeout 15000 ms). Trace:
I20260709 13:09:58.165759 23029 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:58.165938 22949 raft_consensus.cc:1275] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:09:58.167088 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:58.060196 27764 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:58.174407 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:58.178040 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:58.179180 23021 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 12.
I20260709 13:09:58.179121 23026 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11.
W20260709 13:09:58.183545 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:58.185014 22773 catalog_manager.cc:5697] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 4 to 12. New cstate: current_term: 12 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:09:58.180953 27520 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:58.187147 27520 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:58.186973 22773 catalog_manager.cc:5697] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 8 to 14, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 14 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
W20260709 13:09:58.188833 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 126065us
I20260709 13:09:58.190402 27520 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:58.191087 27520 raft_consensus.cc:2804] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:09:58.191471 27520 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:58.191756 22992 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:58.191867 27520 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:58.193879 27878 consensus_queue.cc:1048] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:09:58.158006 27484 ts_tablet_manager.cc:1403] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2: Time spent bootstrapping tablet: real 2.740s	user 0.042s	sys 0.014s
I20260709 13:09:58.130290 27830 raft_consensus.cc:697] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 11 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:58.206960 22876 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 14.
I20260709 13:09:58.209412 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:58.210021 27720 raft_consensus.cc:697] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:58.210310 22880 raft_consensus.cc:2393] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 12: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:58.211128 27720 consensus_queue.cc:237] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:58.178894 27698 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:58.213110 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:58.105939 27839 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:58.096585 27805 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:58.217741 22874 raft_consensus.cc:1275] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:09:58.206501 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:58.219056 22871 raft_consensus.cc:1275] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
I20260709 13:09:58.219880 23022 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 14.
I20260709 13:09:58.228770 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:58.229588 22876 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 9.
I20260709 13:09:58.231181 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:58.231248 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:58.232246 27805 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:58.235061 22871 raft_consensus.cc:1275] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:09:58.235447 27484 raft_consensus.cc:359] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:09:58.236940 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:58.237591 22948 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
I20260709 13:09:58.242144 23024 raft_consensus.cc:2393] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 16: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:58.241729 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:58.249394 22950 raft_consensus.cc:2368] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 15. Current term is 16.
I20260709 13:09:58.251349 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:58.251986 22994 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:58.252883 27179 consensus_queue.cc:1048] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:09:58.258587 27915 consensus_queue.cc:1059] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:09:58.216439 22773 catalog_manager.cc:5697] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 8 to 11, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 11 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:09:58.263578 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:58.264308 23616 raft_consensus.cc:2393] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 16: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:58.236449 27484 raft_consensus.cc:740] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 LEARNER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Initialized, Role: LEARNER
I20260709 13:09:58.157902 27821 consensus_queue.cc:260] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:58.204339 27736 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:58.213595 27356 consensus_queue.cc:1048] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 13:09:58.213156 27698 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:58.285089 22843 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:09:58.159230 22777 rpcz_store.cc:269] 0709 13:09:36.205471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:36.205805 (+   334us) service_pool.cc:224] Handling call
0709 13:09:58.101597 (+21895792us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:09:52.014537 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:09:52.272900 (+258363us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:09:52.290816 (+ 17916us) write_op.cc:435] Acquired schema lock
0709 13:09:52.294105 (+  3289us) tablet.cc:662] Decoding operations
0709 13:09:52.432044 (+137939us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:09:52.432142 (+    98us) write_op.cc:641] Partition lock acquired for write op
0709 13:09:52.432168 (+    26us) tablet.cc:685] Acquiring locks for 23 operations
0709 13:09:52.433950 (+  1782us) tablet.cc:701] Row locks acquired
0709 13:09:52.433997 (+    47us) write_op.cc:260] PREPARE: finished
0709 13:09:52.434167 (+   170us) write_op.cc:270] Start()
0709 13:09:52.434485 (+   318us) write_op.cc:276] Timestamp: P: 1783602592434109 usec, L: 0
0709 13:09:52.434525 (+    40us) op_driver.cc:348] REPLICATION: starting
0709 13:09:52.435280 (+   755us) log.cc:844] Serialized 8946 byte log entry
0709 13:09:52.807066 (+371786us) op_driver.cc:464] REPLICATION: finished
0709 13:09:52.996431 (+189365us) write_op.cc:301] APPLY: starting
0709 13:09:52.996586 (+   155us) tablet.cc:1367] starting BulkCheckPresence
0709 13:09:53.437430 (+440844us) tablet.cc:1370] finished BulkCheckPresence
0709 13:09:53.437568 (+   138us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:09:54.171876 (+734308us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:09:54.172304 (+   428us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=46,key_file_lookups=46,delta_file_lookups=46,mrs_lookups=0
0709 13:09:54.172346 (+    42us) write_op.cc:312] APPLY: finished
0709 13:09:54.172989 (+   643us) log.cc:844] Serialized 204 byte log entry
0709 13:09:54.355275 (+182286us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:54.420093 (+ 64818us) write_op.cc:454] Released schema lock
0709 13:09:54.451361 (+ 31268us) write_op.cc:341] FINISH: Updating metrics
0709 13:09:54.534131 (+ 82770us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:09:54.545656 (+ 11525us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2464872,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":189045,"apply.run_cpu_time_us":53096,"apply.run_wall_time_us":1494492,"cfile_cache_hit":90,"cfile_cache_hit_bytes":94095,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6218,"cfile_init":2,"lbm_read_time_us":9724,"lbm_reads_1-10_ms":1,"lbm_reads_lt_1ms":11,"num_ops":23,"prepare.queue_time_us":514219,"prepare.run_cpu_time_us":33213,"prepare.run_wall_time_us":677391,"raft.queue_time_us":172054,"raft.run_cpu_time_us":20088,"raft.run_wall_time_us":168630,"replication_time_us":372338,"spinlock_wait_cycles":8192,"thread_start_us":904107,"threads_started":5,"wal-append.queue_time_us":436555,"wal-append.run_cpu_time_us":12571,"wal-append.run_wall_time_us":2250739}]]}
I20260709 13:09:58.292114 27736 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:58.292987 27830 consensus_queue.cc:237] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:58.297377 27520 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:58.299803 27724 raft_consensus.cc:2749] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:09:58.300573 27862 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:58.301956 27839 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:58.328575 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:09:58.331305 22879 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 11
I20260709 13:09:58.301265 22773 catalog_manager.cc:5697] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 5 to 8, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 8 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
W20260709 13:09:58.303334 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 51572.3us
I20260709 13:09:58.309634 22954 raft_consensus.cc:1275] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 13:09:58.311375 27646 consensus_queue.cc:1059] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:09:58.311705 27764 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:58.340729 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:58.312556 22920 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:58.314337 27912 raft_consensus.cc:3055] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 13:09:58.329324 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:58.329270 23029 raft_consensus.cc:2468] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11.
I20260709 13:09:58.355841 27912 raft_consensus.cc:740] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:09:58.246974 27625 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:58.356244 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:58.356931 27912 consensus_queue.cc:260] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:58.357930 27625 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:58.358489 27912 raft_consensus.cc:3060] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:58.358634 23617 raft_consensus.cc:1240] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 14. Current term is 15. Ops: []
W20260709 13:09:58.340229 27746 consensus_meta.cc:287] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.550s	user 0.009s	sys 0.004s
I20260709 13:09:58.389542 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:58.329270 23027 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 14.
W20260709 13:09:58.393412 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 267294us
I20260709 13:09:58.325770 27588 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:58.363719 27709 raft_consensus.cc:2749] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:09:58.386773 27862 raft_consensus.cc:2749] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:09:58.386217 27764 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:58.341869 27484 consensus_queue.cc:260] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:09:58.602775 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:58.602787 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:58.606884 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 218951us
W20260709 13:09:58.449839 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:58.608150 22994 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:09:58.400125 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 115238us
W20260709 13:09:58.520308 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 137819us
W20260709 13:09:58.609143 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:58.447749 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:58.563396 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:58.400378 22919 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:58.576740 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:58.618309 22954 raft_consensus.cc:2393] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 16: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:58.489233 23026 raft_consensus.cc:1275] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 13:09:58.542546 27923 log.cc:927] Time spent T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 13:09:58.636801 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:58.637197 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:58.637961 22952 raft_consensus.cc:2393] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 9: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:58.638389 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:58.632352 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:58.633270 22994 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:58.639623 22955 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:58.632503 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:58.633090 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 320580us
I20260709 13:09:58.633704 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:58.635419 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 235087us
I20260709 13:09:58.488440 27807 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:58.501431 27289 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:58.637010 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:58.645321 22949 raft_consensus.cc:2393] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 14: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
W20260709 13:09:58.615392 27604 consensus_meta.cc:287] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.540s	user 0.011s	sys 0.000s
I20260709 13:09:58.580655 27697 raft_consensus.cc:3055] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:09:58.400838 27356 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:58.389963 27746 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:58.658443 22919 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:58.666612 23022 raft_consensus.cc:1240] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:09:58.436650 27719 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:58.585556 27520 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:58.554286 27803 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:58.649119 22948 raft_consensus.cc:2393] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 12: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:58.636780 27919 raft_consensus.cc:2749] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:09:58.636868 27646 raft_consensus.cc:2804] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:09:58.663290 22920 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:58.673368 27807 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:58.674275 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:58.662469 27289 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:58.675381 27926 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
W20260709 13:09:58.675601 22918 leader_election.cc:336] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.792s (SENT)
I20260709 13:09:58.662201 27697 raft_consensus.cc:740] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:58.675925 27356 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:58.676589 27500 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:09:58.677129 27879 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:58.678985 27860 raft_consensus.cc:2749] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:58.680526 27821 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:58.717150 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:58.717324 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:58.717806 23621 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:58.718228 23619 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:58.680693 27724 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:58.680917 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:58.682026 22994 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:58.683600 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:58.693831 22955 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 13.
I20260709 13:09:58.644383 27720 raft_consensus.cc:2804] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:58.698890 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:58.728421 22950 raft_consensus.cc:2393] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 12: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:09:58.651660 27588 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:58.694394 27646 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:58.728976 22953 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 13.
I20260709 13:09:58.707525 23588 leader_election.cc:400] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 15. Current term is 16.
I20260709 13:09:58.662788 27604 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:58.645043 27629 consensus_queue.cc:1048] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 13:09:58.709218 22843 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:58.709326 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:58.693958 27698 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:58.689754 27882 consensus_queue.cc:1048] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 13:09:58.726219 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:58.735304 23616 raft_consensus.cc:1275] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:09:58.735648 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:09:58.744799 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:58.736064 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:09:58.748786 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 57926.4us
I20260709 13:09:58.692687 27803 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:58.736059 27484 ts_tablet_manager.cc:1434] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2: Time spent starting tablet: real 0.540s	user 0.004s	sys 0.012s
I20260709 13:09:58.763551 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:58.735159 27720 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:58.771310 22953 raft_consensus.cc:2393] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 12: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:58.729619 27500 raft_consensus.cc:697] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:09:58.736635 27646 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:09:58.768446 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:09:58.773766 27720 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:09:58.774173 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 98601us
W20260709 13:09:58.776543 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:58.778852 23026 raft_consensus.cc:1275] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 11. (index mismatch)
W20260709 13:09:58.783581 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 140868us
I20260709 13:09:58.784435 22994 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:58.705653 27926 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:58.770054 27698 raft_consensus.cc:2804] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:09:58.794646 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 85525.4us
I20260709 13:09:58.789888 27873 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:58.797801 27707 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:58.802548 22843 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:58.745216 27724 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:58.771703 27484 ts_tablet_manager.cc:1467] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2: Tablet startup took 568ms
I20260709 13:09:58.790084 27925 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:58.749663 27879 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:58.791015 27520 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:09:58.803900 27698 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:58.828558 22994 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:58.650171 22773 catalog_manager.cc:5697] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 7 to 10, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 10 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
W20260709 13:09:58.842777 27930 log.cc:927] Time spent T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 13:09:58.843403 27811 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:58.843972 27811 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:58.833931 27698 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:09:58.774744 27697 consensus_queue.cc:260] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:58.858050 23619 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 16.
I20260709 13:09:58.874210 23588 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:58.822059 27873 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:58.845475 27807 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:58.880138 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 57076.5us
I20260709 13:09:58.876013 27911 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:58.823000 27707 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:58.858412 27356 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:58.867127 27929 raft_consensus.cc:2749] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 13:09:58.900409 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:58.901283 23621 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 12.
I20260709 13:09:58.876150 27826 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:58.831065 27925 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:58.882849 27746 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:58.877944 27882 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:09:58.881902 27289 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:58.853451 27520 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:58.924963 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:58.889598 27867 raft_consensus.cc:2749] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:09:58.896796 27179 consensus_queue.cc:1048] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 13:09:58.896489 27764 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:58.976780 27941 log.cc:927] Time spent T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
I20260709 13:09:58.929960 22992 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:09:58.977272 27936 log.cc:927] Time spent T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.182s	user 0.000s	sys 0.001s
W20260709 13:09:58.946641 22879 consensus_meta.cc:287] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.584s	user 0.007s	sys 0.004s
I20260709 13:09:58.977928 22879 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 11.
I20260709 13:09:58.977200 27764 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:58.979199 27678 raft_consensus.cc:2804] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:09:58.979803 27678 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:58.980190 27678 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:09:58.819905 27484 ts_tablet_manager.cc:1469] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2: Trace:
0709 13:09:47.218094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:47.221780 (+  3686us) service_pool.cc:224] Handling call
0709 13:09:47.553312 (+331532us) ts_tablet_manager.cc:934] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2: Initiating tablet copy from peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
0709 13:09:50.376307 (+2822995us) tablet_copy_client.cc:368] Tablet copy session begun
0709 13:09:51.342183 (+965876us) tablet_metadata.cc:665] Metadata flushed
0709 13:09:51.357859 (+ 15676us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 13:09:52.329756 (+971897us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 13:09:52.420435 (+ 90679us) tablet_replica.cc:206] Creating consensus instance
0709 13:09:52.420839 (+   404us) ts_tablet_manager.cc:989] Replica registered.
0709 13:09:52.420922 (+    83us) inbound_call.cc:177] Queueing success response
0709 13:09:55.393601 (+2972679us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 13:09:58.196428 (+2802827us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 13:09:58.232698 (+ 36270us) tablet_replica.cc:261] Starting instrumentation
0709 13:09:58.233437 (+   739us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":19589,"mutex_wait_us":4709,"raft.queue_time_us":8796,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":4460,"tablet-copy.queue_time_us":164721,"tablet-download-pool.queue_time_us":6981,"tablet-download-pool.run_cpu_time_us":1308,"tablet-download-pool.run_wall_time_us":2433709,"thread_start_us":175254,"threads_started":2}
I20260709 13:09:58.983126 27720 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:58.973176 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:58.984225 27945 consensus_queue.cc:1048] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:09:58.894698 27500 consensus_queue.cc:237] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:58.985631 27720 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:58.985795 27945 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
W20260709 13:09:58.924223 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 246171us
W20260709 13:09:58.937915 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 274673us
I20260709 13:09:58.986508 27945 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:58.987020 22920 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:58.979249 27764 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:58.939620 27931 log.cc:927] Time spent T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260709 13:09:58.963526 27919 raft_consensus.cc:2749] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:09:58.977272 27928 log.cc:927] Time spent T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.153s	user 0.000s	sys 0.000s
I20260709 13:09:58.990325 27764 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:58.940400 27882 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:58.988368 22920 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:58.992769 22920 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:09:58.993295 27933 log.cc:927] Time spent T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 13:09:58.994742 22920 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:09:58.995525 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 65661.7us
I20260709 13:09:58.997467 22992 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:09:58.998057 27520 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:58.998606 27520 raft_consensus.cc:2804] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:58.998946 27520 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:58.999257 27520 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:58.991113 27764 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:58.965778 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:58.897854 27911 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:59.010862 23024 raft_consensus.cc:2393] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 14: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:58.907848 27915 consensus_queue.cc:1048] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 13:09:58.988654 27588 raft_consensus.cc:2764] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 13:09:58.936919 27839 raft_consensus.cc:2749] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:58.999289 22993 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:09:58.985965 27873 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:59.025094 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 222646us
I20260709 13:09:59.000159 27697 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:09:58.999830 27882 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:59.006413 27356 raft_consensus.cc:2749] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:59.013851 27289 raft_consensus.cc:2804] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:09:59.012276 27604 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:58.999928 27811 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:59.033704 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:59.030293 27720 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:59.028683 27803 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:58.967568 27826 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:59.070783 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 283535us
W20260709 13:09:59.080945 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 188336us
I20260709 13:09:59.081142 22843 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:59.037212 27289 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:09:59.093446 22918 leader_election.cc:336] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.939s (SENT)
I20260709 13:09:59.099840 22843 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:59.095875 22918 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:09:59.025010 27764 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
W20260709 13:09:59.128702 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:59.131428 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:59.133984 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 139308us
I20260709 13:09:59.050432 27724 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:59.138407 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:59.138655 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:59.139168 22952 raft_consensus.cc:2368] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 15. Current term is 16.
I20260709 13:09:59.092453 27356 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:09:59.136075 23026 raft_consensus.cc:1240] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 11. Current term is 12. Ops: []
I20260709 13:09:59.138455 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:59.080034 27926 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:59.098822 27289 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:09:59.153101 23024 raft_consensus.cc:1240] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 10. Current term is 11. Ops: [9.9-10.10]
I20260709 13:09:59.153792 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:59.154381 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:59.135283 27945 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:59.155057 23024 raft_consensus.cc:2393] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 12: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:59.155288 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:59.155896 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:59.156438 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:59.143994 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:59.052448 27720 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:59.094614 27830 consensus_queue.cc:1059] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 13:09:59.145733 23022 raft_consensus.cc:1240] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 13:09:59.138240 27604 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:59.146603 27698 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:59.147670 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:59.139926 27879 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:59.098807 22773 catalog_manager.cc:5697] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 8 to 12, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 12 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:09:59.101631 27929 consensus_queue.cc:1059] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.052s
I20260709 13:09:59.098613 27803 raft_consensus.cc:2804] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:09:59.193020 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:09:59.160190 23024 raft_consensus.cc:2410] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
W20260709 13:09:59.162668 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:59.177649 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:59.174867 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:59.199119 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:59.203490 23026 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
I20260709 13:09:59.204633 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:09:59.217581 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 117855us
I20260709 13:09:59.237124 23023 raft_consensus.cc:2368] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 14. Current term is 15.
I20260709 13:09:59.168134 27954 raft_consensus.cc:2804] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:09:59.247431 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:59.170250 27926 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:59.217746 23025 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.580s	user 0.000s	sys 0.029s
W20260709 13:09:59.249205 23025 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.805s	user 0.000s	sys 0.030s
I20260709 13:09:59.241546 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:09:59.193590 27803 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:09:59.244748 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:59.213991 27830 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:09:59.248414 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 113424us
I20260709 13:09:59.175197 27945 consensus_queue.cc:1048] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
I20260709 13:09:59.215349 23619 raft_consensus.cc:1275] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
W20260709 13:09:59.252910 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 85589.4us
I20260709 13:09:59.250118 22843 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:59.254940 23026 raft_consensus.cc:2393] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 15: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:59.261317 23028 raft_consensus.cc:1240] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 13. Current term is 14. Ops: []
I20260709 13:09:59.177668 27604 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:59.260607 23588 leader_election.cc:400] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 election: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 15. Current term is 16.
I20260709 13:09:59.275048 23588 leader_election.cc:403] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:09:59.269034 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:59.261333 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:59.260569 22920 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:09:59.257467 27925 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:59.269529 27953 raft_consensus.cc:2804] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:09:59.269876 27911 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:09:59.270582 27500 raft_consensus.cc:2749] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:09:59.272176 27678 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:59.238880 27950 consensus_queue.cc:1048] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 13:09:59.278672 23022 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
W20260709 13:09:59.654062 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:59.914323 22950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 814) took 1077 ms. Trace:
W20260709 13:09:59.872015 23025 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.012s	sys 0.020s
I20260709 13:09:59.254762 27803 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:09:59.721450 27945 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:59.269366 27956 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:59.952311 27945 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:59.952483 27956 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:59.954506 27945 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:59.902801 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 393049us
I20260709 13:09:59.285060 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:09:59.956841 27697 consensus_meta.cc:287] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.922s	user 0.007s	sys 0.004s
I20260709 13:09:59.957239 23026 raft_consensus.cc:2393] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 14: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:09:59.279084 27954 raft_consensus.cc:697] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 13 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:09:59.957461 22954 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 16.
W20260709 13:09:59.957852 27821 consensus_meta.cc:287] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 1.233s	user 0.012s	sys 0.000s
I20260709 13:09:59.299013 27520 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:09:59.294677 27939 log.cc:927] Time spent T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.315s	user 0.004s	sys 0.006s
I20260709 13:09:59.958484 27954 consensus_queue.cc:237] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:59.720456 27958 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:59.445632 27826 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:59.436128 27970 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:09:59.961051 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:59.782837 22845 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.007s	sys 0.010s
I20260709 13:09:59.962525 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:59.281299 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:09:59.471065 27720 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:59.876014 27965 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:09:59.963845 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:09:59.963932 27965 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:09:59.964505 23022 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:09:59.965556 27646 consensus_meta.cc:287] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 1.162s	user 0.006s	sys 0.005s
W20260709 13:09:59.962538 22845 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.748s	user 0.010s	sys 0.012s
I20260709 13:09:59.966140 27965 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:59.876332 27969 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
W20260709 13:09:59.966745 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:09:59.967392 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:59.967011 27969 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:59.290535 27934 log.cc:927] Time spent T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.300s	user 0.001s	sys 0.001s
I20260709 13:09:59.295425 27959 raft_consensus.cc:2749] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:09:59.358939 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:09:59.926467 22946 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.002s	sys 0.013s
W20260709 13:09:59.968948 22946 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.002s	sys 0.014s
I20260709 13:09:59.969583 22946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2876) took 1349 ms. Trace:
I20260709 13:09:59.970125 23028 raft_consensus.cc:1240] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 13:09:59.970094 22946 rpcz_store.cc:276] 0709 13:09:58.620592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:58.645589 (+ 24997us) service_pool.cc:224] Handling call
0709 13:09:58.686855 (+ 41266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:58.706539 (+ 19684us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:09:58.928865 (+222326us) log.cc:844] Serialized 14 byte log entry
0709 13:09:59.969264 (+1040399us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:59.969312 (+    48us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:09:59.969357 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:59.969434 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:59.969532 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45162,"prepare.queue_time_us":277729,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":451377,"spinlock_wait_cycles":7132032,"thread_start_us":1086351,"threads_started":3,"wal-append.queue_time_us":896370}
I20260709 13:09:59.972805 23028 raft_consensus.cc:1240] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 13. Current term is 14. Ops: []
I20260709 13:09:59.973083 27826 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:59.973649 27826 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
W20260709 13:09:59.902462 22947 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.000s	sys 0.012s
W20260709 13:09:59.975666 22947 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.728s	user 0.001s	sys 0.013s
I20260709 13:09:59.976248 23024 raft_consensus.cc:2410] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:09:59.292410 22773 catalog_manager.cc:5697] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:09:59.290750 27879 raft_consensus.cc:2749] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 15. Current term is 16.
I20260709 13:09:59.942857 22950 rpcz_store.cc:276] 0709 13:09:58.837105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:58.837250 (+   145us) service_pool.cc:224] Handling call
0709 13:09:58.853140 (+ 15890us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:59.081568 (+228428us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:59.081617 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:09:59.102188 (+ 20571us) log.cc:844] Serialized 26 byte log entry
0709 13:09:59.268648 (+166460us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:09:59.268711 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:59.268793 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:59.914130 (+645337us) raft_consensus.cc:1685] finished
0709 13:09:59.914147 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:59.914268 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":125177,"raft.run_cpu_time_us":77,"raft.run_wall_time_us":78,"thread_start_us":316262,"threads_started":2,"wal-append.queue_time_us":772808}
I20260709 13:09:59.959919 27520 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:09:59.866598 22955 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.731s	user 0.011s	sys 0.020s
W20260709 13:09:59.995491 22955 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.870s	user 0.014s	sys 0.023s
I20260709 13:09:59.961184 27956 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:59.284704 27873 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:09:59.961685 27977 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:09:59.922598 27975 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:59.966004 27646 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:09:59.782907 22918 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.000s	sys 0.017s
I20260709 13:09:59.969513 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:00.096320 23026 raft_consensus.cc:2393] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 11: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:09:59.689534 27860 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:09:59.449254 23027 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 806) took 1012 ms. Trace:
I20260709 13:10:00.011024 23020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 810) took 1566 ms. Trace:
I20260709 13:09:59.285079 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:00.101814 22952 raft_consensus.cc:2410] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:09:59.976824 27982 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:59.288887 27955 consensus_queue.cc:1048] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.861s
I20260709 13:10:00.102470 27982 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:00.106231 27982 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:00.012784 27962 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:59.358294 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:59.977003 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:09:59.982422 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:09:59.287510 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:09:59.986850 23028 raft_consensus.cc:1240] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 10. Current term is 11. Ops: [7.6-10.8]
W20260709 13:09:59.944878 23025 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.682s	user 0.016s	sys 0.020s
I20260709 13:09:59.978809 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:59.452876 23590 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:09:59.987912 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:09:59.378471 22843 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:09:59.989717 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:59.295593 27912 consensus_meta.cc:287] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.937s	user 0.013s	sys 0.000s
I20260709 13:09:59.960944 27970 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:09:59.998895 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:00.098554 27746 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:09:59.959975 27958 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:00.000034 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:09:59.989073 27289 consensus_meta.cc:287] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.828s	user 0.009s	sys 0.009s
W20260709 13:09:59.416795 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 156292us
I20260709 13:10:00.106983 23020 rpcz_store.cc:276] 0709 13:09:58.444593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:58.444982 (+   389us) service_pool.cc:224] Handling call
0709 13:09:58.697284 (+252302us) spinlock_profiling.cc:243] Waited 9.16 ms on lock 0x7b58001ca198. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0a22 00007f613bbd9c09 00007f613ad33ee0 00007f613adfd0ab 00007f613adfd131 00007f613adbefc0 00007f613ada9929 00007f613ada93d2 00007f6134320bd4
0709 13:09:58.697495 (+   211us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:58.829791 (+132296us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:58.829842 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:58.834763 (+  4921us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:09:58.834809 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:00.010663 (+1175854us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b540010c054. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:00.010718 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:00.010861 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":303751296}
I20260709 13:09:59.297930 27356 raft_consensus.cc:2749] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:09:59.459009 27698 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:09:59.295727 27830 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:00.068305 27969 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:00.070744 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:00.073045 22773 catalog_manager.cc:5697] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:00.075073 23024 raft_consensus.cc:1275] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 13:10:00.083123 27764 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:00.085395 27826 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:59.260787 27926 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:00.164469 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:00.165097 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:00.165551 22880 raft_consensus.cc:2393] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 16: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:00.165889 22873 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 13.
I20260709 13:10:00.166635 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:00.166811 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:00.094059 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:00.095227 23022 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
I20260709 13:10:00.166706 22773 catalog_manager.cc:5697] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 6 to 9, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 9 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:00.170097 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:00.170945 22873 raft_consensus.cc:2410] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 15 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 13:10:00.170387 22773 catalog_manager.cc:5697] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 8 to 10, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 10 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:00.172124 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:00.016554 23623 raft_consensus.cc:1275] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:10:00.172734 22873 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
I20260709 13:09:59.917876 27604 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:59.275699 27952 raft_consensus.cc:2749] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:10:00.097955 22918 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.920s	user 0.000s	sys 0.024s
W20260709 13:10:00.174430 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 1.113e+06us
I20260709 13:10:00.014863 27821 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:00.175374 27821 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:00.177937 27821 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:09:59.980177 27911 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:00.096792 23027 rpcz_store.cc:276] 0709 13:09:58.437283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:58.437466 (+   183us) service_pool.cc:224] Handling call
0709 13:09:58.604993 (+167527us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:58.769362 (+164369us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:09:58.809279 (+ 39917us) log.cc:844] Serialized 14 byte log entry
0709 13:09:59.060537 (+251258us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:59.096891 (+ 36354us) log.cc:844] Serialized 52 byte log entry
0709 13:09:59.097042 (+   151us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:09:59.098086 (+  1044us) log.cc:844] Serialized 14 byte log entry
0709 13:09:59.117108 (+ 19022us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:09:59.448711 (+331603us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b5400135454. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:09:59.448777 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:09:59.448946 (+   169us) raft_consensus.cc:1685] finished
0709 13:09:59.448968 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:09:59.449178 (+   210us) inbound_call.cc:177] Queueing success response
0709 13:09:59.925652 (+476474us) spinlock_profiling.cc:243] Waited 15.4 ms on lock 0x7b180043a140. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cc62a6 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cf8522 00007f6132cfa7c1
Metrics: {"mutex_wait_us":12709,"prepare.queue_time_us":36815,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":76,"raft.queue_time_us":70822,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":36402,"spinlock_wait_cycles":372748032,"thread_start_us":187698,"threads_started":1,"wal-append.queue_time_us":308730}
I20260709 13:09:59.946830 27953 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:00.179112 27953 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:00.178766 27911 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:00.098768 27975 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:00.107187 27962 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:00.107878 23026 raft_consensus.cc:1275] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 13:10:00.131486 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 678641us
I20260709 13:10:00.132121 22949 raft_consensus.cc:2393] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 11: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:00.000429 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:00.003934 27924 log.cc:927] Time spent T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 1.490s	user 0.009s	sys 0.020s
I20260709 13:10:00.132510 22947 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 14.
I20260709 13:10:00.139510 22875 raft_consensus.cc:2410] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 13 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 13:10:00.140221 22879 raft_consensus.cc:2410] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 9 because replica has last-logged OpId of term: 8 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:10:00.143083 23021 raft_consensus.cc:2468] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 11.
I20260709 13:10:00.145864 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:00.190819 22871 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:00.147234 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:00.093086 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 63534.7us
I20260709 13:10:00.167693 22880 raft_consensus.cc:2410] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:10:00.167667 23025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 2142) took 2264 ms. Trace:
I20260709 13:10:00.173492 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:00.139412 27982 raft_consensus.cc:2804] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:10:00.197852 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:00.198203 23588 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:00.200517 27911 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:00.200933 22918 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 2.151s (SENT) [suppressed 9 similar messages]
I20260709 13:10:00.201452 27826 raft_consensus.cc:2804] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:10:00.202067 27826 raft_consensus.cc:697] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 12 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:00.203022 27826 consensus_queue.cc:237] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:00.204180 27678 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:00.207546 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:00.205266 22874 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 14.
I20260709 13:10:00.208240 22952 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 13.
I20260709 13:10:00.108122 27977 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:00.209491 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:00.192982 27964 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:00.146874 27289 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:00.144183 27746 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:00.213376 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:00.215018 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:00.229964 23588 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:00.191715 23025 rpcz_store.cc:276] 0709 13:09:57.903497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:57.909068 (+  5571us) service_pool.cc:224] Handling call
0709 13:09:57.909438 (+   370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:09:58.160321 (+250883us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:09:58.421414 (+261093us) log.cc:844] Serialized 14 byte log entry
0709 13:09:59.262679 (+841265us) log.cc:844] Serialized 14 byte log entry
0709 13:10:00.132103 (+869424us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:00.139710 (+  7607us) log.cc:844] Serialized 26 byte log entry
0709 13:10:00.139950 (+   240us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:10:00.139989 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:00.140054 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:00.167464 (+ 27410us) raft_consensus.cc:1685] finished
0709 13:10:00.167483 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:00.167593 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7073,"prepare.queue_time_us":105500,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":72,"raft.queue_time_us":119643,"raft.run_cpu_time_us":233,"raft.run_wall_time_us":2200,"thread_start_us":2107774,"threads_started":4,"wal-append.queue_time_us":725232}
I20260709 13:10:00.265692 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:00.268728 23623 raft_consensus.cc:1240] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 14. Current term is 15. Ops: []
I20260709 13:10:00.269083 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:00.270004 23619 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:10:00.268774 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:00.276973 27873 raft_consensus.cc:697] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 12 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
W20260709 13:10:00.277966 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 998589us
I20260709 13:10:00.214148 27982 raft_consensus.cc:697] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:00.287292 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:00.282641 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:00.206769 27821 raft_consensus.cc:2804] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:00.213161 27956 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:00.315917 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:00.318389 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:00.319041 22873 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:00.291545 23029 raft_consensus.cc:1240] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 13. Current term is 14. Ops: []
I20260709 13:10:00.317629 22994 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:00.205374 27803 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:00.319916 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:00.172298 22773 catalog_manager.cc:5697] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 7 to 11, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 11 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:00.322263 22879 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 14.
I20260709 13:10:00.285738 27950 raft_consensus.cc:2804] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:10:00.319839 27971 log.cc:927] Time spent T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:10:00.323361 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:00.323626 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:00.328168 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 127266us
I20260709 13:10:00.332034 23027 raft_consensus.cc:2410] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:10:00.328879 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:00.323860 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:00.281982 27954 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
W20260709 13:10:00.334172 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 53204us
I20260709 13:10:00.334741 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:00.316347 27821 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
W20260709 13:10:00.339249 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 50294.1us
I20260709 13:10:00.340484 22877 raft_consensus.cc:2368] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 12. Current term is 13.
I20260709 13:10:00.331152 22875 raft_consensus.cc:2393] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 15: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:00.281917 27764 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:00.338558 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 69714.5us
I20260709 13:10:00.343066 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:00.344638 23025 raft_consensus.cc:2393] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 12: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:00.346027 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:00.321713 27979 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:00.332072 27950 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:10:00.352025 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 122194us
I20260709 13:10:00.352486 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:00.353307 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:00.355705 22950 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 13.
I20260709 13:10:00.327086 27990 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:00.358352 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:00.345909 22918 connection.cc:256] RPC call timeout handler was delayed by -1.2858507299999928s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.26230000000000003s for a response
I20260709 13:10:00.359771 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:00.265161 27964 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:00.361200 22918 connection.cc:256] RPC call timeout handler was delayed by -1.2714507299999696s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.26389999999999997s for a response
I20260709 13:10:00.345065 27821 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:10:00.364574 22918 connection.cc:256] RPC call timeout handler was delayed by -1.2478490659999579s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19890000000000002s for a response
W20260709 13:10:00.365494 22918 connection.cc:256] RPC call timeout handler was delayed by -1.2478490659999579s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19890000000000002s for a response
W20260709 13:10:00.365790 22918 connection.cc:256] RPC call timeout handler was delayed by -1.1785490659999596s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2066s for a response
W20260709 13:10:00.366243 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:00.322888 27956 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:00.369225 27830 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:00.369110 27988 consensus_queue.cc:1048] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 13:10:00.357663 27979 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:00.369831 27983 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:00.392014 22951 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58320 (request call id 2879) took 1708 ms. Trace:
I20260709 13:10:00.392459 22951 rpcz_store.cc:276] 0709 13:09:58.683288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:58.728245 (+ 44957us) service_pool.cc:224] Handling call
0709 13:10:00.388957 (+1660712us) spinlock_profiling.cc:243] Waited 473 ms on lock 0x7b54003796d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:00.391937 (+  2980us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1088165376}
I20260709 13:10:00.392478 27983 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:00.377501 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:00.378022 27873 consensus_queue.cc:237] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:00.360352 27764 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:00.375072 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:00.395869 22875 raft_consensus.cc:2410] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:10:00.357357 27950 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:10:00.381157 22918 connection.cc:256] RPC call timeout handler was delayed by -1.0845973850000519s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1763s for a response
I20260709 13:10:00.381078 27993 consensus_queue.cc:1048] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:10:00.381978 27990 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:00.383442 27977 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:00.383644 22955 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 810) took 1703 ms. Trace:
I20260709 13:10:00.384269 27958 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:00.418773 22955 rpcz_store.cc:276] 0709 13:09:58.679736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:58.732516 (+ 52780us) service_pool.cc:224] Handling call
0709 13:09:58.732948 (+   432us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:09:59.013614 (+280666us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:09:59.013658 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:09:59.038282 (+ 24624us) log.cc:844] Serialized 52 byte log entry
0709 13:09:59.996024 (+957742us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:09:59.996062 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:00.383062 (+387000us) spinlock_profiling.cc:243] Waited 232 ms on lock 0x7b540023d7d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:00.383113 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:00.383162 (+    49us) raft_consensus.cc:1685] finished
0709 13:10:00.383174 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:00.383576 (+   402us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24730,"raft.queue_time_us":227815,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":7900,"spinlock_wait_cycles":534749440,"thread_start_us":870216,"threads_started":1,"wal-append.queue_time_us":830153}
I20260709 13:10:00.384562 27944 raft_consensus.cc:2749] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:00.386183 27807 raft_consensus.cc:3055] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:10:00.393352 27979 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:00.351171 27912 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:00.405831 22874 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 14.
I20260709 13:10:00.398176 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:00.394025 27970 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:00.408313 27764 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:00.359941 27709 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:00.409807 27911 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:00.411407 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:00.379129 27821 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:00.416124 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:00.379808 22873 raft_consensus.cc:2410] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:10:00.467145 22875 raft_consensus.cc:1240] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 14. Current term is 15. Ops: []
W20260709 13:10:00.417914 22918 connection.cc:256] RPC call timeout handler was delayed by -1.036349065999957s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22240000000000004s for a response
I20260709 13:10:00.422605 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:00.468719 22946 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 15.
I20260709 13:10:00.390946 27954 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:00.470124 27956 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:00.470875 22873 raft_consensus.cc:1240] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 15. Current term is 16. Ops: []
I20260709 13:10:00.471504 27954 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:00.472046 27988 consensus_queue.cc:1059] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:10:00.457849 22843 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:00.466774 27709 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:00.427052 27975 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:00.426285 27807 raft_consensus.cc:740] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
W20260709 13:10:00.480818 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 163225us
I20260709 13:10:00.498848 23026 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 12.
I20260709 13:10:00.499881 22994 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:10:00.500835 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 148358us
I20260709 13:10:00.501447 22918 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:00.501852 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:00.394299 27982 consensus_queue.cc:237] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:00.506034 22874 raft_consensus.cc:1275] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:10:00.506435 22994 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:00.509080 22955 raft_consensus.cc:1275] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 13:10:00.501000 27956 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:00.512472 22994 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:00.513538 22992 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:00.521571 27988 raft_consensus.cc:3055] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:10:00.522064 27988 raft_consensus.cc:740] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:00.523021 27988 consensus_queue.cc:260] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:00.465930 27912 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:00.524389 27988 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:10:00.456427 22773 catalog_manager.cc:5697] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 3 to 7. New cstate: current_term: 7 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
W20260709 13:10:00.529332 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:00.526333 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 68566.7us
I20260709 13:10:00.507195 27764 raft_consensus.cc:2749] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:00.532968 22843 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:00.534796 22871 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 15.
I20260709 13:10:00.536098 22994 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:00.510085 27289 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:00.537879 22843 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:00.539726 22875 raft_consensus.cc:1240] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 14. Current term is 15. Ops: [12.7-14.8]
W20260709 13:10:00.541807 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 143843us
I20260709 13:10:00.476954 27911 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:00.548060 22994 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:00.525900 27807 consensus_queue.cc:260] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:00.549996 27709 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:00.551604 28006 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:10:00.553241 22773 catalog_manager.cc:5697] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 8 to 11, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 11 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:00.526211 27678 raft_consensus.cc:2749] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:00.554479 28005 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 13:10:00.581352 22918 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:00.555011 27944 raft_consensus.cc:2749] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:00.583304 22918 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:00.583911 27764 raft_consensus.cc:2749] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:00.584969 27807 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:00.586016 27764 raft_consensus.cc:2804] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:00.586465 27764 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:00.585484 22773 catalog_manager.cc:5697] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 9 to 10, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 10 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:00.586830 27764 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:00.555800 27983 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:00.556245 27803 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:00.556838 27289 raft_consensus.cc:2749] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:00.531359 27956 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:00.591182 27956 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:00.591234 27707 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:00.542564 27588 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:00.558528 27912 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:00.560570 27911 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:00.539538 27975 raft_consensus.cc:2749] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:00.524559 27953 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:00.581010 27709 raft_consensus.cc:2804] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:00.599027 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:10:00.576283 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 62852.5us
I20260709 13:10:00.584554 22918 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:00.600541 22992 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:10:00.601132 22843 connection.cc:256] RPC call timeout handler was delayed by -2.3932327220001071s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20790000000000003s for a response
W20260709 13:10:00.601704 22843 connection.cc:256] RPC call timeout handler was delayed by -2.308632722000084s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21730000000000002s for a response
I20260709 13:10:00.576311 28006 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:00.561478 27873 raft_consensus.cc:2804] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:10:00.604485 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:10:00.605167 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:00.602093 22992 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:00.602535 27709 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:00.606480 27911 raft_consensus.cc:2749] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:00.607336 27500 raft_consensus.cc:2749] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:00.603657 27678 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:00.602638 28006 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:00.606518 27709 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:00.609692 27953 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:00.593205 27990 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:00.610019 28004 consensus_queue.cc:1048] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 13:10:00.604272 27873 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:00.611797 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:00.617297 27873 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:00.615832 27975 raft_consensus.cc:2804] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:10:00.622287 27975 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:00.622622 27975 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:00.625063 22919 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:00.625885 27736 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:00.626504 27736 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:00.628795 27960 raft_consensus.cc:2804] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:10:00.628935 27678 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:00.629607 27736 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:00.631538 27678 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:00.633750 22949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52222 (request call id 811) took 1953 ms. Trace:
I20260709 13:10:00.614202 22918 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:00.634148 22949 rpcz_store.cc:276] 0709 13:09:58.680161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:58.680647 (+   486us) service_pool.cc:224] Handling call
0709 13:10:00.633551 (+1952904us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b54003af054. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:00.633680 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3231482752}
I20260709 13:10:00.634738 27959 raft_consensus.cc:2804] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:10:00.635133 27959 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:00.635489 27959 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:00.629405 27960 raft_consensus.cc:697] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:00.650254 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:00.651063 22955 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 13.
I20260709 13:10:00.654893 27960 consensus_queue.cc:237] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:00.656822 28006 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:00.658727 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 44591.8us
I20260709 13:10:00.588008 22773 catalog_manager.cc:5697] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 7 to 11. New cstate: current_term: 11 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
W20260709 13:10:00.748082 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 88530.8us
I20260709 13:10:00.756637 22918 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:00.758957 27915 raft_consensus.cc:3055] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:10:00.759377 27915 raft_consensus.cc:740] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:00.763470 27709 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:00.769116 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:00.769896 23026 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 11.
I20260709 13:10:00.770640 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:00.771021 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:00.771757 23026 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 16.
I20260709 13:10:00.771786 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:00.773233 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:00.773952 23025 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 14.
I20260709 13:10:00.774461 23039 tablet_copy_service.cc:342] P f3c799b1249e40119d2a5864231c6c75: Request end of tablet copy session 5b346767a89745fa82928a5dac9cc0b2-ac5cc94926a742e6a0c1340ec33c0ff0 received from {username='slave'} at 127.0.0.1:32794
I20260709 13:10:00.774935 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:00.775767 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:00.776468 27764 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:00.775130 23039 tablet_copy_service.cc:434] P f3c799b1249e40119d2a5864231c6c75: ending tablet copy session 5b346767a89745fa82928a5dac9cc0b2-ac5cc94926a742e6a0c1340ec33c0ff0 on tablet ac5cc94926a742e6a0c1340ec33c0ff0 with peer 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:00.777987 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:00.776484 23027 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 14.
I20260709 13:10:00.776008 23025 raft_consensus.cc:2410] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:10:00.658919 27990 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:00.747258 27730 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:00.794909 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:00.794390 23026 raft_consensus.cc:2468] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 11.
I20260709 13:10:00.796315 22951 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11.
I20260709 13:10:00.803836 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:00.803700 22949 raft_consensus.cc:1275] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:10:00.760149 27915 consensus_queue.cc:260] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:00.813045 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:00.813383 22992 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:00.813949 22952 raft_consensus.cc:2410] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 13:10:00.792909 27625 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:00.753057 27979 consensus_queue.cc:1048] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.044s
W20260709 13:10:00.810043 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 195054us
I20260709 13:10:00.814637 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:00.833156 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:00.833823 22951 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:10:00.818557 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 61951.3us
I20260709 13:10:00.804333 27730 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:00.802070 27990 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:00.820124 22877 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35370 (request call id 2715) took 1545 ms (client timeout 1713 ms). Trace:
I20260709 13:10:00.820412 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:00.824950 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:00.826555 27746 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:00.826862 28006 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:00.827445 27962 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:00.836699 22877 rpcz_store.cc:269] 0709 13:09:59.274416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:59.274717 (+   301us) service_pool.cc:224] Handling call
0709 13:10:00.819897 (+1545180us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b540022b5d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:00.820038 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":460958080}
I20260709 13:10:00.836930 22946 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
I20260709 13:10:00.834797 27625 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:00.842512 22949 raft_consensus.cc:1240] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 14. Current term is 15. Ops: []
I20260709 13:10:00.810367 27500 raft_consensus.cc:2804] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 13:10:00.844316 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 282413us
W20260709 13:10:00.845135 22843 connection.cc:256] RPC call timeout handler was delayed by -1.8460327220000181s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2687s for a response
W20260709 13:10:00.847257 22843 connection.cc:256] RPC call timeout handler was delayed by -1.8190153299999565s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15940000000000001s for a response
W20260709 13:10:00.847625 22843 connection.cc:256] RPC call timeout handler was delayed by -1.7965153300000338s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16190000000000002s for a response
I20260709 13:10:00.849721 27990 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:00.851786 27730 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:00.852772 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:00.853684 27764 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:00.854416 22843 connection.cc:256] RPC call timeout handler was delayed by -1.601232722000077s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2959s for a response
W20260709 13:10:00.854811 22843 connection.cc:256] RPC call timeout handler was delayed by -1.585858442000017s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20320000000000002s for a response
W20260709 13:10:00.855157 22843 connection.cc:256] RPC call timeout handler was delayed by -1.4758088540000927s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1878s for a response
W20260709 13:10:00.855523 22843 connection.cc:256] RPC call timeout handler was delayed by -1.3942153299999518s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2066s for a response
W20260709 13:10:00.855911 22843 connection.cc:256] RPC call timeout handler was delayed by -1.2913544700001012s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1623s for a response
W20260709 13:10:00.856289 22843 connection.cc:256] RPC call timeout handler was delayed by -1.280317445000037s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19840000000000002s for a response
I20260709 13:10:00.843019 27873 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:00.853439 22955 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
I20260709 13:10:00.862241 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:00.863365 22955 raft_consensus.cc:2410] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:10:00.864305 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:00.859831 22843 connection.cc:256] RPC call timeout handler was delayed by -1.1671088540000483s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22210000000000002s for a response
W20260709 13:10:00.865047 22843 connection.cc:256] RPC call timeout handler was delayed by -1.0015088540001216s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2405s for a response
I20260709 13:10:00.839483 27977 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:00.860966 22951 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 14.
I20260709 13:10:00.870867 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:00.843704 27915 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:00.872725 28023 consensus_queue.cc:1059] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:10:00.874281 22994 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:00.859159 27678 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:00.887818 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:00.904336 22875 raft_consensus.cc:2393] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 11: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:00.888900 27990 raft_consensus.cc:2749] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:10:00.888944 28004 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
W20260709 13:10:00.893147 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:00.892441 22773 catalog_manager.cc:5697] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 8 to 10, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 10 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:00.854300 27807 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:00.894438 27709 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:00.894820 27873 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:00.895967 27625 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:00.928877 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:00.897591 27959 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:00.897941 22920 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:00.898559 22843 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:00.899755 27826 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:00.901255 22955 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
I20260709 13:10:00.933070 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:00.933720 22955 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:10:00.883263 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 70021.3us
I20260709 13:10:00.906889 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:00.909626 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:00.906687 22953 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2883) took 2140 ms. Trace:
I20260709 13:10:00.914089 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:00.865023 27500 raft_consensus.cc:697] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:00.924002 22877 raft_consensus.cc:1275] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:10:00.924819 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:00.936543 22992 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:00.936910 22874 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:10:00.925678 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:00.937695 22879 raft_consensus.cc:2393] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 15: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
W20260709 13:10:00.927915 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 53719.6us
I20260709 13:10:00.938757 22994 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:00.931317 22920 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:00.940819 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:00.929451 27919 raft_consensus.cc:3055] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:10:00.901295 27977 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:00.933152 27925 raft_consensus.cc:2804] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:10:00.942790 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:00.933780 27988 raft_consensus.cc:2749] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:00.932443 27719 raft_consensus.cc:2804] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:10:00.915547 28004 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:00.959141 22992 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:00.960644 27964 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:00.947139 22954 raft_consensus.cc:2393] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 10: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
W20260709 13:10:01.058770 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:00.967265 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:01.059990 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:10:00.957733 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 59297.9us
I20260709 13:10:01.067793 22949 raft_consensus.cc:2393] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 13: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:01.067924 22955 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15.
I20260709 13:10:01.067875 22951 raft_consensus.cc:2393] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 16: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:00.957677 22994 leader_election.cc:400] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 12. Current term is 13.
I20260709 13:10:00.927131 27950 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:00.914674 27678 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:10:01.065181 22876 raft_consensus.cc:2368] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 15. Current term is 16.
I20260709 13:10:00.935359 22953 rpcz_store.cc:276] 0709 13:09:58.760650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:58.791164 (+ 30514us) service_pool.cc:224] Handling call
0709 13:09:58.809523 (+ 18359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:09:58.996978 (+187455us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:09:58.997349 (+   371us) log.cc:844] Serialized 14 byte log entry
0709 13:09:59.175645 (+178296us) log.cc:844] Serialized 14 byte log entry
0709 13:09:59.247963 (+ 72318us) log.cc:844] Serialized 14 byte log entry
0709 13:09:59.248375 (+   412us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:09:59.248489 (+   114us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:09:59.248568 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:00.876389 (+1627821us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b540020ec54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:00.876432 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:00.901089 (+ 24657us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":199798,"prepare.queue_time_us":28334,"prepare.run_cpu_time_us":276,"prepare.run_wall_time_us":1790,"raft.queue_time_us":578768,"raft.run_cpu_time_us":11840,"raft.run_wall_time_us":615144,"spinlock_wait_cycles":884845184,"thread_start_us":244043,"threads_started":2,"wal-append.queue_time_us":255134,"wal-append.run_cpu_time_us":1909,"wal-append.run_wall_time_us":1032978}
I20260709 13:10:00.947212 27911 raft_consensus.cc:2804] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:10:00.961815 27990 raft_consensus.cc:2749] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:10:01.070598 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:01.069056 22994 leader_election.cc:403] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:10:01.068872 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:00.930411 27807 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:00.945864 27919 raft_consensus.cc:740] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:00.961308 27839 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:10:01.074139 23021 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32794 (request call id 807) took 2619 ms (client timeout 1714 ms). Trace:
I20260709 13:10:00.960978 27912 consensus_queue.cc:1048] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.119s
I20260709 13:10:01.075451 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:00.942082 27709 consensus_queue.cc:1048] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.128s
I20260709 13:10:00.967016 27925 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:10:01.080468 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 122996us
I20260709 13:10:01.082396 23621 raft_consensus.cc:1240] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14. Ops: []
I20260709 13:10:01.082489 27950 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:01.082819 27807 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:00.938575 27826 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:01.083757 22871 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:01.084167 28023 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:10:01.084327 27678 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:01.085027 27839 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:01.071807 27719 raft_consensus.cc:697] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:01.073222 27289 raft_consensus.cc:2804] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:10:01.074668 27911 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:01.064882 27952 consensus_queue.cc:1048] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
W20260709 13:10:01.070515 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 111441us
I20260709 13:10:01.066475 27956 consensus_queue.cc:1059] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 13:10:01.101022 22953 raft_consensus.cc:1275] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:10:01.097990 27925 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:10:01.105607 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:01.118460 22872 raft_consensus.cc:2393] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 16: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:01.105980 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:01.118841 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:01.102623 23024 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:10:01.119743 23024 raft_consensus.cc:606] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:01.120105 23024 raft_consensus.cc:614] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:10:01.114565 27289 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:01.118871 22875 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
I20260709 13:10:01.120882 27289 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:01.121455 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:01.122527 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:01.123068 22875 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 14.
I20260709 13:10:01.124132 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:01.124739 23024 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:01.125414 27977 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:01.126883 28004 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:01.127888 27821 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:01.129182 27956 raft_consensus.cc:3055] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:10:01.129545 27956 raft_consensus.cc:740] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:01.110347 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:01.113816 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:01.132016 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:01.132578 23022 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
I20260709 13:10:01.133396 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:01.134109 23022 raft_consensus.cc:2410] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 15 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:10:01.096431 27709 consensus_queue.cc:1243] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer aed3f089ec8b49af9c8cf5cdee76e8e7 log is divergent from this leader: its last log entry 13.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:10:01.114144 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:01.112798 22843 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:01.119844 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:01.115772 22951 raft_consensus.cc:1240] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 13. Current term is 14. Ops: [11.9-13.10]
I20260709 13:10:01.106797 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:01.129626 27975 consensus_meta.cc:287] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.507s	user 0.010s	sys 0.000s
I20260709 13:10:01.115664 22992 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:10:01.085376 23021 rpcz_store.cc:269] 0709 13:09:58.443567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:58.485949 (+ 42382us) service_pool.cc:224] Handling call
0709 13:10:01.062404 (+2576455us) spinlock_profiling.cc:243] Waited 1.54 s on lock 0x7b54001c70d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:01.062494 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3532541824}
I20260709 13:10:01.137285 22876 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
I20260709 13:10:01.139480 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:01.140268 22873 raft_consensus.cc:2410] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 12 because replica has last-logged OpId of term: 11 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 13:10:01.112885 27839 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:01.112638 27990 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:01.165572 27990 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:01.167855 27990 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:01.169240 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:01.169955 22873 raft_consensus.cc:2468] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 10.
I20260709 13:10:01.096791 27500 consensus_queue.cc:237] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:01.142167 23024 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15.
I20260709 13:10:01.150452 27919 consensus_queue.cc:260] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:01.151312 28022 log.cc:927] Time spent T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 13:10:01.152040 28027 log.cc:927] Time spent T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
W20260709 13:10:01.157369 28026 log.cc:927] Time spent T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:10:01.158139 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:01.159829 27826 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:01.116467 27911 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:01.161057 27912 raft_consensus.cc:2804] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:10:01.161734 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:01.184222 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:01.185248 23027 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 12.
I20260709 13:10:01.185537 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:01.186192 23022 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 12.
I20260709 13:10:01.187080 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:01.188208 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:10:01.162510 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 49819.6us
I20260709 13:10:01.163311 28004 raft_consensus.cc:2749] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:10:01.195482 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:01.171865 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:01.174273 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:01.176120 22871 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 16.
I20260709 13:10:01.176422 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:01.201021 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:01.178442 22874 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 16.
I20260709 13:10:01.178728 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:01.189545 23022 raft_consensus.cc:2376] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 9. Re-sending same reply.
I20260709 13:10:01.197166 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:01.197932 23020 raft_consensus.cc:1240] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:10:01.203158 23024 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 10.
W20260709 13:10:01.201967 23023 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33418 (request call id 2081) took 2048 ms (client timeout 2038 ms). Trace:
I20260709 13:10:01.145908 27975 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:01.204435 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:01.203688 23023 rpcz_store.cc:269] 0709 13:09:59.150806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:59.156197 (+  5391us) service_pool.cc:224] Handling call
0709 13:10:01.197148 (+2040951us) spinlock_profiling.cc:243] Waited 47.9 ms on lock 0x7b5400249854. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:01.199216 (+  2068us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":110097664}
I20260709 13:10:01.184352 27912 raft_consensus.cc:697] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 14 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:10:01.207003 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:01.211826 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:01.136960 27719 consensus_queue.cc:237] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:01.216230 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:01.217768 22878 raft_consensus.cc:1240] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:10:01.218321 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:01.220247 22843 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:01.220502 23023 raft_consensus.cc:2410] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 15 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:10:01.222239 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:01.222796 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:01.223268 23022 raft_consensus.cc:2376] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 14. Re-sending same reply.
I20260709 13:10:01.224884 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:01.226178 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:01.226712 22953 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11.
I20260709 13:10:01.235093 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:01.235426 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:01.181124 27956 consensus_queue.cc:260] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:01.237586 22948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 820) took 1977 ms. Trace:
I20260709 13:10:01.245800 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:01.246802 23024 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 15.
I20260709 13:10:01.237648 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:01.249085 22992 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:01.245748 22948 rpcz_store.cc:276] 0709 13:09:59.260245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:59.329893 (+ 69648us) service_pool.cc:224] Handling call
0709 13:09:59.330299 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:00.180967 (+850668us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:00.181008 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:00.181063 (+    55us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:10:00.181107 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:01.237392 (+1056285us) spinlock_profiling.cc:243] Waited 850 ms on lock 0x7b5400379bd4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:01.237434 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:01.237538 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1955717888}
I20260709 13:10:01.237500 22878 raft_consensus.cc:2393] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 16: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:01.251593 23022 raft_consensus.cc:2393] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 14: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:01.212956 28004 consensus_queue.cc:1059] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 13:10:01.237100 27826 raft_consensus.cc:2749] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:10:01.262885 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:01.267591 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:01.263649 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 47664.7us
I20260709 13:10:01.264601 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:01.249939 27646 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:01.270098 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:01.272254 23022 raft_consensus.cc:2393] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 15: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:01.254801 27709 raft_consensus.cc:2804] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:10:01.267778 23023 raft_consensus.cc:2376] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11. Re-sending same reply.
I20260709 13:10:01.273247 22843 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:01.275852 28029 consensus_queue.cc:1048] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:10:01.276196 22920 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:01.280184 28029 raft_consensus.cc:2804] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:10:01.280725 28029 raft_consensus.cc:697] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:01.281617 28029 consensus_queue.cc:237] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:01.286988 27678 raft_consensus.cc:2749] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:01.294821 27289 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:01.297348 23022 raft_consensus.cc:1275] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
I20260709 13:10:01.297696 23022 pending_rounds.cc:85] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75: Aborting all ops after (but not including) 7
I20260709 13:10:01.297935 23022 pending_rounds.cc:107] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:10:01.298148 23022 raft_consensus.cc:2889] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:01.299288 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:01.299782 23022 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:01.301244 23023 raft_consensus.cc:1275] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:10:01.301738 27911 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:01.302238 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:01.302309 22994 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:01.303192 27724 raft_consensus.cc:2749] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:01.302817 23023 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:01.237223 27912 consensus_queue.cc:237] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:01.312376 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:01.277395 27709 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:10:01.322408 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:01.324715 27289 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:01.325062 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:01.325975 23025 raft_consensus.cc:2393] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 13: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:01.319233 23022 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 15.
I20260709 13:10:01.334607 23021 raft_consensus.cc:2368] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 12. Current term is 13.
I20260709 13:10:01.305653 28023 raft_consensus.cc:2749] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 12. Current term is 13.
I20260709 13:10:01.338752 27975 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:01.339525 27925 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:01.341229 22948 raft_consensus.cc:1240] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 10. Current term is 11. Ops: []
I20260709 13:10:01.341945 22843 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:01.343187 22843 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:01.345122 22843 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:01.322568 27709 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:01.360522 27975 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:01.367022 27975 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:01.369331 27975 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:01.370514 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:01.347931 28004 raft_consensus.cc:2749] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:01.353912 27988 raft_consensus.cc:2749] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:01.353083 22843 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:01.375571 23029 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 2093) took 2162 ms. Trace:
I20260709 13:10:01.386535 23029 rpcz_store.cc:276] 0709 13:09:59.213044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:00.011753 (+798709us) service_pool.cc:224] Handling call
0709 13:10:00.012120 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:00.320599 (+308479us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:01.375287 (+1054688us) spinlock_profiling.cc:243] Waited 279 ms on lock 0x7b54001c89d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:01.375498 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":642204032}
I20260709 13:10:01.353139 23022 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:10:01.389922 23022 raft_consensus.cc:606] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Received request to transfer leadership to TS aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:01.389793 22992 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:01.372282 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:01.368422 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:01.391203 22949 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:01.393925 23023 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 16.
W20260709 13:10:01.397432 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 121291us
I20260709 13:10:01.348223 27807 raft_consensus.cc:2749] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:01.400688 27996 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:01.366111 27911 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:01.353549 27925 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:01.365828 27977 raft_consensus.cc:3055] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:10:01.412856 27977 raft_consensus.cc:740] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:01.413664 27977 consensus_queue.cc:260] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:01.413997 23027 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
W20260709 13:10:01.408914 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 55893.6us
I20260709 13:10:01.418196 22843 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:01.409433 27996 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:01.420315 27996 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:01.415020 27977 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:01.422462 27826 raft_consensus.cc:2749] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:01.426642 27926 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:01.428434 22843 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:01.431674 27826 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:10:01.434885 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:01.435461 22947 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:01.401624 27975 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:01.446539 22954 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 16.
I20260709 13:10:01.447508 22947 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 14.
I20260709 13:10:01.448033 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:01.449018 22992 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:01.450282 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:01.442443 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:01.443403 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:01.460978 22843 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:01.460805 22948 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:10:01.404703 23588 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:01.463048 23028 raft_consensus.cc:1240] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 14. Current term is 15. Ops: []
I20260709 13:10:01.464313 27996 raft_consensus.cc:2804] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:10:01.464798 27996 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:01.464888 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:01.465240 27996 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:01.465695 22954 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:10:01.462954 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 58341.9us
I20260709 13:10:01.469511 23588 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:01.470358 27821 raft_consensus.cc:2749] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:10:01.471881 23588 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:01.473281 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:01.398638 27988 raft_consensus.cc:2749] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:10:01.475162 23588 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:01.483876 22954 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 13.
I20260709 13:10:01.490569 23588 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:01.476078 22843 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:01.492429 22992 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:01.494860 23588 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:01.493170 22843 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:01.503862 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:01.491437 27915 consensus_meta.cc:287] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.586s	user 0.011s	sys 0.000s
I20260709 13:10:01.518265 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:01.510928 22954 pending_rounds.cc:85] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting all ops after (but not including) 12
I20260709 13:10:01.505682 27867 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:01.524696 23024 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 15.
I20260709 13:10:01.521095 23025 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 12.
I20260709 13:10:01.525681 22949 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 14.
I20260709 13:10:01.527869 22947 raft_consensus.cc:1240] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14. Ops: [11.9-13.10]
I20260709 13:10:01.528254 22954 pending_rounds.cc:107] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting uncommitted NO_OP operation due to leader change: 13.13
I20260709 13:10:01.528713 22954 raft_consensus.cc:2889] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:01.495340 27911 raft_consensus.cc:2804] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:10:01.495258 27821 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:01.503451 28004 raft_consensus.cc:2749] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:01.505934 27811 raft_consensus.cc:2804] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:01.505689 27873 raft_consensus.cc:2804] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:01.515671 28023 raft_consensus.cc:2804] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:10:01.551628 22950 raft_consensus.cc:1275] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:10:01.474999 27807 consensus_queue.cc:1048] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.060s
I20260709 13:10:01.491016 27926 consensus_queue.cc:1059] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.060s
W20260709 13:10:01.560873 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 67777.4us
I20260709 13:10:01.490556 27678 consensus_queue.cc:1048] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 13:10:01.436640 27839 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
W20260709 13:10:01.562042 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 57159.4us
I20260709 13:10:01.534189 27988 raft_consensus.cc:2804] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:10:01.535753 27867 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:01.552798 27821 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:01.566190 22843 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:01.561542 27873 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:01.557865 27811 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:01.557879 27911 raft_consensus.cc:697] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 14 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:01.576395 22955 raft_consensus.cc:2393] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 15: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:01.577988 27768 raft_consensus.cc:2804] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:10:01.578482 27768 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:01.578862 27768 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:01.581893 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:01.570046 28023 raft_consensus.cc:697] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:01.559100 27709 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:01.587770 22876 raft_consensus.cc:1275] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:10:01.572886 27867 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:01.573019 27821 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:01.584211 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:01.592682 22874 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 14.
I20260709 13:10:01.575012 27873 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:01.576166 27811 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:01.600924 28047 consensus_queue.cc:1059] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:10:01.604956 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:01.605832 28023 consensus_queue.cc:237] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:01.585968 28004 raft_consensus.cc:2804] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:10:01.608991 27709 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:01.609160 28004 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:01.609582 28004 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:10:01.606806 27873 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:01.610669 27839 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:01.605173 27807 raft_consensus.cc:3055] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:10:01.574527 27988 raft_consensus.cc:697] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 15 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:01.620268 22948 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 13.
I20260709 13:10:01.630600 27911 consensus_queue.cc:237] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:01.623517 27807 raft_consensus.cc:740] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:01.638166 27807 consensus_queue.cc:260] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:01.645125 27867 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:01.645586 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:01.647765 27867 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:01.653280 27929 raft_consensus.cc:3055] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:10:01.653889 27929 raft_consensus.cc:740] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:01.654824 27929 consensus_queue.cc:260] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:01.655859 27929 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:10:01.668599 27919 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:01.656435 27826 raft_consensus.cc:2749] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:10:01.644346 27996 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:01.681430 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:10:01.681483 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 105983us
I20260709 13:10:01.682614 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:01.683213 23619 raft_consensus.cc:2393] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 16: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:01.685427 23590 leader_election.cc:400] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 election: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 15. Current term is 16.
I20260709 13:10:01.675256 27873 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:01.687270 22952 raft_consensus.cc:1240] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:10:01.687688 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:01.697016 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:01.697624 22954 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:01.700121 27975 raft_consensus.cc:2749] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 13:10:01.703939 27821 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:01.705080 22954 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
I20260709 13:10:01.706246 27988 consensus_queue.cc:237] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:01.707293 27811 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:01.735405 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:01.736217 22955 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:01.708938 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:01.709898 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:01.712492 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:01.716025 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:01.738235 22952 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 9.
I20260709 13:10:01.718452 22949 raft_consensus.cc:1275] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:10:01.719830 27996 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:01.729483 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:01.738373 22950 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 16.
I20260709 13:10:01.716920 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:01.739828 23026 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 9.
I20260709 13:10:01.740296 23619 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 16.
I20260709 13:10:01.740634 22953 raft_consensus.cc:2393] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 14: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
W20260709 13:10:01.745111 22918 leader_election.cc:336] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.690s (SENT)
I20260709 13:10:01.748515 28051 consensus_queue.cc:1059] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:10:01.746888 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:01.749370 28051 consensus_queue.cc:1243] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer aed3f089ec8b49af9c8cf5cdee76e8e7 log is divergent from this leader: its last log entry 13.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:10:01.749992 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:01.750828 23026 raft_consensus.cc:2368] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 14. Current term is 15.
I20260709 13:10:01.751575 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:01.752084 23026 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:10:01.756093 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:01.758060 22843 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:01.750047 27977 raft_consensus.cc:3055] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:10:01.760867 27977 raft_consensus.cc:740] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:01.761364 28004 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:01.761955 27977 consensus_queue.cc:260] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:01.763095 27977 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:10:01.764056 28050 log.cc:927] Time spent T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 13:10:01.764508 28004 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:01.766680 22843 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:01.770213 22843 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:01.780229 22918 leader_election.cc:336] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.764s (SENT)
W20260709 13:10:01.781867 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 94759.8us
I20260709 13:10:01.782721 22918 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:01.812613 27839 raft_consensus.cc:2749] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:10:01.814504 27950 consensus_queue.cc:1059] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:10:01.765117 27768 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:01.778520 28004 raft_consensus.cc:2804] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:10:01.773123 27873 raft_consensus.cc:2749] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:10:01.821159 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 134460us
W20260709 13:10:01.784492 22993 connection.cc:256] RPC call timeout handler was delayed by -1.0575499050000872s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18420000000000003s for a response
I20260709 13:10:01.821936 23588 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:01.785831 27811 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:01.822664 27768 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:01.823262 27996 raft_consensus.cc:3055] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:10:01.823608 27996 raft_consensus.cc:740] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:01.786252 27821 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:01.781628 27678 raft_consensus.cc:2749] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:01.793919 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:01.794528 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:01.829154 22871 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
I20260709 13:10:01.795313 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:01.795984 28047 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:01.796725 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:01.830888 22873 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:01.800181 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:01.832028 22875 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:01.806187 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:01.832894 22879 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:01.807255 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:01.810142 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:01.834328 23021 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:10:01.810739 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:01.778244 23026 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15.
I20260709 13:10:01.774317 22843 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:01.782070 22955 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15.
I20260709 13:10:01.824385 27996 consensus_queue.cc:260] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:01.798372 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:10:01.844657 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 70455us
I20260709 13:10:01.846071 22843 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:01.826553 22993 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:01.844530 23588 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:01.815429 23590 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:01.850939 22993 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:01.852530 22843 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:01.856880 22993 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:01.858081 23027 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11.
I20260709 13:10:01.859256 22993 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:01.860857 22993 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:01.862218 22993 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:01.863214 27873 raft_consensus.cc:2749] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:10:01.874410 23029 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:10:01.875312 23029 raft_consensus.cc:606] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:01.875757 23029 raft_consensus.cc:614] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:10:01.877990 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 62640.2us
I20260709 13:10:01.881912 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:01.863389 27977 raft_consensus.cc:2749] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:01.850025 27678 raft_consensus.cc:2749] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:01.882678 23616 raft_consensus.cc:2376] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 16. Re-sending same reply.
I20260709 13:10:01.891893 23023 raft_consensus.cc:1240] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:10:01.891750 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:01.854291 27811 raft_consensus.cc:2804] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:01.863556 27724 raft_consensus.cc:2749] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:10:01.889638 22879 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 13.
I20260709 13:10:01.844129 28004 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:01.869758 27709 raft_consensus.cc:2749] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:10:01.863591 27929 raft_consensus.cc:2749] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:01.910864 22873 raft_consensus.cc:2410] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 14 because replica has last-logged OpId of term: 12 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 13:10:01.895572 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:01.863476 27356 raft_consensus.cc:2749] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:01.863348 28005 raft_consensus.cc:2749] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:10:01.868229 27919 raft_consensus.cc:2804] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:10:01.897540 27811 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:01.891613 27815 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:10:01.936959 27815 raft_consensus.cc:697] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:01.937958 27815 consensus_queue.cc:237] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:01.940356 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:01.884083 23588 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:01.902992 23617 raft_consensus.cc:1240] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:01.888859 27911 consensus_queue.cc:1059] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:10:01.909977 23029 raft_consensus.cc:1240] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 12. Current term is 13. Ops: []
I20260709 13:10:01.913290 23023 raft_consensus.cc:1240] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 11. Current term is 13. Ops: []
I20260709 13:10:01.914915 23623 raft_consensus.cc:2393] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 15: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:01.886245 28059 raft_consensus.cc:2749] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:10:01.863294 28023 consensus_queue.cc:1048] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 13:10:01.855050 28047 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:01.948882 27979 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:01.954504 27979 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:01.955850 28047 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:01.900990 27996 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:10:01.894932 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 82951.9us
I20260709 13:10:01.933063 23616 raft_consensus.cc:1240] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 14. Current term is 15. Ops: []
I20260709 13:10:01.946616 23024 raft_consensus.cc:1240] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 14. Current term is 15. Ops: [11.5-14.7]
I20260709 13:10:01.908843 23619 raft_consensus.cc:1240] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:10:01.905172 28004 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:10:01.948276 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:01.950002 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 66072us
I20260709 13:10:01.916775 27919 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:01.988191 27979 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:01.989825 23623 raft_consensus.cc:1275] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:10:01.995487 23023 raft_consensus.cc:1240] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:10:01.961833 23588 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:01.947681 27811 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:01.997125 22875 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 13.
I20260709 13:10:01.915694 27977 consensus_queue.cc:1048] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 13:10:01.998102 22918 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:02.006477 22918 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:02.006507 23623 raft_consensus.cc:1275] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:10:01.999379 23588 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:01.965209 27768 raft_consensus.cc:2804] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:10:02.019299 23024 raft_consensus.cc:1275] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:10:01.995026 27919 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:10:02.012624 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 157584us
I20260709 13:10:02.011945 22918 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:02.039619 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:02.044015 22955 raft_consensus.cc:2393] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 15: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:02.041481 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:02.042307 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:02.040961 22844 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:02.009832 27821 raft_consensus.cc:2804] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:10:02.048270 22918 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:02.048489 22948 raft_consensus.cc:1240] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 12. Current term is 13. Ops: []
I20260709 13:10:02.049177 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:02.051751 22954 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
I20260709 13:10:02.011448 27979 raft_consensus.cc:2749] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:02.046797 22843 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:02.055862 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:02.057247 22946 raft_consensus.cc:1240] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:10:02.017823 27768 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:02.057286 22843 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:02.050211 28062 raft_consensus.cc:2804] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:02.021005 27955 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:02.017498 28059 raft_consensus.cc:2804] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:10:02.063922 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:02.065485 22872 raft_consensus.cc:1240] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:10:02.065979 22879 raft_consensus.cc:1240] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 10. Current term is 11. Ops: []
I20260709 13:10:02.046972 27625 consensus_queue.cc:1048] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:10:02.060544 22919 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:02.077240 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:02.078536 22919 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:02.070890 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:02.070749 28062 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:02.075033 22875 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:02.089597 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:02.073891 22918 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:02.090339 22876 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:02.091303 22918 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:02.055317 27839 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:10:02.092965 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:02.091207 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:02.098474 22843 leader_election.cc:400] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 14. Current term is 15.
I20260709 13:10:02.103487 23023 raft_consensus.cc:1275] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:10:02.108491 23623 raft_consensus.cc:1240] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEADER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 15. Current term is 16. Ops: []
I20260709 13:10:02.110739 22843 leader_election.cc:403] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:10:02.072197 27873 raft_consensus.cc:2804] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 13:10:02.072844 27768 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:02.077006 28047 raft_consensus.cc:2749] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:10:02.079139 28032 raft_consensus.cc:2749] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:02.097492 28062 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:02.097309 27972 raft_consensus.cc:2749] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:10:02.097126 28046 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:02.096334 28029 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:10:02.132159 27873 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:02.153721 27821 raft_consensus.cc:697] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:02.154037 27873 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:02.154814 27821 consensus_queue.cc:237] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:02.143827 22875 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 15.
W20260709 13:10:02.141103 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 50513.9us
I20260709 13:10:02.159647 22843 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:02.110011 27839 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:02.103137 27953 raft_consensus.cc:2804] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:10:02.166121 22764 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 2959101199a44a4583a4ee456ecf044d with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER failed with leader 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 0a407504ed1741bf89a276558adf4c0f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 12 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
W20260709 13:10:02.170166 22764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 120562us
W20260709 13:10:02.167186 28061 log.cc:927] Time spent T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 13:10:02.109738 28059 raft_consensus.cc:697] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 14 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
W20260709 13:10:02.172173 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:02.105847 27500 consensus_queue.cc:1059] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 13:10:02.105506 27960 consensus_queue.cc:1059] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:10:02.141301 27988 raft_consensus.cc:2749] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 14. Current term is 15.
I20260709 13:10:02.175058 27953 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:02.201447 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:02.201771 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:02.201496 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:02.202414 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:02.176515 27678 raft_consensus.cc:2804] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:10:02.177101 28032 raft_consensus.cc:2804] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 13:10:02.205673 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:02.205924 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:02.201189 27953 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:02.175288 27839 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:02.178130 28029 raft_consensus.cc:697] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:02.221778 23021 raft_consensus.cc:2410] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 15 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 13:10:02.222065 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:02.215238 23590 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:02.222937 23624 raft_consensus.cc:2393] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 15: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
W20260709 13:10:02.223603 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 129535us
I20260709 13:10:02.228787 28058 raft_consensus.cc:2804] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:02.229319 28058 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:02.229761 28058 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:02.177947 28046 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:02.234503 28046 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:02.224061 27950 consensus_queue.cc:1048] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 13:10:02.183069 27972 consensus_queue.cc:1059] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 13:10:02.212142 27678 raft_consensus.cc:697] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 13 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:02.227753 28032 raft_consensus.cc:697] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:02.245747 27678 consensus_queue.cc:237] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:02.248864 28059 consensus_queue.cc:237] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:02.247538 28032 consensus_queue.cc:237] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:10:02.255934 28076 consensus_queue.cc:1048] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:10:02.223356 27500 raft_consensus.cc:3055] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:10:02.258399 27500 raft_consensus.cc:740] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:02.259231 27500 consensus_queue.cc:260] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:02.260392 27500 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:10:02.271292 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 180088us
I20260709 13:10:02.257690 28076 consensus_queue.cc:1048] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.014s
I20260709 13:10:02.261525 27959 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:02.271063 27996 consensus_queue.cc:1059] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 13:10:02.272776 22918 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:02.288354 23020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 818) took 1522 ms. Trace:
W20260709 13:10:02.339629 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 66957.9us
I20260709 13:10:02.339500 23020 rpcz_store.cc:276] 0709 13:10:00.765472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:00.768751 (+  3279us) service_pool.cc:224] Handling call
0709 13:10:01.197686 (+428935us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:01.203536 (+  5850us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:02.288166 (+1084630us) spinlock_profiling.cc:243] Waited 428 ms on lock 0x7b5400249850. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:02.288298 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":985443072}
I20260709 13:10:02.340515 22918 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:02.292531 22947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2905) took 1146 ms. Trace:
I20260709 13:10:02.254971 27873 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:10:02.341578 22947 rpcz_store.cc:276] 0709 13:10:01.145804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:01.498188 (+352384us) service_pool.cc:224] Handling call
0709 13:10:01.527549 (+ 29361us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:10:01.528379 (+   830us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:02.292339 (+763960us) spinlock_profiling.cc:243] Waited 28.9 ms on lock 0x7b540035d750. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:02.292475 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":66461952}
I20260709 13:10:02.293794 22876 raft_consensus.cc:2468] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 12.
I20260709 13:10:02.335510 22880 raft_consensus.cc:1240] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:10:02.308252 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:02.307520 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:10:02.281646 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:02.309374 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:02.315367 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:02.337387 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:02.338488 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:02.348491 23023 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:02.349052 22877 raft_consensus.cc:2393] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 15: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:02.350340 28058 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:02.355144 28058 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:02.345683 28047 consensus_queue.cc:1059] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:10:02.342770 22918 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:02.361625 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:02.362002 27953 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:02.364132 27953 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:02.364089 28062 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:02.365530 27873 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:02.295732 27768 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:02.366453 28062 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:02.368085 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:02.369035 23020 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:02.357928 22993 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:02.374498 23616 raft_consensus.cc:1240] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 13. Current term is 14. Ops: []
I20260709 13:10:02.358798 22992 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:02.378551 22993 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:02.380280 22993 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:02.294535 27996 raft_consensus.cc:3055] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:10:02.341621 27646 raft_consensus.cc:2749] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:02.342911 27960 raft_consensus.cc:2749] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:02.396804 23029 raft_consensus.cc:1240] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 14. Current term is 15. Ops: []
W20260709 13:10:02.397496 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 54802.2us
I20260709 13:10:02.324174 27988 consensus_queue.cc:1059] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260709 13:10:02.398409 22918 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:02.355923 27811 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:02.399405 27960 raft_consensus.cc:2749] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:10:02.399976 22918 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:02.401940 27811 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:02.403631 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:02.403846 27807 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:02.408026 23024 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 15.
I20260709 13:10:02.389936 27912 raft_consensus.cc:2804] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:10:02.413869 22918 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:02.415321 22918 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:02.416591 22918 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:02.420166 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:02.421289 23588 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:02.381475 27943 raft_consensus.cc:2749] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:10:02.389185 28003 raft_consensus.cc:2749] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:10:02.278972 27867 consensus_queue.cc:1059] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.108s
W20260709 13:10:02.431583 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 109518us
I20260709 13:10:02.431782 22918 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:02.432544 22843 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:02.433034 22918 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:02.381621 27724 raft_consensus.cc:2749] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:02.391428 28078 raft_consensus.cc:2764] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader election decision vote started in defunct term 14: won
I20260709 13:10:02.392030 27996 raft_consensus.cc:740] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:02.436329 23588 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:02.435056 22951 raft_consensus.cc:1275] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:10:02.439062 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:02.386694 22993 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:02.444569 22880 raft_consensus.cc:1275] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:10:02.420356 27646 raft_consensus.cc:2749] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:02.423221 28032 raft_consensus.cc:2749] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:02.420264 27960 raft_consensus.cc:2749] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:02.443562 22843 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:02.462682 27988 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:02.464059 22843 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:02.434561 27953 raft_consensus.cc:2804] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:10:02.458714 22918 leader_election.cc:400] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 12. Current term is 13.
I20260709 13:10:02.455861 27625 raft_consensus.cc:2749] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:02.452080 27873 raft_consensus.cc:2749] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:02.273438 28029 consensus_queue.cc:237] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:02.465274 22918 leader_election.cc:403] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:10:02.433961 27912 raft_consensus.cc:697] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:02.439705 23023 raft_consensus.cc:2468] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 12.
I20260709 13:10:02.444482 28084 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:02.443689 23020 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 17.
W20260709 13:10:02.457511 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 70869.4us
I20260709 13:10:02.459972 27911 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:02.475003 22993 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:10:02.460793 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 231141us
I20260709 13:10:02.444314 27500 consensus_queue.cc:1048] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 13:10:02.425823 27915 consensus_queue.cc:1059] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:10:02.475417 22947 raft_consensus.cc:1275] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:10:02.458247 27950 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:02.466651 22843 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:02.477865 23024 raft_consensus.cc:1275] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 13:10:02.464632 28054 raft_consensus.cc:2804] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:10:02.455052 28004 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:02.486229 22843 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:02.467507 27988 raft_consensus.cc:2804] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:10:02.526599 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:02.464984 27919 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:02.534595 22843 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:02.488855 27678 raft_consensus.cc:2749] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:02.543038 23028 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 2173) took 1394 ms. Trace:
W20260709 13:10:02.551262 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:02.488937 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:02.562000 23022 raft_consensus.cc:2376] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 12. Re-sending same reply.
I20260709 13:10:02.472602 27958 consensus_queue.cc:1059] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 13:10:02.472506 28089 raft_consensus.cc:2749] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:02.553107 23028 rpcz_store.cc:276] 0709 13:10:01.148684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:01.278160 (+129476us) service_pool.cc:224] Handling call
0709 13:10:01.285479 (+  7319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:01.463615 (+178136us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:02.542785 (+1079170us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b540018a954. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:02.542980 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":407755008}
I20260709 13:10:02.479108 27646 raft_consensus.cc:2804] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:10:02.534713 27873 raft_consensus.cc:2749] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:02.567122 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:02.535086 27915 raft_consensus.cc:2749] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:10:02.491684 27768 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:02.476603 27839 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:02.541733 28080 log.cc:927] Time spent T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:10:02.542189 27929 raft_consensus.cc:993] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75: : Instructing follower aed3f089ec8b49af9c8cf5cdee76e8e7 to start an election
W20260709 13:10:02.599041 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 124111us
I20260709 13:10:02.608042 22993 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:02.593910 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:10:02.597569 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 128181us
I20260709 13:10:02.526090 27950 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:02.612736 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:02.543782 28086 consensus_queue.cc:1048] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.084s
I20260709 13:10:02.489542 28032 consensus_queue.cc:1059] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.112s
I20260709 13:10:02.566795 27983 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:02.625367 23029 raft_consensus.cc:1240] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:10:02.473044 27996 consensus_queue.cc:260] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:02.477878 28084 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:02.595508 28079 raft_consensus.cc:2749] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 12. Current term is 13.
I20260709 13:10:02.602051 27646 raft_consensus.cc:697] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
W20260709 13:10:02.618929 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 84405.6us
I20260709 13:10:02.613243 27915 raft_consensus.cc:1081] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Signalling peer aed3f089ec8b49af9c8cf5cdee76e8e7 to start an election
I20260709 13:10:02.611570 27678 raft_consensus.cc:2804] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:10:02.613909 28023 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:02.637167 28004 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:02.638798 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:02.639650 27867 raft_consensus.cc:3055] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:10:02.640074 27867 raft_consensus.cc:740] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
W20260709 13:10:02.641273 28048 log.cc:927] Time spent T 00000000000000000000000000000000 P 8d175b4ae72f4fc48971c2f31668971c: Append to log took a long time: real 0.102s	user 0.000s	sys 0.002s
I20260709 13:10:02.641629 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:02.629796 23024 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
W20260709 13:10:02.600533 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 159570us
I20260709 13:10:02.633282 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:02.635638 22843 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:02.560549 28054 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:02.650729 22994 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:02.655777 22875 raft_consensus.cc:1275] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:10:02.656190 22875 pending_rounds.cc:85] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 7
I20260709 13:10:02.657207 22875 pending_rounds.cc:107] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:10:02.657518 22875 raft_consensus.cc:2889] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:02.703526 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:02.704640 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:02.705478 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:02.776086 22871 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 16.
I20260709 13:10:02.607709 27912 consensus_queue.cc:237] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:02.706717 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 53647us
I20260709 13:10:02.708055 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:02.647348 27996 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:02.646849 27678 raft_consensus.cc:697] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 15 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:02.648767 28054 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:02.656755 28029 raft_consensus.cc:3055] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:10:02.742568 28091 consensus_queue.cc:1048] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 13:10:02.631032 27983 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:02.683321 27988 raft_consensus.cc:697] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 14 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:02.655642 27911 raft_consensus.cc:2749] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:02.658373 27873 raft_consensus.cc:2804] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:10:02.753666 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:02.756001 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 108276us
I20260709 13:10:02.646936 28085 consensus_queue.cc:1059] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.099s
W20260709 13:10:02.762899 28088 log.cc:927] Time spent T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.160s	user 0.000s	sys 0.001s
I20260709 13:10:02.809104 28074 raft_consensus.cc:3055] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:10:02.809566 28074 raft_consensus.cc:740] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:02.809717 27839 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:02.810477 28074 consensus_queue.cc:260] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:02.763016 27956 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:02.766145 22946 raft_consensus.cc:1275] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:10:02.770398 23590 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:02.703941 27960 consensus_queue.cc:1059] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 13:10:02.757671 28004 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:02.789016 28029 raft_consensus.cc:740] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:02.820848 28004 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:02.781358 27678 consensus_queue.cc:237] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:02.830499 28004 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:02.831187 22877 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 15.
I20260709 13:10:02.834657 27356 consensus_queue.cc:1048] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:10:02.835166 28090 consensus_queue.cc:1048] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:10:02.844758 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:02.811762 28074 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:10:02.793634 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:02.799046 27953 raft_consensus.cc:697] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
W20260709 13:10:02.795126 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:02.742538 27867 consensus_queue.cc:260] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:02.806253 27988 consensus_queue.cc:237] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:02.815678 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 45365.7us
I20260709 13:10:02.773499 22994 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:02.818576 27768 raft_consensus.cc:2804] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:10:02.820395 27983 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:02.821345 27960 consensus_queue.cc:1243] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer 5b346767a89745fa82928a5dac9cc0b2 log is divergent from this leader: its last log entry 16.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 13:10:02.816649 28098 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:02.812178 27873 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:02.658710 23029 raft_consensus.cc:1275] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 13:10:02.859258 27955 consensus_meta.cc:287] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.780s	user 0.009s	sys 0.002s
I20260709 13:10:02.774854 23588 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:02.855538 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:02.856420 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:02.874109 22948 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:02.874917 22950 raft_consensus.cc:1240] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 13:10:02.859583 28086 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:02.858211 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:02.855588 22920 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:10:02.862269 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 88894.3us
I20260709 13:10:02.864240 23590 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:02.766973 27646 consensus_queue.cc:237] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:02.857546 22951 raft_consensus.cc:1275] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:10:02.895555 28084 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:02.896981 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:02.897579 23029 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:02.868935 28090 raft_consensus.cc:2804] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:10:02.901206 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 45703.6us
I20260709 13:10:02.873978 28098 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:02.904984 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:02.906879 28098 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:02.875644 27873 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:02.910208 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:02.882961 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 108230us
I20260709 13:10:02.888497 22948 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 17.
I20260709 13:10:02.915810 23588 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:02.889053 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:02.889662 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:02.916778 23024 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:02.919989 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:02.921672 23588 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:02.926011 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:02.888476 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:02.927820 22950 raft_consensus.cc:1240] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 13:10:02.928344 23623 raft_consensus.cc:1240] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16. Ops: [15.14-15.14]
I20260709 13:10:02.849174 28054 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:02.935001 22876 raft_consensus.cc:1275] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:10:02.935981 23028 raft_consensus.cc:1275] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:10:02.937001 22948 raft_consensus.cc:1240] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 12. Current term is 13. Ops: []
I20260709 13:10:02.937475 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:02.938175 23028 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
I20260709 13:10:02.939282 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:02.876288 27768 raft_consensus.cc:697] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
W20260709 13:10:02.879372 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 57587.9us
I20260709 13:10:02.879626 27955 raft_consensus.cc:2749] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:02.942430 23029 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 13.
I20260709 13:10:02.950619 23024 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 10.
I20260709 13:10:02.951678 23618 raft_consensus.cc:1240] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 14 LEADER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:10:02.985575 23024 raft_consensus.cc:1240] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:10:02.951787 23025 raft_consensus.cc:1275] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:10:02.952723 22953 raft_consensus.cc:1275] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
W20260709 13:10:02.953346 28070 log.cc:927] Time spent T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.726s	user 0.001s	sys 0.004s
I20260709 13:10:02.952971 27953 consensus_queue.cc:237] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:02.918722 28086 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:02.990326 28086 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:02.954298 28100 consensus_queue.cc:1059] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:10:02.956369 28084 consensus_queue.cc:1048] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:10:02.959892 27815 raft_consensus.cc:2749] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:10:02.992627 27815 raft_consensus.cc:2749] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:02.960954 22843 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:02.942149 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:02.996068 23028 raft_consensus.cc:2468] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
I20260709 13:10:02.997342 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:02.913652 27821 raft_consensus.cc:2749] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:10:02.917935 28058 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:02.918533 28090 raft_consensus.cc:697] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:02.919513 28089 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:03.004577 22951 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15.
I20260709 13:10:03.002590 23620 raft_consensus.cc:1240] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
I20260709 13:10:03.003187 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:03.130201 28100 consensus_queue.cc:1059] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 13:10:03.140226 23028 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15.
I20260709 13:10:03.140439 28100 consensus_queue.cc:1243] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer f3c799b1249e40119d2a5864231c6c75 log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:10:03.007174 28089 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:03.141297 28089 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:03.142793 22877 raft_consensus.cc:3055] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:10:03.143173 22877 raft_consensus.cc:740] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:03.143909 22877 consensus_queue.cc:260] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:03.034969 28084 consensus_queue.cc:1059] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:10:03.145133 28084 raft_consensus.cc:3055] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:10:03.145486 28084 raft_consensus.cc:740] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:03.146407 28084 consensus_queue.cc:260] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:03.147535 28084 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:03.147693 22877 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:03.148444 23616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 924) took 1100 ms. Trace:
I20260709 13:10:03.002096 28058 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:03.149117 28058 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:03.148938 23616 rpcz_store.cc:276] 0709 13:10:02.048353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:02.048564 (+   211us) service_pool.cc:224] Handling call
0709 13:10:02.048892 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:02.377777 (+328885us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:03.148130 (+770353us) spinlock_profiling.cc:243] Waited 319 ms on lock 0x7b54000c08d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:03.148340 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":734188416}
I20260709 13:10:03.061827 28106 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:03.093146 23022 raft_consensus.cc:1275] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 13:10:03.093878 22918 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:03.152940 28106 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:03.153558 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 59802.8us
I20260709 13:10:03.065871 28094 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:03.155066 28106 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:03.154877 28094 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:03.157297 28094 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:03.003185 28098 raft_consensus.cc:2804] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:10:03.157991 28098 raft_consensus.cc:697] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 17 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:03.158854 28098 consensus_queue.cc:237] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:03.111517 27873 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:03.162308 27873 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:03.163851 27764 raft_consensus.cc:2804] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:10:03.164382 27764 raft_consensus.cc:697] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:03.165223 27764 consensus_queue.cc:237] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:03.166307 27500 consensus_queue.cc:1048] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:10:03.168900 28005 raft_consensus.cc:3055] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:10:03.169284 28005 raft_consensus.cc:740] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:03.170018 28005 consensus_queue.cc:260] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:03.171000 28005 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:03.173949 22918 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:03.174656 27955 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:10:03.175598 22918 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:03.176169 27955 consensus_queue.cc:1048] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:10:03.094856 28077 raft_consensus.cc:3055] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:10:03.178063 28077 raft_consensus.cc:740] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:03.178208 28078 consensus_queue.cc:1048] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:10:03.178952 28077 consensus_queue.cc:260] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:03.179975 28077 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:03.101012 27988 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:03.181532 27988 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:03.184689 27988 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:03.109999 28062 raft_consensus.cc:3055] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:10:03.185202 28062 raft_consensus.cc:740] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
W20260709 13:10:03.089180 28068 log.cc:927] Time spent T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.857s	user 0.002s	sys 0.002s
I20260709 13:10:03.187542 22874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2748) took 1398 ms. Trace:
I20260709 13:10:03.128644 22843 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:03.133937 22872 raft_consensus.cc:1275] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:10:03.189417 22872 pending_rounds.cc:85] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 7
I20260709 13:10:03.189740 22872 pending_rounds.cc:107] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:10:03.190016 22872 raft_consensus.cc:2889] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:03.136399 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:03.190781 22878 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 15.
I20260709 13:10:03.138967 22880 raft_consensus.cc:1275] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:10:03.139127 28054 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:03.194119 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 77157.1us
I20260709 13:10:03.065685 27839 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:03.073880 28090 consensus_queue.cc:237] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:03.177670 28032 raft_consensus.cc:2749] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:03.068755 28086 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:03.187911 22874 rpcz_store.cc:276] 0709 13:10:01.789648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:01.791658 (+  2010us) service_pool.cc:224] Handling call
0709 13:10:01.792063 (+   405us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:10:01.889862 (+ 97799us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:01.889980 (+   118us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:10:01.896278 (+  6298us) log.cc:844] Serialized 52 byte log entry
0709 13:10:02.098131 (+201853us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:10:02.137170 (+ 39039us) log.cc:844] Serialized 14 byte log entry
0709 13:10:02.139701 (+  2531us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:02.139811 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:03.088490 (+948679us) spinlock_profiling.cc:243] Waited 25.4 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:10:03.187306 (+ 98816us) raft_consensus.cc:1685] finished
0709 13:10:03.187381 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:03.187494 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2196,"prepare.queue_time_us":39121,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":21125,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":15507,"spinlock_wait_cycles":58482432,"thread_start_us":232149,"threads_started":2,"wal-append.queue_time_us":298605}
I20260709 13:10:03.195757 22918 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:03.005828 27960 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:03.198362 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:03.198359 23026 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f603724630e44ef78014e7719f096cb3"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:60044
W20260709 13:10:03.201247 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 279615us
I20260709 13:10:03.227530 23026 raft_consensus.cc:606] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:03.227981 23026 raft_consensus.cc:614] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:10:03.228109 23588 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:10:03.201174 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 72631.7us
I20260709 13:10:03.230273 22918 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:03.230492 22843 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:03.205965 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:03.154917 27950 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:03.219062 22877 raft_consensus.cc:1275] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 13:10:03.241825 22877 pending_rounds.cc:85] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 7
I20260709 13:10:03.241811 23588 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:03.242144 22877 pending_rounds.cc:107] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 13:10:03.243587 22877 raft_consensus.cc:2889] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:03.218340 22994 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:03.159857 28029 consensus_queue.cc:260] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:03.246387 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 305470us
I20260709 13:10:03.250778 22843 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:03.245064 22946 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:58320
I20260709 13:10:03.254010 22946 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:03.254448 22946 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:03.259943 27919 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:03.214002 27768 consensus_queue.cc:237] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:03.209753 27988 raft_consensus.cc:2749] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:10:03.265190 22843 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:03.271646 22843 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:03.266999 22918 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:03.191951 27678 consensus_queue.cc:1059] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 13:10:03.245674 28079 raft_consensus.cc:2749] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:10:03.280596 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:03.209568 28023 consensus_queue.cc:1048] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 13:10:03.277791 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:03.272460 22843 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:03.289315 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:03.281828 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:03.247897 28078 raft_consensus.cc:2804] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:10:03.247520 28074 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:10:03.301848 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:03.310804 23020 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 15.
I20260709 13:10:03.269140 27873 raft_consensus.cc:2749] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:10:03.215296 28058 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:03.268096 28003 raft_consensus.cc:2804] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:03.306344 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:03.299831 27975 consensus_queue.cc:1059] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.015s
I20260709 13:10:03.315979 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:03.317549 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:03.332343 28062 consensus_queue.cc:260] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:03.336889 28029 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:03.339337 23021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 846) took 1031 ms. Trace:
I20260709 13:10:03.214501 27839 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:03.213971 28089 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:03.441520 22948 raft_consensus.cc:1240] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:03.268246 27830 consensus_queue.cc:1048] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.142s
I20260709 13:10:03.432273 23021 rpcz_store.cc:276] 0709 13:10:02.307194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:02.362490 (+ 55296us) service_pool.cc:224] Handling call
0709 13:10:02.362765 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:02.362962 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:10:02.363190 (+   228us) log.cc:844] Serialized 14 byte log entry
0709 13:10:02.473351 (+110161us) log.cc:844] Serialized 14 byte log entry
0709 13:10:02.710659 (+237308us) spinlock_profiling.cc:243] Waited 3.23 ms on lock 0x7b18000a6cc0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cf83f3 00007f6132cfa7c1 00007f6132cfa779 00007f6132cfa709 00007f6132cfa6d1 00007f6132cf99cd 000000000038f354 000000000038f199 00007f6132cdb83a
0709 13:10:03.191179 (+480520us) spinlock_profiling.cc:243] Waited 1.8 ms on lock 0x7b18000a6cc0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cdae07 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf5455 00007f6135ddd060 00007f6135def716 00007f6135e00287 00007f6135e00208
0709 13:10:03.337444 (+146265us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:03.337643 (+   199us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:10:03.338011 (+   368us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:03.338386 (+   375us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:03.338954 (+   568us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8296,"prepare.queue_time_us":11347,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":71,"raft.queue_time_us":18965,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":40647,"spinlock_wait_cycles":11558016,"thread_start_us":245391,"threads_started":2,"wal-append.queue_time_us":173273}
W20260709 13:10:03.345103 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 54477.6us
I20260709 13:10:03.274241 28085 raft_consensus.cc:2804] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:03.247737 27955 consensus_queue.cc:1059] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 13:10:03.293463 27958 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:10:03.459801 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:03.461396 22878 raft_consensus.cc:1275] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:10:03.244567 28054 consensus_queue.cc:1059] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 13:10:03.286691 28047 raft_consensus.cc:2804] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader election won for term 10
I20260709 13:10:03.432114 22918 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:03.293447 27956 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:10:03.294565 28099 raft_consensus.cc:2804] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:10:03.246642 27960 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:03.249163 28086 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:03.517892 23020 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "5b346767a89745fa82928a5dac9cc0b2"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:10:03.518904 23020 raft_consensus.cc:606] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Received request to transfer leadership to TS 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:03.519402 23020 raft_consensus.cc:614] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:10:03.268354 27983 consensus_queue.cc:1048] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.147s
I20260709 13:10:03.349399 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:03.348007 27678 raft_consensus.cc:2804] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:03.454717 23588 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:03.457993 28085 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:03.457384 22947 raft_consensus.cc:1275] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:10:03.286289 28032 consensus_queue.cc:1048] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.149s
I20260709 13:10:03.409554 27988 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:03.286600 27943 consensus_queue.cc:1048] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.149s
I20260709 13:10:03.424927 28003 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:03.468006 22880 raft_consensus.cc:2393] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 14: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:03.473867 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:03.474818 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:03.297200 27979 consensus_queue.cc:1059] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.144s
I20260709 13:10:03.293143 28096 consensus_queue.cc:1048] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.149s
I20260709 13:10:03.435070 28078 raft_consensus.cc:697] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:03.481312 27958 raft_consensus.cc:697] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 17 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:03.286865 27807 consensus_queue.cc:1048] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.145s
I20260709 13:10:03.457973 27821 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
W20260709 13:10:03.567477 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 112933us
I20260709 13:10:03.565061 27867 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:03.475628 27912 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:03.646817 23588 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:03.481725 28099 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:03.590642 22876 raft_consensus.cc:1275] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 13:10:03.655593 22876 pending_rounds.cc:85] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 9
I20260709 13:10:03.655997 22876 pending_rounds.cc:107] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 13:10:03.656294 22876 raft_consensus.cc:2889] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:10:03.527839 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 95782.5us
I20260709 13:10:03.538525 27911 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:03.541395 28004 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 13 current term: 13 is higher.
I20260709 13:10:03.513238 27873 consensus_queue.cc:1048] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.152s
I20260709 13:10:03.538467 28085 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:03.589226 28074 raft_consensus.cc:697] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:03.649147 21756 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:10:03.632238 28072 raft_consensus.cc:2749] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:10:03.664216 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 334117us
W20260709 13:10:03.732367 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 85648us
I20260709 13:10:03.606081 28062 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:03.661360 22946 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:03.748781 23619 raft_consensus.cc:1240] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 11. Current term is 12. Ops: []
W20260709 13:10:03.753633 22846 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.21.63.62:33399, user_credentials={real_user=slave}} blocked reactor thread for 99898us
I20260709 13:10:03.739341 21756 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:10:03.532497 27678 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:03.540045 28077 raft_consensus.cc:2804] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:03.776768 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:03.655674 27953 raft_consensus.cc:3055] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 13:10:03.780874 22946 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:03.874707 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 236828us
I20260709 13:10:03.634378 28100 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:03.776813 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:03.778901 23624 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 7 LEARNER]: Advancing to term 9
I20260709 13:10:03.540913 27956 raft_consensus.cc:697] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:03.786096 23618 raft_consensus.cc:1240] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:03.716084 28003 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:03.788996 27953 raft_consensus.cc:740] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
W20260709 13:10:03.650604 27972 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.409s	user 0.006s	sys 0.028s
I20260709 13:10:03.776752 27678 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:03.663255 28099 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:03.732473 27955 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:03.785826 28077 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:03.649272 28023 consensus_queue.cc:1048] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.137s
I20260709 13:10:03.664291 27912 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:03.801892 28118 log.cc:927] Time spent T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.234s	user 0.000s	sys 0.000s
W20260709 13:10:03.802646 28117 log.cc:927] Time spent T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.362s	user 0.000s	sys 0.001s
W20260709 13:10:03.803647 28126 log.cc:927] Time spent T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
W20260709 13:10:03.804486 28113 log.cc:927] Time spent T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.329s	user 0.001s	sys 0.000s
W20260709 13:10:03.805264 28121 log.cc:927] Time spent T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.363s	user 0.001s	sys 0.000s
I20260709 13:10:03.653810 27821 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:03.805984 28132 log.cc:927] Time spent T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.279s	user 0.000s	sys 0.000s
I20260709 13:10:03.592976 27988 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:03.815964 28134 consensus_queue.cc:1059] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 13:10:03.819996 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:03.821048 27958 consensus_queue.cc:237] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:03.942021 28077 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:03.775609 27911 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:03.821722 27764 consensus_queue.cc:1059] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:10:03.769177 28004 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:03.823825 28086 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:03.824836 28078 consensus_queue.cc:237] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:03.826056 22905 heartbeater.cc:646] Failed to heartbeat to 127.21.63.62:33399 (2 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.21.63.62:33399 timed out after 15.000s (SENT)
W20260709 13:10:03.828274 28110 log.cc:927] Time spent T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.565s	user 0.000s	sys 0.001s
I20260709 13:10:03.828912 28074 consensus_queue.cc:237] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:03.840121 28058 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:04.007586 28106 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:04.031523 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:10:03.842597 28115 log.cc:927] Time spent T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.578s	user 0.001s	sys 0.000s
I20260709 13:10:03.843559 28089 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:03.845202 28005 consensus_meta.cc:287] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.674s	user 0.007s	sys 0.007s
I20260709 13:10:03.851482 27915 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:03.754650 27950 raft_consensus.cc:2749] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:10:03.775082 27709 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:03.859522 27929 raft_consensus.cc:3055] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:10:03.863641 22955 raft_consensus.cc:1240] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14. Ops: [11.9-13.10]
I20260709 13:10:03.865104 22954 raft_consensus.cc:1275] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:10:03.774827 23616 raft_consensus.cc:1240] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 14. Current term is 15. Ops: []
W20260709 13:10:03.905607 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:03.905776 22877 raft_consensus.cc:1240] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14. Ops: []
I20260709 13:10:03.904592 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:03.904455 28100 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:04.066696 28005 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:04.067247 28005 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:04.067610 28100 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:04.068200 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:04.069679 28005 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:04.072831 27709 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:04.075037 27709 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:03.928658 23624 raft_consensus.cc:1275] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 LEARNER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 13:10:03.889433 28084 consensus_meta.cc:287] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.741s	user 0.008s	sys 0.003s
W20260709 13:10:03.931442 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 473311us
I20260709 13:10:03.875941 27919 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:03.880995 27811 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:03.929452 27972 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.688s	user 0.006s	sys 0.028s
W20260709 13:10:03.993315 23025 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.008s	sys 0.014s
W20260709 13:10:03.998606 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 221570us
W20260709 13:10:04.001461 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 223785us
I20260709 13:10:04.113484 23588 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:03.644701 28098 consensus_queue.cc:1059] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.141s
W20260709 13:10:03.945631 28059 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.002s	sys 0.009s
I20260709 13:10:04.028296 28004 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:04.121840 28059 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.006s	sys 0.017s
I20260709 13:10:03.996376 28130 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:04.122824 28130 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:04.123922 28004 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:04.125075 28130 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:04.026942 23028 raft_consensus.cc:1275] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 13:10:04.126034 22843 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:03.841743 27960 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:04.035418 22873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 2234) took 1111 ms. Trace:
I20260709 13:10:04.128158 28098 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:04.127315 22873 rpcz_store.cc:276] 0709 13:10:02.924265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:02.924380 (+   115us) service_pool.cc:224] Handling call
0709 13:10:02.924648 (+   268us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:02.925000 (+   352us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:02.925027 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:02.925477 (+   450us) log.cc:844] Serialized 26 byte log entry
0709 13:10:03.211291 (+285814us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:10:03.211346 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:03.211431 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:04.034965 (+823534us) raft_consensus.cc:1685] finished
0709 13:10:04.035183 (+   218us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:04.035359 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1604,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":58177,"thread_start_us":285424,"threads_started":1,"wal-append.queue_time_us":498497}
I20260709 13:10:04.129379 22873 raft_consensus.cc:1240] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 13:10:03.942287 27955 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:04.132439 27955 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:04.038525 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:04.132751 27911 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:04.053047 23026 raft_consensus.cc:1275] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 13:10:04.133991 23026 pending_rounds.cc:85] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75: Aborting all ops after (but not including) 9
I20260709 13:10:04.134265 23026 pending_rounds.cc:107] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 13:10:04.134512 23026 raft_consensus.cc:2889] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:04.060868 22950 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 15.
I20260709 13:10:04.063988 22872 raft_consensus.cc:1275] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:10:04.065580 27929 raft_consensus.cc:740] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:04.083670 28058 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:04.136718 27929 consensus_queue.cc:260] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:04.137133 28058 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:04.137560 28098 consensus_queue.cc:1048] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:10:04.090732 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:04.139000 28098 raft_consensus.cc:3055] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:10:04.139333 28098 raft_consensus.cc:740] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:04.093386 22877 raft_consensus.cc:1240] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 11. Current term is 12. Ops: []
W20260709 13:10:04.098173 23025 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.749s	user 0.009s	sys 0.014s
I20260709 13:10:04.011755 27953 consensus_queue.cc:260] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:10:04.048099 28134 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:04.110659 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:03.946265 27956 consensus_queue.cc:237] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:04.112457 28142 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:04.137818 27929 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:04.107614 23029 raft_consensus.cc:1275] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:10:04.161199 22879 raft_consensus.cc:2410] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 16 because replica has last-logged OpId of term: 14 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 13:10:04.048647 28106 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:04.177757 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:04.120931 27830 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
W20260709 13:10:04.104983 28132 log.cc:927] Time spent T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:10:04.166987 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:04.191104 22878 raft_consensus.cc:2410] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
W20260709 13:10:04.170363 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 56965.8us
I20260709 13:10:04.173625 23022 raft_consensus.cc:1275] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:10:04.140079 28058 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:04.112923 28047 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.005s	sys 0.022s
I20260709 13:10:04.180322 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:04.182256 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:04.183235 22948 raft_consensus.cc:1275] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:10:04.087296 27698 consensus_queue.cc:1048] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.087s
I20260709 13:10:04.184566 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:04.200799 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:04.202066 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:04.202878 22955 raft_consensus.cc:1275] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:10:04.106527 27919 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:04.189345 22920 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:04.098078 28023 consensus_queue.cc:1059] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.082s
I20260709 13:10:04.131768 28003 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:04.152253 27955 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:04.198271 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:04.200997 22954 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
I20260709 13:10:04.215539 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:04.058970 27915 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:04.176020 28135 raft_consensus.cc:2749] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:10:04.145622 27811 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:04.207729 27958 consensus_queue.cc:1048] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:10:04.192333 27830 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:04.198904 28047 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.005s	sys 0.022s
I20260709 13:10:04.171670 27975 consensus_queue.cc:1048] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 13:10:04.214761 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:04.168754 27960 consensus_queue.cc:1048] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 13:10:04.140045 28098 consensus_queue.cc:260] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:04.222549 22873 raft_consensus.cc:1275] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:10:04.222855 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:04.223419 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:04.187929 28140 consensus_queue.cc:1048] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 13:10:04.246416 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:04.250890 23022 raft_consensus.cc:2393] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 14: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:04.176311 28134 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:04.269338 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:04.204610 27867 consensus_queue.cc:1059] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 13:10:04.180405 28142 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:04.243525 28151 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:04.249987 28047 raft_consensus.cc:697] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 10 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:04.243610 27709 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:04.287051 22954 raft_consensus.cc:1275] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:10:04.269145 28064 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:04.289516 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 72490us
I20260709 13:10:04.290880 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:04.222229 28076 consensus_queue.cc:1048] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.061s
I20260709 13:10:04.221582 27955 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:04.330657 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:04.331043 22992 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:04.280812 28090 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:04.303421 23620 raft_consensus.cc:1275] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:10:04.279881 27988 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:04.247196 28077 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:04.314682 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:04.316262 27915 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:04.316579 28134 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:04.317440 27919 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:04.329649 22844 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:04.344463 28134 raft_consensus.cc:2749] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:10:04.343713 22844 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:04.346071 28134 raft_consensus.cc:2749] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:04.347013 27919 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:04.347532 27919 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:04.347923 27955 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:04.349643 27919 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:04.352825 28160 log.cc:927] Time spent T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:10:04.353559 27830 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:04.331444 23618 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
I20260709 13:10:04.302970 28078 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:04.335639 22920 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:04.324631 28062 consensus_meta.cc:287] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.548s	user 0.008s	sys 0.002s
I20260709 13:10:04.338932 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:04.303197 28023 consensus_queue.cc:1048] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 13:10:04.363399 22843 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:04.332134 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:04.361410 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:04.364995 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:04.335888 28098 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:04.368822 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:04.369208 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:04.370260 22950 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
I20260709 13:10:04.371104 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:04.371294 22843 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:04.336766 28005 raft_consensus.cc:2749] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:04.337507 28140 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:04.376317 22876 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:10:04.378791 22873 raft_consensus.cc:2393] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 12: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:04.377597 28136 consensus_queue.cc:1048] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.158s
I20260709 13:10:04.332687 27709 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:04.385768 28152 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:04.386009 28146 log.cc:927] Time spent T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.220s	user 0.001s	sys 0.001s
I20260709 13:10:04.365358 28023 consensus_queue.cc:1243] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer aed3f089ec8b49af9c8cf5cdee76e8e7 log is divergent from this leader: its last log entry 14.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:10:04.386395 28152 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:04.378235 28130 raft_consensus.cc:2804] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:10:04.376274 28062 raft_consensus.cc:2804] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:04.370012 27988 consensus_queue.cc:1048] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 13:10:04.400861 23022 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15.
I20260709 13:10:04.339416 28090 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:04.323513 28151 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:04.331790 28064 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:04.396843 28029 consensus_meta.cc:287] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.955s	user 0.011s	sys 0.000s
I20260709 13:10:04.381278 28106 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:04.412766 28091 raft_consensus.cc:3055] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:10:04.414520 28062 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:04.408550 27919 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:04.418246 28148 log.cc:927] Time spent T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.194s	user 0.000s	sys 0.001s
I20260709 13:10:04.419330 28100 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:04.419884 28152 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:04.420188 28089 raft_consensus.cc:3055] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:10:04.464111 28089 raft_consensus.cc:740] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:04.465101 28089 consensus_queue.cc:260] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:04.466383 28089 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:04.430583 28003 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:04.454705 27764 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:04.469780 27764 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:04.472756 28164 raft_consensus.cc:2749] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:04.397804 28047 consensus_queue.cc:237] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:10:04.420363 28156 consensus_queue.cc:1048] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.056s
I20260709 13:10:04.476886 28156 consensus_queue.cc:1243] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer 0a407504ed1741bf89a276558adf4c0f log is divergent from this leader: its last log entry 14.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:10:04.424880 27811 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:04.425024 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:04.479887 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:04.480800 27764 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:04.374955 28150 consensus_queue.cc:1048] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.067s
I20260709 13:10:04.443552 22876 raft_consensus.cc:1275] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:10:04.457083 22951 raft_consensus.cc:1275] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 10. (term mismatch)
I20260709 13:10:04.481703 22951 pending_rounds.cc:85] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting all ops after (but not including) 9
I20260709 13:10:04.481949 22951 pending_rounds.cc:107] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting uncommitted NO_OP operation due to leader change: 15.10
I20260709 13:10:04.482185 22951 raft_consensus.cc:2889] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:04.384148 28140 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:04.482779 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:04.483137 28064 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:04.484174 28140 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:04.460435 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:04.407824 28130 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:04.484900 28130 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:04.481951 28151 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:04.479962 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:04.487285 22950 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:04.426079 22954 raft_consensus.cc:1240] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 15. Current term is 16. Ops: []
W20260709 13:10:04.491447 28099 consensus_meta.cc:287] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.550s	user 0.004s	sys 0.004s
I20260709 13:10:04.491904 28099 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:04.492982 28130 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:04.494453 27709 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:04.494983 28130 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:04.455116 28091 raft_consensus.cc:740] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:04.496186 28091 consensus_queue.cc:260] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:04.497332 28091 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
W20260709 13:10:04.502564 28155 log.cc:927] Time spent T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.221s	user 0.001s	sys 0.000s
I20260709 13:10:04.503222 22874 pending_rounds.cc:85] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 6
I20260709 13:10:04.503538 22874 pending_rounds.cc:107] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 14.7
I20260709 13:10:04.503777 22874 raft_consensus.cc:2889] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:04.454921 27830 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:04.507148 27830 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:04.509227 27830 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:04.459414 28106 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:04.510017 28106 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:04.511958 28106 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:04.512454 28100 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:04.514124 28116 log.cc:927] Time spent T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 1.050s	user 0.000s	sys 0.004s
I20260709 13:10:04.514861 28100 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:04.515651 22949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 2201) took 1661 ms. Trace:
I20260709 13:10:04.516999 27839 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:04.516010 22949 rpcz_store.cc:276] 0709 13:10:02.854612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:02.857323 (+  2711us) service_pool.cc:224] Handling call
0709 13:10:02.857602 (+   279us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:02.869929 (+ 12327us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:02.869973 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:02.876811 (+  6838us) log.cc:844] Serialized 26 byte log entry
0709 13:10:03.005723 (+128912us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:10:03.005771 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:03.005852 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:04.513562 (+1507710us) spinlock_profiling.cc:243] Waited 4.95 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:10:04.515397 (+  1835us) raft_consensus.cc:1685] finished
0709 13:10:04.515418 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:04.515593 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1364,"raft.queue_time_us":4156,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":8161,"spinlock_wait_cycles":11381120,"thread_start_us":92192,"threads_started":1,"wal-append.queue_time_us":385444}
I20260709 13:10:04.519311 27959 raft_consensus.cc:3055] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:10:04.519529 23588 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:04.519761 27959 raft_consensus.cc:740] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:04.520840 27959 consensus_queue.cc:260] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:04.521459 28143 raft_consensus.cc:2749] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:04.462767 27919 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:04.522998 28003 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:04.523334 27919 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:04.523589 28003 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:04.458036 28062 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:04.524596 27807 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:04.537781 27807 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:04.540293 28141 raft_consensus.cc:3055] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:10:04.540558 27807 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:04.540818 28141 raft_consensus.cc:740] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:04.542214 28141 consensus_queue.cc:260] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:04.547670 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:04.549515 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:04.550000 28090 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:04.550149 23021 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 16.
W20260709 13:10:04.550748 28167 log.cc:927] Time spent T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 13:10:04.551035 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:04.552089 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:04.552830 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:04.553884 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:04.554407 23028 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 14.
I20260709 13:10:04.554751 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:04.555169 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:04.555311 23024 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 17.
I20260709 13:10:04.555846 23028 raft_consensus.cc:2393] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 11: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:04.525512 28084 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:04.557516 28084 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:04.559188 28077 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:04.526605 28003 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:04.527916 28142 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:04.525388 23588 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:04.563928 28124 raft_consensus.cc:2749] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:10:04.552563 23021 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 15.
I20260709 13:10:04.566787 28124 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:04.561592 28158 log.cc:927] Time spent T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 13:10:04.567301 28124 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:04.567616 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:04.568284 23021 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 15.
I20260709 13:10:04.569569 28084 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:04.569586 27807 consensus_queue.cc:1048] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:10:04.570725 23023 raft_consensus.cc:1240] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 13:10:04.572958 22993 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:04.574321 22950 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 16.
W20260709 13:10:04.576189 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 80808.3us
I20260709 13:10:04.569710 22844 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
W20260709 13:10:04.583000 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:04.583474 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:04.583983 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:04.584221 22879 raft_consensus.cc:1240] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 17 LEADER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:04.585170 22992 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:04.586244 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:04.589331 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:04.562237 23617 raft_consensus.cc:3055] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:10:04.589977 23617 raft_consensus.cc:740] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:04.559799 28077 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:04.598029 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:04.599568 23028 raft_consensus.cc:1275] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:10:04.600358 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:04.601199 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:10:04.611305 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:04.605283 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:04.612844 22919 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:04.614379 22918 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:04.615445 22918 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:04.616291 22918 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:04.569489 27912 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:10:04.581384 28085 consensus_meta.cc:287] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.809s	user 0.006s	sys 0.007s
I20260709 13:10:04.621829 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:04.622925 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:04.621011 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:04.530185 27988 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:04.581215 28142 raft_consensus.cc:2749] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:10:04.636299 23023 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 15.
I20260709 13:10:04.640331 22918 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:04.626958 23617 consensus_queue.cc:260] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:04.642050 23617 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:04.602646 28084 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:10:04.602943 28003 raft_consensus.cc:2804] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:10:04.641207 22918 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:04.648200 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:04.656199 22950 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 14.
I20260709 13:10:04.601971 27958 consensus_queue.cc:1059] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 13:10:04.628872 27764 raft_consensus.cc:2749] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:10:04.610672 28062 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:04.666821 22873 raft_consensus.cc:1240] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:10:04.668085 28062 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:04.637590 27972 raft_consensus.cc:2804] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:04.629307 28124 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:04.622385 28077 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:04.637418 28136 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:04.694530 23588 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:04.652382 28140 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:04.712342 22843 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:04.662322 27678 consensus_meta.cc:287] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.722s	user 0.011s	sys 0.000s
I20260709 13:10:04.655300 28029 raft_consensus.cc:2804] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:04.650295 28084 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
W20260709 13:10:04.762357 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 90341.6us
W20260709 13:10:04.762439 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 68014.9us
I20260709 13:10:04.764804 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:04.771953 23617 raft_consensus.cc:1275] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 13:10:04.630076 28085 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:04.709208 28156 raft_consensus.cc:2749] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:10:04.772213 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 59959.4us
I20260709 13:10:04.781454 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:04.782420 22843 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:04.782891 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:04.783447 23021 raft_consensus.cc:2468] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
I20260709 13:10:04.785171 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:04.772495 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 173379us
I20260709 13:10:04.643597 28163 consensus_queue.cc:1048] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.133s
W20260709 13:10:04.775838 23588 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:04.776959 23617 pending_rounds.cc:85] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2: Aborting all ops after (but not including) 12
I20260709 13:10:04.708786 27972 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:04.763370 28003 raft_consensus.cc:697] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 16 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:04.771377 28124 raft_consensus.cc:2804] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:04.644115 27988 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:04.785216 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:04.785426 22871 raft_consensus.cc:1240] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 10 LEADER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 9. Current term is 10. Ops: []
I20260709 13:10:04.796133 23617 pending_rounds.cc:107] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 13:10:04.801884 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:04.712148 27500 consensus_queue.cc:1059] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.073s
I20260709 13:10:04.802564 22843 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:04.779611 28084 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:04.805862 23020 raft_consensus.cc:2393] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 15: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:04.773185 28127 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:04.806008 22875 raft_consensus.cc:1240] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:10:04.805454 23617 raft_consensus.cc:2889] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:04.804227 28124 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:04.777829 27983 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:04.771842 28029 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
W20260709 13:10:04.816286 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:04.774492 27678 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:04.795943 27977 raft_consensus.cc:2749] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:10:04.793277 28130 raft_consensus.cc:2749] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:04.801821 27972 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:10:04.871064 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 62376.3us
W20260709 13:10:04.874667 27956 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.036s	sys 0.054s
W20260709 13:10:04.904245 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 110157us
I20260709 13:10:04.804446 28047 consensus_queue.cc:1048] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.077s
I20260709 13:10:04.904415 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:04.769532 28136 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:04.902190 22843 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:04.908540 22877 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11.
I20260709 13:10:04.873016 27975 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:04.779017 23027 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 16.
I20260709 13:10:04.881326 28124 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:04.887827 28029 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:10:04.904302 27956 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.037s	sys 0.054s
I20260709 13:10:04.927812 28144 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:04.874571 28127 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:04.933597 23618 raft_consensus.cc:1240] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16. Ops: [15.14-15.14]
I20260709 13:10:04.930908 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:04.936972 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:10:04.933437 28176 log.cc:927] Time spent T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
W20260709 13:10:04.934587 28174 log.cc:927] Time spent T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
W20260709 13:10:04.933898 28172 log.cc:927] Time spent T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:10:04.979700 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:04.934245 28077 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:04.981343 28147 raft_consensus.cc:2804] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:10:04.895550 27983 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:04.983320 23028 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:04.924736 28099 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:04.932165 27988 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:04.944695 28051 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:04.985862 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:04.992280 23617 raft_consensus.cc:2410] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 16 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:10:04.986018 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:04.986536 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:04.988116 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:04.994038 23624 raft_consensus.cc:2393] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 15: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:04.934706 28173 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:04.996263 28127 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:04.932940 28120 consensus_queue.cc:1059] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.083s
I20260709 13:10:04.999992 27983 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:05.002195 28029 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:05.003602 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:05.004812 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:05.005695 23625 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 14.
I20260709 13:10:04.925086 27975 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:04.990700 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:04.924813 28003 consensus_queue.cc:237] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:04.992997 22878 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 15.
I20260709 13:10:05.015151 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:05.027477 23028 raft_consensus.cc:2468] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
I20260709 13:10:04.988821 28147 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:05.029603 22843 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:05.029749 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:05.026887 28099 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:05.035598 28099 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:05.015661 28173 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:05.037597 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:10:05.039165 28180 log.cc:927] Time spent T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 13:10:05.018739 28074 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:05.014683 27983 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
W20260709 13:10:05.044399 28184 log.cc:927] Time spent T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 13:10:05.037182 28064 raft_consensus.cc:2804] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election won for term 12
W20260709 13:10:05.042128 28183 log.cc:927] Time spent T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 13:10:05.034662 28147 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:05.051461 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:05.038538 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:05.060808 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:05.060171 22878 raft_consensus.cc:2393] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 16: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:05.040740 28187 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:05.062817 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: true dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:05.047971 28085 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:04.984721 28144 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:05.075968 23622 raft_consensus.cc:2393] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 16: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:05.058755 28064 raft_consensus.cc:697] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 12 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:05.083169 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:05.058485 28074 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:05.052472 27983 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:05.089682 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:05.091449 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:05.093658 28166 log.cc:927] Time spent T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.767s	user 0.003s	sys 0.000s
W20260709 13:10:05.094746 28178 log.cc:927] Time spent T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
I20260709 13:10:05.099999 23023 raft_consensus.cc:1240] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:10:05.099511 28029 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:05.105356 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:05.081631 28187 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:05.108606 28064 consensus_queue.cc:237] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:05.119614 28173 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:05.120021 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:05.121382 28074 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:05.121979 28188 log.cc:927] Time spent T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 13:10:05.123587 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:05.123710 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:05.110972 27953 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 9 FOLLOWER]: Advancing to term 10
I20260709 13:10:05.124265 22876 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 11.
I20260709 13:10:05.125303 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:05.127213 28136 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:05.134202 22843 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:05.145685 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:05.165347 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:05.166038 23625 raft_consensus.cc:1240] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 14. Current term is 15. Ops: []
I20260709 13:10:05.166916 28187 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:05.169245 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:05.170210 23618 raft_consensus.cc:2393] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 15: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:05.158223 22993 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:05.179672 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:05.180315 28098 consensus_meta.cc:287] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.803s	user 0.009s	sys 0.001s
I20260709 13:10:05.180505 23022 raft_consensus.cc:2393] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 11: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
W20260709 13:10:05.180653 28058 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 17 current term: 17 is higher.
I20260709 13:10:05.156534 28179 raft_consensus.cc:2804] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:05.181635 28179 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:05.182077 28179 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:10:05.191735 28089 consensus_meta.cc:287] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.725s	user 0.012s	sys 0.000s
I20260709 13:10:05.193522 27983 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:05.193451 28197 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:05.233927 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:05.233807 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 60530.2us
I20260709 13:10:05.234723 23024 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 11.
I20260709 13:10:05.235535 22993 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:05.237413 28154 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:05.237836 28154 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:05.238161 28154 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:05.238248 23590 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:05.240856 22993 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:05.242998 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:05.243633 23025 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 10.
I20260709 13:10:05.124265 28051 raft_consensus.cc:2749] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:05.258174 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:05.280009 22993 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:05.277824 22843 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:05.256979 28197 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:10:05.221568 28127 raft_consensus.cc:2749] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:10:05.196949 27975 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:05.287335 22993 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:05.287436 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:05.290570 28131 raft_consensus.cc:3055] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:10:05.291019 28131 raft_consensus.cc:740] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:05.288657 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:05.292379 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:05.293156 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:05.289443 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:05.293154 23618 raft_consensus.cc:2393] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 12: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:05.293983 23028 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
I20260709 13:10:05.295807 28200 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:10:05.296255 28078 consensus_meta.cc:287] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.931s	user 0.008s	sys 0.003s
I20260709 13:10:05.297600 22876 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 15.
I20260709 13:10:05.297765 22843 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:05.283387 27919 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:05.278581 28058 raft_consensus.cc:2749] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:10:05.284593 27983 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:05.301401 28084 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:05.303324 22878 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 14.
I20260709 13:10:05.303414 28124 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:05.303614 27972 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:05.288580 28090 raft_consensus.cc:2749] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:05.305107 22948 raft_consensus.cc:1240] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:05.306033 27972 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:05.301451 27919 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:05.304562 23619 raft_consensus.cc:1240] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:10:05.296373 28200 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:05.304273 27983 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:05.283529 27698 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:05.288475 28136 raft_consensus.cc:2804] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:10:05.296685 28078 raft_consensus.cc:2749] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:05.294119 27929 raft_consensus.cc:2749] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:05.309551 27975 raft_consensus.cc:2749] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:10:05.308180 27919 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:05.309873 28051 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:10:05.319689 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:05.315389 28136 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:05.324923 28136 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:10:05.325875 28124 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:05.320495 28051 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:05.324949 28144 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:05.330610 27983 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:05.337867 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:05.339591 23618 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11.
I20260709 13:10:05.341043 22993 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:05.326494 28051 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:05.349031 27975 raft_consensus.cc:2804] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:10:05.349423 27975 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:05.349799 27975 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:10:05.328469 28124 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:05.326706 28131 consensus_queue.cc:260] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:05.315095 27698 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:05.354544 28179 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:05.364082 28200 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:05.372397 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:05.374557 27959 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:05.358330 28124 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:05.381343 28202 log.cc:927] Time spent T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:10:05.381271 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:05.382521 23023 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 14.
I20260709 13:10:05.378968 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:05.383832 22878 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 15.
I20260709 13:10:05.385604 27975 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:05.380797 28147 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:05.379732 28131 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:05.361796 28154 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:05.379209 28124 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:05.400003 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:05.398408 27975 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:05.396255 28046 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:05.405462 23028 raft_consensus.cc:2368] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 14. Current term is 15.
I20260709 13:10:05.407584 27950 raft_consensus.cc:3055] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:10:05.408028 27950 raft_consensus.cc:740] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:05.412314 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:05.410292 27698 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:05.421352 28179 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:05.423842 28186 log.cc:927] Time spent T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.377s	user 0.000s	sys 0.002s
I20260709 13:10:05.415930 28147 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:05.409264 28046 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:05.425287 23618 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11.
I20260709 13:10:05.408836 27950 consensus_queue.cc:260] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:05.430044 22993 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:05.434175 28046 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:05.434669 23616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60044 (request call id 927) took 1688 ms. Trace:
I20260709 13:10:05.435045 23616 rpcz_store.cc:276] 0709 13:10:03.746771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:03.746912 (+   141us) service_pool.cc:224] Handling call
0709 13:10:03.747204 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:04.058615 (+311411us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:05.434440 (+1375825us) spinlock_profiling.cc:243] Waited 27.4 ms on lock 0x7b540045bfd4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:05.434614 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":63052544}
I20260709 13:10:05.435515 28154 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:05.437876 23027 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33420 (request call id 2211) took 1411 ms (client timeout 1631 ms). Trace:
W20260709 13:10:05.438254 23027 rpcz_store.cc:269] 0709 13:10:04.026249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:04.031235 (+  4986us) service_pool.cc:224] Handling call
0709 13:10:05.437731 (+1406496us) spinlock_profiling.cc:243] Waited 631 ms on lock 0x7b54002da5d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:05.437824 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1451122944}
I20260709 13:10:05.442667 22871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 2240) took 1984 ms. Trace:
I20260709 13:10:05.443023 22871 rpcz_store.cc:276] 0709 13:10:03.457827 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:03.474333 (+ 16506us) service_pool.cc:224] Handling call
0709 13:10:03.526361 (+ 52028us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:04.805075 (+1278714us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:05.442462 (+637387us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54002d6254. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:05.442615 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2894863232}
I20260709 13:10:05.444012 28023 raft_consensus.cc:2804] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:10:05.444603 28023 raft_consensus.cc:697] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:05.445569 28023 consensus_queue.cc:237] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:05.447924 28164 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:05.448427 28164 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:05.449697 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 808463us
I20260709 13:10:05.450328 22918 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:05.453951 23588 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:05.459692 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:05.460464 23616 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:10:05.466277 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:05.449221 27821 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:05.475050 22871 raft_consensus.cc:1240] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14. Ops: []
I20260709 13:10:05.473270 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:05.476008 23027 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 15.
I20260709 13:10:05.478814 23588 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:05.487622 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:05.488634 22871 raft_consensus.cc:1240] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:10:05.489677 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:05.488137 23588 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:05.490919 28164 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:05.490980 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:05.490696 28199 raft_consensus.cc:2804] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:10:05.492920 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:05.494853 23588 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:05.493072 28141 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:05.476953 28120 raft_consensus.cc:2804] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:05.487111 27839 raft_consensus.cc:2804] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:05.485693 28106 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:05.512051 27821 raft_consensus.cc:2804] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:05.519704 23026 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 850) took 2240 ms. Trace:
I20260709 13:10:05.521603 23588 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:05.525547 22878 raft_consensus.cc:1240] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:10:05.520061 23026 rpcz_store.cc:276] 0709 13:10:03.278967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:03.279085 (+   118us) service_pool.cc:224] Handling call
0709 13:10:03.279365 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:04.134934 (+855569us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:05.519492 (+1384558us) spinlock_profiling.cc:243] Waited 773 ms on lock 0x7b54001c84d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:05.519652 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1778450176}
I20260709 13:10:05.511240 28199 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:05.485052 28058 consensus_queue.cc:1048] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.034s
W20260709 13:10:05.533480 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:05.479494 28051 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:05.516798 28120 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:05.544241 23625 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 9 LEARNER]: Advancing to term 10
I20260709 13:10:05.519279 27839 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:05.544654 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:05.518090 28191 raft_consensus.cc:2804] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:05.524854 27821 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:05.520166 28079 raft_consensus.cc:3055] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:10:05.519558 28106 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:05.548449 22844 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:10:05.553032 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 87133.6us
I20260709 13:10:05.554399 22918 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:05.555181 23590 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:05.555327 22918 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:05.555440 22919 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:05.550827 27839 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:05.558321 22946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 2217) took 1376 ms. Trace:
I20260709 13:10:05.558703 22946 rpcz_store.cc:276] 0709 13:10:04.181765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:04.181875 (+   110us) service_pool.cc:224] Handling call
0709 13:10:04.198893 (+ 17018us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:04.268709 (+ 69816us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:10:04.282837 (+ 14128us) log.cc:844] Serialized 14 byte log entry
0709 13:10:04.417685 (+134848us) spinlock_profiling.cc:243] Waited 2.37 ms on lock 0x7b180020c560. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cf83f3 00007f6132cfa7c1 00007f6132cfa779 00007f6132cfa709 00007f6132cfa6d1 00007f6132cf99cd 000000000038f354 000000000038f199 00007f6132cdb83a
0709 13:10:04.648426 (+230741us) log.cc:844] Serialized 14 byte log entry
0709 13:10:05.441558 (+793132us) spinlock_profiling.cc:243] Waited 2.38 ms on lock 0x7b180020c560. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132c72c0a 00007f613c3b2219 00007f6132cfaa40 00007f6132cf54e4 00007f6132cf5455 00007f6135ddd060 00007f6135def716 00007f6135e00287 00007f6135e00208
0709 13:10:05.482704 (+ 41146us) log.cc:844] Serialized 14 byte log entry
0709 13:10:05.553594 (+ 70890us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:05.554051 (+   457us) log.cc:844] Serialized 26 byte log entry
0709 13:10:05.554209 (+   158us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:10:05.554243 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:05.554310 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:05.558123 (+  3813us) raft_consensus.cc:1685] finished
0709 13:10:05.558143 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:05.558266 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":167585,"prepare.queue_time_us":59929,"prepare.run_cpu_time_us":106,"prepare.run_wall_time_us":112,"raft.queue_time_us":955944,"raft.run_cpu_time_us":269,"raft.run_wall_time_us":12611,"spinlock_wait_cycles":21892480,"thread_start_us":371155,"threads_started":3,"wal-append.queue_time_us":407173}
I20260709 13:10:05.560837 28100 raft_consensus.cc:2749] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:05.562789 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:05.563956 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:05.569120 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:05.540719 28058 consensus_queue.cc:1243] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer f3c799b1249e40119d2a5864231c6c75 log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:10:05.570772 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:05.542457 28046 raft_consensus.cc:2749] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:10:05.540856 28199 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:05.577001 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:05.580911 22878 raft_consensus.cc:1240] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 12 LEADER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:10:05.581104 23027 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11.
I20260709 13:10:05.578976 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:05.545925 27975 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:05.550295 28120 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:05.601580 23616 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 15.
I20260709 13:10:05.585269 22878 raft_consensus.cc:1275] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:10:05.554453 28191 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:05.565706 28106 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:05.563935 27839 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:05.587741 27867 raft_consensus.cc:2804] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:10:05.594823 22875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 2239) took 2137 ms. Trace:
I20260709 13:10:05.564597 27821 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:05.598805 28084 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:05.603888 22879 raft_consensus.cc:1240] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:10:05.579473 23625 raft_consensus.cc:1275] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 LEARNER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 13:10:05.604746 28207 log.cc:927] Time spent T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
I20260709 13:10:05.606117 28197 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:05.614565 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:05.553874 28079 raft_consensus.cc:740] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:05.616178 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:05.616998 23026 pending_rounds.cc:85] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75: Aborting all ops after (but not including) 7
I20260709 13:10:05.551362 28135 consensus_queue.cc:1048] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.058s
W20260709 13:10:05.618944 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:05.609612 22875 rpcz_store.cc:276] 0709 13:10:03.457572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:03.462020 (+  4448us) service_pool.cc:224] Handling call
0709 13:10:03.462363 (+   343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:04.876051 (+1413688us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:05.594549 (+718498us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b540040ae54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:05.594739 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3089243904}
I20260709 13:10:05.647286 23026 pending_rounds.cc:107] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:10:05.647836 23616 raft_consensus.cc:2368] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 14. Current term is 15.
I20260709 13:10:05.618291 22843 leader_election.cc:400] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 14. Current term is 15.
I20260709 13:10:05.649566 22880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2769) took 1587 ms. Trace:
I20260709 13:10:05.673200 22843 leader_election.cc:403] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:10:05.672448 23026 raft_consensus.cc:2889] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:05.674754 22844 leader_election.cc:400] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 election: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 14. Current term is 15.
I20260709 13:10:05.660969 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:05.662882 27839 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:05.663475 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:05.571925 28100 consensus_queue.cc:1048] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.074s
I20260709 13:10:05.664772 27956 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:05.664870 28214 raft_consensus.cc:2804] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:10:05.665248 28089 raft_consensus.cc:2804] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:10:05.665558 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:10:05.684229 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:05.673355 22880 rpcz_store.cc:276] 0709 13:10:04.062010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:04.062670 (+   660us) service_pool.cc:224] Handling call
0709 13:10:04.063106 (+   436us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:10:04.103100 (+ 39994us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:04.111512 (+  8412us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:10:04.120996 (+  9484us) log.cc:844] Serialized 78 byte log entry
0709 13:10:05.333744 (+1212748us) spinlock_profiling.cc:243] Waited 3.8 ms on lock 0x7b1800434500. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cdae07 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf7c02 00007f613597f20b 00007f6135984d23 00007f61359850d1 00007f6135da0a0f
0709 13:10:05.358589 (+ 24845us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:10:05.358631 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:05.358987 (+   356us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:05.649295 (+290308us) raft_consensus.cc:1685] finished
0709 13:10:05.649315 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:05.649511 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":27924,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":184,"spinlock_wait_cycles":8802944,"thread_start_us":23978,"threads_started":1,"wal-append.queue_time_us":1282187}
I20260709 13:10:05.665809 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:05.619746 28191 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:05.667217 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:05.667627 28091 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:05.668273 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:05.669412 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 114025us
I20260709 13:10:05.690791 22919 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:05.669826 28051 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:05.688037 27979 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:05.693405 27979 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:05.693337 28079 consensus_queue.cc:260] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:05.694454 28079 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:05.669883 28153 raft_consensus.cc:2749] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:10:05.646814 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:05.677867 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:05.676316 22952 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 15.
I20260709 13:10:05.698886 23027 raft_consensus.cc:2410] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 16 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
W20260709 13:10:05.681545 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 63441.5us
I20260709 13:10:05.682734 27956 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:05.704727 27956 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:05.705183 28051 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:05.705686 28051 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:05.609227 28047 consensus_queue.cc:1059] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.080s
I20260709 13:10:05.683516 28214 raft_consensus.cc:697] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:05.665856 27972 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:05.686460 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:05.666584 22946 pending_rounds.cc:85] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting all ops after (but not including) 6
I20260709 13:10:05.762075 22946 pending_rounds.cc:107] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 13:10:05.762377 22946 raft_consensus.cc:2889] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:05.762694 22946 pending_rounds.cc:107] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:10:05.762959 22946 raft_consensus.cc:2889] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:05.695505 22954 raft_consensus.cc:2393] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 15: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:05.610471 27867 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:05.700404 23024 raft_consensus.cc:1275] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 13:10:05.701462 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:05.702736 27983 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:05.676400 28197 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:05.702883 28084 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:05.687089 28179 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:10:05.786573 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 95872.2us
I20260709 13:10:05.786674 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:05.787734 22919 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:05.707448 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:05.609150 28136 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:05.683262 23028 raft_consensus.cc:1275] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 13:10:05.713816 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:05.718698 22876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35370 (request call id 2782) took 1098 ms. Trace:
I20260709 13:10:05.686146 28164 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:05.793011 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:05.793356 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 58875.4us
I20260709 13:10:05.794301 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:05.620291 27975 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:05.721786 22992 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:05.724699 22918 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:05.795136 22880 raft_consensus.cc:2393] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 11: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:05.796957 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:05.797556 22878 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 14.
I20260709 13:10:05.721217 22871 raft_consensus.cc:1240] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 14. Current term is 15. Ops: []
W20260709 13:10:05.729907 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 147798us
I20260709 13:10:05.645787 28086 consensus_queue.cc:1048] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 13:10:05.761418 28152 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:05.763164 28077 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:05.808595 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:05.785398 22874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2789) took 1115 ms. Trace:
I20260709 13:10:05.789852 22876 rpcz_store.cc:276] 0709 13:10:04.620265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:05.137600 (+517335us) service_pool.cc:224] Handling call
0709 13:10:05.718498 (+580898us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b5400265554. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:05.718628 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":347415808}
I20260709 13:10:05.813181 22876 raft_consensus.cc:1275] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:10:05.697633 27953 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:05.799537 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:05.815376 27953 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:05.816126 22879 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
W20260709 13:10:05.801482 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:05.802826 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:05.817619 27953 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:05.766404 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:05.818221 22950 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 13.
I20260709 13:10:05.818385 23029 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:10:05.785176 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 76351.3us
I20260709 13:10:05.819262 22843 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:05.820828 28100 raft_consensus.cc:2749] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:05.805608 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:05.768206 27807 raft_consensus.cc:3055] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:10:05.821902 27807 raft_consensus.cc:740] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:05.822722 27807 consensus_queue.cc:260] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:05.823854 27807 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:05.769858 27867 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:05.827028 23029 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
W20260709 13:10:05.808864 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 87167.5us
I20260709 13:10:05.830199 22843 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:05.831858 27867 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:05.834573 27867 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:05.834851 28064 raft_consensus.cc:2749] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:10:05.698309 28091 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:10:05.836575 22878 raft_consensus.cc:2393] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 11: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:05.784756 28141 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:05.793459 28029 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:05.768478 27972 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:05.862494 22993 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:05.793061 28089 raft_consensus.cc:697] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:05.807693 28047 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:05.841185 27943 raft_consensus.cc:2804] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:05.835583 28153 raft_consensus.cc:2749] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:05.871640 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:05.882622 22876 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:05.789644 28197 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:05.890202 22992 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:05.786612 27983 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:05.850524 28064 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:05.889333 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:05.794109 28084 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:05.886106 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:05.806582 28164 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:05.847855 28135 consensus_queue.cc:1059] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 13:10:05.849289 27956 consensus_queue.cc:1059] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 13:10:05.849826 28077 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:05.934059 22876 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 12.
W20260709 13:10:05.918272 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 193656us
I20260709 13:10:05.840729 28152 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:05.863442 27867 consensus_queue.cc:1048] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 13:10:05.873698 28143 consensus_queue.cc:1059] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 13:10:05.945189 23590 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:05.873510 28076 consensus_queue.cc:1059] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
I20260709 13:10:05.928059 27912 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:10:05.946343 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 83925us
I20260709 13:10:05.884433 28228 raft_consensus.cc:2804] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:10:05.859284 28141 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:05.973793 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:05.988180 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:05.989441 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:05.881155 28051 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:05.868533 27979 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:05.959164 22953 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
I20260709 13:10:05.882051 27943 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:05.931667 28153 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:05.931097 27709 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:10:05.885553 28120 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:05.884789 28131 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:05.893507 28100 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:05.956861 27911 raft_consensus.cc:3055] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:10:05.972774 27956 consensus_queue.cc:1243] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer 0a407504ed1741bf89a276558adf4c0f log is divergent from this leader: its last log entry 12.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:10:05.881590 28047 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:05.974073 28091 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:05.923782 28214 consensus_queue.cc:237] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:05.976097 28197 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:05.977420 28136 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:06.056416 28076 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:05.929773 27953 consensus_queue.cc:1059] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:10:05.982101 27972 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:05.984019 28084 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:05.929162 27839 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:05.997999 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:06.086971 22952 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 14
I20260709 13:10:06.001247 22955 raft_consensus.cc:2393] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 16: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:06.003211 28077 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:06.004877 28106 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:06.010232 28191 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:06.090251 28191 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:06.090638 22951 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 859) took 2803 ms. Trace:
I20260709 13:10:06.091024 22951 rpcz_store.cc:276] 0709 13:10:03.287402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:03.431005 (+143603us) service_pool.cc:224] Handling call
0709 13:10:03.431288 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:04.482609 (+1051321us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:06.090430 (+1607821us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54000758d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:06.090583 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2358134016}
I20260709 13:10:06.091686 27953 raft_consensus.cc:3055] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:10:06.092121 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:06.093384 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:06.093154 27839 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:06.094369 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:06.026654 28141 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:06.028075 27983 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:05.964156 28135 raft_consensus.cc:2804] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:10:05.933718 28064 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:06.028576 28199 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:06.030555 28233 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:06.033520 27678 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:06.035007 22948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 869) took 1387 ms. Trace:
I20260709 13:10:06.065567 28131 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:05.997355 28228 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:06.038407 28089 consensus_queue.cc:237] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:06.049654 22876 raft_consensus.cc:1240] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 10. Current term is 11. Ops: []
I20260709 13:10:06.055220 22878 raft_consensus.cc:1240] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 13. Current term is 14. Ops: []
I20260709 13:10:06.058072 22843 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:06.016453 28029 raft_consensus.cc:2749] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:10:05.973758 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:06.108229 28197 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:06.055801 28059 raft_consensus.cc:3055] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:10:06.056814 27943 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:05.978711 27975 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:06.064623 27979 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:06.062398 27955 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:06.064298 27911 raft_consensus.cc:740] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:06.077730 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:06.080291 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 270831us
I20260709 13:10:06.227190 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:06.085728 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 195591us
I20260709 13:10:06.227636 22919 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:06.092182 27953 raft_consensus.cc:740] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:06.095724 27839 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:06.097132 22947 raft_consensus.cc:2393] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 18: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
W20260709 13:10:06.012048 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:06.098678 28106 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:06.017583 28164 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:06.064266 27709 raft_consensus.cc:697] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:06.104069 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:06.109303 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:06.113400 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:06.115031 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:06.121820 22952 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
I20260709 13:10:06.122798 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:06.241835 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:06.174239 22948 rpcz_store.cc:276] 0709 13:10:04.647461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:04.647618 (+   157us) service_pool.cc:224] Handling call
0709 13:10:04.647982 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:05.305548 (+657566us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:06.034764 (+729216us) spinlock_profiling.cc:243] Waited 657 ms on lock 0x7b5400411754. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:06.034944 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1510797568}
I20260709 13:10:06.243306 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:06.222276 27867 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:06.229656 27955 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:10:06.174983 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:06.244400 27867 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:06.245707 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:06.227783 27979 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:06.247041 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:06.247897 27912 raft_consensus.cc:2749] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:06.248104 27979 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:06.248894 27911 consensus_queue.cc:260] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:06.065474 28100 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:06.250610 27911 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:06.060945 28153 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:06.254514 28153 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:06.213351 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:06.099617 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:06.214560 28214 consensus_queue.cc:1059] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:10:06.214658 28232 consensus_queue.cc:1059] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:10:06.252475 28100 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:06.248203 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:06.221947 27811 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:06.259958 27811 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:06.260391 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:06.262706 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:05.998665 28216 log.cc:927] Time spent T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:10:06.219594 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:06.267392 22875 raft_consensus.cc:1240] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 15. Current term is 16. Ops: []
I20260709 13:10:06.228343 22992 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:10:06.277523 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 49248.2us
I20260709 13:10:06.278463 22992 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:06.280624 27975 raft_consensus.cc:2749] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:06.281487 27975 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:06.281368 27953 consensus_queue.cc:260] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:06.283593 27953 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:06.083093 28076 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:06.284327 28214 consensus_queue.cc:1243] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer 0a407504ed1741bf89a276558adf4c0f log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:10:06.284417 28232 consensus_queue.cc:1243] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer 0a407504ed1741bf89a276558adf4c0f log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:10:06.287462 27709 consensus_queue.cc:237] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:06.289525 28076 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:06.293817 28047 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:06.295524 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:06.297219 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:06.298303 22955 raft_consensus.cc:2393] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 13: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:06.299345 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:06.300104 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:06.127445 28135 raft_consensus.cc:697] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:06.302531 28131 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:06.302968 28135 consensus_queue.cc:237] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:06.217141 27821 consensus_meta.cc:287] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.566s	user 0.008s	sys 0.003s
I20260709 13:10:06.304961 27821 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:06.307849 27821 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:06.244068 22948 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
I20260709 13:10:06.308887 28199 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:06.309504 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:06.188288 28233 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:10:06.311013 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:06.312155 28233 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:10:06.312752 28239 log.cc:927] Time spent T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.230s	user 0.001s	sys 0.000s
I20260709 13:10:06.315059 28120 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:06.315876 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:06.317106 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:06.320178 23023 raft_consensus.cc:1275] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:10:06.320317 28064 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:06.322510 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:06.323243 23023 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:06.215898 28059 raft_consensus.cc:740] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:06.213076 28228 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:06.309692 23618 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
W20260709 13:10:06.218007 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 160016us
I20260709 13:10:06.218880 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:06.337926 22876 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 13.
I20260709 13:10:06.340199 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:06.234277 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:06.342866 23023 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 16.
I20260709 13:10:06.267920 22878 raft_consensus.cc:1240] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 13:10:06.356868 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:06.357384 22951 raft_consensus.cc:2410] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 11 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 13:10:06.227629 28130 raft_consensus.cc:2749] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:06.384765 22948 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15.
I20260709 13:10:06.238555 28179 raft_consensus.cc:2749] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 14. Current term is 15.
I20260709 13:10:06.227841 28197 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:06.471124 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 243539us
I20260709 13:10:06.417171 23026 raft_consensus.cc:1240] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 14. Current term is 15. Ops: []
W20260709 13:10:06.492007 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:06.475013 22879 raft_consensus.cc:1240] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 12 LEADER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 13:10:06.479440 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:06.470039 22992 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:06.478224 22843 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:06.398862 22951 raft_consensus.cc:1275] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:10:06.502341 22843 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:06.502769 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:06.503494 23029 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:10:06.504897 28233 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:10:06.404676 22877 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35370 (request call id 2788) took 1743 ms (client timeout 1757 ms). Trace:
W20260709 13:10:06.506491 22877 rpcz_store.cc:269] 0709 13:10:04.661121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:04.784937 (+123816us) service_pool.cc:224] Handling call
0709 13:10:06.404383 (+1619446us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b5400367754. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:06.404579 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":259806080}
I20260709 13:10:06.507117 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:06.493654 22919 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:06.508284 23588 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:06.507807 22951 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
I20260709 13:10:06.510748 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:06.511735 22843 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:06.513628 23029 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 13.
I20260709 13:10:06.513430 22843 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:06.515332 22843 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:06.515812 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:06.516347 22920 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:06.516734 22919 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:06.519281 27830 raft_consensus.cc:2749] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:10:06.519321 28233 consensus_queue.cc:1048] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:10:06.519688 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:06.359472 27960 raft_consensus.cc:2749] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:06.520898 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:06.521169 22955 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 16.
I20260709 13:10:06.521148 22843 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:06.337422 28059 consensus_queue.cc:260] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:06.521684 23028 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:06.522187 27830 raft_consensus.cc:2804] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:06.522711 27830 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:06.523077 27830 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:06.523531 28076 consensus_queue.cc:1059] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:10:06.517421 27678 raft_consensus.cc:2749] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:10:06.517735 28051 raft_consensus.cc:2804] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:10:06.530206 23028 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 16.
I20260709 13:10:06.530305 28051 raft_consensus.cc:697] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 14 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:06.532120 23588 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:06.516626 22951 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
I20260709 13:10:06.533099 23588 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:06.522143 28192 raft_consensus.cc:2804] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:10:06.534438 22843 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:06.447082 28219 raft_consensus.cc:3055] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:10:06.492220 28215 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:06.359278 28120 consensus_queue.cc:1048] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.151s
I20260709 13:10:06.543720 22951 raft_consensus.cc:1240] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14. Ops: []
I20260709 13:10:06.509481 28179 raft_consensus.cc:2749] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:10:06.548350 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:06.544441 22880 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 15.
I20260709 13:10:06.515275 28134 raft_consensus.cc:2749] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:10:06.551678 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:06.531670 28051 consensus_queue.cc:237] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:06.421464 27955 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:06.548969 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:06.553069 22951 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 11.
I20260709 13:10:06.558683 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:06.545823 27839 raft_consensus.cc:2804] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:10:06.554283 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:06.533388 28232 raft_consensus.cc:2749] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:10:06.565928 22952 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:06.566445 22951 raft_consensus.cc:1240] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 15. Current term is 16. Ops: []
I20260709 13:10:06.567458 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:06.568017 22951 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 15.
I20260709 13:10:06.554091 22843 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:06.549708 28190 raft_consensus.cc:2804] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:10:06.549665 28091 raft_consensus.cc:3055] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:10:06.547070 28191 raft_consensus.cc:2804] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:10:06.562695 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:06.550014 27678 raft_consensus.cc:2749] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:06.547587 28192 raft_consensus.cc:697] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:06.551324 28214 raft_consensus.cc:2749] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:10:06.545071 28219 raft_consensus.cc:740] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:06.569540 27839 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:06.543452 27960 consensus_queue.cc:1059] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:10:06.580770 22878 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:10:06.585103 28240 log.cc:927] Time spent T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 13:10:06.586575 27943 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:06.590549 27811 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:06.593286 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:06.632144 22955 raft_consensus.cc:1275] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 8. (term mismatch)
I20260709 13:10:06.632725 22955 pending_rounds.cc:85] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting all ops after (but not including) 7
I20260709 13:10:06.633046 22955 pending_rounds.cc:107] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting uncommitted NO_OP operation due to leader change: 9.8
I20260709 13:10:06.561595 27830 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:06.570672 22843 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:06.635509 28192 consensus_queue.cc:237] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:06.636950 28197 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:05.809645 22874 rpcz_store.cc:276] 0709 13:10:04.670330 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:05.447020 (+776690us) service_pool.cc:224] Handling call
0709 13:10:05.447261 (+   241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:05.447420 (+   159us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:10:05.552498 (+105078us) log.cc:844] Serialized 14 byte log entry
0709 13:10:05.785074 (+232576us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:05.785122 (+    48us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:10:05.785164 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:05.785229 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:05.785346 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55253,"prepare.queue_time_us":34607,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":31,"raft.queue_time_us":56243,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":14856,"thread_start_us":281680,"threads_started":2,"wal-append.queue_time_us":173781}
I20260709 13:10:06.637598 28219 consensus_queue.cc:260] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:06.625998 28191 raft_consensus.cc:697] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:06.639551 28191 consensus_queue.cc:237] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:06.648097 22878 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 16.
I20260709 13:10:06.616176 27839 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:06.577252 28179 raft_consensus.cc:2804] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:10:06.576638 28091 raft_consensus.cc:740] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:06.552862 28215 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:06.633358 22955 raft_consensus.cc:2889] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:06.603659 23026 raft_consensus.cc:1275] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:10:06.652714 22992 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:06.654247 23023 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:10:06.663077 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:06.665331 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 94713.1us
I20260709 13:10:06.667814 23023 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 16.
I20260709 13:10:06.670545 22843 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:06.625088 22952 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 15.
I20260709 13:10:06.672508 22947 raft_consensus.cc:1240] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 13:10:06.677014 28243 consensus_queue.cc:1048] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 13:10:06.682796 28120 consensus_queue.cc:1048] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:10:06.683630 27943 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:06.684238 28120 consensus_queue.cc:1059] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:10:06.685117 28244 log.cc:927] Time spent T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:10:06.685318 28120 raft_consensus.cc:3055] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:10:06.685676 28120 raft_consensus.cc:740] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
W20260709 13:10:06.685606 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:06.687795 28228 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:06.689044 28120 consensus_queue.cc:260] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:06.689692 28099 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:06.689976 23588 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:06.690274 28091 consensus_queue.cc:260] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:06.691162 28243 raft_consensus.cc:2804] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 13:10:06.691584 28243 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:06.691606 28091 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:06.673666 28200 raft_consensus.cc:2804] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:06.693082 28200 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:06.692123 28243 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:10:06.693557 28200 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:06.675107 28197 consensus_queue.cc:1059] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:10:06.695999 28134 raft_consensus.cc:1064] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
I20260709 13:10:06.696379 28197 raft_consensus.cc:3055] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:10:06.696856 28197 raft_consensus.cc:740] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:06.697674 28197 consensus_queue.cc:260] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:06.696475 28134 raft_consensus.cc:1068] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f: Unable to promote non-voter 5b346767a89745fa82928a5dac9cc0b2: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:10:06.698887 28197 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:06.699355 28120 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:06.698133 27830 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:06.702047 22954 raft_consensus.cc:2368] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 12. Current term is 13.
I20260709 13:10:06.821413 28232 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:06.821966 28232 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:06.823428 22954 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58320 (request call id 2949) took 1109 ms. Trace:
I20260709 13:10:06.654663 27811 consensus_queue.cc:1059] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 13:10:06.824041 28232 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:06.824638 27811 raft_consensus.cc:3055] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:10:06.820808 28231 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:06.702260 28228 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:06.715209 27807 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:06.825968 28231 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:06.826860 27807 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:06.827454 27807 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:06.828737 27955 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:06.829211 27955 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:06.655920 28179 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:06.830628 28179 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:06.831696 27955 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:06.714824 28215 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:06.716720 22946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2935) took 1069 ms. Trace:
W20260709 13:10:06.745141 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:06.834980 22843 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:06.836125 22873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 2257) took 1289 ms. Trace:
I20260709 13:10:06.836800 27830 consensus_queue.cc:1048] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:10:06.837742 22878 raft_consensus.cc:1240] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 13:10:06.838434 27830 raft_consensus.cc:2804] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:10:06.838827 27830 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:06.839190 27830 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:06.840318 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:06.843307 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:06.843900 22876 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 12.
I20260709 13:10:06.833429 22946 rpcz_store.cc:276] 0709 13:10:05.646922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:05.647819 (+   897us) service_pool.cc:224] Handling call
0709 13:10:05.666366 (+ 18547us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:10:05.814012 (+147646us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:05.814039 (+    27us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:10:06.041852 (+227813us) spinlock_profiling.cc:243] Waited 22.1 ms on lock 0x7b2c0005beb8. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f61346dea11 00007f61346ee372 00007f6135e20a01 00007f6135de543a 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2
0709 13:10:06.042442 (+   590us) log.cc:844] Serialized 78 byte log entry
0709 13:10:06.114006 (+ 71564us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:10:06.114040 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:06.114102 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:06.716274 (+602172us) raft_consensus.cc:1685] finished
0709 13:10:06.716311 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:06.716626 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45127,"raft.queue_time_us":22373,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":14974,"spinlock_wait_cycles":50934016,"thread_start_us":121536,"threads_started":2,"wal-append.queue_time_us":354023}
I20260709 13:10:06.823802 22954 rpcz_store.cc:276] 0709 13:10:05.713997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:06.226946 (+512949us) service_pool.cc:224] Handling call
0709 13:10:06.823242 (+596296us) spinlock_profiling.cc:243] Waited 474 ms on lock 0x7b54000753d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:06.823371 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1090265856}
I20260709 13:10:06.846248 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:06.846508 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:06.847031 22954 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:10:06.778527 22871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53724 (request call id 898) took 1971 ms. Trace:
I20260709 13:10:06.847412 22871 rpcz_store.cc:276] 0709 13:10:04.807184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:05.509215 (+702031us) service_pool.cc:224] Handling call
0709 13:10:05.721014 (+211799us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:05.807367 (+ 86353us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:06.778316 (+970949us) spinlock_profiling.cc:243] Waited 210 ms on lock 0x7b5400265550. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:06.778468 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":482146048}
I20260709 13:10:06.849097 22843 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:06.850050 28062 raft_consensus.cc:2749] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:06.850265 22874 raft_consensus.cc:1240] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 13:10:06.851809 28231 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:06.779277 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:06.781224 27356 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:06.853070 22843 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:06.853245 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:06.854562 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:06.855104 22953 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 17.
I20260709 13:10:06.755131 22950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58320 (request call id 2944) took 1090 ms. Trace:
I20260709 13:10:06.855796 23623 raft_consensus.cc:1240] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 11. Current term is 12. Ops: []
I20260709 13:10:06.855713 22950 rpcz_store.cc:276] 0709 13:10:05.673582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:05.686223 (+ 12641us) service_pool.cc:224] Handling call
0709 13:10:06.754953 (+1068730us) spinlock_profiling.cc:243] Waited 56.3 ms on lock 0x7b54000ec9d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:06.755074 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":129531776}
I20260709 13:10:06.784049 28153 raft_consensus.cc:3055] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:10:06.856644 22992 leader_election.cc:400] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 12. Current term is 13.
I20260709 13:10:06.856901 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:06.857313 22992 leader_election.cc:403] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:10:06.857121 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:06.857689 22950 raft_consensus.cc:2393] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 13: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:06.858446 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:06.858958 22950 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 15.
I20260709 13:10:06.859454 27943 raft_consensus.cc:2749] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 12. Current term is 13.
I20260709 13:10:06.859782 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:06.860954 28228 consensus_queue.cc:1059] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:10:06.861249 28153 raft_consensus.cc:740] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:06.803529 22949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2939) took 1140 ms. Trace:
I20260709 13:10:06.809101 28229 raft_consensus.cc:3055] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:10:06.863189 28229 raft_consensus.cc:740] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:06.862684 28153 consensus_queue.cc:260] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:06.818035 27960 consensus_queue.cc:1243] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer 0a407504ed1741bf89a276558adf4c0f log is divergent from this leader: its last log entry 12.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:10:06.818974 28089 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:06.864467 28089 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:06.862323 22949 rpcz_store.cc:276] 0709 13:10:05.662939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:05.666304 (+  3365us) service_pool.cc:224] Handling call
0709 13:10:05.666537 (+   233us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:05.892285 (+225748us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:10:05.929841 (+ 37556us) log.cc:844] Serialized 14 byte log entry
0709 13:10:06.085031 (+155190us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:06.085081 (+    50us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:10:06.085108 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:06.803291 (+718183us) spinlock_profiling.cc:243] Waited 225 ms on lock 0x7b5400285854. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:06.803340 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:06.803468 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22217,"prepare.queue_time_us":64741,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":25040,"raft.run_cpu_time_us":294,"raft.run_wall_time_us":22320,"spinlock_wait_cycles":518494464,"thread_start_us":132403,"threads_started":1,"wal-append.queue_time_us":151493}
I20260709 13:10:06.865871 22949 raft_consensus.cc:1240] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14. Ops: []
I20260709 13:10:06.866539 28089 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:06.820302 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:06.868026 22918 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:06.868786 28229 consensus_queue.cc:260] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:06.870550 27709 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:06.870343 27956 consensus_queue.cc:1048] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:10:06.870990 27960 raft_consensus.cc:2749] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:10:06.871481 27975 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:06.871953 27975 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:06.871019 27709 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:06.874044 27975 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:06.874569 27975 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:06.875000 27709 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:06.875000 27975 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:06.876837 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:06.877396 23021 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 12.
I20260709 13:10:06.877621 27975 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:06.878692 23021 raft_consensus.cc:1240] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 13. Current term is 14. Ops: []
I20260709 13:10:06.879913 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:06.880507 23021 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 13.
I20260709 13:10:06.881085 28089 consensus_queue.cc:1059] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:10:06.881749 28089 consensus_queue.cc:1243] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Peer 0a407504ed1741bf89a276558adf4c0f log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:10:06.882447 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:06.883134 23029 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
I20260709 13:10:06.883684 28153 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:06.884770 23021 raft_consensus.cc:1275] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 13:10:06.885903 22874 raft_consensus.cc:1240] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:10:06.825162 27811 raft_consensus.cc:740] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:06.886873 27356 raft_consensus.cc:2749] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:06.887581 27811 consensus_queue.cc:260] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:06.888866 27811 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:06.890515 27960 consensus_queue.cc:1059] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:10:06.789862 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 227436us
I20260709 13:10:06.894831 28253 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:06.895364 28253 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:06.897526 28253 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:06.899784 28131 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:06.900334 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:06.901384 23618 raft_consensus.cc:1240] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:10:06.901614 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:06.888576 28136 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:10:06.904093 28136 raft_consensus.cc:697] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 14 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:06.900243 28131 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:06.905390 27975 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:06.906400 28131 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:06.861668 28228 consensus_queue.cc:1243] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer 0a407504ed1741bf89a276558adf4c0f log is divergent from this leader: its last log entry 9.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 13:10:06.800336 27979 raft_consensus.cc:3055] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:10:06.907274 27979 raft_consensus.cc:740] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:06.907537 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:06.908515 22880 raft_consensus.cc:2410] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:10:06.908246 27979 consensus_queue.cc:260] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:06.909567 27979 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:10:06.909413 28136 consensus_queue.cc:237] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:06.814606 28059 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:06.913727 28152 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:06.914685 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:06.885699 28099 consensus_queue.cc:1048] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:10:06.915117 22992 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
W20260709 13:10:06.915421 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:06.916000 28099 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:06.916424 28099 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:06.916836 27911 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:10:06.917234 27911 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:06.917642 27911 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:06.918576 28099 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:06.919675 28248 log.cc:927] Time spent T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.306s	user 0.001s	sys 0.002s
I20260709 13:10:06.921650 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:06.922590 22879 raft_consensus.cc:2393] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 15: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:06.923550 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:06.924764 27912 consensus_queue.cc:1059] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:10:06.925243 22843 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:06.925405 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:06.869951 28229 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:06.925895 27912 raft_consensus.cc:3055] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:10:06.926311 27912 raft_consensus.cc:740] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:06.926337 28130 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:06.926887 28130 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:06.927253 28130 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:06.928010 22875 raft_consensus.cc:1275] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:10:06.929765 22879 raft_consensus.cc:1240] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 11. Current term is 12. Ops: []
W20260709 13:10:06.930074 28247 log.cc:927] Time spent T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.317s	user 0.000s	sys 0.004s
I20260709 13:10:06.927380 27912 consensus_queue.cc:260] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:06.931809 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:06.932250 22843 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:06.932428 27912 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:06.890205 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:06.933036 28134 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:06.933295 22874 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 16.
I20260709 13:10:06.933485 28134 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:06.933936 28134 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:06.833953 28078 raft_consensus.cc:3055] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:10:06.937319 28078 raft_consensus.cc:740] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:06.943588 22954 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15.
I20260709 13:10:06.944180 28078 consensus_queue.cc:260] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:06.954231 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:06.955458 22948 raft_consensus.cc:2410] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 7.
I20260709 13:10:06.948704 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:06.949761 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:06.982774 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:10:06.820330 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 127582us
I20260709 13:10:06.984117 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:06.950256 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:06.984858 23588 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:06.951290 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:06.985499 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:06.954548 28078 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:06.955642 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:06.955955 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:06.956604 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:06.957056 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:06.957579 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:06.997790 23020 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11.
I20260709 13:10:06.958484 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:06.958784 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:06.998651 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:06.768909 27839 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:06.905881 27975 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:06.997051 23024 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 15.
I20260709 13:10:06.802995 28200 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:07.004190 22920 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:07.013062 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:06.963537 22950 raft_consensus.cc:1240] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:10:07.014905 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:07.015571 22950 raft_consensus.cc:1275] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:10:06.966636 27830 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:06.968091 22955 raft_consensus.cc:1240] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:10:07.019223 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:07.019789 23588 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:06.941990 28179 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:06.966377 22951 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 2226) took 1129 ms. Trace:
I20260709 13:10:07.021930 22950 raft_consensus.cc:1240] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:10:07.021189 22951 rpcz_store.cc:276] 0709 13:10:05.837437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:06.568546 (+731109us) service_pool.cc:224] Handling call
0709 13:10:06.568894 (+   348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:06.811967 (+243073us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:06.812012 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:06.812590 (+   578us) log.cc:844] Serialized 26 byte log entry
0709 13:10:06.966071 (+153481us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:10:06.966109 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:06.966173 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:06.966212 (+    39us) raft_consensus.cc:1685] finished
0709 13:10:06.966223 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:06.966326 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":242707,"raft.queue_time_us":244182,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":165,"thread_start_us":152936,"threads_started":1,"wal-append.queue_time_us":151526}
I20260709 13:10:07.029700 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:07.029608 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:07.029356 27979 raft_consensus.cc:2804] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:06.974201 22953 raft_consensus.cc:1275] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
W20260709 13:10:07.017851 28260 log.cc:927] Time spent T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
I20260709 13:10:07.040083 22951 raft_consensus.cc:2955] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Committing config change with OpId 7.6: config changed from index -1 to 6, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:10:07.041522 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:07.042076 23027 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:10:06.976030 27911 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:06.979041 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 46862.1us
I20260709 13:10:06.957883 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:06.982978 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:06.853415 28190 raft_consensus.cc:697] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:06.986519 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:06.953820 23022 raft_consensus.cc:1240] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 11. Current term is 12. Ops: [7.6-11.8]
I20260709 13:10:06.932461 22949 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 13.
W20260709 13:10:07.049124 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:07.049479 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:06.846624 28243 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:06.836524 22873 rpcz_store.cc:276] 0709 13:10:05.546498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:05.546664 (+   166us) service_pool.cc:224] Handling call
0709 13:10:06.404587 (+857923us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:06.464613 (+ 60026us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:06.464647 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:06.465245 (+   598us) log.cc:844] Serialized 26 byte log entry
0709 13:10:06.834694 (+369449us) raft_consensus.cc:1645] Marking committed up to 4
0709 13:10:06.834730 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:06.834799 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:06.834839 (+    40us) raft_consensus.cc:1685] finished
0709 13:10:06.834849 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:06.835868 (+  1019us) spinlock_profiling.cc:243] Waited 839 ms on lock 0x7b5400367750. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:06.836068 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2505,"raft.queue_time_us":59588,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":5833,"spinlock_wait_cycles":1930100992,"thread_start_us":369109,"threads_started":1,"wal-append.queue_time_us":338832}
I20260709 13:10:06.947413 28130 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:06.994987 22948 raft_consensus.cc:1240] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:10:06.995019 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:06.958199 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:07.057122 23624 raft_consensus.cc:2368] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16.
I20260709 13:10:07.003978 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:06.959586 23621 raft_consensus.cc:1275] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 14 index: 13. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 13:10:07.016637 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:07.017139 22919 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:07.058758 22871 raft_consensus.cc:2393] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 15: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
W20260709 13:10:07.059405 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:07.020498 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:07.036861 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:07.037777 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:06.974872 22946 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 15.
I20260709 13:10:07.060987 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:07.061606 23026 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
I20260709 13:10:06.945312 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:07.063117 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:07.062429 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:07.037531 27979 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:07.037667 27996 raft_consensus.cc:2804] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:07.091617 27996 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:07.091694 27950 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:07.092120 27996 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:07.039470 22952 raft_consensus.cc:1240] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 12. Current term is 14. Ops: []
I20260709 13:10:06.995900 28197 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:07.051196 22992 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:07.018790 28135 raft_consensus.cc:2804] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:10:07.051929 22876 raft_consensus.cc:1240] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:10:07.053277 22880 raft_consensus.cc:2368] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14.
I20260709 13:10:07.054292 23025 raft_consensus.cc:2393] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 13: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:07.050099 22949 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
I20260709 13:10:07.047827 22843 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:07.000980 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:07.057111 23625 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 13.
I20260709 13:10:07.061738 22993 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:07.066550 23029 raft_consensus.cc:1275] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:10:07.067610 22919 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:07.064241 23026 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 16.
W20260709 13:10:07.069311 28252 log.cc:927] Time spent T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.409s	user 0.006s	sys 0.003s
I20260709 13:10:07.074596 22947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 2238) took 1126 ms. Trace:
W20260709 13:10:07.077029 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:07.080703 27839 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:07.052333 28136 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:06.981942 28134 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:07.067341 22994 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:10:07.159065 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 91784.1us
I20260709 13:10:06.987118 27807 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:07.063665 23021 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
I20260709 13:10:07.099856 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:10:07.098380 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 47302.4us
I20260709 13:10:07.140826 28154 raft_consensus.cc:2749] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:10:07.098379 23622 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11.
I20260709 13:10:07.094192 27979 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:07.131158 27943 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:07.131914 27912 raft_consensus.cc:2804] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:10:07.130623 23024 raft_consensus.cc:1240] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 13. Current term is 14. Ops: []
I20260709 13:10:07.136778 27811 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:07.137506 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 89748.6us
W20260709 13:10:07.140641 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 78952.8us
I20260709 13:10:07.105599 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:07.141563 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:07.142309 23027 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
I20260709 13:10:07.144414 22947 rpcz_store.cc:276] 0709 13:10:05.948737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:06.641490 (+692753us) service_pool.cc:224] Handling call
0709 13:10:06.671461 (+ 29971us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:06.673037 (+  1576us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:07.074339 (+401302us) spinlock_profiling.cc:243] Waited 29.6 ms on lock 0x7b540037aad0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:07.074522 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":68022528}
W20260709 13:10:07.149556 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 82010.1us
I20260709 13:10:07.090351 28229 raft_consensus.cc:2749] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:07.165417 23619 raft_consensus.cc:2393] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 17: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:07.100917 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:07.169796 23029 raft_consensus.cc:3055] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Stepping down as leader of term 11
W20260709 13:10:07.181332 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:07.282097 23625 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
I20260709 13:10:07.285181 23027 raft_consensus.cc:1240] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:10:07.289650 22919 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:07.290541 23029 raft_consensus.cc:740] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:07.206871 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:07.153048 28089 raft_consensus.cc:2749] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:07.154058 28084 raft_consensus.cc:2804] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:10:07.298599 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:07.320847 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 158325us
I20260709 13:10:07.143010 28197 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:07.281420 23624 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:07.162443 28135 raft_consensus.cc:697] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 15 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:07.103914 28250 consensus_queue.cc:1048] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 5, Time since last communication: 0.202s
I20260709 13:10:07.298143 22992 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:07.291823 23029 consensus_queue.cc:260] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:07.306349 28153 raft_consensus.cc:2749] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:07.307497 22919 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:07.307385 28078 consensus_queue.cc:1048] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:10:07.331261 23029 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:07.176501 27912 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
W20260709 13:10:07.344691 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:07.351428 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 44011.9us
I20260709 13:10:07.354169 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:07.354674 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:07.355118 22949 raft_consensus.cc:2410] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 9.
I20260709 13:10:07.355875 22946 raft_consensus.cc:2410] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:10:07.356917 28153 raft_consensus.cc:2804] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:10:07.357443 28153 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:07.357849 28153 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:07.312865 28179 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:07.160866 28136 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:07.326635 28046 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:07.311515 28154 consensus_queue.cc:1059] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.057s
I20260709 13:10:07.295156 27943 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:07.373628 23588 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:07.345952 27912 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:07.376996 27950 consensus_queue.cc:1059] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:10:07.372592 28272 log.cc:927] Time spent T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 13:10:07.378007 28243 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:07.382511 27807 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:07.409503 28253 raft_consensus.cc:2749] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:07.426887 23624 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15.
I20260709 13:10:07.428401 28136 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:07.409853 27950 consensus_queue.cc:1243] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer aed3f089ec8b49af9c8cf5cdee76e8e7 log is divergent from this leader: its last log entry 16.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:10:07.429312 27356 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:07.429805 27356 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:07.432024 27356 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:07.388464 27975 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:07.390241 27830 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:07.391816 28197 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:07.393338 28135 consensus_queue.cc:237] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:07.362282 28120 consensus_meta.cc:287] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.540s	user 0.005s	sys 0.004s
I20260709 13:10:07.326855 27811 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:07.397538 23029 raft_consensus.cc:1275] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 13:10:07.487708 23029 pending_rounds.cc:85] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75: Aborting all ops after (but not including) 9
I20260709 13:10:07.488112 23029 pending_rounds.cc:107] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 13:10:07.488400 23029 raft_consensus.cc:2889] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:07.344242 28200 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:07.296243 28190 consensus_queue.cc:237] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:07.407969 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:07.491333 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:07.413758 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:07.417187 27979 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:07.422842 27943 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:07.424365 28134 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:07.466411 28274 raft_consensus.cc:2764] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election decision vote started in defunct term 15: won
W20260709 13:10:07.467281 27678 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.764s	user 0.006s	sys 0.010s
I20260709 13:10:07.502513 27911 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:07.395208 28046 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:07.508576 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 208211us
I20260709 13:10:07.432507 28153 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:07.514122 22955 raft_consensus.cc:1240] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:10:07.529539 22947 raft_consensus.cc:1275] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:10:07.512069 22843 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:10:07.538246 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 216046us
W20260709 13:10:07.550107 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:07.583901 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 286629us
I20260709 13:10:07.589867 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:07.590137 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:07.590778 23619 raft_consensus.cc:2393] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 17: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:07.591336 22950 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 11.
I20260709 13:10:07.592209 22993 leader_election.cc:400] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16.
I20260709 13:10:07.597512 22871 raft_consensus.cc:1275] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 13:10:07.593956 22994 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:07.593369 23590 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:07.599735 22994 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:07.595181 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:07.602022 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:10:07.575421 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 63423.3us
I20260709 13:10:07.582382 28197 raft_consensus.cc:2804] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:10:07.606765 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:07.606634 22949 raft_consensus.cc:1275] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:10:07.612773 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:07.613106 22871 raft_consensus.cc:1240] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:07.613659 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:07.611344 23588 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:07.614691 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:07.615825 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:07.581670 28134 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:07.608461 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 310405us
I20260709 13:10:07.610472 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:07.619014 22948 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:07.611012 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:07.620985 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:07.622129 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:07.622823 22954 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:07.624976 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:07.627866 22950 raft_consensus.cc:1275] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:10:07.596390 27830 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:07.631173 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:07.603370 28194 raft_consensus.cc:2749] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:07.608235 28233 raft_consensus.cc:2749] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:07.612881 28197 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:07.594790 27950 consensus_queue.cc:1048] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:10:07.649565 27911 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:07.636152 28249 raft_consensus.cc:2804] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 13:10:07.622977 27958 consensus_queue.cc:1048] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 13:10:07.649166 28046 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:07.641904 27709 raft_consensus.cc:2749] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:10:07.660239 27678 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.957s	user 0.007s	sys 0.010s
I20260709 13:10:07.664554 28197 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:07.629478 28134 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:07.695326 27911 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:07.638960 27830 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:07.720279 27979 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:07.706902 28249 raft_consensus.cc:697] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:07.708909 28084 raft_consensus.cc:697] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 15 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:07.726639 27911 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:07.725968 27811 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:07.739254 27996 consensus_meta.cc:287] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.647s	user 0.007s	sys 0.004s
I20260709 13:10:07.741875 28153 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:07.745049 28084 consensus_queue.cc:237] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:07.745754 28134 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:07.752818 28249 consensus_queue.cc:237] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:07.755517 28179 raft_consensus.cc:1064] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
W20260709 13:10:07.756003 28179 raft_consensus.cc:1068] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f: Unable to promote non-voter 5b346767a89745fa82928a5dac9cc0b2: Illegal state: Leader has not yet committed an operation in its own term
W20260709 13:10:07.795943 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:07.809487 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 190136us
W20260709 13:10:07.813215 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 196786us
I20260709 13:10:07.816471 28091 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:07.817446 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 202481us
I20260709 13:10:07.818161 28091 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:07.747014 27996 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:07.821280 28191 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:07.822392 28191 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:07.825028 27830 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:07.830197 27768 raft_consensus.cc:3055] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:10:07.830744 27768 raft_consensus.cc:740] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:07.837285 28285 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:07.838208 28285 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:07.844609 27678 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:07.845621 27678 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:07.850068 28091 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:07.851007 28134 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:07.851991 28134 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:07.853439 28285 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:07.854383 23023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32794 (request call id 880) took 1352 ms. Trace:
I20260709 13:10:07.854951 23023 rpcz_store.cc:276] 0709 13:10:06.501909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:06.502095 (+   186us) service_pool.cc:224] Handling call
0709 13:10:07.853816 (+1351721us) spinlock_profiling.cc:243] Waited 134 ms on lock 0x7b5400248454. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:07.854063 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":307393920}
I20260709 13:10:07.855959 27996 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:07.857750 22994 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:07.861274 28191 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:07.848281 27678 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:07.865765 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:07.864643 28153 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:07.871717 28153 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:07.891901 28292 consensus_queue.cc:1048] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:10:07.893991 28134 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:07.896132 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:07.897693 22873 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 15.
I20260709 13:10:07.908010 28130 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:07.910503 28274 raft_consensus.cc:3055] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:10:07.911149 28274 raft_consensus.cc:740] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:10:07.875982 22843 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.532s (SENT) [suppressed 9 similar messages]
I20260709 13:10:07.885210 27768 consensus_queue.cc:260] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:07.924077 28274 consensus_queue.cc:260] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:07.928066 22876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53724 (request call id 916) took 1026 ms. Trace:
I20260709 13:10:07.908365 27867 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:07.865963 28285 raft_consensus.cc:2804] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:07.930706 22876 rpcz_store.cc:276] 0709 13:10:06.901632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:06.913220 (+ 11588us) service_pool.cc:224] Handling call
0709 13:10:06.913499 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:07.136371 (+222872us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:07.832617 (+696246us) spinlock_profiling.cc:243] Waited 138 ms on lock 0x7b5400301e54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:07.927807 (+ 95190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":317941504}
I20260709 13:10:07.936997 28285 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:07.952069 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:07.953172 22873 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 15.
I20260709 13:10:07.954694 28153 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:07.955967 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:07.956768 23624 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:07.961241 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:07.961963 22950 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 16.
I20260709 13:10:07.963135 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:07.963943 22950 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:10:07.965145 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 68378.4us
I20260709 13:10:07.966717 22994 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:07.969141 28291 raft_consensus.cc:2804] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:10:07.969549 28291 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:07.969941 28291 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:07.980736 27768 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:07.982425 28130 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:07.986413 23590 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:07.987133 28194 consensus_queue.cc:1059] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:10:07.990957 28249 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:07.991235 28120 raft_consensus.cc:3055] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:10:07.992038 28120 raft_consensus.cc:740] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:07.983726 28130 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:07.991487 28249 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:10:07.994024 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:07.992372 23588 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:07.951363 28285 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:08.008713 23624 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 14.
W20260709 13:10:08.015195 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:08.002058 28249 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:10:08.017587 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 141811us
I20260709 13:10:08.020851 22919 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:08.022612 28153 raft_consensus.cc:2804] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:10:08.023157 28153 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:08.023658 28153 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:08.026247 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:08.027007 23624 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15.
I20260709 13:10:08.027225 23622 raft_consensus.cc:1240] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEADER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 15. Current term is 16. Ops: []
I20260709 13:10:08.029795 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:08.030833 22873 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 15.
I20260709 13:10:08.031070 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:08.030936 28197 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:08.033743 22952 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 2243) took 1992 ms. Trace:
I20260709 13:10:08.034495 28197 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:08.035514 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:08.036252 22876 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:10:08.036717 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:08.037516 22951 raft_consensus.cc:2393] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 16: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:08.039577 27625 consensus_queue.cc:1059] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:10:08.042481 22954 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 16.
I20260709 13:10:08.043087 28233 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:08.044092 22954 raft_consensus.cc:1217] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Deduplicated request from leader. Original: 12.11->[14.12-17.13]   Dedup: 14.12->[17.13-17.13]
I20260709 13:10:08.043771 28233 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:08.045545 22946 raft_consensus.cc:1240] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 15. Current term is 16. Ops: [12.9-15.10]
I20260709 13:10:08.075068 22954 raft_consensus.cc:1240] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 17. Current term is 18. Ops: [14.12-17.13]
I20260709 13:10:08.049783 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:08.034121 22952 rpcz_store.cc:276] 0709 13:10:06.041119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:06.671620 (+630501us) service_pool.cc:224] Handling call
0709 13:10:06.671886 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:07.092888 (+421002us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:08.033275 (+940387us) spinlock_profiling.cc:243] Waited 367 ms on lock 0x7b54000aa0d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:08.033577 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":844909440}
I20260709 13:10:08.051680 22919 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:08.053378 22994 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:08.052770 28122 raft_consensus.cc:2955] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 LEADER]: Committing config change with OpId 12.9: config changed from index 5 to 9, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:08.048482 23590 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:08.055938 22880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35370 (request call id 2804) took 1139 ms. Trace:
I20260709 13:10:08.057572 28153 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:08.058986 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:08.029887 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:08.061653 22947 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 17.
I20260709 13:10:08.063557 23588 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:08.063949 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:08.067636 28134 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:08.068790 28141 consensus_queue.cc:1048] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:10:08.070786 27678 consensus_queue.cc:1048] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:10:08.071688 22843 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:08.027149 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:08.076845 22880 rpcz_store.cc:276] 0709 13:10:06.916583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:06.923261 (+  6678us) service_pool.cc:224] Handling call
0709 13:10:08.055461 (+1132200us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b540030d4d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:08.055693 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":296354688}
W20260709 13:10:08.089995 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:08.091615 22873 raft_consensus.cc:1240] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 15. Current term is 16. Ops: []
W20260709 13:10:08.093552 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 45295.7us
I20260709 13:10:08.098533 22950 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 16.
I20260709 13:10:08.091615 22955 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 2963) took 1271 ms. Trace:
W20260709 13:10:08.099653 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:08.082700 28192 raft_consensus.cc:2749] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:08.099889 22955 rpcz_store.cc:276] 0709 13:10:06.819979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:07.020334 (+200355us) service_pool.cc:224] Handling call
0709 13:10:07.020641 (+   307us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:07.528468 (+507827us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:08.077627 (+549159us) spinlock_profiling.cc:243] Waited 493 ms on lock 0x7b540020ec54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:08.091414 (+ 13787us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1133967744}
W20260709 13:10:08.108007 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:08.108963 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:08.109012 28051 raft_consensus.cc:2749] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:10:08.105743 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:08.101329 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:08.112592 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:08.082824 28141 consensus_queue.cc:1243] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer aed3f089ec8b49af9c8cf5cdee76e8e7 log is divergent from this leader: its last log entry 11.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:10:08.115983 27356 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:08.116722 27356 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:08.116943 28091 consensus_queue.cc:1059] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:10:08.120600 22876 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 16.
I20260709 13:10:08.108443 27768 raft_consensus.cc:2804] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:08.119974 23590 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:08.109788 28078 raft_consensus.cc:2804] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:10:08.128155 28153 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:08.128420 22946 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 16.
I20260709 13:10:08.132541 27975 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:08.136196 28091 consensus_queue.cc:1243] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer 5b346767a89745fa82928a5dac9cc0b2 log is divergent from this leader: its last log entry 16.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:10:08.137722 28078 raft_consensus.cc:697] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:08.138265 22947 raft_consensus.cc:1275] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:10:08.138588 27979 raft_consensus.cc:3055] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:10:08.139227 27979 raft_consensus.cc:740] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:08.138996 28078 consensus_queue.cc:237] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:08.123203 27867 consensus_queue.cc:1059] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 13:10:08.105293 28134 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:08.122394 27768 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:08.140970 22952 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:08.135504 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:08.141108 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:08.151656 28130 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:08.153790 27912 consensus_meta.cc:287] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.742s	user 0.020s	sys 0.001s
I20260709 13:10:08.154304 27912 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:08.157488 23028 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 13.
W20260709 13:10:08.161052 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 107912us
I20260709 13:10:08.162092 22994 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:08.162494 27678 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:08.163249 27678 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:08.162983 28120 consensus_queue.cc:260] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:08.164773 27912 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:08.165112 28120 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:08.166517 27678 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:08.134686 28192 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:08.168890 28192 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:08.169414 28192 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:08.160419 28285 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:08.171190 28285 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:08.142156 27625 raft_consensus.cc:3055] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 13:10:08.178231 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 114808us
I20260709 13:10:08.172930 27625 raft_consensus.cc:740] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:08.148224 27768 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:10:08.189916 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 118330us
I20260709 13:10:08.192526 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:08.194867 28134 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:08.199185 28192 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:08.151676 27356 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:08.201403 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:08.201781 28194 consensus_queue.cc:1048] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:10:08.202286 23023 raft_consensus.cc:2393] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 11: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:08.203081 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:08.203739 23617 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:08.204026 23588 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:08.204480 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:08.205228 23616 raft_consensus.cc:2368] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14.
I20260709 13:10:08.206003 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:08.140628 27979 consensus_queue.cc:260] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:08.206707 23624 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 15.
I20260709 13:10:08.203836 28194 consensus_queue.cc:1059] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 13:10:08.213294 27625 consensus_queue.cc:260] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:08.215189 27625 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:08.207072 27979 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:08.173729 28233 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:08.222391 22844 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:08.224874 23617 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
W20260709 13:10:08.178560 28298 log.cc:927] Time spent T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:10:08.230897 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:08.232407 23026 raft_consensus.cc:1275] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:10:08.230971 28243 raft_consensus.cc:2804] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:10:08.216233 27768 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:08.217305 27912 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:10:08.245221 28219 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:10:08.249686 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 166147us
I20260709 13:10:08.253474 28134 raft_consensus.cc:2804] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:08.253963 28134 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:08.254475 28134 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:08.256513 28243 raft_consensus.cc:697] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:08.257768 28243 consensus_queue.cc:237] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:08.253289 28058 consensus_queue.cc:1048] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:10:08.261546 28291 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:08.264029 28291 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:08.253877 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:08.265491 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 103541us
I20260709 13:10:08.265940 28106 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:08.266614 28106 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:08.267411 22994 leader_election.cc:400] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14.
I20260709 13:10:08.267716 22994 leader_election.cc:403] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:10:08.269797 23028 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33420 (request call id 2249) took 1311 ms. Trace:
I20260709 13:10:08.270249 23028 rpcz_store.cc:276] 0709 13:10:06.958159 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:07.000769 (+ 42610us) service_pool.cc:224] Handling call
0709 13:10:08.269568 (+1268799us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b540024a254. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:08.269736 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2358376704}
I20260709 13:10:08.273555 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:08.277163 27356 consensus_queue.cc:1048] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:10:08.278626 27356 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:10:08.283326 28274 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:08.284991 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:08.289110 22993 leader_election.cc:400] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14.
I20260709 13:10:08.289440 22993 leader_election.cc:403] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:10:08.290117 22993 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:08.291004 28277 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:08.294173 22953 raft_consensus.cc:1240] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:10:08.280377 22875 raft_consensus.cc:1275] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 13:10:08.296015 22875 pending_rounds.cc:85] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 11
I20260709 13:10:08.296301 22875 pending_rounds.cc:107] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 13:10:08.296562 22875 raft_consensus.cc:2889] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:08.288554 28245 consensus_queue.cc:1059] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.009s
I20260709 13:10:08.297865 27912 raft_consensus.cc:2804] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:08.298316 27912 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:08.298702 27912 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:08.251935 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:08.301295 23023 raft_consensus.cc:2393] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 16: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:08.221947 27911 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:08.281783 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:08.237036 28233 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:08.303289 22873 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 15.
W20260709 13:10:08.307089 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 54323.2us
I20260709 13:10:08.308542 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:10:08.305609 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 155482us
I20260709 13:10:08.316041 22876 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:08.324174 23622 raft_consensus.cc:2393] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 16: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:08.315588 22952 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
I20260709 13:10:08.327119 22844 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:08.335707 28228 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:08.346257 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:08.351712 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:08.352231 27678 raft_consensus.cc:2804] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader election won for term 13
W20260709 13:10:08.373520 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:08.371109 22843 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:08.375242 27768 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:08.380815 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:08.381414 22872 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:08.386008 28194 consensus_queue.cc:1059] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:10:08.361701 28291 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:08.386970 28143 raft_consensus.cc:3055] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:10:08.387467 28143 raft_consensus.cc:740] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:08.387409 28291 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:08.388705 28143 consensus_queue.cc:260] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:08.389680 28291 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:08.361070 28106 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:08.348932 28233 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:08.397763 28233 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:08.373134 28192 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:08.402678 28106 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:08.403048 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:08.403345 28106 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:08.404968 23026 raft_consensus.cc:1275] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:10:08.405821 28106 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:08.406050 27911 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:08.406499 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:10:08.406819 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:08.407670 22948 consensus_meta.cc:287] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.690s	user 0.009s	sys 0.009s
I20260709 13:10:08.408149 22948 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15.
I20260709 13:10:08.409508 28051 raft_consensus.cc:2749] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 13:10:08.409634 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 46107.3us
I20260709 13:10:08.412062 28062 consensus_queue.cc:1048] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:10:08.412732 28062 consensus_queue.cc:1243] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer aed3f089ec8b49af9c8cf5cdee76e8e7 log is divergent from this leader: its last log entry 13.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 13:10:08.415061 27678 raft_consensus.cc:697] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 13 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:08.417721 27678 consensus_queue.cc:237] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:08.418166 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:08.418864 23622 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 17.
I20260709 13:10:08.419325 28058 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:08.419831 28058 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:08.420569 22919 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:08.421772 27960 raft_consensus.cc:2804] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:08.422062 28058 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:08.422207 28143 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:08.424820 28078 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:08.425436 28078 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:08.426796 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:08.427577 23021 raft_consensus.cc:2393] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 16: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:08.427616 28078 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:08.430598 28280 raft_consensus.cc:1064] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
I20260709 13:10:08.390146 28249 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:08.431757 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:08.348519 28228 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:08.432267 23621 raft_consensus.cc:2368] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 16. Current term is 17.
I20260709 13:10:08.434024 28228 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:08.435093 23023 raft_consensus.cc:2410] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 17 because replica has last-logged OpId of term: 15 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 13.
I20260709 13:10:08.440866 28253 consensus_queue.cc:1048] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:10:08.441670 28305 log.cc:927] Time spent T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:10:08.444197 22918 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:08.448128 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:08.451189 22876 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 17.
I20260709 13:10:08.380358 27356 raft_consensus.cc:2749] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14.
I20260709 13:10:08.478132 22844 leader_election.cc:400] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 16. Current term is 17.
I20260709 13:10:08.478542 22844 leader_election.cc:403] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:10:08.422209 27960 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:08.485001 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:08.487937 22992 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:08.488646 27911 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:08.490814 23588 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:08.492457 28058 raft_consensus.cc:3055] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:10:08.496184 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:08.497179 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:08.497776 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:08.501510 27911 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:08.502513 22949 raft_consensus.cc:1275] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 13. (term mismatch)
I20260709 13:10:08.502701 28179 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:10:08.492193 28291 consensus_queue.cc:1059] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:10:08.508126 22873 raft_consensus.cc:1275] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:10:08.510409 22872 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 16.
I20260709 13:10:08.477823 28277 raft_consensus.cc:2749] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14.
I20260709 13:10:08.510311 22949 pending_rounds.cc:85] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting all ops after (but not including) 12
I20260709 13:10:08.514467 22949 pending_rounds.cc:107] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting uncommitted NO_OP operation due to leader change: 15.13
I20260709 13:10:08.514817 22949 raft_consensus.cc:2889] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:08.514942 23590 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:08.426780 27912 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:08.498898 27960 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:08.519431 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:10:08.518671 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 183087us
I20260709 13:10:08.520790 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:08.505088 28058 raft_consensus.cc:740] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
W20260709 13:10:08.538932 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 51164.9us
I20260709 13:10:08.539824 28062 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:08.541414 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:08.542337 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:08.550766 22948 raft_consensus.cc:2410] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 13:10:08.509255 28106 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:10:08.546113 22920 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:10:08.556538 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 144370us
I20260709 13:10:08.559564 22992 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:08.543742 28285 raft_consensus.cc:2749] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:08.534530 28249 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:08.513806 23028 raft_consensus.cc:2955] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Committing config change with OpId 12.9: config changed from index 5 to 9, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:08.572283 22951 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 15.
I20260709 13:10:08.577906 22920 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:08.557946 28106 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:08.565032 27625 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:10:08.593654 28245 consensus_queue.cc:1048] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.006s
I20260709 13:10:08.586911 28078 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:08.598954 27625 raft_consensus.cc:697] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:08.599242 28078 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:08.599696 28078 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:08.600018 27625 consensus_queue.cc:237] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:08.591602 28106 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:08.605047 28089 raft_consensus.cc:2804] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:10:08.606297 22918 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:08.608769 27768 raft_consensus.cc:2804] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:10:08.609359 27768 raft_consensus.cc:697] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:08.610293 27768 consensus_queue.cc:237] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:08.616811 28311 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:08.617036 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 126374us
I20260709 13:10:08.617563 28311 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:08.618191 22992 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:08.619899 28311 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:08.620957 27943 raft_consensus.cc:2749] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:10:08.622962 28304 log.cc:927] Time spent T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.197s	user 0.002s	sys 0.000s
I20260709 13:10:08.622859 28134 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:08.624984 22874 raft_consensus.cc:2393] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 16: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:08.625038 28134 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:08.627254 27821 consensus_queue.cc:1048] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:10:08.631654 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:08.606329 28058 consensus_queue.cc:260] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:08.629590 27912 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:08.629874 28299 log.cc:927] Time spent T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 13:10:08.631358 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:08.556087 28062 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:08.574422 28194 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:08.628278 27911 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:08.579878 28228 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:08.647261 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:08.647261 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:08.647480 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:08.652235 28280 consensus_queue.cc:237] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 7, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:08.688349 27975 consensus_meta.cc:287] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.515s	user 0.007s	sys 0.002s
I20260709 13:10:08.688817 27975 raft_consensus.cc:2749] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:08.695055 22948 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 14.
W20260709 13:10:08.695549 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 116845us
I20260709 13:10:08.652514 28277 raft_consensus.cc:2804] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:08.698055 28277 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:08.698520 28277 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:10:08.699039 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:08.700219 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:08.701315 23621 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:08.705125 28153 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:08.705788 28153 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:08.708890 28074 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:08.709525 28074 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:08.708148 28153 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:08.690371 28078 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:08.711127 22774 catalog_manager.cc:5697] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 10 to 11, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 11 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:08.712528 28078 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:08.714732 22774 catalog_manager.cc:5697] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 10 to 13, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 13 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:08.717021 23620 raft_consensus.cc:2468] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 13.
I20260709 13:10:08.723767 28315 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:08.724885 27911 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:08.724637 28315 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:08.725549 27911 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:08.727180 28315 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:08.727795 28315 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:08.728114 27911 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:08.730259 28062 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:08.731159 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:08.731971 23622 raft_consensus.cc:2393] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 17: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:08.734004 27821 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:08.734781 27821 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:08.737668 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 163691us
I20260709 13:10:08.740842 23588 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:08.646979 27979 raft_consensus.cc:2749] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:08.646198 28051 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:08.646409 28091 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:10:08.744071 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 95873.5us
I20260709 13:10:08.716490 22774 catalog_manager.cc:5697] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 8 to 12. New cstate: current_term: 12 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:08.751504 22872 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11.
I20260709 13:10:08.756254 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:08.758714 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:08.760567 23026 raft_consensus.cc:1275] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:10:08.762620 23588 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:08.763773 22946 raft_consensus.cc:2393] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 17: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:08.764241 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:08.728446 28315 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:08.772954 22918 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:08.778517 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:08.694985 28194 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:08.790948 22843 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:08.796428 22954 raft_consensus.cc:1240] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 15. Current term is 16. Ops: [12.9-15.10]
I20260709 13:10:08.793869 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:08.798506 28099 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:08.798389 22774 catalog_manager.cc:5697] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 7 to 12. New cstate: current_term: 12 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:08.799274 28099 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:08.800786 28054 raft_consensus.cc:2749] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:10:08.802502 23624 raft_consensus.cc:1275] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:10:08.802801 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:08.803715 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:08.804528 23616 raft_consensus.cc:2393] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 16: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:08.806327 27356 raft_consensus.cc:3055] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:10:08.806744 27356 raft_consensus.cc:740] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:08.807747 27356 consensus_queue.cc:260] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:08.808341 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:08.809125 22872 raft_consensus.cc:2393] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 13: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:08.809576 23620 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 1063) took 1207 ms. Trace:
W20260709 13:10:08.810101 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 52731.9us
I20260709 13:10:08.770743 28249 raft_consensus.cc:2749] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:08.798384 23029 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 16.
I20260709 13:10:08.799806 28179 raft_consensus.cc:2749] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 16. Current term is 17.
I20260709 13:10:08.809964 23620 rpcz_store.cc:276] 0709 13:10:07.601976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:07.602087 (+   111us) service_pool.cc:224] Handling call
0709 13:10:08.809408 (+1207321us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b5400469454. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:08.809516 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2518315904}
I20260709 13:10:08.824491 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:10:08.825146 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 46572.8us
I20260709 13:10:08.822098 22920 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:08.827118 28311 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:08.827783 28311 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:08.830349 28311 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:08.831760 22773 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33466 (request call id 292) took 20800 ms (client timeout 15000 ms). Trace:
W20260709 13:10:08.832459 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:08.833042 22948 raft_consensus.cc:1275] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:10:08.833859 28078 raft_consensus.cc:2804] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:10:08.833922 22920 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:08.834451 28078 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:08.834890 28078 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:08.811079 28293 consensus_queue.cc:1048] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 13:10:08.837173 22774 catalog_manager.cc:5697] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 5 to 11, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 11 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:08.839872 22918 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:08.840243 27356 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:08.844730 27979 raft_consensus.cc:2749] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:08.845510 22879 raft_consensus.cc:1240] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:10:08.846789 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:08.847019 27979 raft_consensus.cc:2749] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:10:08.847904 27979 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:08.848075 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:08.826534 22843 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:10:08.854915 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:08.839769 22774 catalog_manager.cc:5697] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 8 to 9, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 9 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:08.775218 28091 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:08.862192 28315 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:08.863796 28091 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:08.864899 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:08.865830 22872 raft_consensus.cc:2410] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:10:08.866240 22873 raft_consensus.cc:2393] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 16: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
W20260709 13:10:08.874764 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 48370.4us
I20260709 13:10:08.773212 28228 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:08.881248 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:08.893716 23029 raft_consensus.cc:1240] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 15. Current term is 16. Ops: []
I20260709 13:10:08.776490 28051 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:08.904109 22876 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 17.
W20260709 13:10:08.909724 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 106515us
I20260709 13:10:08.914494 22951 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:08.915984 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:08.925155 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:08.926671 23621 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 16.
I20260709 13:10:08.880900 28091 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:08.881629 28134 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:10:08.893499 28089 raft_consensus.cc:697] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:08.905795 27911 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:08.936087 22878 raft_consensus.cc:2393] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 16: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:08.907080 28315 raft_consensus.cc:2804] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:10:08.876892 28274 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:08.931344 22948 pending_rounds.cc:85] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting all ops after (but not including) 7
I20260709 13:10:08.933964 22871 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 15.
I20260709 13:10:08.945766 22948 pending_rounds.cc:107] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 13:10:08.934661 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:08.946075 22948 raft_consensus.cc:2889] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:08.937422 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:08.952102 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:08.955194 28315 raft_consensus.cc:697] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 15 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:08.955734 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:08.956384 23021 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 14.
I20260709 13:10:08.956326 28315 consensus_queue.cc:237] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:08.958086 22872 raft_consensus.cc:2393] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 15: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:08.960322 28091 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:08.926107 27821 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:08.960608 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:08.971668 28325 consensus_queue.cc:1048] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:10:08.972221 23020 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:08.972823 28274 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:08.975247 28274 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:08.976217 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:08.944998 28134 raft_consensus.cc:697] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 17 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:08.965144 23024 raft_consensus.cc:1275] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 13:10:08.977809 28134 consensus_queue.cc:237] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:08.965183 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:08.978698 27960 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:08.979563 22947 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 16.
I20260709 13:10:08.980449 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:08.981007 22947 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 16.
I20260709 13:10:08.978199 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:08.965940 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:08.984479 23027 raft_consensus.cc:2410] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
W20260709 13:10:08.971365 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 60874.7us
I20260709 13:10:08.985051 23029 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 14.
I20260709 13:10:08.987847 22955 raft_consensus.cc:2393] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 18: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:08.970654 22774 catalog_manager.cc:5697] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 10 to 12, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 12 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:08.991628 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:08.994612 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 154873us
I20260709 13:10:08.998508 22918 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:08.999498 28330 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:09.000519 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:08.943960 28074 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:09.004688 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:09.004925 22918 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:09.005391 23026 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 14.
I20260709 13:10:09.010317 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:09.011869 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:09.013175 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:09.014230 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:08.970815 27911 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:09.015986 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:09.018260 23026 raft_consensus.cc:2393] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 12: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
W20260709 13:10:09.022926 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 80690.6us
I20260709 13:10:09.024067 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:09.025960 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:09.026108 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:09.027197 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:09.027281 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:09.028124 23024 raft_consensus.cc:2393] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 16: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:09.028976 22774 catalog_manager.cc:5697] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 9 to 10, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 10 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:09.030824 22774 catalog_manager.cc:5697] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 9 to 12, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 12 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:09.032181 22950 raft_consensus.cc:1275] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 13:10:09.028427 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:09.032382 22774 catalog_manager.cc:5697] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 8 to 12, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 12 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:09.034766 23027 raft_consensus.cc:2393] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 16: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:09.037058 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:09.037806 23020 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 12.
I20260709 13:10:08.991055 28179 consensus_queue.cc:1048] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 13:10:09.038261 28277 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:09.040805 23020 raft_consensus.cc:1275] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 13:10:09.041146 23020 pending_rounds.cc:85] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75: Aborting all ops after (but not including) 7
I20260709 13:10:09.041241 28326 raft_consensus.cc:2749] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:09.041519 23020 pending_rounds.cc:107] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 13:10:09.041826 23020 raft_consensus.cc:2889] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:09.042219 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:09.043694 28324 log.cc:927] Time spent T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 13:10:09.045640 23029 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 16.
I20260709 13:10:09.054314 27911 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:09.054936 23024 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:09.040427 28194 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:09.068099 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:09.042281 28277 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:09.035488 22774 catalog_manager.cc:5697] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 6 to 7, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 7 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
W20260709 13:10:09.074326 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:09.074563 22950 pending_rounds.cc:85] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting all ops after (but not including) 6
I20260709 13:10:09.082530 22950 pending_rounds.cc:107] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 13:10:09.082815 22950 raft_consensus.cc:2889] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:09.016616 27960 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:09.085558 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 85676.9us
I20260709 13:10:09.033778 28315 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:09.030821 28051 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:08.971350 28089 consensus_queue.cc:237] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:09.110355 23027 raft_consensus.cc:1275] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:10:09.105036 22951 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 16.
I20260709 13:10:09.056774 28099 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:09.115025 22994 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:09.127830 23024 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15.
I20260709 13:10:09.131621 23023 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15.
I20260709 13:10:09.137033 22918 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:09.143646 28277 raft_consensus.cc:2749] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:09.147470 28078 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:09.148592 23025 raft_consensus.cc:2410] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 13:10:09.151275 23621 raft_consensus.cc:1275] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 LEARNER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:10:09.152331 28333 consensus_queue.cc:1048] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.012s
W20260709 13:10:08.832319 22773 rpcz_store.cc:269] 0709 13:09:48.031628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:48.055869 (+ 24241us) service_pool.cc:224] Handling call
0709 13:10:08.831445 (+20775576us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:10:01.496992 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:10:01.497763 (+   771us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:10:01.497837 (+    74us) write_op.cc:435] Acquired schema lock
0709 13:10:01.497871 (+    34us) tablet.cc:662] Decoding operations
0709 13:10:01.499902 (+  2031us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:10:01.500004 (+   102us) write_op.cc:641] Partition lock acquired for write op
0709 13:10:01.500035 (+    31us) tablet.cc:685] Acquiring locks for 21 operations
0709 13:10:01.501829 (+  1794us) tablet.cc:701] Row locks acquired
0709 13:10:01.501878 (+    49us) write_op.cc:260] PREPARE: finished
0709 13:10:01.502069 (+   191us) write_op.cc:270] Start()
0709 13:10:01.502229 (+   160us) write_op.cc:276] Timestamp: P: 1783602601501998 usec, L: 0
0709 13:10:01.502266 (+    37us) op_driver.cc:348] REPLICATION: starting
0709 13:10:01.502920 (+   654us) log.cc:844] Serialized 8182 byte log entry
0709 13:10:02.539063 (+1036143us) spinlock_profiling.cc:243] Waited 1.81 ms on lock 0x7b1800146080. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cf83f3 00007f6132cfa7c1 00007f6132cfa779 00007f6132cfa709 00007f6132cfa6d1 00007f6132cf99cd 000000000038f354 000000000038f199 00007f6132cdb83a
0709 13:10:02.875802 (+336739us) op_driver.cc:464] REPLICATION: finished
0709 13:10:03.118606 (+242804us) write_op.cc:301] APPLY: starting
0709 13:10:03.118700 (+    94us) tablet.cc:1367] starting BulkCheckPresence
0709 13:10:04.319225 (+1200525us) tablet.cc:1370] finished BulkCheckPresence
0709 13:10:04.319280 (+    55us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:10:05.110035 (+790755us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:10:05.110603 (+   568us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=42,key_file_lookups=42,delta_file_lookups=42,mrs_lookups=0
0709 13:10:05.110675 (+    72us) write_op.cc:312] APPLY: finished
0709 13:10:05.128743 (+ 18068us) log.cc:844] Serialized 188 byte log entry
0709 13:10:05.246977 (+118234us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:05.295152 (+ 48175us) write_op.cc:454] Released schema lock
0709 13:10:05.311484 (+ 16332us) write_op.cc:341] FINISH: Updating metrics
0709 13:10:05.332865 (+ 21381us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:05.332911 (+    46us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":472670,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":158360,"apply.run_cpu_time_us":66597,"apply.run_wall_time_us":2210621,"cfile_cache_hit":86,"cfile_cache_hit_bytes":91949,"num_ops":21,"prepare.queue_time_us":64452,"prepare.run_cpu_time_us":27753,"prepare.run_wall_time_us":77001,"raft.queue_time_us":207305,"raft.run_cpu_time_us":10591,"raft.run_wall_time_us":151302,"replication_time_us":1351455,"spinlock_wait_cycles":4163840,"thread_start_us":378014,"threads_started":5,"wal-append.queue_time_us":260196,"wal-append.run_cpu_time_us":11280,"wal-append.run_wall_time_us":2261335}]]}
W20260709 13:10:09.180521 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:09.131409 28099 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:09.188349 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 53274.3us
I20260709 13:10:09.190026 22843 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:09.112622 28315 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:09.177448 27960 raft_consensus.cc:2749] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 13:10:09.208949 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 78377.4us
W20260709 13:10:09.217221 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 111442us
I20260709 13:10:09.218142 22994 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:09.204223 28343 raft_consensus.cc:2804] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:09.234843 28343 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:09.235342 28343 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:09.238651 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:09.239427 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:09.239928 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:09.240257 22880 raft_consensus.cc:2410] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 17 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 13:10:09.240895 22875 raft_consensus.cc:1240] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:10:09.241082 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:09.229077 27911 raft_consensus.cc:2749] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:10:09.243142 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:09.231477 28245 consensus_queue.cc:1048] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:10:09.193673 28099 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:09.245688 28099 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:09.196431 28319 consensus_queue.cc:1048] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.032s
W20260709 13:10:09.217432 28106 consensus_meta.cc:287] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.612s	user 0.010s	sys 0.002s
I20260709 13:10:09.222033 28315 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:09.247303 22918 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:09.247450 28106 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:09.249076 28326 consensus_queue.cc:1048] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:10:09.249486 22918 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:09.249894 22953 raft_consensus.cc:1240] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 13. Current term is 14. Ops: []
I20260709 13:10:09.250154 28228 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:09.250535 28326 raft_consensus.cc:2749] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:10:09.240522 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:09.252497 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:09.255149 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:09.258533 22948 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:09.256906 22994 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:09.264600 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:09.264879 28154 raft_consensus.cc:2749] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:09.266080 22994 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:09.267977 27356 raft_consensus.cc:2749] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:10:09.268262 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:09.269891 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:09.270433 23623 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:09.270651 22994 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:09.271894 22994 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:09.272042 28277 consensus_queue.cc:1059] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
W20260709 13:10:09.272877 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 83044.8us
I20260709 13:10:09.273031 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:09.273274 28291 raft_consensus.cc:2749] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:09.273972 23616 raft_consensus.cc:2393] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 16: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:09.274044 28291 raft_consensus.cc:3055] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:10:09.274479 28291 raft_consensus.cc:740] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:09.275355 28291 consensus_queue.cc:260] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:09.276247 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:09.249540 28106 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:09.284816 23590 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:09.292860 28344 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:09.293502 28344 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:09.276507 28291 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:09.304914 27975 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:09.305532 27975 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:09.306207 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:09.307474 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:09.307642 27975 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:09.307758 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:09.308079 22872 raft_consensus.cc:2468] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 13.
I20260709 13:10:09.308162 27975 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:09.272934 28277 raft_consensus.cc:2749] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:09.296923 23623 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18.
I20260709 13:10:09.301592 28344 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:09.310583 23590 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:09.310861 22992 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:09.311834 22994 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:09.323480 22992 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:09.329986 22875 raft_consensus.cc:1275] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:10:09.329216 22919 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:09.332087 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:09.336089 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:09.337114 23588 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:09.315603 27975 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:09.337963 28277 raft_consensus.cc:2749] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:10:09.338449 27821 raft_consensus.cc:2764] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:10:09.338680 28194 raft_consensus.cc:2804] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:10:09.339133 28194 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:09.339483 28194 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:09.319830 28054 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:10:09.340775 28054 raft_consensus.cc:697] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:09.341869 27975 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:09.341720 28054 consensus_queue.cc:237] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:09.345242 28058 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:09.328545 28330 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:09.329222 28292 raft_consensus.cc:2749] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:10:09.329124 28078 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:09.349026 22948 raft_consensus.cc:1240] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 LEADER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:10:09.349138 28106 raft_consensus.cc:2804] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:10:09.351512 28106 raft_consensus.cc:697] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:09.349226 28330 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:09.352799 28330 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:09.352507 28106 consensus_queue.cc:237] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:09.353183 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:09.356060 22946 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 16.
W20260709 13:10:09.356132 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:09.365756 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:09.367169 22947 raft_consensus.cc:2393] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 15: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:09.375808 23028 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 901) took 1050 ms. Trace:
I20260709 13:10:09.376284 23028 rpcz_store.cc:276] 0709 13:10:08.324955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:08.330229 (+  5274us) service_pool.cc:224] Handling call
0709 13:10:08.368841 (+ 38612us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:08.477398 (+108557us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:10:08.647454 (+170056us) log.cc:844] Serialized 14 byte log entry
0709 13:10:09.256499 (+609045us) spinlock_profiling.cc:243] Waited 2.18 ms on lock 0x7b1800410fe0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cdae07 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf5455 00007f61397dd3c7 00007f6135def9a7 00007f6135dff0bf 00007f6135dff038
0709 13:10:09.374021 (+117522us) log.cc:844] Serialized 14 byte log entry
0709 13:10:09.375422 (+  1401us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:09.375480 (+    58us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:10:09.375527 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:09.375597 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:09.375718 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":53672,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":13123,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":5690,"spinlock_wait_cycles":5006592,"thread_start_us":299605,"threads_started":2,"wal-append.queue_time_us":307121}
I20260709 13:10:09.328876 27912 raft_consensus.cc:2804] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:09.407299 23623 raft_consensus.cc:1275] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:10:09.401754 27912 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:10:09.413520 28309 log.cc:927] Time spent T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 1.018s	user 0.007s	sys 0.000s
I20260709 13:10:09.422087 22843 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:09.424993 28280 consensus_queue.cc:1059] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:10:09.368273 28099 consensus_queue.cc:1048] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 13:10:09.429199 28343 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:09.431609 28091 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:09.434203 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:09.434474 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:09.436565 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:10:09.447515 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 135771us
I20260709 13:10:09.448915 22994 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:09.451247 23616 raft_consensus.cc:1240] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEADER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16. Ops: []
I20260709 13:10:09.447181 28089 consensus_queue.cc:1048] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 13:10:09.441372 27912 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:09.462733 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:09.464437 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:09.465991 22994 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:09.467792 27768 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:09.475162 27356 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:09.478636 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:09.479229 23024 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 16.
I20260709 13:10:09.479277 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:09.482797 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:09.474803 22918 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:09.483462 23029 raft_consensus.cc:2376] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 12. Re-sending same reply.
I20260709 13:10:09.483677 27356 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:09.485088 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:09.485257 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:09.485441 23028 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 15.
I20260709 13:10:09.486140 23027 raft_consensus.cc:2410] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 13:10:09.486253 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:09.487001 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:09.487398 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:09.487808 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:09.487947 23023 raft_consensus.cc:2376] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 14. Re-sending same reply.
I20260709 13:10:09.489504 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:09.490248 23023 raft_consensus.cc:2410] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:10:09.490661 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:09.491624 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:09.492627 23023 raft_consensus.cc:2393] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 16: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:09.492810 27807 raft_consensus.cc:2749] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:09.494367 23023 raft_consensus.cc:1240] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 14. Current term is 15. Ops: []
I20260709 13:10:09.496279 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:09.496991 23023 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 15.
I20260709 13:10:09.498631 22920 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:09.498848 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:09.499514 23023 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:10:09.499441 28333 consensus_queue.cc:1048] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:10:09.501295 28091 raft_consensus.cc:2804] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:09.501730 28091 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:09.502080 28091 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:09.503312 28343 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:09.477471 27768 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:09.504932 28344 raft_consensus.cc:3055] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:10:09.505352 28344 raft_consensus.cc:740] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:09.506235 28344 consensus_queue.cc:260] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:09.508355 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:09.509011 23024 raft_consensus.cc:2468] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 12.
I20260709 13:10:09.499456 28277 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:09.511597 28277 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:09.513311 23026 raft_consensus.cc:2410] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:10:09.471457 27911 raft_consensus.cc:2749] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:09.474896 28274 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:09.517066 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:09.517906 23029 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11.
I20260709 13:10:09.477648 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:09.474050 22993 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:09.521252 23020 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 13.
W20260709 13:10:09.521469 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 47538us
I20260709 13:10:09.522181 22993 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:09.525516 22875 raft_consensus.cc:2393] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 16: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:09.526813 28089 raft_consensus.cc:2804] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:10:09.527374 28089 raft_consensus.cc:697] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:09.528998 28291 raft_consensus.cc:2804] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:10:09.529472 28291 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:09.529907 28291 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:09.531293 28359 raft_consensus.cc:2804] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:10:09.531838 28359 raft_consensus.cc:697] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 16 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:09.535331 28311 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:09.535898 28311 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:09.507462 28344 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:09.554224 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:09.585036 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 163055us
I20260709 13:10:09.586603 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:09.587932 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:09.588032 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:09.588629 22878 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:09.588913 23029 raft_consensus.cc:1240] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:10:09.588195 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:09.589706 23023 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18.
I20260709 13:10:09.589901 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:09.590514 23028 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:09.590574 28194 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:09.595258 27356 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:09.515873 28274 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:09.608601 23021 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 15.
I20260709 13:10:09.609043 23020 raft_consensus.cc:1275] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:10:09.597820 23024 raft_consensus.cc:2393] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 14: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
W20260709 13:10:09.599568 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 127508us
I20260709 13:10:09.604920 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:10:09.611698 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 286544us
I20260709 13:10:09.612917 22878 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 12.
I20260709 13:10:09.613547 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:09.614214 22876 raft_consensus.cc:1240] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:09.617046 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:09.615234 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:09.625267 22920 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:09.627738 22843 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:09.528215 28089 consensus_queue.cc:237] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:09.632751 23028 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 12.
I20260709 13:10:09.638288 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:09.639101 22992 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:09.640218 22992 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:09.641445 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:09.641940 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:09.642637 22879 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:09.635147 28194 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:09.647814 28194 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:09.647995 23025 raft_consensus.cc:1275] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
I20260709 13:10:09.648319 28194 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:09.650632 28194 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:09.648330 23025 pending_rounds.cc:85] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75: Aborting all ops after (but not including) 10
I20260709 13:10:09.652021 23025 pending_rounds.cc:107] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 13:10:09.652298 23025 raft_consensus.cc:2889] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:09.652910 27911 raft_consensus.cc:2749] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:09.656505 22843 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:09.657265 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:09.657871 28274 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:09.658154 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:09.658860 23619 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 18.
I20260709 13:10:09.659650 28335 consensus_queue.cc:1059] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 13:10:09.656598 27807 raft_consensus.cc:2749] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:10:09.667024 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:09.663057 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:09.641294 27960 raft_consensus.cc:2804] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:10:09.657908 22843 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:09.666419 22992 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:09.689383 23625 raft_consensus.cc:1275] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:10:09.661815 28326 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:09.641501 28277 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:09.697356 22992 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:09.630705 28228 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:09.665230 28343 raft_consensus.cc:2749] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:10:09.663369 28292 raft_consensus.cc:2804] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:10:09.694437 22948 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 18.
W20260709 13:10:09.716898 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 59090.3us
I20260709 13:10:09.717489 28274 raft_consensus.cc:2749] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:09.701325 28134 raft_consensus.cc:2749] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:09.722385 27768 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:10:09.725625 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 120787us
I20260709 13:10:09.727799 22992 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:09.734938 22879 raft_consensus.cc:2468] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 14.
I20260709 13:10:09.739223 22843 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:09.693313 27960 raft_consensus.cc:697] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:09.743139 22843 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:09.751354 22994 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:09.752243 27356 raft_consensus.cc:2749] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:10:09.753684 22994 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:10:09.765105 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:09.741254 28292 raft_consensus.cc:697] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 16 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:09.783535 27356 raft_consensus.cc:2749] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:09.788968 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:09.790170 22843 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:09.787519 28291 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:09.746431 28099 raft_consensus.cc:2749] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:09.725890 28326 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:09.750218 28343 raft_consensus.cc:2749] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:09.750459 28154 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:09.797246 28154 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:09.797417 28343 raft_consensus.cc:2749] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:09.797632 22843 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:09.797750 28154 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:09.737816 28359 consensus_queue.cc:237] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:09.799103 22843 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:09.801056 28343 raft_consensus.cc:2749] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:09.802276 22843 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:09.726411 28228 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:09.803520 22843 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:09.804931 28062 raft_consensus.cc:2749] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:10:09.805127 28343 raft_consensus.cc:2749] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:09.800988 28134 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:10:09.804996 22843 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:09.807158 28134 raft_consensus.cc:2804] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:09.807466 28343 consensus_queue.cc:1048] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:10:09.807713 28134 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:09.808931 28134 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:09.810572 22843 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:09.811123 28311 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:09.812038 28343 raft_consensus.cc:2804] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:09.812451 28343 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:09.812798 28343 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:09.805784 28062 consensus_queue.cc:1048] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 13:10:09.815212 28361 log.cc:927] Time spent T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.167s	user 0.000s	sys 0.002s
I20260709 13:10:09.817826 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:09.818945 22876 raft_consensus.cc:2393] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 14: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:09.821285 28292 consensus_queue.cc:237] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:09.822995 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:09.777145 28078 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:09.824399 28078 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:09.826861 28078 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:09.724437 28330 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:09.829365 28330 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:09.778612 28106 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:09.830399 28106 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:09.833173 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:09.833990 22951 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 16.
I20260709 13:10:09.836797 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:09.837819 22951 raft_consensus.cc:2410] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:10:09.821298 22876 raft_consensus.cc:1275] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:10:09.826658 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:09.843437 22952 raft_consensus.cc:1275] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:10:09.845507 22952 raft_consensus.cc:1275] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 13:10:09.849589 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:09.851593 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:09.855304 22843 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:09.861855 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:09.862416 28326 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:09.862602 28343 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:09.863591 28058 consensus_meta.cc:287] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.518s	user 0.017s	sys 0.006s
I20260709 13:10:09.864008 22951 raft_consensus.cc:1275] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:10:09.864181 23620 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 16.
I20260709 13:10:09.788621 27960 consensus_queue.cc:237] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:09.869140 28291 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:09.871394 28134 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:09.872206 28343 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:09.890574 22843 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:09.894745 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:09.895785 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:09.804971 28228 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:09.893512 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:09.835204 27912 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:09.903314 22876 raft_consensus.cc:2393] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 17: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:09.880330 27975 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:09.895383 28343 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:10:09.915139 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 286380us
I20260709 13:10:09.920552 22918 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:09.901345 28191 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:09.929834 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:09.930006 22843 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:09.930773 28099 consensus_queue.cc:1048] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:10:09.939428 28153 raft_consensus.cc:3055] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:10:09.939973 28153 raft_consensus.cc:740] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:09.961659 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:09.962505 22879 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 17.
W20260709 13:10:09.964339 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 337509us
I20260709 13:10:09.895056 28330 consensus_queue.cc:1048] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 13:10:09.908175 28106 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:09.970713 27356 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:09.972914 27356 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:09.977552 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 46160.6us
I20260709 13:10:09.926524 28051 raft_consensus.cc:2749] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:09.989794 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:09.990446 23029 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:09.990501 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:09.989821 22918 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:09.942032 28335 raft_consensus.cc:2804] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:10:10.016170 27768 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:10.017009 27768 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:10.017089 28153 consensus_queue.cc:260] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:10.017763 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:10.018340 22951 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 18.
I20260709 13:10:09.927201 27975 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:10:09.989140 28280 raft_consensus.cc:2749] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:10.018404 28153 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:10.053299 23021 raft_consensus.cc:1275] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:10:09.997642 28332 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:09.996045 28319 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:10.060271 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:10.062667 22949 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 16.
I20260709 13:10:09.959791 28191 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:10.020593 28106 raft_consensus.cc:2749] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:10:10.076560 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 101757us
I20260709 13:10:10.016153 28154 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:10.077943 22918 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:10.085702 23029 raft_consensus.cc:2468] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 14.
I20260709 13:10:10.024572 28335 raft_consensus.cc:697] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
W20260709 13:10:10.105304 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 349607us
I20260709 13:10:10.110977 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:10.111847 22873 raft_consensus.cc:2368] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14.
I20260709 13:10:10.116485 23026 raft_consensus.cc:2393] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 19: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:10.119946 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:10.119643 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:10.124831 22871 raft_consensus.cc:2393] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 16: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:10.124042 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:10.020744 28078 consensus_queue.cc:1175] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f3c799b1249e40119d2a5864231c6c75" responder_term: 16 status { last_received { term: 16 index: 8 } last_committed_idx: 8 last_received_current_leader { term: 16 index: 8 } }
I20260709 13:10:10.069458 28332 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:10.142091 23619 raft_consensus.cc:1275] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 13:10:10.143703 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:10.143592 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:10.144608 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:10.145457 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:10.148336 23028 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15.
I20260709 13:10:10.149937 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:10.147897 23029 raft_consensus.cc:2410] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:10:10.152288 22994 leader_election.cc:400] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14.
I20260709 13:10:10.156245 22994 leader_election.cc:403] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:10:10.114064 28373 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 13:10:10.154069 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 76176us
I20260709 13:10:10.156402 23588 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:10.162871 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:10.163798 23023 raft_consensus.cc:2368] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 11. Current term is 12.
I20260709 13:10:10.164384 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:10.158260 22918 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:10.100816 28319 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:10.153728 28091 consensus_meta.cc:287] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.651s	user 0.011s	sys 0.001s
I20260709 13:10:10.181020 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:10.188845 27356 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:10.140368 28362 raft_consensus.cc:2749] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:10.146075 28134 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:10.205124 23026 raft_consensus.cc:1275] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 13:10:10.214236 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:10.207198 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:10.221769 23028 raft_consensus.cc:2410] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:10:10.171712 27912 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:10.229418 23619 raft_consensus.cc:2368] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14.
I20260709 13:10:10.230340 23588 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:10.192342 27975 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:10.180481 27625 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:10.211509 28364 raft_consensus.cc:2749] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14.
I20260709 13:10:10.262915 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:10.261080 23624 raft_consensus.cc:1275] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:10:10.264586 27356 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:10.265293 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:10:10.265877 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 107695us
I20260709 13:10:10.213124 28373 raft_consensus.cc:2749] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:10.210570 28370 raft_consensus.cc:2749] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:10.265990 23620 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 14.
I20260709 13:10:10.274132 27356 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:10.281572 27356 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:10.285290 28191 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:10.285557 28365 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:10.286342 28365 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:10.262691 28351 log.cc:927] Time spent T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 13:10:10.300959 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:10.290838 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:10.204809 28091 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:10.306742 22948 raft_consensus.cc:2393] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 16: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
W20260709 13:10:10.308161 28366 log.cc:927] Time spent T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:10:10.310137 28319 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:10.311342 23027 raft_consensus.cc:1240] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:10:10.272308 28134 consensus_queue.cc:1048] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:10:10.321473 23026 raft_consensus.cc:1275] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:10:10.263747 28333 raft_consensus.cc:2749] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:10.302366 23619 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 17.
I20260709 13:10:10.348089 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:10.348691 28311 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:10.349188 22949 raft_consensus.cc:2393] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 16: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:10.349450 28311 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:10.352206 28311 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:10.360389 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:10.355131 28332 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:10:10.353009 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 86360.5us
W20260709 13:10:10.399312 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 145256us
I20260709 13:10:10.399819 22918 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:10.255283 28335 consensus_queue.cc:237] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:10.383684 28372 log.cc:927] Time spent T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.166s	user 0.000s	sys 0.000s
I20260709 13:10:10.384312 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:10.384423 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:10.446087 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:10.447124 22952 raft_consensus.cc:2410] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 13:10:10.447388 22954 raft_consensus.cc:1275] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:10:10.387857 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:10.449127 22951 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:10.453198 28078 raft_consensus.cc:2749] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:10:10.452473 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 52759.4us
I20260709 13:10:10.455027 22918 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:10.456859 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 186285us
I20260709 13:10:10.264365 28058 consensus_queue.cc:1048] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.073s
I20260709 13:10:10.395767 22946 raft_consensus.cc:1275] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 13:10:10.462893 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 61339.7us
I20260709 13:10:10.375584 22875 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 14.
I20260709 13:10:10.396901 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:10.273175 27625 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:10.403841 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:10.408440 28319 consensus_queue.cc:1048] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:10:10.411998 28365 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:10.416342 28343 consensus_meta.cc:287] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.500s	user 0.008s	sys 0.001s
I20260709 13:10:10.417840 23588 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:10.420509 28154 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:10.476320 22874 raft_consensus.cc:1275] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:10:10.427235 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:10.477866 22877 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:10.478892 22951 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 19.
I20260709 13:10:10.435284 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:10.443650 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:10.459072 22918 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:10.460121 22871 raft_consensus.cc:1275] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 13:10:10.457692 23590 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:10.482245 22871 pending_rounds.cc:85] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 5
I20260709 13:10:10.482591 22871 pending_rounds.cc:107] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 9.6
W20260709 13:10:10.475397 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 57636.4us
I20260709 13:10:10.482776 23590 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:10.475235 22992 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:10.474023 22953 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 13.
I20260709 13:10:10.481711 22949 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 13.
I20260709 13:10:10.492391 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:10.482879 22871 raft_consensus.cc:2889] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:10.499022 22918 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:10.460410 28078 raft_consensus.cc:2749] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:10.505076 22994 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:10.507422 22994 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:10.480911 28343 raft_consensus.cc:2749] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 13:10:10.503762 28245 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.026s	sys 0.020s
I20260709 13:10:10.470736 28106 consensus_queue.cc:1059] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 13:10:10.500093 28243 raft_consensus.cc:2804] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:10:10.503121 28362 raft_consensus.cc:2749] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:10:10.500209 28194 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:10.505117 28373 raft_consensus.cc:2804] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:10:10.554158 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 56163us
W20260709 13:10:10.530958 28360 log.cc:927] Time spent T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 13:10:10.507026 28330 raft_consensus.cc:2804] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:10:10.548265 22879 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.007s	sys 0.024s
I20260709 13:10:10.548714 22878 raft_consensus.cc:1240] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 11. Current term is 12. Ops: []
I20260709 13:10:10.570343 28330 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:10.570551 28131 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:10.559638 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:10.570976 28330 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:10.571122 28131 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:10.571604 28131 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:10.565347 22876 raft_consensus.cc:1240] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:10.534876 28274 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:10.578586 28330 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:10.580114 27768 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:10.554273 23588 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:10.584478 28330 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:10.542388 28245 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.676s	user 0.026s	sys 0.020s
I20260709 13:10:10.538687 28359 raft_consensus.cc:2749] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:10.566664 28373 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:10.503173 28311 consensus_queue.cc:1048] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 13:10:10.597985 27356 consensus_queue.cc:1048] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:10:10.587463 22879 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.007s	sys 0.024s
W20260709 13:10:10.543982 28292 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.013s	sys 0.014s
I20260709 13:10:10.535601 28091 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:10.542667 27960 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.003s	sys 0.055s
I20260709 13:10:10.591133 23588 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:10:10.593822 22843 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.010s	sys 0.019s
W20260709 13:10:10.603816 22843 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.010s	sys 0.019s
W20260709 13:10:10.604110 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 533222us
I20260709 13:10:10.555799 28194 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:10.605087 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:10.542059 28365 consensus_queue.cc:1048] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:10:10.544341 27867 consensus_queue.cc:1048] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 13:10:10.591873 28243 raft_consensus.cc:697] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 14 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:10.612703 22843 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:10:10.604117 27960 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.737s	user 0.003s	sys 0.056s
I20260709 13:10:10.597807 28377 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:10.633540 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:10.635739 28089 raft_consensus.cc:3055] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:10:10.615907 28274 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:10.637982 28243 consensus_queue.cc:237] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:10.598434 28370 raft_consensus.cc:2804] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:10.598613 28373 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:10.641132 28370 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:10.641539 28370 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:10.616074 28106 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:10.642315 28106 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:10.649607 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:10.651033 28398 consensus_queue.cc:1048] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:10:10.651734 27625 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:10.651953 28099 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:10.654964 22875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35360 (request call id 2307) took 1081 ms. Trace:
I20260709 13:10:10.655488 22875 rpcz_store.cc:276] 0709 13:10:09.573106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:09.586250 (+ 13144us) service_pool.cc:224] Handling call
0709 13:10:10.654754 (+1068504us) spinlock_profiling.cc:243] Waited 371 ms on lock 0x7b54003bd654. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:10.654894 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":854048640}
I20260709 13:10:10.610072 28054 raft_consensus.cc:2804] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:10:10.609784 28194 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:10.636335 28089 raft_consensus.cc:740] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
W20260709 13:10:10.665912 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 163636us
I20260709 13:10:10.666769 22918 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:10.652376 28099 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:10.670701 22843 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:10.664323 22920 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:10.677685 22843 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:10.679513 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:10.681393 22949 raft_consensus.cc:1240] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 14. Current term is 15. Ops: []
W20260709 13:10:10.687994 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 107172us
I20260709 13:10:10.689086 22992 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:10.690227 28291 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:10:10.691061 28291 raft_consensus.cc:697] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 19 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:10.692381 28291 consensus_queue.cc:237] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:10.701499 28062 raft_consensus.cc:2749] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:10:10.707571 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 138983us
I20260709 13:10:10.664036 28054 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:10.673146 28319 consensus_queue.cc:1048] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 13:10:10.660030 27807 consensus_queue.cc:1048] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 13:10:10.628945 28377 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:10.710373 22918 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:10.714574 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:10.719383 22949 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
I20260709 13:10:10.708483 28335 raft_consensus.cc:2804] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:10:10.712929 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:10.723592 22948 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:10.708696 28054 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:10.670352 28315 consensus_queue.cc:1048] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 13:10:10.741613 22949 raft_consensus.cc:1240] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 12. Current term is 13. Ops: []
I20260709 13:10:10.742053 22877 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 16.
I20260709 13:10:10.674611 28099 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:10.749109 22920 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:10.752945 22993 leader_election.cc:400] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14.
I20260709 13:10:10.759102 22878 raft_consensus.cc:1275] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:10:10.769280 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:10.769982 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:10.771710 23624 raft_consensus.cc:1275] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:10:10.773141 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:10.724180 28330 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:10.797876 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:10.797514 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 119875us
I20260709 13:10:10.810153 22843 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:10.811465 28377 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:10.811568 22843 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:10.813138 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 102857us
I20260709 13:10:10.813489 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:10.814149 22918 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:10.815970 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:10.816615 23621 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 16.
I20260709 13:10:10.736683 28335 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:10.803479 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:10.737488 28389 raft_consensus.cc:2749] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:10.818702 22877 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 14.
I20260709 13:10:10.724599 27960 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:10:10.777685 22993 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:10.774066 22994 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:10:10.822392 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 48437.8us
I20260709 13:10:10.813058 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:10:10.822395 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 44824.4us
W20260709 13:10:10.818045 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 205702us
I20260709 13:10:10.774740 28091 raft_consensus.cc:2749] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:10:10.830585 22843 leader_election.cc:400] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 11. Current term is 12.
I20260709 13:10:10.830940 22843 leader_election.cc:403] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 13:10:10.818871 28062 raft_consensus.cc:2749] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:10:10.799331 28106 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:10.817505 28365 raft_consensus.cc:2804] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:10:10.821259 28335 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:10:10.876866 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 56712.1us
I20260709 13:10:10.879592 22948 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 17.
I20260709 13:10:10.765614 28153 consensus_queue.cc:1059] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.101s
W20260709 13:10:10.872030 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 48166.2us
W20260709 13:10:10.863523 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 114473us
W20260709 13:10:10.881687 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 56259.2us
W20260709 13:10:10.882404 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 63099.4us
I20260709 13:10:10.827811 28291 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:10:10.893303 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:10.871726 27975 raft_consensus.cc:2749] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:10.872489 28344 raft_consensus.cc:2749] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:10.902230 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:10.911567 23028 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:10:10.912938 28153 raft_consensus.cc:2749] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:10.913678 28153 consensus_queue.cc:1059] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 13:10:10.903021 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:10.798957 28089 consensus_queue.cc:260] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:10.862156 28330 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:10.882930 28359 consensus_queue.cc:1048] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 13:10:10.918073 28419 raft_consensus.cc:3055] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:10:10.918476 28330 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:10.924873 23028 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 17.
I20260709 13:10:10.890688 28365 raft_consensus.cc:697] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:10.909933 23590 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:10.933035 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:11.010300 22843 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:11.011519 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 62496.5us
I20260709 13:10:10.889215 28333 consensus_queue.cc:1048] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 13:10:10.918486 28419 raft_consensus.cc:740] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:10.899888 28364 consensus_queue.cc:1059] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.140s
I20260709 13:10:11.022755 28377 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:11.044041 22992 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:11.053073 22843 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:11.026818 28292 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:11.057742 22843 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:11.050804 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 235323us
I20260709 13:10:11.030721 27768 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:11.051071 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 140564us
W20260709 13:10:11.051250 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 299730us
I20260709 13:10:11.046828 28062 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:11.071056 22946 raft_consensus.cc:1275] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:10:11.064078 22918 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:11.080164 22919 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:11.058322 27960 raft_consensus.cc:697] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:11.091668 27356 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:11.098783 27356 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:11.093600 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:11.096848 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:11.049188 28401 raft_consensus.cc:2749] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:11.101315 23023 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 12.
I20260709 13:10:11.066684 27678 raft_consensus.cc:2749] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 11. Current term is 12.
I20260709 13:10:11.065301 28315 raft_consensus.cc:2804] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:10:11.034444 28417 consensus_queue.cc:1048] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 13:10:11.104866 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:11.122700 22879 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 15.
I20260709 13:10:11.060995 28389 consensus_queue.cc:1048] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
W20260709 13:10:11.120838 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 56818.9us
I20260709 13:10:11.070677 28409 raft_consensus.cc:2749] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:10:11.122505 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 64846.3us
I20260709 13:10:11.171386 22843 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:11.125519 22994 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:11.091240 28051 raft_consensus.cc:2749] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:11.060652 28377 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:11.175376 23022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 2215) took 1028 ms. Trace:
I20260709 13:10:11.069305 28292 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:11.063783 28418 consensus_queue.cc:1059] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 13:10:11.075699 27768 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:11.117367 28280 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
W20260709 13:10:11.237162 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 133071us
I20260709 13:10:11.175792 23022 rpcz_store.cc:276] 0709 13:10:10.147337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:10.170295 (+ 22958us) service_pool.cc:224] Handling call
0709 13:10:10.182073 (+ 11778us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 13:10:10.233787 (+ 51714us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:10.233832 (+    45us) raft_consensus.cc:1539] Triggering prepare for 4 ops
0709 13:10:10.298034 (+ 64202us) log.cc:844] Serialized 104 byte log entry
0709 13:10:10.591217 (+293183us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:10:10.602509 (+ 11292us) log.cc:844] Serialized 14 byte log entry
0709 13:10:10.632068 (+ 29559us) spinlock_profiling.cc:243] Waited 8.34 ms on lock 0x7b18000bb780. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cf83f3 00007f6132cfa7c1 00007f6132cfa779 00007f6132cfa709 00007f6132cfa6d1 00007f6132cf99cd 000000000038f354 000000000038f199 00007f6132cdb83a
0709 13:10:10.920300 (+288232us) spinlock_profiling.cc:243] Waited 4.93 ms on lock 0x7b18000bb780. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132c72c0a 00007f613c3b2219 00007f6132cfaa40 00007f6132cf54e4 00007f6132cf5455 00007f61397dd3c7 00007f6135def9a7 00007f6135dff0bf 00007f6135dff038
0709 13:10:10.920484 (+   184us) log.cc:844] Serialized 14 byte log entry
0709 13:10:10.920699 (+   215us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:10.920773 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:11.175143 (+254370us) raft_consensus.cc:1685] finished
0709 13:10:11.175163 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:11.175318 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":298100,"prepare.queue_time_us":332065,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":73,"raft.queue_time_us":38793,"raft.run_cpu_time_us":319,"raft.run_wall_time_us":10907,"spinlock_wait_cycles":30528896,"thread_start_us":286605,"threads_started":1,"wal-append.queue_time_us":501476}
I20260709 13:10:11.123600 28074 raft_consensus.cc:2749] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:11.085619 28062 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:11.256881 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 176826us
I20260709 13:10:11.257790 22919 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f
W20260709 13:10:11.239853 28384 log.cc:927] Time spent T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.362s	user 0.000s	sys 0.001s
I20260709 13:10:11.175608 28122 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:11.265880 22919 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:11.183027 28319 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:11.262110 23588 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:11.296792 28292 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:11.123203 27960 consensus_queue.cc:237] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:11.329617 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 158354us
W20260709 13:10:11.330592 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 153449us
I20260709 13:10:11.304087 28343 raft_consensus.cc:2749] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:10:11.139168 28365 consensus_queue.cc:237] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:11.308890 28418 raft_consensus.cc:3055] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:10:11.335738 22843 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:11.260013 28280 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:11.348841 28373 consensus_meta.cc:287] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.703s	user 0.004s	sys 0.006s
I20260709 13:10:11.376223 22918 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:11.139465 28419 consensus_queue.cc:260] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:11.309062 28122 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:11.357815 28370 consensus_meta.cc:287] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.716s	user 0.004s	sys 0.007s
I20260709 13:10:11.320348 28319 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:11.383446 28424 log.cc:927] Time spent T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:10:11.384835 28435 raft_consensus.cc:2804] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:10:11.385807 28194 consensus_meta.cc:287] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.711s	user 0.007s	sys 0.002s
I20260709 13:10:11.385708 28438 consensus_queue.cc:1048] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:10:11.387202 27768 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:11.388080 28401 raft_consensus.cc:2804] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:10:11.373762 28131 consensus_meta.cc:287] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.802s	user 0.012s	sys 0.000s
W20260709 13:10:11.389643 28379 log.cc:927] Time spent T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:10:11.389959 28431 consensus_queue.cc:1059] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.123s
I20260709 13:10:11.390583 28423 raft_consensus.cc:2749] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:11.390395 28421 consensus_queue.cc:1048] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 13:10:11.391017 28411 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:11.391817 28364 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:11.348654 28418 raft_consensus.cc:740] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:11.378132 28389 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:11.405000 28435 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:11.378872 28373 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:11.431087 28401 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:11.432018 28433 raft_consensus.cc:2749] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:11.429272 28194 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:11.470649 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 204862us
W20260709 13:10:11.471156 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 135514us
I20260709 13:10:11.471349 22919 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:11.445685 28431 raft_consensus.cc:3055] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:10:11.435037 28131 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:11.400497 28370 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:11.445144 28364 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:11.450525 28389 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:11.453696 28401 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:11.474411 22919 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:11.448244 28435 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:11.445431 28411 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:11.526835 28429 raft_consensus.cc:2804] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:10:11.572918 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 61617us
W20260709 13:10:11.573763 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 197603us
I20260709 13:10:11.597886 23625 raft_consensus.cc:2393] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 15: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
W20260709 13:10:11.599119 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 124775us
I20260709 13:10:11.604548 22919 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:11.469749 28377 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:11.474823 28423 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:10:11.522075 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 47700.1us
I20260709 13:10:11.609601 23624 raft_consensus.cc:2393] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 19: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:11.576490 28315 raft_consensus.cc:697] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 17 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:11.477567 28051 consensus_queue.cc:1059] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.130s
I20260709 13:10:11.609951 28438 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:11.602450 28429 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:11.609923 28423 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:11.575711 28431 raft_consensus.cc:740] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:11.620217 28280 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:11.646806 22947 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.025s	sys 0.047s
W20260709 13:10:11.647881 22947 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.025s	sys 0.047s
I20260709 13:10:11.632776 28319 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:11.636476 28437 raft_consensus.cc:2804] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:11.636775 28122 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:11.642689 28425 log.cc:927] Time spent T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.282s	user 0.000s	sys 0.010s
I20260709 13:10:11.640139 28429 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:11.643263 28423 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:10:11.660948 28427 log.cc:927] Time spent T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.216s	user 0.001s	sys 0.000s
I20260709 13:10:11.660622 28437 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:11.638779 28443 consensus_queue.cc:1048] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
W20260709 13:10:11.666531 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 63667.2us
I20260709 13:10:11.668642 28365 raft_consensus.cc:2749] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:11.674726 28343 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:11.486187 28418 consensus_queue.cc:260] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:11.676725 28373 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:11.680366 28194 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:11.682138 28437 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:11.645605 28438 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:11.688858 28335 consensus_meta.cc:287] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.794s	user 0.006s	sys 0.006s
I20260709 13:10:11.703580 28243 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
W20260709 13:10:11.705554 28453 log.cc:927] Time spent T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:10:11.709286 28389 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:11.747102 28403 log.cc:927] Time spent T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 1.096s	user 0.000s	sys 0.013s
W20260709 13:10:11.755139 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 629635us
I20260709 13:10:11.758831 23020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 2218) took 1609 ms. Trace:
I20260709 13:10:11.765244 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:11.766490 23620 raft_consensus.cc:2410] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:10:11.766472 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:11.767488 22879 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 14.
I20260709 13:10:11.759228 23020 rpcz_store.cc:276] 0709 13:10:10.149196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:10.302977 (+153781us) service_pool.cc:224] Handling call
0709 13:10:10.303455 (+   478us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:10:10.329705 (+ 26250us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:10.329749 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:10:10.343468 (+ 13719us) log.cc:844] Serialized 52 byte log entry
0709 13:10:10.600477 (+257009us) raft_consensus.cc:1645] Marking committed up to 6
0709 13:10:10.600521 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:10.600589 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:11.746398 (+1145809us) spinlock_profiling.cc:243] Waited 1.94 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:10:11.754575 (+  8177us) raft_consensus.cc:1685] finished
0709 13:10:11.754596 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:11.758773 (+  4177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25939,"raft.queue_time_us":25806,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":3914,"spinlock_wait_cycles":4460928,"thread_start_us":196968,"threads_started":1,"wal-append.queue_time_us":252396}
I20260709 13:10:11.698278 28343 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:11.764334 28370 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:11.764434 27912 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:11.733966 28335 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:11.694696 28315 consensus_queue.cc:237] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:11.781486 28292 raft_consensus.cc:2804] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:11.775032 28409 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:11.781708 28131 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:11.677682 28431 consensus_queue.cc:260] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:11.753908 28243 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:11.811743 22949 raft_consensus.cc:1240] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 13. Current term is 14. Ops: []
W20260709 13:10:11.814787 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 210715us
I20260709 13:10:11.798146 28359 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:11.801921 28292 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:11.817404 28449 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:11.820055 28131 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:11.825332 28438 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:11.829273 28243 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:11.830348 28335 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:11.831209 28099 consensus_meta.cc:287] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 1.058s	user 0.012s	sys 0.002s
W20260709 13:10:11.832203 28054 consensus_meta.cc:287] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 1.072s	user 0.013s	sys 0.000s
I20260709 13:10:11.835103 28437 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:11.835879 28423 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:11.837445 28458 log.cc:927] Time spent T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
W20260709 13:10:11.838256 28454 log.cc:927] Time spent T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:10:11.837831 28401 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:11.797094 28411 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:11.841341 28373 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:11.802232 28377 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:11.849858 27356 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:11.896768 27356 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:11.797354 27912 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:11.876375 28450 raft_consensus.cc:2749] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:11.874751 28099 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:11.878700 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:11.845999 28292 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:11.845188 28431 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:11.887565 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:11.870779 28365 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:11.898669 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:11.897315 27356 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:11.807533 28409 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:11.901285 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:11.852918 28359 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:11.909530 28472 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:11.912118 28466 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:11.916039 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:11.937989 23029 raft_consensus.cc:2393] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 15: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:11.916085 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:11.918255 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:11.869575 28131 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:11.921633 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:11.904533 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:11.922420 23023 raft_consensus.cc:1275] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 13:10:11.878091 28054 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:11.926741 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:11.943007 22951 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 15.
I20260709 13:10:11.927582 28099 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:11.928289 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:11.928828 27912 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:11.933817 28423 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:11.956349 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:11.958077 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:11.958817 23024 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:11.934581 28409 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:11.936141 27356 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:11.939824 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:11.942155 23023 pending_rounds.cc:85] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75: Aborting all ops after (but not including) 9
I20260709 13:10:11.959909 23023 pending_rounds.cc:107] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 13:10:11.960108 23617 raft_consensus.cc:2368] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 17. Current term is 18.
I20260709 13:10:11.960327 23023 raft_consensus.cc:2889] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:11.926034 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:11.938586 23021 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 16.
I20260709 13:10:11.951928 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:11.962492 23021 raft_consensus.cc:1275] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:10:11.963946 23021 raft_consensus.cc:1275] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:10:11.881901 28449 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:11.900629 28373 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:11.969978 22919 leader_election.cc:400] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 election: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 17. Current term is 18.
I20260709 13:10:11.974406 22919 leader_election.cc:403] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:10:11.974905 27356 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:11.975335 27356 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:11.970881 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:11.971247 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:11.976688 22875 raft_consensus.cc:2393] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 15: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:11.973888 23619 raft_consensus.cc:1240] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:10:11.914256 28377 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:11.978655 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:11.985328 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:11.986222 22872 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:11.981629 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:11.986857 22875 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 16.
I20260709 13:10:11.988231 27356 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:11.965219 27912 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:11.923535 28365 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:11.985319 28423 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:12.014822 28465 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:12.010673 23025 raft_consensus.cc:2468] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
I20260709 13:10:11.941803 28472 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:12.014955 22992 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:12.015630 28467 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:12.015407 28461 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:12.015576 27356 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:12.016469 28441 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:12.014134 27912 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:12.041294 23590 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:12.058152 23024 raft_consensus.cc:2468] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 13.
I20260709 13:10:12.061048 27356 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:11.946421 28466 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:12.004801 28343 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:12.069017 23027 raft_consensus.cc:1275] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:10:12.070060 22918 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:12.071581 22918 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:12.073906 28449 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:12.074275 28398 raft_consensus.cc:2804] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:12.074645 28449 raft_consensus.cc:2749] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:12.074755 27960 raft_consensus.cc:2749] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:10:12.075937 22843 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:12.064781 28435 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:12.077041 28437 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:12.077131 23620 raft_consensus.cc:1240] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:10:12.077316 28134 raft_consensus.cc:2804] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:12.077821 28134 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:12.077906 28435 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:12.075277 22994 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:12.079106 28359 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:12.081290 28429 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:12.082063 22871 raft_consensus.cc:1240] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:10:12.067636 28441 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:12.084432 28365 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:12.059159 28051 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:12.085254 28051 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:12.086479 28441 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:12.087575 28051 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:12.089092 28402 log.cc:927] Time spent T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 1.154s	user 0.005s	sys 0.014s
I20260709 13:10:12.044602 28465 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:12.091473 28465 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:12.092346 23029 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:12.093772 28089 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:12.097059 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:12.015461 28277 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:12.124871 22872 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 18.
I20260709 13:10:12.133051 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:12.133613 27356 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:12.134300 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:12.073967 28401 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:12.078182 28134 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:12.074666 28398 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:12.138068 23590 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:12.143585 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:12.137321 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 62126.4us
I20260709 13:10:12.077137 28131 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:12.124709 28429 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:12.148628 23029 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 16.
I20260709 13:10:12.153214 23024 raft_consensus.cc:1240] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 12. Current term is 13. Ops: []
I20260709 13:10:12.164311 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:12.156510 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 85023.2us
I20260709 13:10:12.162673 27356 consensus_queue.cc:1059] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 13:10:12.159050 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:12.157094 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:12.167145 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:12.167165 22873 raft_consensus.cc:2393] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 17: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:12.149349 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:12.170334 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:12.170560 22993 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:12.171360 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:12.172302 28468 raft_consensus.cc:2749] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:12.172363 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:12.173462 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:12.174431 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:12.175930 22871 raft_consensus.cc:2393] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 16: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:12.176192 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:12.176679 22918 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:12.177070 23024 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:12.178593 22918 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:12.178931 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 126551us
W20260709 13:10:12.153522 28470 log.cc:927] Time spent T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.121s	user 0.000s	sys 0.001s
I20260709 13:10:12.132819 28277 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:12.066644 28461 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:12.149848 28359 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:12.064411 28467 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:12.187711 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:12.154829 28398 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:12.194299 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:12.161994 28243 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:10:12.201818 28472 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:12.202083 23625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 989) took 1590 ms. Trace:
I20260709 13:10:12.209863 23625 rpcz_store.cc:276] 0709 13:10:10.611296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:10.611433 (+   137us) service_pool.cc:224] Handling call
0709 13:10:12.201891 (+1590458us) spinlock_profiling.cc:243] Waited 963 ms on lock 0x7b54002bc854. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:12.202025 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2215602048}
I20260709 13:10:12.193228 28449 consensus_queue.cc:1048] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:10:12.188530 28315 consensus_queue.cc:1059] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 13:10:12.205359 22918 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:12.204373 28344 raft_consensus.cc:2955] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 10 LEADER]: Committing config change with OpId 10.9: config changed from index 6 to 9, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:12.205822 27912 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:12.180508 28429 raft_consensus.cc:2749] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:12.214705 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:12.217223 28243 raft_consensus.cc:697] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:12.230535 28449 raft_consensus.cc:2804] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:10:12.236969 28480 raft_consensus.cc:2749] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:12.230942 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:12.207465 28373 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:12.246977 28464 log.cc:927] Time spent T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.277s	user 0.001s	sys 0.000s
I20260709 13:10:12.186077 27960 consensus_queue.cc:1048] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.063s
I20260709 13:10:12.195766 28359 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:12.229877 28420 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:12.239104 28461 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:12.277451 23024 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 17.
I20260709 13:10:12.186586 28437 consensus_queue.cc:1048] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:10:12.252843 28377 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:10:12.297312 28476 log.cc:927] Time spent T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 13:10:12.288436 23020 raft_consensus.cc:1275] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 13:10:12.258666 28292 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:12.294361 28243 consensus_queue.cc:237] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:12.295221 28481 log.cc:927] Time spent T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.113s	user 0.000s	sys 0.000s
W20260709 13:10:12.302099 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 96815.5us
I20260709 13:10:12.282874 28449 raft_consensus.cc:697] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:12.289634 28359 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:12.299795 22955 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 2312) took 1239 ms. Trace:
I20260709 13:10:12.304201 22918 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:12.283344 28466 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:10:12.306732 28482 log.cc:927] Time spent T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 13:10:12.306845 28134 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:12.308780 23618 consensus_queue.cc:237] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:12.305230 23024 raft_consensus.cc:2955] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Committing config change with OpId 10.9: config changed from index 6 to 9, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:12.310453 28446 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:12.298558 28420 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:12.318750 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:12.319181 22951 raft_consensus.cc:1275] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:10:12.321729 23023 raft_consensus.cc:2368] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 15. Current term is 16.
I20260709 13:10:12.317560 22955 rpcz_store.cc:276] 0709 13:10:11.060235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:11.060393 (+   158us) service_pool.cc:224] Handling call
0709 13:10:11.071060 (+ 10667us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:11.075233 (+  4173us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:11.075265 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:11.836873 (+761608us) spinlock_profiling.cc:243] Waited 1.89 ms on lock 0x7b2c0005beb8. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f61346dea11 00007f61346ee372 00007f6135e20a01 00007f6135de543a 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2
0709 13:10:11.837564 (+   691us) log.cc:844] Serialized 548 byte log entry
0709 13:10:11.884317 (+ 46753us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:10:11.884363 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:11.884426 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:12.289260 (+404834us) raft_consensus.cc:1685] finished
0709 13:10:12.289281 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:12.299729 (+ 10448us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4349184,"thread_start_us":46458,"threads_started":1,"wal-append.queue_time_us":132186}
I20260709 13:10:12.324996 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:12.325629 22951 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 17.
I20260709 13:10:12.329535 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:12.331018 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:12.335737 22952 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 2313) took 1275 ms. Trace:
I20260709 13:10:12.335930 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:12.336858 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:12.337471 22992 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:12.337114 28449 consensus_queue.cc:237] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:12.338081 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:12.338429 22992 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:12.339272 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:12.339637 28466 consensus_queue.cc:1048] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:10:12.321750 27960 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:12.336207 22952 rpcz_store.cc:276] 0709 13:10:11.060695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:11.060874 (+   179us) service_pool.cc:224] Handling call
0709 13:10:11.061289 (+   415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:11.119020 (+ 57731us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:10:11.119063 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:11.740519 (+621456us) spinlock_profiling.cc:243] Waited 5.69 ms on lock 0x7b2c0005beb8. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f61346dea11 00007f61346ee372 00007f6135e20a01 00007f6135de543a 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2
0709 13:10:11.740946 (+   427us) log.cc:844] Serialized 26 byte log entry
0709 13:10:11.782014 (+ 41068us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:10:11.782053 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:11.782160 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:12.335419 (+553259us) raft_consensus.cc:1685] finished
0709 13:10:12.335443 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:12.335678 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31885,"raft.queue_time_us":247948,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":12345,"spinlock_wait_cycles":13095296,"thread_start_us":97943,"threads_started":2,"wal-append.queue_time_us":67460}
I20260709 13:10:12.340646 28362 raft_consensus.cc:2804] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:12.341173 28362 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:12.341048 27960 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:12.341526 28467 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:12.341701 28362 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:12.342144 28472 raft_consensus.cc:2749] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:12.322576 28330 raft_consensus.cc:2749] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:12.343652 27960 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:12.344509 28466 consensus_queue.cc:1243] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer aed3f089ec8b49af9c8cf5cdee76e8e7 log is divergent from this leader: its last log entry 16.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:10:12.347131 22954 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 15.
I20260709 13:10:12.347285 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:12.348251 22992 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:12.356917 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:12.359025 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:12.361440 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:12.362721 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:12.363310 23026 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 15.
I20260709 13:10:12.357959 28409 raft_consensus.cc:2749] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:12.364264 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:12.346571 28292 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:12.366796 28423 raft_consensus.cc:2749] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 17. Current term is 18.
I20260709 13:10:12.327329 28446 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:12.389540 28319 raft_consensus.cc:3055] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:10:12.435551 28432 raft_consensus.cc:2749] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:12.441792 28420 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:12.444219 28446 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:12.446019 22949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 2309) took 1390 ms. Trace:
I20260709 13:10:12.446134 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:12.447059 28154 raft_consensus.cc:3055] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:10:12.447515 28154 raft_consensus.cc:740] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:12.447541 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:12.446888 22873 raft_consensus.cc:2468] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
I20260709 13:10:12.448707 28154 consensus_queue.cc:260] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:12.449898 28154 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:12.451177 28418 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:12.425743 28319 raft_consensus.cc:740] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:12.454573 28319 consensus_queue.cc:260] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:12.455554 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:12.456081 23624 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 1084) took 1683 ms (client timeout 2158 ms). Trace:
W20260709 13:10:12.437095 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:12.456236 22876 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
I20260709 13:10:12.457183 23623 raft_consensus.cc:1275] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:10:12.431188 28362 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:12.458832 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:12.459223 28362 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:12.459949 23024 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 17.
I20260709 13:10:12.448657 22871 raft_consensus.cc:2410] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
W20260709 13:10:12.460601 23624 rpcz_store.cc:269] 0709 13:10:10.772829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:10.772940 (+   111us) service_pool.cc:224] Handling call
0709 13:10:12.455866 (+1682926us) spinlock_profiling.cc:243] Waited 835 ms on lock 0x7b540046b254. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:12.456024 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1921571072}
I20260709 13:10:12.456100 28319 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:12.464166 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:12.464941 22873 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 14.
I20260709 13:10:12.465319 22994 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:12.466583 22994 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:12.467535 28292 raft_consensus.cc:2749] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:12.468217 28292 raft_consensus.cc:2749] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:12.432313 28466 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:12.471472 28466 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:12.476318 28277 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:12.483001 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:12.483238 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:12.483846 22955 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:12.484864 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:12.485673 22952 raft_consensus.cc:2393] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 19: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:12.486495 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:12.487494 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:12.490252 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:12.491530 28483 log.cc:927] Time spent T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.279s	user 0.001s	sys 0.002s
I20260709 13:10:12.493892 23616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 1085) took 1465 ms. Trace:
I20260709 13:10:12.494354 23619 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 991) took 1882 ms. Trace:
I20260709 13:10:12.494845 23619 rpcz_store.cc:276] 0709 13:10:10.611864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:10.612391 (+   527us) service_pool.cc:224] Handling call
0709 13:10:10.621611 (+  9220us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:11.978526 (+1356915us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:12.494122 (+515596us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b54002bb6d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:12.494298 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3109297024}
I20260709 13:10:12.497157 28062 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:12.437901 28134 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:12.494313 23616 rpcz_store.cc:276] 0709 13:10:11.028038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:11.028186 (+   148us) service_pool.cc:224] Handling call
0709 13:10:11.037675 (+  9489us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:11.325033 (+287358us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:11.325065 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:11.342612 (+ 17547us) log.cc:844] Serialized 26 byte log entry
0709 13:10:11.621582 (+278970us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:10:11.621641 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:12.493479 (+871838us) spinlock_profiling.cc:243] Waited 231 ms on lock 0x7b5400638dd4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:12.493529 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:12.493578 (+    49us) raft_consensus.cc:1685] finished
0709 13:10:12.493594 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:12.493805 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54537,"raft.queue_time_us":103903,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":13721,"spinlock_wait_cycles":530282240,"thread_start_us":274861,"threads_started":1,"wal-append.queue_time_us":332071}
I20260709 13:10:12.446439 22949 rpcz_store.cc:276] 0709 13:10:11.055302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:11.055455 (+   153us) service_pool.cc:224] Handling call
0709 13:10:11.055714 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:11.814902 (+759188us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:12.445766 (+630864us) spinlock_profiling.cc:243] Waited 753 ms on lock 0x7b54000a96d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:12.445968 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1731739904}
I20260709 13:10:12.506928 22950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 2301) took 1829 ms. Trace:
I20260709 13:10:12.446686 28398 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:12.509663 28398 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:12.510948 28091 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:12.511499 28091 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:12.511968 28475 log.cc:927] Time spent T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.272s	user 0.000s	sys 0.001s
I20260709 13:10:12.514261 28466 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:12.518442 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:12.521497 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:12.522820 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:12.523563 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:12.523862 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:12.524402 23029 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 16.
I20260709 13:10:12.524654 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:12.525298 22877 raft_consensus.cc:2368] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 14. Current term is 15.
I20260709 13:10:12.525723 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:12.526772 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:12.527462 22876 raft_consensus.cc:2368] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 17. Current term is 18.
I20260709 13:10:12.529599 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:12.530367 28091 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:12.531040 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:12.531646 22871 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 17.
I20260709 13:10:12.534363 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:12.535521 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:12.535779 22955 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 15.
I20260709 13:10:12.536242 22949 raft_consensus.cc:2393] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 14: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:12.536114 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:12.536793 22878 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 14.
I20260709 13:10:12.542035 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:12.542276 28465 consensus_queue.cc:1048] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:10:12.542721 22955 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:12.468602 22994 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:10:12.545605 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 77081.7us
I20260709 13:10:12.547333 28291 raft_consensus.cc:2804] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:10:12.547720 28291 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:12.548094 28291 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:12.553050 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:12.554776 28447 consensus_queue.cc:1059] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:10:12.556318 22994 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:12.556625 27678 raft_consensus.cc:3055] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:10:12.557308 27678 raft_consensus.cc:740] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:12.560289 28409 raft_consensus.cc:2804] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:10:12.560725 28409 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:12.561055 28409 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:10:12.567842 28428 log.cc:927] Time spent T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 1.134s	user 0.002s	sys 0.002s
I20260709 13:10:12.569154 28419 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:12.569269 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:12.558238 27678 consensus_queue.cc:260] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:12.571911 27678 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:12.575033 23622 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 990) took 1963 ms. Trace:
I20260709 13:10:12.575731 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:10:12.579650 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 60435.3us
I20260709 13:10:12.580866 22955 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 16.
I20260709 13:10:12.581924 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:12.582651 22949 raft_consensus.cc:1217] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Deduplicated request from leader. Original: 12.11->[14.12-19.14]   Dedup: 14.12->[17.13-19.14]
I20260709 13:10:12.583161 22949 pending_rounds.cc:85] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting all ops after (but not including) 12
I20260709 13:10:12.583398 22949 pending_rounds.cc:107] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 13:10:12.583616 22949 raft_consensus.cc:2889] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:12.584555 22994 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:12.575488 23622 rpcz_store.cc:276] 0709 13:10:10.611598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:10.611713 (+   115us) service_pool.cc:224] Handling call
0709 13:10:10.612018 (+   305us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:10:10.832483 (+220465us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:10.832537 (+    54us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:10:10.874916 (+ 42379us) log.cc:844] Serialized 52 byte log entry
0709 13:10:11.121609 (+246693us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:10:11.121751 (+   142us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:11.122245 (+   494us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:12.567121 (+1444876us) spinlock_profiling.cc:243] Waited 1.58 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:10:12.573988 (+  6867us) raft_consensus.cc:1685] finished
0709 13:10:12.574040 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:12.574970 (+   930us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":33601,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":10979,"spinlock_wait_cycles":3628416,"thread_start_us":359544,"threads_started":2,"wal-append.queue_time_us":479624}
I20260709 13:10:12.590701 22872 raft_consensus.cc:1275] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:10:12.593068 22994 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:12.597069 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:12.598156 22873 raft_consensus.cc:2393] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 15: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:12.599567 27911 raft_consensus.cc:2749] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:10:12.605044 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 60750.6us
I20260709 13:10:12.612021 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:12.613013 22955 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15.
I20260709 13:10:12.616694 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:12.622394 22875 raft_consensus.cc:1275] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:10:12.507522 22950 rpcz_store.cc:276] 0709 13:10:10.677208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:10.677334 (+   126us) service_pool.cc:224] Handling call
0709 13:10:10.677647 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:10.715492 (+ 37845us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:10:12.334981 (+1619489us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b180046b340. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132c72c0a 00007f613c3b2219 00007f6132cfaa40 00007f6132cf54e4 00007f6132cf5455 00007f61397dd3c7 00007f6135def9a7 00007f6135dff0bf 00007f6135dff038
0709 13:10:12.344606 (+  9625us) log.cc:844] Serialized 14 byte log entry
0709 13:10:12.430726 (+ 86120us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:12.431423 (+   697us) log.cc:844] Serialized 26 byte log entry
0709 13:10:12.431596 (+   173us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:10:12.433334 (+  1738us) log.cc:844] Serialized 14 byte log entry
0709 13:10:12.433689 (+   355us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:12.433774 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:12.506721 (+ 72947us) raft_consensus.cc:1685] finished
0709 13:10:12.506743 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:12.506871 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":82273,"prepare.queue_time_us":1629711,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":72,"raft.queue_time_us":13707,"raft.run_cpu_time_us":344,"raft.run_wall_time_us":61147,"spinlock_wait_cycles":30214400,"thread_start_us":85239,"threads_started":1,"wal-append.queue_time_us":159325}
I20260709 13:10:12.616443 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:12.626724 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:12.630185 22951 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 18.
I20260709 13:10:12.618644 22994 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:12.625219 22873 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 14.
I20260709 13:10:12.642654 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:12.649087 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:12.655069 22876 raft_consensus.cc:1275] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 13:10:12.651932 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:12.663367 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:12.663398 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 44836.5us
I20260709 13:10:12.664238 22878 raft_consensus.cc:2393] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 17: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:12.660786 22873 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 15.
I20260709 13:10:12.666843 22955 raft_consensus.cc:1240] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 14. Current term is 15. Ops: []
I20260709 13:10:12.671054 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:12.613654 28377 raft_consensus.cc:2804] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:10:12.699841 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:12.621057 27911 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:12.626375 27867 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:12.621183 28122 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:12.621273 28333 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:12.702307 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:12.747392 22876 raft_consensus.cc:2393] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 17: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:12.747076 28333 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:12.698949 27821 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:12.748955 27821 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:12.703116 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:12.751340 22878 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:12.702520 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:12.752277 23623 raft_consensus.cc:2393] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 17: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:12.703603 23020 raft_consensus.cc:1240] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 12. Current term is 13. Ops: []
I20260709 13:10:12.753480 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:12.754675 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:12.755200 23020 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:12.755935 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:12.713217 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:12.717600 28442 consensus_queue.cc:1059] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.103s
I20260709 13:10:12.721381 22918 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:12.721863 28364 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:12.721927 28465 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:12.723573 28091 consensus_queue.cc:1048] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:10:12.691885 28277 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:12.667083 28466 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:12.723618 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:12.687729 28377 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:12.690696 28444 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:12.727831 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:12.762200 28362 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:12.701395 22843 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:12.734335 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:12.737809 28344 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:12.746096 28122 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:12.749362 28333 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:12.753357 28369 consensus_queue.cc:1048] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:10:12.756747 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 62680.6us
I20260709 13:10:12.757095 23027 raft_consensus.cc:2393] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 15: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:12.662714 28062 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:12.755225 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:12.713501 22875 raft_consensus.cc:1240] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:10:12.738966 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
W20260709 13:10:12.758919 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:12.767501 23023 raft_consensus.cc:1275] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
W20260709 13:10:13.252743 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:12.769940 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:12.771476 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
W20260709 13:10:12.777618 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:12.783730 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 62424.6us
I20260709 13:10:12.787115 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:10:13.253187 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:12.791298 23020 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 17.
I20260709 13:10:12.730357 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:13.254371 22918 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:12.800480 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:13.254374 22946 raft_consensus.cc:2393] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 15: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
W20260709 13:10:13.121901 23024 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.543s	user 0.005s	sys 0.008s
W20260709 13:10:13.254530 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:13.255661 23024 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.677s	user 0.008s	sys 0.010s
I20260709 13:10:12.793354 28438 raft_consensus.cc:2749] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:12.779189 28277 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:10:12.848798 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:12.761325 27911 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:12.881327 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:13.261718 23616 raft_consensus.cc:2393] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 15: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:12.744894 28359 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:12.792810 28466 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:12.804064 28409 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:12.780000 28465 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:13.239952 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 538594us
I20260709 13:10:13.240317 28417 raft_consensus.cc:2804] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:10:13.240902 28389 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:13.240708 28344 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:13.241755 28419 consensus_meta.cc:287] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.672s	user 0.002s	sys 0.009s
W20260709 13:10:13.023818 28243 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.579s	user 0.020s	sys 0.049s
I20260709 13:10:12.867617 28362 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:12.780205 28442 consensus_queue.cc:1243] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer 5b346767a89745fa82928a5dac9cc0b2 log is divergent from this leader: its last log entry 11.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:10:12.804561 28444 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:12.763713 27867 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:13.143967 22948 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.005s	sys 0.006s
W20260709 13:10:13.446883 22948 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.834s	user 0.006s	sys 0.008s
I20260709 13:10:12.792305 28377 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:12.762688 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:13.416414 28422 raft_consensus.cc:2749] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:12.796263 22878 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 16.
W20260709 13:10:13.127107 23619 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.005s	sys 0.018s
I20260709 13:10:13.257911 23618 raft_consensus.cc:2393] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 17: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
W20260709 13:10:13.467697 23619 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.899s	user 0.012s	sys 0.020s
W20260709 13:10:13.258443 28154 consensus_meta.cc:287] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.808s	user 0.008s	sys 0.013s
W20260709 13:10:13.310932 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 56663.9us
I20260709 13:10:13.475033 22918 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:13.304572 23588 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f
W20260709 13:10:13.476382 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 171881us
I20260709 13:10:13.266623 28364 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:13.272310 28277 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:13.260027 28446 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:13.265991 28122 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:13.341130 22946 raft_consensus.cc:1275] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:10:13.436425 22947 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
I20260709 13:10:13.356294 28319 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
W20260709 13:10:13.433699 28243 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.994s	user 0.024s	sys 0.050s
I20260709 13:10:13.451722 22955 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:13.253968 22951 raft_consensus.cc:1240] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 15. Current term is 16. Ops: []
I20260709 13:10:13.452507 22877 raft_consensus.cc:2468] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 14.
I20260709 13:10:13.265698 28062 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:13.356348 28417 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:13.460633 23625 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:13.461350 27867 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:13.463982 22948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 2311) took 2407 ms. Trace:
I20260709 13:10:13.428320 28434 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:13.415854 28389 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:13.495051 22918 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:10:13.555732 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 60753.1us
W20260709 13:10:13.528249 27678 consensus_meta.cc:287] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.956s	user 0.004s	sys 0.019s
I20260709 13:10:13.447477 28450 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:13.540036 23588 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:13.553145 22948 rpcz_store.cc:276] 0709 13:10:11.056979 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:11.057360 (+   381us) service_pool.cc:224] Handling call
0709 13:10:11.071414 (+ 14054us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:11.178264 (+106850us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:10:11.178292 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:11.182634 (+  4342us) log.cc:844] Serialized 26 byte log entry
0709 13:10:12.229433 (+1046799us) spinlock_profiling.cc:243] Waited 3.98 ms on lock 0x7b4800190158. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613ad0eaa1 00007f613ad05b37 00007f613ad059ca 00007f613adfd26f 00007f613adfcf37 00007f613ae1155c 00007f613ae11508 00007f613ae11488 00007f613ae11443 00007f613ae106ec 00007f6139d3aa96
0709 13:10:12.612566 (+383133us) spinlock_profiling.cc:243] Waited 1.62 ms on lock 0x7b18002ed980. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cdae07 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf7c02 00007f613597f20b 00007f6135984d23 00007f61359850d1 00007f6135da0a0f
0709 13:10:13.447263 (+834697us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:10:13.447302 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:13.447377 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:13.463528 (+ 16151us) raft_consensus.cc:1685] finished
0709 13:10:13.463547 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:13.463870 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":229967,"raft.run_cpu_time_us":5833,"raft.run_wall_time_us":907429,"spinlock_wait_cycles":12880128,"thread_start_us":920131,"threads_started":2,"wal-append.queue_time_us":2120622}
I20260709 13:10:13.492511 28369 raft_consensus.cc:2804] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:10:13.505864 28154 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:13.575888 22918 leader_election.cc:400] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 15. Current term is 16.
I20260709 13:10:13.593719 22918 leader_election.cc:403] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:10:13.486770 27821 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:13.509279 28421 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:13.594324 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 54368.6us
W20260709 13:10:13.595074 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 343654us
I20260709 13:10:13.530893 28419 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:13.538906 28438 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:13.555634 28417 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:13.616060 23588 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:13.632483 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:13.635088 22955 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 17.
I20260709 13:10:13.701210 22955 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58310 (request call id 2321) took 1004 ms. Trace:
I20260709 13:10:13.701603 22955 rpcz_store.cc:276] 0709 13:10:12.697196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:12.697687 (+   491us) service_pool.cc:224] Handling call
0709 13:10:13.701142 (+1003455us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:13.553369 28442 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:13.505610 28364 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:13.679701 28421 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:13.678614 28450 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:13.575690 28422 raft_consensus.cc:2804] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:13.719401 28422 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:13.719869 28422 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:13.584887 28434 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:13.720782 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:13.722600 28434 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:13.608590 28243 raft_consensus.cc:2749] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:13.723541 28433 consensus_queue.cc:1048] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:10:13.725970 28344 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:13.726294 28364 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:13.725602 22955 raft_consensus.cc:2955] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Committing config change with OpId 10.9: config changed from index 6 to 9, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:13.726701 28344 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:13.727594 28280 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:13.683017 28495 log.cc:927] Time spent T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.383s	user 0.000s	sys 0.001s
I20260709 13:10:13.531761 28446 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:13.731580 23622 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 995) took 1045 ms. Trace:
I20260709 13:10:13.731720 28054 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:13.732523 28446 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:13.615844 28154 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:13.734751 28154 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:13.735146 23619 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 1096) took 1187 ms. Trace:
I20260709 13:10:13.735519 23619 rpcz_store.cc:276] 0709 13:10:12.548137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:12.551185 (+  3048us) service_pool.cc:224] Handling call
0709 13:10:12.567510 (+ 16325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:12.567861 (+   351us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:12.567891 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:12.568367 (+   476us) log.cc:844] Serialized 26 byte log entry
0709 13:10:13.468149 (+899782us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:10:13.468193 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:13.468271 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:13.734940 (+266669us) raft_consensus.cc:1685] finished
0709 13:10:13.734960 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:13.735093 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1055,"raft.run_cpu_time_us":72,"raft.run_wall_time_us":73,"thread_start_us":899435,"threads_started":1,"wal-append.queue_time_us":1164919}
I20260709 13:10:13.723346 27911 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:13.732227 28442 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:13.739456 28442 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:13.615492 28369 raft_consensus.cc:697] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 15 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:13.740018 28442 raft_consensus.cc:2749] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 15. Current term is 16.
I20260709 13:10:13.740473 22949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 3050) took 1213 ms. Trace:
I20260709 13:10:13.741003 28359 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:13.732003 23622 rpcz_store.cc:276] 0709 13:10:12.686525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:12.698296 (+ 11771us) service_pool.cc:224] Handling call
0709 13:10:12.698652 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:12.751380 (+ 52728us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:12.751412 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:12.751456 (+    44us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:10:12.751496 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:13.731368 (+979872us) spinlock_profiling.cc:243] Waited 52.5 ms on lock 0x7b54000c08d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:13.731413 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:13.731525 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":120779008}
I20260709 13:10:13.538964 28319 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:13.743479 28319 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:13.743614 22953 raft_consensus.cc:1240] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 LEADER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 12. Current term is 13. Ops: []
I20260709 13:10:13.728183 28280 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:13.745044 22950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 2319) took 1129 ms. Trace:
I20260709 13:10:13.745658 28280 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:13.631910 28419 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:10:13.746311 28499 log.cc:927] Time spent T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 13:10:13.746654 28419 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:13.748951 28512 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:13.749575 28512 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:13.745437 22950 rpcz_store.cc:276] 0709 13:10:12.615851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:12.624061 (+  8210us) service_pool.cc:224] Handling call
0709 13:10:12.624384 (+   323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:12.663487 (+ 39103us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:12.663517 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:12.729499 (+ 65982us) log.cc:844] Serialized 26 byte log entry
0709 13:10:13.131290 (+401791us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:10:13.131358 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:13.131463 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:13.723241 (+591778us) spinlock_profiling.cc:243] Waited 24.3 ms on lock 0x7b4800190158. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613ad0eaa1 00007f613ad05b37 00007f613ad059ca 00007f613adfd26f 00007f613adfcf37 00007f613ae1155c 00007f613ae11508 00007f613ae11488 00007f613ae11443 00007f613ae106ec 00007f6139d3aa96
0709 13:10:13.744745 (+ 21504us) raft_consensus.cc:1685] finished
0709 13:10:13.744766 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:13.744976 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":130,"raft.run_cpu_time_us":18169,"raft.run_wall_time_us":1097506,"spinlock_wait_cycles":55800576,"thread_start_us":338937,"threads_started":1,"wal-append.queue_time_us":955664}
I20260709 13:10:13.748258 28442 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:13.751644 28442 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:13.748265 28498 log.cc:927] Time spent T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 13:10:13.753759 28389 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:13.754741 23025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 2313) took 1426 ms. Trace:
I20260709 13:10:13.756151 23621 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 1000) took 1017 ms. Trace:
I20260709 13:10:13.727191 28344 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:13.752800 28466 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:13.740875 22949 rpcz_store.cc:276] 0709 13:10:12.527044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:12.574673 (+ 47629us) service_pool.cc:224] Handling call
0709 13:10:12.582467 (+  7794us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:10:12.629977 (+ 47510us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:10:12.690632 (+ 60655us) log.cc:844] Serialized 14 byte log entry
0709 13:10:13.471735 (+781103us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:10:13.494562 (+ 22827us) log.cc:844] Serialized 52 byte log entry
0709 13:10:13.495294 (+   732us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:10:13.495352 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:13.495425 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:13.740246 (+244821us) raft_consensus.cc:1685] finished
0709 13:10:13.740266 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:13.740414 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":216850,"prepare.queue_time_us":66652,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":206775,"raft.run_cpu_time_us":347,"raft.run_wall_time_us":43679,"thread_start_us":556413,"threads_started":2,"wal-append.queue_time_us":1021325}
I20260709 13:10:13.752609 23617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 997) took 1057 ms. Trace:
I20260709 13:10:13.752103 28512 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:13.756527 23621 rpcz_store.cc:276] 0709 13:10:12.738293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:12.750303 (+ 12010us) service_pool.cc:224] Handling call
0709 13:10:12.750623 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:13.241710 (+491087us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:13.241741 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:13.241792 (+    51us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:10:13.241825 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:13.755945 (+514120us) spinlock_profiling.cc:243] Waited 491 ms on lock 0x7b54003d9ad4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:13.755988 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:13.756099 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1128893824}
I20260709 13:10:13.768857 23617 rpcz_store.cc:276] 0709 13:10:12.695449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:12.695572 (+   123us) service_pool.cc:224] Handling call
0709 13:10:12.695864 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:13.493406 (+797542us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:13.493496 (+    90us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:13.493547 (+    51us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:10:13.493591 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:13.752381 (+258790us) spinlock_profiling.cc:243] Waited 797 ms on lock 0x7b5400650ed4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:13.752442 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:13.752557 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1833362816}
I20260709 13:10:13.772902 23625 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 17.
W20260709 13:10:13.774114 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 158120us
I20260709 13:10:13.777168 28426 raft_consensus.cc:2749] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:10:13.778620 28062 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:13.770941 28291 consensus_meta.cc:287] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 1.222s	user 0.019s	sys 0.004s
I20260709 13:10:13.779529 28291 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:13.779932 28422 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:13.774675 28409 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:13.781414 28444 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:13.782281 28422 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:13.784564 28344 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:13.785656 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:13.785925 23588 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:13.786495 22954 raft_consensus.cc:2368] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 17. Current term is 18.
I20260709 13:10:13.786957 22954 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58320 (request call id 3051) took 1259 ms. Trace:
I20260709 13:10:13.787252 22954 rpcz_store.cc:276] 0709 13:10:12.527271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:12.551154 (+ 23883us) service_pool.cc:224] Handling call
0709 13:10:13.785479 (+1234325us) spinlock_profiling.cc:243] Waited 15.1 ms on lock 0x7b5800043598. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0a22 00007f613bbd9c09 00007f613ad33ee0 00007f613adfd0ab 00007f613adfd131 00007f613adab0a0 00007f613adaa984 00007f61343256d4 00007f6134325662
0709 13:10:13.786876 (+  1397us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34690304}
I20260709 13:10:13.789755 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:13.790545 22952 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58320 (request call id 3047) took 1264 ms. Trace:
I20260709 13:10:13.791049 22952 rpcz_store.cc:276] 0709 13:10:12.526348 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:12.540579 (+ 14231us) service_pool.cc:224] Handling call
0709 13:10:13.789601 (+1249022us) spinlock_profiling.cc:243] Waited 4.01 ms on lock 0x7b5800043598. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0a22 00007f613bbd9c09 00007f613ad33ee0 00007f613adfd0ab 00007f613adfd131 00007f613adab0a0 00007f613adaa984 00007f61343256d4 00007f6134325662
0709 13:10:13.790492 (+   891us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9214720}
I20260709 13:10:13.792356 28420 raft_consensus.cc:3055] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:10:13.792829 28420 raft_consensus.cc:740] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:13.793442 23027 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 2244) took 1088 ms. Trace:
I20260709 13:10:13.793746 28420 consensus_queue.cc:260] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:13.793895 23027 rpcz_store.cc:276] 0709 13:10:12.705055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:13.265374 (+560319us) service_pool.cc:224] Handling call
0709 13:10:13.300949 (+ 35575us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:13.437330 (+136381us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:13.437363 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:13.437405 (+    42us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:10:13.437441 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:13.793191 (+355750us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b54001f7cd4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:13.793237 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:13.793385 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":313144064}
I20260709 13:10:13.795210 28420 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:13.796518 28444 raft_consensus.cc:2749] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:13.797799 23588 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:13.804333 28377 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:13.805775 28058 raft_consensus.cc:2749] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:13.806627 28377 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:13.807304 23020 raft_consensus.cc:1240] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:10:13.807384 23588 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:13.808681 23020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 952) took 1033 ms. Trace:
I20260709 13:10:13.809042 23020 rpcz_store.cc:276] 0709 13:10:12.774883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:13.269236 (+494353us) service_pool.cc:224] Handling call
0709 13:10:13.269545 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:13.807747 (+538202us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:13.808451 (+   704us) spinlock_profiling.cc:243] Waited 538 ms on lock 0x7b5400134a54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:13.808611 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1236291328}
I20260709 13:10:13.810094 28465 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:13.811856 23029 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 941) took 1320 ms. Trace:
I20260709 13:10:13.755517 23025 rpcz_store.cc:276] 0709 13:10:12.328452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:12.365114 (+ 36662us) service_pool.cc:224] Handling call
0709 13:10:12.365423 (+   309us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:10:12.365801 (+   378us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:12.365829 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:10:12.366451 (+   622us) log.cc:844] Serialized 52 byte log entry
0709 13:10:12.782986 (+416535us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:10:12.783026 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:12.783094 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:13.754532 (+971438us) raft_consensus.cc:1685] finished
0709 13:10:13.754551 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:13.754686 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3077,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":142,"thread_start_us":416141,"threads_started":1,"wal-append.queue_time_us":1185710}
I20260709 13:10:13.819906 23620 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 999) took 1087 ms. Trace:
I20260709 13:10:13.819680 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:13.820340 23620 rpcz_store.cc:276] 0709 13:10:12.732714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:12.732830 (+   116us) service_pool.cc:224] Handling call
0709 13:10:12.733109 (+   279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:12.750642 (+ 17533us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:10:12.750675 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:12.750719 (+    44us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:10:12.750750 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:13.819684 (+1068934us) spinlock_profiling.cc:243] Waited 17.3 ms on lock 0x7b54008f43d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:13.819726 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:13.819855 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39808768}
I20260709 13:10:13.822276 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:13.812347 23029 rpcz_store.cc:276] 0709 13:10:12.491536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:12.524961 (+ 33425us) service_pool.cc:224] Handling call
0709 13:10:12.584356 (+ 59395us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:12.599024 (+ 14668us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:12.599073 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:12.614135 (+ 15062us) log.cc:844] Serialized 26 byte log entry
0709 13:10:13.249447 (+635312us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:10:13.513776 (+264329us) log.cc:844] Serialized 14 byte log entry
0709 13:10:13.514128 (+   352us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:13.514217 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:13.811615 (+297398us) raft_consensus.cc:1685] finished
0709 13:10:13.811643 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:13.811791 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5616,"prepare.queue_time_us":535438,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":221469,"raft.run_cpu_time_us":203,"raft.run_wall_time_us":9638,"thread_start_us":580178,"threads_started":2,"wal-append.queue_time_us":1126388}
I20260709 13:10:13.824512 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:13.827061 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:13.831606 22843 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:13.821779 28344 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:13.834358 28344 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:13.834612 28417 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:13.836753 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:13.837606 22872 raft_consensus.cc:2393] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 17: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:13.836400 28417 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:13.825095 28058 raft_consensus.cc:2749] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:10:13.827876 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:13.878527 22952 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 15.
I20260709 13:10:13.840447 22877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35370 (request call id 2886) took 1126 ms. Trace:
I20260709 13:10:13.848062 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:13.856043 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:13.881501 22949 raft_consensus.cc:2393] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 17: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:13.879891 22877 rpcz_store.cc:276] 0709 13:10:12.714368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:12.723402 (+  9034us) service_pool.cc:224] Handling call
0709 13:10:13.840269 (+1116867us) spinlock_profiling.cc:243] Waited 669 ms on lock 0x7b54002d6754. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:13.840381 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1538078976}
I20260709 13:10:13.889761 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:13.819126 28343 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:13.909446 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:13.913473 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 355413us
I20260709 13:10:13.903086 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:13.916177 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:13.892532 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 61015.7us
I20260709 13:10:13.899032 22878 raft_consensus.cc:1275] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:10:13.906304 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:13.822666 28438 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:13.922924 22994 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.641s	user 0.021s	sys 0.031s
I20260709 13:10:13.924253 22877 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 13.
I20260709 13:10:13.926828 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:13.877737 28344 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:13.929330 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:13.929925 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:13.930722 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:13.932255 23588 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:13.932271 22946 raft_consensus.cc:1275] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 13:10:13.924496 22994 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.022s	sys 0.031s
W20260709 13:10:13.889941 28507 log.cc:927] Time spent T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.534s	user 0.005s	sys 0.000s
I20260709 13:10:13.896425 28319 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:13.946622 23588 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:13.947576 28457 raft_consensus.cc:2804] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:13.948058 28457 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:13.948467 28457 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:13.948863 28438 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:13.953756 28122 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:10:13.893131 28280 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:13.943572 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:13.955329 23588 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:13.921139 28343 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:13.969518 22843 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:13.965952 22994 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:13.973264 23625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 1001) took 1234 ms. Trace:
I20260709 13:10:13.947041 28319 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:13.973483 28468 raft_consensus.cc:2804] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:10:13.975683 28319 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:13.943842 28519 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:13.954893 28362 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:13.957527 28062 consensus_queue.cc:1059] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 13:10:13.973750 23625 rpcz_store.cc:276] 0709 13:10:12.738605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:12.738720 (+   115us) service_pool.cc:224] Handling call
0709 13:10:13.973106 (+1234386us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b540048dd54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:13.973213 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":478821248}
I20260709 13:10:13.893432 28369 consensus_queue.cc:237] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:13.979223 22994 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:13.980759 28417 raft_consensus.cc:2749] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:13.961580 22844 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:13.976128 28468 raft_consensus.cc:697] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 16 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:13.961206 28280 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:14.003696 28280 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:13.959906 22920 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:14.001233 28468 consensus_queue.cc:237] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:13.966481 28122 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:10:14.025605 22843 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.63.2:33309 timed out after 1.974s (SENT) [suppressed 12 similar messages]
I20260709 13:10:13.985700 28291 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:10:14.056497 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 77395.9us
W20260709 13:10:14.057418 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 97433.2us
I20260709 13:10:14.058576 22920 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:13.973959 28243 consensus_queue.cc:1048] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
W20260709 13:10:14.056185 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 94671.5us
I20260709 13:10:14.065348 22844 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:14.060252 28438 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:13.986132 28519 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:14.062681 28417 raft_consensus.cc:2749] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:14.072798 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:14.073881 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:14.081250 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:14.081760 22874 raft_consensus.cc:2410] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:10:14.059840 28377 raft_consensus.cc:2804] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:14.059748 28442 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:14.060433 28441 raft_consensus.cc:2749] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:10:14.064925 28465 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:14.079038 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:14.079861 22844 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:14.082136 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:14.082841 22920 leader_election.cc:400] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 election: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 14. Current term is 15.
I20260709 13:10:14.084903 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:14.078917 22947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58320 (request call id 3043) took 1564 ms. Trace:
I20260709 13:10:14.085525 28513 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:14.107807 22947 rpcz_store.cc:276] 0709 13:10:12.525279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:12.525429 (+   150us) service_pool.cc:224] Handling call
0709 13:10:14.078752 (+1553323us) spinlock_profiling.cc:243] Waited 910 ms on lock 0x7b5400148054. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:14.078866 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2092815616}
I20260709 13:10:14.108558 28513 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:14.081146 28062 raft_consensus.cc:2749] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:14.085778 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:14.111403 22873 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:14.098436 27912 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:14.115979 23020 raft_consensus.cc:1275] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:10:14.085568 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:14.116588 22872 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:14.117089 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:14.067384 28291 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:14.161844 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:14.118197 22844 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:14.112242 22920 leader_election.cc:403] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 13:10:14.118593 23025 raft_consensus.cc:1275] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:10:14.119588 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:14.071828 23024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 2317) took 1730 ms. Trace:
I20260709 13:10:14.098778 28377 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:10:14.171628 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 53554.2us
W20260709 13:10:14.174224 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 91534.8us
I20260709 13:10:14.175335 22920 leader_election.cc:400] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 election: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 17. Current term is 18.
I20260709 13:10:14.177186 28377 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:14.178293 23027 raft_consensus.cc:1240] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 14. Current term is 15. Ops: []
I20260709 13:10:14.163941 23024 rpcz_store.cc:276] 0709 13:10:12.341687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:12.463199 (+121512us) service_pool.cc:224] Handling call
0709 13:10:12.463505 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:12.463862 (+   357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:12.463889 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:12.578063 (+114174us) log.cc:844] Serialized 26 byte log entry
0709 13:10:13.256129 (+678066us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:10:13.608274 (+352145us) log.cc:844] Serialized 14 byte log entry
0709 13:10:14.071522 (+463248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:14.071593 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:14.071637 (+    44us) raft_consensus.cc:1685] finished
0709 13:10:14.071651 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:14.071771 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37015,"prepare.queue_time_us":529202,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":30,"raft.queue_time_us":123126,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":145,"thread_start_us":1330947,"threads_started":3,"wal-append.queue_time_us":1222469}
I20260709 13:10:14.077440 28280 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:14.079610 28526 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:14.066799 28419 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:14.085914 28527 raft_consensus.cc:2749] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:10:14.201599 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:14.185482 22993 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:14.202636 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:14.111250 28362 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:14.162294 28417 raft_consensus.cc:2749] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:14.162024 28466 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:14.117300 28409 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:14.161485 28441 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:14.226027 28524 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:14.215802 28344 raft_consensus.cc:3055] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:10:14.226739 28344 raft_consensus.cc:740] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:14.226594 28524 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:14.227661 28344 consensus_queue.cc:260] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:14.215997 28280 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:14.234691 28280 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:14.236179 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 63221.4us
I20260709 13:10:14.223297 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:14.201653 28389 raft_consensus.cc:2749] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:10:14.177886 28518 raft_consensus.cc:2749] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 14. Current term is 15.
I20260709 13:10:14.083328 28438 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:14.118120 28465 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:14.219347 28364 raft_consensus.cc:2749] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:14.167604 27912 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:14.225279 28362 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:14.220507 28513 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:14.178906 28291 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:14.176360 22920 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:14.246515 22875 raft_consensus.cc:2393] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 13: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:14.278816 22844 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:14.240607 22872 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 15.
W20260709 13:10:14.275143 23589 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.859s	user 0.011s	sys 0.011s
I20260709 13:10:14.278313 22873 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 13.
I20260709 13:10:14.258455 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:14.282326 23027 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:14.237715 28466 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:14.282229 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:14.275193 22992 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.844s	user 0.011s	sys 0.040s
W20260709 13:10:14.277508 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 101228us
W20260709 13:10:14.309121 22992 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.878s	user 0.011s	sys 0.040s
I20260709 13:10:14.309494 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:14.279995 23589 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.923s	user 0.011s	sys 0.011s
I20260709 13:10:14.309947 22920 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:14.280357 28442 raft_consensus.cc:2749] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:14.237344 28526 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:14.286212 23590 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
W20260709 13:10:14.291458 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:14.313186 23025 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 16.
I20260709 13:10:14.272181 28524 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:14.307410 23026 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.002s	sys 0.025s
I20260709 13:10:14.255928 28343 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:14.302284 23021 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.012s	sys 0.014s
I20260709 13:10:14.284780 28280 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:10:14.320683 23026 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.002s	sys 0.025s
W20260709 13:10:14.320361 23021 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.012s	sys 0.015s
I20260709 13:10:14.324844 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:14.329429 22920 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:14.284865 28518 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:14.273418 28319 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:14.244725 28409 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:14.247156 28441 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:14.343593 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:14.325558 27975 raft_consensus.cc:2749] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:14.344357 28442 raft_consensus.cc:2804] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:14.347208 28457 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:14.352797 28515 consensus_queue.cc:1048] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 13:10:14.355193 28529 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:14.355033 28517 consensus_queue.cc:1048] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:10:14.395205 28519 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:14.476779 22955 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.666s	user 0.020s	sys 0.046s
I20260709 13:10:14.361394 22992 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:14.321894 28122 raft_consensus.cc:2804] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:10:14.362867 28518 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:14.336361 28280 raft_consensus.cc:697] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 17 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:14.326072 28430 raft_consensus.cc:2749] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:14.449226 22880 raft_consensus.cc:1275] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:10:14.450232 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:14.451987 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:14.499554 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:14.371526 28442 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:14.355667 28319 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:14.479283 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:14.501232 22871 raft_consensus.cc:2393] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 17: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:14.518833 22873 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:10:14.481186 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 119887us
I20260709 13:10:14.486584 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:14.494490 22955 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.020s	sys 0.046s
I20260709 13:10:14.496212 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:14.504850 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:14.493546 28122 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:14.500640 28466 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:14.517279 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:14.474238 28419 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:14.521045 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:14.532152 23028 raft_consensus.cc:2468] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
W20260709 13:10:14.520488 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 494886us
I20260709 13:10:14.529906 22880 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:14.530400 23022 raft_consensus.cc:2393] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 15: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:14.495744 28514 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:14.498381 28527 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:14.517581 28420 consensus_meta.cc:287] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.722s	user 0.012s	sys 0.003s
I20260709 13:10:14.543545 22873 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18.
I20260709 13:10:14.553212 28280 consensus_queue.cc:237] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:14.557699 22880 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 16.
I20260709 13:10:14.561095 28319 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:14.561820 28319 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:14.564303 28319 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:14.568533 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:14.569473 23022 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 17.
I20260709 13:10:14.572412 28421 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:14.574872 28518 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:14.577255 28362 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:14.494958 28519 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:14.517542 28442 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:14.579842 23021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 2303) took 2657 ms. Trace:
W20260709 13:10:14.521844 28520 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.517s	user 0.004s	sys 0.015s
I20260709 13:10:14.580371 23021 rpcz_store.cc:276] 0709 13:10:11.939626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:11.972061 (+ 32435us) service_pool.cc:224] Handling call
0709 13:10:11.972451 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:11.973721 (+  1270us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:11.973765 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:11.985055 (+ 11290us) log.cc:844] Serialized 26 byte log entry
0709 13:10:13.739410 (+1754355us) spinlock_profiling.cc:243] Waited 2.05 ms on lock 0x7b1800220960. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cdae07 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf7c02 00007f613597f20b 00007f6135984d23 00007f61359850d1 00007f6135da0a0f
0709 13:10:14.324731 (+585321us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:10:14.324767 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:14.324902 (+   135us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:14.579451 (+254549us) raft_consensus.cc:1685] finished
0709 13:10:14.579479 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:14.579746 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":703,"raft.queue_time_us":840,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":5400,"spinlock_wait_cycles":4724096,"thread_start_us":585124,"threads_started":1,"wal-append.queue_time_us":2544826}
I20260709 13:10:14.528203 28122 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:14.612118 28457 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:14.613296 27912 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:14.613466 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:14.615677 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:14.616312 28526 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:14.616278 23620 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 16.
I20260709 13:10:14.617877 22993 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:14.619390 27912 raft_consensus.cc:2804] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:14.619851 27912 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:14.620299 27912 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:14.621835 28441 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:14.623327 28277 consensus_meta.cc:287] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 1.061s	user 0.006s	sys 0.006s
I20260709 13:10:14.623827 28277 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:14.626386 28277 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:14.629913 28450 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:14.631294 28409 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:14.632074 28465 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:14.633741 28291 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:14.534029 28430 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:14.634732 28430 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:14.638098 28438 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:14.478206 28529 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:14.640089 27912 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:14.641849 28529 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:14.642587 27912 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:14.647040 28527 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:14.649573 28527 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:14.653308 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:14.654088 23622 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:10:14.663653 23027 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18.
I20260709 13:10:14.665567 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:14.669152 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:14.669878 23622 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:14.667809 28457 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:14.685619 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:14.687409 23021 raft_consensus.cc:2393] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 13: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:14.579890 28519 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:14.689440 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:14.690814 23022 raft_consensus.cc:2410] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 17 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:10:14.697750 28446 consensus_queue.cc:1059] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:10:14.718004 28343 raft_consensus.cc:3055] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:10:14.718519 28343 raft_consensus.cc:740] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:14.719578 28343 consensus_queue.cc:260] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:14.730664 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:14.740780 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:14.741888 23025 raft_consensus.cc:2410] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:10:14.743435 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:14.744226 23028 raft_consensus.cc:2468] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
I20260709 13:10:14.744576 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:14.754094 28344 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:14.729029 28430 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:14.745813 23022 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
I20260709 13:10:14.758231 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:14.762037 23028 raft_consensus.cc:2368] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 15. Current term is 16.
I20260709 13:10:14.685868 28457 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:14.762612 28377 consensus_meta.cc:287] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.561s	user 0.010s	sys 0.002s
I20260709 13:10:14.785557 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:14.799551 23029 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.007s	sys 0.037s
W20260709 13:10:14.799950 23029 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.007s	sys 0.037s
W20260709 13:10:14.793164 28459 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.718s	user 0.014s	sys 0.059s
W20260709 13:10:14.829306 23023 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.011s	sys 0.049s
I20260709 13:10:14.792013 28519 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:14.778683 28377 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:14.834395 22994 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:10:14.839464 23617 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.025s	sys 0.024s
W20260709 13:10:14.866600 23617 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.025s	sys 0.024s
W20260709 13:10:14.844429 23023 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.011s	sys 0.049s
W20260709 13:10:14.839820 27911 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.003s	sys 0.046s
I20260709 13:10:14.869534 22994 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:10:14.845412 28459 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.777s	user 0.014s	sys 0.059s
I20260709 13:10:14.829783 28525 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:14.874241 22994 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:14.875929 23020 raft_consensus.cc:2393] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 15: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
W20260709 13:10:14.869756 22918 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.016s	sys 0.017s
W20260709 13:10:14.877624 22918 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.524s	user 0.016s	sys 0.017s
I20260709 13:10:14.879011 22994 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:14.880044 22994 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:10:14.883893 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 530794us
I20260709 13:10:14.891561 28277 raft_consensus.cc:2749] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:14.893005 23027 raft_consensus.cc:1240] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:14.871662 28465 raft_consensus.cc:2749] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:10:14.893959 23023 raft_consensus.cc:1240] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:10:14.892535 28291 raft_consensus.cc:2804] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:10:14.898547 28377 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:14.895496 22918 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:14.899016 28519 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:14.899537 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:14.900425 28485 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:14.892285 28512 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:10:14.919188 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:14.875706 28528 raft_consensus.cc:2749] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:10:14.875291 28525 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:14.921372 23622 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 17.
I20260709 13:10:14.924466 28466 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:14.929319 28291 raft_consensus.cc:697] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:14.937707 23026 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 2315) took 2605 ms. Trace:
I20260709 13:10:14.929821 28527 raft_consensus.cc:2749] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:14.943825 22955 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 2328) took 1457 ms. Trace:
I20260709 13:10:14.930905 22918 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:14.932723 28523 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:14.954126 22955 rpcz_store.cc:276] 0709 13:10:13.486096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:13.724975 (+238879us) service_pool.cc:224] Handling call
0709 13:10:13.725299 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:13.725506 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:10:13.810821 (+ 85315us) log.cc:844] Serialized 14 byte log entry
0709 13:10:14.939661 (+1128840us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:14.939713 (+    52us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:10:14.939760 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:14.939838 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:14.943754 (+  3916us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":53284,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":102,"thread_start_us":1128436,"threads_started":2,"wal-append.queue_time_us":977239}
I20260709 13:10:14.955678 22918 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:10:14.874917 27911 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.004s	sys 0.046s
I20260709 13:10:14.947166 23026 rpcz_store.cc:276] 0709 13:10:12.332150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:12.369775 (+ 37625us) service_pool.cc:224] Handling call
0709 13:10:12.370242 (+   467us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:12.371654 (+  1412us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:12.371703 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:12.372298 (+   595us) log.cc:844] Serialized 26 byte log entry
0709 13:10:13.783961 (+1411663us) spinlock_profiling.cc:243] Waited 3.29 ms on lock 0x7b1800370a80. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cdae07 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf7c02 00007f613597f20b 00007f6135984d23 00007f61359850d1 00007f6135da0a0f
0709 13:10:14.321224 (+537263us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:10:14.924594 (+603370us) log.cc:844] Serialized 14 byte log entry
0709 13:10:14.932122 (+  7528us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:14.932196 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:14.932232 (+    36us) raft_consensus.cc:1685] finished
0709 13:10:14.937354 (+  5122us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:14.937641 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13328,"prepare.queue_time_us":201397,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":2327,"raft.queue_time_us":10159,"raft.run_cpu_time_us":302,"raft.run_wall_time_us":10998,"spinlock_wait_cycles":7570304,"thread_start_us":943874,"threads_started":2,"wal-append.queue_time_us":2328237}
W20260709 13:10:14.958292 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:14.932948 28438 consensus_queue.cc:1048] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
W20260709 13:10:14.965075 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 398110us
W20260709 13:10:14.968330 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 425499us
I20260709 13:10:14.942097 28512 raft_consensus.cc:697] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:14.931936 28485 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:14.935549 28517 consensus_queue.cc:1048] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:10:14.969054 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:14.946934 27975 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:14.937105 28450 consensus_queue.cc:1059] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 13:10:14.959576 28441 raft_consensus.cc:2804] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:10:14.957724 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 439697us
I20260709 13:10:14.971689 22843 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:14.978694 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:14.973809 22874 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 15.
I20260709 13:10:14.929570 28519 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:14.963810 28457 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:14.982609 22843 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:14.982996 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:14.990286 22874 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 17.
I20260709 13:10:14.981101 22992 leader_election.cc:400] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 election: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 17. Current term is 18.
W20260709 13:10:14.991194 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 262865us
I20260709 13:10:14.992498 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:14.992868 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:14.993103 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:14.993942 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:15.004882 22992 leader_election.cc:403] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:10:15.004632 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:14.971009 28389 consensus_queue.cc:1048] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 13:10:15.010403 23588 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:15.007666 22877 raft_consensus.cc:2393] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 16: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:14.953773 28466 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:15.008796 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:15.008002 22843 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:15.010072 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:15.012514 22953 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 17.
W20260709 13:10:15.005128 28369 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.638s	user 0.013s	sys 0.010s
I20260709 13:10:14.980016 28419 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:15.006754 22992 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:15.018661 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:14.959389 28523 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:14.981683 28447 raft_consensus.cc:2804] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:10:14.983494 28421 raft_consensus.cc:2804] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:15.021271 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:15.022732 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:15.005978 28524 raft_consensus.cc:2804] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 13:10:15.019820 28369 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.663s	user 0.014s	sys 0.010s
I20260709 13:10:15.019860 22843 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:15.024609 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:15.026247 23588 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:15.019027 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:15.024463 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:15.023602 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:15.028723 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:15.031332 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:15.041096 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:15.009057 28277 raft_consensus.cc:2749] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 17. Current term is 18.
I20260709 13:10:15.032096 23026 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 17.
I20260709 13:10:15.027722 22953 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
I20260709 13:10:15.023751 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:15.052412 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:15.053519 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:15.053989 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:15.054308 23027 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 12.
I20260709 13:10:15.055408 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:15.056109 23027 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 14.
I20260709 13:10:14.979718 27975 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:15.057686 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:15.058305 22947 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15.
I20260709 13:10:15.043339 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:15.049058 23023 raft_consensus.cc:2393] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 16: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:15.048902 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:15.049177 22994 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:15.061220 22955 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
I20260709 13:10:14.993476 28518 consensus_queue.cc:1059] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 16.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.035s
I20260709 13:10:15.025391 28521 raft_consensus.cc:2749] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:10:15.062415 23026 raft_consensus.cc:2393] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 15: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:15.019625 28441 raft_consensus.cc:697] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:15.079428 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:15.077419 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:15.080595 23027 raft_consensus.cc:1240] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:10:15.062021 22948 raft_consensus.cc:2410] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:10:15.084075 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:15.085737 23027 raft_consensus.cc:2393] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 17: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:15.087270 22994 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:15.089605 23028 raft_consensus.cc:2393] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 17: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:15.041055 28421 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
W20260709 13:10:15.063933 28546 log.cc:927] Time spent T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.169s	user 0.000s	sys 0.001s
I20260709 13:10:15.025772 28442 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:15.025134 28319 consensus_queue.cc:1048] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 13:10:15.110787 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:15.042371 28524 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:15.060139 28062 raft_consensus.cc:2804] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:10:15.129840 28554 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:15.130005 28062 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:15.120059 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:15.130426 28554 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:15.130952 22947 raft_consensus.cc:2368] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 15. Current term is 16.
W20260709 13:10:15.131108 28122 consensus_meta.cc:287] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.530s	user 0.012s	sys 0.000s
I20260709 13:10:15.131466 27975 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:15.130465 28062 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:15.131498 28122 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:15.134549 28122 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:15.135123 23625 raft_consensus.cc:1240] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14. Ops: []
I20260709 13:10:15.082360 28409 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:15.135836 28409 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:15.137279 23026 raft_consensus.cc:1275] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:10:15.138094 28409 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:15.138747 23588 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:15.139424 28277 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:15.139935 28277 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:15.140836 23588 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:15.141166 28560 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:15.141650 28560 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:15.142009 28560 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:15.143139 28369 raft_consensus.cc:2749] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:10:15.143935 28369 consensus_queue.cc:1059] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:10:15.144811 28369 raft_consensus.cc:2749] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:10:15.145434 28369 raft_consensus.cc:3055] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:10:15.145798 28369 raft_consensus.cc:740] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:15.147145 28532 consensus_queue.cc:1059] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:10:15.146742 28369 consensus_queue.cc:260] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:15.148350 28532 raft_consensus.cc:3055] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:10:15.148911 28532 raft_consensus.cc:740] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:15.149936 28532 consensus_queue.cc:260] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:15.151288 28532 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:10:15.151564 28514 consensus_meta.cc:287] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.601s	user 0.009s	sys 0.001s
I20260709 13:10:15.152124 28514 raft_consensus.cc:2749] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:15.033718 28419 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:15.057272 28377 raft_consensus.cc:2749] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:15.064777 28408 raft_consensus.cc:2749] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:15.121592 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:15.121850 28421 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:10:15.127213 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 107402us
I20260709 13:10:15.154063 23021 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
W20260709 13:10:15.086588 28549 log.cc:927] Time spent T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.111s	user 0.000s	sys 0.001s
I20260709 13:10:15.130237 28485 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:15.180284 22843 leader_election.cc:400] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 15. Current term is 16.
I20260709 13:10:15.180804 22843 leader_election.cc:403] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:10:15.059495 28447 raft_consensus.cc:697] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 17 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:15.148438 28369 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:15.178094 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:15.107311 28513 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:15.063575 28333 consensus_queue.cc:1048] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 13:10:15.111977 27912 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 13:10:15.186288 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:15.027706 28291 consensus_queue.cc:237] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:15.123171 28520 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:15.192212 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:15.197820 22843 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:15.192413 27678 raft_consensus.cc:2749] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 15. Current term is 16.
I20260709 13:10:15.152872 28524 raft_consensus.cc:3060] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:15.198300 22877 raft_consensus.cc:2393] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 13: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:15.050312 28512 consensus_queue.cc:237] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:15.211365 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 124164us
W20260709 13:10:15.193228 28542 log.cc:927] Time spent T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.246s	user 0.000s	sys 0.001s
I20260709 13:10:15.200904 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:15.201606 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:15.200937 22879 raft_consensus.cc:1240] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:10:15.208112 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:15.209750 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:15.170197 28450 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:15.076284 28553 consensus_queue.cc:1059] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.132s
I20260709 13:10:15.210865 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:15.132519 28554 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:15.213321 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:15.197829 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 55200.5us
W20260709 13:10:15.576160 28547 log.cc:927] Time spent T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.551s	user 0.001s	sys 0.007s
I20260709 13:10:15.193184 28513 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:15.190831 27912 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:15.211354 28520 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:15.421622 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:15.432637 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 261457us
I20260709 13:10:15.585151 22920 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:15.529516 28519 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:15.530020 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:15.211542 28417 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:15.132517 28441 consensus_queue.cc:237] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:15.531193 28466 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:15.532361 28277 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:10:15.533130 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 335351us
I20260709 13:10:15.534183 28421 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:15.535449 28524 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:15.537520 28438 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:15.537948 28527 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:15.538686 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:15.542207 28419 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:15.549075 28459 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:15.555430 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 343115us
I20260709 13:10:15.547425 28062 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:15.565711 28442 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:15.225483 28447 consensus_queue.cc:237] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:15.556448 28560 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:15.562950 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:15.564304 23625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 1106) took 1219 ms. Trace:
I20260709 13:10:15.564924 22875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53724 (request call id 1014) took 1249 ms. Trace:
I20260709 13:10:15.617522 22948 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 17.
W20260709 13:10:15.531226 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:15.617641 22875 rpcz_store.cc:276] 0709 13:10:14.315061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:14.315761 (+   700us) service_pool.cc:224] Handling call
0709 13:10:14.316433 (+   672us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:14.316858 (+   425us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:14.316894 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:14.324094 (+  7200us) log.cc:844] Serialized 487 byte log entry
0709 13:10:14.828192 (+504098us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:10:14.828231 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:14.828300 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:15.564456 (+736156us) raft_consensus.cc:1685] finished
0709 13:10:15.564481 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:15.564605 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":483671,"threads_started":1,"wal-append.queue_time_us":598748}
I20260709 13:10:15.229105 28377 raft_consensus.cc:2804] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:15.580885 28513 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:15.588274 22843 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:15.626921 22843 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:15.583563 27912 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:10:15.569947 28543 log.cc:927] Time spent T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.559s	user 0.003s	sys 0.005s
W20260709 13:10:15.582926 28344 consensus_meta.cc:287] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.819s	user 0.008s	sys 0.008s
I20260709 13:10:15.557734 28557 raft_consensus.cc:2749] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:15.604429 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:15.644809 22871 raft_consensus.cc:2393] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 16: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:15.606549 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:15.608991 22918 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:15.609669 22873 raft_consensus.cc:1275] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:10:15.617522 23625 rpcz_store.cc:276] 0709 13:10:14.345283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:14.346008 (+   725us) service_pool.cc:224] Handling call
0709 13:10:14.346347 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:15.135501 (+789154us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:15.564069 (+428568us) spinlock_profiling.cc:243] Waited 789 ms on lock 0x7b54006383d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:15.564240 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1813567744}
I20260709 13:10:15.627743 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:15.636170 23588 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:15.629292 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:15.647177 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:15.581734 28450 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:15.647933 23027 raft_consensus.cc:2393] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 15: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:15.622134 28377 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:15.665853 28344 raft_consensus.cc:2749] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:15.667376 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:15.668493 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 59586us
I20260709 13:10:15.618988 28568 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:15.597047 28417 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:15.630928 28576 raft_consensus.cc:2749] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:15.635715 28426 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:15.641819 28442 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:15.612968 28438 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:15.646063 28570 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:15.687727 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:15.689275 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:15.681677 22918 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:15.691754 22918 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:15.640182 28567 raft_consensus.cc:2749] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:15.612704 28459 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:15.690022 23027 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 14.
I20260709 13:10:15.634673 28419 consensus_queue.cc:1048] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 13:10:15.718317 28450 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:15.695549 28570 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:15.719600 28438 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:15.698596 28062 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:15.720496 28515 raft_consensus.cc:2804] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:10:15.711498 28524 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:15.643535 28572 consensus_queue.cc:1048] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.070s
I20260709 13:10:15.715430 28520 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:15.716882 28421 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:15.717739 28417 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:15.720965 28515 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:15.723991 28515 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:15.725728 28560 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:15.727795 28459 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:15.679921 28377 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:15.617187 28527 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:15.735267 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 108446us
I20260709 13:10:15.679857 28574 raft_consensus.cc:2804] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:15.736507 22843 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:15.693751 28426 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:15.697042 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:15.739670 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 145433us
I20260709 13:10:15.733935 28576 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:15.743140 23027 raft_consensus.cc:2410] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 18 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 13:10:15.746421 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:15.748762 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 101356us
I20260709 13:10:15.746977 23027 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 12.
I20260709 13:10:15.745441 22843 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:15.753019 22843 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:15.738453 28574 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:15.752211 22918 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:15.756418 28567 raft_consensus.cc:2749] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:10:15.730190 28442 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:15.745093 28576 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:15.760443 22843 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:15.761322 22920 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:15.762354 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:15.759891 28579 raft_consensus.cc:2804] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:10:15.764281 22843 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:15.694228 28568 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:15.758560 28459 raft_consensus.cc:2804] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:10:15.757541 28574 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:15.758757 28421 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:15.777585 23622 raft_consensus.cc:1240] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:10:15.777649 23617 raft_consensus.cc:1240] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 15. Current term is 16. Ops: []
W20260709 13:10:15.779320 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:15.764741 28576 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:15.764308 28122 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:10:15.771986 28544 log.cc:927] Time spent T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.796s	user 0.002s	sys 0.002s
I20260709 13:10:15.781623 23622 raft_consensus.cc:1275] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:10:15.780555 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:15.831946 23027 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15.
I20260709 13:10:15.783618 23588 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:15.769565 28579 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:15.832382 22843 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:15.833524 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 50014.1us
I20260709 13:10:15.772496 28567 raft_consensus.cc:2804] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:15.776602 28459 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:15.817883 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:15.775902 27821 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:15.823045 22920 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:15.827948 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:15.779953 28421 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:15.812741 28524 consensus_queue.cc:1059] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.018s
W20260709 13:10:15.878638 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 55735.8us
I20260709 13:10:15.876397 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:15.852752 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:15.821066 28578 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:10:15.826529 28447 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:15.849521 22843 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:15.842896 28557 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:15.845304 28408 raft_consensus.cc:2804] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:15.887332 28408 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:15.887746 28408 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:15.887848 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:15.827453 28515 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:15.841616 28567 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:15.886257 22920 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:15.895195 22920 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:15.889855 28527 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:15.845052 28459 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:15.839171 28579 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:15.903502 22843 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:15.905782 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:15.883912 28441 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:15.847187 28344 consensus_queue.cc:1048] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 13:10:15.838989 28122 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:15.892478 28524 raft_consensus.cc:2804] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:15.894439 28578 raft_consensus.cc:697] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:15.902221 28553 raft_consensus.cc:2749] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:15.875561 27821 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:15.902575 28567 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:15.911695 28417 raft_consensus.cc:2749] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:10:15.891675 28438 consensus_queue.cc:1059] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:10:15.912761 28527 raft_consensus.cc:2749] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:15.893715 28447 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:15.932548 28343 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:15.920463 28524 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:15.927860 23625 raft_consensus.cc:2955] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Committing config change with OpId 10.9: config changed from index 6 to 9, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
W20260709 13:10:15.958854 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 63734us
I20260709 13:10:15.959838 22920 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:15.962373 28581 log.cc:927] Time spent T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 13:10:15.962922 28568 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:15.963716 28585 log.cc:927] Time spent T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 13:10:15.964171 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:15.964833 28578 consensus_queue.cc:237] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:15.965380 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:15.965941 23023 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:10:15.957572 28580 log.cc:927] Time spent T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
I20260709 13:10:15.934108 28572 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:15.975500 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:15.975818 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:15.977128 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:15.960429 28408 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:15.985805 23023 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 17.
I20260709 13:10:15.989722 23588 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:15.961409 28524 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:15.993419 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:15.997087 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:15.997249 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:15.997862 22946 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
I20260709 13:10:15.998310 22950 raft_consensus.cc:1240] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 12. Current term is 13. Ops: []
I20260709 13:10:15.998984 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:16.000124 22951 raft_consensus.cc:1275] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:10:16.001186 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:16.001762 22951 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15.
I20260709 13:10:16.001979 22920 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:16.002621 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:16.004767 22920 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:16.005111 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:16.005683 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:16.008019 23024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 2326) took 1802 ms. Trace:
I20260709 13:10:16.008695 28532 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:16.009210 28532 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:16.009620 28532 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:15.969525 28527 raft_consensus.cc:2749] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:16.008558 23024 rpcz_store.cc:276] 0709 13:10:14.206606 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:14.206758 (+   152us) service_pool.cc:224] Handling call
0709 13:10:14.207132 (+   374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:14.207584 (+   452us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:14.207628 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:14.224351 (+ 16723us) log.cc:844] Serialized 26 byte log entry
0709 13:10:14.731648 (+507297us) spinlock_profiling.cc:243] Waited 8.38 ms on lock 0x7b1800476140. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cdae07 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf7c02 00007f613597f20b 00007f6135984d23 00007f61359850d1 00007f6135da0a0f
0709 13:10:14.991174 (+259526us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:10:16.007262 (+1016088us) log.cc:844] Serialized 14 byte log entry
0709 13:10:16.007505 (+   243us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:16.007577 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:16.007616 (+    39us) raft_consensus.cc:1685] finished
0709 13:10:16.007698 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:16.007840 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1002722,"prepare.queue_time_us":1017657,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":1301,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":6910,"spinlock_wait_cycles":19281664,"thread_start_us":259346,"threads_started":1,"wal-append.queue_time_us":999662}
I20260709 13:10:16.016185 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:10:16.016875 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 67998.3us
I20260709 13:10:16.017495 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:16.018221 22877 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 16.
I20260709 13:10:16.019150 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:16.019622 22843 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:16.019904 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:16.021474 23029 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 2321) took 1908 ms. Trace:
I20260709 13:10:16.017179 22953 raft_consensus.cc:2376] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 17. Re-sending same reply.
I20260709 13:10:16.020654 23621 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
I20260709 13:10:16.021880 23029 rpcz_store.cc:276] 0709 13:10:14.113319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:14.113478 (+   159us) service_pool.cc:224] Handling call
0709 13:10:14.113881 (+   403us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:10:14.169697 (+ 55816us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:14.169741 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:10:14.195591 (+ 25850us) log.cc:844] Serialized 52 byte log entry
0709 13:10:14.800386 (+604795us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:10:14.832088 (+ 31702us) log.cc:844] Serialized 14 byte log entry
0709 13:10:14.945797 (+113709us) spinlock_profiling.cc:243] Waited 7.15 ms on lock 0x7b18003ee1e0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132c72c0a 00007f613c3b2219 00007f6132cfaa40 00007f6132cf54e4 00007f6132cf5455 00007f61397dd3c7 00007f6135def9a7 00007f6135dff0bf 00007f6135dff038
0709 13:10:14.946058 (+   261us) log.cc:844] Serialized 14 byte log entry
0709 13:10:14.947582 (+  1524us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:14.947668 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:15.679405 (+731737us) spinlock_profiling.cc:243] Waited 2.37 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:10:15.991201 (+311796us) spinlock_profiling.cc:243] Waited 1.15 ms on lock 0x7b18003ee1e0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132c72c0a 00007f613c3b2219 00007f6139cb958f 00007f6132cf8571 00007f6132cfa7c1 00007f6132cfa779 00007f6132cfa709 00007f6132cfa6d1 00007f6132cf99cd
0709 13:10:16.021256 (+ 30055us) raft_consensus.cc:1685] finished
0709 13:10:16.021277 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:16.021418 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":81169,"prepare.queue_time_us":1191069,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":87,"raft.queue_time_us":14943,"raft.run_cpu_time_us":444,"raft.run_wall_time_us":36961,"spinlock_wait_cycles":24542080,"thread_start_us":552882,"threads_started":1,"wal-append.queue_time_us":716207}
I20260709 13:10:16.024791 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:16.030350 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:16.028527 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:16.033082 22843 leader_election.cc:400] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 15. Current term is 16.
I20260709 13:10:16.004091 28553 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:16.038827 23024 raft_consensus.cc:1240] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:10:16.011492 28450 raft_consensus.cc:2749] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:16.037266 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:16.003952 28518 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:16.034224 22843 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:16.035554 22920 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:16.011031 28468 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:16.041611 22953 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
I20260709 13:10:16.036438 22844 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:16.045858 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:16.022099 28417 raft_consensus.cc:2749] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:15.985963 28570 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:16.027698 28591 raft_consensus.cc:2749] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:15.992177 28572 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:16.040850 28421 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:16.095516 28280 raft_consensus.cc:2804] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:16.125953 22993 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:16.140022 22993 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:16.130955 28590 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:16.131232 28576 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:16.093906 28527 raft_consensus.cc:2804] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:10:16.116109 28587 log.cc:927] Time spent T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 13:10:16.168169 23024 raft_consensus.cc:1275] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:10:16.118631 28515 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:16.170740 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:16.121585 28418 raft_consensus.cc:2804] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:16.172901 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:16.174331 22873 raft_consensus.cc:1240] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:10:16.169350 22952 raft_consensus.cc:1240] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:16.094811 28518 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:16.185210 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:16.154618 27975 raft_consensus.cc:2804] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:10:16.135630 28417 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:16.188512 22950 raft_consensus.cc:2393] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 16: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:16.148874 28466 raft_consensus.cc:2749] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:10:16.154615 28450 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:16.138617 28280 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:16.096045 28468 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:16.188205 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:16.047041 28553 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:16.195330 23028 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 961) took 1062 ms. Trace:
I20260709 13:10:16.124511 28570 consensus_queue.cc:1048] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 5, Time since last communication: 0.058s
I20260709 13:10:16.165230 28527 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:16.174659 28561 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:16.202409 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:16.203866 22954 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
I20260709 13:10:16.186383 28418 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:16.209489 23028 rpcz_store.cc:276] 0709 13:10:15.133329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:15.157646 (+ 24317us) service_pool.cc:224] Handling call
0709 13:10:15.158029 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:15.158277 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:10:16.010787 (+852510us) log.cc:844] Serialized 14 byte log entry
0709 13:10:16.194918 (+184131us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:16.194972 (+    54us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:10:16.195018 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:16.195083 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:16.195197 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":861096,"prepare.queue_time_us":856170,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":34,"raft.queue_time_us":3378,"spinlock_wait_cycles":6365568,"thread_start_us":180904,"threads_started":1,"wal-append.queue_time_us":181236}
I20260709 13:10:16.166985 28590 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:16.223990 28590 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:16.157024 28528 consensus_queue.cc:1059] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 13:10:16.188781 28408 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:16.180900 22953 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
mode: GRACEFUL
new_leader_uuid: "5b346767a89745fa82928a5dac9cc0b2"
 from {username='slave'} at 127.0.0.1:39742
I20260709 13:10:16.225196 22953 raft_consensus.cc:606] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Received request to transfer leadership to TS 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:16.225612 22953 raft_consensus.cc:614] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:10:16.205157 27975 raft_consensus.cc:697] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 17 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:16.227355 27975 consensus_queue.cc:237] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:16.232504 28519 raft_consensus.cc:3055] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:10:16.201536 28450 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:16.233019 28519 raft_consensus.cc:740] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:16.233927 28519 consensus_queue.cc:260] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:16.234795 28450 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:16.235225 28519 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:16.225514 28552 raft_consensus.cc:3055] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:10:16.236949 28552 raft_consensus.cc:740] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:16.237826 28552 consensus_queue.cc:260] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 16.15, Last appended by leader: 15, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:16.238806 28552 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:10:16.239142 28377 consensus_meta.cc:287] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.501s	user 0.009s	sys 0.000s
I20260709 13:10:16.239578 28377 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:16.245180 28520 raft_consensus.cc:3055] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:10:16.245733 27912 consensus_meta.cc:287] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.599s	user 0.010s	sys 0.000s
I20260709 13:10:16.245675 28520 raft_consensus.cc:740] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:16.246292 27912 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:16.246714 28520 consensus_queue.cc:260] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:16.251955 22872 raft_consensus.cc:1240] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:10:16.252614 28525 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:16.253602 22877 raft_consensus.cc:1240] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:10:16.200119 28417 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:16.258865 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:16.260615 28417 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:16.262683 28518 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:16.263907 23027 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:16.272011 28426 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:16.206167 28280 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:16.260273 28519 consensus_queue.cc:1059] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:10:16.278071 27821 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:16.276714 28420 raft_consensus.cc:3055] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:10:16.290493 28556 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:16.301656 22918 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:16.284343 28442 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:16.294680 28466 consensus_queue.cc:1048] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:10:16.287694 28523 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:16.305722 28554 raft_consensus.cc:3055] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:10:16.291196 28529 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:16.315050 28576 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:16.196041 28122 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:16.281311 28528 raft_consensus.cc:3055] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:10:16.320875 22871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35360 (request call id 2361) took 1112 ms. Trace:
I20260709 13:10:16.322463 22875 raft_consensus.cc:1275] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:10:16.405085 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:16.405905 22875 raft_consensus.cc:2368] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16.
W20260709 13:10:16.287628 23021 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33418 (request call id 2257) took 1557 ms (client timeout 1781 ms). Trace:
I20260709 13:10:16.257959 28468 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:16.357945 23590 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:16.331285 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:16.332885 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:16.363092 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:16.366330 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:16.367417 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:16.368420 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:16.365160 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:16.374814 23022 raft_consensus.cc:1240] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 17 LEADER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:16.332518 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:16.377117 22954 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:39742
W20260709 13:10:16.377877 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 76320.7us
I20260709 13:10:16.383764 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:16.316007 28418 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:16.374893 22879 raft_consensus.cc:1275] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:10:16.389875 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:16.221439 28527 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:16.393847 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:16.395799 23027 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 16.
I20260709 13:10:16.355860 28447 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:16.383378 28574 consensus_meta.cc:287] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.583s	user 0.008s	sys 0.004s
I20260709 13:10:16.355412 28420 raft_consensus.cc:740] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:16.401592 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:16.248646 27912 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:16.374511 28438 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:16.406906 22994 leader_election.cc:400] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16.
W20260709 13:10:16.407107 23021 rpcz_store.cc:269] 0709 13:10:14.729938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:14.730302 (+   364us) service_pool.cc:224] Handling call
0709 13:10:16.287407 (+1557105us) spinlock_profiling.cc:243] Waited 417 ms on lock 0x7b540010c554. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:16.287549 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":959833472}
I20260709 13:10:16.403358 22871 rpcz_store.cc:276] 0709 13:10:15.208336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:15.209446 (+  1110us) service_pool.cc:224] Handling call
0709 13:10:16.320648 (+1111202us) spinlock_profiling.cc:243] Waited 77.1 ms on lock 0x7b54001bb554. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:16.320815 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":177363840}
I20260709 13:10:16.374269 28485 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:16.375797 28556 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:16.318147 28561 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:16.409444 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 347805us
I20260709 13:10:16.597218 22872 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 17.
I20260709 13:10:16.597793 22843 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:16.473804 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:16.583946 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:16.556538 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 198671us
I20260709 13:10:16.602648 22955 raft_consensus.cc:2410] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
W20260709 13:10:16.596185 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 443823us
I20260709 13:10:16.612746 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:16.614359 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:16.615075 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:16.616751 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:16.617560 22994 leader_election.cc:403] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:10:16.619757 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:16.388933 28442 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:10:16.598799 22950 raft_consensus.cc:2468] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 13.
I20260709 13:10:16.621510 22947 raft_consensus.cc:2393] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 13: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:16.621752 22918 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:16.622895 23023 raft_consensus.cc:1240] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:16.621762 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:16.623452 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:16.624621 23024 raft_consensus.cc:2410] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 14 because replica has last-logged OpId of term: 12 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 13:10:16.387739 28554 raft_consensus.cc:740] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
W20260709 13:10:16.626258 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 219487us
I20260709 13:10:16.618461 22876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 2358) took 1417 ms. Trace:
I20260709 13:10:16.640312 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:16.617455 22949 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 16.
I20260709 13:10:16.632601 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:16.631429 22953 raft_consensus.cc:2393] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 17: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:16.632395 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:16.644236 22843 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:10:16.684544 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 54164.6us
I20260709 13:10:16.705281 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:16.510430 28459 consensus_meta.cc:287] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.595s	user 0.005s	sys 0.005s
I20260709 13:10:16.518458 28553 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:16.625381 28560 raft_consensus.cc:2749] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:16.751783 22946 raft_consensus.cc:2468] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 13.
I20260709 13:10:16.682076 23020 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 18.
I20260709 13:10:16.735860 22873 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 16.
I20260709 13:10:16.678261 22994 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:16.677387 22950 raft_consensus.cc:2368] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16.
I20260709 13:10:16.403210 28528 raft_consensus.cc:740] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:16.571007 28571 raft_consensus.cc:2749] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:10:16.627223 28532 consensus_meta.cc:287] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.617s	user 0.011s	sys 0.000s
I20260709 13:10:16.498008 28343 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:16.626114 28596 raft_consensus.cc:2749] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:16.408842 28525 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:16.744932 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:16.407420 28441 raft_consensus.cc:2749] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:16.475762 28597 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:16.400959 28529 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:16.780946 28557 consensus_meta.cc:287] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.894s	user 0.005s	sys 0.006s
W20260709 13:10:16.755280 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 150859us
I20260709 13:10:16.679929 27912 raft_consensus.cc:2749] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16.
I20260709 13:10:16.640542 28572 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:16.684132 28513 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
W20260709 13:10:16.761613 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 83411.2us
I20260709 13:10:16.767742 22918 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:16.625566 28438 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:16.810343 22918 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:16.782039 22843 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:16.783188 22947 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 17.
W20260709 13:10:16.811595 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 72244.9us
I20260709 13:10:16.745434 27678 raft_consensus.cc:2804] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:10:16.743050 28602 raft_consensus.cc:2804] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:10:16.744968 28457 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:16.736410 28417 consensus_queue.cc:1059] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.082s
I20260709 13:10:16.661875 28574 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:16.789701 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:16.625945 28280 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:16.683698 28485 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:16.408746 27975 consensus_queue.cc:1048] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.332s
I20260709 13:10:16.764544 28344 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:16.757121 28459 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:16.816819 28468 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:16.840237 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:16.811857 28609 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:16.804374 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:16.823333 28450 raft_consensus.cc:2749] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:10:16.817314 27678 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:10:16.806823 28611 log.cc:927] Time spent T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:10:16.809018 23029 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 15.
I20260709 13:10:16.781466 28532 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:16.788470 28377 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:16.749136 28420 consensus_queue.cc:260] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:16.789809 27821 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:16.850098 22953 raft_consensus.cc:2393] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 14: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:16.821895 22952 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 13.
W20260709 13:10:16.818122 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 80680.4us
W20260709 13:10:16.822391 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:16.801589 28557 raft_consensus.cc:2749] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:16.779978 28343 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:16.807909 28447 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
W20260709 13:10:16.823539 28603 log.cc:927] Time spent T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.450s	user 0.001s	sys 0.000s
I20260709 13:10:16.776787 28291 raft_consensus.cc:2804] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:16.835120 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:16.789012 28525 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:16.823807 28604 log.cc:927] Time spent T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.262s	user 0.000s	sys 0.001s
I20260709 13:10:16.817730 27912 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:16.799073 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:16.940138 22946 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 19.
I20260709 13:10:16.841943 22951 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 17.
I20260709 13:10:16.818246 28442 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:16.841248 22950 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:10:16.763655 28554 consensus_queue.cc:260] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:16.796324 28597 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:16.847532 22954 raft_consensus.cc:606] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:16.825682 28062 raft_consensus.cc:2749] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:10:16.861178 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:16.949651 22949 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 17.
I20260709 13:10:16.827392 28602 raft_consensus.cc:697] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:16.824999 28457 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:16.889338 28420 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:16.879076 27821 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:16.898720 28610 log.cc:927] Time spent T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 13:10:16.900449 28438 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:16.847543 27678 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
W20260709 13:10:16.879449 28576 consensus_queue.cc:1175] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f3c799b1249e40119d2a5864231c6c75" responder_term: 15 status { last_received { term: 15 index: 10 } last_committed_idx: 9 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 14. Current term is 15. Ops: [14.10-14.10]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:10:16.881592 28450 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
W20260709 13:10:16.903960 28524 consensus_meta.cc:287] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.891s	user 0.010s	sys 0.001s
I20260709 13:10:16.904919 28343 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:16.906250 28605 log.cc:927] Time spent T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 13:10:16.909000 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:16.998225 22872 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:10:16.910413 28567 consensus_meta.cc:287] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.956s	user 0.007s	sys 0.002s
I20260709 13:10:16.912362 23027 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33418 (request call id 2269) took 1133 ms. Trace:
I20260709 13:10:16.912705 28532 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:16.915197 28459 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:16.917292 28608 log.cc:927] Time spent T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 13:10:16.817824 28513 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:16.928843 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:16.930279 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:16.928275 22880 raft_consensus.cc:1275] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:10:16.834542 28344 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:17.020169 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:16.941895 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 96369.3us
I20260709 13:10:16.928407 28520 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:16.850023 28609 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:16.846357 28468 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:16.947036 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:17.034868 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:16.947233 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:17.036300 22950 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 16.
I20260709 13:10:16.910413 28418 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:16.846103 22877 raft_consensus.cc:1240] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 14. Current term is 15. Ops: []
I20260709 13:10:16.951344 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:16.942116 28579 consensus_meta.cc:287] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 1.022s	user 0.008s	sys 0.005s
I20260709 13:10:16.932366 28291 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
W20260709 13:10:16.957718 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 105078us
I20260709 13:10:17.046355 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:16.973237 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 44023.8us
I20260709 13:10:17.047668 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:17.047822 23590 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:16.957787 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:16.975412 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:16.822826 28528 consensus_queue.cc:260] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:16.946159 28556 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:16.999137 23027 rpcz_store.cc:276] 0709 13:10:15.778687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:15.887565 (+108878us) service_pool.cc:224] Handling call
0709 13:10:16.912206 (+1024641us) spinlock_profiling.cc:243] Waited 375 ms on lock 0x7b54000bc7d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:16.912307 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":863184768}
I20260709 13:10:16.932076 28447 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:17.003715 22948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 981) took 1016 ms. Trace:
I20260709 13:10:16.972913 28606 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:16.946529 27912 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:17.023813 23622 raft_consensus.cc:1240] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 17. Current term is 18. Ops: []
I20260709 13:10:17.025807 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:17.026746 22878 raft_consensus.cc:2393] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 16: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:17.031776 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:16.959043 28377 consensus_queue.cc:1048] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 13:10:17.065660 22879 raft_consensus.cc:2368] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 14. Current term is 15.
I20260709 13:10:17.002394 28524 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:16.976075 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 135775us
I20260709 13:10:17.048389 23025 raft_consensus.cc:2393] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 16: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:17.048832 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:17.048336 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:17.090077 22946 raft_consensus.cc:2368] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16.
I20260709 13:10:17.090814 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:17.004372 28450 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:17.054463 23617 raft_consensus.cc:1275] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:10:17.060298 23020 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
I20260709 13:10:17.058509 22948 rpcz_store.cc:276] 0709 13:10:15.987722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:15.990244 (+  2522us) service_pool.cc:224] Handling call
0709 13:10:15.990626 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:15.990987 (+   361us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:15.991014 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:15.991454 (+   440us) log.cc:844] Serialized 26 byte log entry
0709 13:10:16.373453 (+381999us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:10:16.373519 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:16.373621 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:17.003371 (+629750us) raft_consensus.cc:1685] finished
0709 13:10:17.003409 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:17.003629 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8061,"raft.queue_time_us":1075,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":16541,"thread_start_us":381517,"threads_started":1,"wal-append.queue_time_us":381956}
I20260709 13:10:17.090656 22843 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:10:17.096150 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:17.061193 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:17.046466 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:17.050583 28291 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:17.086874 22872 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 18.
I20260709 13:10:17.087244 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:17.104947 22878 raft_consensus.cc:1240] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:10:17.101758 23026 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 970) took 1319 ms. Trace:
I20260709 13:10:17.052731 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:17.076328 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:17.112706 23027 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 13.
I20260709 13:10:17.083284 23621 raft_consensus.cc:1240] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 15. Current term is 16. Ops: []
I20260709 13:10:17.093669 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:17.012465 28567 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:17.097873 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:17.064364 28606 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:17.125608 22874 raft_consensus.cc:2368] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 15. Current term is 16.
I20260709 13:10:17.116271 28418 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:17.059633 28529 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:17.105782 23026 rpcz_store.cc:276] 0709 13:10:15.782752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:16.040214 (+257462us) service_pool.cc:224] Handling call
0709 13:10:16.044948 (+  4734us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:16.045417 (+   469us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:16.045458 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:16.107693 (+ 62235us) log.cc:844] Serialized 487 byte log entry
0709 13:10:16.628304 (+520611us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:10:16.628372 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:16.628499 (+   127us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:17.099978 (+471479us) raft_consensus.cc:1685] finished
0709 13:10:17.100005 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:17.101705 (+  1700us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":463263,"threads_started":1,"wal-append.queue_time_us":625091}
I20260709 13:10:17.107769 22994 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:17.119491 23025 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:10:17.120221 22950 raft_consensus.cc:1275] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:10:17.123327 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:17.131072 22872 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 17.
I20260709 13:10:17.075285 28521 raft_consensus.cc:2749] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:10:17.038370 28579 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:17.075168 28343 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:17.048494 28527 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:17.131758 27975 raft_consensus.cc:2804] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:10:17.116262 28609 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:17.137197 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:17.146945 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:17.099400 28574 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:17.062124 28614 consensus_queue.cc:1059] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 11.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.074s
I20260709 13:10:17.116814 28280 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:17.147590 23026 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
I20260709 13:10:17.155287 22875 raft_consensus.cc:1240] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 12. Current term is 13. Ops: []
I20260709 13:10:17.142295 28418 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:10:17.174130 22947 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 17.
I20260709 13:10:17.176210 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:17.151787 27975 raft_consensus.cc:697] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 18 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:10:17.186352 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 95803.7us
W20260709 13:10:17.175559 28615 log.cc:927] Time spent T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.174s	user 0.001s	sys 0.000s
I20260709 13:10:17.189586 23027 raft_consensus.cc:1275] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:10:17.192170 27821 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:17.192553 28525 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:17.192955 28606 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:17.202471 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:17.074846 28602 consensus_queue.cc:237] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:17.193602 28597 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:17.193883 28567 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:17.194829 28524 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:17.185732 23588 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:17.148442 28343 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:17.195595 28527 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:17.229061 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:17.233132 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 185377us
I20260709 13:10:17.234579 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:17.236083 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:17.245458 27912 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:17.246449 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:17.181744 28532 consensus_queue.cc:1059] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 13:10:17.204644 23026 raft_consensus.cc:1275] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:10:17.205557 28450 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:17.185034 28418 raft_consensus.cc:697] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 16 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:17.217898 23588 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:17.221897 27975 consensus_queue.cc:237] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:17.223250 28521 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:10:17.224843 28344 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:16.676585 22876 rpcz_store.cc:276] 0709 13:10:15.201236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:15.201344 (+   108us) service_pool.cc:224] Handling call
0709 13:10:15.201689 (+   345us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:10:15.532483 (+330794us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:15.532544 (+    61us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:10:15.549194 (+ 16650us) log.cc:844] Serialized 52 byte log entry
0709 13:10:15.581241 (+ 32047us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:10:15.626540 (+ 45299us) log.cc:844] Serialized 14 byte log entry
0709 13:10:15.965198 (+338658us) spinlock_profiling.cc:243] Waited 5.06 ms on lock 0x7b1800087d00. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cf8522 00007f6132cfa7c1 00007f6132cfa779 00007f6132cfa709 00007f6132cfa6d1 00007f6132cf99cd 000000000038f354 000000000038f199 00007f6132cdb83a
0709 13:10:16.135245 (+170047us) spinlock_profiling.cc:243] Waited 8.83 ms on lock 0x7b1800087d00. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cdae07 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf5455 00007f6135ddd060 00007f6135def716 00007f6135e00287 00007f6135e00208
0709 13:10:16.618119 (+482874us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:16.618199 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:16.618243 (+    44us) raft_consensus.cc:1685] finished
0709 13:10:16.618260 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:16.618390 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":327233,"prepare.queue_time_us":44592,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":343327,"raft.run_cpu_time_us":207,"raft.run_wall_time_us":7754,"spinlock_wait_cycles":31971072,"thread_start_us":514247,"threads_started":2,"wal-append.queue_time_us":137131,"wal-append.run_cpu_time_us":2245,"wal-append.run_wall_time_us":1042753}
I20260709 13:10:17.250351 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:17.259399 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:17.326481 22950 raft_consensus.cc:1275] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 13:10:17.265007 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:17.287472 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:17.255159 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:17.247268 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:17.247591 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:17.328117 22880 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 16.
I20260709 13:10:17.256033 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:17.329469 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:17.278502 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:17.301398 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:17.332091 23027 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:17.253310 23619 raft_consensus.cc:1240] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:17.281946 23617 raft_consensus.cc:1240] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 14. Current term is 15. Ops: []
W20260709 13:10:17.295233 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:17.266014 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:17.303148 22874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35360 (request call id 2367) took 1307 ms. Trace:
W20260709 13:10:17.309037 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 91212.7us
I20260709 13:10:17.307179 23028 raft_consensus.cc:2393] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 15: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:17.313310 22877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2922) took 1049 ms. Trace:
I20260709 13:10:17.248361 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:17.246941 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:17.335302 22877 rpcz_store.cc:276] 0709 13:10:16.263571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:16.358263 (+ 94692us) service_pool.cc:224] Handling call
0709 13:10:16.358678 (+   415us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:17.037528 (+678850us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:17.312995 (+275467us) spinlock_profiling.cc:243] Waited 487 ms on lock 0x7b54002882d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:17.313235 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1120487680}
I20260709 13:10:17.279110 22918 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:10:17.281308 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:17.336326 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:17.336647 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 57609.3us
I20260709 13:10:17.336695 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:17.337642 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:17.338337 22952 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:10:17.308210 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:17.339743 23026 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:10:17.340379 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 68124us
W20260709 13:10:17.340508 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 63838.7us
I20260709 13:10:17.342777 23620 raft_consensus.cc:2393] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 18: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:17.325547 28582 raft_consensus.cc:2804] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:17.325647 28521 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:17.343593 23029 raft_consensus.cc:1275] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 13:10:17.343950 28521 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:17.344410 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:17.345211 22876 raft_consensus.cc:2393] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 15: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:17.345678 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:17.346292 22877 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 14.
I20260709 13:10:17.342917 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:17.330912 23020 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 16.
I20260709 13:10:17.334609 22874 rpcz_store.cc:276] 0709 13:10:15.996152 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:15.996268 (+   116us) service_pool.cc:224] Handling call
0709 13:10:17.302872 (+1306604us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b540022abd4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:17.303070 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2542681856}
I20260709 13:10:17.356885 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:17.346045 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:17.359974 22876 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 14.
I20260709 13:10:17.316466 23025 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 14.
I20260709 13:10:17.366868 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:17.367127 22875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2932) took 1043 ms. Trace:
I20260709 13:10:17.367513 22875 rpcz_store.cc:276] 0709 13:10:16.323377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:16.406505 (+ 83128us) service_pool.cc:224] Handling call
0709 13:10:16.406786 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:17.155699 (+748913us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:17.366919 (+211220us) spinlock_profiling.cc:243] Waited 747 ms on lock 0x7b54001bc454. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:17.367074 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1717108480}
I20260709 13:10:17.369156 23588 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:17.370105 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:17.252123 28447 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:17.343639 28582 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:17.370723 28426 raft_consensus.cc:2749] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:10:17.371134 28582 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:17.370855 28369 consensus_queue.cc:1059] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:10:17.371666 22875 raft_consensus.cc:2368] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 17. Current term is 18.
I20260709 13:10:17.371841 27821 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:17.256003 28513 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:17.372412 27821 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:17.248888 28532 raft_consensus.cc:3055] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:10:17.374722 28532 raft_consensus.cc:740] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:17.299569 28468 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:17.282686 28485 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:17.375703 28532 consensus_queue.cc:260] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:17.259279 28600 log.cc:927] Time spent T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 1.067s	user 0.005s	sys 0.003s
I20260709 13:10:17.378257 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:17.361210 22876 raft_consensus.cc:2955] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Committing config change with OpId 12.11: config changed from index 6 to 11, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:17.380686 23588 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:17.254114 28343 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:17.382633 22873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35360 (request call id 2374) took 1131 ms. Trace:
I20260709 13:10:17.383150 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:17.383864 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:17.383982 22875 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 16.
I20260709 13:10:17.384004 22920 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:17.385476 22920 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:17.369513 22918 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:17.383173 22873 rpcz_store.cc:276] 0709 13:10:16.251223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:16.258533 (+  7310us) service_pool.cc:224] Handling call
0709 13:10:17.382457 (+1123924us) spinlock_profiling.cc:243] Waited 476 ms on lock 0x7b5400287dd4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:17.382576 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1094956800}
I20260709 13:10:17.392447 28591 consensus_queue.cc:1048] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.010s
I20260709 13:10:17.273792 27912 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:17.381788 22994 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:17.396548 23588 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:17.256945 28579 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:17.398983 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:17.405198 23027 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 16.
I20260709 13:10:17.406488 22843 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:17.252271 28561 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:17.420944 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:17.420508 22955 raft_consensus.cc:2393] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 16: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:17.368196 28517 raft_consensus.cc:2804] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:10:17.366493 28527 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:17.415398 22994 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:17.424166 22952 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15.
W20260709 13:10:17.432327 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 62882.5us
I20260709 13:10:17.439158 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:17.439996 22952 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:17.394284 28532 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:17.398331 28426 raft_consensus.cc:2749] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:17.452935 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:17.459846 23026 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15.
I20260709 13:10:17.460554 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:17.464428 23027 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15.
I20260709 13:10:17.416540 28468 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:17.420923 28369 raft_consensus.cc:2804] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:17.283026 27678 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:17.484943 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 99537.5us
I20260709 13:10:17.414637 28450 consensus_queue.cc:1048] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 13:10:17.421821 28062 consensus_queue.cc:1048] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 13:10:17.309378 28418 consensus_queue.cc:237] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:17.466151 22918 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:17.430619 28517 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:17.478035 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:17.485504 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:17.501479 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:17.502004 22955 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:17.503479 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:17.504567 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:17.505630 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:17.506750 22952 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 16.
I20260709 13:10:17.480841 22844 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:17.414829 28485 raft_consensus.cc:2749] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:10:17.489815 27678 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:17.490417 22878 raft_consensus.cc:1240] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 12. Current term is 13. Ops: []
W20260709 13:10:17.474215 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 45385us
W20260709 13:10:17.557720 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 76945.7us
I20260709 13:10:17.486654 22920 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:17.394060 27912 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:17.452303 28529 raft_consensus.cc:2804] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:10:17.558511 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 71934.7us
I20260709 13:10:17.558508 22844 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:17.559653 22920 leader_election.cc:400] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 14. Current term is 15.
I20260709 13:10:17.559561 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:17.559993 22920 leader_election.cc:403] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:10:17.463510 28421 raft_consensus.cc:2804] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:17.427433 28513 consensus_queue.cc:1059] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 13:10:17.486219 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:17.506563 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:17.463667 28626 raft_consensus.cc:2804] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:10:17.494783 28599 log.cc:927] Time spent T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 1.302s	user 0.007s	sys 0.004s
I20260709 13:10:17.493939 28426 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:17.652875 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:17.653435 28426 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:17.654196 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:17.465281 28468 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:17.655885 28468 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:17.478621 28369 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:17.657764 28369 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:17.444690 28527 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:17.666172 28572 raft_consensus.cc:2955] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Committing config change with OpId 12.11: config changed from index 6 to 11, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:17.667260 28521 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:17.462138 28377 consensus_queue.cc:1059] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
I20260709 13:10:17.669512 28377 raft_consensus.cc:3055] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:10:17.669920 28377 raft_consensus.cc:740] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:17.670805 28377 consensus_queue.cc:260] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:17.672282 23625 raft_consensus.cc:2393] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 20: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:17.677243 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:17.678102 23623 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 13.
I20260709 13:10:17.494421 28553 raft_consensus.cc:2749] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:17.527499 22955 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 18.
W20260709 13:10:17.519802 28622 log.cc:927] Time spent T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.259s	user 0.000s	sys 0.003s
I20260709 13:10:17.680069 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:17.680222 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:17.509685 28517 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:17.681809 22955 raft_consensus.cc:1240] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 17. Current term is 18. Ops: []
I20260709 13:10:17.683027 22955 raft_consensus.cc:1240] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:17.491807 28568 consensus_queue.cc:1059] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.037s
I20260709 13:10:17.497445 28561 consensus_queue.cc:1048] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
W20260709 13:10:17.551292 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 85244.8us
I20260709 13:10:17.538298 28441 raft_consensus.cc:2804] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:10:17.522605 27975 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:17.554529 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:10:17.548930 28420 consensus_meta.cc:287] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.568s	user 0.009s	sys 0.002s
I20260709 13:10:17.556504 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:17.536928 28459 consensus_queue.cc:1048] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 13:10:17.545995 28576 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
W20260709 13:10:17.626410 28632 log.cc:927] Time spent T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:10:17.745926 22946 raft_consensus.cc:1275] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 10. (term mismatch)
W20260709 13:10:17.627596 28627 log.cc:927] Time spent T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
W20260709 13:10:17.627750 28629 log.cc:927] Time spent T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:10:17.599558 28421 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:17.602698 28574 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:17.747694 28421 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:17.747772 28574 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:17.748611 28574 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:17.750013 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:17.751261 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:17.751899 23619 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15.
I20260709 13:10:17.635090 27912 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:17.635651 28291 consensus_meta.cc:287] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.507s	user 0.010s	sys 0.001s
I20260709 13:10:17.752969 28291 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:17.755471 28291 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:17.561614 28529 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:17.756227 28529 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:17.610112 28485 raft_consensus.cc:2749] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 14. Current term is 15.
I20260709 13:10:17.601469 28343 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:17.607052 22843 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:17.652472 28626 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:17.685061 22953 raft_consensus.cc:1275] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
I20260709 13:10:17.721268 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:17.692307 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:17.690428 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:10:17.689802 22994 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:17.695068 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:17.691334 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:17.689428 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:17.723636 22951 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:17.723748 22948 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:17.724869 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:17.899801 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:17.900564 23623 raft_consensus.cc:2393] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 16: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:17.901321 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:17.902050 23623 raft_consensus.cc:2393] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 18: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:17.902755 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:17.903725 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:17.726974 22844 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:17.727676 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:17.743866 27975 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:17.745734 28576 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:17.905766 22877 raft_consensus.cc:2468] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 15.
I20260709 13:10:17.746439 22946 pending_rounds.cc:85] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting all ops after (but not including) 9
I20260709 13:10:17.907158 22946 pending_rounds.cc:107] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting uncommitted NO_OP operation due to leader change: 14.10
I20260709 13:10:17.907505 22946 raft_consensus.cc:2889] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:17.760793 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:17.909019 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:17.909194 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:17.909739 23620 raft_consensus.cc:2393] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 14: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:17.909065 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:17.911348 22951 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 18.
I20260709 13:10:17.912045 22948 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15.
I20260709 13:10:17.893054 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:17.672256 28377 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:17.685295 28568 raft_consensus.cc:3055] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:10:17.920646 23621 raft_consensus.cc:2393] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 13: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:17.693849 28389 raft_consensus.cc:3055] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:10:17.939908 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:17.940677 23621 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 15.
I20260709 13:10:17.937778 22948 raft_consensus.cc:1240] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 12. Current term is 13. Ops: []
I20260709 13:10:17.916850 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:17.970677 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:17.971513 23624 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:17.741389 28426 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:18.002365 23620 raft_consensus.cc:1275] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 17 index: 16. (index mismatch)
I20260709 13:10:17.896936 28441 raft_consensus.cc:697] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:17.907233 28626 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:17.897461 28417 consensus_queue.cc:1048] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.110s
I20260709 13:10:17.936385 28640 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:18.021054 28421 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:18.021497 28521 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:18.004274 28557 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:18.053577 28557 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:18.054211 28557 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:18.054971 23022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 2355) took 1723 ms. Trace:
I20260709 13:10:18.055441 23022 rpcz_store.cc:276] 0709 13:10:16.331278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:16.331783 (+   505us) service_pool.cc:224] Handling call
0709 13:10:16.359975 (+ 28192us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:16.623229 (+263254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:18.054680 (+1431451us) spinlock_profiling.cc:243] Waited 14.6 ms on lock 0x7b54002da5d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:18.054901 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":33604352}
I20260709 13:10:18.060797 23621 raft_consensus.cc:1240] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 19. Current term is 20. Ops: []
I20260709 13:10:18.028379 28601 raft_consensus.cc:3055] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:10:18.031190 28644 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:17.920907 27911 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:17.971554 28389 raft_consensus.cc:740] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:17.971359 28568 raft_consensus.cc:740] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:18.020551 28527 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:17.896255 28582 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:18.023567 28447 raft_consensus.cc:2749] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:18.037652 28441 consensus_queue.cc:237] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:17.913515 28343 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:18.037793 28640 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:18.040783 28421 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:18.041458 22952 raft_consensus.cc:2393] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 17: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:18.046478 22954 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:39742 (request call id 1) took 1727 ms. Trace:
I20260709 13:10:18.083653 22954 rpcz_store.cc:276] 0709 13:10:16.318955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:16.319803 (+   848us) service_pool.cc:224] Handling call
0709 13:10:18.046297 (+1726494us) spinlock_profiling.cc:243] Waited 225 ms on lock 0x7b5400411ed4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135ddf853 00007f613adad60a 00007f61343337d4 00007f6134333762 00007f6134333691 00007f613433360c 00007f61343328c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:18.046386 (+    89us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":516782336}
I20260709 13:10:18.081007 27911 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:18.094681 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:18.095844 27911 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:18.101264 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:18.035559 28576 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:18.069306 28601 raft_consensus.cc:740] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:18.079319 23624 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 14.
W20260709 13:10:18.076824 28532 consensus_meta.cc:287] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.615s	user 0.005s	sys 0.004s
I20260709 13:10:18.110695 23618 raft_consensus.cc:2393] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 17: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:18.112151 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:18.113126 23625 raft_consensus.cc:2368] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14.
I20260709 13:10:18.123257 22993 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:18.125072 23590 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:18.125339 23619 raft_consensus.cc:1240] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14. Ops: []
I20260709 13:10:18.133103 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:18.134037 23620 raft_consensus.cc:2410] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:10:18.079434 28644 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:18.121898 27911 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:18.137032 28607 raft_consensus.cc:2804] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:18.130501 28468 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:18.160861 28607 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:18.166873 28466 raft_consensus.cc:2749] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:10:18.122151 28389 consensus_queue.cc:260] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:18.131218 28568 consensus_queue.cc:260] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:18.188081 28607 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:18.221020 28528 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:18.222599 28626 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:18.222529 28557 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:18.227502 28640 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:18.217494 28389 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:18.234408 28557 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:18.235677 28607 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:18.237061 28343 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:18.238217 28607 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:18.239889 28582 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:18.241792 28468 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:18.243372 28644 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:18.244025 28457 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:18.245564 28572 consensus_meta.cc:287] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.577s	user 0.010s	sys 0.003s
I20260709 13:10:18.247174 28574 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:18.248309 28554 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:18.241835 28529 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:18.248413 28369 consensus_meta.cc:287] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.590s	user 0.005s	sys 0.001s
I20260709 13:10:18.252933 28529 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:18.267410 28574 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:18.259299 28369 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:18.269294 28517 consensus_meta.cc:287] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.587s	user 0.008s	sys 0.001s
I20260709 13:10:18.269923 28517 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:18.271538 28369 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:18.182252 28602 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.920s	user 0.007s	sys 0.007s
W20260709 13:10:18.278811 28659 log.cc:927] Time spent T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.146s	user 0.000s	sys 0.001s
I20260709 13:10:18.277578 28626 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:18.073993 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 347091us
W20260709 13:10:18.312108 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 704803us
I20260709 13:10:18.314530 22843 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:18.316531 22843 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:18.318116 22843 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:10:18.319691 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:18.318380 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:18.321590 22880 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.890s	user 0.035s	sys 0.076s
W20260709 13:10:18.322136 22880 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.895s	user 0.035s	sys 0.076s
I20260709 13:10:18.327082 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:18.332909 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:18.334856 22844 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:18.080160 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 520560us
I20260709 13:10:18.346815 22920 leader_election.cc:400] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 election: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 15. Current term is 16.
I20260709 13:10:18.347214 22920 leader_election.cc:403] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:10:18.349288 22949 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
I20260709 13:10:18.351171 22920 leader_election.cc:400] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 election: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 17. Current term is 18.
I20260709 13:10:18.351704 22920 leader_election.cc:403] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:10:18.354785 22920 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:18.356637 22844 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:18.141317 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 219037us
I20260709 13:10:18.184969 27975 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:18.274611 28602 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.012s	user 0.007s	sys 0.009s
I20260709 13:10:18.387396 28438 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:18.167230 27911 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:18.390297 28578 raft_consensus.cc:2749] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 15. Current term is 16.
I20260709 13:10:18.390415 28641 raft_consensus.cc:2749] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 17. Current term is 18.
I20260709 13:10:18.391601 28578 raft_consensus.cc:2804] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:18.392138 28578 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:18.272145 28517 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:18.388741 28438 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:18.376832 28062 raft_consensus.cc:2804] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:10:18.397166 22949 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58320 (request call id 3081) took 1486 ms (client timeout 1876 ms). Trace:
I20260709 13:10:18.397161 28520 raft_consensus.cc:2804] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:18.406175 28520 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
W20260709 13:10:18.405948 22949 rpcz_store.cc:269] 0709 13:10:16.911038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:16.950933 (+ 39895us) service_pool.cc:224] Handling call
0709 13:10:18.396799 (+1445866us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54004123d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:18.397035 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3014559616}
I20260709 13:10:18.406800 28520 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:18.407644 28561 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:18.392648 28578 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:18.409155 27821 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:18.410372 23616 raft_consensus.cc:1240] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:18.411420 28420 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:18.412024 27912 raft_consensus.cc:3055] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:10:18.412216 28420 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:18.388288 28572 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:41960 (request call id 0) took 6398 ms. Trace:
I20260709 13:10:18.408524 28561 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:18.191928 28601 consensus_queue.cc:260] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:18.423400 22877 raft_consensus.cc:1240] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:10:18.423573 28344 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:18.461217 22878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 2941) took 1445 ms. Trace:
I20260709 13:10:18.461913 28567 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:18.416966 28343 raft_consensus.cc:2804] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:18.462930 28343 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:18.463287 28343 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:18.463402 22952 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58310 (request call id 2367) took 1088 ms. Trace:
I20260709 13:10:18.463773 22952 rpcz_store.cc:276] 0709 13:10:17.375218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:17.559331 (+184113us) service_pool.cc:224] Handling call
0709 13:10:18.463009 (+903678us) spinlock_profiling.cc:243] Waited 481 ms on lock 0x7b540037a0d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:18.463220 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1106160000}
I20260709 13:10:18.415665 28420 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:18.423974 22876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53724 (request call id 1037) took 1099 ms. Trace:
I20260709 13:10:18.276415 28568 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:18.469812 23025 raft_consensus.cc:1240] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:10:18.469132 22876 rpcz_store.cc:276] 0709 13:10:17.324173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:17.360639 (+ 36466us) service_pool.cc:224] Handling call
0709 13:10:17.360920 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:17.361105 (+   185us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:10:17.481771 (+120666us) log.cc:844] Serialized 14 byte log entry
0709 13:10:18.422842 (+941071us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:18.422990 (+   148us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:10:18.423189 (+   199us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:18.423379 (+   190us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:18.423645 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":110495,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":92,"thread_start_us":940330,"threads_started":2,"wal-append.queue_time_us":651558}
I20260709 13:10:18.433199 27911 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:18.437527 28561 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:18.422963 28438 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:18.473328 28438 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:18.402853 23621 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 1120) took 1130 ms. Trace:
I20260709 13:10:18.395493 22946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 3085) took 1483 ms. Trace:
I20260709 13:10:18.428864 22955 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 998) took 1064 ms. Trace:
I20260709 13:10:18.438745 23616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 1118) took 1167 ms. Trace:
I20260709 13:10:18.401646 23618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60044 (request call id 1000) took 1147 ms. Trace:
I20260709 13:10:18.481477 22950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 3089) took 1568 ms. Trace:
I20260709 13:10:18.476162 22955 rpcz_store.cc:276] 0709 13:10:17.364413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:17.724659 (+360246us) service_pool.cc:224] Handling call
0709 13:10:17.725068 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:17.909074 (+184006us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:10:18.351959 (+442885us) log.cc:844] Serialized 14 byte log entry
0709 13:10:18.418851 (+ 66892us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:18.418960 (+   109us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:10:18.419081 (+   121us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:18.428381 (+  9300us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b5400148554. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:18.428429 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:18.428642 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33693,"prepare.queue_time_us":316084,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":34008,"raft.run_cpu_time_us":345,"raft.run_wall_time_us":345,"spinlock_wait_cycles":422494464,"thread_start_us":407724,"threads_started":2}
I20260709 13:10:18.439049 28389 raft_consensus.cc:2804] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:10:18.454676 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 331349us
I20260709 13:10:18.447639 28369 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:18.481904 22950 rpcz_store.cc:276] 0709 13:10:16.913440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:17.135037 (+221597us) service_pool.cc:224] Handling call
0709 13:10:17.324432 (+189395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:17.326887 (+  2455us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:18.481076 (+1154189us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b5400148050. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:18.481320 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":434622208}
I20260709 13:10:18.457543 22843 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:18.474193 28438 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:18.448896 28418 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:18.482401 28520 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:18.491168 28418 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:18.396612 28447 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:18.493409 28447 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:18.494901 22880 raft_consensus.cc:1240] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 11. Current term is 12. Ops: []
I20260709 13:10:18.481518 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:18.475435 23621 rpcz_store.cc:276] 0709 13:10:17.272297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:18.001010 (+728713us) service_pool.cc:224] Handling call
0709 13:10:18.060518 (+ 59508us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:18.061385 (+   867us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:18.402240 (+340855us) spinlock_profiling.cc:243] Waited 59.1 ms on lock 0x7b540046b250. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:18.402589 (+   349us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":135825280}
I20260709 13:10:18.495535 28520 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:18.484954 28062 raft_consensus.cc:697] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 16 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:18.461627 22878 rpcz_store.cc:276] 0709 13:10:17.015410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:17.144452 (+129042us) service_pool.cc:224] Handling call
0709 13:10:17.144740 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:17.561586 (+416846us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:18.460701 (+899115us) spinlock_profiling.cc:243] Waited 345 ms on lock 0x7b540022bad4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:18.461034 (+   333us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":794407296}
I20260709 13:10:18.484158 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:18.483551 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:18.449018 28579 raft_consensus.cc:2749] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:18.457154 28601 raft_consensus.cc:3060] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:18.482810 28389 raft_consensus.cc:697] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 15 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
W20260709 13:10:18.399209 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 1.05673e+06us
I20260709 13:10:18.497627 23020 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:10:18.501317 23026 raft_consensus.cc:2393] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 15: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:18.477459 23616 rpcz_store.cc:276] 0709 13:10:17.271790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:18.003731 (+731941us) service_pool.cc:224] Handling call
0709 13:10:18.409769 (+406038us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:18.410881 (+  1112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:18.438532 (+ 27651us) spinlock_profiling.cc:243] Waited 374 ms on lock 0x7b5400638dd0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:18.438681 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":860516224}
I20260709 13:10:18.445904 23590 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:18.501106 28369 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:18.511488 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 54201.6us
I20260709 13:10:18.512648 28528 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:18.512797 22843 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:18.513721 28369 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:18.513569 28528 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:18.511637 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:18.412631 27912 raft_consensus.cc:740] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:18.509356 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:18.518548 22878 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:18.481779 23618 rpcz_store.cc:276] 0709 13:10:17.253755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:17.254934 (+  1179us) service_pool.cc:224] Handling call
0709 13:10:18.401022 (+1146088us) spinlock_profiling.cc:243] Waited 781 ms on lock 0x7b5400638dd4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:18.401357 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1796644736}
I20260709 13:10:18.475963 22946 rpcz_store.cc:276] 0709 13:10:16.911996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:17.128341 (+216345us) service_pool.cc:224] Handling call
0709 13:10:17.143804 (+ 15463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:17.908053 (+764249us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:18.394992 (+486939us) spinlock_profiling.cc:243] Waited 602 ms on lock 0x7b54000ec9d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:18.395297 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1383851776}
I20260709 13:10:18.503103 28520 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:18.500928 28062 consensus_queue.cc:237] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:18.519737 28567 raft_consensus.cc:2749] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:18.501735 28561 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:18.520236 28520 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:18.521816 28561 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:18.496606 28447 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:18.500219 28438 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:18.524370 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 78882us
I20260709 13:10:18.520364 27912 consensus_queue.cc:260] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:18.527068 27912 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:18.420745 28572 rpcz_store.cc:276] 0709 13:10:11.989271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:12.278640 (+289369us) service_pool.cc:224] Handling call
0709 13:10:12.333563 (+ 54923us) log.cc:844] Serialized 487 byte log entry
0709 13:10:13.719274 (+1385711us) spinlock_profiling.cc:243] Waited 7.83 ms on lock 0x7b540059a220. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135d5e142 00007f6135d3eeaa 00007f6135d45d53 00007f6135d45cd9 00007f6135d45c69 00007f6135d45c31 00007f6135d44dfd 000000000038f354 000000000038f199 00007f6132cf8475
0709 13:10:13.747384 (+ 28110us) spinlock_profiling.cc:243] Waited 4.44 ms on lock 0x7b540059a220. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135d65b26 00007f6135d5f49a 00007f6135d5e714 00007f6135d3eeaa 00007f6135d45d53 00007f6135d45cd9 00007f6135d45c69 00007f6135d45c31 00007f6135d44dfd 000000000038f354
0709 13:10:18.387849 (+4640465us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65564,"raft.queue_time_us":85419,"raft.run_cpu_time_us":4091,"raft.run_wall_time_us":1928718,"spinlock_wait_cycles":28226688,"thread_start_us":327231,"threads_started":1,"wal-append.queue_time_us":525321,"wal-append.run_cpu_time_us":16626,"wal-append.run_wall_time_us":1771234}
I20260709 13:10:18.531961 27821 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:18.535094 22763 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 2959101199a44a4583a4ee456ecf044d with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 11)
I20260709 13:10:18.535559 27821 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:18.536638 28528 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:18.538167 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:18.538285 27821 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:18.531977 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:18.539331 23025 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 15.
I20260709 13:10:18.540597 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:10:18.544368 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:18.544950 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:18.536059 28426 raft_consensus.cc:2804] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:18.548804 28426 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:18.554562 28426 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:18.558818 23020 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 17.
I20260709 13:10:18.565045 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:18.567797 23025 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:18.568081 22951 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 1000) took 1201 ms. Trace:
I20260709 13:10:18.569236 28343 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:18.568485 22951 rpcz_store.cc:276] 0709 13:10:17.366992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:17.918402 (+551410us) service_pool.cc:224] Handling call
0709 13:10:17.918780 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:17.919048 (+   268us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:10:18.446611 (+527563us) log.cc:844] Serialized 14 byte log entry
0709 13:10:18.551334 (+104723us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:18.559640 (+  8306us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:10:18.567628 (+  7988us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:18.567746 (+   118us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:18.567899 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57896,"prepare.queue_time_us":309153,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":26,"raft.queue_time_us":10595,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":120,"thread_start_us":511746,"threads_started":2}
I20260709 13:10:18.570489 22872 raft_consensus.cc:2393] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 17: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:18.573259 22992 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:18.575685 28561 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:18.556425 28389 consensus_queue.cc:237] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:18.578815 28578 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:18.581369 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:18.584101 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 71452.4us
I20260709 13:10:18.584772 22878 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 16.
I20260709 13:10:18.554557 28154 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:18.581025 27821 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:18.589470 28528 raft_consensus.cc:2749] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:18.599682 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:18.608224 28524 raft_consensus.cc:2749] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:18.603567 22843 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:18.612735 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:18.614449 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:18.616235 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:18.617367 23623 raft_consensus.cc:2368] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 15. Current term is 16.
I20260709 13:10:18.617568 28420 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:18.623526 28532 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:18.624279 28532 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:18.645617 23616 raft_consensus.cc:1240] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 17. Current term is 18. Ops: []
I20260709 13:10:18.645808 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:18.605360 28520 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:18.608855 28652 consensus_queue.cc:1059] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
W20260709 13:10:18.825866 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 229256us
I20260709 13:10:18.774278 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:18.829734 22951 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:18.746145 22874 raft_consensus.cc:1240] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:18.654183 23588 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:18.654412 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:18.839880 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 274389us
I20260709 13:10:18.774557 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:18.841460 28553 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:18.841657 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:18.821223 28418 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:18.851544 28668 log.cc:927] Time spent T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.246s	user 0.007s	sys 0.010s
I20260709 13:10:18.851847 28441 consensus_queue.cc:1059] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 13:10:18.777695 28670 log.cc:927] Time spent T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 13:10:18.864221 23617 raft_consensus.cc:2468] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
W20260709 13:10:18.895274 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 322201us
I20260709 13:10:18.768677 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:18.695778 28520 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:18.654963 28420 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:18.855906 22776 catalog_manager.cc:5697] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 9 to 10, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 10 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:18.712844 28578 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:18.747263 28438 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:18.753219 28526 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:18.766887 28343 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:18.898985 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:18.689229 28447 raft_consensus.cc:2749] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:18.836179 28532 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:18.629374 27821 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:18.901837 22843 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:18.649992 28154 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:18.922423 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:18.927313 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:18.916749 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:18.920262 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:18.935329 23025 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 14.
I20260709 13:10:18.905311 28572 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:10:18.931857 22946 raft_consensus.cc:2468] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
I20260709 13:10:18.944599 28602 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:18.945376 28343 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:18.945503 28602 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:18.946041 28343 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:18.948195 28602 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:18.948717 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:18.944985 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:18.949795 28447 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:18.950452 28447 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:18.951293 28526 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:18.912680 28553 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:18.953306 28438 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:18.954826 28553 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:18.955452 28438 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:18.955524 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:18.956425 22872 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
I20260709 13:10:18.956804 28520 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:18.959020 28520 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:18.952829 28418 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:10:18.961153 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 307303us
I20260709 13:10:18.916620 28528 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:18.979022 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:18.980330 22872 raft_consensus.cc:2393] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 15: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:18.948189 28343 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:18.959082 28447 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:18.982849 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 102844us
I20260709 13:10:18.984037 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:18.933332 27912 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:18.992002 22992 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:18.994423 22918 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:18.995920 22918 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:18.997539 28441 raft_consensus.cc:2804] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:18.997761 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:18.999542 28420 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:19.005288 22843 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:19.006100 28528 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:19.006583 28447 raft_consensus.cc:2749] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:19.007382 22843 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:19.008354 28447 raft_consensus.cc:2749] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:10:18.961025 28572 raft_consensus.cc:697] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
W20260709 13:10:19.009516 28673 log.cc:927] Time spent T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 13:10:19.009985 28572 consensus_queue.cc:237] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:19.011700 28526 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:19.012383 28526 raft_consensus.cc:2804] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:10:18.997937 28441 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:19.012898 28441 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:19.014173 22993 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:19.005622 22871 raft_consensus.cc:1275] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 13:10:19.004448 28554 consensus_meta.cc:287] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.755s	user 0.007s	sys 0.010s
I20260709 13:10:18.987641 28520 raft_consensus.cc:2804] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:10:19.027812 28450 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:19.028453 28450 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:19.030227 22994 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:19.032829 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:19.032995 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:19.034870 28450 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:19.035044 28664 consensus_queue.cc:1048] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:10:19.033030 28457 raft_consensus.cc:3055] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:10:19.036053 28457 raft_consensus.cc:740] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:19.036237 28529 raft_consensus.cc:2804] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:10:19.036789 28529 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:19.037173 28529 raft_consensus.cc:3060] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:19.037185 28457 consensus_queue.cc:260] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:19.039328 28457 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:19.039624 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:19.040269 23021 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:19.047008 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:19.047780 23025 raft_consensus.cc:2393] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 17: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:19.013231 28418 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:19.045850 23590 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:19.055760 23588 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:19.055637 27912 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:19.057931 28652 raft_consensus.cc:2749] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:19.058276 27912 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:19.012924 28526 raft_consensus.cc:697] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:19.074152 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:10:19.081730 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:19.074517 22920 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:19.084976 22873 raft_consensus.cc:2393] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 16: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
W20260709 13:10:19.085136 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 91982.3us
I20260709 13:10:19.055663 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:19.050343 28528 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:19.063759 28520 raft_consensus.cc:697] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 15 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:19.097651 28526 consensus_queue.cc:237] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:19.095743 28652 raft_consensus.cc:2804] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:10:19.099855 28426 consensus_meta.cc:287] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.519s	user 0.008s	sys 0.005s
W20260709 13:10:19.115590 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 206994us
I20260709 13:10:19.110947 23028 raft_consensus.cc:2468] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
I20260709 13:10:19.114909 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:19.099105 27821 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:19.122187 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 95096.1us
W20260709 13:10:19.123945 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 215752us
I20260709 13:10:19.124991 22951 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
I20260709 13:10:19.119568 28438 raft_consensus.cc:2749] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:19.148643 22843 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:19.150182 28520 consensus_queue.cc:237] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:19.157327 28634 consensus_queue.cc:1059] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.058s
I20260709 13:10:19.158744 28634 raft_consensus.cc:3055] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:10:19.159145 28634 raft_consensus.cc:740] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:19.160140 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:19.165588 22994 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:19.166450 28154 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:19.160195 28634 consensus_queue.cc:260] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:19.167309 28154 consensus_queue.cc:1059] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:10:19.169193 28684 raft_consensus.cc:2804] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:10:19.169670 28684 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:19.175683 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:19.170075 28684 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:19.178682 27678 raft_consensus.cc:2749] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:10:19.185561 28485 raft_consensus.cc:3055] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:10:19.186097 28485 raft_consensus.cc:740] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:19.190627 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:19.190856 28485 consensus_queue.cc:260] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:19.192124 23622 raft_consensus.cc:2393] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 14: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:19.193202 22992 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:19.168545 28634 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:19.118746 28426 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:19.160427 28613 consensus_queue.cc:1059] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.081s
I20260709 13:10:19.207542 23021 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
I20260709 13:10:19.192453 28438 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:19.205785 28450 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:19.202390 28532 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:19.209790 27912 raft_consensus.cc:2749] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:10:19.233134 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:19.237263 28450 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:19.240227 28450 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:19.240968 28529 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:19.244370 28529 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:19.240967 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 56461.5us
I20260709 13:10:19.245828 22843 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:19.246829 22843 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:19.247242 28602 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:19.248191 28602 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:19.248934 28579 raft_consensus.cc:2804] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:10:19.249429 28579 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:19.249809 28579 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:19.252785 28602 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:19.257213 28418 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:19.257668 22951 raft_consensus.cc:1275] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:10:19.212783 28528 raft_consensus.cc:3055] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:10:19.253017 27678 raft_consensus.cc:2804] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:10:19.260281 28438 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:19.269016 27678 raft_consensus.cc:697] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 13 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:19.270619 27678 consensus_queue.cc:237] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:19.272308 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:19.273685 22776 catalog_manager.cc:5697] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 5 to 8, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 8 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
W20260709 13:10:19.289422 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:19.260456 28528 raft_consensus.cc:740] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:19.276787 28529 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:19.294363 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:19.295027 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:19.271778 28438 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:19.292655 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:19.300865 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 158372us
I20260709 13:10:19.282722 28601 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:19.295499 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:19.304677 22949 raft_consensus.cc:2410] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:10:19.276371 28532 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:19.306041 22955 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 16.
I20260709 13:10:19.306154 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:19.309177 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:19.295248 28524 consensus_queue.cc:1048] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.010s
W20260709 13:10:19.311090 23588 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.779s (SENT) [suppressed 18 similar messages]
W20260709 13:10:19.314654 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 339841us
I20260709 13:10:19.315397 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:19.317668 28684 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:19.320693 22844 leader_election.cc:400] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 15. Current term is 16.
I20260709 13:10:19.321420 22844 leader_election.cc:403] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:10:19.311049 22948 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
W20260709 13:10:19.323248 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 309181us
I20260709 13:10:19.323289 22953 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
I20260709 13:10:19.331491 22843 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:19.335493 28369 raft_consensus.cc:3055] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:10:19.335937 28369 raft_consensus.cc:740] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:19.336910 28369 consensus_queue.cc:260] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:19.338151 28369 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:10:19.340828 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:19.332420 28344 raft_consensus.cc:2749] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 15. Current term is 16.
I20260709 13:10:19.344208 28426 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:19.308413 28601 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:19.355126 28601 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:19.356552 28441 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:19.359501 22946 raft_consensus.cc:1240] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 15. Current term is 16. Ops: []
I20260709 13:10:19.358963 28441 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:19.361469 28532 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:19.348616 28528 consensus_queue.cc:260] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:19.373507 28528 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:19.362643 22992 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:19.369500 22993 leader_election.cc:400] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14.
I20260709 13:10:19.369971 28529 raft_consensus.cc:2749] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:19.370409 28684 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:19.377416 22993 leader_election.cc:403] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:10:19.377970 22844 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
W20260709 13:10:19.381470 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 50072.9us
W20260709 13:10:19.376643 28685 consensus_queue.cc:1175] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" responder_term: 12 status { last_received { term: 12 index: 8 } last_committed_idx: 8 last_received_current_leader { term: 12 index: 8 } }
I20260709 13:10:19.383221 28524 raft_consensus.cc:2804] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:19.383846 28062 raft_consensus.cc:2804] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:10:19.389950 28485 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:10:19.387401 22774 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33478 (request call id 278) took 39965 ms (client timeout 15000 ms). Trace:
I20260709 13:10:19.390265 22992 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:19.391080 28664 raft_consensus.cc:2749] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14.
I20260709 13:10:19.432561 28062 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:19.446307 22843 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:19.391937 22993 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:19.392827 28578 consensus_queue.cc:1059] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:10:19.395848 22919 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:19.396605 28579 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:19.388978 28684 raft_consensus.cc:2804] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:19.401190 28418 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:19.388571 22776 catalog_manager.cc:5697] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 6 to 8, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 8 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:19.403105 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:19.404614 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:19.409813 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:19.412644 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:19.413774 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:19.411845 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:19.427085 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:19.462561 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:19.451084 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 59229.7us
I20260709 13:10:19.431806 28524 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:19.460008 22871 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 13.
I20260709 13:10:19.470785 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:19.445762 28529 raft_consensus.cc:2749] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:19.448002 28062 raft_consensus.cc:3060] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:19.477471 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:19.454135 27912 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:19.486310 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:19.487594 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:19.488238 22878 raft_consensus.cc:2468] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 14.
I20260709 13:10:19.489005 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:19.489758 22876 raft_consensus.cc:2393] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 13: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
W20260709 13:10:19.490532 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 94747.1us
I20260709 13:10:19.491341 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:19.491519 22919 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:19.492501 22873 raft_consensus.cc:2393] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 15: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:19.486021 28579 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:19.478315 28389 raft_consensus.cc:2804] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:10:19.455227 28684 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:19.493103 28389 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:19.493289 28684 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:19.486027 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:19.481096 28343 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:19.494235 28343 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:19.496376 28343 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:19.471901 28524 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:19.496801 28438 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:19.497125 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 106951us
I20260709 13:10:19.497354 28438 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:19.490615 28602 raft_consensus.cc:2749] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:10:19.498860 28602 raft_consensus.cc:2749] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:19.499636 22992 leader_election.cc:400] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16.
I20260709 13:10:19.493572 28389 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:19.501739 28602 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:19.502241 28420 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:19.502323 28602 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:19.502988 28420 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:19.442581 22774 rpcz_store.cc:269] 0709 13:09:39.421785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:39.422507 (+   722us) service_pool.cc:224] Handling call
0709 13:10:19.387179 (+39964672us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:10:09.226063 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:10:09.226798 (+   735us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:10:09.226849 (+    51us) write_op.cc:435] Acquired schema lock
0709 13:10:09.226869 (+    20us) tablet.cc:662] Decoding operations
0709 13:10:09.228062 (+  1193us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:10:09.228159 (+    97us) write_op.cc:641] Partition lock acquired for write op
0709 13:10:09.228188 (+    29us) tablet.cc:685] Acquiring locks for 11 operations
0709 13:10:09.274948 (+ 46760us) tablet.cc:701] Row locks acquired
0709 13:10:09.274997 (+    49us) write_op.cc:260] PREPARE: finished
0709 13:10:09.275180 (+   183us) write_op.cc:270] Start()
0709 13:10:09.275330 (+   150us) write_op.cc:276] Timestamp: P: 1783602609275114 usec, L: 0
0709 13:10:09.275372 (+    42us) op_driver.cc:348] REPLICATION: starting
0709 13:10:09.276088 (+   716us) log.cc:844] Serialized 4374 byte log entry
0709 13:10:10.097265 (+821177us) op_driver.cc:464] REPLICATION: finished
0709 13:10:10.626308 (+529043us) write_op.cc:301] APPLY: starting
0709 13:10:10.626416 (+   108us) tablet.cc:1367] starting BulkCheckPresence
0709 13:10:10.639092 (+ 12676us) tablet.cc:1370] finished BulkCheckPresence
0709 13:10:10.639222 (+   130us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:10:11.239170 (+599948us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:10:11.388068 (+148898us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=22,key_file_lookups=22,delta_file_lookups=22,mrs_lookups=0
0709 13:10:11.388138 (+    70us) write_op.cc:312] APPLY: finished
0709 13:10:11.388534 (+   396us) log.cc:844] Serialized 105 byte log entry
0709 13:10:11.388784 (+   250us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:11.388913 (+   129us) write_op.cc:454] Released schema lock
0709 13:10:11.388974 (+    61us) write_op.cc:341] FINISH: Updating metrics
0709 13:10:11.389520 (+   546us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:11.389548 (+    28us) write_op.cc:454] Released schema lock
0709 13:10:12.090046 (+700498us) spinlock_profiling.cc:243] Waited 9.1 ms on lock 0x7b18002bd440. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cf8522 00007f6132cfa7c1 00007f6132cfa779 00007f6132cfa709 00007f6132cfa6d1 00007f6132cf99cd 000000000038f354 000000000038f199 00007f6132cdb83a
Metrics: {"mutex_wait_us":854667,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":517778,"apply.run_cpu_time_us":19180,"apply.run_wall_time_us":763189,"cfile_cache_hit":46,"cfile_cache_hit_bytes":50129,"num_ops":11,"prepare.queue_time_us":102671,"prepare.run_cpu_time_us":19708,"prepare.run_wall_time_us":183967,"raft.queue_time_us":631343,"raft.run_cpu_time_us":36313,"raft.run_wall_time_us":428747,"replication_time_us":821679,"spinlock_wait_cycles":20927360,"thread_start_us":864959,"threads_started":4,"wal-append.queue_time_us":860372,"wal-append.run_cpu_time_us":44819,"wal-append.run_wall_time_us":2518803}]]}
I20260709 13:10:19.480378 22843 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:19.505200 22919 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:19.499573 28438 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:19.518059 22920 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:19.520157 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:19.525730 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:19.527544 22920 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:19.513792 28602 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:19.517277 28579 raft_consensus.cc:2804] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:10:19.533754 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:19.514546 28343 consensus_queue.cc:1059] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 13:10:19.528373 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:19.535342 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:19.538738 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 227671us
I20260709 13:10:19.537791 22876 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:19.539963 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:19.521396 28369 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:19.525326 28685 consensus_queue.cc:1059] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.030s
I20260709 13:10:19.537186 28579 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:19.521431 28418 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:19.542691 28578 raft_consensus.cc:2749] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:10:19.585224 28687 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:19.579157 28369 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:19.586448 28688 log.cc:927] Time spent T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
W20260709 13:10:19.591647 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 86574.3us
I20260709 13:10:19.552029 28602 raft_consensus.cc:2804] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:10:19.622493 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 121699us
I20260709 13:10:19.625000 28420 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:19.638885 22873 raft_consensus.cc:1275] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:10:19.540349 28526 raft_consensus.cc:2749] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:19.581405 28579 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:10:19.659662 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 93116.4us
W20260709 13:10:19.661859 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 94606.1us
W20260709 13:10:19.667240 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:19.637506 28602 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:19.668061 22919 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:19.678476 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 151009us
I20260709 13:10:19.679754 22920 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:19.684873 28578 raft_consensus.cc:2749] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:10:19.685660 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:19.678133 22994 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.047s	sys 0.034s
W20260709 13:10:19.686877 22994 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.047s	sys 0.034s
W20260709 13:10:19.687184 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 520586us
I20260709 13:10:19.689162 28529 consensus_queue.cc:1059] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.014s
I20260709 13:10:19.690412 28529 consensus_queue.cc:1048] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 13:10:19.691892 28529 raft_consensus.cc:3055] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:10:19.692286 28529 raft_consensus.cc:740] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:19.692823 27912 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:19.693392 27912 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:19.697090 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:19.697546 28418 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:19.697834 22992 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:19.698747 28450 raft_consensus.cc:3055] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:10:19.699183 22992 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:19.699550 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:19.700928 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:19.701337 28528 raft_consensus.cc:3055] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:10:19.701910 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:19.701028 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:19.700399 22992 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:19.686331 22919 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:19.671175 28457 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:19.701797 28528 raft_consensus.cc:740] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:19.714458 22920 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:10:19.678481 28652 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.547s	user 0.024s	sys 0.059s
I20260709 13:10:19.720770 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:19.678284 28602 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:19.726485 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:19.700465 28685 raft_consensus.cc:2749] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:10:19.699201 28450 raft_consensus.cc:740] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:19.726647 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:10:19.722553 28652 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.024s	sys 0.060s
I20260709 13:10:19.663401 28687 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:19.741670 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 51529.1us
I20260709 13:10:19.742416 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:19.746765 22948 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:19.748132 22874 raft_consensus.cc:1275] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:10:19.748117 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:19.749469 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:19.750038 22946 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 16.
I20260709 13:10:19.740603 28441 raft_consensus.cc:2804] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:19.745620 28459 raft_consensus.cc:2749] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:19.733991 28693 raft_consensus.cc:2749] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:19.758858 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:19.755641 23623 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 16.
I20260709 13:10:19.744535 28652 raft_consensus.cc:697] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:19.721269 28062 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:19.764098 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:19.759487 23622 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 16.
W20260709 13:10:19.768591 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 68282.5us
I20260709 13:10:19.757403 28441 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:19.758167 27912 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:19.774968 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:19.775084 28701 raft_consensus.cc:2749] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:19.751228 28457 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:19.771523 28529 consensus_queue.cc:260] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:19.779266 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 93002.3us
I20260709 13:10:19.783854 28709 raft_consensus.cc:2804] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 13:10:19.776093 28695 log.cc:927] Time spent T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:10:19.774923 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:19.798048 22844 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:19.803683 22844 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:19.774145 28441 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:19.802129 23624 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 14.
I20260709 13:10:19.811533 23616 raft_consensus.cc:2393] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 15: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
W20260709 13:10:19.809303 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 62664.2us
I20260709 13:10:19.810321 22992 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:19.765777 28578 consensus_queue.cc:1048] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:10:19.815968 22876 raft_consensus.cc:2468] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 16.
I20260709 13:10:19.818578 22879 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 16.
I20260709 13:10:19.800030 28709 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:19.827723 28457 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:19.827891 28709 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:10:19.766315 28528 consensus_queue.cc:260] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:19.829141 28686 log.cc:927] Time spent T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.153s	user 0.003s	sys 0.000s
I20260709 13:10:19.814114 28707 raft_consensus.cc:2804] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:19.841135 22919 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:19.842954 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:10:19.845836 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 61026.9us
I20260709 13:10:19.846704 22992 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:19.734751 27678 raft_consensus.cc:2955] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 17 LEADER]: Committing config change with OpId 12.9: config changed from index 5 to 9, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:19.852638 28709 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:19.861951 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:19.859225 23618 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 17.
W20260709 13:10:19.863246 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:19.829468 28528 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:19.826437 27912 raft_consensus.cc:2804] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:19.869674 23624 raft_consensus.cc:2393] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 16: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:19.835152 28447 raft_consensus.cc:2804] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:19.871574 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:19.856650 28687 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:19.834548 28485 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:19.838913 28450 consensus_queue.cc:260] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:19.869366 28532 consensus_queue.cc:1048] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.022s
I20260709 13:10:19.861467 22918 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:19.878432 23590 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:19.881027 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:19.881716 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:19.840041 28707 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:10:19.894820 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 52182.3us
I20260709 13:10:19.889210 23026 raft_consensus.cc:1275] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:10:19.896941 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 67401.3us
I20260709 13:10:19.889741 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:19.907115 23025 raft_consensus.cc:2468] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 13.
I20260709 13:10:19.915249 22948 raft_consensus.cc:2468] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
I20260709 13:10:19.917145 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:19.919615 22993 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:19.887424 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:19.920619 23024 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 14.
I20260709 13:10:19.806867 28684 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:19.889535 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:19.921130 22918 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:19.950626 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:19.890094 28450 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:19.884963 28529 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:19.958047 22993 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:19.922890 28154 raft_consensus.cc:2749] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:19.884629 28579 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:19.924046 28687 raft_consensus.cc:2804] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:10:19.885753 28062 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:19.961848 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:19.998582 23616 raft_consensus.cc:3060] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:19.949002 28701 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:19.964030 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:19.898986 28707 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:19.886988 28485 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:19.965814 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 63420.7us
I20260709 13:10:19.924825 23022 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 15.
I20260709 13:10:19.890132 28438 raft_consensus.cc:2749] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:19.969578 28664 raft_consensus.cc:2749] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:19.875941 27912 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
W20260709 13:10:20.004474 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 157797us
I20260709 13:10:19.886744 28389 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:19.895116 28706 raft_consensus.cc:2749] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:19.998334 28459 raft_consensus.cc:2804] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:10:19.965322 28369 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:19.968969 28685 raft_consensus.cc:2749] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:10:19.999469 28672 log.cc:927] Time spent T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:10:19.971359 22993 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:10:20.010272 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:20.018778 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:20.020751 28459 raft_consensus.cc:697] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:20.005566 28687 raft_consensus.cc:697] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
W20260709 13:10:20.021945 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 53508.9us
I20260709 13:10:20.023007 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:20.023249 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:20.023461 22877 raft_consensus.cc:2393] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 16: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:20.024322 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:19.971598 28524 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:20.024008 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:20.013195 27912 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:20.027856 22871 raft_consensus.cc:2393] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 14: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:19.885089 28652 consensus_queue.cc:237] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:20.029181 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 75923.7us
I20260709 13:10:20.029848 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:20.036506 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:20.037412 28684 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:20.038704 28459 consensus_queue.cc:237] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:20.041559 28687 consensus_queue.cc:237] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:20.045414 28711 log.cc:927] Time spent T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 13:10:20.046861 28447 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:20.047354 28447 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:20.047446 27912 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:20.049484 28418 raft_consensus.cc:3055] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:10:20.049918 28418 raft_consensus.cc:740] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:20.017887 28701 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:20.026083 28664 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:20.049641 22993 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:20.052635 28701 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:20.052898 27912 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:20.059535 22993 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
W20260709 13:10:20.064038 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:20.069336 23616 raft_consensus.cc:2468] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
I20260709 13:10:20.069875 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:20.069262 27912 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:10:20.072772 23024 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 13.
I20260709 13:10:20.073354 27912 raft_consensus.cc:697] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 17 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:20.059839 28664 raft_consensus.cc:2804] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:20.090202 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:20.091053 23025 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 16.
I20260709 13:10:20.092240 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:20.093302 23025 raft_consensus.cc:2368] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 15. Current term is 16.
W20260709 13:10:20.098212 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:20.081171 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:20.082811 28389 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:20.084555 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:20.086614 28709 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:20.087376 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:20.099268 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:20.100760 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 73859.5us
I20260709 13:10:20.101377 23022 raft_consensus.cc:2393] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 11: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:20.109735 22992 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:20.113957 22918 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:20.115592 22918 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:20.117179 22918 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:20.120219 23021 raft_consensus.cc:1275] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:10:20.114933 22844 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:20.124610 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:20.125363 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 163710us
I20260709 13:10:20.089139 28664 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:20.126781 22843 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:20.125531 27912 consensus_queue.cc:237] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:20.124929 23026 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 16.
I20260709 13:10:20.125522 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:20.127363 22876 raft_consensus.cc:1240] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14. Ops: []
I20260709 13:10:20.130648 22918 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
W20260709 13:10:20.130878 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:20.134302 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:20.134708 22994 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:20.138833 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:20.146641 23026 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:20.147466 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
W20260709 13:10:20.155588 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 122376us
I20260709 13:10:20.157222 22919 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:20.159221 28524 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:20.127235 28526 raft_consensus.cc:2749] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:20.160719 28526 raft_consensus.cc:2749] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:20.160780 22919 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:20.133129 28709 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:20.132647 28664 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:20.161983 28709 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:20.162425 22919 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:20.163584 23623 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:20.154696 22994 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:20.163648 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:20.164804 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 55196.9us
I20260709 13:10:20.137825 28553 consensus_queue.cc:1048] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.028s
I20260709 13:10:20.167100 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:20.164412 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:20.133157 28154 raft_consensus.cc:2749] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:20.146872 28062 raft_consensus.cc:2804] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:10:20.173458 22992 leader_election.cc:400] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16.
W20260709 13:10:20.172331 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 57482.3us
I20260709 13:10:20.126614 28602 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:20.152999 28459 raft_consensus.cc:2804] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:20.169848 28579 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:20.179201 28703 raft_consensus.cc:2804] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:20.190146 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:20.169368 28526 raft_consensus.cc:2749] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:20.173100 28485 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:20.133488 28707 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:20.172868 28702 raft_consensus.cc:2749] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:20.194061 23027 raft_consensus.cc:2393] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 17: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:20.189827 28459 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:20.179556 28520 raft_consensus.cc:2804] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:10:20.198832 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:20.203599 23026 raft_consensus.cc:2468] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
I20260709 13:10:20.195678 28706 raft_consensus.cc:2749] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:10:20.208191 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:20.204262 28459 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:20.210278 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:20.211395 28707 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:20.195374 28728 raft_consensus.cc:3060] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:20.206959 28062 raft_consensus.cc:697] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 14 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:20.213485 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:20.214116 23024 raft_consensus.cc:2468] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
I20260709 13:10:20.213925 28062 consensus_queue.cc:237] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:20.217397 28709 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:20.218935 22872 raft_consensus.cc:2393] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 17: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:20.221624 22843 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:20.221990 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:20.222893 23026 raft_consensus.cc:2410] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 13:10:20.192359 28703 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
W20260709 13:10:20.230223 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 75593.7us
W20260709 13:10:20.231331 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 64818.1us
I20260709 13:10:20.234243 28718 raft_consensus.cc:2749] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:10:20.235394 28723 log.cc:927] Time spent T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 13:10:20.237200 28520 raft_consensus.cc:697] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 13 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:20.237864 28602 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:20.238269 28520 consensus_queue.cc:237] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 11.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:20.239933 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 66600.9us
I20260709 13:10:20.231642 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:20.220080 28441 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:20.245002 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:20.245375 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:20.232966 23026 raft_consensus.cc:1240] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 10. Current term is 11. Ops: []
I20260709 13:10:20.258875 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:20.259718 22878 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:20.265646 23026 raft_consensus.cc:1275] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:10:20.269349 28664 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:20.229766 28685 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:10:20.271548 22993 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:20.276888 23022 raft_consensus.cc:2368] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 14. Current term is 15.
W20260709 13:10:20.276940 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 55386.9us
I20260709 13:10:20.279999 23026 raft_consensus.cc:1275] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:10:20.284448 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:20.286604 22875 raft_consensus.cc:1240] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:20.287216 22946 raft_consensus.cc:1275] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:10:20.288337 22952 raft_consensus.cc:1275] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:10:20.288679 22950 raft_consensus.cc:1275] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:10:20.231773 28703 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:20.292311 23028 raft_consensus.cc:1275] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:10:20.292596 28714 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 13:10:20.292510 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:20.293440 22955 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:20.297323 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:20.301223 22955 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
I20260709 13:10:20.301771 28664 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:20.302855 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:20.303682 23616 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:20.270727 28602 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:20.305038 28602 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:20.305553 22992 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:20.307091 28602 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:20.307456 28729 raft_consensus.cc:2749] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:20.308569 22992 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:20.309688 22948 raft_consensus.cc:1275] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:10:20.310010 22994 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:20.313695 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:20.314496 22955 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:20.317324 22843 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:20.313761 22920 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:20.320420 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:20.323201 28441 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:20.288854 22953 raft_consensus.cc:2955] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Committing config change with OpId 12.9: config changed from index 5 to 9, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:20.327817 23616 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 15.
I20260709 13:10:20.330113 28447 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:20.335590 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:20.308269 28154 consensus_queue.cc:1048] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 13:10:20.345368 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:20.353754 23027 raft_consensus.cc:3060] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 13
I20260709 13:10:20.276696 28709 consensus_queue.cc:1059] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 13:10:20.362869 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:20.336782 28420 raft_consensus.cc:2804] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:10:20.334379 28459 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:20.339385 28447 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:20.344282 28664 raft_consensus.cc:2749] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:10:20.332229 28418 consensus_queue.cc:260] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:20.344509 28714 raft_consensus.cc:2749] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:20.403362 22955 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
I20260709 13:10:20.362763 28685 consensus_queue.cc:1059] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.039s
W20260709 13:10:20.408075 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 90867.6us
I20260709 13:10:20.365629 28728 raft_consensus.cc:2749] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:20.399760 28520 raft_consensus.cc:2804] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:10:20.417332 22843 leader_election.cc:400] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 15. Current term is 16.
I20260709 13:10:20.422375 22843 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:20.413749 28420 raft_consensus.cc:697] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:20.428716 28719 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:20.431640 22843 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:20.426831 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:20.434254 23023 raft_consensus.cc:2410] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 19 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
W20260709 13:10:20.435077 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:20.436328 22878 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 14.
I20260709 13:10:20.418701 28418 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:20.410336 28729 consensus_queue.cc:1048] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.032s
I20260709 13:10:20.429354 28719 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:20.448035 28447 raft_consensus.cc:2749] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:20.430287 28520 raft_consensus.cc:697] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 13 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:20.448603 28710 raft_consensus.cc:2749] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:10:20.449507 28520 consensus_queue.cc:237] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:20.451481 28703 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:20.453094 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:20.459816 23027 raft_consensus.cc:2468] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
I20260709 13:10:20.440707 22843 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:20.463630 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 333029us
I20260709 13:10:20.465114 28710 raft_consensus.cc:2749] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:20.466353 22843 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:20.466449 22918 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:20.467836 22918 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:20.467855 28701 raft_consensus.cc:2804] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:10:20.469215 28438 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:20.469556 28710 raft_consensus.cc:2749] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:20.469861 22918 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:20.469871 28438 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:20.470597 28438 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:20.470723 28450 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:10:20.470873 22918 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:20.471601 28710 consensus_queue.cc:1059] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:10:20.469310 28701 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:20.472477 22918 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:20.472946 28447 raft_consensus.cc:3055] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 10 LEADER]: Stepping down as leader of term 10
I20260709 13:10:20.473070 28701 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:20.473378 28447 raft_consensus.cc:740] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:20.473762 22918 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:20.474574 28526 raft_consensus.cc:2749] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:20.475265 28554 raft_consensus.cc:2749] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:10:20.474323 28447 consensus_queue.cc:260] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:20.475941 28554 raft_consensus.cc:2804] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:20.476341 28554 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:20.476423 28578 consensus_queue.cc:1048] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 13:10:20.477027 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:20.479859 27678 consensus_queue.cc:1048] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:10:20.481170 28554 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:20.482597 27678 consensus_queue.cc:1048] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:10:20.484314 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:20.488152 27678 consensus_queue.cc:1048] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 13:10:20.489267 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 164081us
I20260709 13:10:20.489457 23623 raft_consensus.cc:1275] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:10:20.490310 22843 leader_election.cc:400] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 14. Current term is 15.
I20260709 13:10:20.490696 22843 leader_election.cc:403] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:10:20.492224 22843 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:20.490057 22993 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:20.494998 27678 raft_consensus.cc:2749] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:20.461014 28369 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
W20260709 13:10:20.504058 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 341669us
W20260709 13:10:20.503345 28529 consensus_meta.cc:287] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.529s	user 0.011s	sys 0.000s
I20260709 13:10:20.500391 28707 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:10:20.507154 28369 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:20.516870 28703 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:20.518544 22946 raft_consensus.cc:1240] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:10:20.520285 28343 consensus_queue.cc:1048] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:10:20.521613 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:20.521772 22843 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:20.522320 22950 raft_consensus.cc:1240] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:10:20.510680 28719 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:20.504812 28712 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
W20260709 13:10:20.522917 28737 log.cc:927] Time spent T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 13:10:20.529563 22953 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:20.524525 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:20.534114 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 199639us
I20260709 13:10:20.535715 22955 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 17.
I20260709 13:10:20.537379 22949 raft_consensus.cc:1240] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:10:20.519588 28718 consensus_queue.cc:1048] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 13:10:20.540850 28715 consensus_queue.cc:1048] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.004s
I20260709 13:10:20.544742 28719 consensus_queue.cc:1048] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:10:20.546211 28712 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:20.551267 28450 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:10:20.526310 28420 consensus_queue.cc:237] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:20.537958 22843 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:20.558516 28719 raft_consensus.cc:2804] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:10:20.558382 28701 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:20.559155 28719 raft_consensus.cc:697] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 14 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:20.560027 28719 consensus_queue.cc:237] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:20.563372 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:20.566468 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:20.562322 28369 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:20.548583 28712 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:20.579530 28447 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 10 FOLLOWER]: Advancing to term 11
I20260709 13:10:20.580296 28438 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:20.591197 28459 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:20.533921 28062 raft_consensus.cc:2804] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:10:20.617305 22953 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 20.
I20260709 13:10:20.619304 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:20.623018 22950 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:20.564553 28701 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:20.612854 28438 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:20.619769 28062 raft_consensus.cc:697] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 14 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:20.619565 28652 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:20.649611 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:20.655032 28736 log.cc:927] Time spent T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.215s	user 0.003s	sys 0.004s
I20260709 13:10:20.681803 22953 raft_consensus.cc:1240] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 15. Current term is 16. Ops: []
I20260709 13:10:20.682161 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:20.682804 22948 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 18.
I20260709 13:10:20.682981 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:20.683341 22952 raft_consensus.cc:1240] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:20.689438 28652 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:20.704236 22950 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 17.
I20260709 13:10:20.609203 28554 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:20.706550 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 216536us
I20260709 13:10:20.639092 28707 raft_consensus.cc:2749] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 14. Current term is 15.
I20260709 13:10:20.715879 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:20.721192 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:20.722208 22949 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:20.705897 28154 raft_consensus.cc:3055] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:10:20.730096 28154 raft_consensus.cc:740] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:20.731148 28154 consensus_queue.cc:260] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:20.737833 28652 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:20.738577 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:20.739357 22946 raft_consensus.cc:2393] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 17: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:20.740192 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:20.746423 22843 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:20.746340 28062 consensus_queue.cc:237] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:20.748826 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 194660us
I20260709 13:10:20.752983 28750 consensus_queue.cc:1048] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 13:10:20.753669 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:20.753847 28750 consensus_queue.cc:1048] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:10:20.754915 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:20.755964 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:20.756729 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:20.757077 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:20.757768 22953 raft_consensus.cc:2393] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 16: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:20.757813 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:20.758482 22951 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 15.
I20260709 13:10:20.758579 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:20.759274 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:20.760285 28712 consensus_queue.cc:1059] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:10:20.728701 28719 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:20.761137 28712 raft_consensus.cc:3055] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:10:20.761639 28712 raft_consensus.cc:740] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:20.761416 28719 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:20.756567 22947 raft_consensus.cc:2368] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14.
I20260709 13:10:20.762476 28712 consensus_queue.cc:260] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:20.763552 28719 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:20.763625 28712 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:20.760553 22951 raft_consensus.cc:1275] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:10:20.765606 22993 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:10:20.766539 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 59351.7us
I20260709 13:10:20.767858 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:20.771347 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:20.771979 22951 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
I20260709 13:10:20.773201 22843 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:20.776695 22953 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 18.
I20260709 13:10:20.783198 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:20.776871 22950 raft_consensus.cc:1240] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:10:20.744557 28745 raft_consensus.cc:2804] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:10:20.778811 23025 raft_consensus.cc:1275] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 13:10:20.792984 23028 raft_consensus.cc:1275] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 13:10:20.780256 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:20.787606 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:20.803481 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:20.803859 22920 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:20.804147 23617 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:20.805063 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:20.808233 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:20.808775 22880 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:20.808933 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:20.809671 22872 raft_consensus.cc:2393] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 15: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:20.811726 28578 raft_consensus.cc:2804] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:10:20.812263 28578 raft_consensus.cc:697] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:20.813192 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:20.813144 28578 consensus_queue.cc:237] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:20.817641 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:20.818413 22875 raft_consensus.cc:2393] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 13: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:20.819407 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:20.821108 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:20.821858 22953 raft_consensus.cc:2368] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 14. Current term is 15.
I20260709 13:10:20.765343 28447 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:20.778283 28715 raft_consensus.cc:2804] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:10:20.825623 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:20.826195 23025 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 17.
I20260709 13:10:20.831462 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:20.832612 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:20.787624 28750 raft_consensus.cc:2749] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:10:20.818650 28718 consensus_queue.cc:1048] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.011s
I20260709 13:10:20.836242 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:20.836536 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:20.835047 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:20.835513 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:20.839104 22954 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:20.841114 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:20.821127 28745 raft_consensus.cc:697] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:20.828240 28634 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:20.824031 28447 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:20.828442 28715 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:20.881507 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:20.882438 23029 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:20.885563 23617 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 17.
I20260709 13:10:20.894124 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:20.894881 22953 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 15.
I20260709 13:10:20.858201 28447 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:20.852393 28715 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:20.906010 23020 raft_consensus.cc:1240] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:10:20.908617 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:20.916742 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:20.917626 23020 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 15
W20260709 13:10:20.923712 28753 log.cc:927] Time spent T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:10:20.924990 28745 consensus_queue.cc:237] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:20.925755 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:20.871928 28634 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:20.936798 22949 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 17.
I20260709 13:10:20.942858 22875 raft_consensus.cc:1275] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:10:20.943755 22843 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:20.946496 23028 raft_consensus.cc:1240] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:10:20.947180 22947 raft_consensus.cc:1240] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:10:20.949321 22918 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:20.949892 22949 raft_consensus.cc:1275] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:10:20.951579 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:20.962690 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:20.963526 23623 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 17.
W20260709 13:10:20.967497 28746 log.cc:927] Time spent T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.428s	user 0.002s	sys 0.000s
I20260709 13:10:20.969748 22954 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 17.
I20260709 13:10:20.970424 22953 raft_consensus.cc:1240] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:20.972796 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:20.972922 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:20.973896 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:20.974469 22953 raft_consensus.cc:2376] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 17. Re-sending same reply.
I20260709 13:10:20.975368 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:20.975865 22953 raft_consensus.cc:2376] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 20. Re-sending same reply.
I20260709 13:10:20.974190 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:20.976651 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:20.985899 22955 raft_consensus.cc:1275] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:10:20.961850 28718 consensus_queue.cc:1048] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 13:10:20.977725 28756 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:20.988183 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:20.988960 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:20.990581 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:20.960114 28554 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:20.958551 28520 raft_consensus.cc:2749] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:10:20.965049 28578 raft_consensus.cc:2749] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:20.983529 28750 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:20.962431 28418 consensus_queue.cc:1048] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 13:10:21.021253 28634 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:21.031086 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:21.032020 23624 raft_consensus.cc:2368] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 16. Current term is 17.
I20260709 13:10:21.035243 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:21.037334 22949 raft_consensus.cc:1240] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14. Ops: []
I20260709 13:10:21.038691 23616 raft_consensus.cc:1240] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 18. Current term is 19. Ops: []
W20260709 13:10:21.041559 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:21.041568 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:21.043557 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:21.044227 23026 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 17.
I20260709 13:10:21.045123 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:21.046180 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:21.046939 22873 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 16.
I20260709 13:10:21.048103 23026 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 17.
I20260709 13:10:20.997243 28756 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:21.049789 23022 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 13.
I20260709 13:10:21.050446 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:21.051113 28756 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:21.051949 22844 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:21.052937 28344 raft_consensus.cc:2804] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:10:21.053467 28344 raft_consensus.cc:697] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 17 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:21.054440 28344 consensus_queue.cc:237] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:21.060907 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:10:21.062692 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:21.063560 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:21.063258 23020 raft_consensus.cc:2468] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 15.
I20260709 13:10:21.072604 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:21.074162 22918 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:21.081075 23029 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 14.
I20260709 13:10:21.081715 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:21.081075 22880 raft_consensus.cc:2468] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 15.
I20260709 13:10:21.094337 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:21.095371 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:21.096203 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:21.093379 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:21.099236 22875 raft_consensus.cc:1275] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:10:21.105198 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:21.105787 23618 raft_consensus.cc:2376] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 14. Re-sending same reply.
I20260709 13:10:21.107659 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:21.108246 23618 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 15.
I20260709 13:10:20.996927 22955 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:39742
I20260709 13:10:21.109612 22955 raft_consensus.cc:606] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:21.110018 23624 raft_consensus.cc:2393] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 17: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:21.110040 22955 raft_consensus.cc:614] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:10:21.112569 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:21.081910 22992 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:21.122309 23617 raft_consensus.cc:1240] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 14. Current term is 15. Ops: []
I20260709 13:10:21.124406 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:21.126590 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:21.127254 22949 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 17.
I20260709 13:10:21.127655 28426 consensus_queue.cc:1059] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:10:21.087857 28554 raft_consensus.cc:2804] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:10:21.092322 28707 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:21.011497 28750 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:21.135219 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:21.137362 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:21.143997 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:21.146027 23020 raft_consensus.cc:2393] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 11: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:21.146751 23025 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 13.
I20260709 13:10:21.147078 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:21.147771 23029 raft_consensus.cc:2393] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 16: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:21.147712 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:21.149186 22918 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:21.150617 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:21.151592 28750 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:21.158288 23025 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 20.
I20260709 13:10:21.159701 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:21.161103 23025 raft_consensus.cc:2376] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 20. Re-sending same reply.
I20260709 13:10:21.138108 28554 raft_consensus.cc:697] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:21.176023 23026 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 17.
I20260709 13:10:21.177517 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:21.120258 28634 consensus_queue.cc:1059] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 13:10:21.179004 23617 raft_consensus.cc:1275] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:10:21.179347 28764 log.cc:927] Time spent T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:10:21.171965 28441 raft_consensus.cc:2804] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:10:21.154452 28707 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:21.183382 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:21.184188 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:21.184273 23028 raft_consensus.cc:2468] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 16.
W20260709 13:10:21.185868 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 113054us
I20260709 13:10:21.205688 23029 raft_consensus.cc:2393] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 16: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:21.205873 22992 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:21.206022 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:21.130877 28756 raft_consensus.cc:2749] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:21.209362 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:21.210907 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:21.212157 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:21.208410 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:21.198330 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:21.177662 28554 consensus_queue.cc:237] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:21.199097 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 72506us
I20260709 13:10:21.200363 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:21.206501 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:21.223707 23026 raft_consensus.cc:2393] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 16: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:21.244923 23024 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:21.247210 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:21.249863 22992 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:21.253595 23025 raft_consensus.cc:1240] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:10:21.254837 23025 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:10:21.255416 23025 raft_consensus.cc:606] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:21.255546 23023 raft_consensus.cc:1275] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:10:21.255800 23025 raft_consensus.cc:614] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:10:21.256358 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:21.208839 28693 raft_consensus.cc:2749] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:21.260059 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:21.208750 28441 raft_consensus.cc:697] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:21.266901 28154 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:21.274463 23588 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:21.284037 23588 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:21.260919 28634 raft_consensus.cc:3055] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:10:21.286599 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 152923us
I20260709 13:10:21.287575 23020 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 15.
I20260709 13:10:21.291795 22872 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 17.
I20260709 13:10:21.261348 28709 raft_consensus.cc:2749] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:10:21.297065 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 55371.1us
I20260709 13:10:21.310418 22992 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:21.316651 28441 consensus_queue.cc:237] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:21.294049 28634 raft_consensus.cc:740] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:21.378158 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:21.297925 28062 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:21.330019 28685 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:21.383056 22880 raft_consensus.cc:1240] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:21.386137 23025 raft_consensus.cc:1240] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 17 LEADER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 16. Current term is 17. Ops: []
W20260709 13:10:21.391947 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 93292.3us
I20260709 13:10:21.393036 22992 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:21.415998 23025 raft_consensus.cc:1240] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 12. Current term is 13. Ops: []
I20260709 13:10:21.392632 28719 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:21.431906 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 147963us
I20260709 13:10:21.432288 28781 raft_consensus.cc:2804] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:10:21.434996 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 158429us
I20260709 13:10:21.438860 23025 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f603724630e44ef78014e7719f096cb3"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:10:21.375752 28771 consensus_queue.cc:1059] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.093s
I20260709 13:10:21.410823 28745 raft_consensus.cc:2804] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:21.411932 28772 raft_consensus.cc:2749] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:21.455906 23021 raft_consensus.cc:1240] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 11. Current term is 13. Ops: []
I20260709 13:10:21.452509 23029 raft_consensus.cc:1240] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:21.446264 28745 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:21.441341 28781 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:21.414932 28578 consensus_queue.cc:1048] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 13:10:21.411067 28062 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:21.471630 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 177133us
W20260709 13:10:21.472633 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 60109.9us
I20260709 13:10:21.457770 28344 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:21.443289 28719 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:21.474887 28766 log.cc:927] Time spent T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
I20260709 13:10:21.467417 28745 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:21.495581 22992 leader_election.cc:400] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14.
I20260709 13:10:21.467695 28781 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:21.514703 22992 leader_election.cc:403] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:10:21.495069 23029 raft_consensus.cc:1275] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:10:21.462196 28707 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:21.519328 28062 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:21.520329 28783 consensus_queue.cc:1048] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:10:21.454339 28634 consensus_queue.cc:260] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:21.526109 28768 log.cc:927] Time spent T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 13:10:21.542866 27912 raft_consensus.cc:2749] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14.
I20260709 13:10:21.546272 23025 raft_consensus.cc:606] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:21.546869 23025 raft_consensus.cc:614] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:10:21.561985 23024 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 17.
W20260709 13:10:21.565521 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 86146.6us
W20260709 13:10:21.567052 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 70219.4us
W20260709 13:10:21.569473 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 131942us
I20260709 13:10:21.525880 28344 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:21.571367 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 96998.9us
W20260709 13:10:21.572096 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 133947us
I20260709 13:10:21.573381 22880 raft_consensus.cc:1275] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 13:10:21.566846 28447 consensus_meta.cc:287] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.652s	user 0.013s	sys 0.007s
I20260709 13:10:21.532752 28756 consensus_queue.cc:1059] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 13:10:21.579322 22918 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
W20260709 13:10:21.566634 28715 consensus_meta.cc:287] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.652s	user 0.006s	sys 0.006s
I20260709 13:10:21.582782 22992 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:21.589040 22879 raft_consensus.cc:1275] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 13:10:21.599591 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:21.603452 27912 raft_consensus.cc:2804] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:10:21.582477 28693 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:21.608511 23588 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:21.682749 28771 consensus_queue.cc:1243] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer aed3f089ec8b49af9c8cf5cdee76e8e7 log is divergent from this leader: its last log entry 14.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:10:21.591723 28715 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:21.759095 28715 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:21.617316 28709 consensus_queue.cc:1048] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.091s
I20260709 13:10:21.589066 28447 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:21.762145 28447 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:21.598601 28789 consensus_queue.cc:1048] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 5, Time since last communication: 0.107s
I20260709 13:10:21.762473 28344 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:21.765550 22878 raft_consensus.cc:1240] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:10:21.659581 28693 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:21.593222 28634 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:21.767624 23026 raft_consensus.cc:1275] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 13:10:21.770928 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 204070us
I20260709 13:10:21.706163 22873 raft_consensus.cc:1275] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:10:21.740374 27912 raft_consensus.cc:697] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 18 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:21.696269 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:10:21.697275 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 117135us
I20260709 13:10:21.774019 22992 leader_election.cc:400] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 14. Current term is 15.
I20260709 13:10:21.774528 22992 leader_election.cc:403] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:10:21.604825 28787 raft_consensus.cc:2804] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:10:21.748955 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:21.749452 22920 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:21.695986 22880 raft_consensus.cc:1240] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 17 LEADER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 16. Current term is 17. Ops: []
W20260709 13:10:21.659866 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:21.752694 28728 raft_consensus.cc:3055] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:10:21.746417 28154 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
W20260709 13:10:21.777489 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:21.777148 22992 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:21.780589 22880 raft_consensus.cc:1275] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:10:21.780871 22992 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:21.782820 22992 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:21.771714 28687 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:21.787303 23590 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:21.746747 28796 consensus_queue.cc:1059] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 13:10:21.781966 28794 raft_consensus.cc:2749] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:10:21.784216 28154 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:21.785521 28344 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:21.786243 28806 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:21.766288 28756 consensus_queue.cc:1048] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 13:10:21.782148 28805 raft_consensus.cc:2749] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 14. Current term is 15.
I20260709 13:10:21.800400 28706 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:10:21.793247 22992 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:10:21.796852 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 188448us
I20260709 13:10:21.775064 28719 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:21.805550 28791 consensus_queue.cc:1059] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.012s
I20260709 13:10:21.794128 27912 consensus_queue.cc:237] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:21.775511 28780 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:21.782611 28789 raft_consensus.cc:2749] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:21.778841 28728 raft_consensus.cc:740] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:21.810297 22873 raft_consensus.cc:1240] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:21.876447 23588 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:21.812249 28808 raft_consensus.cc:2749] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:10:21.878796 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 94413.5us
I20260709 13:10:21.807336 28799 consensus_queue.cc:1048] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.076s
I20260709 13:10:21.825776 28687 raft_consensus.cc:2804] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:10:21.894222 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 114154us
W20260709 13:10:21.896090 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 59804.7us
I20260709 13:10:21.814107 28792 consensus_queue.cc:1048] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 13:10:21.811136 28745 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:21.811203 28344 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:21.900324 28789 consensus_queue.cc:1059] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:10:21.901165 28745 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:21.901440 28344 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:21.901784 28745 consensus_queue.cc:1059] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:10:21.903002 28819 raft_consensus.cc:3055] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:10:21.903378 28819 raft_consensus.cc:740] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:21.903898 28745 raft_consensus.cc:3055] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:10:21.904296 28745 raft_consensus.cc:740] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:21.905489 28745 consensus_queue.cc:260] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:21.899044 23588 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:21.873346 28447 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:21.917573 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:21.934506 23588 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:21.812767 28806 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:21.898620 28687 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:21.908488 28794 raft_consensus.cc:2804] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:10:21.930321 28706 raft_consensus.cc:697] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:21.904398 28819 consensus_queue.cc:260] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:21.920106 28802 raft_consensus.cc:2804] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:10:21.922176 28787 raft_consensus.cc:697] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:21.947320 23588 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:21.942420 28771 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:10:21.967304 28771 raft_consensus.cc:697] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 20 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:21.932641 28447 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:21.945799 28687 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:21.959015 23617 raft_consensus.cc:1275] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:10:21.886703 28780 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:21.961143 28745 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:21.961757 28819 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:21.962929 28799 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 13:10:21.963662 28706 consensus_queue.cc:237] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:21.949767 28809 raft_consensus.cc:2749] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:21.948415 28728 consensus_queue.cc:260] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:21.957610 28797 log.cc:927] Time spent T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.170s	user 0.000s	sys 0.000s
I20260709 13:10:21.970147 23616 raft_consensus.cc:1275] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 11 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
W20260709 13:10:21.980064 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 165282us
I20260709 13:10:21.984386 23028 raft_consensus.cc:1275] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:10:21.952917 28794 raft_consensus.cc:697] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 17 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:21.995234 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:21.996331 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:21.998845 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:21.999233 23616 raft_consensus.cc:1275] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:10:22.000590 28154 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:22.001814 28816 consensus_queue.cc:1048] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:10:22.002048 28780 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:21.968312 28771 consensus_queue.cc:237] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:22.002660 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:22.003302 23028 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
I20260709 13:10:22.003490 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:22.004464 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:22.005276 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:22.005445 23028 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 16.
I20260709 13:10:22.006348 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:22.010831 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:22.027875 23028 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15.
I20260709 13:10:22.015893 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:22.016604 22844 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:22.017999 28447 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:22.018922 22951 raft_consensus.cc:1275] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:10:22.016614 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:22.018030 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:22.019558 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:22.019709 22949 raft_consensus.cc:1240] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 14. Current term is 15. Ops: []
I20260709 13:10:21.975819 28802 raft_consensus.cc:697] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 17 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:22.020233 28781 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:22.023104 28685 consensus_meta.cc:287] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.631s	user 0.009s	sys 0.000s
I20260709 13:10:22.018529 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:22.054900 22879 raft_consensus.cc:1813] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Handling vote request from an unknown peer aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:22.055630 22879 raft_consensus.cc:2410] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:10:22.034178 22950 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:22.034891 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:22.056869 22948 raft_consensus.cc:2393] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 18: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:22.036867 23020 raft_consensus.cc:1275] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 13:10:22.041625 22946 raft_consensus.cc:1275] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:10:22.041807 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:22.041894 28802 consensus_queue.cc:237] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:22.042528 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:22.061035 23021 raft_consensus.cc:2393] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 13: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:22.062554 22843 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:22.043156 28806 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:22.058542 28715 raft_consensus.cc:2804] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:10:22.084522 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 286574us
I20260709 13:10:22.058348 28685 raft_consensus.cc:2749] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:22.057866 28520 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:22.095785 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:22.097007 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:22.097648 22953 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 20.
I20260709 13:10:22.070825 28745 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:22.099802 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 303694us
I20260709 13:10:22.105484 28447 consensus_queue.cc:1048] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:10:22.106269 28801 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:22.108458 28745 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:22.112792 22919 leader_election.cc:400] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 16. Current term is 17.
I20260709 13:10:22.117980 22950 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 19.
I20260709 13:10:22.092916 28719 raft_consensus.cc:2804] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:10:22.118579 22919 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:22.088737 28715 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:22.071130 28062 consensus_queue.cc:1048] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:10:22.068205 28827 consensus_queue.cc:1059] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.058s
I20260709 13:10:22.123663 22919 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:22.122633 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:22.126410 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:22.130023 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 83421.6us
W20260709 13:10:22.131860 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 74093.7us
I20260709 13:10:22.115260 28791 consensus_queue.cc:1243] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer 0a407504ed1741bf89a276558adf4c0f log is divergent from this leader: its last log entry 17.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 13:10:22.119467 28801 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:22.139818 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:22.130874 22992 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:22.140983 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:22.141425 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:22.141059 23618 raft_consensus.cc:1240] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:22.140471 23617 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 13.
I20260709 13:10:22.126166 28715 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:22.160234 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:22.175000 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:22.155833 28815 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:10:22.100205 28520 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:22.178544 28815 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:22.178943 28815 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:22.176532 23625 raft_consensus.cc:3055] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:10:22.179682 23625 raft_consensus.cc:740] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:22.180292 28520 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:22.122061 28811 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:22.195081 23624 raft_consensus.cc:2368] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16.
I20260709 13:10:22.188120 22992 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:22.195353 22947 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 15.
I20260709 13:10:22.130234 28719 raft_consensus.cc:697] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 17 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:22.068282 28787 consensus_queue.cc:237] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:22.180539 23625 consensus_queue.cc:260] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:22.124950 28780 consensus_queue.cc:1059] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.054s
I20260709 13:10:22.199862 22992 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:22.211206 23625 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:22.180918 28520 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:22.214926 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 151013us
W20260709 13:10:22.220628 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 83642.9us
I20260709 13:10:22.206959 28154 raft_consensus.cc:2749] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:22.198192 28796 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:22.197904 28799 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:22.212742 28791 raft_consensus.cc:2749] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:22.210683 28827 raft_consensus.cc:2804] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:10:22.115841 28794 consensus_queue.cc:237] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:22.214058 28829 raft_consensus.cc:2749] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:10:22.247840 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 124252us
I20260709 13:10:22.222548 28344 consensus_queue.cc:1048] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 13:10:22.244626 28062 consensus_queue.cc:1059] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
W20260709 13:10:22.275065 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:22.277210 28843 consensus_queue.cc:1048] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 13:10:22.276427 22919 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:22.283466 22919 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:22.252812 28827 raft_consensus.cc:697] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:22.242174 28796 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:22.224551 28520 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:22.301192 23023 raft_consensus.cc:1275] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 13:10:22.277712 28831 log.cc:927] Time spent T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:10:22.209714 28811 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:22.276378 28830 log.cc:927] Time spent T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 13:10:22.263752 28791 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:22.274108 28781 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:22.358599 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 96955.8us
I20260709 13:10:22.363044 23588 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:22.360052 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 76661.7us
I20260709 13:10:22.249361 28799 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:22.303968 28829 raft_consensus.cc:2804] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:22.265981 28418 consensus_queue.cc:1048] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.092s
I20260709 13:10:22.365240 28780 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:22.391376 28780 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:22.391783 22993 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:22.392050 28756 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:22.392570 28756 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:22.393816 28780 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:22.394215 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:22.265365 28719 consensus_queue.cc:237] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:22.395262 28756 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:22.381410 28844 raft_consensus.cc:2749] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:22.369686 22919 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:22.383078 28791 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:22.398403 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:22.398593 28791 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:22.384711 28832 log.cc:927] Time spent T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 13:10:22.399022 23025 raft_consensus.cc:1813] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Handling vote request from an unknown peer aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:22.399566 23025 raft_consensus.cc:2393] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 13: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:22.384835 28520 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:22.386159 23625 raft_consensus.cc:1275] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 13:10:22.382200 28811 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:22.392970 28154 raft_consensus.cc:2804] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:10:22.400444 28154 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:22.400588 23624 raft_consensus.cc:1275] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:10:22.400893 28154 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:22.401640 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:22.395323 23021 raft_consensus.cc:1275] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 13:10:22.373980 28799 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:22.382161 28827 consensus_queue.cc:237] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:22.405591 23588 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:22.417840 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:22.422046 21756 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:10:22.425446 28801 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:22.428927 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 177211us
I20260709 13:10:22.429203 22993 leader_election.cc:400] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16.
I20260709 13:10:22.360999 28745 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:22.431010 22993 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:22.369659 28829 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:22.439240 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:22.441676 22954 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 14.
I20260709 13:10:22.444000 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:22.444638 22875 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11.
I20260709 13:10:22.444841 28829 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:22.445297 28775 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:22.445837 28775 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:22.447916 28775 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:22.450452 28796 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:22.451943 28775 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:22.419550 28062 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:22.452541 28775 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:22.453181 28062 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:22.454938 28775 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:22.455439 28062 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:22.428165 28772 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 13:10:22.456773 28745 raft_consensus.cc:2749] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:22.457401 28791 raft_consensus.cc:2749] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:22.398613 28687 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:22.458190 28819 consensus_queue.cc:1048] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:10:22.459157 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:22.459599 28687 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:22.460325 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:22.428813 28418 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:22.429633 28652 raft_consensus.cc:2749] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:22.472632 23588 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:22.430014 28634 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:10:22.479720 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 227191us
I20260709 13:10:22.483016 23588 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:22.483695 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:22.484233 23623 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 10.
I20260709 13:10:22.486074 22992 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:22.487762 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:22.487628 28802 raft_consensus.cc:2749] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:10:22.489802 22954 raft_consensus.cc:1275] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:10:22.488389 22953 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11.
W20260709 13:10:22.506685 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 142472us
I20260709 13:10:22.508381 22919 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:22.504832 23022 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32794 (request call id 1015) took 1468 ms (client timeout 1520 ms). Trace:
W20260709 13:10:22.510367 23022 rpcz_store.cc:269] 0709 13:10:21.029569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:21.034970 (+  5401us) service_pool.cc:224] Handling call
0709 13:10:22.497764 (+1462794us) spinlock_profiling.cc:243] Waited 13.8 ms on lock 0x7b5400135454. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:22.497932 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31751552}
I20260709 13:10:22.498157 28818 raft_consensus.cc:2749] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:10:22.520889 22843 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:22.522118 22843 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:22.523231 28344 raft_consensus.cc:2804] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:10:22.523627 28344 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:22.523655 28850 raft_consensus.cc:2804] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:22.524156 28344 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:22.528036 28520 consensus_queue.cc:1048] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:10:22.519263 22872 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:22.533222 28788 raft_consensus.cc:3055] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:10:22.533738 28788 raft_consensus.cc:740] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:22.534837 28788 consensus_queue.cc:260] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:22.535983 28788 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:22.537163 22843 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:22.501058 28418 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:22.538112 28719 raft_consensus.cc:2804] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:22.538621 28719 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:22.538988 28719 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:22.539908 28418 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:22.504397 28685 raft_consensus.cc:2804] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:22.540895 28685 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:22.541286 28685 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:22.512658 28715 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:22.545141 28715 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:22.524233 28850 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:22.546810 22843 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:22.547857 22872 raft_consensus.cc:1275] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 13:10:22.553447 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 65066.8us
I20260709 13:10:22.558830 21756 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:10:22.518782 28794 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:22.571430 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:22.573299 23025 raft_consensus.cc:1240] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 17. Current term is 18. Ops: []
I20260709 13:10:22.573966 28154 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:22.576287 28154 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:22.553870 28850 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:22.578007 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:22.578881 23026 raft_consensus.cc:1240] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 13. Current term is 14. Ops: []
I20260709 13:10:22.580624 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:22.572038 23021 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
I20260709 13:10:22.580807 23022 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 16.
I20260709 13:10:22.582254 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:22.577329 28807 consensus_queue.cc:1048] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 13:10:22.585788 22843 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:22.586530 28847 raft_consensus.cc:2804] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:22.586910 28847 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:22.587278 28847 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:22.589390 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:22.593753 28685 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:22.594928 28847 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:22.595979 28685 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:10:22.597035 28839 log.cc:927] Time spent T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.156s	user 0.000s	sys 0.001s
I20260709 13:10:22.597152 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:22.598449 28847 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:22.600235 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:22.601027 28783 consensus_queue.cc:1059] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:10:22.601238 23588 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:22.602041 28783 raft_consensus.cc:3055] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:10:22.602432 28783 raft_consensus.cc:740] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:22.602983 28802 raft_consensus.cc:2749] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:22.603277 28783 consensus_queue.cc:260] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:22.603786 28802 consensus_queue.cc:1059] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:10:22.578423 28418 raft_consensus.cc:2749] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:10:22.600835 23025 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:22.604574 28783 raft_consensus.cc:3060] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:22.609326 28818 consensus_queue.cc:1048] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:10:22.609742 23588 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:22.609954 28796 consensus_queue.cc:1048] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:10:22.613492 28802 raft_consensus.cc:2804] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:10:22.613894 28802 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:22.614260 28802 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:22.615147 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 11 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:22.581578 28715 consensus_queue.cc:1048] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
W20260709 13:10:22.618031 28804 log.cc:927] Time spent T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.857s	user 0.000s	sys 0.003s
I20260709 13:10:22.618170 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:22.618805 23020 raft_consensus.cc:2376] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 16. Re-sending same reply.
I20260709 13:10:22.620029 23020 raft_consensus.cc:1240] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 13:10:22.621493 28344 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:22.622198 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:22.611585 28846 consensus_queue.cc:1059] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:10:22.623474 28344 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:22.623641 28693 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:22.614195 28818 raft_consensus.cc:3055] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:10:22.624615 28818 raft_consensus.cc:740] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:22.625437 28818 consensus_queue.cc:260] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:22.626565 28818 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:22.629940 28850 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:22.631990 28850 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:22.634550 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:22.635072 23021 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:22.637028 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:22.643553 23020 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:22.650223 23025 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 17.
I20260709 13:10:22.652594 22843 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:22.653303 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:22.656173 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:22.653946 23616 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 10.
I20260709 13:10:22.623654 28846 raft_consensus.cc:3055] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:10:22.665037 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:22.665675 23021 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 17.
I20260709 13:10:22.665529 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:22.643567 28369 raft_consensus.cc:3055] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:10:22.668025 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:22.670239 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:22.668756 28369 raft_consensus.cc:740] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:22.580948 28794 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:22.672505 28369 consensus_queue.cc:260] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:22.673957 28369 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:22.654398 28815 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:22.681221 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:22.687888 28728 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:22.662492 28829 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:22.644814 28447 consensus_queue.cc:1059] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 13:10:22.662917 28693 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:22.689657 28344 raft_consensus.cc:3055] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:10:22.690076 28693 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:22.692368 28693 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:22.690088 28344 raft_consensus.cc:740] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
W20260709 13:10:22.697037 28851 log.cc:927] Time spent T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.208s	user 0.001s	sys 0.000s
I20260709 13:10:22.697216 28420 raft_consensus.cc:2749] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:10:22.697574 22952 raft_consensus.cc:3055] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:10:22.704203 28811 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:22.772878 22952 raft_consensus.cc:740] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:22.713241 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:22.713836 22843 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:22.776536 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:22.776996 22880 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 15.
W20260709 13:10:22.777135 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 63383.2us
I20260709 13:10:22.778198 22843 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:22.777307 23025 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11.
I20260709 13:10:22.780532 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:22.711349 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:22.636581 28719 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:22.663125 28850 raft_consensus.cc:2804] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:10:22.714285 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:22.786116 23020 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
I20260709 13:10:22.787174 22949 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 16.
I20260709 13:10:22.787269 22843 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:22.715313 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:10:22.697086 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 188793us
W20260709 13:10:22.731834 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:22.717901 28791 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:22.801172 28791 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:22.803508 28791 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:22.708261 28554 consensus_queue.cc:1048] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:10:22.744287 28829 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:22.745771 28800 log.cc:927] Time spent T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.989s	user 0.002s	sys 0.002s
I20260709 13:10:22.746711 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:22.805238 22953 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:22.747416 28821 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:22.807096 28821 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:22.749205 28786 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:22.809012 28786 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:22.807127 23029 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 2331) took 1677 ms. Trace:
I20260709 13:10:22.667294 28846 raft_consensus.cc:740] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:22.811007 28846 consensus_queue.cc:260] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:22.754392 28790 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:22.755278 28693 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:22.765873 28745 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:22.816597 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:22.756536 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:22.843768 22951 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15.
I20260709 13:10:22.766803 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:22.767294 22877 raft_consensus.cc:1240] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14. Ops: []
I20260709 13:10:22.770936 22947 raft_consensus.cc:1240] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 15. Current term is 16. Ops: []
I20260709 13:10:22.846351 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:22.774434 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:22.711778 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:22.847674 23618 raft_consensus.cc:2368] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 17. Current term is 18.
I20260709 13:10:22.847808 22879 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 11.
I20260709 13:10:22.782050 22872 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 17.
W20260709 13:10:22.716589 28770 log.cc:927] Time spent T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 1.269s	user 0.003s	sys 0.003s
I20260709 13:10:22.796504 22919 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:22.798411 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:22.851063 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 54658.2us
I20260709 13:10:22.851161 22950 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 11.
I20260709 13:10:22.718644 28779 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:22.799717 28715 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:22.800237 28578 consensus_queue.cc:1059] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:10:22.810271 23029 rpcz_store.cc:276] 0709 13:10:21.129262 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:21.516170 (+386908us) service_pool.cc:224] Handling call
0709 13:10:21.516603 (+   433us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:21.533038 (+ 16435us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:21.533065 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:21.566074 (+ 33009us) log.cc:844] Serialized 26 byte log entry
0709 13:10:21.704330 (+138256us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:10:21.704397 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:21.704486 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:22.744964 (+1040478us) spinlock_profiling.cc:243] Waited 9.79 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:10:22.806698 (+ 61734us) raft_consensus.cc:1685] finished
0709 13:10:22.806728 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:22.807025 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5921,"raft.queue_time_us":15556,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":1674,"spinlock_wait_cycles":22550784,"thread_start_us":136344,"threads_started":1,"wal-append.queue_time_us":189094}
I20260709 13:10:22.859764 23588 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:22.786864 28805 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:22.823103 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:22.717345 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:22.812172 28846 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:22.841854 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:22.841989 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:22.788712 28775 raft_consensus.cc:2804] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:10:22.873350 22953 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 15.
I20260709 13:10:22.875455 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:22.856853 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:22.858008 23025 raft_consensus.cc:1240] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 13:10:22.861934 22875 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 15.
I20260709 13:10:22.862411 22948 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 16.
I20260709 13:10:22.871084 22955 raft_consensus.cc:2393] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 17: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:22.867759 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:22.879299 22951 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:22.838860 28850 raft_consensus.cc:697] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 17 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:22.877194 22946 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15.
I20260709 13:10:22.892220 23029 raft_consensus.cc:1240] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 13. Current term is 14. Ops: []
W20260709 13:10:22.884058 22994 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:22.872843 23588 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:22.888248 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:22.814529 28447 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:10:22.786490 28811 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:22.894328 23024 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33420 (request call id 2423) took 1937 ms (client timeout 2164 ms). Trace:
I20260709 13:10:22.895303 23027 raft_consensus.cc:1240] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 16. Current term is 17. Ops: [14.8-16.9]
I20260709 13:10:22.886615 22843 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:22.856470 28829 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:22.902667 22948 raft_consensus.cc:1240] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 14. Current term is 15. Ops: [13.7-14.8]
W20260709 13:10:22.900265 23024 rpcz_store.cc:269] 0709 13:10:20.956638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:21.193171 (+236533us) service_pool.cc:224] Handling call
0709 13:10:22.894123 (+1700952us) spinlock_profiling.cc:243] Waited 28.3 ms on lock 0x7b5400189a54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:22.894263 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":65036160}
I20260709 13:10:22.796074 28837 consensus_queue.cc:1048] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.083s
W20260709 13:10:22.905510 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:22.902066 28791 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:22.911334 22918 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:22.913028 23024 raft_consensus.cc:1275] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:10:22.887290 28796 raft_consensus.cc:2749] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:22.922135 22955 raft_consensus.cc:1240] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
W20260709 13:10:22.925186 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 52441.8us
I20260709 13:10:22.850109 28693 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:22.963722 22878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53724 (request call id 1086) took 2038 ms. Trace:
I20260709 13:10:22.965274 22844 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:22.964110 22878 rpcz_store.cc:276] 0709 13:10:20.925693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:20.929839 (+  4146us) service_pool.cc:224] Handling call
0709 13:10:20.930174 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:21.766011 (+835837us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:22.963517 (+1197506us) spinlock_profiling.cc:243] Waited 834 ms on lock 0x7b5400369054. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:22.963666 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1919247616}
I20260709 13:10:22.896209 28775 raft_consensus.cc:697] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 17 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:22.959282 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:22.974265 22953 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:22.962033 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:22.855397 28745 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:22.812716 28825 consensus_queue.cc:1048] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.096s
W20260709 13:10:22.989378 23588 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:22.989892 23588 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:22.990293 23588 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:22.926342 28485 consensus_queue.cc:1059] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.067s
I20260709 13:10:22.874511 28805 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:22.995900 22951 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 17.
I20260709 13:10:22.926453 28801 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:22.959232 22955 raft_consensus.cc:1240] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:22.968325 23023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 2335) took 1573 ms. Trace:
I20260709 13:10:22.854126 28554 consensus_queue.cc:1048] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 13:10:22.977816 22950 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:22.862179 28344 consensus_queue.cc:260] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:22.962831 28844 raft_consensus.cc:2804] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:10:23.000152 22953 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 16.
I20260709 13:10:22.909049 28826 raft_consensus.cc:2749] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:22.926944 28847 raft_consensus.cc:2804] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:10:22.909970 28750 consensus_queue.cc:1059] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.088s
I20260709 13:10:22.862823 28715 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:22.850660 28790 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:22.870872 28779 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:23.000268 23023 rpcz_store.cc:276] 0709 13:10:21.394742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:21.892733 (+497991us) service_pool.cc:224] Handling call
0709 13:10:21.927362 (+ 34629us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:22.304374 (+377012us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:22.963136 (+658762us) spinlock_profiling.cc:243] Waited 352 ms on lock 0x7b54001c7fd4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:22.968265 (+  5129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":810270592}
I20260709 13:10:22.974208 28792 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:22.994091 28825 consensus_queue.cc:1243] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer 5b346767a89745fa82928a5dac9cc0b2 log is divergent from this leader: its last log entry 11.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 13:10:22.938004 28447 raft_consensus.cc:697] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 19 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:23.028450 22953 raft_consensus.cc:1275] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:10:23.037639 22955 raft_consensus.cc:1275] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:10:22.998534 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:23.079372 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:23.079807 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:23.025110 28802 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:23.087314 28802 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:23.052352 28844 raft_consensus.cc:697] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:23.089473 28844 consensus_queue.cc:237] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:23.053133 28344 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:23.093271 28447 consensus_queue.cc:237] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:23.021127 28850 consensus_queue.cc:237] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:23.011720 28801 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:23.054005 22950 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 18.
I20260709 13:10:23.000032 28780 consensus_queue.cc:1048] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:10:23.054291 28794 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:23.055806 28693 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:23.057001 28775 consensus_queue.cc:237] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:23.059388 28745 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:22.993736 28783 consensus_queue.cc:1059] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.052s
I20260709 13:10:23.059748 28812 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
W20260709 13:10:23.060276 28840 log.cc:927] Time spent T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 13:10:23.060639 28842 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:23.027954 28834 raft_consensus.cc:2804] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:10:23.028244 28786 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:23.061283 28750 raft_consensus.cc:3055] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:10:22.993976 28728 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:22.997136 28485 raft_consensus.cc:3055] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:10:23.062948 28816 raft_consensus.cc:3055] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:10:23.063608 28774 raft_consensus.cc:2749] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:10:23.063978 28861 log.cc:927] Time spent T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:10:23.066124 28715 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:23.066007 22952 consensus_queue.cc:260] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:22.957110 28791 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:23.068878 23588 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:23.074158 28154 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:23.080607 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:23.097499 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:10:23.173719 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 104906us
I20260709 13:10:23.144361 28834 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:23.042426 28847 raft_consensus.cc:697] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 14 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:23.103500 23023 raft_consensus.cc:1240] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 17. Current term is 18. Ops: []
I20260709 13:10:23.075147 28828 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
W20260709 13:10:23.107019 28634 consensus_meta.cc:287] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.620s	user 0.005s	sys 0.004s
I20260709 13:10:23.121587 28745 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:23.123160 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:23.078881 28792 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:23.101322 28789 consensus_queue.cc:1048] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:10:23.171264 22952 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:10:23.163751 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 198565us
I20260709 13:10:23.173848 22946 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:23.097515 23625 raft_consensus.cc:1240] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 13:10:23.098672 22992 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:23.187407 28847 consensus_queue.cc:237] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:23.149721 28750 raft_consensus.cc:740] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:23.154619 28485 raft_consensus.cc:740] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
W20260709 13:10:23.194959 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 96396.3us
I20260709 13:10:23.157562 28816 raft_consensus.cc:740] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:23.197149 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:23.152194 28829 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:23.199009 22844 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:23.199604 22992 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:23.120651 28812 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:23.183539 28834 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:23.177425 28783 raft_consensus.cc:2749] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:23.214461 28812 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:23.215919 28870 raft_consensus.cc:2749] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:23.228338 22946 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 21.
I20260709 13:10:23.194808 28634 raft_consensus.cc:2749] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:23.199504 28794 raft_consensus.cc:2804] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:10:23.150182 28842 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:23.157409 28728 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:23.181295 28687 consensus_queue.cc:1059] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 13:10:23.195281 28745 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:23.225221 28807 raft_consensus.cc:2804] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:10:23.233970 28818 consensus_meta.cc:287] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.607s	user 0.013s	sys 0.000s
I20260709 13:10:23.292786 23588 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:23.182163 28154 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:23.252283 23024 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:23.258198 22951 raft_consensus.cc:2393] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 15: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:23.259699 23027 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 2325) took 2484 ms. Trace:
W20260709 13:10:23.300422 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 88714us
W20260709 13:10:23.250078 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 289982us
I20260709 13:10:23.288970 23025 raft_consensus.cc:1240] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:10:23.301699 22844 leader_election.cc:400] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 17. Current term is 18.
I20260709 13:10:23.302085 22844 leader_election.cc:403] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
W20260709 13:10:23.303335 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 99271.1us
I20260709 13:10:23.300704 23027 rpcz_store.cc:276] 0709 13:10:20.775297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:20.775408 (+   111us) service_pool.cc:224] Handling call
0709 13:10:21.292915 (+517507us) spinlock_profiling.cc:243] Waited 1.52 ms on lock 0x7b58001ca198. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0a22 00007f613bbd9c09 00007f613ad33ee0 00007f613adfd0ab 00007f613adfd131 00007f613adbefc0 00007f613ada9929 00007f613ada93d2 00007f6134320bd4
0709 13:10:22.894939 (+1602024us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:10:22.900399 (+  5460us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:23.259326 (+358927us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b5400189a50. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:23.259513 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3687095936}
I20260709 13:10:23.189518 28828 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:23.240586 28862 log.cc:927] Time spent T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.237s	user 0.003s	sys 0.000s
W20260709 13:10:23.276244 28865 log.cc:927] Time spent T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
W20260709 13:10:23.277681 28866 log.cc:927] Time spent T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.074s	user 0.000s	sys 0.001s
I20260709 13:10:23.304395 22992 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:23.307890 23623 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
I20260709 13:10:23.270404 28794 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:23.316893 23588 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:23.276491 28805 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:10:23.342255 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:23.343310 23027 raft_consensus.cc:1275] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:10:23.345162 22954 raft_consensus.cc:1275] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:10:23.332331 23024 raft_consensus.cc:1275] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:10:23.338737 22953 raft_consensus.cc:1240] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16. Ops: []
I20260709 13:10:23.275130 28745 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:23.351123 23588 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:23.306675 28827 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:23.357004 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:23.312120 28807 raft_consensus.cc:697] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:23.320544 28826 raft_consensus.cc:2804] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:10:23.365042 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:23.371224 28826 raft_consensus.cc:697] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:23.368007 22920 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:23.372720 22920 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:23.313927 28811 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:23.304062 28774 consensus_queue.cc:1048] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 13:10:23.369372 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:23.371315 22950 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 14.
I20260709 13:10:23.341761 28794 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:23.341991 28812 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:10:23.369422 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:23.373749 22920 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:23.342093 28520 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:23.378393 22992 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:23.382216 23024 raft_consensus.cc:2393] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 14: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:23.343664 28685 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:23.320013 28715 consensus_queue.cc:1059] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.052s
I20260709 13:10:23.385569 22920 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:23.357705 28788 raft_consensus.cc:2749] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:23.343081 28779 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:23.370694 28426 raft_consensus.cc:2804] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:10:23.395309 28860 log.cc:927] Time spent T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.281s	user 0.000s	sys 0.001s
I20260709 13:10:23.368814 28801 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:23.476536 22994 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:23.340602 28818 consensus_queue.cc:1048] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.040s
W20260709 13:10:23.478475 28800 log.cc:927] Time spent T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 13:10:23.319404 28485 consensus_queue.cc:260] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:23.496300 28485 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:23.366019 28796 consensus_queue.cc:1048] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 13:10:23.386983 28578 raft_consensus.cc:2804] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:23.391541 28795 raft_consensus.cc:2804] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:10:23.390641 28850 consensus_queue.cc:1059] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.104s
I20260709 13:10:23.497797 23024 raft_consensus.cc:1240] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:10:23.394134 28844 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:23.441980 28745 raft_consensus.cc:2804] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:23.475414 28811 raft_consensus.cc:2804] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:23.476904 28879 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:23.372195 28827 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:23.478163 28870 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:23.386453 28805 consensus_queue.cc:1048] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.119s
I20260709 13:10:23.497794 28426 raft_consensus.cc:697] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 18 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:23.496935 28447 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
W20260709 13:10:23.526861 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 174665us
I20260709 13:10:23.386785 28878 consensus_queue.cc:1059] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.122s
I20260709 13:10:23.501864 28578 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:23.541185 22947 raft_consensus.cc:1240] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 13:10:23.398046 28685 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:23.507468 28795 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:23.512174 28811 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
W20260709 13:10:23.536303 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 150801us
W20260709 13:10:23.536319 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 140770us
I20260709 13:10:23.507579 28788 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:23.529251 28791 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:23.510406 28745 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:23.510247 28844 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:23.555648 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:23.558271 22843 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:23.636361 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 78174.2us
I20260709 13:10:23.636844 22947 raft_consensus.cc:1275] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 13:10:23.543309 23588 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:23.630934 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:23.627013 28795 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 11 FOLLOWER]: Advancing to term 12
W20260709 13:10:23.638784 28880 log.cc:927] Time spent T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.320s	user 0.000s	sys 0.001s
W20260709 13:10:23.641881 28884 log.cc:927] Time spent T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 13:10:23.642328 28685 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:23.643215 28885 raft_consensus.cc:3055] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:10:23.642910 28788 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:23.643601 28885 raft_consensus.cc:740] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:23.488854 28826 consensus_queue.cc:237] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:23.644470 28885 consensus_queue.cc:260] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:23.645824 28885 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:23.646867 22878 raft_consensus.cc:1275] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:10:23.627089 28811 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:23.648814 28792 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:23.648953 28886 consensus_queue.cc:1048] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:10:23.649863 28842 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:23.650467 28842 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:23.650939 28842 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:23.652640 28844 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:23.511229 28807 consensus_queue.cc:237] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:23.544885 28850 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:23.655256 28850 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:23.657338 28850 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:23.527806 28879 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:23.657459 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:23.657878 28850 raft_consensus.cc:2749] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:23.658720 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:23.659092 28745 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:23.659372 23623 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60026 (request call id 1075) took 1007 ms. Trace:
I20260709 13:10:23.659578 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:23.659811 23623 rpcz_store.cc:276] 0709 13:10:22.652347 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:22.652741 (+   394us) service_pool.cc:224] Handling call
0709 13:10:23.659222 (+1006481us) spinlock_profiling.cc:243] Waited 641 ms on lock 0x7b54001108d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:23.659320 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1473736960}
I20260709 13:10:23.660697 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:23.661736 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:23.663009 22880 raft_consensus.cc:1240] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 14 LEADER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 13. Current term is 14. Ops: [13.10-13.10]
I20260709 13:10:23.537377 28774 raft_consensus.cc:3055] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:10:23.638063 28829 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:23.634529 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:23.673008 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:23.666159 22872 raft_consensus.cc:1240] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 15. Current term is 16. Ops: []
W20260709 13:10:23.676021 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 291005us
W20260709 13:10:23.654637 28846 consensus_meta.cc:287] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.776s	user 0.007s	sys 0.001s
I20260709 13:10:23.545153 28578 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:23.665582 22873 raft_consensus.cc:1275] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:10:23.678786 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:23.674515 23022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 2350) took 1092 ms. Trace:
I20260709 13:10:23.725449 22877 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18.
I20260709 13:10:23.700695 23022 rpcz_store.cc:276] 0709 13:10:22.582310 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:22.582412 (+   102us) service_pool.cc:224] Handling call
0709 13:10:22.582673 (+   261us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:22.582980 (+   307us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:22.583010 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:22.612772 (+ 29762us) log.cc:844] Serialized 26 byte log entry
0709 13:10:22.760164 (+147392us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:10:22.760203 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:22.760266 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:23.674311 (+914045us) raft_consensus.cc:1685] finished
0709 13:10:23.674331 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:23.674463 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":98,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":81,"thread_start_us":147028,"threads_started":1,"wal-append.queue_time_us":281138}
I20260709 13:10:23.659210 28879 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:23.736680 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:23.737336 22951 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 17.
I20260709 13:10:23.739141 22875 raft_consensus.cc:1240] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 17 LEADER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 16. Current term is 17. Ops: [16.14-16.14]
I20260709 13:10:23.546161 28818 consensus_queue.cc:1059] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.180s
I20260709 13:10:23.664888 28774 raft_consensus.cc:740] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:23.673063 28154 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:23.538271 28447 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:23.750226 22952 consensus_meta.cc:287] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.556s	user 0.009s	sys 0.001s
I20260709 13:10:23.627707 28867 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:23.759147 22952 raft_consensus.cc:1275] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 17 index: 14. (term mismatch)
I20260709 13:10:23.761202 22952 pending_rounds.cc:85] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting all ops after (but not including) 13
I20260709 13:10:23.535074 28870 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:23.726708 28812 raft_consensus.cc:697] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 21 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:23.738967 28805 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:23.761457 22952 pending_rounds.cc:107] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting uncommitted NO_OP operation due to leader change: 16.14
I20260709 13:10:23.772454 22952 raft_consensus.cc:2889] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:23.773188 22952 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 2419) took 1779 ms. Trace:
I20260709 13:10:23.773566 22952 rpcz_store.cc:276] 0709 13:10:21.994077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:22.000622 (+  6545us) service_pool.cc:224] Handling call
0709 13:10:22.000926 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:22.697475 (+696549us) spinlock_profiling.cc:243] Waited 146 ms on lock 0x7b5400082b20. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135d5cdc0 00007f6135de71c0 00007f6135de8231 00007f6135de4d8f 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c
0709 13:10:23.065686 (+368211us) spinlock_profiling.cc:243] Waited 71.7 ms on lock 0x7b5400082b20. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135d6306f 00007f6135ddbf60 00007f6135ddd7de 00007f6135de7b49 00007f6135de8416 00007f6135de4d8f 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62
0709 13:10:23.772939 (+707253us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:23.773133 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":500328576}
I20260709 13:10:23.683373 28829 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:23.685210 28791 consensus_queue.cc:1048] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.077s
I20260709 13:10:23.783790 22843 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:23.784716 28712 raft_consensus.cc:2804] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:23.785105 28712 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:23.785462 28712 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:23.794869 22843 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:23.797792 22843 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:23.796324 28712 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:23.809716 28783 raft_consensus.cc:3055] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:10:23.810197 28783 raft_consensus.cc:740] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:23.811019 28783 consensus_queue.cc:260] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:23.815500 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:23.819568 22880 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 17.
I20260709 13:10:23.820410 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:23.818284 23590 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:23.821816 28888 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:23.766227 28867 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:23.822265 28888 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:23.822724 28888 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:23.826455 23590 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:23.827389 28867 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:23.778889 28842 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:23.828346 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 159363us
I20260709 13:10:23.831084 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:23.829783 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:23.757275 28426 consensus_queue.cc:237] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:23.777859 28805 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:23.828011 28818 raft_consensus.cc:2749] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:23.824607 28846 consensus_queue.cc:1059] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:10:23.833926 28772 raft_consensus.cc:2749] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:10:23.837984 28822 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 17 current term: 17 is higher.
I20260709 13:10:23.837218 28783 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:10:23.885705 23588 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:23.887013 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:23.887686 23617 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15.
I20260709 13:10:23.888343 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:23.889163 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:23.889739 23616 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 17.
I20260709 13:10:23.889853 28709 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:23.834106 28890 raft_consensus.cc:2804] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:23.869100 28818 consensus_queue.cc:1059] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.003s
W20260709 13:10:23.863377 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:23.857585 28344 consensus_meta.cc:287] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.765s	user 0.011s	sys 0.000s
I20260709 13:10:23.868623 28745 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:23.873472 22920 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:23.875357 28886 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:23.879777 28815 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:23.869047 28827 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:10:23.892524 22994 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:23.895813 22844 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:23.782055 28774 consensus_queue.cc:260] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:23.998095 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 102614us
W20260709 13:10:24.011431 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 211004us
I20260709 13:10:24.014489 23021 raft_consensus.cc:1240] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
W20260709 13:10:24.013775 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 312663us
W20260709 13:10:24.065709 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 169977us
I20260709 13:10:23.907176 28890 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:23.836575 28812 consensus_queue.cc:237] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:23.890339 28709 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:23.986702 28807 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
W20260709 13:10:24.223507 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:24.214274 22994 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:24.233302 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 392838us
I20260709 13:10:24.239190 22844 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:24.210681 28892 log.cc:927] Time spent T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.376s	user 0.001s	sys 0.000s
W20260709 13:10:24.302490 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 68978.7us
W20260709 13:10:24.216699 28770 log.cc:927] Time spent T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.219s	user 0.001s	sys 0.000s
I20260709 13:10:24.208719 28344 raft_consensus.cc:2804] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:10:24.208734 28685 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:24.311287 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 72171.5us
I20260709 13:10:24.321677 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:24.324059 23025 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 13.
I20260709 13:10:24.342226 28911 raft_consensus.cc:2804] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:24.342769 22844 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:24.344398 28846 raft_consensus.cc:2804] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:24.344848 28846 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:24.344646 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:24.345556 28846 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:24.346063 23025 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 17.
I20260709 13:10:24.216133 28792 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.348037 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:24.359683 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 46107.1us
I20260709 13:10:24.219352 28827 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.238552 28887 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.222297 28890 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:10:24.625686 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 256907us
I20260709 13:10:24.638264 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:24.302500 28815 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.247610 28842 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:24.639335 23025 raft_consensus.cc:2393] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 15: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:24.639897 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:24.212867 28772 consensus_queue.cc:1059] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.145s
I20260709 13:10:24.145533 28886 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:24.323102 28774 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:24.323740 28344 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:24.342799 28911 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:24.623037 28447 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:24.310001 28807 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:24.363093 28685 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:24.670761 28829 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:24.670749 28772 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:24.630962 28792 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:24.678745 28344 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:24.640352 28827 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:24.680881 28911 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:24.668205 28815 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:24.687485 28895 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:10:24.714334 28899 log.cc:927] Time spent T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.494s	user 0.000s	sys 0.001s
I20260709 13:10:24.714617 28895 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:24.715806 22955 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 2439) took 1087 ms. Trace:
I20260709 13:10:24.709558 28772 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:24.716248 22955 rpcz_store.cc:276] 0709 13:10:23.628633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:23.629887 (+  1254us) service_pool.cc:224] Handling call
0709 13:10:23.630281 (+   394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:23.665195 (+ 34914us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:23.665227 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:23.678752 (+ 13525us) log.cc:844] Serialized 26 byte log entry
0709 13:10:23.830012 (+151260us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:10:23.830067 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:23.830155 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:24.715550 (+885395us) raft_consensus.cc:1685] finished
0709 13:10:24.715576 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:24.715739 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43732,"raft.queue_time_us":34603,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":10644,"thread_start_us":75908,"threads_started":1,"wal-append.queue_time_us":158747}
W20260709 13:10:24.718966 28896 log.cc:927] Time spent T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.573s	user 0.001s	sys 0.000s
I20260709 13:10:24.719810 28772 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:24.689039 28418 consensus_queue.cc:1048] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.022s
I20260709 13:10:24.721901 28886 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:24.664901 28906 consensus_queue.cc:1048] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.060s
I20260709 13:10:24.661850 28887 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:24.729614 28709 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:24.730330 28887 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:24.730840 28709 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:24.731352 28709 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:24.732105 28792 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:24.695206 28520 consensus_meta.cc:287] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 1.304s	user 0.009s	sys 0.001s
I20260709 13:10:24.732756 28520 raft_consensus.cc:2749] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 17. Current term is 18.
W20260709 13:10:24.735351 28844 consensus_meta.cc:287] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 1.082s	user 0.008s	sys 0.000s
I20260709 13:10:24.735844 28844 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:24.713243 28891 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.736940 28891 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:24.738967 28891 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:24.690546 28447 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:24.741003 28447 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:24.743341 28447 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:24.743937 28783 consensus_meta.cc:287] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.864s	user 0.005s	sys 0.001s
I20260709 13:10:24.678233 28842 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:24.745211 28895 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:24.680145 22994 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:24.759090 23025 raft_consensus.cc:1275] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:10:24.768791 22875 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 18.
I20260709 13:10:24.709903 28846 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:24.754357 28834 consensus_meta.cc:287] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 1.524s	user 0.005s	sys 0.006s
I20260709 13:10:24.781536 28344 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:24.780601 28834 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:24.783813 28827 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:24.785419 28815 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:24.787058 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:24.787554 28805 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:24.788990 28745 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:10:24.789286 28885 consensus_meta.cc:287] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 1.143s	user 0.008s	sys 0.003s
I20260709 13:10:24.789677 28709 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:24.791610 23021 raft_consensus.cc:2393] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 18: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
W20260709 13:10:24.712399 28811 consensus_meta.cc:287] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 1.064s	user 0.010s	sys 0.001s
W20260709 13:10:24.799428 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:24.809096 23026 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 2334) took 3610 ms. Trace:
I20260709 13:10:24.810773 28915 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:24.869854 28578 consensus_meta.cc:287] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 1.135s	user 0.008s	sys 0.005s
I20260709 13:10:24.877957 22880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35360 (request call id 2470) took 1009 ms. Trace:
I20260709 13:10:24.813655 28773 raft_consensus.cc:3055] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:10:24.814320 28792 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.900405 23026 rpcz_store.cc:276] 0709 13:10:21.198405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:21.768633 (+570228us) service_pool.cc:224] Handling call
0709 13:10:22.497937 (+729304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:22.580913 (+ 82976us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:22.709790 (+128877us) spinlock_profiling.cc:243] Waited 80.8 ms on lock 0x7b5400135454. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:24.808855 (+2099065us) spinlock_profiling.cc:243] Waited 729 ms on lock 0x7b5400135450. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:24.809027 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1862118016}
I20260709 13:10:24.813817 28834 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:24.920782 22880 rpcz_store.cc:276] 0709 13:10:23.868499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:24.786786 (+918287us) service_pool.cc:224] Handling call
0709 13:10:24.877891 (+ 91105us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:10:24.843772 22994 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:24.928778 22994 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.63.1:33929 timed out after 1.577s (SENT) [suppressed 18 similar messages]
W20260709 13:10:24.930969 22871 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.150s	user 0.016s	sys 0.036s
W20260709 13:10:24.931393 22871 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.153s	user 0.016s	sys 0.036s
W20260709 13:10:24.906603 28795 consensus_meta.cc:287] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 1.261s	user 0.011s	sys 0.001s
I20260709 13:10:24.897898 28887 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:24.905571 28885 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:24.934927 22871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 2464) took 1279 ms. Trace:
W20260709 13:10:24.992499 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 54956.5us
W20260709 13:10:24.997756 23588 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.676s	user 0.031s	sys 0.044s
W20260709 13:10:25.045053 23588 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.723s	user 0.033s	sys 0.044s
W20260709 13:10:25.045655 23588 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:24.930027 28773 raft_consensus.cc:740] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:24.936542 28895 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.937153 28447 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:24.992412 28834 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:24.915162 28811 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:25.039191 22871 rpcz_store.cc:276] 0709 13:10:23.655167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:23.664810 (+  9643us) service_pool.cc:224] Handling call
0709 13:10:23.673130 (+  8320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:23.673323 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:10:23.778480 (+105157us) log.cc:844] Serialized 14 byte log entry
0709 13:10:24.934433 (+1155953us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:24.934491 (+    58us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:10:24.934550 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:24.934629 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:24.934866 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2512,"prepare.queue_time_us":160703,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":51,"raft.queue_time_us":3226,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":57568,"spinlock_wait_cycles":1792,"thread_start_us":1257814,"threads_started":2}
I20260709 13:10:24.934957 28520 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:25.054446 28891 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:25.078011 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:25.087592 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:25.055820 28886 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:24.931298 28578 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:25.079231 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:25.040120 28369 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:25.080174 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:24.995069 28913 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:25.048280 28827 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:24.933820 28792 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:25.084117 28834 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:25.105692 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:25.105113 28844 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:25.103623 23616 raft_consensus.cc:2393] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 14: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:25.178936 23616 raft_consensus.cc:1275] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:10:25.124010 28802 raft_consensus.cc:3055] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:10:25.110759 28794 consensus_meta.cc:287] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 1.713s	user 0.011s	sys 0.000s
I20260709 13:10:25.125303 28788 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:25.134008 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:25.185940 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:25.133354 28687 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:25.189239 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:25.190068 22871 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 16.
I20260709 13:10:25.090516 28891 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:25.133570 28807 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:25.133186 28344 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:25.131788 28916 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:25.104998 28886 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:25.106350 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:25.192687 23625 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18.
I20260709 13:10:25.080725 28783 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:25.050354 28885 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:24.939852 28795 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:25.196322 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:25.078954 28888 consensus_meta.cc:287] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 1.256s	user 0.012s	sys 0.000s
W20260709 13:10:25.105646 28903 log.cc:927] Time spent T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.227s	user 0.001s	sys 0.000s
I20260709 13:10:25.156747 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:25.115831 22876 raft_consensus.cc:2410] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:10:25.121989 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:24.925595 28915 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:25.201635 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:25.202288 23625 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 15.
I20260709 13:10:25.202394 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:24.992133 28745 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:25.050019 28887 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:25.088110 28520 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:25.163369 28912 log.cc:927] Time spent T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 13:10:25.165064 28811 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:25.166546 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:25.167943 22874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 2461) took 1513 ms. Trace:
I20260709 13:10:25.086115 28447 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:25.177012 23620 raft_consensus.cc:1240] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:25.079108 28895 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:25.132262 28894 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:25.170568 28369 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:25.199450 22880 raft_consensus.cc:2410] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
W20260709 13:10:25.200584 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 271833us
I20260709 13:10:25.159500 23617 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 10.
I20260709 13:10:25.219310 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:25.203253 22871 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 16.
I20260709 13:10:25.206727 28870 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:25.209307 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:25.172806 28827 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:25.213485 22874 rpcz_store.cc:276] 0709 13:10:23.654234 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:23.656761 (+  2527us) service_pool.cc:224] Handling call
0709 13:10:23.657054 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:23.769794 (+112740us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:23.769837 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:23.769908 (+    71us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:10:23.769952 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:25.167628 (+1397676us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b5400287dd4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:25.167690 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:25.167861 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":258719616}
I20260709 13:10:25.222851 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:25.194347 28687 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:25.181335 28802 raft_consensus.cc:740] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:25.209869 28745 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:25.228827 28802 consensus_queue.cc:260] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:25.229882 28802 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:25.231344 28447 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:25.231474 28745 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:25.232963 28885 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:25.221238 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:25.233532 28895 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:25.221205 28894 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:25.234319 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:25.235024 23623 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 17.
I20260709 13:10:25.235479 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:25.236105 23620 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 17.
I20260709 13:10:25.172487 28913 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:25.238512 28827 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:25.238595 28913 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:25.188706 28794 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:25.243199 22871 raft_consensus.cc:1240] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14. Ops: []
I20260709 13:10:25.236133 28894 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:25.242444 28891 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:25.240351 28687 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:25.159729 28773 consensus_queue.cc:260] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:25.246184 23622 raft_consensus.cc:1275] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:10:25.198978 28916 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:25.198300 28783 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:25.250150 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:25.252332 23617 raft_consensus.cc:2468] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 14.
I20260709 13:10:25.199541 28888 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:25.222350 28807 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:25.254760 23022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 1030) took 1365 ms. Trace:
I20260709 13:10:25.255908 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:25.255147 23022 rpcz_store.cc:276] 0709 13:10:23.889650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:23.890647 (+   997us) service_pool.cc:224] Handling call
0709 13:10:23.891052 (+   405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:23.891473 (+   421us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:23.891511 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:23.891947 (+   436us) log.cc:844] Serialized 26 byte log entry
0709 13:10:24.673203 (+781256us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:10:24.673277 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:24.673360 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:25.254553 (+581193us) raft_consensus.cc:1685] finished
0709 13:10:25.254573 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:25.254697 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":238,"raft.queue_time_us":15105,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":12825,"thread_start_us":464973,"threads_started":1,"wal-append.queue_time_us":911530}
I20260709 13:10:25.257011 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:25.257596 23617 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 14.
I20260709 13:10:25.259539 28578 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:25.262162 28911 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:25.265013 22878 raft_consensus.cc:1240] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:10:25.267870 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:25.268906 22871 raft_consensus.cc:2393] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 18: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
W20260709 13:10:25.273500 28890 consensus_meta.cc:287] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.601s	user 0.008s	sys 0.002s
I20260709 13:10:25.274041 28890 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:25.275249 22880 raft_consensus.cc:1240] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:10:25.316407 23619 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:25.360816 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:10:25.489177 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 222561us
I20260709 13:10:25.280944 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:25.490172 23621 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 13.
I20260709 13:10:25.491727 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:25.324723 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:25.493630 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:25.494989 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 228840us
I20260709 13:10:25.483062 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:25.509796 23620 raft_consensus.cc:2393] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 19: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:25.496096 22994 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:25.516369 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:25.517314 22878 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 13.
W20260709 13:10:25.325081 28924 log.cc:927] Time spent T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
W20260709 13:10:25.325564 28928 log.cc:927] Time spent T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 13:10:25.498872 28447 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:25.315670 28795 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:25.412038 28790 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:25.281656 28828 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:25.426076 28915 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:25.368443 28834 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:25.366559 28886 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:25.314452 28887 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:25.367381 28792 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:25.543756 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:25.566987 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:25.568369 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:25.569003 22871 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 18.
I20260709 13:10:25.569801 23619 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 16.
I20260709 13:10:25.545152 28794 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:25.551105 23021 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33420 (request call id 2451) took 1305 ms (client timeout 1600 ms). Trace:
I20260709 13:10:25.276434 28890 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:25.254192 28773 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:25.559835 28796 raft_consensus.cc:3055] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 20 LEADER]: Stepping down as leader of term 20
W20260709 13:10:25.561246 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 45266.9us
I20260709 13:10:25.574904 28796 raft_consensus.cc:740] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:25.575526 22994 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:25.575793 28796 consensus_queue.cc:260] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:25.277887 28923 log.cc:927] Time spent T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 13:10:25.557555 22993 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:25.576997 28821 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:25.577127 28796 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:25.578141 22993 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:25.579926 22993 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:25.580963 28794 raft_consensus.cc:2749] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:25.410005 28916 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:25.581396 28790 consensus_queue.cc:1048] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:10:25.477140 28911 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:25.580912 28887 raft_consensus.cc:2804] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:10:25.583078 28794 raft_consensus.cc:2804] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:25.583428 28887 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:25.411474 28719 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:25.313144 28520 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:25.572019 23021 rpcz_store.cc:269] 0709 13:10:24.245921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:24.321723 (+ 75802us) service_pool.cc:224] Handling call
0709 13:10:25.550912 (+1229189us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b5400359b54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:25.551046 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":285464960}
I20260709 13:10:25.572320 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:25.584321 22994 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:25.492901 28888 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:25.581033 28756 raft_consensus.cc:2804] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:25.583655 28794 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:10:25.591009 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:25.591475 22994 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:25.569824 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:25.586635 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:25.592649 23617 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 15.
I20260709 13:10:25.586459 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:25.585852 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:25.588783 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:25.530629 28745 raft_consensus.cc:2749] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:25.594087 23621 raft_consensus.cc:2393] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 14: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:25.600791 28790 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:25.590739 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:25.594899 28794 raft_consensus.cc:3060] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:25.587729 28916 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:25.602450 28916 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:25.590739 28756 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:25.595325 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:25.323081 28807 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:25.583915 28887 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:25.385326 28750 consensus_queue.cc:260] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:25.521824 28891 consensus_queue.cc:1048] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.078s
I20260709 13:10:25.610510 28709 consensus_queue.cc:1059] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:10:25.611101 28805 consensus_queue.cc:1059] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:10:25.612088 28745 raft_consensus.cc:3055] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:10:25.612478 28745 raft_consensus.cc:740] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:25.601315 28790 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:25.613353 28745 consensus_queue.cc:260] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:25.606734 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:25.614518 28745 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:25.614765 23027 raft_consensus.cc:2410] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:10:25.595887 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:25.615693 23021 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:25.616578 23028 raft_consensus.cc:1240] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:10:25.618430 28756 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:25.618706 22948 raft_consensus.cc:1240] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 13. Current term is 14. Ops: []
I20260709 13:10:25.619413 22948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 1029) took 2698 ms. Trace:
I20260709 13:10:25.619748 22948 rpcz_store.cc:276] 0709 13:10:22.920786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:22.922602 (+  1816us) service_pool.cc:224] Handling call
0709 13:10:22.972150 (+ 49548us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:25.618526 (+2646376us) spinlock_profiling.cc:243] Waited 57.6 ms on lock 0x7b5400083020. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135d5cdc0 00007f6135de71c0 00007f6135de8231 00007f6135de4d8f 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c
0709 13:10:25.619144 (+   618us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:25.619330 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":132369152}
I20260709 13:10:25.621001 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:25.622088 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:25.607250 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:25.622879 23022 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
I20260709 13:10:25.623116 22878 raft_consensus.cc:2393] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 16: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:25.608343 22874 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 16.
W20260709 13:10:25.588056 28869 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.805s	user 0.068s	sys 0.100s
I20260709 13:10:25.628747 23021 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 16.
I20260709 13:10:25.636065 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:25.590802 28528 raft_consensus.cc:2804] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:25.639201 23022 raft_consensus.cc:1275] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 13:10:25.639427 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:25.639977 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:25.604723 28916 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:25.642360 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:25.633734 28869 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.861s	user 0.069s	sys 0.100s
I20260709 13:10:25.608134 28917 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:25.648454 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:25.587061 28447 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:25.649147 23022 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 16.
W20260709 13:10:25.649892 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 70036.9us
W20260709 13:10:25.645154 28931 log.cc:927] Time spent T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.147s	user 0.001s	sys 0.001s
I20260709 13:10:25.652125 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:25.650656 22993 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:25.622071 28807 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:25.652750 22874 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 17.
I20260709 13:10:25.654706 23022 raft_consensus.cc:1275] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:10:25.642154 28528 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:25.653889 22993 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:25.659082 22993 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:25.653573 28789 raft_consensus.cc:2749] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:25.677263 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:25.655577 28528 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:25.678388 23023 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 17.
I20260709 13:10:25.649629 28917 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:25.678978 23020 raft_consensus.cc:1275] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:10:25.660156 28807 raft_consensus.cc:2804] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:25.666595 28750 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:25.682770 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:25.664752 28685 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:25.742036 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:25.677259 28790 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:25.757122 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:25.757786 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:25.744093 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:25.758852 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:25.680924 22993 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:25.750139 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:25.750535 28930 log.cc:927] Time spent T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:10:25.760506 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:25.752516 28712 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:25.754346 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:25.754575 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:25.755024 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:25.755645 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:25.760433 23028 raft_consensus.cc:2393] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 12: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:25.759411 23022 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
I20260709 13:10:25.762741 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:25.762878 22994 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:25.761178 23020 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 17.
I20260709 13:10:25.770718 23023 raft_consensus.cc:2410] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:10:25.682034 28885 raft_consensus.cc:2804] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:25.769147 22843 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:25.773618 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:25.774581 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:25.775827 28916 consensus_queue.cc:1059] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:10:25.776980 28790 raft_consensus.cc:2804] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:25.742167 28728 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:25.781840 28917 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:25.682464 28807 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:25.782032 28447 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:25.785374 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 104503us
I20260709 13:10:25.785347 22843 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:25.796303 23029 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 2434) took 3137 ms. Trace:
I20260709 13:10:25.803944 22843 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:25.803619 23029 rpcz_store.cc:276] 0709 13:10:22.658915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:22.893213 (+234298us) service_pool.cc:224] Handling call
0709 13:10:22.893495 (+   282us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:23.103070 (+209575us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:23.103113 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:23.103166 (+    53us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:10:23.103211 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:25.796020 (+2692809us) spinlock_profiling.cc:243] Waited 209 ms on lock 0x7b5400248454. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:25.796068 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:25.796228 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":481410944}
W20260709 13:10:25.796815 22875 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53724 (request call id 1096) took 1984 ms (client timeout 1608 ms). Trace:
I20260709 13:10:25.798483 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:25.802974 23625 raft_consensus.cc:1240] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEADER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16. Ops: []
I20260709 13:10:25.756443 28887 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:25.789700 28790 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:25.802541 28807 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:25.807446 28712 raft_consensus.cc:2804] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:25.807456 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:25.788146 28062 raft_consensus.cc:2749] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:25.808076 22993 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:25.788285 28885 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
W20260709 13:10:25.805542 22875 rpcz_store.cc:269] 0709 13:10:23.808385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:23.812193 (+  3808us) service_pool.cc:224] Handling call
0709 13:10:25.792940 (+1980747us) spinlock_profiling.cc:243] Waited 897 ms on lock 0x7b540030c5d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:25.793073 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2063181184}
I20260709 13:10:25.789659 28745 raft_consensus.cc:2749] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:10:25.812258 22843 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:25.813143 22872 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 17.
I20260709 13:10:25.822218 23027 raft_consensus.cc:2368] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 13. Current term is 14.
I20260709 13:10:25.823812 22843 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:25.790380 28838 consensus_queue.cc:1059] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 13:10:25.812184 28790 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:25.813217 28447 raft_consensus.cc:2804] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:10:25.831743 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:25.839125 22843 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:25.815821 28712 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:25.822254 28885 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:25.843187 22875 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 16.
I20260709 13:10:25.828732 28775 raft_consensus.cc:2804] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:25.844556 28775 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:25.845081 28775 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:25.850900 28528 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:25.856266 22876 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53724 (request call id 1098) took 2041 ms (client timeout 1953 ms). Trace:
I20260709 13:10:25.856966 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:25.857775 22872 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 14.
I20260709 13:10:25.858737 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:25.859560 22872 raft_consensus.cc:2410] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:10:25.841624 28712 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:25.861315 28887 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:23.749631 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:25.864722 28520 raft_consensus.cc:2804] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:10:25.824929 28728 raft_consensus.cc:2749] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:25.824780 28869 raft_consensus.cc:2804] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:10:25.856837 22876 rpcz_store.cc:269] 0709 13:10:23.814694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:23.815198 (+   504us) service_pool.cc:224] Handling call
0709 13:10:25.856008 (+2040810us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b5400412654. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:25.856187 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2941995392}
W20260709 13:10:25.867749 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:25.842042 22843 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:25.872159 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:25.872972 28528 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:25.874712 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:25.875555 22873 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 17.
I20260709 13:10:25.842535 28906 consensus_queue.cc:1048] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 13:10:25.827858 28062 consensus_queue.cc:1048] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 13:10:25.816246 28917 consensus_queue.cc:1048] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 13:10:25.876654 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:25.879163 23024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 2448) took 1634 ms. Trace:
I20260709 13:10:25.880546 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:25.881577 23024 rpcz_store.cc:276] 0709 13:10:24.244727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:24.244903 (+   176us) service_pool.cc:224] Handling call
0709 13:10:24.321227 (+ 76324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:24.703921 (+382694us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:24.703955 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:24.704000 (+    45us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:10:24.704040 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:25.878827 (+1174787us) spinlock_profiling.cc:243] Waited 382 ms on lock 0x7b5400359654. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:25.878894 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:25.879092 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":879633280}
I20260709 13:10:25.884081 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:25.880266 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:25.884851 22875 raft_consensus.cc:2393] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 12: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:25.883051 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:25.885330 22874 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:25.886507 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:25.886667 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:25.865175 28520 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:25.887869 22876 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:25.822777 28756 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:25.867571 28447 raft_consensus.cc:697] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 16 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:25.897187 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:25.869134 28869 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:25.883028 28868 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:25.886334 28794 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:25.903331 23021 raft_consensus.cc:2393] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 18: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:25.904012 22872 raft_consensus.cc:1275] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 13:10:25.904381 22872 pending_rounds.cc:85] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 7
I20260709 13:10:25.904603 22872 pending_rounds.cc:107] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 13:10:25.904923 22872 raft_consensus.cc:2889] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:25.905011 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:25.906368 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:25.906939 23024 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 18.
I20260709 13:10:25.908367 22873 raft_consensus.cc:1240] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:10:25.891378 28520 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:25.918787 22876 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 14.
I20260709 13:10:25.919140 22843 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:25.901628 28869 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:25.921612 22872 raft_consensus.cc:1240] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 19 LEADER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:10:25.930691 22872 raft_consensus.cc:1275] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:10:25.936937 28447 consensus_queue.cc:237] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:25.940011 22876 raft_consensus.cc:1240] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 16. Current term is 17. Ops: []
W20260709 13:10:25.946189 28876 log.cc:927] Time spent T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 2.436s	user 0.059s	sys 0.164s
I20260709 13:10:25.946297 28796 raft_consensus.cc:3055] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:10:25.946933 28796 raft_consensus.cc:740] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:25.902940 28868 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:10:25.949542 22877 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35360 (request call id 2467) took 2262 ms (client timeout 1600 ms). Trace:
I20260709 13:10:25.949277 22992 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:10:25.949947 22877 rpcz_store.cc:269] 0709 13:10:23.656051 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:23.658485 (+  2434us) service_pool.cc:224] Handling call
0709 13:10:25.918414 (+2259929us) spinlock_profiling.cc:243] Waited 66.1 ms on lock 0x7b5400369554. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:25.918530 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":151988608}
I20260709 13:10:25.954427 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:25.954715 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:25.955871 23020 raft_consensus.cc:2410] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 15 because replica has last-logged OpId of term: 14 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 10.
I20260709 13:10:25.958521 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:25.959661 23028 raft_consensus.cc:2393] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 15: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:25.969988 23028 raft_consensus.cc:1240] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 13:10:25.948629 28900 raft_consensus.cc:3055] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:10:25.952322 28913 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:25.956573 28868 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:25.982960 23028 raft_consensus.cc:1275] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:10:25.962756 28887 raft_consensus.cc:2804] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:25.962888 28062 raft_consensus.cc:2804] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:10:25.992805 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:25.990031 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:25.996394 28062 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:25.980418 28900 raft_consensus.cc:740] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:26.004710 22953 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 3135) took 2653 ms. Trace:
I20260709 13:10:25.996650 28775 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:26.020941 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:25.990000 28887 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:26.022933 28838 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:26.014711 28520 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:26.027067 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:26.026906 28838 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:25.986954 28796 consensus_queue.cc:260] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:26.030629 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 1.82186e+06us
I20260709 13:10:26.003751 28706 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:26.031164 28842 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:26.031605 28706 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:26.032819 28850 raft_consensus.cc:3055] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:10:26.033316 28850 raft_consensus.cc:740] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:26.035616 28898 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:26.008082 28810 consensus_queue.cc:1048] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.029s
I20260709 13:10:26.037937 28810 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:26.038430 28810 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:26.000198 28062 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:26.042917 28756 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:26.045393 28846 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:26.047753 28900 consensus_queue.cc:260] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:26.049443 28900 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:26.005270 22953 rpcz_store.cc:276] 0709 13:10:23.351264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:23.358655 (+  7391us) service_pool.cc:224] Handling call
0709 13:10:23.359061 (+   406us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:23.380413 (+ 21352us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:23.380448 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:23.386890 (+  6442us) log.cc:844] Serialized 26 byte log entry
0709 13:10:23.730165 (+343275us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:10:23.740957 (+ 10792us) log.cc:844] Serialized 14 byte log entry
0709 13:10:26.000893 (+2259936us) spinlock_profiling.cc:243] Waited 2.01 ms on lock 0x7b18004424e0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132c72c0a 00007f613c3b2219 00007f6132cfaa40 00007f6132cf54e4 00007f6132cf5455 00007f6135ddd060 00007f6135def716 00007f6135e00287 00007f6135e00208
0709 13:10:26.001112 (+   219us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:26.001211 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:26.001265 (+    54us) raft_consensus.cc:1685] finished
0709 13:10:26.004417 (+  3152us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:26.004607 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43777,"prepare.queue_time_us":20950,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":47,"raft.queue_time_us":2285972,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":88341,"spinlock_wait_cycles":4633472,"thread_start_us":230901,"threads_started":1,"wal-append.queue_time_us":349185,"wal-append.run_cpu_time_us":3906,"wal-append.run_wall_time_us":1965900}
I20260709 13:10:26.055744 28828 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:26.057010 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:26.057652 22949 raft_consensus.cc:2468] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 13.
I20260709 13:10:26.057844 28783 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:26.058666 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:26.059303 22946 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 17.
I20260709 13:10:26.060611 22955 raft_consensus.cc:1275] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 13:10:26.060817 22874 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 19.
I20260709 13:10:26.061685 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:26.065099 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:26.065750 22953 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 15.
W20260709 13:10:26.078876 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:26.080402 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:26.081245 23619 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 17.
I20260709 13:10:26.082036 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:26.082854 22874 raft_consensus.cc:2393] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 18: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:26.083704 22994 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:26.085719 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
W20260709 13:10:26.095690 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 2.22202e+06us
I20260709 13:10:26.095772 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:26.100225 23588 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:26.102363 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:26.103276 23616 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 17.
I20260709 13:10:26.104887 23618 raft_consensus.cc:1240] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 18. Current term is 19. Ops: []
I20260709 13:10:26.107679 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:26.110765 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:26.111599 23618 raft_consensus.cc:2468] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 11.
I20260709 13:10:26.056248 28828 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:26.116914 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:26.117215 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:25.998574 28917 consensus_queue.cc:1048] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.095s
I20260709 13:10:26.119431 22877 raft_consensus.cc:2393] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 12: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:26.119585 22876 raft_consensus.cc:2393] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 16: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:26.113590 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:26.119983 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:26.120597 23617 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 10 FOLLOWER]: Advancing to term 12
I20260709 13:10:26.120894 23619 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:26.114980 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:25.998757 28888 consensus_queue.cc:1059] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.095s
I20260709 13:10:26.117699 22872 raft_consensus.cc:1240] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 18. Current term is 19. Ops: []
I20260709 13:10:26.165372 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:26.125335 23624 raft_consensus.cc:1217] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Deduplicated request from leader. Original: 8.7->[10.8-14.10]   Dedup: 10.8->[12.9-14.10]
I20260709 13:10:26.166011 23624 pending_rounds.cc:85] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2: Aborting all ops after (but not including) 8
I20260709 13:10:26.166391 23624 pending_rounds.cc:107] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 13:10:26.127857 22879 raft_consensus.cc:2393] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 16: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:26.166613 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:25.992981 28913 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:26.167657 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:26.117820 28846 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:26.168743 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:26.169159 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:26.134020 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:26.166630 23624 raft_consensus.cc:2889] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:26.108122 28783 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:26.096966 28821 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:26.097095 28796 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:26.133702 28794 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:26.186318 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:26.186704 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:26.139238 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:26.036278 28898 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:26.139446 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:26.138945 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:26.140414 23588 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:26.197849 22874 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Advancing to term 19
W20260709 13:10:26.198132 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 57824.9us
I20260709 13:10:26.203845 22874 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 19.
I20260709 13:10:26.203486 28850 consensus_queue.cc:260] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:26.195997 28885 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:26.165971 22952 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:26.126528 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:26.087567 28887 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:26.170049 22953 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 17.
W20260709 13:10:26.134853 22920 connection.cc:256] RPC call timeout handler was delayed by -1.1060369669999091s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16000000000000003s for a response
I20260709 13:10:26.164196 23619 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 19.
I20260709 13:10:26.179001 23617 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
I20260709 13:10:26.126572 28867 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:26.135010 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:26.115121 28905 raft_consensus.cc:2749] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:26.185618 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:26.136092 28779 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:26.133466 28369 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:26.185926 28821 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:26.190786 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:26.191434 28828 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:26.191794 22954 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 2441) took 2562 ms. Trace:
I20260709 13:10:26.279347 22946 raft_consensus.cc:2393] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 12: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
W20260709 13:10:26.196903 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 165644us
I20260709 13:10:26.124115 23590 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
W20260709 13:10:26.210458 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:26.281399 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 157358us
I20260709 13:10:26.246863 22952 raft_consensus.cc:2410] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 15 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:10:26.266968 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:26.283103 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:26.283423 22947 raft_consensus.cc:2468] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 14.
I20260709 13:10:26.279496 22954 rpcz_store.cc:276] 0709 13:10:23.629268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:23.629404 (+   136us) service_pool.cc:224] Handling call
0709 13:10:23.629662 (+   258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:23.825449 (+195787us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:23.825482 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:23.825524 (+    42us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:10:23.825555 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:26.191560 (+2366005us) spinlock_profiling.cc:243] Waited 196 ms on lock 0x7b54000a91d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:26.191608 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:26.191742 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":449763200}
I20260709 13:10:26.284881 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:26.181125 28846 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:26.267396 22953 raft_consensus.cc:1240] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 LEADER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:10:26.248373 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:26.291971 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:26.273840 23618 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:26.293215 22947 raft_consensus.cc:2393] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 12: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:26.219691 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:26.197139 28812 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:26.250977 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:26.285336 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:26.286721 23590 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:26.183310 28783 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:26.293977 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:26.296483 23625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 1163) took 1108 ms. Trace:
I20260709 13:10:26.314030 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:26.324157 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:26.324465 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 240879us
I20260709 13:10:26.310484 22918 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:26.297065 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:26.345965 23619 raft_consensus.cc:2393] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 12: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:26.245254 28916 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:26.350167 22843 leader_election.cc:400] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 13. Current term is 14.
I20260709 13:10:26.294783 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:26.238662 22919 connection.cc:256] RPC call timeout handler was delayed by -1.7825326610000047s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15480000000000002s for a response
I20260709 13:10:26.350479 22843 leader_election.cc:403] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 13:10:26.295894 23623 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60032 (request call id 1159) took 1135 ms. Trace:
W20260709 13:10:26.350772 22919 connection.cc:256] RPC call timeout handler was delayed by -1.6889326609999671s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1652s for a response
I20260709 13:10:26.294304 22952 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15.
I20260709 13:10:26.316298 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:26.343246 23625 rpcz_store.cc:276] 0709 13:10:25.188453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:25.208065 (+ 19612us) service_pool.cc:224] Handling call
0709 13:10:25.217705 (+  9640us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:25.805953 (+588248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:26.296249 (+490296us) spinlock_profiling.cc:243] Waited 585 ms on lock 0x7b540046a854. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:26.296405 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1345533696}
I20260709 13:10:26.275594 28578 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:26.343535 22946 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 18.
I20260709 13:10:26.329010 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:26.344777 22994 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:26.338997 23618 raft_consensus.cc:2468] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 14.
W20260709 13:10:26.292928 28773 consensus_meta.cc:287] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.697s	user 0.011s	sys 0.002s
I20260709 13:10:26.244354 28813 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:26.348471 23620 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 16.
I20260709 13:10:26.350046 22918 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:26.351068 23623 rpcz_store.cc:276] 0709 13:10:25.162205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:25.166187 (+  3982us) service_pool.cc:224] Handling call
0709 13:10:26.295741 (+1129554us) spinlock_profiling.cc:243] Waited 21.7 ms on lock 0x7b5400671454. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:26.295840 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":49847040}
W20260709 13:10:26.311482 28942 log.cc:927] Time spent T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
I20260709 13:10:26.385560 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:26.386313 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:26.386547 22946 raft_consensus.cc:2393] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 16: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:26.351737 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:26.387530 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:26.388229 22946 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:26.352032 23621 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:10:26.345028 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:26.368594 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:26.282657 28850 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:26.282060 28810 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:26.345201 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 58560.4us
I20260709 13:10:26.371434 22994 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:26.315582 28838 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:26.306013 28898 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:26.307565 28821 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:26.308197 28706 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:26.400102 22994 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:26.405119 22994 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:26.375145 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:26.376925 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:26.407442 22949 raft_consensus.cc:3060] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:10:26.377617 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:26.379925 23623 raft_consensus.cc:1275] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:10:26.380286 28905 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:26.369850 28815 raft_consensus.cc:2804] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:26.316792 28846 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:26.381031 28889 raft_consensus.cc:2749] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:26.381263 22918 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:26.382161 22875 raft_consensus.cc:2410] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 13:10:26.372879 28828 raft_consensus.cc:2804] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:10:26.371652 22919 connection.cc:256] RPC call timeout handler was delayed by -1.5449326609999616s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18120000000000003s for a response
W20260709 13:10:26.429740 22919 connection.cc:256] RPC call timeout handler was delayed by -1.2938326610000104s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20910000000000004s for a response
I20260709 13:10:26.311009 28901 consensus_queue.cc:1048] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.074s
I20260709 13:10:26.287587 28867 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:26.384795 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:26.431231 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 50039.7us
I20260709 13:10:26.453661 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:26.453846 23623 raft_consensus.cc:1240] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 15. Current term is 16. Ops: []
I20260709 13:10:26.454396 22953 raft_consensus.cc:2393] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 16: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:26.306344 28913 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:26.389595 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:26.410267 28922 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:26.310005 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:26.392220 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:26.395123 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:26.394937 28578 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:26.279328 28779 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:26.396494 28813 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:26.372103 28786 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:26.398928 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 325830us
I20260709 13:10:26.310277 28790 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:26.348960 28802 raft_consensus.cc:2749] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:10:26.289162 28826 raft_consensus.cc:2764] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:10:26.411334 22994 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:26.412266 28935 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:26.413009 28369 consensus_queue.cc:1059] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:10:26.413760 22954 raft_consensus.cc:1275] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 13:10:26.417564 22993 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:26.370728 28715 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 13. Current term is 14.
I20260709 13:10:26.418359 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:26.418529 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
W20260709 13:10:26.572088 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 47689.1us
I20260709 13:10:26.572407 23618 raft_consensus.cc:3060] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:26.573380 22992 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:26.574163 28794 raft_consensus.cc:2804] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:26.389748 28685 raft_consensus.cc:2749] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:26.575034 22954 pending_rounds.cc:85] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting all ops after (but not including) 9
I20260709 13:10:26.427198 22946 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 17.
I20260709 13:10:26.575639 28935 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:26.424042 23621 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 18.
I20260709 13:10:26.575999 28935 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:26.576151 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:26.384210 28887 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:26.311271 28712 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:26.294931 28812 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:26.465348 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:26.370481 28062 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:26.410063 28916 consensus_queue.cc:1059] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.059s
I20260709 13:10:26.422077 28750 raft_consensus.cc:2804] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:26.470059 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:26.472160 23625 raft_consensus.cc:1275] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 15. (term mismatch)
I20260709 13:10:26.596983 23625 pending_rounds.cc:85] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2: Aborting all ops after (but not including) 14
I20260709 13:10:26.597247 23625 pending_rounds.cc:107] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2: Aborting uncommitted NO_OP operation due to leader change: 20.15
I20260709 13:10:26.597571 23625 raft_consensus.cc:2889] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:26.417877 28815 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:26.622567 28709 raft_consensus.cc:2804] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:10:26.623337 28709 raft_consensus.cc:697] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 14 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:26.624305 28709 consensus_queue.cc:237] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:26.450623 28898 raft_consensus.cc:2749] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:26.429467 28828 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:26.627555 28828 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:26.596302 28750 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:26.629158 28750 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:10:26.633374 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:26.635018 28783 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:26.596491 22955 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 17.
I20260709 13:10:26.638973 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:26.639590 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
W20260709 13:10:26.640717 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 58317.7us
W20260709 13:10:26.642943 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:26.643865 22950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 3141) took 2961 ms. Trace:
I20260709 13:10:26.644330 22950 rpcz_store.cc:276] 0709 13:10:23.682087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:23.682306 (+   219us) service_pool.cc:224] Handling call
0709 13:10:23.682579 (+   273us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:26.008249 (+2325670us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:26.008280 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:26.008750 (+   470us) log.cc:844] Serialized 26 byte log entry
0709 13:10:26.418939 (+410189us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:10:26.418984 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:26.597022 (+178038us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b5400149454. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:26.597069 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:26.598485 (+  1416us) spinlock_profiling.cc:243] Waited 12.8 ms on lock 0x7b4800190158. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613ad0eaa1 00007f613ad05b37 00007f613ad059ca 00007f613adfd26f 00007f613adfcf37 00007f613ae1155c 00007f613ae11508 00007f613ae11488 00007f613ae11443 00007f613ae106ec 00007f6139d3aa96
0709 13:10:26.643606 (+ 45121us) raft_consensus.cc:1685] finished
0709 13:10:26.643636 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:26.643800 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2188699,"raft.queue_time_us":2191942,"raft.run_cpu_time_us":3702,"raft.run_wall_time_us":590090,"spinlock_wait_cycles":329580032,"thread_start_us":294401,"threads_started":1,"wal-append.queue_time_us":519371}
I20260709 13:10:26.647334 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:26.571034 22949 raft_consensus.cc:2468] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 15.
I20260709 13:10:26.496733 28889 consensus_queue.cc:1048] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.078s
I20260709 13:10:26.677114 22875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35370 (request call id 3023) took 1414 ms. Trace:
I20260709 13:10:26.678385 22875 rpcz_store.cc:276] 0709 13:10:25.262366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:25.885733 (+623367us) service_pool.cc:224] Handling call
0709 13:10:26.676847 (+791114us) spinlock_profiling.cc:243] Waited 483 ms on lock 0x7b540022abd4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:26.677036 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1110691584}
I20260709 13:10:26.574787 22992 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:26.575337 22954 pending_rounds.cc:107] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 13:10:26.679612 22954 raft_consensus.cc:2889] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:10:26.679901 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 105178us
I20260709 13:10:26.309371 28807 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:26.513568 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:26.516112 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:26.422412 28756 consensus_queue.cc:1048] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 13:10:26.580443 23618 raft_consensus.cc:2468] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 15.
W20260709 13:10:26.581748 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:26.417529 28796 consensus_queue.cc:1048] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.061s
I20260709 13:10:26.583505 22947 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 17.
W20260709 13:10:26.430277 22919 connection.cc:256] RPC call timeout handler was delayed by -1.175032660999932s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2223s for a response
W20260709 13:10:26.690692 22919 connection.cc:256] RPC call timeout handler was delayed by -1.1488042470000437s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20630000000000004s for a response
I20260709 13:10:26.651988 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:26.692267 22919 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:26.583341 28902 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:26.587236 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:26.497450 23023 raft_consensus.cc:1240] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 21 LEADER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
W20260709 13:10:26.591815 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 180568us
I20260709 13:10:26.698182 23021 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 18.
I20260709 13:10:26.501581 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:26.693897 22919 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:26.586663 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:26.701716 22994 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:26.595125 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:26.703099 22946 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 16.
I20260709 13:10:26.398056 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:26.601807 28887 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:26.604261 28520 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:26.605846 28813 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:26.696607 28810 raft_consensus.cc:2749] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:26.608458 28790 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:26.610001 28528 raft_consensus.cc:2804] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 13:10:26.609752 28812 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:26.612394 23029 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
I20260709 13:10:26.496945 28786 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:26.612818 28779 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:26.614914 28578 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:26.615888 28062 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:26.616798 28802 raft_consensus.cc:3055] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:10:26.622071 28775 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:26.622602 28815 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:26.303287 28816 consensus_queue.cc:260] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:26.513828 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:26.426657 28905 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:26.677783 22880 raft_consensus.cc:1275] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 13:10:26.751010 22880 pending_rounds.cc:85] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 8
I20260709 13:10:26.751336 22880 pending_rounds.cc:107] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 13:10:26.751681 22880 raft_consensus.cc:2889] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:26.682612 22992 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:26.684284 23028 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 17.
I20260709 13:10:26.651330 22844 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:26.689585 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 46156.9us
I20260709 13:10:26.754521 22920 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:26.469274 28922 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:26.755554 28922 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:26.574869 28794 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:26.761089 28794 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:26.571237 28821 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:26.765056 28821 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:26.695504 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:26.768972 22954 raft_consensus.cc:1240] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 15. Current term is 16. Ops: []
I20260709 13:10:26.699306 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:26.701704 22952 raft_consensus.cc:1275] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:10:26.588444 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:26.774688 23025 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 15.
W20260709 13:10:26.690852 28868 consensus_meta.cc:287] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.692s	user 0.009s	sys 0.002s
I20260709 13:10:26.775790 28868 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:26.778810 28868 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:26.714727 28712 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:26.716007 28745 raft_consensus.cc:2804] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:10:26.684485 28728 raft_consensus.cc:2749] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:26.689459 28908 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:26.781036 28908 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:26.726565 22946 raft_consensus.cc:1275] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:10:26.697029 28369 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
W20260709 13:10:26.732856 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:26.732856 22776 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48906 (request call id 200) took 74075 ms (client timeout 15000 ms). Trace:
I20260709 13:10:26.689718 28706 consensus_queue.cc:1048] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.039s
W20260709 13:10:26.789053 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:26.792590 22950 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15.
I20260709 13:10:26.724203 28802 raft_consensus.cc:740] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:26.720404 28528 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:26.722388 28885 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:26.813104 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 161861us
I20260709 13:10:26.814798 22844 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:26.816361 22954 raft_consensus.cc:1275] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:10:26.718578 28685 consensus_queue.cc:1048] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.080s
W20260709 13:10:26.813194 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 130649us
I20260709 13:10:26.725952 28779 raft_consensus.cc:2749] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:10:26.817147 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 90620.5us
I20260709 13:10:26.824571 22919 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f
W20260709 13:10:26.815496 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 61058.2us
I20260709 13:10:26.717528 28946 consensus_queue.cc:1059] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.110s
I20260709 13:10:26.830375 22992 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:26.831717 22920 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:26.839881 22919 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:26.805474 28816 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:26.846634 22844 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:26.711892 28902 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:26.803165 28805 raft_consensus.cc:2749] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:26.846256 28791 consensus_queue.cc:1059] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.030s
I20260709 13:10:26.876879 28791 raft_consensus.cc:3055] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:10:26.877354 28791 raft_consensus.cc:740] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:26.878461 28791 consensus_queue.cc:260] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:26.879985 28791 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:10:26.785919 22776 rpcz_store.cc:269] 0709 13:09:12.657644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:12.657836 (+   192us) service_pool.cc:224] Handling call
0709 13:10:26.732791 (+74074955us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:10:19.838969 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:10:19.853603 (+ 14634us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:10:19.853660 (+    57us) write_op.cc:435] Acquired schema lock
0709 13:10:19.853684 (+    24us) tablet.cc:662] Decoding operations
0709 13:10:19.854305 (+   621us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:10:19.854391 (+    86us) write_op.cc:641] Partition lock acquired for write op
0709 13:10:19.854423 (+    32us) tablet.cc:685] Acquiring locks for 3 operations
0709 13:10:20.008501 (+154078us) tablet.cc:701] Row locks acquired
0709 13:10:20.017618 (+  9117us) write_op.cc:260] PREPARE: finished
0709 13:10:20.032233 (+ 14615us) write_op.cc:270] Start()
0709 13:10:20.076341 (+ 44108us) write_op.cc:276] Timestamp: P: 1783602620029186 usec, L: 0
0709 13:10:20.096751 (+ 20410us) op_driver.cc:348] REPLICATION: starting
0709 13:10:20.097475 (+   724us) log.cc:844] Serialized 1284 byte log entry
0709 13:10:20.445250 (+347775us) op_driver.cc:464] REPLICATION: finished
0709 13:10:20.526571 (+ 81321us) write_op.cc:301] APPLY: starting
0709 13:10:20.528514 (+  1943us) tablet.cc:1367] starting BulkCheckPresence
0709 13:10:20.547083 (+ 18569us) tablet.cc:1370] finished BulkCheckPresence
0709 13:10:20.547216 (+   133us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:10:20.549480 (+  2264us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:10:20.549926 (+   446us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=6,key_file_lookups=6,delta_file_lookups=6,mrs_lookups=0
0709 13:10:20.549996 (+    70us) write_op.cc:312] APPLY: finished
0709 13:10:20.550316 (+   320us) log.cc:844] Serialized 41 byte log entry
0709 13:10:20.556921 (+  6605us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:20.557060 (+   139us) write_op.cc:454] Released schema lock
0709 13:10:20.557132 (+    72us) write_op.cc:341] FINISH: Updating metrics
0709 13:10:20.557593 (+   461us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:20.557623 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4273783,"reported_tablets":32,"child_traces":[["op",{"apply.queue_time_us":77643,"apply.run_cpu_time_us":7272,"apply.run_wall_time_us":33297,"cfile_cache_hit":14,"cfile_cache_hit_bytes":16673,"num_ops":3,"prepare.queue_time_us":182044,"prepare.run_cpu_time_us":9140,"prepare.run_wall_time_us":546471,"raft.queue_time_us":78183,"raft.run_cpu_time_us":4605,"raft.run_wall_time_us":61845,"replication_time_us":348308,"thread_start_us":431198,"threads_started":4,"wal-append.queue_time_us":259014,"wal-append.run_cpu_time_us":4454,"wal-append.run_wall_time_us":1194457}]]}
I20260709 13:10:26.796550 28745 raft_consensus.cc:697] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 19 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:10:26.802963 28873 consensus_queue.cc:1175] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f3c799b1249e40119d2a5864231c6c75" responder_term: 18 status { last_received { term: 18 index: 12 } last_committed_idx: 12 last_received_current_leader { term: 18 index: 12 } }
I20260709 13:10:26.807859 28898 raft_consensus.cc:2804] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:26.808094 28528 raft_consensus.cc:3060] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:26.841248 28822 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:26.784586 28369 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:26.817662 28868 raft_consensus.cc:2749] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:26.850198 28846 raft_consensus.cc:2749] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:10:26.846926 28801 raft_consensus.cc:2804] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:10:26.860111 23624 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.055s	sys 0.050s
I20260709 13:10:26.820971 28807 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:26.908218 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 68426.8us
I20260709 13:10:26.937393 22919 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:26.857533 28813 raft_consensus.cc:3055] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:10:26.855895 22773 catalog_manager.cc:5697] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 10 to 16. New cstate: current_term: 16 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
W20260709 13:10:26.916867 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 85260.9us
I20260709 13:10:26.851413 22992 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:10:26.933355 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:26.933357 23624 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.751s	user 0.055s	sys 0.051s
I20260709 13:10:26.938611 22919 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
W20260709 13:10:26.913992 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 496449us
I20260709 13:10:26.830919 28775 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 13:10:26.943784 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 92491.3us
I20260709 13:10:26.945091 22992 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:26.946084 22992 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:26.947428 22992 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:26.916651 28706 raft_consensus.cc:2804] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:26.925627 28787 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:10:26.954666 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 253014us
I20260709 13:10:26.915886 28898 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:26.860849 28908 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:26.939711 28779 raft_consensus.cc:2749] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:10:26.961236 22919 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:26.960983 22994 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:26.961825 22993 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:26.959318 28775 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:26.939671 28813 raft_consensus.cc:740] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:26.949757 28867 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:26.967011 22919 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:26.953279 28805 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:10:26.856841 28885 raft_consensus.cc:2804] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:26.980345 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:26.968856 22992 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:26.949558 28807 raft_consensus.cc:2749] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:10:26.969030 22773 catalog_manager.cc:5697] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 9 to 12, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 12 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:26.944226 28801 raft_consensus.cc:697] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:26.970922 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:26.962460 28898 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:26.953289 28706 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:26.996274 23028 raft_consensus.cc:2393] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 12: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:26.915488 28802 consensus_queue.cc:260] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:26.960379 28810 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:26.982808 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:26.984022 23025 raft_consensus.cc:1275] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:10:26.956753 28756 raft_consensus.cc:2749] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:26.972956 28787 raft_consensus.cc:697] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:27.030196 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
W20260709 13:10:26.993301 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 351004us
I20260709 13:10:26.994354 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:26.855947 28520 consensus_queue.cc:1048] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.115s
I20260709 13:10:26.959689 28685 raft_consensus.cc:2804] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:26.995510 28867 raft_consensus.cc:2804] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:10:26.999711 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:27.000911 28706 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:26.973217 28821 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:27.005219 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:27.049556 22871 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 14.
I20260709 13:10:26.856627 28922 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:26.992625 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:26.975510 28945 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 13:10:26.980826 28885 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:27.006290 28810 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:26.925419 28822 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:27.041134 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:27.041869 23025 raft_consensus.cc:1240] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 14. Current term is 15. Ops: []
I20260709 13:10:27.001371 28790 raft_consensus.cc:2804] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:10:26.986048 22773 catalog_manager.cc:5697] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 10 to 14, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 14 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:27.028647 28887 raft_consensus.cc:2749] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:26.963130 28901 consensus_queue.cc:1048] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:10:26.981226 28775 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:27.052188 28802 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:27.054785 28786 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:27.055186 28728 raft_consensus.cc:2804] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:10:27.052914 28867 raft_consensus.cc:697] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:27.004487 28805 consensus_queue.cc:1048] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.048s
I20260709 13:10:27.041270 28685 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:27.093197 23618 raft_consensus.cc:1275] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:10:27.055636 28810 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:27.056434 28885 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:27.070525 22844 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:27.104940 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:27.040827 28745 consensus_queue.cc:237] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:27.116796 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 170667us
W20260709 13:10:27.091305 28952 log.cc:927] Time spent T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
W20260709 13:10:27.149273 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:27.148459 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 78027.9us
I20260709 13:10:27.111135 22773 catalog_manager.cc:5697] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 12 to 17, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 17 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:27.129492 23022 raft_consensus.cc:1240] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:27.116099 28902 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:27.065177 28828 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:27.116842 28950 log.cc:927] Time spent T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.204s	user 0.001s	sys 0.000s
I20260709 13:10:27.100239 28790 raft_consensus.cc:697] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 18 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:10:27.163224 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 210165us
W20260709 13:10:27.158679 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:27.105818 28905 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:27.165138 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:27.134352 28685 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:10:27.179076 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:27.049055 28813 consensus_queue.cc:260] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:27.182833 22947 raft_consensus.cc:1240] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 13. Current term is 14. Ops: []
I20260709 13:10:27.134970 28728 raft_consensus.cc:697] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:27.187374 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:27.187674 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:27.189407 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:27.175467 22843 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:10:27.193502 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 226556us
I20260709 13:10:27.195065 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:27.197523 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:27.154470 28369 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:27.143034 28829 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:27.200505 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:27.221997 22954 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 17.
I20260709 13:10:27.163558 28520 raft_consensus.cc:2749] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:10:27.205516 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:27.224843 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:27.220019 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 44629.7us
I20260709 13:10:27.201701 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:27.227065 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:27.227674 22946 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 15.
I20260709 13:10:27.228444 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:27.229794 22843 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:27.231057 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:27.232314 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:27.233479 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:27.233501 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:27.234875 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:27.243649 22954 raft_consensus.cc:2393] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 15: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:27.244177 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:27.244843 23621 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
I20260709 13:10:27.140810 28750 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:27.245422 23622 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 1181) took 1141 ms. Trace:
I20260709 13:10:27.246827 22843 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:27.248983 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:27.251848 22876 raft_consensus.cc:1240] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:10:27.251366 22773 catalog_manager.cc:5697] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 12 to 16, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 16 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:27.252058 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:27.252592 22873 raft_consensus.cc:1813] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Handling vote request from an unknown peer aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:27.252776 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:27.253103 22873 raft_consensus.cc:2393] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 14: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:27.253790 22875 raft_consensus.cc:1240] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:10:27.253862 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:27.254329 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:27.252943 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:27.256024 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:27.177701 28827 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:27.240249 23617 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 15.
I20260709 13:10:27.259167 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:27.260408 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:27.261794 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:27.263005 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:27.263599 22952 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 19.
I20260709 13:10:27.264367 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:27.265669 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:27.265839 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:27.266397 23617 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 15.
I20260709 13:10:27.266754 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:27.267766 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:27.268519 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:27.245898 23622 rpcz_store.cc:276] 0709 13:10:26.104084 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:26.104567 (+   483us) service_pool.cc:224] Handling call
0709 13:10:26.126182 (+ 21615us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:26.140311 (+ 14129us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:26.140346 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:26.171058 (+ 30712us) log.cc:844] Serialized 26 byte log entry
0709 13:10:26.584398 (+413340us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:10:26.584456 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:26.584524 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:27.245234 (+660710us) raft_consensus.cc:1685] finished
0709 13:10:27.245252 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:27.245368 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13648,"raft.queue_time_us":42561,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":10461,"thread_start_us":368995,"threads_started":1,"wal-append.queue_time_us":630435}
I20260709 13:10:27.246526 28846 raft_consensus.cc:2804] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:27.259709 22993 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:27.273432 22954 raft_consensus.cc:1240] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 18. Current term is 19. Ops: [18.14-18.14]
W20260709 13:10:27.283207 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:27.273072 22773 catalog_manager.cc:5697] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 11 to 15, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 15 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:27.275689 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:27.284099 23619 raft_consensus.cc:2368] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 14. Current term is 15.
I20260709 13:10:27.280020 23617 raft_consensus.cc:1813] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Handling vote request from an unknown peer aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:27.270320 22952 raft_consensus.cc:1240] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:10:27.133458 28801 consensus_queue.cc:237] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:27.286082 23617 raft_consensus.cc:2393] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 14: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:27.300690 22954 raft_consensus.cc:1240] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:27.300340 22773 catalog_manager.cc:5697] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 8 to 13, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 13 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:27.304178 22876 raft_consensus.cc:1240] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:10:27.304361 23616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 1182) took 1199 ms. Trace:
I20260709 13:10:27.304792 23616 rpcz_store.cc:276] 0709 13:10:26.105005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:26.113296 (+  8291us) service_pool.cc:224] Handling call
0709 13:10:26.114050 (+   754us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:26.114533 (+   483us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:26.114573 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:26.115253 (+   680us) log.cc:844] Serialized 26 byte log entry
0709 13:10:26.477047 (+361794us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:10:26.477086 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:26.477146 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:27.304163 (+827017us) raft_consensus.cc:1685] finished
0709 13:10:27.304183 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:27.304309 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":93,"raft.queue_time_us":100,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":15563,"spinlock_wait_cycles":694656,"thread_start_us":295106,"threads_started":1,"wal-append.queue_time_us":658785}
I20260709 13:10:27.274986 28715 raft_consensus.cc:2804] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:27.251576 28712 consensus_queue.cc:1048] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 13:10:27.310297 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:27.317207 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:27.316898 22773 catalog_manager.cc:5697] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 12 to 16, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 16 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:27.318287 22773 catalog_manager.cc:5697] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 10 to 11, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 11 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:27.320029 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:27.320962 28828 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:27.321103 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:27.324728 23020 raft_consensus.cc:1240] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 14. Current term is 15. Ops: []
I20260709 13:10:27.288827 28965 raft_consensus.cc:2804] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:10:27.288285 28956 log.cc:927] Time spent T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.246s	user 0.000s	sys 0.001s
I20260709 13:10:27.282382 28846 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:27.244750 28829 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:27.284777 28850 consensus_meta.cc:287] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.877s	user 0.007s	sys 0.004s
I20260709 13:10:27.319852 22773 catalog_manager.cc:5697] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 6 to 17, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 17 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:27.312297 22954 raft_consensus.cc:1275] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 13:10:27.311239 28869 consensus_meta.cc:287] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 1.348s	user 0.011s	sys 0.002s
I20260709 13:10:27.176973 28787 consensus_queue.cc:237] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:27.321127 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:27.272156 28827 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:27.342320 22773 catalog_manager.cc:5697] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 10 to 14. New cstate: current_term: 14 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
W20260709 13:10:27.360901 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 91493.2us
I20260709 13:10:27.385895 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:27.386695 23024 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:27.315418 28715 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:27.339536 28868 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:27.259389 28867 consensus_queue.cc:237] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:27.343386 28846 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:27.343039 28965 raft_consensus.cc:697] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:27.353286 28922 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:10:27.403488 28967 log.cc:927] Time spent T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
W20260709 13:10:27.404417 28968 log.cc:927] Time spent T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:10:27.405764 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:10:27.408912 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:27.409268 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:10:27.389284 28958 log.cc:927] Time spent T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.296s	user 0.000s	sys 0.001s
I20260709 13:10:27.421712 23024 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
I20260709 13:10:27.342489 28728 consensus_queue.cc:237] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:27.284332 28790 consensus_queue.cc:237] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:27.393882 28715 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:27.429898 22879 raft_consensus.cc:1240] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 14. Current term is 15. Ops: []
W20260709 13:10:27.432310 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:27.431051 22773 catalog_manager.cc:5697] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 11 to 12, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 12 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:27.397639 28868 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
W20260709 13:10:27.409528 28956 log.cc:927] Time spent T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:10:27.383389 28869 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:27.409799 28786 consensus_queue.cc:1059] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:10:27.396612 28822 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:27.435963 22773 catalog_manager.cc:5697] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 8 to 15, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 15 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:27.422820 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:10:27.427577 28957 log.cc:927] Time spent T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.261s	user 0.001s	sys 0.001s
I20260709 13:10:27.443614 28965 consensus_queue.cc:237] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:27.410262 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 163393us
I20260709 13:10:27.455031 22955 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 2451) took 1233 ms. Trace:
I20260709 13:10:27.462811 22955 rpcz_store.cc:276] 0709 13:10:26.221465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:26.701126 (+479661us) service_pool.cc:224] Handling call
0709 13:10:26.701422 (+   296us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:10:26.717663 (+ 16241us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:26.717712 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:10:26.728754 (+ 11042us) log.cc:844] Serialized 52 byte log entry
0709 13:10:27.001208 (+272454us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:10:27.257300 (+256092us) log.cc:844] Serialized 14 byte log entry
0709 13:10:27.269112 (+ 11812us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:27.269191 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:27.454758 (+185567us) raft_consensus.cc:1685] finished
0709 13:10:27.454779 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:27.454972 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21749,"prepare.queue_time_us":333904,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":2975,"raft.queue_time_us":28662,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":5002,"thread_start_us":404055,"threads_started":2,"wal-append.queue_time_us":460160}
I20260709 13:10:27.465849 28869 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:27.466147 22877 raft_consensus.cc:2393] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 17: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:27.463971 22843 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:27.468900 28922 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:27.469332 28869 raft_consensus.cc:2749] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:10:27.469409 28922 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:27.472445 22843 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:27.473703 22843 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:27.474872 22843 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:27.456118 28972 log.cc:927] Time spent T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
W20260709 13:10:27.447881 28966 log.cc:927] Time spent T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.248s	user 0.001s	sys 0.000s
I20260709 13:10:27.441816 28868 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:10:27.519433 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 44625.4us
I20260709 13:10:27.522958 22843 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:27.440392 28775 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:27.515565 28869 raft_consensus.cc:2749] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:10:27.535423 28956 log.cc:927] Time spent T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:10:27.517081 28985 raft_consensus.cc:2749] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:10:27.541909 22773 catalog_manager.cc:5697] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 8 to 13, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 13 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:27.535089 28900 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:10:27.546387 28750 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:27.526993 28984 raft_consensus.cc:2749] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:10:27.558378 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:27.555335 22843 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:27.590837 22773 catalog_manager.cc:5697] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 10 to 12, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 12 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:27.541433 28902 consensus_queue.cc:1048] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 13:10:27.599370 22773 catalog_manager.cc:5697] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 9 to 13, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 13 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:27.593379 22994 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:10:27.564760 28816 consensus_meta.cc:287] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.712s	user 0.010s	sys 0.000s
I20260709 13:10:27.595194 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:27.603598 22877 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:27.595752 22918 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:27.597481 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:27.602563 28838 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:27.606206 28838 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:27.606706 22875 raft_consensus.cc:1275] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:10:27.607604 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:27.608170 28838 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:27.609500 28987 log.cc:927] Time spent T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 13:10:27.609243 22773 catalog_manager.cc:5697] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 8 to 11, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 11 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
W20260709 13:10:27.610510 28993 log.cc:927] Time spent T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 13:10:27.611079 22773 catalog_manager.cc:5697] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 6 to 12. New cstate: current_term: 12 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:27.614125 22773 catalog_manager.cc:5697] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 9 to 15. New cstate: current_term: 15 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:27.615733 22773 catalog_manager.cc:5697] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 10 to 15, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 15 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:10:27.618940 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:27.560909 28850 consensus_queue.cc:1048] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.042s
I20260709 13:10:27.623754 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:27.627707 28969 log.cc:927] Time spent T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.278s	user 0.001s	sys 0.000s
W20260709 13:10:27.636222 28706 consensus_meta.cc:287] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.578s	user 0.008s	sys 0.001s
I20260709 13:10:27.599598 28485 consensus_queue.cc:1048] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 13:10:27.637732 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:27.638922 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:10:27.640337 28982 log.cc:927] Time spent T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 13:10:27.602735 28869 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:27.644279 28869 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:27.560643 28983 consensus_queue.cc:1048] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.084s
I20260709 13:10:27.644544 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:27.650625 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:27.651700 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:27.603024 28786 consensus_queue.cc:1059] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:10:27.653739 28786 raft_consensus.cc:3055] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:10:27.564889 28900 raft_consensus.cc:697] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 17 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:27.647249 22948 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 19.
I20260709 13:10:27.655947 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:27.662416 22918 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:27.664175 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:27.664780 22955 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15.
I20260709 13:10:27.591805 28985 raft_consensus.cc:2804] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:27.667557 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:27.625298 22773 catalog_manager.cc:5697] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: config changed from index -1 to 6, term changed from 7 to 10, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1), NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added, now has a pending config: VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1), VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2), VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3), VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 10 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:27.667196 28795 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:27.649196 28706 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:27.647229 28974 raft_consensus.cc:2749] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:27.646768 28869 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:27.667759 28873 consensus_queue.cc:1059] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.008s
W20260709 13:10:27.675928 28810 consensus_meta.cc:287] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.518s	user 0.007s	sys 0.000s
I20260709 13:10:27.676753 28970 consensus_queue.cc:1059] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:10:27.654119 28786 raft_consensus.cc:740] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:27.678167 28786 consensus_queue.cc:260] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:27.679338 28786 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:27.599151 28922 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:27.681862 28985 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:27.682345 28985 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:27.682408 28827 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:27.682866 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:27.683455 23021 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:27.676828 28913 consensus_queue.cc:1048] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:10:27.686390 22773 catalog_manager.cc:5697] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 8 to 14, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 14 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:27.668722 23028 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 17.
I20260709 13:10:27.689556 22843 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:27.701715 23021 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 14.
I20260709 13:10:27.635004 28709 raft_consensus.cc:2804] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:27.708945 22773 catalog_manager.cc:5697] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 11 to 15. New cstate: current_term: 15 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:27.710988 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:27.711849 23028 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
I20260709 13:10:27.713033 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:27.713701 23028 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15.
W20260709 13:10:27.715548 22992 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.721s	user 0.025s	sys 0.031s
W20260709 13:10:27.716102 22992 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.721s	user 0.025s	sys 0.031s
W20260709 13:10:27.700994 28528 consensus_meta.cc:287] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.764s	user 0.006s	sys 0.005s
I20260709 13:10:27.670017 22918 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:27.722232 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:27.722644 22919 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:10:27.673957 28815 consensus_meta.cc:287] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.871s	user 0.010s	sys 0.001s
I20260709 13:10:27.714864 28984 raft_consensus.cc:2749] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:27.672930 28706 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:27.670939 28816 raft_consensus.cc:2749] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:27.670917 28795 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:27.724375 28815 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:27.727216 28815 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:27.728015 23024 raft_consensus.cc:2393] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 18: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:27.673538 28775 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:27.729754 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:27.729794 28775 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:27.730940 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:27.731992 28775 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:27.717515 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:27.716682 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 747789us
I20260709 13:10:27.674969 28715 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:27.719142 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:27.654541 28885 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:27.707224 28709 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:27.676393 28810 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:27.733189 22919 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:27.745183 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:27.765663 28974 raft_consensus.cc:2749] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:27.767851 23026 raft_consensus.cc:1240] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 19 LEADER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 18. Current term is 19. Ops: []
I20260709 13:10:27.770539 23028 raft_consensus.cc:1240] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 13. Current term is 14. Ops: []
I20260709 13:10:27.768774 23021 raft_consensus.cc:2393] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 17: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
W20260709 13:10:27.732472 28898 consensus_meta.cc:287] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.705s	user 0.006s	sys 0.004s
W20260709 13:10:27.805688 28685 consensus_meta.cc:287] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.600s	user 0.010s	sys 0.000s
W20260709 13:10:27.793779 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 123812us
I20260709 13:10:27.800442 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:27.817180 22876 raft_consensus.cc:2393] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 18: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:27.819725 28715 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:27.814273 28900 consensus_queue.cc:237] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:27.787674 22877 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 17.
I20260709 13:10:27.825999 22918 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:27.828817 22871 raft_consensus.cc:1240] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:27.828868 22920 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:27.800635 22773 catalog_manager.cc:5697] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 9 to 14, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 14 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:27.766830 22992 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:27.763309 28795 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:27.796458 28995 log.cc:927] Time spent T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 13:10:27.830456 28709 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:27.733835 28528 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:27.798460 28787 raft_consensus.cc:2749] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:27.830056 22920 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:27.806201 23023 raft_consensus.cc:2393] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 19: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
W20260709 13:10:27.969130 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 139133us
I20260709 13:10:27.830255 22918 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:27.969950 22920 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:27.970710 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 140504us
I20260709 13:10:27.970922 28974 raft_consensus.cc:2749] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:10:27.971794 28974 raft_consensus.cc:2749] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:27.972402 28974 raft_consensus.cc:2749] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:27.973768 22920 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:27.768940 28868 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:27.814092 28970 raft_consensus.cc:2749] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:27.816817 28898 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:27.829213 28794 consensus_meta.cc:287] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 1.067s	user 0.009s	sys 0.004s
I20260709 13:10:27.825968 28685 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:27.977432 28794 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:27.831007 22994 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:27.978770 28685 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:27.848879 22952 raft_consensus.cc:1240] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:10:27.978979 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:27.979521 28794 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:27.980122 22953 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:27.849895 28846 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:27.982700 28709 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:27.983886 28846 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:27.848773 28873 raft_consensus.cc:2804] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:10:27.984835 28709 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:27.985065 23024 raft_consensus.cc:1240] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 17. Current term is 18. Ops: []
I20260709 13:10:27.852815 28821 raft_consensus.cc:3055] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:10:27.985651 28821 raft_consensus.cc:740] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:27.856288 28810 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:27.856444 28885 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:10:27.858877 28935 consensus_meta.cc:287] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 1.282s	user 0.003s	sys 0.013s
I20260709 13:10:27.987450 28935 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:27.989007 28578 raft_consensus.cc:3055] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:10:27.989545 28578 raft_consensus.cc:740] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:27.989667 28935 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:27.862586 28829 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:27.831947 28485 raft_consensus.cc:3060] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:27.990502 28578 consensus_queue.cc:260] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:27.992015 28578 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:27.993105 23027 raft_consensus.cc:1240] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:10:27.993422 28898 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:27.986514 28821 consensus_queue.cc:260] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:27.995174 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:10:27.811405 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 78280.5us
W20260709 13:10:27.877142 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 110417us
I20260709 13:10:27.882642 28985 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:27.879601 22773 catalog_manager.cc:5697] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 10 to 12, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 12 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:27.959317 22947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 2452) took 1737 ms. Trace:
I20260709 13:10:27.889266 28991 raft_consensus.cc:2804] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:27.977759 28868 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:27.851509 28791 consensus_meta.cc:287] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.971s	user 0.006s	sys 0.008s
I20260709 13:10:27.995231 28821 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:27.965363 28826 raft_consensus.cc:3055] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:10:28.002404 28826 raft_consensus.cc:740] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:27.997277 22919 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:27.997731 22992 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:27.979560 23022 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18.
I20260709 13:10:27.998808 28985 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:28.004933 23027 raft_consensus.cc:1240] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 14. Current term is 15. Ops: []
I20260709 13:10:27.999390 22773 catalog_manager.cc:5697] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 12 to 17, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 17 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:28.006152 23022 raft_consensus.cc:1275] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:10:28.006891 23027 raft_consensus.cc:1240] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:10:27.999687 28991 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
W20260709 13:10:28.007699 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:27.999531 22947 rpcz_store.cc:276] 0709 13:10:26.221847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:26.705459 (+483612us) service_pool.cc:224] Handling call
0709 13:10:26.705734 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:27.185385 (+479651us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:27.958969 (+773584us) spinlock_profiling.cc:243] Waited 477 ms on lock 0x7b540020dd54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:27.959213 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1096626944}
I20260709 13:10:28.008157 28991 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:28.003643 28826 consensus_queue.cc:260] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:28.010998 28813 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:28.006383 22773 catalog_manager.cc:5697] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 13 to 17, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 17 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:28.018172 22953 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 19.
I20260709 13:10:28.018663 28485 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:10:28.020157 28977 log.cc:927] Time spent T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.626s	user 0.003s	sys 0.000s
I20260709 13:10:28.020884 23020 raft_consensus.cc:1240] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 11. Current term is 12. Ops: []
I20260709 13:10:28.022017 28528 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:28.023355 23020 raft_consensus.cc:1240] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 16. Current term is 17. Ops: []
W20260709 13:10:28.024003 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:28.025223 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 51525.3us
I20260709 13:10:28.025939 23020 raft_consensus.cc:1240] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:10:28.027141 23020 raft_consensus.cc:1240] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:10:28.034629 22773 catalog_manager.cc:5697] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 10 to 13, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 13 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:28.034418 28787 raft_consensus.cc:2749] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:28.036437 22773 catalog_manager.cc:5697] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 11 to 14. New cstate: current_term: 14 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:28.037412 28873 raft_consensus.cc:697] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:28.037822 22773 catalog_manager.cc:5697] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 7 to 12, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 12 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:28.038390 28873 consensus_queue.cc:237] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:28.040786 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:28.045028 23624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60044 (request call id 1055) took 1962 ms. Trace:
W20260709 13:10:28.045176 22948 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58320 (request call id 3158) took 1877 ms (client timeout 1592 ms). Trace:
W20260709 13:10:28.045529 22948 rpcz_store.cc:269] 0709 13:10:26.169108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:26.169754 (+   646us) service_pool.cc:224] Handling call
0709 13:10:28.045030 (+1875276us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b5400411754. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:28.045120 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3357073792}
I20260709 13:10:28.046533 22949 raft_consensus.cc:1275] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 13:10:28.046923 22949 pending_rounds.cc:85] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting all ops after (but not including) 8
I20260709 13:10:28.047158 22949 pending_rounds.cc:107] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:10:28.047394 22949 raft_consensus.cc:2889] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:10:28.047436 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 50254us
I20260709 13:10:28.048281 22919 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:28.052417 22919 leader_election.cc:400] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 14. Current term is 15.
I20260709 13:10:28.052587 23623 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 1101) took 1842 ms. Trace:
I20260709 13:10:28.052824 22919 leader_election.cc:403] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:10:28.054224 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:10:28.055135 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 348543us
I20260709 13:10:28.055902 22874 raft_consensus.cc:1240] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:10:28.053074 23623 rpcz_store.cc:276] 0709 13:10:26.210245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:26.470206 (+259961us) service_pool.cc:224] Handling call
0709 13:10:26.572111 (+101905us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:26.572355 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:10:26.978577 (+406222us) log.cc:844] Serialized 14 byte log entry
0709 13:10:28.052226 (+1073649us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:28.052294 (+    68us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:10:28.052338 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:28.052402 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:28.052534 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":635904,"prepare.queue_time_us":340518,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":10950,"thread_start_us":829065,"threads_started":2,"wal-append.queue_time_us":560024}
I20260709 13:10:28.056988 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:28.058519 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:28.060238 22875 raft_consensus.cc:2368] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 18. Current term is 19.
W20260709 13:10:28.061213 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 63571.7us
I20260709 13:10:28.049302 28813 raft_consensus.cc:2804] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:10:28.049753 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 791298us
I20260709 13:10:28.049661 28898 consensus_queue.cc:1059] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:10:28.050382 28485 consensus_queue.cc:1059] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:10:28.051064 28791 consensus_queue.cc:1059] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:10:28.065053 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 234093us
W20260709 13:10:28.077145 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:28.069562 28813 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:28.080212 28813 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:28.066251 22843 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:28.066473 23588 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:28.081688 22843 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:28.082590 28985 raft_consensus.cc:2804] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:10:28.082986 22843 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:28.083595 28868 raft_consensus.cc:2749] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:10:28.084815 28868 raft_consensus.cc:2749] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:10:28.085971 22873 raft_consensus.cc:1240] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16. Ops: []
I20260709 13:10:28.083072 28985 raft_consensus.cc:697] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 13 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:28.087740 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:28.088088 28985 consensus_queue.cc:237] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:28.088876 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:28.090103 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:28.090816 22876 raft_consensus.cc:2393] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 15: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:28.091645 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:28.092383 22878 raft_consensus.cc:2393] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 18: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:28.087098 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:28.093791 22875 raft_consensus.cc:2393] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 19: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:28.095562 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:28.096639 28813 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:28.097936 22843 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:28.102033 28868 raft_consensus.cc:2804] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:28.102169 22843 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:28.102439 28868 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:28.102824 28868 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:28.102963 28900 raft_consensus.cc:2804] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:28.103379 28900 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:28.103601 28813 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:28.103706 28900 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:28.104403 22843 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:28.105316 22843 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:28.106755 28906 raft_consensus.cc:2804] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:28.107043 28775 raft_consensus.cc:2749] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:10:28.107142 28906 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:28.107832 28906 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:28.108881 28902 consensus_queue.cc:1059] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:10:28.109948 28902 consensus_queue.cc:1059] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:10:28.110857 28902 consensus_queue.cc:1048] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:10:28.111657 28984 raft_consensus.cc:3055] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:10:28.112066 28984 raft_consensus.cc:740] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:28.112888 28984 consensus_queue.cc:260] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:28.113462 28900 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:28.120275 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 53921us
I20260709 13:10:28.121070 23588 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:10:28.122843 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 45234.5us
W20260709 13:10:28.123087 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 63365.6us
W20260709 13:10:28.124825 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 72583.8us
I20260709 13:10:28.123566 22992 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:28.067592 28829 raft_consensus.cc:2749] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:28.134229 23588 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f
W20260709 13:10:28.129534 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:28.130591 22875 raft_consensus.cc:1275] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:10:28.131254 22920 leader_election.cc:400] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 18. Current term is 19.
I20260709 13:10:28.076143 28528 raft_consensus.cc:2804] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:28.136703 22920 leader_election.cc:403] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:10:28.142292 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:28.139942 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:28.140719 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:28.146553 22950 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 13.
I20260709 13:10:28.141528 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:28.139256 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:28.157789 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:28.161299 22952 raft_consensus.cc:1240] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 17. Current term is 18. Ops: []
I20260709 13:10:28.162593 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:28.163431 22952 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:28.166668 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:28.168119 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:28.119688 28885 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:10:28.115779 28867 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:28.170135 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:28.171540 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:10:28.173519 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 80201.5us
I20260709 13:10:28.174037 22950 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:10:28.171780 23028 raft_consensus.cc:1240] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:10:28.122071 28984 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:28.215309 28775 consensus_queue.cc:1059] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 13:10:28.150076 28783 raft_consensus.cc:2749] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:28.215471 22954 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 15.
I20260709 13:10:28.304551 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:28.305557 22954 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:28.236832 28818 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:10:28.129046 28709 consensus_queue.cc:1048] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.096s
I20260709 13:10:28.173370 28578 raft_consensus.cc:2749] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 14. Current term is 15.
I20260709 13:10:28.322535 22954 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 18.
I20260709 13:10:28.334511 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:28.142643 28796 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:28.230688 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:28.224413 28801 raft_consensus.cc:3055] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:10:28.129287 28908 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:28.172480 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:28.142565 28528 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:28.173240 28795 consensus_queue.cc:1059] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 13:10:28.229094 28885 raft_consensus.cc:697] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 17 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:28.370571 28796 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:28.371685 28885 consensus_queue.cc:237] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:28.373296 28796 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:28.258977 22952 raft_consensus.cc:2468] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 15.
I20260709 13:10:28.171005 28756 raft_consensus.cc:2804] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:28.141919 28369 raft_consensus.cc:2804] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:10:28.237124 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 106059us
I20260709 13:10:28.174444 28935 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:28.197435 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:28.392143 22948 raft_consensus.cc:2368] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16.
I20260709 13:10:28.123100 28715 raft_consensus.cc:3055] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:10:28.223325 22946 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 17.
I20260709 13:10:28.279377 22918 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:28.231698 22950 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 15.
W20260709 13:10:28.299194 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:28.399809 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 120520us
I20260709 13:10:28.300258 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:28.199613 22992 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:28.230373 28901 consensus_queue.cc:1059] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 13:10:28.215179 28867 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:28.239068 28787 raft_consensus.cc:2749] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 18. Current term is 19.
I20260709 13:10:28.237149 28991 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:28.311050 28818 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
W20260709 13:10:28.365460 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:28.350004 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:28.375985 28756 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
W20260709 13:10:28.401834 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 226268us
I20260709 13:10:28.402772 22875 raft_consensus.cc:1275] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 13:10:28.375025 29005 log.cc:927] Time spent T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.341s	user 0.000s	sys 0.002s
I20260709 13:10:28.405012 22878 raft_consensus.cc:1275] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:10:28.406500 22955 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 15.
I20260709 13:10:28.374823 28528 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:28.366633 28801 raft_consensus.cc:740] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:28.405540 28791 raft_consensus.cc:2804] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:10:28.417866 22992 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:28.394886 28715 raft_consensus.cc:740] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:28.416378 28818 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:28.404428 28976 raft_consensus.cc:2749] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:10:28.419266 28946 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
W20260709 13:10:28.417861 29007 log.cc:927] Time spent T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.193s	user 0.000s	sys 0.002s
I20260709 13:10:28.431480 23027 raft_consensus.cc:1240] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 17. Current term is 18. Ops: []
W20260709 13:10:28.419263 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 295381us
I20260709 13:10:28.408268 28795 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:28.391882 28369 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:28.376118 28908 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:28.419972 28756 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:28.401784 28935 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:28.450959 22992 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:28.336959 28783 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:28.433779 28791 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:28.416363 28578 consensus_queue.cc:1059] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 13:10:28.433146 28901 consensus_queue.cc:1059] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.027s
W20260709 13:10:28.477160 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 342937us
I20260709 13:10:28.492141 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:28.493764 23625 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 19.
I20260709 13:10:28.493927 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:28.478572 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:28.445350 28485 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:28.480297 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:28.422387 28787 consensus_queue.cc:1048] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 13:10:28.427345 28889 consensus_queue.cc:1048] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.028s
I20260709 13:10:28.453969 28709 raft_consensus.cc:2749] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:10:28.433180 28816 consensus_queue.cc:1048] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:10:28.418241 28873 consensus_queue.cc:1059] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
W20260709 13:10:28.498123 29011 log.cc:927] Time spent T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.183s	user 0.000s	sys 0.002s
I20260709 13:10:28.492224 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
W20260709 13:10:28.500164 29014 log.cc:927] Time spent T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 13:10:28.494572 23588 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:28.494728 23623 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 21.
I20260709 13:10:28.497079 23616 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 17.
I20260709 13:10:28.484855 28369 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:28.498651 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:28.483898 28959 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:28.454286 28887 consensus_queue.cc:1059] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.041s
I20260709 13:10:28.491287 28791 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:28.503228 23625 raft_consensus.cc:1275] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:10:28.454326 28946 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:28.504284 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:28.492116 28913 raft_consensus.cc:2749] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:10:28.495251 23618 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:28.507475 23622 raft_consensus.cc:1275] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:10:28.484404 28795 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:28.509574 23620 raft_consensus.cc:1275] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:10:28.509358 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:28.492308 28715 consensus_queue.cc:260] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:28.501397 28935 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:28.487833 28783 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:28.508997 29012 log.cc:927] Time spent T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
W20260709 13:10:28.510888 29016 log.cc:927] Time spent T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 13:10:28.533463 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:28.500294 28485 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:28.538463 28715 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:28.497277 28801 consensus_queue.cc:260] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:28.541605 28801 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:28.517385 28959 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:28.523947 28709 raft_consensus.cc:3055] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:10:28.523999 28796 raft_consensus.cc:2749] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:28.551741 28709 raft_consensus.cc:740] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:28.560465 28709 consensus_queue.cc:260] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:28.561676 28709 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:28.556181 28783 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:28.556355 28868 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:28.557758 28946 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:28.528980 28906 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:28.558458 28867 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:28.545874 28991 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:28.565371 28791 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:28.570392 28791 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:28.557597 28908 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:28.571996 28795 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:28.572010 28959 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:28.572435 28868 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:28.573103 28906 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:28.173579 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 51199us
W20260709 13:10:28.615698 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:28.621503 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:28.435454 22878 raft_consensus.cc:1240] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 14. Current term is 15. Ops: []
W20260709 13:10:28.504572 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:28.490427 22992 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:10:28.635496 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 145062us
I20260709 13:10:28.639716 23026 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 2460) took 1559 ms. Trace:
W20260709 13:10:28.642159 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 111987us
I20260709 13:10:28.644318 23029 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 2459) took 1590 ms. Trace:
I20260709 13:10:28.657661 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:28.658835 23620 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:28.640378 23026 rpcz_store.cc:276] 0709 13:10:27.080586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:27.080778 (+   192us) service_pool.cc:224] Handling call
0709 13:10:27.100300 (+ 19522us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:27.810165 (+709865us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:28.638840 (+828675us) spinlock_profiling.cc:243] Waited 667 ms on lock 0x7b540010bb54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:28.639372 (+   532us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1534629248}
I20260709 13:10:28.662344 23623 raft_consensus.cc:1275] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:10:28.662776 28485 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:28.664211 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:28.665223 23625 raft_consensus.cc:3060] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:28.666393 28787 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:28.666636 22992 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:28.670271 22992 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:28.671458 28712 raft_consensus.cc:3055] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 11 LEADER]: Stepping down as leader of term 11
I20260709 13:10:28.673157 28783 consensus_queue.cc:1048] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:10:28.644912 23029 rpcz_store.cc:276] 0709 13:10:27.054216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:27.054340 (+   124us) service_pool.cc:224] Handling call
0709 13:10:27.055163 (+   823us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:27.055495 (+   332us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:10:27.055527 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:27.086316 (+ 30789us) log.cc:844] Serialized 26 byte log entry
0709 13:10:27.969905 (+883589us) spinlock_profiling.cc:243] Waited 7.88 ms on lock 0x7b48001bc9d8. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613ad0eaa1 00007f613ad05b37 00007f613ad059ca 00007f613adfd26f 00007f613adfcf37 00007f613ae1155c 00007f613ae11508 00007f613ae11488 00007f613ae11443 00007f613ae106ec 00007f6139d3aa96
0709 13:10:28.048574 (+ 78669us) spinlock_profiling.cc:243] Waited 3.36 ms on lock 0x7b18000803e0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cdae07 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf7c02 00007f613597f20b 00007f6135984d23 00007f61359850d1 00007f6135da0a0f
0709 13:10:28.283616 (+235042us) raft_consensus.cc:1645] Marking committed up to 7
0709 13:10:28.283748 (+   132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:28.284084 (+   336us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:28.643682 (+359598us) raft_consensus.cc:1685] finished
0709 13:10:28.643722 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:28.644112 (+   390us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65206,"raft.queue_time_us":67865,"raft.run_cpu_time_us":3125,"raft.run_wall_time_us":841877,"spinlock_wait_cycles":26049920,"thread_start_us":234667,"threads_started":1,"wal-append.queue_time_us":1296171}
I20260709 13:10:28.677075 28685 raft_consensus.cc:2749] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:28.677337 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:28.677762 23024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 2403) took 1357 ms. Trace:
I20260709 13:10:28.678414 23622 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 15.
I20260709 13:10:28.682013 22875 raft_consensus.cc:1240] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:10:28.682935 28062 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:28.678228 23024 rpcz_store.cc:276] 0709 13:10:27.320472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:27.731835 (+411363us) service_pool.cc:224] Handling call
0709 13:10:27.732116 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:27.985465 (+253349us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:28.677215 (+691750us) spinlock_profiling.cc:243] Waited 253 ms on lock 0x7b54001c89d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:28.677576 (+   361us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":581314944}
I20260709 13:10:28.667467 28787 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:28.672209 28712 raft_consensus.cc:740] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:28.676307 28783 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:28.691740 23023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33418 (request call id 2406) took 1302 ms. Trace:
W20260709 13:10:28.693075 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:28.693555 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:28.699391 28779 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:28.045423 23624 rpcz_store.cc:276] 0709 13:10:26.082724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:26.084772 (+  2048us) service_pool.cc:224] Handling call
0709 13:10:26.117834 (+ 33062us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:10:26.942132 (+824298us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:10:26.958534 (+ 16402us) log.cc:844] Serialized 14 byte log entry
0709 13:10:27.224399 (+265865us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:10:27.230648 (+  6249us) log.cc:844] Serialized 52 byte log entry
0709 13:10:27.230813 (+   165us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:10:27.231028 (+   215us) log.cc:844] Serialized 14 byte log entry
0709 13:10:27.231325 (+   297us) log.cc:844] Serialized 14 byte log entry
0709 13:10:27.260053 (+ 28728us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:27.260150 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:27.817687 (+557537us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b18003ebf00. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cf83f3 00007f6132cfa7c1 00007f6132cfa779 00007f6132cfa709 00007f6132cfa6d1 00007f6132cf99cd 000000000038f354 000000000038f199 00007f6132cdb83a
0709 13:10:28.019595 (+201908us) spinlock_profiling.cc:243] Waited 9.17 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:10:28.044031 (+ 24436us) spinlock_profiling.cc:243] Waited 19.5 ms on lock 0x7b18003ebf00. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cf83f3 00007f6132cfa7c1 00007f6132cfa779 00007f6132cfa709 00007f6132cfa6d1 00007f6132cf99cd 000000000038f354 000000000038f199 00007f6132cdb83a
0709 13:10:28.044806 (+   775us) raft_consensus.cc:1685] finished
0709 13:10:28.044829 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:28.044974 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":840909,"prepare.queue_time_us":871936,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":95,"raft.queue_time_us":80060,"raft.run_cpu_time_us":450,"raft.run_wall_time_us":47156,"spinlock_wait_cycles":127335552,"thread_start_us":963650,"threads_started":2,"wal-append.queue_time_us":309469}
I20260709 13:10:28.709846 22953 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58310 (request call id 2473) took 1522 ms. Trace:
I20260709 13:10:28.717252 23023 rpcz_store.cc:276] 0709 13:10:27.389481 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:27.409015 (+ 19534us) service_pool.cc:224] Handling call
0709 13:10:28.691263 (+1282248us) spinlock_profiling.cc:243] Waited 384 ms on lock 0x7b5400359b54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:28.691483 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":883287936}
I20260709 13:10:28.734561 22878 raft_consensus.cc:1240] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 15. Current term is 16. Ops: []
I20260709 13:10:28.734019 28712 consensus_queue.cc:260] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:28.729358 22953 rpcz_store.cc:276] 0709 13:10:27.186987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:27.187141 (+   154us) service_pool.cc:224] Handling call
0709 13:10:28.709319 (+1522178us) spinlock_profiling.cc:243] Waited 792 ms on lock 0x7b5400425754. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:28.709548 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1821613440}
I20260709 13:10:28.750918 28787 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:28.761083 28813 raft_consensus.cc:3055] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:10:28.774916 28813 raft_consensus.cc:740] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:28.775236 28827 raft_consensus.cc:2804] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:10:28.775988 28827 raft_consensus.cc:697] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:28.776703 22993 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:28.776376 28813 consensus_queue.cc:260] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:28.777899 28813 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:28.779052 28991 raft_consensus.cc:2749] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:28.779932 28991 consensus_queue.cc:1048] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:10:28.775098 28796 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:28.781518 28796 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:28.777350 28827 consensus_queue.cc:237] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:28.763347 28062 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:28.787228 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 48593.3us
I20260709 13:10:28.759462 29018 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:28.787515 28712 raft_consensus.cc:3060] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 11 FOLLOWER]: Advancing to term 12
I20260709 13:10:28.772276 28887 raft_consensus.cc:2804] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:10:28.794406 23625 raft_consensus.cc:2468] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
I20260709 13:10:28.574831 28818 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:28.795222 29023 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:28.797114 29023 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:28.797724 28062 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:28.797520 29018 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:28.798321 28818 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:28.799502 29023 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:28.799962 29018 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:10:28.800648 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 71627.5us
I20260709 13:10:28.744522 28783 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:28.766659 28779 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:28.803819 28887 raft_consensus.cc:697] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 18 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:28.813040 28887 consensus_queue.cc:237] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:28.814432 28783 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:28.814854 29021 log.cc:927] Time spent T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.258s	user 0.001s	sys 0.000s
I20260709 13:10:28.816529 22951 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 2450) took 2595 ms. Trace:
I20260709 13:10:28.818975 28796 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:28.817862 22951 rpcz_store.cc:276] 0709 13:10:26.221120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:26.470620 (+249500us) service_pool.cc:224] Handling call
0709 13:10:26.573498 (+102878us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:26.610774 (+ 37276us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:26.610806 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:26.611267 (+   461us) log.cc:844] Serialized 26 byte log entry
0709 13:10:26.926097 (+314830us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:10:27.136816 (+210719us) log.cc:844] Serialized 14 byte log entry
0709 13:10:27.138033 (+  1217us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:27.138134 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:28.800263 (+1662129us) spinlock_profiling.cc:243] Waited 4.22 ms on lock 0x7b240041fd08. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132b7e9c4 00007f6132b7cacb 00007f6132d37dc9 00007f6132d37a55 00007f6132d36fb0 00007f61359ad488 00007f61359adbe8 00007f6135985398 00007f613598076a 00007f613597fbb6
0709 13:10:28.816028 (+ 15765us) raft_consensus.cc:1685] finished
0709 13:10:28.816135 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:28.816331 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36858,"prepare.queue_time_us":212911,"raft.queue_time_us":53477,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":12501,"spinlock_wait_cycles":28708352,"thread_start_us":495639,"threads_started":2,"wal-append.queue_time_us":437852}
W20260709 13:10:28.801393 23588 connection.cc:256] RPC call timeout handler was delayed by -1.1910645090000571s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1559s for a response
I20260709 13:10:28.822517 22949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 2475) took 1634 ms. Trace:
W20260709 13:10:28.827216 23588 connection.cc:256] RPC call timeout handler was delayed by -1.1037645090000296s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1656s for a response
I20260709 13:10:28.827160 23620 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 20.
W20260709 13:10:28.827785 23588 connection.cc:256] RPC call timeout handler was delayed by -1.0326645090000284s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17350000000000002s for a response
I20260709 13:10:28.826035 22949 rpcz_store.cc:276] 0709 13:10:27.187621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:27.189011 (+  1390us) service_pool.cc:224] Handling call
0709 13:10:27.225552 (+ 36541us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:28.047945 (+822393us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:28.821955 (+774010us) spinlock_profiling.cc:243] Waited 821 ms on lock 0x7b5400411254. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:28.822283 (+   328us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1887204736}
I20260709 13:10:28.826838 29023 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:28.833973 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:28.844398 22992 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:10:28.846566 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:28.839982 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 138846us
I20260709 13:10:28.835386 28779 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:28.850991 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:28.853822 23021 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:28.853097 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:10:28.856894 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:28.854507 29034 log.cc:927] Time spent T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.069s	user 0.000s	sys 0.002s
I20260709 13:10:28.866110 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:28.868700 23025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 2479) took 1173 ms. Trace:
I20260709 13:10:28.874055 28885 raft_consensus.cc:2804] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:10:28.874706 28885 raft_consensus.cc:697] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:28.876036 28821 raft_consensus.cc:3055] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 13:10:28.875862 28885 consensus_queue.cc:237] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:28.878996 28821 raft_consensus.cc:740] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:28.879256 23618 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18.
I20260709 13:10:28.890102 23588 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:28.898979 23588 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:28.900996 23023 raft_consensus.cc:2368] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 18. Current term is 19.
I20260709 13:10:28.904893 28821 consensus_queue.cc:260] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:28.890614 29023 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:28.906530 28821 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:28.909155 28796 raft_consensus.cc:2749] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 13:10:28.910018 28796 raft_consensus.cc:2764] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:10:28.909679 28369 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:28.912056 28369 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:28.912814 22992 leader_election.cc:400] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16.
I20260709 13:10:28.913136 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:28.915230 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:28.915341 28826 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:10:28.916285 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:28.917845 28709 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:28.918541 28709 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:28.929642 22951 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 18.
I20260709 13:10:28.933701 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:28.921039 28709 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:28.869432 23025 rpcz_store.cc:276] 0709 13:10:27.695062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:28.016759 (+321697us) service_pool.cc:224] Handling call
0709 13:10:28.017042 (+   283us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:10:28.069271 (+ 52229us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 13:10:28.238958 (+169687us) log.cc:844] Serialized 14 byte log entry
0709 13:10:28.491836 (+252878us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:10:28.498482 (+  6646us) log.cc:844] Serialized 52 byte log entry
0709 13:10:28.498630 (+   148us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:10:28.498853 (+   223us) log.cc:844] Serialized 14 byte log entry
0709 13:10:28.499055 (+   202us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:28.499122 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:28.813932 (+314810us) spinlock_profiling.cc:243] Waited 2.28 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:10:28.868064 (+ 54132us) raft_consensus.cc:1685] finished
0709 13:10:28.868096 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:28.868372 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46266,"prepare.queue_time_us":224861,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":4435,"raft.queue_time_us":56533,"raft.run_cpu_time_us":361,"raft.run_wall_time_us":46359,"spinlock_wait_cycles":5246336,"thread_start_us":285058,"threads_started":2,"wal-append.queue_time_us":225218}
I20260709 13:10:28.938205 23588 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:28.911134 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:28.941109 23617 raft_consensus.cc:2393] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 16: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:28.942078 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:28.947095 22843 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:28.949174 22843 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:28.949410 29006 raft_consensus.cc:2749] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:10:28.953750 29036 log.cc:927] Time spent T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:10:28.955209 29035 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:28.956141 29035 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:28.957263 28984 raft_consensus.cc:2749] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:10:28.961804 23021 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18.
I20260709 13:10:28.962324 22993 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:28.963528 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:28.965080 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:28.969337 29008 log.cc:927] Time spent T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.537s	user 0.000s	sys 0.005s
I20260709 13:10:28.969708 28898 raft_consensus.cc:3055] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:10:28.970288 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:28.971673 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 78260.6us
I20260709 13:10:28.973272 22877 raft_consensus.cc:1240] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:10:28.978209 23029 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 15.
I20260709 13:10:28.976212 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:28.991753 28900 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:28.992122 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:28.993221 22880 raft_consensus.cc:2393] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 16: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:28.993125 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:28.994261 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:28.989306 28898 raft_consensus.cc:740] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
W20260709 13:10:28.997207 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 217955us
I20260709 13:10:28.997813 28898 consensus_queue.cc:260] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:28.998356 22993 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:28.998896 28898 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:28.999342 22993 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:29.004588 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:29.005353 22955 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:29.005260 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:28.992050 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:29.009579 22950 raft_consensus.cc:2393] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 15: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
W20260709 13:10:29.010740 28528 consensus_meta.cc:287] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.570s	user 0.012s	sys 0.008s
I20260709 13:10:29.011229 28528 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:29.013391 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:29.013695 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:29.014199 28709 consensus_queue.cc:1048] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:10:29.015095 28827 raft_consensus.cc:2804] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:29.015657 28685 raft_consensus.cc:2804] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 13:10:29.034729 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:29.034749 28827 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:29.035410 28827 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:29.037056 28818 raft_consensus.cc:2749] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:29.016191 28991 raft_consensus.cc:2749] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:29.037493 28805 consensus_queue.cc:1059] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:10:29.038597 28805 raft_consensus.cc:3055] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:10:29.038966 28805 raft_consensus.cc:740] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:29.027313 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:29.041685 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:29.021538 29035 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:29.023741 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:29.025177 23028 raft_consensus.cc:1240] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:10:29.016434 23026 raft_consensus.cc:1240] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:29.025666 28945 raft_consensus.cc:3055] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:10:29.025184 22947 raft_consensus.cc:1240] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 17. Current term is 18. Ops: []
I20260709 13:10:29.026232 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:29.025702 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:29.028149 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:29.029506 23024 raft_consensus.cc:1275] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:10:29.029179 28756 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:29.030474 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:28.908553 29023 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:29.032649 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 171590us
W20260709 13:10:29.033972 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 133392us
I20260709 13:10:29.031744 23029 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 16.
I20260709 13:10:29.010751 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:29.049278 22948 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:29.052829 22950 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
I20260709 13:10:29.056727 22994 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:29.056969 22954 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 18.
I20260709 13:10:29.040053 28805 consensus_queue.cc:260] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:29.058099 28805 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:29.058130 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:29.059343 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:29.048307 28685 raft_consensus.cc:697] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 12 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:29.061380 23588 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:29.069144 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:29.069681 22948 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 16.
I20260709 13:10:29.069816 22951 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:29.045082 28945 raft_consensus.cc:740] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:29.071628 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:29.072641 23021 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:29.065888 22919 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:29.085618 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:29.088142 22874 raft_consensus.cc:1275] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 13:10:29.088479 28685 consensus_queue.cc:237] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:29.088831 28945 consensus_queue.cc:260] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:29.089859 22877 raft_consensus.cc:2393] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 16: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:29.090230 28945 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:29.091656 28528 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:29.093065 22880 raft_consensus.cc:1240] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 13 LEADER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 12. Current term is 13. Ops: []
I20260709 13:10:29.095027 22871 raft_consensus.cc:1240] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:29.097218 28756 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:29.098843 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:29.105578 28965 raft_consensus.cc:2749] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:10:29.105166 28827 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:29.110574 23021 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 20.
I20260709 13:10:29.111272 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:29.113655 22873 raft_consensus.cc:1240] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 17. Current term is 18. Ops: []
W20260709 13:10:29.114295 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:29.118040 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:29.125248 28062 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:10:29.134394 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:29.080498 28818 consensus_queue.cc:1059] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
W20260709 13:10:29.140740 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 79545us
W20260709 13:10:29.143823 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 81382.1us
I20260709 13:10:29.144130 29022 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:29.144780 22919 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:29.144734 29022 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:29.146397 28802 raft_consensus.cc:3055] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:10:29.146821 28802 raft_consensus.cc:740] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:29.147850 29022 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:29.147722 28802 consensus_queue.cc:260] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:29.148605 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 92348.3us
I20260709 13:10:29.154026 28062 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:29.165181 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:29.167071 28827 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:29.167253 29035 consensus_queue.cc:1048] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:10:29.171562 28964 raft_consensus.cc:2749] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:29.172840 28786 raft_consensus.cc:2749] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:10:29.175613 22919 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:29.176819 28821 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:10:29.177294 28821 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:29.177788 28062 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:29.180320 28802 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:29.180610 29035 raft_consensus.cc:2804] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:29.181121 29035 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:29.181502 29035 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:29.177649 28821 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:29.182193 22994 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:29.170768 23588 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:29.183863 22919 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:29.192445 22955 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 19.
W20260709 13:10:29.193742 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 110725us
I20260709 13:10:29.199115 28813 raft_consensus.cc:2804] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:10:29.199745 28813 raft_consensus.cc:697] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:29.200764 28813 consensus_queue.cc:237] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:29.204710 29038 log.cc:927] Time spent T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 13:10:29.226001 23617 raft_consensus.cc:1240] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16. Ops: []
I20260709 13:10:29.218243 28964 raft_consensus.cc:2749] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:29.236241 28578 consensus_queue.cc:1048] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.045s
I20260709 13:10:29.242262 22992 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:29.243593 28868 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:29.195314 28945 consensus_queue.cc:1048] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 13:10:29.219656 28991 raft_consensus.cc:2749] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:10:29.245903 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:29.194754 28786 consensus_queue.cc:1059] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.039s
W20260709 13:10:29.248291 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 172407us
W20260709 13:10:29.252991 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 91771us
I20260709 13:10:29.244328 28868 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:29.261277 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 105996us
I20260709 13:10:29.262187 23588 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:29.268401 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:29.270048 28805 raft_consensus.cc:2804] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:10:29.270803 28805 raft_consensus.cc:697] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:29.271943 28805 consensus_queue.cc:237] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:29.273149 28898 raft_consensus.cc:3055] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:10:29.278442 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 43861.5us
I20260709 13:10:29.285470 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:29.286160 23618 raft_consensus.cc:2368] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 19. Current term is 20.
I20260709 13:10:29.290406 23625 raft_consensus.cc:1275] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:10:29.291530 28964 raft_consensus.cc:2804] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:29.292023 28964 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:29.292438 28964 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:29.275174 28802 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:29.297657 28802 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:29.299288 28062 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:29.299808 28062 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:29.300261 28802 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:29.300566 22992 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:29.293354 28578 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:29.301944 28062 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:29.302093 28816 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:29.284384 28898 raft_consensus.cc:740] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:29.320459 22949 raft_consensus.cc:1275] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:10:29.310297 22875 raft_consensus.cc:1240] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 17. Current term is 18. Ops: []
W20260709 13:10:29.325048 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 59353.7us
I20260709 13:10:29.319841 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:29.325030 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:29.325362 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:29.303743 28868 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:29.331142 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:29.332371 22948 raft_consensus.cc:1240] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 17. Current term is 18. Ops: []
I20260709 13:10:29.332999 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:29.333787 23622 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 17.
I20260709 13:10:29.337695 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:29.338539 23625 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Advancing to term 17
I20260709 13:10:29.337695 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:29.340505 22993 leader_election.cc:400] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 19. Current term is 20.
I20260709 13:10:29.340958 22993 leader_election.cc:403] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:10:29.337458 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:29.343667 22952 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15.
I20260709 13:10:29.344779 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:29.349592 23620 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 17.
I20260709 13:10:29.348347 22946 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 19.
I20260709 13:10:29.349932 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:29.334235 28709 consensus_queue.cc:1059] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 13:10:29.319836 28578 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:29.327845 28816 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:29.382920 22949 raft_consensus.cc:2393] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 18: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:29.382561 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:29.347211 28801 consensus_queue.cc:1059] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.038s
I20260709 13:10:29.377878 28991 raft_consensus.cc:2804] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:10:29.374734 28827 raft_consensus.cc:2749] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 19. Current term is 20.
I20260709 13:10:29.343179 28885 consensus_queue.cc:1048] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 13:10:29.388777 28062 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:29.391168 23622 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:29.405489 22954 raft_consensus.cc:2393] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 15: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:29.400480 28816 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:29.410410 23625 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 17.
W20260709 13:10:29.413564 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 118847us
I20260709 13:10:29.415970 28869 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:29.420622 23624 raft_consensus.cc:1275] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:10:29.422887 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:29.424153 23625 raft_consensus.cc:1240] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 17. Current term is 18. Ops: []
I20260709 13:10:29.424494 22993 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:29.425010 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:29.420981 28062 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:29.442903 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:29.445132 23619 raft_consensus.cc:1240] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 11. Current term is 12. Ops: []
I20260709 13:10:29.449747 23620 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 16.
I20260709 13:10:29.430187 28991 raft_consensus.cc:697] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:29.453115 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:29.458827 22948 raft_consensus.cc:2393] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 18: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:29.445824 28827 raft_consensus.cc:2804] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:10:29.476866 28062 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:29.478255 22919 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:29.478421 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:29.479933 22946 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 17.
I20260709 13:10:29.420677 28869 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:29.476263 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:29.482928 23624 raft_consensus.cc:1275] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:10:29.489012 22843 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:29.495972 22951 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 16.
I20260709 13:10:29.451848 28821 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:29.500082 28062 raft_consensus.cc:2804] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:29.500545 28062 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:29.500965 28062 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:29.503190 22919 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f
W20260709 13:10:29.503767 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:29.502259 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:29.479609 28827 raft_consensus.cc:697] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 17 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:29.512558 28902 consensus_queue.cc:1048] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 13:10:29.539423 28369 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:29.541646 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:29.540727 22843 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:29.507664 28816 raft_consensus.cc:2804] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:29.549672 23622 raft_consensus.cc:1275] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:10:29.514396 28801 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:29.514192 28821 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:29.570124 22843 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:29.534184 28869 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:29.535645 28887 raft_consensus.cc:3055] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:10:29.580451 28868 consensus_queue.cc:1059] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 13:10:29.585772 28868 raft_consensus.cc:2804] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:10:29.586388 28868 raft_consensus.cc:697] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 16 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:29.587476 28868 consensus_queue.cc:237] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:29.589429 28827 consensus_queue.cc:237] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:29.590224 29022 raft_consensus.cc:3055] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:10:29.590706 29022 raft_consensus.cc:740] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:29.591774 29022 consensus_queue.cc:260] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:29.593303 29054 log.cc:927] Time spent T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 13:10:29.593680 22843 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:29.594561 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:29.595556 28712 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:10:29.596153 28712 raft_consensus.cc:697] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 19 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:29.597317 23622 raft_consensus.cc:1240] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 19. Current term is 20. Ops: []
I20260709 13:10:29.597249 28712 consensus_queue.cc:237] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:29.600466 29022 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:29.557812 29023 raft_consensus.cc:2804] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:29.605041 29023 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:29.565781 28991 consensus_queue.cc:237] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:29.605506 29023 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:29.542976 28369 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:29.581751 28887 raft_consensus.cc:740] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:29.611317 28887 consensus_queue.cc:260] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:29.612398 28887 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:29.613050 28578 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:29.560986 28801 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:29.613801 28801 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:29.617950 23621 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
I20260709 13:10:29.561901 28816 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:29.632390 28062 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:29.637820 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:29.652415 22955 raft_consensus.cc:1240] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16. Ops: []
I20260709 13:10:29.450743 28528 raft_consensus.cc:2955] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 14 LEADER]: Committing config change with OpId 12.11: config changed from index 6 to 11, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:29.633735 28816 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:29.661233 28062 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:29.663395 23622 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:29.667968 29061 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:29.679957 28685 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:29.682889 22946 raft_consensus.cc:1240] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 13:10:29.683236 28685 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:29.685115 28965 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:29.685695 28965 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:29.687995 28965 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:29.689807 28685 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:29.688604 28965 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:29.691200 28965 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:29.692569 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:29.693804 28965 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:29.695611 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:29.696910 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:29.699389 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:10:29.700109 29052 log.cc:927] Time spent T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.245s	user 0.000s	sys 0.001s
I20260709 13:10:29.700284 22954 raft_consensus.cc:2393] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 16: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:29.702149 29062 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:29.703023 29062 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:29.706449 23619 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 19.
I20260709 13:10:29.719575 22954 raft_consensus.cc:1240] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:10:29.731454 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:29.736878 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 143299us
I20260709 13:10:29.743948 23622 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15.
I20260709 13:10:29.745941 22955 raft_consensus.cc:2393] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 15: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:29.750396 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:29.763304 22955 raft_consensus.cc:1275] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:10:29.682823 29061 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:29.769505 28369 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:29.772547 28801 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:29.774469 29062 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:29.774915 29064 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:29.775445 29064 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:29.776808 28485 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:29.777539 29064 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:29.777355 28485 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:29.779043 28801 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:29.779465 28485 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:29.780607 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:29.781363 23029 raft_consensus.cc:2393] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 18: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:29.781482 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:29.782636 23021 raft_consensus.cc:1240] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:10:29.782977 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:29.783317 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:29.784400 29061 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:10:29.784570 29068 log.cc:927] Time spent T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 13:10:29.784915 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:29.785000 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:29.785131 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:29.785768 23616 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 19.
W20260709 13:10:29.786142 29035 consensus_meta.cc:287] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.604s	user 0.007s	sys 0.005s
I20260709 13:10:29.786509 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:29.786697 22952 raft_consensus.cc:1275] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:10:29.787447 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:29.789179 22949 raft_consensus.cc:1275] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:10:29.790475 22953 raft_consensus.cc:1275] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:10:29.791806 22947 raft_consensus.cc:1275] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:10:29.786908 29035 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:29.794597 23022 raft_consensus.cc:2410] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:10:29.798012 28902 consensus_queue.cc:1048] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:10:29.798599 22843 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:29.799466 28902 consensus_queue.cc:1059] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:10:29.799747 22843 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:29.800000 22918 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:29.800841 28869 raft_consensus.cc:3055] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:10:29.801229 28869 raft_consensus.cc:740] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:29.801406 22918 leader_election.cc:400] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 18. Current term is 19.
I20260709 13:10:29.802119 28869 consensus_queue.cc:260] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:29.802678 22918 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:29.803464 28779 raft_consensus.cc:2749] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:10:29.803547 28869 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:29.804550 28779 raft_consensus.cc:2804] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:29.804965 28779 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:29.805418 28779 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:29.766613 28709 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
W20260709 13:10:29.773139 29067 log.cc:927] Time spent T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 13:10:29.813100 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:29.815150 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:29.816627 22949 raft_consensus.cc:1275] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:10:29.816753 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:29.817768 22950 raft_consensus.cc:1240] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16. Ops: []
I20260709 13:10:29.818248 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:29.817792 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:29.798939 28964 consensus_meta.cc:287] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.506s	user 0.000s	sys 0.009s
I20260709 13:10:29.800545 28902 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:29.821473 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:29.822086 22954 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 13.
I20260709 13:10:29.808951 28868 raft_consensus.cc:2804] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:10:29.824378 28062 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:29.815820 29062 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:29.832835 22955 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 16.
I20260709 13:10:29.825847 28868 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:29.833205 29062 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:29.833658 28868 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:29.833688 29035 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:29.835376 29062 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:29.821067 28991 consensus_queue.cc:1059] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.004s
I20260709 13:10:29.826247 22918 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:29.808740 28709 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:29.837594 28709 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:29.826491 22992 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:29.840472 28062 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:29.842931 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:29.843770 22918 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:29.844867 28756 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:29.844556 28898 consensus_queue.cc:260] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:29.845510 28756 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:29.846014 28898 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:29.846709 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:29.847422 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:29.848034 23028 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:29.848187 28756 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:29.853750 28991 raft_consensus.cc:3055] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:10:29.854154 28991 raft_consensus.cc:740] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:29.855064 28991 consensus_queue.cc:260] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:29.856218 28991 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:29.857143 28485 raft_consensus.cc:2749] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:10:29.821936 28964 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:29.860390 29071 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:29.860592 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:29.860945 29071 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:29.861898 28964 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:29.862675 23618 raft_consensus.cc:1240] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 17. Current term is 18. Ops: [17.14-17.14]
W20260709 13:10:29.862027 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:29.822178 28902 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:29.864948 23028 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15.
I20260709 13:10:29.865074 23024 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 16.
I20260709 13:10:29.864010 22918 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:29.864807 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:29.868229 28062 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:29.869336 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 100641us
I20260709 13:10:29.869573 23023 raft_consensus.cc:1240] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 14. Current term is 15. Ops: []
I20260709 13:10:29.871681 22918 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:29.872337 28821 consensus_queue.cc:1048] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:10:29.867883 28902 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:29.873406 28813 raft_consensus.cc:2749] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:29.874190 22950 raft_consensus.cc:1240] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 17. Current term is 18. Ops: []
I20260709 13:10:29.869506 28786 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:29.875438 28786 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:29.871872 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:29.876931 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:29.877058 22949 raft_consensus.cc:2376] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 19. Re-sending same reply.
I20260709 13:10:29.878875 28786 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:29.880815 28802 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:29.881776 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:29.872493 23588 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:29.882375 23024 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:29.863050 29035 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:10:29.857128 28756 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:29.889497 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:29.841403 28887 consensus_queue.cc:1048] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 13:10:29.890787 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:29.897432 22947 raft_consensus.cc:1240] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 17. Current term is 18. Ops: []
W20260709 13:10:29.891603 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 51904.5us
I20260709 13:10:29.902540 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:10:29.902732 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 76292.4us
I20260709 13:10:29.904073 28062 consensus_queue.cc:1059] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:10:29.905740 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:29.874228 28828 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:29.906756 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:29.900005 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:29.909898 23617 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:29.925521 23029 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:29.921063 22992 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:29.929179 23021 raft_consensus.cc:1275] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 13:10:29.882990 28802 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:29.900817 28964 raft_consensus.cc:2749] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:29.926218 28062 raft_consensus.cc:3055] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:10:29.939479 28062 raft_consensus.cc:740] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:29.937983 23619 raft_consensus.cc:1275] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:10:29.892565 28786 consensus_queue.cc:1048] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 13:10:29.898092 29023 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:29.955684 23024 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 18.
W20260709 13:10:29.957793 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 86190us
I20260709 13:10:29.948161 28062 consensus_queue.cc:260] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:29.938203 29062 raft_consensus.cc:2804] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:29.960239 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:29.962052 29035 raft_consensus.cc:697] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:29.956471 23623 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 16.
W20260709 13:10:29.963761 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 93653.9us
I20260709 13:10:29.916862 28965 consensus_queue.cc:1048] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:10:29.945302 28945 raft_consensus.cc:2804] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:29.964547 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:29.961557 22918 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:29.932101 28805 consensus_queue.cc:1048] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 13:10:29.931126 28827 consensus_queue.cc:1048] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.029s
I20260709 13:10:29.970067 28062 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:29.970379 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:29.973312 22955 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 17.
W20260709 13:10:29.970904 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:29.919297 28828 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:29.930434 28756 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:29.969389 23022 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
I20260709 13:10:29.985596 22843 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:29.947321 28685 consensus_queue.cc:1059] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 13:10:29.992730 29077 raft_consensus.cc:2749] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:10:29.996773 29075 consensus_queue.cc:1048] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 13:10:29.961782 29062 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:30.002604 23021 raft_consensus.cc:1240] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 17. Current term is 18. Ops: []
I20260709 13:10:29.968899 28945 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
W20260709 13:10:30.026101 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 56119.6us
I20260709 13:10:30.028659 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:30.025655 29062 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:10:30.048192 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 93420.7us
I20260709 13:10:30.028975 28945 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:30.054849 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:29.991766 28779 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:30.000749 29076 consensus_queue.cc:1048] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.126s
I20260709 13:10:30.031731 29082 raft_consensus.cc:2804] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:10:29.963184 29035 consensus_queue.cc:237] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:30.077349 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:30.081774 28802 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:30.071421 28062 consensus_queue.cc:1059] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.014s
W20260709 13:10:30.087916 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 126411us
I20260709 13:10:30.089473 22954 raft_consensus.cc:1240] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 15. Current term is 16. Ops: []
I20260709 13:10:30.092249 28828 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:30.093031 22955 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:39742
I20260709 13:10:30.093813 22955 raft_consensus.cc:606] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 LEADER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:30.095686 28869 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:30.096189 28869 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:30.098181 28869 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:30.098676 28869 raft_consensus.cc:3055] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:10:30.098575 28902 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:30.099057 28869 raft_consensus.cc:740] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:30.099850 28869 consensus_queue.cc:260] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:30.100896 28902 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:30.072885 29087 consensus_queue.cc:1048] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 13:10:30.105611 28756 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:30.106590 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:30.107391 23021 raft_consensus.cc:2393] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 19: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:30.108700 29023 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:30.120690 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:30.121228 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:30.121678 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:30.122367 28779 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:30.070276 28816 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:30.125654 29023 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:30.125993 29091 raft_consensus.cc:2749] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:30.126332 29023 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:30.130815 29083 log.cc:927] Time spent T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 13:10:30.056804 28786 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:30.131800 28786 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:30.135066 22947 raft_consensus.cc:1240] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 17. Current term is 18. Ops: []
I20260709 13:10:30.054340 29071 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:30.134817 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 166209us
W20260709 13:10:30.136201 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:30.141004 23617 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18.
I20260709 13:10:30.150015 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:30.246129 23620 raft_consensus.cc:1240] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:10:30.248019 23588 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:30.268796 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:30.125677 28816 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:30.272415 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 183576us
I20260709 13:10:30.281288 22946 raft_consensus.cc:1275] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
W20260709 13:10:30.282126 28712 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.009s	sys 0.027s
W20260709 13:10:30.282505 28712 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.009s	sys 0.027s
I20260709 13:10:30.283022 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:30.285087 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:30.285629 22946 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
I20260709 13:10:30.285791 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:30.286433 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:30.269833 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 242983us
I20260709 13:10:30.313529 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:30.314891 23620 raft_consensus.cc:2393] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 18: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:30.313704 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:30.244342 28786 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:30.325879 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:30.326530 23625 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
I20260709 13:10:30.327287 22876 raft_consensus.cc:1240] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16. Ops: []
W20260709 13:10:30.328953 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 79145.5us
I20260709 13:10:30.316927 22843 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:30.325428 29082 raft_consensus.cc:697] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 16 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:30.337792 29082 consensus_queue.cc:237] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:30.339637 29075 consensus_queue.cc:1048] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:10:30.345060 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:30.345907 28827 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:30.346443 28827 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:30.349068 28827 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:10:30.377600 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 240976us
I20260709 13:10:30.379280 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:30.380865 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:30.381114 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:30.381960 29085 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:30.384280 22948 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:39742
I20260709 13:10:30.314283 29023 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:30.387790 22948 raft_consensus.cc:606] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:30.388384 22948 raft_consensus.cc:614] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:10:30.389164 23588 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:30.390476 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:30.390555 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:30.382478 29085 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:30.392872 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 90226.4us
I20260709 13:10:30.360013 28802 raft_consensus.cc:2749] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:30.417862 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:30.357532 28868 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:30.418562 22871 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 12.
I20260709 13:10:30.392921 28816 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:30.400694 28818 raft_consensus.cc:2749] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:30.399531 28813 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:30.437213 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:30.401289 29023 raft_consensus.cc:2804] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:10:30.413923 28898 consensus_meta.cc:287] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.567s	user 0.012s	sys 0.001s
I20260709 13:10:30.429222 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:10:30.430437 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 51865.1us
I20260709 13:10:30.424268 22876 raft_consensus.cc:1275] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:10:30.439812 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:30.437845 22877 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 16.
I20260709 13:10:30.441680 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:30.448812 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:30.449388 22877 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18.
I20260709 13:10:30.446750 23588 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:30.441915 22843 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:30.451550 29088 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:30.446803 22879 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 16.
I20260709 13:10:30.458005 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:30.437783 28816 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:30.462764 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:30.444798 22880 raft_consensus.cc:1240] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:10:30.451488 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:30.464032 28816 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:30.420820 28779 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:30.466270 22953 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 18.
I20260709 13:10:30.466365 28779 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:30.464233 22877 raft_consensus.cc:2393] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 16: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:30.444252 23029 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 16.
I20260709 13:10:30.437037 29094 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:30.477041 28712 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:30.481396 23588 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:30.473745 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:30.473001 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:30.472259 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:30.485525 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:30.490037 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:30.491972 23620 raft_consensus.cc:1275] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:10:30.490943 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:30.493484 22874 raft_consensus.cc:1240] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 18. Current term is 19. Ops: []
I20260709 13:10:30.493686 22880 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18.
I20260709 13:10:30.491020 22876 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 15.
I20260709 13:10:30.452224 29088 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:30.467376 29023 raft_consensus.cc:697] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 19 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:30.517800 23588 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:30.482744 28712 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:30.521370 22873 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 18.
W20260709 13:10:30.523053 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:30.525476 28779 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:30.445079 28813 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:30.492657 28818 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:30.526602 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:30.539196 22876 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:10:30.516512 29089 log.cc:927] Time spent T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
I20260709 13:10:30.527416 23588 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:10:30.534747 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:30.533520 22946 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 15.
I20260709 13:10:30.515340 28369 raft_consensus.cc:2764] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:10:30.530330 28712 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:30.471135 29095 consensus_queue.cc:1059] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.081s
I20260709 13:10:30.550911 22880 raft_consensus.cc:1275] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:10:30.513255 29071 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:30.526602 28902 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:30.484879 29094 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:30.553966 29098 log.cc:927] Time spent T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
W20260709 13:10:30.562538 29062 consensus_meta.cc:287] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.501s	user 0.010s	sys 0.000s
W20260709 13:10:30.589169 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 61865.3us
I20260709 13:10:30.589349 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:30.590705 29105 log.cc:927] Time spent T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 13:10:30.563995 28902 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:30.598639 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:30.606411 28902 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:30.606819 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:30.596767 29023 consensus_queue.cc:237] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:30.608896 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:30.573199 28818 raft_consensus.cc:2749] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:10:30.590003 23588 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:30.610520 22843 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:30.620289 22948 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 16.
I20260709 13:10:30.569099 28868 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:30.614588 22876 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 17.
I20260709 13:10:30.590068 29062 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:30.637990 22843 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:30.609841 28813 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:30.642427 29062 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:30.645160 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:30.645946 22873 raft_consensus.cc:2393] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 18: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:30.571899 29071 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:30.658131 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:30.658493 29094 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:30.659070 22950 raft_consensus.cc:2410] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 13:10:30.665063 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:30.665704 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:30.666042 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:30.667271 22877 raft_consensus.cc:1275] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 13:10:30.667694 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:30.668308 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:30.668998 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:30.669018 22843 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:30.669263 22874 raft_consensus.cc:1275] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:10:30.669974 28868 raft_consensus.cc:2804] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:10:30.669997 29088 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:30.671386 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:30.671574 29071 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:30.640738 29085 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:30.675403 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:30.676858 29094 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:30.677477 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:30.678174 22877 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
I20260709 13:10:30.697610 29101 raft_consensus.cc:3055] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:10:30.698203 29101 raft_consensus.cc:740] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:30.699307 29101 consensus_queue.cc:260] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:30.701471 29101 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:10:30.703766 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 274699us
I20260709 13:10:30.705538 22992 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:30.705878 22950 raft_consensus.cc:1240] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16. Ops: []
I20260709 13:10:30.707901 28827 raft_consensus.cc:2804] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 13:10:30.708624 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 223337us
I20260709 13:10:30.712390 29076 consensus_queue.cc:1048] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:10:30.716694 23617 raft_consensus.cc:1275] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:10:30.717624 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:30.718611 23029 raft_consensus.cc:2393] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 18: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:30.718712 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:30.719496 23025 raft_consensus.cc:2410] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 22 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 13:10:30.719769 23021 raft_consensus.cc:1275] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:10:30.721007 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:30.708518 28827 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:30.723661 28827 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:30.726948 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:30.730136 28965 consensus_queue.cc:1048] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:10:30.733289 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:30.733456 23022 raft_consensus.cc:1240] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:10:30.733531 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:30.733807 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:30.733973 23025 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18.
I20260709 13:10:30.734366 23029 raft_consensus.cc:2393] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 16: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:30.735497 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:30.736307 23029 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 19.
I20260709 13:10:30.738421 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:30.738929 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:30.739514 23022 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18.
I20260709 13:10:30.654803 29022 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:30.745343 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:30.749243 22918 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:30.747592 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 78642.4us
W20260709 13:10:30.751936 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 225220us
I20260709 13:10:30.748490 23026 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 14.
I20260709 13:10:30.755347 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:30.753600 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:30.758854 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:30.759649 28062 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:30.760157 28062 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:30.761116 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:30.759759 22879 raft_consensus.cc:2393] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 15: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:30.741719 28964 raft_consensus.cc:2749] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:30.768648 22918 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:30.779702 28062 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:30.693307 29095 raft_consensus.cc:2804] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader election won for term 15
W20260709 13:10:30.771849 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:30.780758 22918 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:30.750988 28868 raft_consensus.cc:697] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 16 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:30.693166 28712 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:30.790086 22918 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:10:30.791965 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 171875us
I20260709 13:10:30.741288 29094 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:30.796918 22918 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:30.752979 29116 raft_consensus.cc:2749] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 13:10:30.840376 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 79424.3us
I20260709 13:10:30.778966 29085 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:10:30.805828 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:30.841542 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 135183us
W20260709 13:10:30.787163 28945 consensus_meta.cc:287] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.721s	user 0.009s	sys 0.002s
I20260709 13:10:30.782208 29086 raft_consensus.cc:2749] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:10:30.754245 29022 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:30.779021 29077 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:30.847216 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 50412.7us
I20260709 13:10:30.848825 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:30.849469 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:30.850800 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:30.791090 29095 raft_consensus.cc:697] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 15 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:30.784283 29088 consensus_queue.cc:1048] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.123s
I20260709 13:10:30.913760 22874 raft_consensus.cc:1275] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:10:30.854135 22876 raft_consensus.cc:1240] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 17. Current term is 18. Ops: []
W20260709 13:10:30.912393 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 119371us
I20260709 13:10:30.907372 23029 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 18.
I20260709 13:10:30.786458 28965 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:30.924021 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 75606.9us
I20260709 13:10:30.804239 28991 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:30.852600 29097 raft_consensus.cc:2804] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:30.840093 28578 raft_consensus.cc:2804] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:10:30.804013 28813 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:30.934289 29134 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:30.939196 29071 consensus_queue.cc:1059] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:10:30.943063 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 97296.2us
I20260709 13:10:30.948832 22875 raft_consensus.cc:1240] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 16 LEADER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16. Ops: []
W20260709 13:10:31.044570 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 102755us
W20260709 13:10:31.048159 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:30.949671 22992 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:30.957242 22871 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:30.854247 28945 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:30.959586 29077 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:31.063102 29112 log.cc:927] Time spent T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.372s	user 0.000s	sys 0.001s
I20260709 13:10:31.064574 29122 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:31.064985 22920 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:31.066339 28712 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:31.067732 29094 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:30.958817 22844 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.63.4:44043 timed out after 2.225s (SENT) [suppressed 23 similar messages]
I20260709 13:10:31.068349 29127 consensus_queue.cc:1048] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.005s
I20260709 13:10:31.077165 22871 raft_consensus.cc:1275] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 13:10:31.077065 28868 consensus_queue.cc:237] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:31.078123 28827 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:31.078601 29022 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:30.955857 28965 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:31.021375 28898 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:30.958599 28578 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:30.958400 29097 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:31.021557 28813 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:31.058233 29134 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:31.082192 28945 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:30.959698 29082 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:30.853168 29103 consensus_queue.cc:1059] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.224s
I20260709 13:10:31.023038 29023 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
W20260709 13:10:31.134052 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 63922.7us
I20260709 13:10:31.083734 29095 consensus_queue.cc:237] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:31.085770 29117 raft_consensus.cc:3055] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:10:31.085963 29035 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:31.086405 29062 consensus_queue.cc:1048] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:10:31.086611 22919 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:31.057785 28991 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:31.087704 29088 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:31.158530 29088 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:31.160917 29136 consensus_queue.cc:1059] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:10:31.087718 22951 raft_consensus.cc:1240] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:31.161867 29141 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:31.089423 29077 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:31.162416 29141 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:31.162878 29077 consensus_queue.cc:1059] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:10:31.092269 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 149580us
W20260709 13:10:31.094048 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 144452us
I20260709 13:10:31.165027 22992 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:31.094636 29086 raft_consensus.cc:2804] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:31.166008 29086 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:31.135066 22994 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:31.166131 29088 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:31.166224 28991 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:31.121587 28578 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:31.121165 28813 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:31.121357 29097 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:10:31.151916 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 65393.4us
I20260709 13:10:31.175989 22954 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 2511) took 1209 ms. Trace:
I20260709 13:10:31.176378 22954 rpcz_store.cc:276] 0709 13:10:29.966940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:29.967652 (+   712us) service_pool.cc:224] Handling call
0709 13:10:29.971859 (+  4207us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:30.101025 (+129166us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:31.175702 (+1074677us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b54000a96d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:31.175890 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":265203968}
I20260709 13:10:31.177662 29077 raft_consensus.cc:3055] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:10:31.171885 29035 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:31.170679 29086 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:31.179973 29139 raft_consensus.cc:2804] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:10:31.180150 29035 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:31.180492 29139 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:31.180953 29139 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:31.181550 23623 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60044 (request call id 1089) took 1291 ms. Trace:
I20260709 13:10:31.181777 28965 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:31.181957 23623 rpcz_store.cc:276] 0709 13:10:29.890415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:29.890556 (+   141us) service_pool.cc:224] Handling call
0709 13:10:31.181364 (+1290808us) spinlock_profiling.cc:243] Waited 59 ms on lock 0x7b5400469e54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:31.181467 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":135737600}
I20260709 13:10:31.182317 28965 raft_consensus.cc:2764] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:10:31.095141 29122 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:31.170776 29088 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:10:31.178203 29077 raft_consensus.cc:740] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:31.183487 28965 raft_consensus.cc:2804] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:10:31.185660 22876 raft_consensus.cc:1240] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 19. Current term is 20. Ops: []
I20260709 13:10:31.121448 28898 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:31.199257 29085 raft_consensus.cc:2749] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:31.201017 28898 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:31.201305 28827 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:31.164517 29141 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:31.195068 29134 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:31.197283 29122 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:31.154860 29117 raft_consensus.cc:740] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
W20260709 13:10:31.205834 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 451029us
W20260709 13:10:31.220232 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 261452us
W20260709 13:10:31.240981 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 315255us
I20260709 13:10:31.138664 29023 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:31.134589 29082 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:31.242611 22843 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:10:31.251358 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 67002.5us
I20260709 13:10:31.262104 28821 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:31.262737 28821 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:31.259471 29088 raft_consensus.cc:697] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:31.265058 29088 consensus_queue.cc:237] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:31.208880 28965 raft_consensus.cc:697] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 18 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:10:31.273422 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 52313.3us
I20260709 13:10:31.274205 22843 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:31.275559 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:31.242983 29122 raft_consensus.cc:3055] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:10:31.276223 22874 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18.
I20260709 13:10:31.278445 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:31.304613 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 169626us
I20260709 13:10:31.277149 22843 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:31.305651 22876 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 16.
I20260709 13:10:31.306370 22954 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:39742
I20260709 13:10:31.239620 29101 consensus_queue.cc:1048] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
W20260709 13:10:31.312206 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 147241us
I20260709 13:10:31.306175 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:31.313103 22994 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:31.308405 22920 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:31.317704 23588 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:31.305436 29094 raft_consensus.cc:2804] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:10:31.278797 29122 raft_consensus.cc:740] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:31.323410 22843 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:31.323585 22920 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:31.323890 22994 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:10:31.324625 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 276496us
I20260709 13:10:31.274034 28813 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:31.312704 29022 raft_consensus.cc:2804] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 13:10:31.334074 28802 raft_consensus.cc:2749] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:31.334517 29151 consensus_queue.cc:1059] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:10:31.334944 29122 consensus_queue.cc:260] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:31.338723 29154 raft_consensus.cc:2749] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:31.348214 22994 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:31.358448 29154 raft_consensus.cc:2749] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:31.359350 29097 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:31.345963 22992 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:31.376969 29157 raft_consensus.cc:2804] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:31.377401 29157 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:31.313267 29147 consensus_queue.cc:1059] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.074s
I20260709 13:10:31.377776 29157 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:31.381093 28813 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:31.382284 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:31.307272 29145 raft_consensus.cc:2749] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:31.356719 29160 raft_consensus.cc:2749] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:10:31.383531 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:31.348970 29022 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:31.384900 29022 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:31.356290 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:31.327384 28887 raft_consensus.cc:2749] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:31.360657 29154 consensus_queue.cc:1048] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:10:31.361215 29135 raft_consensus.cc:2749] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:31.362377 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:31.365406 22994 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:31.366247 29064 raft_consensus.cc:3055] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:10:31.368992 28991 raft_consensus.cc:3055] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:10:31.350835 28869 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:31.355078 29122 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:31.350455 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:31.373108 28578 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:31.428323 29146 log.cc:927] Time spent T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 13:10:31.375368 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:31.357679 22920 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:31.386785 23619 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 17.
I20260709 13:10:31.360818 28821 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:31.433388 22880 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 15.
I20260709 13:10:31.344388 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:31.389216 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 58968.5us
W20260709 13:10:31.400044 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 121514us
W20260709 13:10:31.436082 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 78473.6us
I20260709 13:10:31.317313 29103 consensus_queue.cc:1048] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.071s
I20260709 13:10:31.398032 22879 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 17.
I20260709 13:10:31.413132 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:31.369078 29159 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:31.311472 29117 consensus_queue.cc:260] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:31.414958 29092 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:31.386971 28965 consensus_queue.cc:237] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:31.310961 29077 consensus_queue.cc:260] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:31.416882 29131 log.cc:927] Time spent T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.148s	user 0.001s	sys 0.000s
I20260709 13:10:31.417958 22874 raft_consensus.cc:1275] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:10:31.394066 29094 raft_consensus.cc:697] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 18 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:31.420202 29157 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:31.421288 22947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 2512) took 1453 ms. Trace:
I20260709 13:10:31.399951 29064 raft_consensus.cc:740] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:31.426294 29119 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:31.374584 29023 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:31.428731 22954 raft_consensus.cc:606] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:31.509265 22954 raft_consensus.cc:614] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:10:31.429324 29097 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:31.429623 22875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 3074) took 1045 ms. Trace:
I20260709 13:10:31.387056 22877 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 18.
I20260709 13:10:31.451051 22876 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 15.
I20260709 13:10:31.451028 22920 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:31.428449 28578 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:31.511830 28578 raft_consensus.cc:2749] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 13:10:31.511922 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 60997.7us
I20260709 13:10:31.513450 22920 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:31.471614 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:31.476193 22843 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:31.519968 29101 raft_consensus.cc:2749] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:10:31.520123 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 44018.2us
I20260709 13:10:31.478616 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:31.529354 23618 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 17.
I20260709 13:10:31.413157 28991 raft_consensus.cc:740] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:31.504587 22947 rpcz_store.cc:276] 0709 13:10:29.967420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:29.967756 (+   336us) service_pool.cc:224] Handling call
0709 13:10:29.971433 (+  3677us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:30.174161 (+202728us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:31.420998 (+1246837us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b54000a91d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:31.421222 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":375758720}
I20260709 13:10:31.455159 28801 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:31.534057 28801 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:31.536772 28801 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:31.539415 29093 log.cc:927] Time spent T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 1.014s	user 0.010s	sys 0.015s
I20260709 13:10:31.541054 23622 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 1211) took 1607 ms. Trace:
I20260709 13:10:31.541504 23622 rpcz_store.cc:276] 0709 13:10:29.933992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:29.934128 (+   136us) service_pool.cc:224] Handling call
0709 13:10:29.934456 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:29.944814 (+ 10358us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:29.944853 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:29.949404 (+  4551us) log.cc:844] Serialized 26 byte log entry
0709 13:10:30.358992 (+409588us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:10:30.359035 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:30.360159 (+  1124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:31.538843 (+1178684us) spinlock_profiling.cc:243] Waited 8.45 ms on lock 0x7b24003465f8. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132b7e9c4 00007f6132b7cacb 00007f6132d37dc9 00007f6132d37a55 00007f6132d3709d 00007f613596910e 00007f613598088c 00007f613597fbb6 00007f613597f7d8 00007f6135989221
0709 13:10:31.540636 (+  1793us) raft_consensus.cc:1685] finished
0709 13:10:31.540680 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:31.540960 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10534,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":4104,"spinlock_wait_cycles":19425536,"thread_start_us":386537,"threads_started":1,"wal-append.queue_time_us":523907}
I20260709 13:10:31.510008 22875 rpcz_store.cc:276] 0709 13:10:30.384027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:30.388524 (+  4497us) service_pool.cc:224] Handling call
0709 13:10:30.400097 (+ 11573us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:31.058672 (+658575us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:31.429380 (+370708us) spinlock_profiling.cc:243] Waited 548 ms on lock 0x7b5400368b54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:31.429551 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1260764416}
I20260709 13:10:31.548111 22920 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:31.551029 22877 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
mode: GRACEFUL
 from {username='slave'} at 127.0.0.1:43122
I20260709 13:10:31.469727 29150 raft_consensus.cc:2749] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 13:10:31.552098 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 85094.4us
I20260709 13:10:31.553062 23588 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:31.554548 23590 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:31.505127 29135 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:31.562757 23622 raft_consensus.cc:1275] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:10:31.508720 29088 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:31.565805 29088 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:31.568063 29088 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:31.569828 22919 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:31.536543 28805 raft_consensus.cc:2749] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:31.574479 23588 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:31.576885 22993 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:31.559191 29156 raft_consensus.cc:2804] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:31.570772 28802 raft_consensus.cc:2804] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:31.581662 29141 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:31.583828 29160 raft_consensus.cc:2804] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:31.584363 28813 raft_consensus.cc:2804] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:31.571487 29082 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:31.585811 29064 consensus_queue.cc:260] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:31.586740 29094 consensus_queue.cc:237] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:31.588485 29157 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:31.590026 22872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35360 (request call id 2535) took 1110 ms. Trace:
I20260709 13:10:31.580456 22843 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:31.483623 29092 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:31.571220 28964 raft_consensus.cc:2749] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:31.583777 29117 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:31.596632 23590 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
W20260709 13:10:31.647202 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 66332.9us
I20260709 13:10:31.596691 23588 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:31.603238 22920 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:31.595852 29160 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:31.599705 28818 raft_consensus.cc:3055] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:10:31.561131 29119 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:31.596172 28802 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:31.593540 29156 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:31.599924 28813 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:31.609258 22872 rpcz_store.cc:276] 0709 13:10:30.479699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:30.485348 (+  5649us) service_pool.cc:224] Handling call
0709 13:10:31.589820 (+1104472us) spinlock_profiling.cc:243] Waited 274 ms on lock 0x7b5400369554. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:31.589956 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":630989696}
W20260709 13:10:31.640038 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 59679.3us
I20260709 13:10:31.578867 29086 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:31.633710 28805 raft_consensus.cc:2804] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:10:31.649466 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 46314.6us
I20260709 13:10:31.676282 22920 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:10:31.656263 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 264723us
I20260709 13:10:31.572058 29135 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:31.666782 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:31.680410 22920 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:31.682194 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:31.682384 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:31.658052 28818 raft_consensus.cc:740] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:31.666700 28802 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:31.645169 28868 raft_consensus.cc:2804] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:31.662118 28801 raft_consensus.cc:2804] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:31.658344 29160 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:31.684545 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:31.647315 29139 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:31.685081 29085 raft_consensus.cc:2804] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:31.702943 22872 raft_consensus.cc:2393] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 18: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
W20260709 13:10:31.686908 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 90472.4us
I20260709 13:10:31.679579 29088 raft_consensus.cc:2804] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:31.681169 29064 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:31.666859 29142 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:31.688728 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:31.673707 28813 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:31.689886 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:10:31.673153 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 97494.1us
I20260709 13:10:31.682957 22949 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 18.
I20260709 13:10:31.672047 29156 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:31.662444 29148 raft_consensus.cc:2804] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:31.675828 28805 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:31.696235 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:31.731211 22876 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:31.685350 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:31.703827 23588 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:31.672076 29134 raft_consensus.cc:2749] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:31.690456 28868 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:31.665630 29150 consensus_queue.cc:1048] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 13:10:31.696163 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:10:31.706215 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:31.687633 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 326167us
I20260709 13:10:31.716310 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:31.693619 28801 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:31.716559 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:31.717451 22947 raft_consensus.cc:2393] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 17: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:31.732398 22873 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:31.703274 29085 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:10:31.763013 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 59307.2us
I20260709 13:10:31.705606 29088 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:31.720337 29148 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:31.700423 29101 consensus_queue.cc:1059] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 13:10:31.700084 28712 consensus_queue.cc:1059] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 13:10:31.732929 28805 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:31.772107 28868 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:10:31.789606 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 75704.5us
I20260709 13:10:31.701581 29023 consensus_queue.cc:1059] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.071s
I20260709 13:10:31.770390 22994 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:31.771749 29141 raft_consensus.cc:2749] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:31.776340 28801 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:31.776453 29075 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:31.782624 29085 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:31.799098 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:31.789517 29088 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:31.776770 28945 consensus_queue.cc:1048] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.027s
I20260709 13:10:31.793079 29148 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:31.798928 29035 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:31.796698 29177 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:31.789142 28964 consensus_queue.cc:1059] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.019s
W20260709 13:10:31.813704 29165 log.cc:927] Time spent T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.176s	user 0.001s	sys 0.000s
I20260709 13:10:31.733246 29142 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:31.826946 22873 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 19.
I20260709 13:10:31.831032 23624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60044 (request call id 1093) took 1779 ms. Trace:
W20260709 13:10:31.825703 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 47251.1us
I20260709 13:10:31.841419 22876 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 19.
W20260709 13:10:31.842666 29172 log.cc:927] Time spent T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 13:10:31.843595 22992 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:31.845074 22878 raft_consensus.cc:2410] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:10:31.845412 28964 consensus_queue.cc:1243] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer f3c799b1249e40119d2a5864231c6c75 log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:10:31.834115 29035 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:31.859620 29035 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:31.861330 29166 log.cc:927] Time spent T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.168s	user 0.000s	sys 0.000s
I20260709 13:10:31.863366 29142 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:31.827042 28898 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:31.847275 23624 rpcz_store.cc:276] 0709 13:10:30.051963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:30.052095 (+   132us) service_pool.cc:224] Handling call
0709 13:10:30.077414 (+ 25319us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:30.086553 (+  9139us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:30.086603 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:30.116942 (+ 30339us) log.cc:844] Serialized 26 byte log entry
0709 13:10:30.388735 (+271793us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:10:31.696609 (+1307874us) spinlock_profiling.cc:243] Waited 2.89 ms on lock 0x7b18001d5ba0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f613598d4bc 00007f6132cdb48b 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf5455 00007f61397dd3c7 00007f6135def9a7 00007f6135dff0bf
0709 13:10:31.696764 (+   155us) log.cc:844] Serialized 14 byte log entry
0709 13:10:31.827607 (+130843us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:31.827716 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:31.827775 (+    59us) raft_consensus.cc:1685] finished
0709 13:10:31.830788 (+  3013us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:31.830960 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":84028,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":32,"raft.queue_time_us":28956,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":1891,"spinlock_wait_cycles":6646016,"thread_start_us":466546,"threads_started":4,"wal-append.queue_time_us":361896,"wal-append.run_cpu_time_us":2686,"wal-append.run_wall_time_us":1172015}
I20260709 13:10:31.853776 23588 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:31.827337 29157 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:31.828743 28369 raft_consensus.cc:2749] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:10:31.869153 28062 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:31.885366 28062 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:31.870770 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:31.831056 29086 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:31.834245 29119 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:10:31.880452 29168 log.cc:927] Time spent T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.217s	user 0.000s	sys 0.001s
I20260709 13:10:31.903090 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:31.905893 23024 raft_consensus.cc:2393] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 13: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:31.810772 29075 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:31.913017 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:31.878598 29092 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:31.912091 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:31.882794 29179 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:31.919872 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:31.921955 22946 raft_consensus.cc:2393] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 19: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:31.922559 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:31.923419 23022 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 17.
I20260709 13:10:31.928169 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:31.909361 29076 raft_consensus.cc:2749] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:31.931396 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:31.931977 22946 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 19.
I20260709 13:10:31.885502 28898 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:31.933400 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:31.944222 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 67239.4us
I20260709 13:10:31.853677 29177 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:31.895161 29157 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:31.946851 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:31.953148 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:31.953275 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:31.953943 29175 log.cc:927] Time spent T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.143s	user 0.000s	sys 0.000s
I20260709 13:10:31.954224 28805 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:31.954725 29135 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:31.962356 23623 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 17.
I20260709 13:10:31.957298 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:31.940806 29185 raft_consensus.cc:2749] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:10:31.926193 28964 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:31.971946 22946 raft_consensus.cc:2393] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 17: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:31.972530 23620 raft_consensus.cc:1275] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:10:31.972262 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:31.972729 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:10:31.973805 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 203501us
I20260709 13:10:31.977468 22994 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:10:31.946208 29122 consensus_meta.cc:287] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.521s	user 0.011s	sys 0.000s
I20260709 13:10:31.934486 29186 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:31.977519 23588 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:31.980250 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:10:31.959543 29022 consensus_meta.cc:287] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.574s	user 0.006s	sys 0.005s
I20260709 13:10:32.022397 22994 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:31.946581 29134 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:32.014516 23623 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 19.
I20260709 13:10:31.946146 29035 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:32.025105 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:32.026351 23029 raft_consensus.cc:2368] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 18. Current term is 19.
I20260709 13:10:31.964428 29183 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:32.021427 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:31.925818 29179 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:32.049333 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 71885.8us
I20260709 13:10:32.050103 23588 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:31.945724 29170 raft_consensus.cc:2804] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:32.030890 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:32.029563 23621 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 16.
I20260709 13:10:31.975996 29076 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:32.041407 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:32.041841 28802 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:32.046800 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:32.041432 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:32.050576 29179 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:32.061982 23616 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 19.
I20260709 13:10:32.064436 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:32.065150 23621 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 12.
I20260709 13:10:32.070003 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:32.070714 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:32.041517 29176 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:10:32.073953 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 51647.3us
I20260709 13:10:31.992805 28964 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:32.070653 22880 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 16.
I20260709 13:10:32.076823 22994 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:32.079756 23617 raft_consensus.cc:2393] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 17: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:32.056010 29185 raft_consensus.cc:2749] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:32.084322 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:32.082675 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:32.064863 29170 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:10:32.090346 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:32.085532 23623 raft_consensus.cc:2376] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 19. Re-sending same reply.
I20260709 13:10:32.091619 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:32.092301 23623 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 18.
I20260709 13:10:32.090833 22844 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:32.036993 29134 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:32.096415 29177 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:32.089663 29062 raft_consensus.cc:2749] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:32.098012 28898 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:32.098676 28898 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:32.099231 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:32.099193 28898 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:32.100390 29159 consensus_meta.cc:287] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.631s	user 0.007s	sys 0.000s
I20260709 13:10:32.066694 29139 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:32.037743 29035 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:32.030252 29022 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:32.093016 29170 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:32.111864 22844 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:32.112450 29135 consensus_queue.cc:1059] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:10:32.083061 29092 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:32.038599 29186 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:32.054499 29183 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:32.062947 29076 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:32.218786 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 102996us
I20260709 13:10:32.067451 29156 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:32.273465 28712 raft_consensus.cc:3055] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:10:32.283865 23620 raft_consensus.cc:2376] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15. Re-sending same reply.
I20260709 13:10:32.279330 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:32.252080 22878 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35360 (request call id 2527) took 1833 ms (client timeout 1926 ms). Trace:
I20260709 13:10:32.281697 29119 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:32.271387 28991 consensus_queue.cc:260] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:32.300845 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 189065us
W20260709 13:10:32.362962 22878 rpcz_store.cc:269] 0709 13:10:30.418673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:30.443512 (+ 24839us) service_pool.cc:224] Handling call
0709 13:10:32.251810 (+1808298us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b54002d7154. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:32.252009 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3129811840}
I20260709 13:10:32.304163 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:32.224337 29092 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:32.305980 28805 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:32.306771 22775 catalog_manager.cc:5697] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 8 to 13, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 13 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:32.307837 29086 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:32.288383 29134 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:32.381795 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:32.310611 29076 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:32.300552 29156 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:32.312148 29075 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:32.297465 29082 raft_consensus.cc:3055] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:10:32.244499 29141 raft_consensus.cc:3055] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:10:32.247499 29145 raft_consensus.cc:3055] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:10:32.318388 22920 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:32.223949 29135 raft_consensus.cc:3055] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:10:32.116328 29191 consensus_queue.cc:1048] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.269s
I20260709 13:10:32.122632 29127 consensus_queue.cc:1059] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.201s
W20260709 13:10:32.247247 29173 log.cc:927] Time spent T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.243s	user 0.001s	sys 0.000s
I20260709 13:10:32.326573 29149 consensus_queue.cc:1048] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 13:10:32.248543 29181 log.cc:927] Time spent T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.236s	user 0.001s	sys 0.001s
W20260709 13:10:32.246819 29143 log.cc:927] Time spent T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
W20260709 13:10:32.243335 29180 log.cc:927] Time spent T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.332s	user 0.000s	sys 0.001s
I20260709 13:10:32.243791 29193 raft_consensus.cc:2804] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:32.431649 29193 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:32.432091 29193 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:32.249089 29187 raft_consensus.cc:2749] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:32.311414 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:32.433926 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:32.434643 22954 raft_consensus.cc:2393] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 16: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:32.435431 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:32.281877 29139 consensus_queue.cc:1059] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:10:32.437074 29139 raft_consensus.cc:3055] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 13:10:32.437201 29160 consensus_meta.cc:287] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.727s	user 0.007s	sys 0.007s
I20260709 13:10:32.437439 29139 raft_consensus.cc:740] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:32.437673 29160 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:32.438340 29139 consensus_queue.cc:260] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:32.439502 29139 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:32.241685 29189 raft_consensus.cc:2749] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:10:32.228137 22877 raft_consensus.cc:606] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Received request to transfer leadership
I20260709 13:10:32.441177 22877 raft_consensus.cc:614] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:10:32.443682 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:32.277279 28802 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:32.245803 29169 consensus_queue.cc:1048] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.124s
I20260709 13:10:32.275465 29157 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:32.334924 29150 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:32.446950 29127 raft_consensus.cc:3055] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:10:32.446674 29150 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:32.447383 29127 raft_consensus.cc:740] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:32.336109 29077 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:32.397024 29135 raft_consensus.cc:740] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:32.337679 29186 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:32.266036 28898 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:32.341632 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:32.450230 23616 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:32.451016 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:32.290436 29035 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:32.268253 29022 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:32.117494 29159 raft_consensus.cc:2749] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:32.455389 22949 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 16.
I20260709 13:10:32.457899 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:32.274138 29148 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:32.462409 29148 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:32.463900 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:32.360754 22918 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:32.374073 22844 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:32.378438 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:32.472594 22874 raft_consensus.cc:2393] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 15: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
W20260709 13:10:32.346889 28813 consensus_meta.cc:287] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.565s	user 0.008s	sys 0.000s
I20260709 13:10:32.380870 22946 raft_consensus.cc:2393] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 18: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:32.381517 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:32.382591 22876 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 17.
I20260709 13:10:32.381517 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:32.383800 22872 raft_consensus.cc:1240] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16. Ops: []
I20260709 13:10:32.383720 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:32.385082 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:32.311393 23025 raft_consensus.cc:1240] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:10:32.385644 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:32.391609 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:32.459102 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:32.482524 23619 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 21.
I20260709 13:10:32.394990 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
W20260709 13:10:32.483436 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 122816us
I20260709 13:10:32.483748 23622 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:10:32.362339 28712 raft_consensus.cc:740] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:32.481299 22953 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 22.
W20260709 13:10:32.401850 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 58541.2us
I20260709 13:10:32.490324 22949 raft_consensus.cc:2410] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:10:32.402940 22875 raft_consensus.cc:1275] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 13:10:32.406946 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:10:32.491796 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 117870us
I20260709 13:10:32.401685 28964 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:32.492501 22844 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:32.389314 29134 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:32.385946 29023 consensus_queue.cc:1048] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 13:10:32.420984 29192 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:32.497558 22873 raft_consensus.cc:1275] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:10:32.497753 29192 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:32.497665 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:32.499624 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:32.500085 29192 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:32.422498 29104 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:32.500989 29104 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:32.502915 29104 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:32.401687 28945 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:10:32.391530 29082 raft_consensus.cc:740] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:32.504184 29150 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:32.395038 29141 raft_consensus.cc:740] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:32.425666 28805 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:32.425621 28991 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:32.522068 28805 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:32.438843 22948 raft_consensus.cc:1240] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
W20260709 13:10:32.261454 29188 log.cc:927] Time spent T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.247s	user 0.001s	sys 0.000s
I20260709 13:10:32.449859 29135 consensus_queue.cc:260] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:32.526201 29135 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:32.526247 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:32.270166 29183 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:32.297587 28801 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:32.381711 29119 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:32.397550 29145 raft_consensus.cc:740] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:32.448314 29127 consensus_queue.cc:260] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:32.121114 28869 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:32.540331 22951 raft_consensus.cc:1240] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16. Ops: []
I20260709 13:10:32.436491 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:32.489586 22878 raft_consensus.cc:2393] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 13: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
W20260709 13:10:32.395494 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 77192.2us
I20260709 13:10:32.542560 22920 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:32.385638 29086 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:32.545049 28827 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:32.545542 28827 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:32.545900 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:32.546411 29086 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:32.546612 23026 raft_consensus.cc:2393] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 17: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:32.546964 29086 raft_consensus.cc:2804] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:10:32.547426 29086 raft_consensus.cc:697] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:32.493624 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:32.495872 22844 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:32.489511 28813 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:32.497296 29187 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:10:32.550557 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:32.550983 29187 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:32.551343 22950 raft_consensus.cc:2393] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 19: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:32.551450 29187 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:32.551738 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:32.496891 28712 consensus_queue.cc:260] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:32.552155 28813 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:32.552806 28712 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:10:32.550940 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 55132.7us
I20260709 13:10:32.555961 22844 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:32.556697 29022 raft_consensus.cc:2804] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:10:32.557181 29022 raft_consensus.cc:697] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 19 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:32.557271 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:32.558066 23023 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 17.
I20260709 13:10:32.559162 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:32.504316 29082 consensus_queue.cc:260] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:32.559875 23023 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:32.500113 22953 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:32.490192 29035 raft_consensus.cc:2749] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:32.509531 29160 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:32.521085 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:32.524928 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:32.552304 23028 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:32.530787 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:32.550592 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:32.565907 23617 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 19.
W20260709 13:10:32.533691 29064 consensus_meta.cc:287] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.822s	user 0.015s	sys 0.002s
I20260709 13:10:32.538416 28801 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:32.538257 28818 consensus_queue.cc:260] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:32.561218 28805 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:32.538987 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:32.544387 29077 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:32.547501 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:32.552037 29023 raft_consensus.cc:2804] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:32.593207 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:32.563647 28869 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:32.560072 29082 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:32.425138 29076 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:32.519006 28945 raft_consensus.cc:697] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:32.539781 23588 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:32.503057 23622 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 19.
W20260709 13:10:32.611416 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 71713.5us
I20260709 13:10:32.572175 23616 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 13.
I20260709 13:10:32.573091 23621 raft_consensus.cc:2393] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 17: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:32.516497 29134 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:32.589287 22952 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.026s	sys 0.088s
I20260709 13:10:32.559008 29187 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:32.597436 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:32.598759 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:32.599313 23623 raft_consensus.cc:2393] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 19: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:32.574717 29150 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:32.627645 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:32.609768 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:32.558120 29022 consensus_queue.cc:237] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:32.629387 23028 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 19.
I20260709 13:10:32.594069 29127 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:32.606747 28869 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:32.644830 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:32.633283 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:32.702570 22876 raft_consensus.cc:2393] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 18: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:32.702533 23617 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 16.
I20260709 13:10:32.700908 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:32.647246 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:32.704107 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:32.704358 23622 raft_consensus.cc:2393] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 19: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
W20260709 13:10:32.617789 29085 consensus_meta.cc:287] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.804s	user 0.013s	sys 0.000s
W20260709 13:10:32.631527 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:32.615214 28902 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:32.625471 28801 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:32.623715 29148 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:10:32.643694 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:32.650023 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:32.660701 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:32.652693 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:32.643468 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:32.649798 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:32.686158 29204 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:32.696237 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:32.691058 29134 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:32.672426 22953 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 13.
I20260709 13:10:32.699705 28062 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:32.646016 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:32.759434 23623 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 16.
I20260709 13:10:32.677306 23590 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:32.685105 22918 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:32.671787 22992 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:10:32.636158 29117 consensus_meta.cc:287] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.978s	user 0.014s	sys 0.000s
I20260709 13:10:32.658226 22919 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:32.687793 23618 raft_consensus.cc:1275] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:10:32.642845 23588 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:32.667112 22775 catalog_manager.cc:5697] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 11 to 14, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 14 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:32.667292 22873 raft_consensus.cc:1240] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 19. Current term is 20. Ops: []
I20260709 13:10:32.563884 29141 consensus_queue.cc:260] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:32.701820 29176 consensus_meta.cc:287] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.551s	user 0.012s	sys 0.000s
I20260709 13:10:32.735898 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:10:32.772395 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 114257us
I20260709 13:10:32.737720 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:32.602253 29077 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:32.739089 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:32.648784 28827 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:32.779788 28369 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:32.698145 29086 consensus_queue.cc:237] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:32.780329 29119 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:32.742614 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:32.781157 22874 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:32.754834 28902 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:32.639653 29201 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:32.698040 28818 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:10:32.748651 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:32.754879 23029 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 19.
I20260709 13:10:32.697208 22950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58320 (request call id 3228) took 1004 ms. Trace:
I20260709 13:10:32.600355 29023 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:32.684849 29092 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:32.758376 22954 raft_consensus.cc:2393] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 17: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
W20260709 13:10:32.761859 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 84718.5us
I20260709 13:10:32.777244 29160 raft_consensus.cc:2749] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:10:32.763481 22948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 3230) took 1052 ms. Trace:
I20260709 13:10:32.740603 29085 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:32.606104 29145 consensus_queue.cc:260] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:32.624810 29076 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:32.768190 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 83188.6us
W20260709 13:10:32.768993 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 97264.3us
I20260709 13:10:32.768363 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:32.767045 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:32.831285 22871 raft_consensus.cc:2393] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 19: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:32.775185 23020 raft_consensus.cc:1275] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:10:32.833213 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:32.833932 22954 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 16.
W20260709 13:10:32.776966 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 134218us
I20260709 13:10:32.778613 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:32.741757 29159 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:32.788081 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:32.841218 22875 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:10:32.779281 28868 consensus_meta.cc:287] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.975s	user 0.010s	sys 0.004s
I20260709 13:10:32.639780 29150 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:32.804175 29203 log.cc:927] Time spent T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:10:32.802466 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:32.849218 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:32.850335 23616 raft_consensus.cc:2393] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 13: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:32.854846 23024 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 15.
W20260709 13:10:32.851897 29088 consensus_meta.cc:287] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 1.025s	user 0.009s	sys 0.007s
I20260709 13:10:32.864995 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:32.865887 23621 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 18.
I20260709 13:10:32.807659 23023 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 20.
I20260709 13:10:32.699077 28945 consensus_queue.cc:237] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:32.802130 29201 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:32.802206 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:32.771360 29138 raft_consensus.cc:2804] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:32.757732 28801 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:32.754133 29148 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:32.806257 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:32.787400 22919 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:10:32.903214 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 115887us
I20260709 13:10:32.816234 22993 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:32.818728 22950 rpcz_store.cc:276] 0709 13:10:31.693249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:31.693581 (+   332us) service_pool.cc:224] Handling call
0709 13:10:32.696975 (+1003394us) spinlock_profiling.cc:243] Waited 777 ms on lock 0x7b5400149454. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:32.697140 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1786250496}
I20260709 13:10:32.784185 28813 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:10:32.777184 29176 raft_consensus.cc:2749] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:32.776825 29191 raft_consensus.cc:2804] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:32.934764 22919 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:32.668366 29187 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:32.823830 22948 rpcz_store.cc:276] 0709 13:10:31.710861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:31.711282 (+   421us) service_pool.cc:224] Handling call
0709 13:10:31.733358 (+ 22076us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:32.525039 (+791681us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:32.763150 (+238111us) spinlock_profiling.cc:243] Waited 705 ms on lock 0x7b5400285854. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:32.763402 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1621873920}
I20260709 13:10:32.763962 22952 raft_consensus.cc:2393] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 22: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:32.764698 22951 raft_consensus.cc:2393] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 18: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:32.826596 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:32.938153 23028 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 16.
I20260709 13:10:32.849273 28868 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:32.835083 23025 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 18.
I20260709 13:10:32.836494 23622 raft_consensus.cc:1240] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 17. Current term is 18. Ops: [17.14-17.14]
I20260709 13:10:32.838131 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:32.942258 23588 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:32.841622 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:32.943210 22871 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 18.
I20260709 13:10:32.943850 22948 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:39742
I20260709 13:10:32.944391 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:32.944834 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:32.945331 22871 raft_consensus.cc:2410] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:10:32.946159 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:32.934054 28813 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:32.946739 29076 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:32.823133 29023 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:32.947147 28813 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:32.845883 22947 raft_consensus.cc:2368] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16.
I20260709 13:10:32.857530 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:32.867249 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:32.786000 29204 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:32.790644 29141 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:32.896167 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:32.896900 23027 raft_consensus.cc:2393] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 13: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:32.847292 29159 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:32.932168 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 142039us
I20260709 13:10:32.936041 23618 raft_consensus.cc:1275] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:10:32.938649 22876 raft_consensus.cc:2393] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 16: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:32.830407 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:32.959848 23029 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
I20260709 13:10:32.946973 22871 raft_consensus.cc:2410] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
W20260709 13:10:32.843948 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 219253us
I20260709 13:10:32.948782 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:32.961374 22951 raft_consensus.cc:2393] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 18: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:32.839555 29145 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:32.900077 29138 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:32.970383 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:32.951957 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:32.952801 23021 raft_consensus.cc:2410] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:10:32.930894 29148 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:32.936007 29160 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:32.889309 29088 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:32.976543 22951 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 19.
I20260709 13:10:32.963647 22878 raft_consensus.cc:2393] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 17: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:32.937165 29191 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:32.967020 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:32.940222 29155 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:10:32.950585 29119 raft_consensus.cc:2749] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:10:32.951823 22952 raft_consensus.cc:1275] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:10:32.970626 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:32.972584 22874 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 16.
I20260709 13:10:32.971277 22993 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:32.951006 29202 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:33.004163 29202 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:32.973567 22843 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:33.013309 29187 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:10:32.952931 29092 raft_consensus.cc:2804] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:32.973701 29138 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:32.982493 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:32.985199 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:32.990010 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:32.988226 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:32.985848 29094 raft_consensus.cc:2749] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:32.992712 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:33.078819 22874 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 20.
W20260709 13:10:32.995646 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 308012us
I20260709 13:10:32.999790 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:10:33.001205 29208 log.cc:927] Time spent T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.173s	user 0.001s	sys 0.001s
I20260709 13:10:33.001474 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:33.003165 29159 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:33.015910 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:33.016383 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 45206.4us
I20260709 13:10:32.991681 22952 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 2523) took 1189 ms. Trace:
I20260709 13:10:32.979943 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:10:33.026600 29193 consensus_meta.cc:287] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.594s	user 0.009s	sys 0.004s
I20260709 13:10:32.984158 29191 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:32.984822 29155 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:33.028995 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:33.032732 29064 raft_consensus.cc:2804] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:10:33.074268 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:33.074761 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:33.118274 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:32.974026 28902 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:33.076862 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:33.080354 22994 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:33.092900 22955 raft_consensus.cc:2393] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 18: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:33.089818 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:33.085264 22871 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18.
W20260709 13:10:33.088140 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:33.077683 22920 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:33.097985 22872 raft_consensus.cc:2468] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 15.
I20260709 13:10:33.084771 23026 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18.
I20260709 13:10:33.099743 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:33.106078 23020 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:10:33.036569 29205 log.cc:927] Time spent T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.348s	user 0.001s	sys 0.004s
I20260709 13:10:33.105036 22952 rpcz_store.cc:276] 0709 13:10:31.801967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:32.947926 (+1145959us) service_pool.cc:224] Handling call
0709 13:10:32.948227 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:32.991426 (+ 43199us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:32.991635 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:33.113765 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:33.123406 22994 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:33.123708 23588 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:33.124514 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:33.008256 29202 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:33.118920 22876 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 20.
W20260709 13:10:33.122821 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 45197us
I20260709 13:10:33.106863 22993 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:33.124646 22994 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:33.020723 29092 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:33.125548 23029 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15.
I20260709 13:10:33.126099 22950 raft_consensus.cc:1275] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:10:33.121462 22843 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:33.119527 23590 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:33.132252 23618 raft_consensus.cc:2393] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 20: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:33.134657 22952 raft_consensus.cc:1240] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 18. Current term is 19. Ops: []
I20260709 13:10:33.158237 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:33.136861 23021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32794 (request call id 1059) took 1238 ms. Trace:
I20260709 13:10:33.159073 22871 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 18.
I20260709 13:10:33.140280 22994 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:33.118979 29155 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:10:33.165268 22952 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 3236) took 1287 ms. Trace:
I20260709 13:10:33.170711 22952 rpcz_store.cc:276] 0709 13:10:31.878119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:33.128134 (+1250015us) service_pool.cc:224] Handling call
0709 13:10:33.128478 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:33.164961 (+ 36483us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:33.165203 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 13:10:33.123510 29170 consensus_meta.cc:287] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.902s	user 0.014s	sys 0.002s
I20260709 13:10:33.166433 22950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 2531) took 1337 ms. Trace:
I20260709 13:10:33.178769 22950 rpcz_store.cc:276] 0709 13:10:31.828940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:33.125384 (+1296444us) service_pool.cc:224] Handling call
0709 13:10:33.125732 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:33.159894 (+ 34162us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:33.166367 (+  6473us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:33.133625 29176 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:33.159466 23021 rpcz_store.cc:276] 0709 13:10:31.898446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:31.898569 (+   123us) service_pool.cc:224] Handling call
0709 13:10:32.046682 (+148113us) spinlock_profiling.cc:243] Waited 2.62 ms on lock 0x7b58001ca198. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0eee 0000000000375939 00007f613ad341e0 00007f613adfd0c4 00007f613adfd131 00007f613adab0a0 00007f613adaa984 00007f61343256d4 00007f6134325662
0709 13:10:33.136576 (+1089894us) spinlock_profiling.cc:243] Waited 893 ms on lock 0x7b5400134a54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:33.136790 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2059634304}
W20260709 13:10:33.173310 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:33.173285 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:33.186982 22876 raft_consensus.cc:1813] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Handling vote request from an unknown peer aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:33.100914 29035 consensus_queue.cc:1048] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 13:10:33.174906 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:33.179584 29061 raft_consensus.cc:2804] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:10:33.122702 29207 log.cc:927] Time spent T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.234s	user 0.000s	sys 0.001s
I20260709 13:10:33.133302 28821 raft_consensus.cc:2804] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:10:33.167603 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 46307.3us
I20260709 13:10:33.138739 28369 raft_consensus.cc:2749] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:33.122517 29159 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:33.157747 28991 raft_consensus.cc:2804] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:10:33.205603 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 375956us
I20260709 13:10:33.207077 22947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58320 (request call id 3234) took 1431 ms. Trace:
I20260709 13:10:33.207566 22947 rpcz_store.cc:276] 0709 13:10:31.775793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:31.788092 (+ 12299us) service_pool.cc:224] Handling call
0709 13:10:33.206856 (+1418764us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b5400074ed4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:33.207012 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2379617152}
I20260709 13:10:33.190188 22876 raft_consensus.cc:2410] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:10:33.192195 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:33.211804 23029 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 18.
I20260709 13:10:33.212018 22873 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 18.
I20260709 13:10:33.143178 29085 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:33.116981 29193 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:33.128921 29127 raft_consensus.cc:2749] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:33.172243 29092 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:33.167057 29202 consensus_queue.cc:1059] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 13:10:33.170001 28902 raft_consensus.cc:2764] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:10:33.168205 29134 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:10:33.218374 22951 raft_consensus.cc:1240] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:10:33.247700 22951 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 1085) took 1327 ms. Trace:
I20260709 13:10:33.248080 22951 rpcz_store.cc:276] 0709 13:10:31.919910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:33.217730 (+1297820us) service_pool.cc:224] Handling call
0709 13:10:33.218141 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:33.247448 (+ 29307us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:33.247645 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:33.220237 22955 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58320 (request call id 3226) took 1538 ms. Trace:
I20260709 13:10:33.248914 22955 rpcz_store.cc:276] 0709 13:10:31.682080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:31.682181 (+   101us) service_pool.cc:224] Handling call
0709 13:10:33.220013 (+1537832us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b5400284e54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:33.220169 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3113249664}
W20260709 13:10:33.221102 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:33.181124 29150 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:33.222611 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 392418us
I20260709 13:10:33.139684 29064 raft_consensus.cc:697] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 19 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:33.223024 22992 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:33.223694 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:33.258363 22947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 3237) took 1280 ms. Trace:
I20260709 13:10:33.224609 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:33.229700 22873 raft_consensus.cc:1275] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 12. (term mismatch)
I20260709 13:10:33.192066 29061 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:33.162700 28827 raft_consensus.cc:2749] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:10:33.240586 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:33.241132 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:33.242182 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:33.205029 28991 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:33.253082 22876 raft_consensus.cc:1240] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:10:33.185477 29170 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:33.253631 22918 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:33.254997 23625 raft_consensus.cc:1240] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 17. Current term is 18. Ops: []
I20260709 13:10:33.228276 29187 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:33.260694 22873 pending_rounds.cc:85] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 11
I20260709 13:10:33.221792 29159 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:33.260069 23026 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 16.
I20260709 13:10:33.314128 23021 raft_consensus.cc:1240] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 13:10:33.262935 22951 raft_consensus.cc:1240] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:33.267498 22875 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 16.
W20260709 13:10:33.271405 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 48490.1us
I20260709 13:10:33.277415 23028 raft_consensus.cc:1275] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:10:33.279608 29077 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:33.280093 29209 log.cc:927] Time spent T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.179s	user 0.000s	sys 0.001s
I20260709 13:10:33.250232 29134 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:10:33.282060 29135 consensus_meta.cc:287] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.755s	user 0.012s	sys 0.000s
I20260709 13:10:33.332692 22951 raft_consensus.cc:1240] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 15. Current term is 16. Ops: []
I20260709 13:10:33.275969 29076 raft_consensus.cc:2749] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:33.250169 29201 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:33.312033 23025 raft_consensus.cc:1240] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:33.322110 22880 raft_consensus.cc:1240] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 13:10:33.323976 22950 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:33.293895 22873 pending_rounds.cc:107] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 17.12
I20260709 13:10:33.273499 29061 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:33.289266 28991 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:10:33.368115 23588 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:33.253394 29127 consensus_queue.cc:1048] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 13:10:33.377436 22951 raft_consensus.cc:1240] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 19. Current term is 20. Ops: []
I20260709 13:10:33.260207 22947 rpcz_store.cc:276] 0709 13:10:31.978368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:33.238437 (+1260069us) service_pool.cc:224] Handling call
0709 13:10:33.257670 (+ 19233us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:33.257869 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:33.257904 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:33.257957 (+    53us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:10:33.258001 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:33.258068 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:33.258199 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:33.309262 29187 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:33.332787 29134 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:33.308996 29088 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:33.391993 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:33.392724 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:33.364359 29204 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:33.402711 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:33.344082 22873 raft_consensus.cc:2889] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:33.422120 29170 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:33.429828 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 176274us
I20260709 13:10:33.389919 22951 raft_consensus.cc:1240] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 17. Current term is 18. Ops: []
I20260709 13:10:33.433415 22918 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:33.434747 22918 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:33.439925 22877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 3112) took 1064 ms. Trace:
I20260709 13:10:33.440537 22877 rpcz_store.cc:276] 0709 13:10:32.375402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:32.492221 (+116819us) service_pool.cc:224] Handling call
0709 13:10:32.492550 (+   329us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:32.493021 (+   471us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:32.493058 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:32.493484 (+   426us) log.cc:844] Serialized 26 byte log entry
0709 13:10:32.732931 (+239447us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:10:32.733403 (+   472us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:32.733476 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:33.439601 (+706125us) raft_consensus.cc:1685] finished
0709 13:10:33.439632 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:33.439835 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1694,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":2608,"thread_start_us":201121,"threads_started":1,"wal-append.queue_time_us":449834}
I20260709 13:10:33.452553 22947 raft_consensus.cc:1275] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 13:10:33.453379 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 133871us
W20260709 13:10:33.454463 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 321041us
I20260709 13:10:33.456833 22920 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:33.457770 22920 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:33.443037 29159 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:33.460106 23590 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:33.454795 22992 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:33.465714 22992 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:33.443979 29086 raft_consensus.cc:2749] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:33.469995 23625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 1140) took 1509 ms. Trace:
I20260709 13:10:33.475605 22844 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:33.490458 23625 rpcz_store.cc:276] 0709 13:10:31.960855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:32.108815 (+147960us) service_pool.cc:224] Handling call
0709 13:10:32.222894 (+114079us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:33.291034 (+1068140us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:33.469749 (+178715us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b5400671454. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:33.469944 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2361926912}
I20260709 13:10:33.475498 22992 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:33.495296 22992 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:33.483775 22920 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:10:33.511868 28818 consensus_meta.cc:287] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.694s	user 0.012s	sys 0.000s
I20260709 13:10:33.512578 28369 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:33.513206 28369 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:33.514806 28818 raft_consensus.cc:2804] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:33.515472 28818 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:33.516182 28818 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:10:33.516649 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:33.506556 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 46431.5us
I20260709 13:10:33.397512 29064 consensus_queue.cc:237] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:33.545836 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:33.488351 29159 raft_consensus.cc:2749] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:10:33.503160 29077 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:33.557974 29077 consensus_queue.cc:1059] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 13:10:33.559248 29077 raft_consensus.cc:2749] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:33.559965 29077 consensus_queue.cc:1059] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:10:33.465938 29170 consensus_queue.cc:1059] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.095s
I20260709 13:10:33.562815 29187 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:33.465265 29119 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:33.563861 29119 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:33.466274 29035 raft_consensus.cc:2804] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:33.565034 29035 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:33.565512 29035 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:33.566568 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:10:33.568279 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:33.504761 29135 consensus_queue.cc:1048] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 13:10:33.497676 29204 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:33.571144 29204 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:33.571578 29204 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:10:33.574764 29023 consensus_meta.cc:287] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.620s	user 0.011s	sys 0.003s
I20260709 13:10:33.575665 22950 raft_consensus.cc:1275] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:10:33.575309 29023 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:33.578135 28869 consensus_meta.cc:287] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.839s	user 0.005s	sys 0.011s
I20260709 13:10:33.578606 28869 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:33.579103 29023 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:33.579267 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:33.532389 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:10:33.591252 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:33.495203 29076 raft_consensus.cc:2804] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 13:10:33.499647 28827 raft_consensus.cc:2749] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:33.600705 22946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 3233) took 1827 ms. Trace:
I20260709 13:10:33.604408 22953 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 3235) took 1824 ms. Trace:
I20260709 13:10:33.606458 22953 rpcz_store.cc:276] 0709 13:10:31.780035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:32.945848 (+1165813us) service_pool.cc:224] Handling call
0709 13:10:32.946182 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:32.946395 (+   213us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:10:33.235317 (+288922us) log.cc:844] Serialized 14 byte log entry
0709 13:10:33.604070 (+368753us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:33.604126 (+    56us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:10:33.604171 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:33.604237 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:33.604356 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":68070,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"thread_start_us":656236,"threads_started":2}
W20260709 13:10:33.594501 29082 consensus_meta.cc:287] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.978s	user 0.011s	sys 0.002s
W20260709 13:10:33.612232 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:33.610859 22952 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 1082) took 1701 ms. Trace:
I20260709 13:10:33.593950 28898 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:10:33.596845 29022 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.014s	sys 0.049s
I20260709 13:10:33.589257 29085 raft_consensus.cc:2749] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:33.615383 22949 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 2533) took 1782 ms. Trace:
I20260709 13:10:33.617444 22949 rpcz_store.cc:276] 0709 13:10:31.832615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:33.129967 (+1297352us) service_pool.cc:224] Handling call
0709 13:10:33.169017 (+ 39050us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:33.169272 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:10:33.191540 (+ 22268us) log.cc:844] Serialized 14 byte log entry
0709 13:10:33.615010 (+423470us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:33.615076 (+    66us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:10:33.615145 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:33.615226 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:33.615328 (+   102us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25140,"prepare.queue_time_us":59132,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"thread_start_us":423083,"threads_started":1}
W20260709 13:10:33.627571 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 438469us
I20260709 13:10:33.615619 22952 rpcz_store.cc:276] 0709 13:10:31.909109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:33.171854 (+1262745us) service_pool.cc:224] Handling call
0709 13:10:33.172199 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:33.172419 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:10:33.226646 (+ 54227us) log.cc:844] Serialized 14 byte log entry
0709 13:10:33.610466 (+383820us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:33.610529 (+    63us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:10:33.610586 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:33.610671 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:33.610797 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18920,"prepare.queue_time_us":131958,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":44,"thread_start_us":383428,"threads_started":1}
I20260709 13:10:33.610893 29088 raft_consensus.cc:2804] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:10:33.620106 29022 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.014s	sys 0.050s
I20260709 13:10:33.610131 28821 raft_consensus.cc:697] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:33.617323 29170 raft_consensus.cc:3055] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 13:10:33.642627 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 462076us
I20260709 13:10:33.605945 22946 rpcz_store.cc:276] 0709 13:10:31.773192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:32.562216 (+789024us) service_pool.cc:224] Handling call
0709 13:10:32.562472 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:32.562643 (+   171us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:10:32.784211 (+221568us) log.cc:844] Serialized 14 byte log entry
0709 13:10:33.600241 (+816030us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:33.600315 (+    74us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:10:33.600376 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:33.600463 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:33.600618 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":392525,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":71523,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":5165,"thread_start_us":1017707,"threads_started":3,"wal-append.queue_time_us":520549}
I20260709 13:10:33.615056 29076 raft_consensus.cc:697] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:33.620020 28898 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:33.604113 29023 consensus_queue.cc:1059] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 13:10:33.633314 28827 raft_consensus.cc:3055] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:10:33.688820 28827 raft_consensus.cc:740] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:33.650099 29170 raft_consensus.cc:740] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:33.660100 28898 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:10:33.695405 28813 consensus_meta.cc:287] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.737s	user 0.015s	sys 0.002s
I20260709 13:10:33.715006 29150 raft_consensus.cc:3055] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:10:33.721097 28869 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:33.721545 28821 consensus_queue.cc:237] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:33.721881 29076 consensus_queue.cc:237] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:33.712927 29148 consensus_meta.cc:287] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.725s	user 0.012s	sys 0.004s
W20260709 13:10:33.735234 23020 consensus_meta.cc:287] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.601s	user 0.014s	sys 0.004s
I20260709 13:10:33.738694 23020 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 19.
I20260709 13:10:33.739521 23020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33420 (request call id 2529) took 1188 ms. Trace:
W20260709 13:10:33.728498 29139 consensus_meta.cc:287] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 1.289s	user 0.017s	sys 0.001s
I20260709 13:10:33.739951 23020 rpcz_store.cc:276] 0709 13:10:32.550695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:32.965684 (+414989us) service_pool.cc:224] Handling call
0709 13:10:33.739439 (+773755us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 13:10:33.751606 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
W20260709 13:10:33.730394 29138 consensus_meta.cc:287] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.623s	user 0.011s	sys 0.000s
I20260709 13:10:33.762423 29138 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:33.720301 29061 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:33.773802 28868 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:33.777662 29151 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:33.778406 29151 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:33.736439 28801 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:33.781116 29170 consensus_queue.cc:260] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:33.782325 29170 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:10:33.783803 29141 consensus_meta.cc:287] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.815s	user 0.010s	sys 0.007s
I20260709 13:10:33.734741 28813 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:33.785773 29134 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:33.786525 28813 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:33.786695 29141 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:33.786832 29119 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:33.787544 22954 raft_consensus.cc:1240] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 19. Current term is 20. Ops: []
I20260709 13:10:33.751332 28827 consensus_queue.cc:260] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:33.689735 28818 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:33.794624 28818 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:33.795640 29022 raft_consensus.cc:3055] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:10:33.796206 29022 raft_consensus.cc:740] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:33.797479 29022 consensus_queue.cc:260] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:33.798471 28369 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:33.731101 29150 raft_consensus.cc:740] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:33.809827 23029 raft_consensus.cc:2393] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 18: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
W20260709 13:10:33.801793 28945 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.023s	sys 0.035s
I20260709 13:10:33.804329 29022 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:10:33.816953 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 166309us
W20260709 13:10:33.810547 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 304944us
I20260709 13:10:33.821466 22992 leader_election.cc:400] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16.
I20260709 13:10:33.822788 22992 leader_election.cc:403] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Cancelling election due to peer responding with higher term
I20260709 13:10:33.805173 29224 raft_consensus.cc:3055] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:10:33.753623 29148 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:33.817817 28945 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.023s	sys 0.035s
I20260709 13:10:33.787189 29141 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:33.838797 22955 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 3242) took 1474 ms. Trace:
I20260709 13:10:33.840603 22948 raft_consensus.cc:606] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 LEADER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:33.807761 29064 consensus_queue.cc:1048] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.022s
I20260709 13:10:33.844413 23616 raft_consensus.cc:1240] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 19. Current term is 20. Ops: [19.12-19.12]
W20260709 13:10:33.845779 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 324278us
I20260709 13:10:33.826840 29224 raft_consensus.cc:740] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:33.835714 29151 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:10:33.861405 29155 consensus_meta.cc:287] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.674s	user 0.009s	sys 0.002s
I20260709 13:10:33.894970 29155 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:33.897320 29155 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:33.898023 23023 raft_consensus.cc:2368] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 22. Current term is 23.
I20260709 13:10:33.863838 29139 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:33.898617 29139 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:33.899262 29141 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:33.866220 29239 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:10:33.901994 29176 consensus_meta.cc:287] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.707s	user 0.010s	sys 0.000s
I20260709 13:10:33.901652 29204 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:33.902391 29176 raft_consensus.cc:2749] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:33.902786 29193 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:33.903054 23026 raft_consensus.cc:1240] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 15. Current term is 16. Ops: []
I20260709 13:10:33.868537 22874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 2565) took 1200 ms. Trace:
I20260709 13:10:33.841532 29138 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:33.904068 22874 rpcz_store.cc:276] 0709 13:10:32.667699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:33.233127 (+565428us) service_pool.cc:224] Handling call
0709 13:10:33.233494 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:33.233870 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:33.233915 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:33.241530 (+  7615us) log.cc:844] Serialized 26 byte log entry
0709 13:10:33.650061 (+408531us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:10:33.650099 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:33.650170 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:33.868259 (+218089us) raft_consensus.cc:1685] finished
0709 13:10:33.868284 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:33.868471 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5786,"raft.queue_time_us":1024,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":11216,"thread_start_us":321809,"threads_started":1,"wal-append.queue_time_us":552392}
I20260709 13:10:33.869444 29061 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:33.873057 28369 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:33.875591 29224 consensus_queue.cc:260] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:33.876075 28991 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:33.841045 29150 consensus_queue.cc:260] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:33.877446 22826 auto_rebalancer.cc:893] Removing replace marker failed: Replica 0a407504ed1741bf89a276558adf4c0f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 13 committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
W20260709 13:10:33.887730 29160 consensus_meta.cc:287] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.901s	user 0.010s	sys 0.011s
I20260709 13:10:33.890337 29134 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 21 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:33.904996 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:33.872687 29191 consensus_meta.cc:287] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.750s	user 0.010s	sys 0.007s
I20260709 13:10:33.905675 23026 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 2459) took 1229 ms. Trace:
I20260709 13:10:33.905512 29148 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:33.911885 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:33.906471 23027 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 20.
I20260709 13:10:33.906957 22920 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:33.918882 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:33.919907 22826 auto_rebalancer.cc:901] Could not move replica: Illegal state: not currently leader
W20260709 13:10:33.921608 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 461548us
I20260709 13:10:33.922016 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:33.933967 22949 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 18.
I20260709 13:10:33.922649 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:33.923273 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:33.934958 22920 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:33.923496 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:33.925347 23026 rpcz_store.cc:276] 0709 13:10:32.676569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:33.314278 (+637709us) service_pool.cc:224] Handling call
0709 13:10:33.314661 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:33.903483 (+588822us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:33.905463 (+  1980us) spinlock_profiling.cc:243] Waited 588 ms on lock 0x7b540010c554. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:33.905618 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1352693504}
I20260709 13:10:33.905696 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:10:33.925009 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 87406.4us
I20260709 13:10:33.926788 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:33.925812 29160 raft_consensus.cc:2749] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:33.906042 29139 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:33.912953 29204 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:33.933373 23024 raft_consensus.cc:2393] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 17: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:33.938558 23617 raft_consensus.cc:2393] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 17: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:33.938995 22952 raft_consensus.cc:2393] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 18: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
W20260709 13:10:33.932323 22826 auto_rebalancer.cc:330] scheduled replica move failed to complete: Illegal state: not currently leader
I20260709 13:10:33.943475 23590 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:33.930945 29150 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:33.945025 29159 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:33.929651 29191 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:33.949483 29035 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:33.950023 23028 raft_consensus.cc:1240] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:33.951696 29191 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:33.955735 28898 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:33.941124 29145 consensus_meta.cc:287] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.967s	user 0.013s	sys 0.008s
I20260709 13:10:33.924311 29238 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:33.972694 29238 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:33.839694 22955 rpcz_store.cc:276] 0709 13:10:32.364221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:33.258921 (+894700us) service_pool.cc:224] Handling call
0709 13:10:33.259209 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:33.259471 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:10:33.649355 (+389884us) log.cc:844] Serialized 14 byte log entry
0709 13:10:33.838354 (+188999us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:33.838441 (+    87us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:10:33.838500 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:33.838586 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:33.838728 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":95877,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":72,"thread_start_us":557003,"threads_started":2}
I20260709 13:10:33.974967 29238 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:33.975719 29094 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:33.976217 29094 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:33.978314 23023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32794 (request call id 1067) took 1307 ms. Trace:
I20260709 13:10:33.978694 23023 rpcz_store.cc:276] 0709 13:10:32.671074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:32.895868 (+224794us) service_pool.cc:224] Handling call
0709 13:10:33.978128 (+1082260us) spinlock_profiling.cc:243] Waited 942 ms on lock 0x7b5400134554. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:33.978259 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2165517696}
I20260709 13:10:33.979033 29094 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:33.983889 23027 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33418 (request call id 2447) took 1353 ms. Trace:
I20260709 13:10:33.936455 29224 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:33.985924 22954 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 2526) took 2183 ms. Trace:
I20260709 13:10:33.986320 22954 rpcz_store.cc:276] 0709 13:10:31.802956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:32.972996 (+1170040us) service_pool.cc:224] Handling call
0709 13:10:33.015036 (+ 42040us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:33.788041 (+773005us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:33.985671 (+197630us) spinlock_profiling.cc:243] Waited 772 ms on lock 0x7b54000758d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:33.985867 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1776011520}
I20260709 13:10:33.959465 29088 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:10:33.989113 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 45672.8us
I20260709 13:10:33.958609 28991 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:33.943997 22920 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:33.967308 22879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 3116) took 1282 ms. Trace:
I20260709 13:10:33.964075 29035 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:33.984326 23027 rpcz_store.cc:276] 0709 13:10:32.630135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:32.970343 (+340208us) service_pool.cc:224] Handling call
0709 13:10:33.983726 (+1013383us) spinlock_profiling.cc:243] Waited 777 ms on lock 0x7b540010bb54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:33.983833 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1786653440}
I20260709 13:10:33.963519 29234 raft_consensus.cc:2749] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:10:33.993139 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:33.996104 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 52711.9us
I20260709 13:10:33.970783 28827 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:10:34.006827 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:34.007365 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:33.994537 22879 rpcz_store.cc:276] 0709 13:10:32.684510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:33.227429 (+542919us) service_pool.cc:224] Handling call
0709 13:10:33.227868 (+   439us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:10:33.253652 (+ 25784us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:33.253696 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:10:33.259470 (+  5774us) log.cc:844] Serialized 52 byte log entry
0709 13:10:33.654216 (+394746us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:10:33.700407 (+ 46191us) log.cc:844] Serialized 14 byte log entry
0709 13:10:33.788635 (+ 88228us) spinlock_profiling.cc:243] Waited 3.52 ms on lock 0x7b1800341e60. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cf83f3 00007f6132cfa7c1 00007f6132cfa779 00007f6132cfa709 00007f6132cfa6d1 00007f6132cf99cd 000000000038f354 000000000038f199 00007f6132cdb83a
0709 13:10:33.966600 (+177965us) spinlock_profiling.cc:243] Waited 3.02 ms on lock 0x7b1800341e60. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132c72c0a 00007f613c3b2219 00007f6132cfaa40 00007f6132cf54e4 00007f6132cf5455 00007f61397dd3c7 00007f6135def9a7 00007f6135dff0bf 00007f6135dff038
0709 13:10:33.966788 (+   188us) log.cc:844] Serialized 14 byte log entry
0709 13:10:33.966998 (+   210us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:33.967073 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:33.967113 (+    40us) raft_consensus.cc:1685] finished
0709 13:10:33.967125 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:33.967257 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":231717,"prepare.queue_time_us":315871,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":4928,"raft.queue_time_us":33518,"raft.run_cpu_time_us":354,"raft.run_wall_time_us":31022,"spinlock_wait_cycles":15062656,"thread_start_us":326449,"threads_started":1,"wal-append.queue_time_us":554466}
W20260709 13:10:33.996543 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 159572us
W20260709 13:10:33.997246 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 53330.8us
I20260709 13:10:33.997598 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:33.998423 22948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:39742 (request call id 6) took 1351 ms. Trace:
I20260709 13:10:34.009409 22948 rpcz_store.cc:276] 0709 13:10:32.642341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:32.943698 (+301357us) service_pool.cc:224] Handling call
0709 13:10:33.993478 (+1049780us) spinlock_profiling.cc:243] Waited 896 ms on lock 0x7b540037afd4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135ddf853 00007f613adad60a 00007f61343337d4 00007f6134333762 00007f6134333691 00007f613433360c 00007f61343328c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:33.993595 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2060776192}
I20260709 13:10:33.990564 29088 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:10:34.000413 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:33.919538 28369 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:33.999632 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:10:34.012426 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:34.012532 22872 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:10:34.013229 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:34.013509 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:34.015780 22843 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:34.017868 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:34.018254 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:34.018532 22877 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:34.022059 29247 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:34.001464 29160 raft_consensus.cc:2804] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:10:34.024852 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:34.025566 22950 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:34.001144 29145 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:34.026441 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:34.027652 23616 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 1150) took 1190 ms. Trace:
I20260709 13:10:34.030591 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:34.025063 22843 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:34.028074 23024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33420 (request call id 2537) took 1451 ms. Trace:
I20260709 13:10:34.031071 23026 raft_consensus.cc:1240] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 19. Current term is 20. Ops: []
I20260709 13:10:34.031488 23024 rpcz_store.cc:276] 0709 13:10:32.576722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:33.224306 (+647584us) service_pool.cc:224] Handling call
0709 13:10:34.027865 (+803559us) spinlock_profiling.cc:243] Waited 673 ms on lock 0x7b54001c84d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:34.028016 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1546932480}
I20260709 13:10:34.032292 23020 raft_consensus.cc:1275] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 13:10:34.030298 23616 rpcz_store.cc:276] 0709 13:10:32.837197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:32.862709 (+ 25512us) service_pool.cc:224] Handling call
0709 13:10:32.893181 (+ 30472us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:33.844857 (+951676us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:34.027381 (+182524us) spinlock_profiling.cc:243] Waited 951 ms on lock 0x7b540045d654. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:34.027586 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2186957568}
I20260709 13:10:34.054409 23027 raft_consensus.cc:1275] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 13:10:34.054594 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:34.055263 23025 raft_consensus.cc:2468] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15.
I20260709 13:10:34.055589 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:34.058857 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:34.059433 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:34.060150 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:34.060334 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:34.061359 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:34.062053 23024 raft_consensus.cc:2393] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 19: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:34.061836 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:34.062824 23026 raft_consensus.cc:2410] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 13:10:34.061781 23023 raft_consensus.cc:1240] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 17. Current term is 18. Ops: []
I20260709 13:10:34.063829 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:34.063825 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:34.065313 23020 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
I20260709 13:10:34.067382 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:34.067910 23622 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Advancing to term 23
I20260709 13:10:34.067781 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:34.068516 23619 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:34.069123 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:34.009811 29243 raft_consensus.cc:2804] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:10:34.023562 28821 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:34.099545 23619 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 17.
W20260709 13:10:34.101841 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 84158.8us
I20260709 13:10:34.103030 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:34.103889 23025 raft_consensus.cc:1275] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
W20260709 13:10:34.104609 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:34.074698 23027 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 16.
I20260709 13:10:34.073257 22950 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 21.
I20260709 13:10:34.023013 29247 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:34.075978 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:34.106019 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:34.106359 23624 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:34.106621 23027 raft_consensus.cc:2376] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 20. Re-sending same reply.
I20260709 13:10:34.054461 29145 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:34.028954 29094 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:34.081202 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:34.081274 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:34.082698 23622 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 23.
I20260709 13:10:34.032866 29159 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:34.085682 28898 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:34.089607 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:34.185487 22875 raft_consensus.cc:1240] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:34.090176 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:34.186089 23022 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 16.
I20260709 13:10:34.090791 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:10:34.091693 29092 consensus_meta.cc:287] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.841s	user 0.014s	sys 0.004s
I20260709 13:10:34.094661 28369 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:34.095484 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:34.096082 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:34.019405 29150 consensus_queue.cc:1048] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 13:10:34.069054 29239 raft_consensus.cc:2749] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election lost for term 15. Reason: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16.
I20260709 13:10:34.055665 29148 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:34.058128 29160 raft_consensus.cc:697] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:34.190280 23021 raft_consensus.cc:2393] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 14: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:34.218133 23618 raft_consensus.cc:2410] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 21 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 13:10:34.189523 22875 raft_consensus.cc:1240] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 19. Current term is 20. Ops: []
I20260709 13:10:34.190682 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:34.190315 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:34.189610 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:34.190037 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:34.203325 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:34.201195 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:34.190963 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:34.220072 23022 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18.
I20260709 13:10:34.220315 22877 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 16.
I20260709 13:10:34.120853 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:34.221791 23023 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 16.
I20260709 13:10:34.219245 22954 raft_consensus.cc:2393] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 17: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:34.224412 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:34.098145 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:34.224001 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:34.101854 22872 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 18.
I20260709 13:10:34.226073 23622 raft_consensus.cc:2393] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 20: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:34.105149 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:34.056897 29170 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:34.107386 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:34.107647 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:34.228616 23027 raft_consensus.cc:2376] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 20. Re-sending same reply.
I20260709 13:10:34.111081 22880 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 18.
I20260709 13:10:34.183094 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:34.187006 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:34.187392 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 68086.5us
I20260709 13:10:34.231583 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:34.187063 22873 raft_consensus.cc:3060] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:34.069587 29135 raft_consensus.cc:2749] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:34.188369 23026 raft_consensus.cc:2393] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 17: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:34.111052 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:34.189062 29233 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:34.201319 22993 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:34.226706 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:34.227692 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:34.228741 22955 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 18.
I20260709 13:10:34.110294 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:34.098121 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 162226us
I20260709 13:10:34.240725 22993 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:10:34.243449 23588 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:34.112401 22947 raft_consensus.cc:1275] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:10:34.234299 23022 raft_consensus.cc:2393] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 19: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:34.235167 23024 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 14.
I20260709 13:10:34.237072 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:34.230319 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:34.244787 23588 leader_election.cc:400] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 18. Current term is 19.
I20260709 13:10:34.237767 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:34.070164 29138 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:34.242867 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:34.245078 23588 leader_election.cc:403] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:10:34.110703 28991 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:34.247013 23588 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:34.247639 22872 raft_consensus.cc:1275] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:10:34.248603 23616 raft_consensus.cc:1240] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:34.248797 23588 leader_election.cc:400] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 22 pre-election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 22. Current term is 23.
I20260709 13:10:34.249089 23588 leader_election.cc:403] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:10:34.110787 29245 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:34.252405 22993 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:34.199775 29134 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:34.184155 29243 raft_consensus.cc:697] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 18 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:34.206465 29247 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:34.214080 29145 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:34.297791 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:34.309557 22952 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:34.310552 22873 raft_consensus.cc:2468] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 21.
I20260709 13:10:34.258914 23588 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:34.188192 29094 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:34.249867 28369 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:34.260378 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:34.314960 23023 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18.
I20260709 13:10:34.261164 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:34.254240 29245 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:34.265336 23029 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33418 (request call id 2451) took 1591 ms. Trace:
I20260709 13:10:34.194377 29088 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:34.317165 23029 rpcz_store.cc:276] 0709 13:10:32.674128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:33.225506 (+551378us) service_pool.cc:224] Handling call
0709 13:10:34.265144 (+1039638us) spinlock_profiling.cc:243] Waited 539 ms on lock 0x7b5400249354. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:34.265276 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1239371648}
I20260709 13:10:34.266867 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:34.267390 22993 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:34.318805 23024 raft_consensus.cc:2410] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 18 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:10:34.268448 29155 raft_consensus.cc:2749] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:34.273890 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:34.318938 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 51608.8us
I20260709 13:10:34.274483 22843 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:34.276523 23028 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33418 (request call id 2460) took 1594 ms. Trace:
I20260709 13:10:34.256196 29193 raft_consensus.cc:2749] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 18. Current term is 19.
I20260709 13:10:34.280027 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:34.328770 23616 raft_consensus.cc:2368] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 20. Current term is 21.
I20260709 13:10:34.282796 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:34.098552 28868 consensus_queue.cc:1048] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.146s
I20260709 13:10:34.289811 29224 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:10:34.288436 29237 raft_consensus.cc:2749] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 22. Current term is 23.
I20260709 13:10:34.111994 28821 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:34.290854 22919 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:34.294618 23624 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 19.
I20260709 13:10:34.295943 23020 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 16.
I20260709 13:10:34.300103 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:34.347447 22876 raft_consensus.cc:2368] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 20. Current term is 21.
I20260709 13:10:34.184405 29151 consensus_queue.cc:1048] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 13:10:34.305202 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:34.348515 22946 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 18.
I20260709 13:10:34.349031 22952 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 17.
I20260709 13:10:34.111182 28869 consensus_queue.cc:1059] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.145s
I20260709 13:10:34.308274 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:34.369367 22953 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 16.
I20260709 13:10:34.312237 22994 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:34.245596 23021 raft_consensus.cc:2376] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 19. Re-sending same reply.
I20260709 13:10:34.309449 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:34.314445 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:34.288985 29141 raft_consensus.cc:2804] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:34.227612 29092 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:34.321748 28369 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:34.322844 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 48468.3us
I20260709 13:10:34.233508 29148 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:34.323555 29145 raft_consensus.cc:2749] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:10:34.304598 29134 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:34.239799 29170 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:34.317723 29094 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:34.325071 29247 raft_consensus.cc:2749] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:34.320184 23028 rpcz_store.cc:276] 0709 13:10:32.682290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:33.320359 (+638069us) service_pool.cc:224] Handling call
0709 13:10:33.341635 (+ 21276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:33.950400 (+608765us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:34.276270 (+325870us) spinlock_profiling.cc:243] Waited 608 ms on lock 0x7b54001798d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:34.276460 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1398634880}
W20260709 13:10:34.326586 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 334109us
I20260709 13:10:34.394042 22918 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:34.255537 29176 raft_consensus.cc:2804] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 13:10:34.395287 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 136433us
I20260709 13:10:34.329898 22950 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:10:34.396201 23588 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:34.397464 23588 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:34.227403 29035 consensus_queue.cc:1059] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.076s
I20260709 13:10:34.210749 29159 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:34.323566 29245 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:34.334136 29160 consensus_queue.cc:237] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:34.251914 29233 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:34.290372 28991 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:34.337400 29224 raft_consensus.cc:697] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 23 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:34.336915 29243 consensus_queue.cc:237] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:34.338750 29022 consensus_meta.cc:287] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.511s	user 0.009s	sys 0.000s
I20260709 13:10:34.338629 29193 raft_consensus.cc:2749] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:34.340696 29088 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:34.341225 23029 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:10:34.471369 23029 raft_consensus.cc:606] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Received request to transfer leadership to TS aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:34.471856 23029 raft_consensus.cc:614] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:10:34.349562 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:34.306538 22992 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:34.349942 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:34.351229 23020 raft_consensus.cc:1275] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 13:10:34.351138 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 61340.9us
I20260709 13:10:34.291421 29138 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:34.352566 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:34.476806 22949 raft_consensus.cc:3060] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:34.346846 22993 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:34.366122 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:34.477886 22879 raft_consensus.cc:2468] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 12.
I20260709 13:10:34.367503 22920 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:34.279651 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:34.366899 23590 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:34.373106 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:34.362845 29086 raft_consensus.cc:2749] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:10:34.389032 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 76906us
I20260709 13:10:34.480458 22994 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:34.372247 29237 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:34.485059 29237 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:34.486308 29159 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:34.487274 29237 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:34.412153 22951 raft_consensus.cc:2393] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 17: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:34.412567 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:34.488785 22952 raft_consensus.cc:2393] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 18: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:34.413389 22918 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
W20260709 13:10:34.489880 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 76547.2us
I20260709 13:10:34.489990 22775 catalog_manager.cc:5697] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 3 to 13, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 13 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:34.490797 22918 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:34.372013 29141 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:34.491739 29141 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:34.492492 22775 catalog_manager.cc:5697] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 9 to 12, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 12 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:34.494020 22775 catalog_manager.cc:5697] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:34.415167 22950 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 23.
I20260709 13:10:34.466938 29183 raft_consensus.cc:2804] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:34.425871 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:34.496927 29183 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:34.497081 29234 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:34.497349 29183 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:34.497527 29234 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:34.498087 29234 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:34.498517 28868 raft_consensus.cc:3055] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:10:34.498906 28868 raft_consensus.cc:740] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:34.499691 28868 consensus_queue.cc:260] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:34.432955 22955 raft_consensus.cc:1275] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:10:34.434348 23027 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 16.
I20260709 13:10:34.502166 29086 raft_consensus.cc:2749] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:10:34.502243 29141 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:34.503352 23617 raft_consensus.cc:1240] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 15. Current term is 16. Ops: []
I20260709 13:10:34.503247 28868 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:34.394133 29155 raft_consensus.cc:2749] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:10:34.478873 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 111426us
I20260709 13:10:34.505750 29201 consensus_queue.cc:1059] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:10:34.469080 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 81581us
I20260709 13:10:34.496502 22775 catalog_manager.cc:5697] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 11 to 14, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 14 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:34.394524 29135 consensus_queue.cc:1048] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.083s
I20260709 13:10:34.434669 29176 raft_consensus.cc:697] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 17 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:34.394300 29204 consensus_queue.cc:1048] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.084s
W20260709 13:10:34.522413 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 175625us
W20260709 13:10:34.523188 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 216750us
I20260709 13:10:34.535056 22950 raft_consensus.cc:1240] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16. Ops: []
W20260709 13:10:34.536566 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 148880us
I20260709 13:10:34.507006 28898 raft_consensus.cc:2804] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:10:34.506215 29141 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:34.544176 22879 raft_consensus.cc:1275] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:10:34.506278 29086 consensus_queue.cc:1048] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 13:10:34.469938 28945 consensus_queue.cc:1048] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.069s
I20260709 13:10:34.544128 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:34.552651 22876 raft_consensus.cc:1240] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 13:10:34.556202 29257 raft_consensus.cc:2804] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:10:34.484046 29022 consensus_queue.cc:1059] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.062s
I20260709 13:10:34.557207 29145 consensus_queue.cc:1048] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:10:34.557370 29022 consensus_queue.cc:1059] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:10:34.558247 29224 consensus_queue.cc:237] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:34.558676 29145 raft_consensus.cc:3055] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:10:34.559137 29145 raft_consensus.cc:740] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
W20260709 13:10:34.559563 22773 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33466 (request call id 293) took 21534 ms (client timeout 15000 ms). Trace:
I20260709 13:10:34.560056 29145 consensus_queue.cc:260] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:34.551736 29247 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:34.561335 29247 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:34.562077 29134 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:34.546233 29139 raft_consensus.cc:2804] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:10:34.563078 29139 raft_consensus.cc:697] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 19 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:34.563992 29139 consensus_queue.cc:237] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:34.559949 22773 rpcz_store.cc:269] 0709 13:10:13.239553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:13.239754 (+   201us) service_pool.cc:224] Handling call
0709 13:10:13.839175 (+599421us) spinlock_profiling.cc:243] Waited 30.7 ms on lock 0x7b54002b6e54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4f090 00007f6135dedf3a 00007f613bba41cc 00007f613bca237e 00007f613a1632a4 00007f613a163232 00007f613a163161 00007f613a1630dc 00007f613a162392 00007f613412048c 00007f613411f8c6
0709 13:10:34.559326 (+20720151us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:10:28.231089 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:10:28.392876 (+161787us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:10:28.397485 (+  4609us) write_op.cc:435] Acquired schema lock
0709 13:10:28.401297 (+  3812us) tablet.cc:662] Decoding operations
0709 13:10:28.543199 (+141902us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:10:28.543266 (+    67us) write_op.cc:641] Partition lock acquired for write op
0709 13:10:28.543297 (+    31us) tablet.cc:685] Acquiring locks for 30 operations
0709 13:10:28.574376 (+ 31079us) tablet.cc:701] Row locks acquired
0709 13:10:28.574389 (+    13us) write_op.cc:260] PREPARE: finished
0709 13:10:28.574507 (+   118us) write_op.cc:270] Start()
0709 13:10:28.574593 (+    86us) write_op.cc:276] Timestamp: P: 1783602628574471 usec, L: 0
0709 13:10:28.574604 (+    11us) op_driver.cc:348] REPLICATION: starting
0709 13:10:28.575426 (+   822us) log.cc:844] Serialized 11853 byte log entry
0709 13:10:28.580882 (+  5456us) op_driver.cc:464] REPLICATION: finished
0709 13:10:28.940572 (+359690us) write_op.cc:301] APPLY: starting
0709 13:10:28.940774 (+   202us) tablet.cc:1367] starting BulkCheckPresence
0709 13:10:29.292619 (+351845us) tablet.cc:1370] finished BulkCheckPresence
0709 13:10:29.296690 (+  4071us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:10:29.802364 (+505674us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:10:29.802820 (+   456us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=60,key_file_lookups=60,delta_file_lookups=60,mrs_lookups=0
0709 13:10:29.802885 (+    65us) write_op.cc:312] APPLY: finished
0709 13:10:29.870687 (+ 67802us) log.cc:844] Serialized 260 byte log entry
0709 13:10:30.079996 (+209309us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:30.103655 (+ 23659us) write_op.cc:454] Released schema lock
0709 13:10:30.103751 (+    96us) write_op.cc:341] FINISH: Updating metrics
0709 13:10:30.328406 (+224655us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:30.358569 (+ 30163us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":194147,"reported_tablets":48,"spinlock_wait_cycles":70662912,"child_traces":[["op",{"apply.queue_time_us":340261,"apply.run_cpu_time_us":62734,"apply.run_wall_time_us":1332932,"cfile_cache_hit":122,"cfile_cache_hit_bytes":129587,"num_ops":30,"prepare.queue_time_us":50101,"prepare.run_cpu_time_us":10370,"prepare.run_wall_time_us":377809,"raft.queue_time_us":843,"raft.run_cpu_time_us":18221,"raft.run_wall_time_us":313411,"replication_time_us":6162,"thread_start_us":408857,"threads_started":5,"wal-append.queue_time_us":201431,"wal-append.run_cpu_time_us":5743,"wal-append.run_wall_time_us":2265477}]]}
I20260709 13:10:34.565460 22775 catalog_manager.cc:5697] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 11 to 14. New cstate: current_term: 14 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:34.566964 22775 catalog_manager.cc:5697] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 8 to 16, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 16 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:34.568962 22992 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:34.536126 29237 raft_consensus.cc:2749] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:10:34.570667 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 203914us
I20260709 13:10:34.523725 29255 raft_consensus.cc:2764] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
W20260709 13:10:34.572554 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 175177us
I20260709 13:10:34.579489 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:34.541766 29193 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:34.580950 23588 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:34.568428 22775 catalog_manager.cc:5697] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 11 to 14, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 14 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:34.577507 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:34.582824 22880 raft_consensus.cc:2393] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 18: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:34.581490 22879 raft_consensus.cc:2393] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 20: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
W20260709 13:10:34.573154 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 92809.8us
W20260709 13:10:34.580951 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 102047us
I20260709 13:10:34.584486 22919 leader_election.cc:400] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 election: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 20. Current term is 21.
I20260709 13:10:34.585646 22919 leader_election.cc:403] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 13:10:34.548295 29076 raft_consensus.cc:2749] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:34.578908 23590 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:34.583222 22775 catalog_manager.cc:5697] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 10 to 13, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 13 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:34.606452 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:34.607496 22950 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 20.
I20260709 13:10:34.607642 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:34.591761 22992 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:34.590703 23588 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:34.611387 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:34.578276 29127 raft_consensus.cc:2804] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:10:34.610769 23588 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:34.616783 22880 raft_consensus.cc:1275] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 13:10:34.617210 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:34.578724 29257 raft_consensus.cc:697] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 18 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:34.555785 29259 consensus_queue.cc:1059] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 13:10:34.625435 22992 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:34.625964 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:34.626314 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:34.634492 22879 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 19.
I20260709 13:10:34.589107 29141 raft_consensus.cc:2749] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:10:34.631168 23590 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:34.611711 29134 raft_consensus.cc:2804] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 13:10:34.638340 29251 log.cc:927] Time spent T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 13:10:34.638652 28813 raft_consensus.cc:3055] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:10:34.640172 28827 raft_consensus.cc:2804] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:10:34.641219 29261 raft_consensus.cc:2749] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:10:34.641907 29076 raft_consensus.cc:2749] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:34.648051 22955 raft_consensus.cc:1275] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:10:34.575179 28898 raft_consensus.cc:697] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:34.608374 29201 raft_consensus.cc:2749] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 20. Current term is 21.
I20260709 13:10:34.612797 29242 raft_consensus.cc:2804] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:34.585583 29245 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:34.620097 29237 raft_consensus.cc:2749] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:10:34.667301 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 176546us
I20260709 13:10:34.655791 29255 raft_consensus.cc:2804] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 13:10:34.667943 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 124780us
I20260709 13:10:34.589184 29193 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:34.648788 29127 raft_consensus.cc:697] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 18 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:34.648161 28821 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:34.665313 28369 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:34.663354 29253 consensus_queue.cc:1048] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.041s
I20260709 13:10:34.684458 28827 raft_consensus.cc:697] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 17 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:34.697304 28827 consensus_queue.cc:237] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:34.684914 23624 raft_consensus.cc:1275] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 13:10:34.701973 22949 raft_consensus.cc:2468] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 19.
I20260709 13:10:34.619884 29082 consensus_queue.cc:1059] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 13:10:34.686275 23623 raft_consensus.cc:1275] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:10:34.687168 22918 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
W20260709 13:10:34.652070 29252 log.cc:927] Time spent T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 13:10:34.706362 22918 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:34.689641 22955 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:39742
I20260709 13:10:34.662947 28813 raft_consensus.cc:740] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:34.690827 22843 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:34.707516 22955 raft_consensus.cc:606] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:34.708110 22955 raft_consensus.cc:614] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:10:34.615649 29176 consensus_queue.cc:237] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:34.730217 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 87134us
I20260709 13:10:34.683686 29242 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:34.731173 23588 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:34.718585 22843 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:34.734710 22918 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:34.687805 29170 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:34.738770 22843 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:34.738996 22918 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:34.791061 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 52365.2us
W20260709 13:10:34.791465 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 52564.3us
I20260709 13:10:34.792369 22918 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:34.793262 29160 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:34.793648 22918 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:34.794437 29264 raft_consensus.cc:2749] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:10:34.795069 29264 raft_consensus.cc:2804] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:34.795390 29264 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:34.795814 29264 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:34.796875 29135 raft_consensus.cc:2804] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:34.797367 29135 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:34.653053 29246 consensus_queue.cc:1059] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.064s
I20260709 13:10:34.797766 29135 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:34.798307 29246 raft_consensus.cc:3055] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:10:34.798779 29246 raft_consensus.cc:740] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:34.799129 22918 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:34.686090 29247 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:34.744041 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:34.801003 23619 raft_consensus.cc:2393] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 17: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:34.737591 29138 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:34.737773 29082 raft_consensus.cc:2749] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:10:34.767928 29260 consensus_queue.cc:1048] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 13:10:34.771405 28369 raft_consensus.cc:2749] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:10:34.802359 29082 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:34.802644 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:34.802984 29260 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:34.802852 29082 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:34.803507 29260 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:34.771824 23588 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:34.772048 29262 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:34.679762 29151 consensus_queue.cc:1048] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:10:34.773157 29086 raft_consensus.cc:2804] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:10:34.805292 29265 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:10:34.805538 23588 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:34.805723 29265 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:34.806685 29138 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:34.806550 29262 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:34.772938 29257 consensus_queue.cc:237] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:34.774490 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:34.775285 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:34.690233 29255 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:34.782039 29233 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:34.783077 29127 consensus_queue.cc:237] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:34.784789 29201 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:34.786595 28813 consensus_queue.cc:260] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:34.788151 29254 log.cc:927] Time spent T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.175s	user 0.000s	sys 0.000s
I20260709 13:10:34.789078 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:34.735036 28821 raft_consensus.cc:2804] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:34.740656 29242 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:34.692260 28898 consensus_queue.cc:237] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:34.848812 29266 raft_consensus.cc:2749] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:10:34.849704 29262 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:34.850790 29193 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:34.852777 29255 raft_consensus.cc:3060] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:34.705229 29245 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:34.854385 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:34.900434 23029 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 18.
I20260709 13:10:34.855304 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:34.855494 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:34.856114 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:34.902216 23022 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:34.729867 29259 raft_consensus.cc:2749] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:10:34.738099 29092 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:34.852890 29201 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:10:34.842116 29251 log.cc:927] Time spent T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 13:10:34.799839 29246 consensus_queue.cc:260] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:34.917696 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:34.812063 22918 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:34.918368 23619 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 18.
I20260709 13:10:34.816588 29265 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:34.851050 29086 raft_consensus.cc:697] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:34.845920 28991 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:34.780365 29183 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:34.904834 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:34.908401 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 401820us
I20260709 13:10:34.864275 28821 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:10:34.912556 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:34.938597 22843 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:34.915956 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:10:34.918874 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 106910us
I20260709 13:10:34.876571 29260 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:34.923560 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:34.942768 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:34.924655 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:34.932579 23588 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:34.933573 22844 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:34.935587 22920 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:34.940096 22918 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:34.923234 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:34.944878 22950 raft_consensus.cc:2410] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 17 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 13:10:34.946127 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:34.946779 22954 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 17.
I20260709 13:10:34.911796 29193 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:34.940480 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 280865us
I20260709 13:10:34.918064 29201 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:34.951283 22994 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:34.947096 22918 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:34.929489 29253 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:34.951624 23588 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:34.934620 29233 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:34.954136 21756 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:10:34.951362 29224 consensus_queue.cc:1048] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 13:10:34.935604 29148 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:34.942247 28813 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:34.975762 22920 leader_election.cc:400] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 election: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 20. Current term is 21.
I20260709 13:10:34.950237 29259 consensus_queue.cc:1048] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 13:10:34.941855 28821 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:34.961759 29245 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:34.962729 29092 consensus_queue.cc:1048] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:10:34.963376 29223 raft_consensus.cc:2804] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:34.964643 29280 consensus_queue.cc:1048] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:10:34.966404 29183 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:34.966629 22955 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 19.
I20260709 13:10:34.859529 29094 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:34.920100 29082 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:34.969050 22992 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:34.969517 23588 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:34.972002 21756 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:10:34.959851 29088 raft_consensus.cc:3055] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:10:34.974403 23022 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 16.
I20260709 13:10:34.976621 22994 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:34.948143 29134 raft_consensus.cc:697] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:34.953701 29260 raft_consensus.cc:2804] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:34.992890 29279 raft_consensus.cc:2804] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:10:34.974061 29243 raft_consensus.cc:2749] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:34.976943 29261 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:34.966367 29135 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:35.037541 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 61001us
I20260709 13:10:35.010932 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:35.072026 22953 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 17.
I20260709 13:10:34.994557 29223 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:34.995131 29277 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:34.985726 29160 raft_consensus.cc:2749] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:35.069356 28991 raft_consensus.cc:2749] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:10:35.044770 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:35.063853 29260 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:35.000357 29245 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:35.077261 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:35.078962 29242 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:35.036768 29279 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:35.080431 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:35.109153 22871 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 17.
I20260709 13:10:35.081319 23026 raft_consensus.cc:1275] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
W20260709 13:10:35.085758 29234 consensus_meta.cc:287] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.587s	user 0.011s	sys 0.000s
I20260709 13:10:35.086199 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:35.112313 23028 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 16.
W20260709 13:10:35.101310 29276 log.cc:927] Time spent T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:10:35.087646 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:35.091696 23617 raft_consensus.cc:1240] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:35.091892 29267 raft_consensus.cc:3055] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:10:35.080155 29265 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:35.093091 29255 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:35.001621 29262 raft_consensus.cc:2804] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:35.098591 22878 raft_consensus.cc:1240] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 18. Current term is 19. Ops: []
W20260709 13:10:35.099400 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 165931us
I20260709 13:10:35.107549 29246 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:10:35.181437 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 238868us
I20260709 13:10:35.074878 23588 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:10:35.182435 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 107622us
I20260709 13:10:35.182483 22843 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:35.184092 23588 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:35.005436 29221 raft_consensus.cc:2804] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:35.005352 29088 raft_consensus.cc:740] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:35.065030 29148 raft_consensus.cc:2749] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:10:35.106707 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 130013us
I20260709 13:10:35.037637 29139 raft_consensus.cc:2804] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:34.997376 29278 raft_consensus.cc:2749] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:10:35.190043 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:35.087342 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:34.973233 29253 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:35.099054 29223 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:35.107908 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:35.115450 22953 raft_consensus.cc:1240] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 18. Current term is 19. Ops: []
I20260709 13:10:35.171536 23029 raft_consensus.cc:2393] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 20: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
W20260709 13:10:35.173408 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 55792.3us
I20260709 13:10:35.108950 29279 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:34.967764 29193 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:35.176887 23590 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:35.213003 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:35.213814 22879 raft_consensus.cc:2393] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 20: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:35.179754 29262 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:35.110054 29260 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:35.204291 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:35.189759 29082 raft_consensus.cc:2749] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:35.205394 23028 raft_consensus.cc:2393] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 16: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:35.192790 29139 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:35.116092 29267 raft_consensus.cc:740] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:35.215808 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:35.225014 22873 raft_consensus.cc:2393] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 19: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:35.179622 22872 raft_consensus.cc:1275] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:10:35.225728 29094 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:35.226498 29139 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:35.216050 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:35.231202 29261 raft_consensus.cc:2749] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:35.233922 29193 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:35.238298 29139 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:35.238992 28868 consensus_meta.cc:287] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.702s	user 0.009s	sys 0.002s
I20260709 13:10:35.240243 29233 raft_consensus.cc:2749] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:35.220332 22871 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 17.
I20260709 13:10:35.187538 29221 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:35.231319 29135 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:35.099392 29086 consensus_queue.cc:237] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:35.223556 29262 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:35.210007 29281 raft_consensus.cc:2804] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:35.242623 29255 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:35.242689 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:35.172919 29245 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:35.297596 29245 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:35.298105 29253 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:35.267766 29284 consensus_queue.cc:1059] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.091s
I20260709 13:10:35.213140 29266 consensus_queue.cc:1048] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
I20260709 13:10:35.305616 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:35.244911 29145 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:35.245396 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:35.307829 22875 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 16.
I20260709 13:10:35.114991 29234 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:35.210403 29134 consensus_queue.cc:237] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:35.249262 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:35.250922 29265 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:35.106335 29277 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:35.261011 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:35.219556 29141 raft_consensus.cc:2749] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:35.261585 29139 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:35.252554 29064 consensus_queue.cc:1059] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
W20260709 13:10:35.225126 29282 log.cc:927] Time spent T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 13:10:35.263069 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:35.262704 29267 consensus_queue.cc:260] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:35.325490 22955 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:35.263773 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:35.205664 28827 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.046s	sys 0.067s
I20260709 13:10:35.329180 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:35.329813 23026 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 14.
I20260709 13:10:35.212725 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:35.212533 22843 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:10:35.331584 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 119150us
I20260709 13:10:35.264592 29138 raft_consensus.cc:2804] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 13:10:35.288436 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 73556.7us
I20260709 13:10:35.333823 22843 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:35.337015 22947 raft_consensus.cc:1275] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 13:10:35.338297 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:35.320528 29201 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:35.240269 29170 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:35.323683 29260 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:35.275610 29221 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:35.223274 29092 consensus_queue.cc:1059] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.070s
I20260709 13:10:35.294548 29281 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:10:35.325763 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 111580us
I20260709 13:10:35.326386 22953 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 19.
I20260709 13:10:35.288828 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:35.363749 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:35.309002 22920 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:35.325253 29022 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:35.333027 28827 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.047s	sys 0.067s
I20260709 13:10:35.345738 29253 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:35.360250 22843 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:35.346354 29267 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:35.338819 29280 raft_consensus.cc:2804] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:35.375562 29280 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:35.376012 29280 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:35.377257 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:35.371176 29022 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:35.383073 29022 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:35.384297 29201 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:35.385227 29266 raft_consensus.cc:2804] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:35.385636 29266 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:35.385990 29266 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:35.389880 29260 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:35.394155 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:35.397104 29223 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:35.397521 29204 raft_consensus.cc:3055] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:10:35.397746 22918 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:35.397974 29204 raft_consensus.cc:740] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:35.398036 29253 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:35.398969 22918 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:35.399292 29223 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:35.399034 29204 consensus_queue.cc:260] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:35.399915 29223 consensus_queue.cc:1048] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:10:35.400115 22918 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:35.367904 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:35.422894 29223 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:35.423394 29223 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:35.368621 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:35.359088 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:35.369468 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 60582.3us
I20260709 13:10:35.361955 29265 consensus_queue.cc:1048] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.014s
I20260709 13:10:35.400285 29204 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:35.480611 22874 raft_consensus.cc:1275] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 13:10:35.400908 22950 raft_consensus.cc:1240] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:10:35.400996 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:35.401831 22877 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18.
I20260709 13:10:35.483891 29176 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:35.484683 29176 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:35.402009 29201 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:35.499428 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:35.500068 22878 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 16.
I20260709 13:10:35.401849 29092 raft_consensus.cc:2749] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:10:35.400146 29253 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:35.403923 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:35.501498 22871 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:35.404515 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:35.409598 23021 raft_consensus.cc:1275] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 13:10:35.503069 23021 pending_rounds.cc:85] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75: Aborting all ops after (but not including) 8
I20260709 13:10:35.503391 23021 pending_rounds.cc:107] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:10:35.503676 23021 raft_consensus.cc:2889] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:35.365994 29281 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:35.413335 22880 raft_consensus.cc:1240] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 17. Current term is 18. Ops: []
I20260709 13:10:35.415014 29234 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:35.293656 29088 consensus_queue.cc:260] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:35.347668 29138 raft_consensus.cc:697] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:35.426725 22875 raft_consensus.cc:1275] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:10:35.534749 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:35.362124 29170 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:35.427788 22947 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 20.
I20260709 13:10:35.429231 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:35.539057 23024 raft_consensus.cc:2393] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 16: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:35.431686 23027 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18.
I20260709 13:10:35.432292 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:35.540544 23028 raft_consensus.cc:2393] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 19: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
W20260709 13:10:35.433184 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 73003.8us
I20260709 13:10:35.368378 22955 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 18.
I20260709 13:10:35.478034 22872 raft_consensus.cc:2393] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 20: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:35.541637 22843 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:35.542371 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:35.542623 22843 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:35.543368 22872 raft_consensus.cc:2393] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 16: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:35.341915 29135 raft_consensus.cc:2749] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:10:35.423448 28813 raft_consensus.cc:2749] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:10:35.488489 22874 raft_consensus.cc:1275] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:10:35.488986 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:35.552757 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:35.490725 23025 raft_consensus.cc:1275] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
W20260709 13:10:35.496866 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 61222.7us
I20260709 13:10:35.506564 23619 raft_consensus.cc:1240] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 12. Current term is 13. Ops: []
I20260709 13:10:35.434732 29145 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:35.545998 22994 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:35.547928 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:35.547276 22843 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:35.553741 23025 pending_rounds.cc:85] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75: Aborting all ops after (but not including) 8
I20260709 13:10:35.557952 23025 pending_rounds.cc:107] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:10:35.558235 23025 raft_consensus.cc:2889] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:35.505093 28827 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:35.536055 29285 log.cc:927] Time spent T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.170s	user 0.000s	sys 0.001s
I20260709 13:10:35.555773 23020 raft_consensus.cc:2468] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 16.
I20260709 13:10:35.559440 22878 raft_consensus.cc:2410] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
W20260709 13:10:35.549192 29264 consensus_meta.cc:287] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.753s	user 0.013s	sys 0.000s
I20260709 13:10:35.561062 23619 raft_consensus.cc:1275] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:10:35.560976 22843 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:35.563258 22875 raft_consensus.cc:2393] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 18: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:35.568285 22843 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:35.569375 22843 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:35.570556 22843 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:35.548774 28991 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:35.549391 29267 raft_consensus.cc:2749] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:35.499073 29286 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:35.505288 29237 consensus_queue.cc:1048] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.055s
I20260709 13:10:35.566687 28868 raft_consensus.cc:2804] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:35.595940 28868 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:35.596285 28868 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:35.571761 28801 consensus_queue.cc:1048] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 13:10:35.551941 29022 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:10:35.549008 29176 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:10:35.558568 28821 consensus_meta.cc:287] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.561s	user 0.008s	sys 0.001s
I20260709 13:10:35.577797 23619 raft_consensus.cc:1240] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 17. Current term is 18. Ops: []
I20260709 13:10:35.583648 22871 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 18.
W20260709 13:10:35.586671 29293 log.cc:927] Time spent T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
W20260709 13:10:35.587436 29295 log.cc:927] Time spent T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
W20260709 13:10:35.588013 29294 log.cc:927] Time spent T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.221s	user 0.003s	sys 0.000s
I20260709 13:10:35.571343 29264 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:35.565837 28827 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:35.593799 29280 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:35.593540 29286 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:35.595232 29246 raft_consensus.cc:2804] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:35.622071 22951 raft_consensus.cc:1240] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 14. Current term is 15. Ops: []
I20260709 13:10:35.607393 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:35.571178 29094 raft_consensus.cc:2804] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:10:35.618387 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 47898.6us
I20260709 13:10:35.572626 29135 raft_consensus.cc:2749] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:10:35.576007 29082 raft_consensus.cc:2804] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:10:35.559628 29260 consensus_queue.cc:1048] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 13:10:35.615396 23620 raft_consensus.cc:1275] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:10:35.585825 29145 raft_consensus.cc:2749] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:35.620112 29264 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:35.620867 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:35.628539 22843 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:35.585863 28991 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:35.608652 28821 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:35.612798 29022 raft_consensus.cc:697] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 16 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:35.636824 22948 raft_consensus.cc:1240] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 13:10:35.637532 28821 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:35.637351 29022 consensus_queue.cc:237] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:35.639511 29279 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:35.630272 22873 raft_consensus.cc:1240] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 14. Current term is 15. Ops: []
W20260709 13:10:35.630364 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 94171.5us
I20260709 13:10:35.631382 22872 raft_consensus.cc:1275] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:10:35.641568 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:35.625691 29246 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:35.648367 22843 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:35.649636 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:35.628082 29094 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:35.612643 29291 consensus_queue.cc:1059] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 13:10:35.622746 29141 consensus_queue.cc:1048] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.028s
I20260709 13:10:35.659749 23022 raft_consensus.cc:2393] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 20: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:35.640321 29082 raft_consensus.cc:697] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 20 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:35.647038 29061 consensus_queue.cc:1048] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.008s
I20260709 13:10:35.664819 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:35.666383 29223 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:35.672796 22955 raft_consensus.cc:1240] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:10:35.655107 29260 raft_consensus.cc:2749] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:35.667788 29286 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:35.669230 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:35.669723 29280 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:35.650992 29246 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:35.670548 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:35.671746 22876 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18.
I20260709 13:10:35.666556 28991 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:35.650797 29176 consensus_queue.cc:1048] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.026s
I20260709 13:10:35.634392 29267 consensus_queue.cc:1059] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 13:10:35.656733 29094 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:35.676301 23620 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 18.
I20260709 13:10:35.677557 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:35.677873 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:35.685794 22955 raft_consensus.cc:2393] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 18: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:35.674026 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:35.686592 23029 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 18.
I20260709 13:10:35.681150 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:35.687849 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:35.623167 29138 consensus_queue.cc:237] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:35.682789 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:35.683369 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:35.674585 29312 raft_consensus.cc:2749] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 13:10:35.690457 23025 raft_consensus.cc:2393] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 18: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:35.691074 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:35.658680 29304 consensus_queue.cc:1048] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 13:10:35.647727 29221 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:35.676430 28801 raft_consensus.cc:993] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7: : Instructing follower 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:10:35.686321 22920 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:35.694871 22844 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:35.703500 22947 raft_consensus.cc:1240] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 LEADER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 19. Current term is 20. Ops: [19.10-19.10]
I20260709 13:10:35.704967 22843 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:35.707734 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:35.708627 22843 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:35.714303 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:35.716724 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:35.721117 22952 raft_consensus.cc:1240] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:10:35.693369 29280 raft_consensus.cc:3055] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:10:35.706350 29260 raft_consensus.cc:2804] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:35.734015 29260 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:35.734402 29260 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:35.706568 29223 raft_consensus.cc:1081] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 LEADER]: Signalling peer 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:10:35.739180 29284 consensus_queue.cc:1059] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 13:10:35.717774 29267 raft_consensus.cc:2749] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:10:35.732450 29280 raft_consensus.cc:740] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:35.750094 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:35.717571 29303 raft_consensus.cc:2804] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:10:35.685900 29266 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:35.750900 22947 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 18.
I20260709 13:10:35.729672 29286 raft_consensus.cc:2804] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:10:35.758852 29201 raft_consensus.cc:2749] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:10:35.762486 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:35.748137 29260 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:35.750680 29082 consensus_queue.cc:237] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:35.771176 28868 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:35.761497 29286 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:10:35.780119 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:35.783263 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 67734.3us
I20260709 13:10:35.754015 29303 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:35.789963 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:35.791324 22843 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:35.807539 22843 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:35.810412 22843 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:35.783042 29237 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:35.794013 29286 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:35.789357 29176 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:35.803200 22871 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:35360
I20260709 13:10:35.809927 29303 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:35.827643 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:35.787806 29280 consensus_queue.cc:260] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:35.829355 29267 raft_consensus.cc:2804] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:35.833284 29267 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:35.833621 29267 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:35.836009 29278 raft_consensus.cc:3055] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:10:35.836382 29278 raft_consensus.cc:740] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:35.842954 29278 consensus_queue.cc:260] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:35.844200 29278 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:35.794235 29262 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:35.844762 29221 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:35.844880 29240 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:35.845633 29240 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:35.814850 29279 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:35.814132 29312 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:10:35.845572 22872 raft_consensus.cc:2393] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 19: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:35.818512 29237 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:35.811898 29094 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:35.849049 29240 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:35.841708 29266 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:35.851472 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:35.818660 29145 raft_consensus.cc:2749] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:10:35.856467 29237 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:35.856832 29281 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:35.865008 22874 raft_consensus.cc:2393] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 17: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:35.869470 22871 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:35.850124 28868 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:35.870110 22871 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:35.823181 29176 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:35.891202 22871 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:35.907174 23624 raft_consensus.cc:1240] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEADER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 15. Current term is 16. Ops: []
I20260709 13:10:35.902068 29262 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:35.909996 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 99439.2us
I20260709 13:10:35.911202 29176 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:35.913533 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:35.914299 22954 raft_consensus.cc:2393] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 17: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:35.893524 29094 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:35.940676 22878 raft_consensus.cc:1240] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 15. Current term is 16. Ops: []
I20260709 13:10:35.941682 29281 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:35.943053 29277 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:35.944272 29262 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:35.944855 29262 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:35.929325 29286 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:35.948611 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:35.949246 23022 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:35.950147 29303 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:35.951758 29077 raft_consensus.cc:3055] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:10:35.952163 29077 raft_consensus.cc:740] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:35.953018 29077 consensus_queue.cc:260] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:35.953833 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:35.954504 23028 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:10:35.953040 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:35.957672 22878 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:10:35.949236 29262 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:35.962599 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:35.963874 22954 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 20.
I20260709 13:10:35.967561 22878 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 20.
I20260709 13:10:35.953171 29267 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:35.970479 23022 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 19.
I20260709 13:10:35.963753 29277 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:35.971794 29277 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:35.971975 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:35.990053 23028 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 19.
I20260709 13:10:35.985776 22951 raft_consensus.cc:1275] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:10:35.989231 23588 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:35.985323 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:35.974033 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:36.012281 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:35.987174 22918 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:36.013044 22952 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:36.013168 23022 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:10:35.977659 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:36.014760 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:36.015297 22954 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 17.
I20260709 13:10:35.978274 22992 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:36.017365 23588 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:35.966499 22843 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:10:36.019191 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 52845.7us
I20260709 13:10:36.021984 22918 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:10:36.022239 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 44046.3us
I20260709 13:10:36.022964 22918 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:36.023885 29094 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:10:36.024318 29094 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:36.024684 29094 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:36.027194 29267 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:36.022042 29331 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:36.028405 29331 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:36.029831 29240 raft_consensus.cc:2804] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:10:36.030345 29240 raft_consensus.cc:697] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 18 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:36.030556 29331 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:36.031175 29240 consensus_queue.cc:237] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:36.034363 23028 raft_consensus.cc:1240] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:10:36.033149 29138 consensus_queue.cc:1048] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:10:35.970291 29262 consensus_queue.cc:1059] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 13:10:36.036898 29223 raft_consensus.cc:2749] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:10:36.037277 29262 raft_consensus.cc:3055] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:10:36.037708 28801 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:10:36.038120 29237 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:10:36.038579 29159 raft_consensus.cc:2749] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:10:36.038771 29237 raft_consensus.cc:697] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:36.039363 29242 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:36.039826 29237 consensus_queue.cc:237] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:36.042182 28869 consensus_queue.cc:1059] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.011s
I20260709 13:10:36.043082 28868 raft_consensus.cc:3055] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:10:36.041628 29234 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:36.044040 29234 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:36.046293 29234 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:36.046622 29094 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:36.046954 29246 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:36.047742 29260 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:36.021247 29281 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:36.043429 28868 raft_consensus.cc:740] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:35.974192 29277 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:35.981164 29150 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:36.039371 29201 consensus_queue.cc:1048] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 13:10:36.057307 29141 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:36.049369 29246 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:36.016294 28945 consensus_queue.cc:1048] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.069s
I20260709 13:10:36.053364 22951 raft_consensus.cc:1275] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:10:36.053682 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:36.056465 23588 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:36.060546 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:36.060561 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:35.981887 29266 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:36.063674 23022 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 17.
I20260709 13:10:36.038324 28801 raft_consensus.cc:697] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:36.077526 28801 consensus_queue.cc:237] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:36.064851 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:36.075796 23588 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:35.981941 29127 consensus_queue.cc:1059] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.082s
I20260709 13:10:36.071177 29150 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:35.982134 29176 consensus_queue.cc:1048] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.082s
I20260709 13:10:36.073944 22955 raft_consensus.cc:2393] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 17: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:36.085980 29265 raft_consensus.cc:2749] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:36.075438 22954 raft_consensus.cc:1275] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:10:36.048620 29094 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:36.081239 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:36.089246 22946 raft_consensus.cc:2410] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:10:36.062489 29303 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:36.083889 29281 raft_consensus.cc:2804] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:36.060387 29331 consensus_queue.cc:1059] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.016s
I20260709 13:10:36.084656 23588 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:36.093180 23588 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:36.085659 29127 raft_consensus.cc:3055] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:10:36.071005 22952 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 18.
I20260709 13:10:36.056493 29262 raft_consensus.cc:740] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:36.097892 23588 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:36.072494 29141 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:36.106452 22952 raft_consensus.cc:1275] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:10:36.091822 29281 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:36.095520 29127 raft_consensus.cc:740] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:36.091207 29150 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:36.099013 29134 raft_consensus.cc:2749] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 13:10:36.111157 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:36.111394 29281 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:36.111750 22954 raft_consensus.cc:3060] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:36.098987 29261 raft_consensus.cc:2804] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:36.112757 29261 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:36.113198 29261 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:36.111407 29134 raft_consensus.cc:2804] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:10:36.101311 29265 consensus_queue.cc:1048] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.012s
I20260709 13:10:36.113917 29134 raft_consensus.cc:697] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 18 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:36.114815 29134 consensus_queue.cc:237] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:36.102082 29240 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:36.116196 29127 consensus_queue.cc:260] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:36.116976 29240 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:36.117558 29127 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:36.119006 29240 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:36.087684 28868 consensus_queue.cc:260] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:36.120746 28868 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:36.121577 22954 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 20.
I20260709 13:10:36.122541 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:36.124073 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:36.124715 22955 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 20.
I20260709 13:10:36.125891 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:36.126396 29141 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:36.126057 29281 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:36.129199 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:36.131122 22954 raft_consensus.cc:2410] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 13:10:36.133277 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:36.135177 22946 raft_consensus.cc:2393] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 20: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:36.137015 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:36.089329 29266 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:36.143877 29331 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:36.145305 22992 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:36.145572 29139 consensus_queue.cc:1048] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:10:36.145332 29331 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:36.147119 22992 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:36.148430 22992 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:36.149993 29242 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:36.147133 29139 raft_consensus.cc:2749] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:10:36.148478 29281 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:36.162226 22879 raft_consensus.cc:1275] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 10. (term mismatch)
I20260709 13:10:36.162637 22879 pending_rounds.cc:85] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 9
I20260709 13:10:36.162868 22879 pending_rounds.cc:107] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 19.10
I20260709 13:10:36.163180 22879 raft_consensus.cc:2889] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:36.164207 22992 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:36.166709 29331 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:36.124332 29262 consensus_queue.cc:260] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:36.167791 29262 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:36.170508 22992 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:36.161343 29139 raft_consensus.cc:2749] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:36.169827 29176 consensus_queue.cc:1048] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.009s
I20260709 13:10:36.173327 29275 raft_consensus.cc:2804] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:10:36.173440 29247 raft_consensus.cc:2804] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:10:36.181380 29266 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:36.168654 29317 log.cc:927] Time spent T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.413s	user 0.000s	sys 0.002s
I20260709 13:10:36.169449 29170 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:36.186995 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 89214.4us
I20260709 13:10:36.187925 29247 raft_consensus.cc:697] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 20 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:36.189833 29247 consensus_queue.cc:237] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:36.191026 29160 consensus_queue.cc:1175] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "5b346767a89745fa82928a5dac9cc0b2" responder_term: 17 status { last_received { term: 16 index: 13 } last_committed_idx: 13 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 16. Current term is 17. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:10:36.193436 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:36.193936 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:36.194561 22946 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 23.
I20260709 13:10:36.186900 29275 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:36.196293 29275 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:36.208691 22947 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 17.
I20260709 13:10:36.212270 23588 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:36.214785 29141 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:10:36.215204 29141 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:36.215557 29141 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:10:36.210022 29261 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:36.186709 29134 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:36.222682 29261 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:36.222774 29265 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:36.223562 29141 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:36.225886 29141 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 24 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:36.226084 23020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33420 (request call id 2559) took 1149 ms. Trace:
I20260709 13:10:36.226523 23588 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:36.250020 23020 rpcz_store.cc:276] 0709 13:10:35.076407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:35.077012 (+   605us) service_pool.cc:224] Handling call
0709 13:10:36.225931 (+1148919us) spinlock_profiling.cc:243] Waited 381 ms on lock 0x7b5400358254. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:36.226030 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":877233920}
I20260709 13:10:36.252013 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:36.231227 29242 raft_consensus.cc:2749] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:36.234594 29275 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:36.235723 29335 log.cc:927] Time spent T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 13:10:36.240373 22873 raft_consensus.cc:1240] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 18. Current term is 19. Ops: []
I20260709 13:10:36.261368 23624 raft_consensus.cc:1240] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 14. Current term is 15. Ops: []
I20260709 13:10:36.244510 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:36.262351 22878 raft_consensus.cc:2393] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 17: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:36.245898 22874 raft_consensus.cc:1275] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:10:36.223486 29265 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:36.252156 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:36.263406 22947 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:10:36.228088 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 248272us
I20260709 13:10:36.259119 22872 raft_consensus.cc:1240] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:36.259940 22879 raft_consensus.cc:1240] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 13:10:36.263435 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:36.221997 29134 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:36.286900 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:36.262293 29261 raft_consensus.cc:2804] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:10:36.289296 23617 raft_consensus.cc:1240] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 24 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 23. Current term is 24. Ops: []
I20260709 13:10:36.289402 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:36.295044 22874 raft_consensus.cc:3060] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:36.296139 22843 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:36.290771 22947 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 24.
I20260709 13:10:36.293733 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:36.297485 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:36.298336 23618 raft_consensus.cc:2410] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
W20260709 13:10:36.299268 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:36.300818 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:36.301156 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:36.301870 22878 raft_consensus.cc:2393] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 20: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:36.304342 22918 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:36.304565 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:36.304965 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:36.305840 22880 raft_consensus.cc:1240] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 18. Current term is 19. Ops: []
I20260709 13:10:36.306527 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:36.307219 22843 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:36.307888 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:36.298363 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:36.308341 22843 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:36.298906 23588 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:36.311779 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:36.312572 22873 raft_consensus.cc:2393] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 17: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:36.313432 22843 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:36.314150 23619 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 17.
I20260709 13:10:36.278669 29141 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:36.314764 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:36.315922 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:36.317425 23624 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 18.
I20260709 13:10:36.316321 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:36.320758 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:36.320971 22843 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:36.321736 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:36.318455 23024 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 19.
I20260709 13:10:36.322494 23020 raft_consensus.cc:2393] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 17: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:36.325016 22993 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:36.261533 29170 consensus_queue.cc:1059] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 13:10:36.327325 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:36.327590 22843 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:36.261246 29320 consensus_queue.cc:1059] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.053s
I20260709 13:10:36.328753 22993 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:36.313258 29331 raft_consensus.cc:2749] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:10:36.312310 28801 raft_consensus.cc:2804] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:10:36.285589 28991 consensus_queue.cc:1048] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.052s
I20260709 13:10:36.332352 22874 raft_consensus.cc:2468] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 18.
I20260709 13:10:36.294201 29277 consensus_queue.cc:1048] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.033s
W20260709 13:10:36.359665 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:36.315691 29340 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:10:36.314368 28868 raft_consensus.cc:2749] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:36.314168 29082 raft_consensus.cc:2749] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:10:36.380512 23021 raft_consensus.cc:1275] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:10:36.382763 29261 raft_consensus.cc:697] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 17 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:36.367643 28869 raft_consensus.cc:2804] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:10:36.397656 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:36.326932 29260 raft_consensus.cc:2749] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:10:36.332620 29061 consensus_queue.cc:1059] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.057s
I20260709 13:10:36.329543 29242 raft_consensus.cc:2749] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:36.367761 29145 raft_consensus.cc:2804] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:10:36.386971 22843 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:36.323643 29141 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:36.387940 29313 raft_consensus.cc:2749] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:36.384646 28801 raft_consensus.cc:697] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:36.332294 29159 consensus_queue.cc:1059] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.067s
I20260709 13:10:36.385628 29267 consensus_queue.cc:1059] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.021s
I20260709 13:10:36.414165 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:36.415949 23025 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:10:36.419169 29338 log.cc:927] Time spent T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 13:10:36.419369 22874 raft_consensus.cc:1275] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:10:36.408835 29082 raft_consensus.cc:2804] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:10:36.398885 29340 raft_consensus.cc:697] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 24 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:36.421339 29261 consensus_queue.cc:237] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:36.422423 29340 consensus_queue.cc:237] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:36.421437 29082 raft_consensus.cc:697] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 17 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:36.423913 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:36.424470 29082 consensus_queue.cc:237] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:36.428499 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:36.429292 23023 raft_consensus.cc:1240] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 13:10:36.433105 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:36.401525 28869 raft_consensus.cc:697] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 19 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:36.444888 22918 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:36.413213 29145 raft_consensus.cc:697] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 19 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:36.449555 29145 consensus_queue.cc:237] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:36.450398 29286 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:36.454651 22918 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:36.456766 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:36.457214 28869 consensus_queue.cc:237] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:36.458647 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:36.456805 29313 consensus_queue.cc:1059] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 13:10:36.459146 29264 raft_consensus.cc:3055] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:10:36.459443 22950 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
I20260709 13:10:36.459622 29176 raft_consensus.cc:3055] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:10:36.460145 29176 raft_consensus.cc:740] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:36.461042 22948 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 16.
I20260709 13:10:36.460922 29176 consensus_queue.cc:260] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:36.461578 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:36.462067 29176 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:36.462307 22949 raft_consensus.cc:2393] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 20: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:36.463207 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:36.463490 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:36.463790 22952 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 20.
I20260709 13:10:36.464731 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:36.465360 22947 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:36.459554 29264 raft_consensus.cc:740] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:36.466264 22994 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:36.470187 22843 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:36.473841 29277 raft_consensus.cc:2749] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:10:36.475930 29141 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:36.511934 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:36.483309 28868 raft_consensus.cc:2804] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:36.485762 29266 consensus_queue.cc:1048] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:10:36.486366 22843 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:36.517174 29267 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:10:36.517563 29267 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:36.517885 29267 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:36.518608 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:36.488533 28801 consensus_queue.cc:237] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:36.520478 23025 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 20.
I20260709 13:10:36.520959 29280 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:36.489444 29275 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:36.465627 29159 raft_consensus.cc:2749] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:10:36.468149 29201 raft_consensus.cc:2749] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:10:36.462332 29134 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:36.521590 28868 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:36.526024 22953 raft_consensus.cc:1240] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 14. Current term is 15. Ops: []
I20260709 13:10:36.526232 28868 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:36.495390 29264 consensus_queue.cc:260] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:36.529388 29264 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:36.530838 28813 raft_consensus.cc:3055] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:10:36.531240 28813 raft_consensus.cc:740] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:36.483251 28827 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:36.488125 22918 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:36.533360 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 45308.6us
I20260709 13:10:36.534569 22918 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:36.505949 22947 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 21.
I20260709 13:10:36.512815 23029 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
I20260709 13:10:36.515821 23023 raft_consensus.cc:1275] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:10:36.487891 29265 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:36.522114 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:36.538820 22953 raft_consensus.cc:1275] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:10:36.541724 22953 raft_consensus.cc:1275] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:10:36.544307 23029 raft_consensus.cc:1240] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 20. Current term is 21. Ops: []
I20260709 13:10:36.548748 29266 raft_consensus.cc:2749] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:36.552493 29286 consensus_queue.cc:1059] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:10:36.552958 28868 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:36.549706 22843 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:36.554040 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:36.556337 22876 raft_consensus.cc:1275] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 13:10:36.556779 22876 pending_rounds.cc:85] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 8
I20260709 13:10:36.556977 22877 raft_consensus.cc:1275] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:10:36.557018 22876 pending_rounds.cc:107] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 13:10:36.557824 22876 raft_consensus.cc:2889] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:36.561800 23028 raft_consensus.cc:1240] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 13:10:36.548530 29159 consensus_queue.cc:1059] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 13:10:36.564561 29159 raft_consensus.cc:3055] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:10:36.564934 29159 raft_consensus.cc:740] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:36.565907 29159 consensus_queue.cc:260] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:36.567224 29159 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:36.546586 28945 raft_consensus.cc:2749] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:36.568365 29267 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:36.532091 28813 consensus_queue.cc:260] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:36.564955 22877 raft_consensus.cc:1275] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:10:36.574183 22919 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:36.585670 22918 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:36.555807 29286 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:10:36.589689 23028 raft_consensus.cc:1275] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 13:10:36.593128 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:36.583244 28898 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:36.600695 22953 raft_consensus.cc:1275] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 13:10:36.604347 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:36.613941 29331 raft_consensus.cc:3055] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:10:36.614367 29331 raft_consensus.cc:740] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:36.615219 29331 consensus_queue.cc:260] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:36.616039 29077 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:36.616365 29331 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:36.617537 29088 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:36.620858 29094 raft_consensus.cc:3055] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:10:36.622854 22871 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:36.623495 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:36.625022 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:36.627228 23029 raft_consensus.cc:1275] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 13:10:36.627969 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:36.600533 29201 raft_consensus.cc:2804] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:36.629279 22954 raft_consensus.cc:1240] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 18. Current term is 19. Ops: []
W20260709 13:10:36.620452 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 195004us
I20260709 13:10:36.643756 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:36.630421 22953 raft_consensus.cc:1240] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 24 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 23. Current term is 24. Ops: [21.15-23.16]
I20260709 13:10:36.609460 29286 raft_consensus.cc:697] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 21 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:36.632697 22843 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:36.633024 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 304348us
I20260709 13:10:36.603475 28813 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:36.628676 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:36.649554 22949 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:36.649578 22993 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:36.651295 22946 raft_consensus.cc:1275] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:10:36.601061 29284 consensus_queue.cc:1048] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 13:10:36.646955 22992 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:36.610687 29223 raft_consensus.cc:2804] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:10:36.654971 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:36.602697 29280 consensus_queue.cc:1048] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 13:10:36.655937 29135 raft_consensus.cc:2749] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:10:36.669890 29127 raft_consensus.cc:2764] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 13:10:36.626219 29266 consensus_queue.cc:1048] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 13:10:36.670167 29223 raft_consensus.cc:697] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:36.673846 29223 consensus_queue.cc:237] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:36.634346 29094 raft_consensus.cc:740] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:36.675518 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:36.677541 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:36.607686 28898 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:36.681210 23024 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:36.682477 22992 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:36.659795 22920 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:36.641028 29201 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:36.700501 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:36.697908 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:36.729549 23620 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:36.730634 22844 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:36.736732 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 77031.2us
I20260709 13:10:36.737352 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:10:36.741444 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:36.754540 23624 raft_consensus.cc:1240] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 24 LEADER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 23. Current term is 24. Ops: [23.16-23.16]
I20260709 13:10:36.642633 28991 consensus_queue.cc:1048] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.050s
W20260709 13:10:36.756524 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 68121.5us
I20260709 13:10:36.740170 22949 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 21.
I20260709 13:10:36.757879 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:36.759505 23622 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:36.760726 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:36.738184 23619 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 20.
I20260709 13:10:36.738193 22920 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:36.772182 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:36.772907 23027 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:36.769428 29141 consensus_queue.cc:1059] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.013s
I20260709 13:10:36.691823 29127 raft_consensus.cc:2749] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:10:36.778594 22948 raft_consensus.cc:1275] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 13:10:36.737890 29201 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:10:36.834600 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 152227us
I20260709 13:10:36.793180 23024 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 20.
I20260709 13:10:36.794214 23027 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15.
I20260709 13:10:36.795154 23622 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 19.
I20260709 13:10:36.728917 29280 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:36.797257 22920 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:36.681393 29259 consensus_queue.cc:1059] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.100s
W20260709 13:10:36.837244 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 66887.9us
I20260709 13:10:36.840691 28898 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:36.729187 29134 consensus_queue.cc:1059] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.104s
I20260709 13:10:36.778246 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:36.781179 23620 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 19.
I20260709 13:10:36.739676 29303 raft_consensus.cc:2749] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:10:36.854832 22953 raft_consensus.cc:1240] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:10:36.784721 29160 raft_consensus.cc:2749] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:10:36.688611 29265 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:36.778301 28868 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:36.850836 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 53701.3us
I20260709 13:10:36.782711 29082 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:36.782799 28869 raft_consensus.cc:2749] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:10:36.922161 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:36.838601 29340 raft_consensus.cc:3055] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:10:36.926966 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 90982.3us
I20260709 13:10:36.836828 29266 consensus_queue.cc:1048] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.101s
I20260709 13:10:36.681254 29145 consensus_queue.cc:1048] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.195s
I20260709 13:10:36.834872 29349 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:36.931521 29267 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:36.874433 29183 raft_consensus.cc:2804] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:36.776698 29348 consensus_queue.cc:1048] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.151s
I20260709 13:10:36.954866 22992 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:36.975508 22992 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:10:36.777513 29306 log.cc:927] Time spent T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 13:10:36.979720 29320 raft_consensus.cc:3055] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:10:36.980129 29320 raft_consensus.cc:740] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:36.968112 29183 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:36.980933 29183 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:36.980957 29320 consensus_queue.cc:260] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:36.983304 29135 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:36.832486 29150 consensus_queue.cc:1059] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.102s
I20260709 13:10:36.927958 29345 consensus_queue.cc:1048] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
W20260709 13:10:36.937850 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 207315us
I20260709 13:10:36.869212 29280 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:36.781613 29094 consensus_queue.cc:260] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:36.964854 29160 consensus_queue.cc:1059] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 13:10:36.950058 29265 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:36.953437 29340 raft_consensus.cc:740] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:36.978384 29127 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:36.868855 29138 consensus_queue.cc:1048] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.090s
I20260709 13:10:36.993541 22844 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:10:37.037578 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 44147us
I20260709 13:10:37.001508 22992 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:36.856441 29286 consensus_queue.cc:237] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:37.000461 29221 raft_consensus.cc:3055] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:10:36.984058 29135 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:37.020491 28827 consensus_queue.cc:1048] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:10:37.022645 29280 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:37.024211 29368 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:37.026103 29265 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:37.027552 29141 consensus_queue.cc:1059] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:10:37.028589 29340 consensus_queue.cc:260] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:36.953195 29082 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:37.034833 29127 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:37.001137 29267 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:37.052888 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 111656us
I20260709 13:10:37.036686 28945 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:37.047865 29221 raft_consensus.cc:740] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:37.070899 29135 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:37.012892 29253 raft_consensus.cc:2749] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:10:37.074895 23021 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.007s	sys 0.006s
W20260709 13:10:37.183148 23021 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.008s	sys 0.007s
W20260709 13:10:37.074801 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 73415.5us
I20260709 13:10:36.989557 29277 consensus_queue.cc:1048] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.079s
I20260709 13:10:37.039932 29266 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:37.067421 29264 consensus_meta.cc:287] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.517s	user 0.005s	sys 0.005s
I20260709 13:10:36.968295 29349 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:37.155292 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:37.067284 29259 raft_consensus.cc:2749] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:10:37.159288 29088 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:37.186839 23620 raft_consensus.cc:1240] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 12. Current term is 13. Ops: []
I20260709 13:10:37.190614 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:37.191876 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:37.036054 29183 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:37.089584 29247 raft_consensus.cc:2804] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:10:37.040308 29279 consensus_queue.cc:1059] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.295s
I20260709 13:10:37.335534 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:37.068202 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 127249us
I20260709 13:10:37.311257 29265 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:37.293365 29127 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:37.308619 28991 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:37.183812 29313 raft_consensus.cc:2749] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:10:37.350454 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:37.312479 28801 consensus_queue.cc:1048] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.054s
I20260709 13:10:37.309993 29267 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:37.293012 28945 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:37.086829 29368 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:37.336797 29266 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:37.367137 29369 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:37.391199 29240 raft_consensus.cc:3055] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:10:37.403517 29221 consensus_queue.cc:260] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:37.366581 29280 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:37.371299 29247 raft_consensus.cc:697] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 21 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:10:37.409926 29331 consensus_meta.cc:287] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.768s	user 0.011s	sys 0.001s
I20260709 13:10:37.348269 29088 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:37.367056 28991 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:37.366102 29265 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:37.419480 29384 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:37.451422 22871 raft_consensus.cc:1240] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 13:10:37.425434 29240 raft_consensus.cc:740] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:37.434317 29259 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:37.474210 22872 raft_consensus.cc:3060] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:10:37.475464 22843 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.014s	sys 0.019s
W20260709 13:10:37.475908 22843 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.014s	sys 0.019s
I20260709 13:10:37.485062 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:37.486038 23027 raft_consensus.cc:2393] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 16: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:37.488526 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:37.435294 29280 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:37.428315 29369 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:37.500214 23024 raft_consensus.cc:1275] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:10:37.456394 29385 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:37.523237 23026 raft_consensus.cc:1275] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 13:10:37.488425 29364 log.cc:927] Time spent T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.177s	user 0.000s	sys 0.001s
W20260709 13:10:37.515700 29360 log.cc:927] Time spent T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.441s	user 0.000s	sys 0.001s
I20260709 13:10:37.479687 29253 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:37.494565 28827 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:37.559026 29349 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:37.576313 22872 raft_consensus.cc:1275] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:10:37.529105 29159 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:37.538956 29262 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:37.584869 22918 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.006s	sys 0.018s
W20260709 13:10:37.593402 22918 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.008s	sys 0.020s
I20260709 13:10:37.465322 29259 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:37.601583 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:37.602222 22953 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:37.453729 29384 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:37.531411 29246 raft_consensus.cc:3055] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 13:10:37.579830 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 285229us
I20260709 13:10:37.600826 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: true dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:37.619264 22954 raft_consensus.cc:3055] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:10:37.619707 22954 raft_consensus.cc:740] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:37.620576 22954 consensus_queue.cc:260] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:37.621624 22954 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:37.622418 22880 raft_consensus.cc:1240] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 19. Current term is 20. Ops: []
I20260709 13:10:37.623643 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:37.603402 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:37.604384 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:37.624996 22950 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:37.624516 22880 raft_consensus.cc:2393] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 19: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:37.604266 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:37.625887 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:37.627352 22951 raft_consensus.cc:1275] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 13:10:37.554826 29082 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:37.559657 29077 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:37.632000 23023 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 2597) took 1060 ms. Trace:
I20260709 13:10:37.632400 23026 raft_consensus.cc:1275] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:10:37.626694 22955 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 17.
I20260709 13:10:37.582777 28801 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:37.637662 23023 rpcz_store.cc:276] 0709 13:10:36.571595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:36.571957 (+   362us) service_pool.cc:224] Handling call
0709 13:10:36.584503 (+ 12546us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:36.587891 (+  3388us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:36.587947 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:36.592525 (+  4578us) log.cc:844] Serialized 26 byte log entry
0709 13:10:36.863586 (+271061us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:10:36.863614 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:36.863662 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:37.631792 (+768130us) raft_consensus.cc:1685] finished
0709 13:10:37.631822 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:37.631948 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8300,"raft.queue_time_us":3892,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":4279,"thread_start_us":223800,"threads_started":1,"wal-append.queue_time_us":434427}
I20260709 13:10:37.642774 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:37.638420 29370 log.cc:927] Time spent T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.277s	user 0.001s	sys 0.000s
I20260709 13:10:37.523633 29247 consensus_queue.cc:237] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:37.651510 22879 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 17.
I20260709 13:10:37.532214 29240 consensus_queue.cc:260] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:37.633734 29183 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:37.572312 29385 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:37.644644 29082 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:37.570722 29253 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:37.658298 28801 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:37.580245 28827 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:37.619568 29262 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:37.618302 29159 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:37.650305 29388 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:37.651163 29077 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:37.676648 22880 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 19.
W20260709 13:10:37.703495 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:37.763981 22953 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15.
I20260709 13:10:37.765203 22950 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 21.
I20260709 13:10:37.737535 22948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 3297) took 1138 ms. Trace:
I20260709 13:10:37.769162 22948 rpcz_store.cc:276] 0709 13:10:36.598834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:36.598958 (+   124us) service_pool.cc:224] Handling call
0709 13:10:36.599279 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:36.789989 (+190710us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:37.737206 (+947217us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b5400148054. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:37.737456 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":367211904}
I20260709 13:10:37.762174 22920 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:37.771266 22948 raft_consensus.cc:1275] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:10:37.754045 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:37.772652 22955 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 17.
I20260709 13:10:37.774041 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:37.766340 22954 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 20.
I20260709 13:10:37.777622 22843 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:37.778987 22952 raft_consensus.cc:1240] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 13:10:37.755313 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:37.756376 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:37.757555 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:37.783459 22951 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 17.
I20260709 13:10:37.758215 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:37.766371 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:37.767802 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:37.786715 22953 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 17.
I20260709 13:10:37.787315 22950 raft_consensus.cc:2393] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 19: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:37.754886 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:37.733258 29259 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:37.729988 29279 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:37.795048 29279 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:37.795396 29396 consensus_queue.cc:1048] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:10:37.782315 23023 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 16.
I20260709 13:10:37.798586 22843 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:37.799182 22955 raft_consensus.cc:1275] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:10:37.799770 22843 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:37.799216 23024 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 16.
W20260709 13:10:37.658510 28813 consensus_meta.cc:287] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 1.000s	user 0.011s	sys 0.001s
I20260709 13:10:37.756464 29266 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:37.726152 29371 log.cc:927] Time spent T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.377s	user 0.004s	sys 0.006s
W20260709 13:10:37.707722 29380 log.cc:927] Time spent T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.228s	user 0.001s	sys 0.000s
W20260709 13:10:37.708421 29379 log.cc:927] Time spent T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 13:10:37.629179 29246 raft_consensus.cc:740] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:37.693531 29088 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:37.731846 29267 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:37.760776 29201 consensus_meta.cc:287] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.835s	user 0.009s	sys 0.003s
I20260709 13:10:37.771453 29260 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:37.736495 29253 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:37.734146 29369 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:37.719072 29284 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:37.959408 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:37.749183 29150 consensus_queue.cc:1048] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.170s
I20260709 13:10:37.889918 23625 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 1293) took 1204 ms. Trace:
I20260709 13:10:37.733877 29265 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:37.775218 29183 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:37.758896 28869 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:37.724653 28991 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:37.758502 29373 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:37.906484 22876 raft_consensus.cc:1240] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 21 LEADER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 20. Current term is 21. Ops: []
I20260709 13:10:37.777757 28898 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:37.721133 29368 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:37.921805 22992 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.000s	sys 0.020s
W20260709 13:10:38.134591 22992 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.866s	user 0.000s	sys 0.021s
I20260709 13:10:37.727118 29385 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:38.030366 29279 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:37.842787 29259 raft_consensus.cc:2804] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:10:37.887480 29397 log.cc:927] Time spent T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
W20260709 13:10:37.900799 29138 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.858s	user 0.005s	sys 0.011s
I20260709 13:10:37.722067 29320 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:37.729563 29082 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:37.917851 29266 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:10:38.019615 23618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 1291) took 1334 ms. Trace:
I20260709 13:10:37.754251 29388 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:37.918248 28813 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:38.027012 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:38.191434 23023 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 19.
I20260709 13:10:37.962446 29405 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:38.021975 29280 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:38.064337 28801 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:38.070322 29384 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:37.964780 23625 rpcz_store.cc:276] 0709 13:10:36.685736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:36.687005 (+  1269us) service_pool.cc:224] Handling call
0709 13:10:36.742105 (+ 55100us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:10:36.782411 (+ 40306us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:36.782446 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:10:36.796057 (+ 13611us) log.cc:844] Serialized 52 byte log entry
0709 13:10:36.986652 (+190595us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:10:36.986692 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:36.986758 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:37.853164 (+866406us) raft_consensus.cc:1685] finished
0709 13:10:37.853194 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:37.889844 (+ 36650us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39792,"raft.queue_time_us":40108,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":44284,"thread_start_us":147239,"threads_started":1,"wal-append.queue_time_us":320744}
W20260709 13:10:37.978855 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 179159us
I20260709 13:10:37.928351 29201 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:37.957182 29260 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:38.019104 29387 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:38.022846 29267 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:38.028496 29369 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:10:38.023056 29396 raft_consensus.cc:2804] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:10:38.189620 29266 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:38.133355 23021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 2596) took 1589 ms. Trace:
I20260709 13:10:38.270066 29266 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:38.270105 23021 rpcz_store.cc:276] 0709 13:10:36.545614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:36.545758 (+   144us) service_pool.cc:224] Handling call
0709 13:10:36.546242 (+   484us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:36.546443 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:36.546471 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:36.546866 (+   395us) log.cc:844] Serialized 26 byte log entry
0709 13:10:37.183527 (+636661us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:10:37.183566 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:37.183641 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:38.133080 (+949439us) raft_consensus.cc:1685] finished
0709 13:10:38.133109 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:38.133279 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":614235,"threads_started":1,"wal-append.queue_time_us":897357}
I20260709 13:10:37.967615 29284 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:38.272866 29369 raft_consensus.cc:697] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 21 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:38.274209 29284 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:38.274008 29369 consensus_queue.cc:237] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:38.075160 29393 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:38.275316 29393 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:38.215619 28827 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:38.277405 29393 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:38.211114 29409 consensus_queue.cc:1059] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.088s
I20260709 13:10:38.278465 29409 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:38.279088 29409 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:38.279865 29396 raft_consensus.cc:697] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 15 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:38.074595 29398 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:38.280885 29396 consensus_queue.cc:237] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:38.281359 29398 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:38.282894 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 44823.6us
I20260709 13:10:38.283053 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:38.283684 23024 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 19.
I20260709 13:10:38.113332 28945 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:38.072723 29403 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:38.284711 29403 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:38.180302 29354 log.cc:927] Time spent T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 1.165s	user 0.000s	sys 0.015s
I20260709 13:10:38.189806 23618 rpcz_store.cc:276] 0709 13:10:36.685222 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:36.686433 (+  1211us) service_pool.cc:224] Handling call
0709 13:10:36.686703 (+   270us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:36.692978 (+  6275us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:36.693009 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:36.693507 (+   498us) log.cc:844] Serialized 26 byte log entry
0709 13:10:36.988862 (+295355us) spinlock_profiling.cc:243] Waited 6.42 ms on lock 0x7b1800456220. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cdae07 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf7c02 00007f613597f20b 00007f6135984d23 00007f61359850d1 00007f6135da0a0f
0709 13:10:37.377038 (+388176us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:10:37.377094 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:37.377194 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:38.007112 (+629918us) raft_consensus.cc:1685] finished
0709 13:10:38.007150 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:38.019516 (+ 12366us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42267,"raft.queue_time_us":69221,"raft.run_cpu_time_us":363,"raft.run_wall_time_us":29058,"spinlock_wait_cycles":14768384,"thread_start_us":387907,"threads_started":1,"wal-append.queue_time_us":719133}
I20260709 13:10:38.205044 22843 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:10:38.180649 29138 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.144s	user 0.006s	sys 0.012s
I20260709 13:10:38.194022 29279 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:38.112749 29183 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:38.184470 29259 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:38.177800 29408 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:10:38.293079 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 88150.6us
I20260709 13:10:38.294000 22843 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:38.183158 29368 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:38.131019 28869 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:38.129426 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:38.192910 29159 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:36.084265 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:38.143784 28898 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:38.201511 28813 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:38.137769 29373 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:38.200286 29262 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:38.200775 29402 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:38.217155 29404 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:38.217646 29405 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:38.255295 29127 consensus_meta.cc:287] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.864s	user 0.018s	sys 0.003s
I20260709 13:10:38.219116 29387 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:38.219199 29267 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:38.229636 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:38.257200 29201 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:38.229812 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:38.230602 23022 raft_consensus.cc:1275] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
W20260709 13:10:38.232820 29357 log.cc:927] Time spent T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 1.275s	user 0.000s	sys 0.014s
I20260709 13:10:38.233242 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:38.235643 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:38.336779 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 137308us
I20260709 13:10:38.336980 23027 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 19.
W20260709 13:10:38.236178 29400 log.cc:927] Time spent T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.212s	user 0.001s	sys 0.000s
W20260709 13:10:38.236699 29406 log.cc:927] Time spent T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 13:10:38.244971 28991 consensus_queue.cc:1048] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.015s
I20260709 13:10:38.245860 29242 consensus_queue.cc:1048] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.015s
I20260709 13:10:38.261363 29260 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:38.225039 29286 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:38.234841 29388 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:38.258072 29392 log.cc:927] Time spent T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.339s	user 0.000s	sys 0.001s
I20260709 13:10:38.259467 29082 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:38.343189 29094 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:38.237221 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:38.264461 29422 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:38.258028 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:38.244971 29407 consensus_queue.cc:1048] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.025s
I20260709 13:10:38.210990 29291 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:38.215440 29077 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:38.287562 22920 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:38.283954 29398 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:38.300493 22843 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:10:38.384513 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 84143.8us
I20260709 13:10:38.292318 29259 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:38.292172 29022 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:10:38.309894 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:38.391315 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 103856us
W20260709 13:10:38.310618 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:38.298166 29135 raft_consensus.cc:2804] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:38.302594 29284 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:10:38.326895 23025 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 17.
I20260709 13:10:38.327607 22877 raft_consensus.cc:1275] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:10:38.292397 29279 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:38.337121 22873 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 16.
I20260709 13:10:38.403472 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:38.404579 22873 raft_consensus.cc:2410] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
I20260709 13:10:38.369375 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:38.300773 29368 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:38.372920 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:38.373255 22880 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18.
I20260709 13:10:38.327389 29127 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:38.396792 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:38.397159 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:38.398007 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:38.309455 29402 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:38.396188 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:38.405257 23023 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 17.
I20260709 13:10:38.380522 29201 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:38.325526 29404 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:38.444272 29145 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:38.380105 29409 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:38.444998 29145 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:38.447082 29402 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:38.447265 29145 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:38.447662 29201 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:38.450016 29201 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:38.450430 29362 log.cc:927] Time spent T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 1.480s	user 0.000s	sys 0.011s
W20260709 13:10:38.451596 29366 log.cc:927] Time spent T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 1.472s	user 0.002s	sys 0.011s
I20260709 13:10:38.453217 22875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 3162) took 2149 ms. Trace:
W20260709 13:10:38.453567 29372 log.cc:927] Time spent T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 1.483s	user 0.006s	sys 0.012s
I20260709 13:10:38.453621 22875 rpcz_store.cc:276] 0709 13:10:36.303520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:36.307330 (+  3810us) service_pool.cc:224] Handling call
0709 13:10:36.307903 (+   573us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:36.308348 (+   445us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:36.308378 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:36.524472 (+216094us) spinlock_profiling.cc:243] Waited 1.84 ms on lock 0x7b2c000b6b68. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f61346dea11 00007f61346ee372 00007f6135e20a01 00007f6135de543a 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2
0709 13:10:36.547227 (+ 22755us) log.cc:844] Serialized 26 byte log entry
0709 13:10:36.763658 (+216431us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:10:36.763718 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:36.763803 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:38.449820 (+1686017us) spinlock_profiling.cc:243] Waited 70.4 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:10:38.452936 (+  3116us) raft_consensus.cc:1685] finished
0709 13:10:38.452959 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:38.453132 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3672,"raft.queue_time_us":542,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":4779,"spinlock_wait_cycles":166070912,"thread_start_us":175556,"threads_started":1,"wal-append.queue_time_us":379935}
I20260709 13:10:38.388731 29403 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:38.416325 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:38.300613 29408 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:38.436121 22952 raft_consensus.cc:1240] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 17. Current term is 18. Ops: []
I20260709 13:10:38.412838 29284 raft_consensus.cc:697] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 20 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:38.438072 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:38.440496 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:38.382185 29275 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:38.441509 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:38.463459 22955 raft_consensus.cc:2393] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 17: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:38.463594 22953 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 19.
I20260709 13:10:38.414304 23623 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 1284) took 2139 ms. Trace:
I20260709 13:10:38.462993 22948 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 17.
I20260709 13:10:38.438138 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:38.456210 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:38.371821 29286 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:38.457566 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:38.386045 29247 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:38.410835 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:38.401553 29393 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:38.466099 22946 raft_consensus.cc:2393] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 15: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:38.469987 23028 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 2600) took 1794 ms. Trace:
I20260709 13:10:38.437250 29396 raft_consensus.cc:2804] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:38.471026 23027 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 16.
I20260709 13:10:38.406364 29088 raft_consensus.cc:2804] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:38.400449 29135 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:38.473743 23025 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 19.
I20260709 13:10:38.455984 29421 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:38.464253 23623 rpcz_store.cc:276] 0709 13:10:36.264761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:36.278588 (+ 13827us) service_pool.cc:224] Handling call
0709 13:10:36.306167 (+ 27579us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:36.396423 (+ 90256us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:36.396455 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:36.401259 (+  4804us) log.cc:844] Serialized 26 byte log entry
0709 13:10:36.628657 (+227398us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:10:36.628713 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:36.628779 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:38.070484 (+1441705us) spinlock_profiling.cc:243] Waited 89.2 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:10:38.398102 (+327618us) raft_consensus.cc:1685] finished
0709 13:10:38.398133 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:38.404166 (+  6033us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9844,"raft.queue_time_us":66176,"raft.run_cpu_time_us":46,"raft.run_wall_time_us":47,"spinlock_wait_cycles":205126912,"thread_start_us":241027,"threads_started":2,"wal-append.queue_time_us":430016}
I20260709 13:10:38.477077 23022 raft_consensus.cc:1240] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:10:38.475814 23028 rpcz_store.cc:276] 0709 13:10:36.675346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:36.675502 (+   156us) service_pool.cc:224] Handling call
0709 13:10:36.675867 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:36.688462 (+ 12595us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:36.688495 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:36.688949 (+   454us) log.cc:844] Serialized 26 byte log entry
0709 13:10:36.888751 (+199802us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:10:36.888799 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:36.888886 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:38.450994 (+1562108us) spinlock_profiling.cc:243] Waited 61.2 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:10:38.455569 (+  4575us) raft_consensus.cc:1685] finished
0709 13:10:38.455589 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:38.469920 (+ 14331us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":100217,"raft.queue_time_us":52580,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":23315,"spinlock_wait_cycles":140713216,"thread_start_us":199437,"threads_started":1,"wal-append.queue_time_us":290157}
I20260709 13:10:38.439644 29082 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:38.508345 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:38.509013 23028 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 18.
I20260709 13:10:38.383374 29422 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:38.386581 29291 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:38.403640 29022 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:38.512010 29420 log.cc:927] Time spent T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 13:10:38.512964 29286 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:38.518546 29279 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:38.503690 29407 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:38.474622 29275 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:38.520226 29408 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:38.521021 28869 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:38.521849 28898 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:38.521941 29405 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:38.506470 29135 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:38.524094 29368 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:38.479022 29396 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:38.480472 29393 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:38.502836 29088 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:38.477589 29247 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:38.528456 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:38.510505 29183 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:38.583771 23028 raft_consensus.cc:2393] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 17: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:38.533627 29127 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:38.510205 29082 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:38.534534 29275 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:38.536139 23029 raft_consensus.cc:1275] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:10:38.597275 29303 raft_consensus.cc:3055] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 13:10:38.597689 29303 raft_consensus.cc:740] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:38.597769 29259 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:38.537348 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:38.503162 29409 consensus_queue.cc:1059] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 17.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.026s
I20260709 13:10:38.599409 29409 raft_consensus.cc:3055] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:10:38.599843 29409 raft_consensus.cc:740] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:38.584091 29353 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:38.600461 29353 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:38.600744 29409 consensus_queue.cc:260] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 17.16, Last appended by leader: 16, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:38.546721 29022 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:38.545110 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:38.601425 22876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53724 (request call id 1192) took 1950 ms. Trace:
I20260709 13:10:38.554633 22878 raft_consensus.cc:1275] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:10:38.602329 22954 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 18.
W20260709 13:10:38.545466 22826 auto_rebalancer.cc:629] Failed to schedule move for tablet 778fe3ade80844c48f8408313fc86a26: Illegal state: Replica 0a407504ed1741bf89a276558adf4c0f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 20 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
W20260709 13:10:38.547874 29363 log.cc:927] Time spent T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 1.560s	user 0.006s	sys 0.002s
I20260709 13:10:38.532891 29284 consensus_queue.cc:237] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:38.589327 29082 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:38.590091 29359 log.cc:927] Time spent T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 1.589s	user 0.006s	sys 0.005s
I20260709 13:10:38.591735 29247 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:38.584370 29150 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:38.595625 29266 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:38.582443 29088 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:38.621871 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:38.622628 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:38.623250 22872 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 18.
I20260709 13:10:38.508486 29421 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:38.602339 23020 raft_consensus.cc:2468] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15.
I20260709 13:10:38.574182 29396 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:38.602355 22876 rpcz_store.cc:276] 0709 13:10:36.650949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:36.652750 (+  1801us) service_pool.cc:224] Handling call
0709 13:10:36.688558 (+ 35808us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:38.130734 (+1442176us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:38.601199 (+470465us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b5400367c54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:38.601349 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2800230272}
I20260709 13:10:38.617489 22874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53724 (request call id 1197) took 1159 ms. Trace:
W20260709 13:10:38.619292 22844 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.63.4:44043 timed out after 1.599s (SENT) [suppressed 18 similar messages]
I20260709 13:10:38.585862 29267 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:38.539052 28813 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:38.626663 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:38.627931 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:38.604351 29409 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:38.638067 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:38.642222 22947 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 3296) took 2043 ms. Trace:
I20260709 13:10:38.642817 22947 rpcz_store.cc:276] 0709 13:10:36.598562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:36.598693 (+   131us) service_pool.cc:224] Handling call
0709 13:10:36.599646 (+   953us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:36.608402 (+  8756us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:36.608442 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:36.611553 (+  3111us) log.cc:844] Serialized 26 byte log entry
0709 13:10:36.791401 (+179848us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:10:36.791469 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:36.791557 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:38.547232 (+1755675us) spinlock_profiling.cc:243] Waited 40.4 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:10:38.637677 (+ 90445us) raft_consensus.cc:1685] finished
0709 13:10:38.637711 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:38.642133 (+  4422us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8248,"raft.queue_time_us":8377,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":22130,"spinlock_wait_cycles":92843776,"thread_start_us":179515,"threads_started":1,"wal-append.queue_time_us":327815}
I20260709 13:10:38.638507 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:38.638932 22874 rpcz_store.cc:276] 0709 13:10:37.457885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:37.458026 (+   141us) service_pool.cc:224] Handling call
0709 13:10:37.480419 (+ 22393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:37.916052 (+435633us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:37.916097 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:37.916695 (+   598us) log.cc:844] Serialized 26 byte log entry
0709 13:10:38.286739 (+370044us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:10:38.286794 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:38.286886 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:38.617237 (+330351us) raft_consensus.cc:1685] finished
0709 13:10:38.617257 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:38.617427 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10517,"raft.queue_time_us":90665,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":50782,"thread_start_us":648215,"threads_started":2,"wal-append.queue_time_us":407605}
I20260709 13:10:38.641139 22878 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 18.
I20260709 13:10:38.645185 22880 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 15.
I20260709 13:10:38.645864 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:38.646561 22878 raft_consensus.cc:2393] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 17: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:38.538712 29407 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:38.654287 29393 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:38.656410 29407 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:38.657617 23619 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 1298) took 1363 ms. Trace:
I20260709 13:10:38.658068 23619 rpcz_store.cc:276] 0709 13:10:37.294044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:37.294207 (+   163us) service_pool.cc:224] Handling call
0709 13:10:37.294584 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:37.294842 (+   258us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 13:10:37.295158 (+   316us) log.cc:844] Serialized 14 byte log entry
0709 13:10:38.657074 (+1361916us) spinlock_profiling.cc:243] Waited 22 ms on lock 0x7b1800117520. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132c72c0a 00007f613c3b2219 00007f6132cfaa40 00007f6132cf54e4 00007f6132cf5455 00007f6135ddd060 00007f6135def716 00007f6135e00287 00007f6135e00208
0709 13:10:38.657230 (+   156us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:38.657297 (+    67us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:10:38.657355 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:38.657432 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:38.657557 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16224,"prepare.queue_time_us":14727,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"spinlock_wait_cycles":50493824}
I20260709 13:10:38.661571 29348 raft_consensus.cc:3055] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:10:38.662004 29348 raft_consensus.cc:740] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:38.642107 22879 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 19.
I20260709 13:10:38.622349 29150 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:38.666584 29353 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:38.666734 29262 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:38.667361 29262 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:38.667972 29134 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:38.668551 29134 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:38.669317 29377 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:38.670290 29262 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:38.670671 29134 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:38.662993 29348 consensus_queue.cc:260] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:38.671825 29348 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:38.673170 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:38.677368 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:10:38.678009 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 1.21701e+06us
W20260709 13:10:38.678323 29361 log.cc:927] Time spent T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 1.716s	user 0.000s	sys 0.013s
I20260709 13:10:38.679106 29141 consensus_queue.cc:1059] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:10:38.681052 29340 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:38.681993 29265 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:38.682542 29265 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:38.651068 29421 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:38.640264 29237 raft_consensus.cc:3055] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:10:38.652323 29127 consensus_queue.cc:1048] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 13:10:38.684983 29261 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:38.685359 23590 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:38.685796 29387 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:38.685614 29261 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:38.663410 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:38.687104 22880 raft_consensus.cc:2393] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 19: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:38.687139 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:38.677484 23624 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 1295) took 1991 ms. Trace:
W20260709 13:10:38.664243 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:38.687881 23624 rpcz_store.cc:276] 0709 13:10:36.686256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:36.687851 (+  1595us) service_pool.cc:224] Handling call
0709 13:10:36.740252 (+ 52401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:36.755604 (+ 15352us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:38.677152 (+1921548us) spinlock_profiling.cc:243] Waited 13.9 ms on lock 0x7b540046b254. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:38.677424 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":31867776}
I20260709 13:10:38.688436 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:38.688804 29347 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:38.689136 29346 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:38.689070 23618 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18.
I20260709 13:10:38.685034 29237 raft_consensus.cc:740] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
W20260709 13:10:38.686612 22826 auto_rebalancer.cc:629] Failed to schedule move for tablet 8a996de6f02a4723aef91074b3d56714: Illegal state: Replica 0a407504ed1741bf89a276558adf4c0f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 16 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:38.692488 22994 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:10:38.632151 29281 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.662s	user 0.019s	sys 0.041s
I20260709 13:10:38.689759 29346 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:38.695744 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:38.689422 29347 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:38.697965 29375 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:38.698441 29347 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:38.699927 23618 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 17.
I20260709 13:10:38.695016 22994 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:38.630322 29422 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:38.704516 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 85242.5us
I20260709 13:10:38.707784 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:38.708431 23618 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 17.
I20260709 13:10:38.687835 29291 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:38.710803 22878 raft_consensus.cc:1240] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 17. Current term is 18. Ops: []
I20260709 13:10:38.711457 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:38.703531 22994 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:38.712998 22880 raft_consensus.cc:2393] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 19: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:38.713650 29237 consensus_queue.cc:260] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:38.713902 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:38.714898 29237 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:38.689512 29349 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:38.715560 29349 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:38.694628 29265 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:38.715190 22878 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 18.
I20260709 13:10:38.717116 29291 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:38.709247 29313 raft_consensus.cc:2804] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:10:38.699196 29347 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:38.717904 29291 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:38.718982 22879 raft_consensus.cc:1240] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:38.718837 29347 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:38.719121 29303 consensus_queue.cc:260] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:38.707219 29138 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:38.669919 29377 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:38.720328 29138 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:38.721254 29404 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:38.718608 29313 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:38.722157 29313 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:38.702551 29127 raft_consensus.cc:2749] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:38.698552 29375 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:38.726156 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
W20260709 13:10:38.728214 22826 auto_rebalancer.cc:629] Failed to schedule move for tablet 7cd8492f63fb4193a47cf9fbc574ef58: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:10:38.727828 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:38.730641 23618 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:10:38.694160 29281 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.725s	user 0.022s	sys 0.044s
I20260709 13:10:38.741890 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:38.738355 22994 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:10:38.739517 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:38.743690 22876 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 16.
I20260709 13:10:38.744313 22994 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:38.744627 22920 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:38.748004 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:38.747390 22920 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:38.750701 23029 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 17.
I20260709 13:10:38.749277 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:38.749607 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:38.749974 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:38.752058 22994 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:38.752345 23028 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 19.
I20260709 13:10:38.694821 29135 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:38.753157 29421 raft_consensus.cc:2804] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:10:38.753605 29421 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:38.753969 29421 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:38.754781 29135 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:38.759598 29138 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:38.760820 29387 raft_consensus.cc:2804] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:38.760994 22872 raft_consensus.cc:2368] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 17. Current term is 18.
I20260709 13:10:38.761260 29387 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:38.761606 29387 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:38.761641 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:38.762324 23025 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18.
I20260709 13:10:38.720571 29303 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:38.769511 23588 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:38.768204 23623 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 19.
I20260709 13:10:38.771342 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:38.771960 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:38.772181 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:38.772898 23625 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18.
I20260709 13:10:38.776796 29378 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:38.745301 29127 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:38.785871 29377 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:38.766536 29404 raft_consensus.cc:2804] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:38.786733 29396 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:38.789886 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:38.790984 23588 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:38.796186 23622 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:10:38.796901 23588 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:38.791677 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:38.793294 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:38.794691 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:38.770272 29176 raft_consensus.cc:2804] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:10:38.799314 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:38.799593 23588 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:38.801180 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:38.779759 29266 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:38.802594 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:38.789120 29439 raft_consensus.cc:2749] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 13:10:38.805013 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:38.805802 23616 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:38.794477 29127 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:38.810015 22875 raft_consensus.cc:1240] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:10:38.811800 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:38.798799 29176 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:38.795648 29404 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:38.801889 29422 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:38.817423 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:38.784655 29340 consensus_queue.cc:1048] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:10:38.801234 29377 raft_consensus.cc:2749] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:10:38.815572 23621 raft_consensus.cc:1275] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:10:38.777410 29378 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:38.819979 29443 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:38.820832 23617 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 15.
I20260709 13:10:38.806123 29393 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:38.824427 23624 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 18.
I20260709 13:10:38.828155 23625 raft_consensus.cc:1275] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:10:38.813627 29127 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:38.811481 29264 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:38.881583 23618 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18.
I20260709 13:10:38.883116 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:38.853098 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:38.846963 23622 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 20.
I20260709 13:10:38.885187 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:38.885546 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:38.892048 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:38.892714 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:38.892973 23624 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 18.
I20260709 13:10:38.824091 29404 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 15
W20260709 13:10:38.850723 29395 log.cc:927] Time spent T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 1.000s	user 0.003s	sys 0.004s
I20260709 13:10:38.823736 29176 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:38.897815 29396 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:38.898874 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:38.895299 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:38.906721 23624 raft_consensus.cc:2393] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 13: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:38.901363 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:38.824023 29291 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:38.843263 29439 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:38.843040 29349 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:38.905472 23616 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 21.
I20260709 13:10:38.848464 29150 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:38.847549 29160 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:38.908422 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:38.890908 29265 raft_consensus.cc:2749] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:38.916985 29261 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:38.916826 29347 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:38.905247 22919 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:38.913007 23625 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 18.
I20260709 13:10:38.910171 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:38.920907 22919 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:38.921782 22919 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:38.911856 23618 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 18.
I20260709 13:10:38.915082 29375 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:38.852730 29393 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:38.829598 29422 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:38.924947 23618 raft_consensus.cc:1240] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 17. Current term is 18. Ops: []
I20260709 13:10:38.922092 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:38.895696 29264 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:38.922936 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:38.885867 23622 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 19.
I20260709 13:10:38.931704 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:38.941673 23618 raft_consensus.cc:2376] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 21. Re-sending same reply.
I20260709 13:10:38.890976 29088 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:38.931671 23624 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 18.
I20260709 13:10:38.943189 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:38.942378 23588 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:38.927269 29150 raft_consensus.cc:2804] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:38.927807 29291 raft_consensus.cc:2804] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:10:38.929586 29266 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:10:38.943529 23590 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:38.946839 29349 raft_consensus.cc:2804] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:38.940762 29346 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:38.947197 29150 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:38.927431 29160 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:38.955474 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:38.953532 29349 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:38.961063 23618 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 15.
I20260709 13:10:38.951274 29422 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:38.952391 29377 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:38.952602 29291 raft_consensus.cc:697] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:38.945204 29261 consensus_queue.cc:1059] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 13:10:38.953863 29266 raft_consensus.cc:697] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:38.912129 29439 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:38.961730 29349 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:38.966075 29377 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:38.958950 29340 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:38.959275 29150 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:38.976958 29376 raft_consensus.cc:3055] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:10:38.974936 29377 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:38.991448 29439 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:38.992619 29393 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:38.993046 29246 consensus_queue.cc:260] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:38.987324 29264 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:38.984272 29376 raft_consensus.cc:740] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:38.993438 29266 consensus_queue.cc:237] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:38.994385 29246 raft_consensus.cc:3060] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 12 FOLLOWER]: Advancing to term 13
I20260709 13:10:38.994887 29376 consensus_queue.cc:260] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:38.996194 29376 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:10:38.986455 29447 log.cc:927] Time spent T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:10:38.997505 29387 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:38.979862 29340 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:38.999709 29221 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:38.992755 29378 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:38.999770 29387 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:39.014236 29421 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:39.015872 29421 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:39.047966 22871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53724 (request call id 1195) took 1592 ms. Trace:
I20260709 13:10:39.048988 22871 rpcz_store.cc:276] 0709 13:10:37.454928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:37.466008 (+ 11080us) service_pool.cc:224] Handling call
0709 13:10:37.466353 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:37.466670 (+   317us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:37.466697 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:37.467104 (+   407us) log.cc:844] Serialized 26 byte log entry
0709 13:10:38.268902 (+801798us) spinlock_profiling.cc:243] Waited 7.23 ms on lock 0x7b18003bd160. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f613598d4bc 00007f6132cdb48b 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf7c02 00007f613597f20b 00007f6135984d23 00007f61359850d1
0709 13:10:38.269070 (+   168us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:10:38.269109 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:38.269182 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:38.850063 (+580881us) spinlock_profiling.cc:243] Waited 3.82 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:10:39.047084 (+197021us) raft_consensus.cc:1685] finished
0709 13:10:39.047161 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:39.047629 (+   468us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":56117,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":33501,"spinlock_wait_cycles":25426048,"thread_start_us":86916,"threads_started":1,"wal-append.queue_time_us":289524}
I20260709 13:10:39.051657 22919 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:39.053576 22919 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:39.065153 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:39.067186 22950 raft_consensus.cc:2393] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 18: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:39.068517 22948 raft_consensus.cc:1275] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:10:39.068821 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:39.069815 22950 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 19.
I20260709 13:10:39.070305 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:39.070835 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:39.071949 22950 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 15.
I20260709 13:10:39.072190 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:39.073465 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:39.074384 22952 raft_consensus.cc:1240] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:10:39.074535 22953 raft_consensus.cc:1275] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 24 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:10:38.958106 22993 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:39.091115 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:39.091868 23621 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 17.
I20260709 13:10:39.092156 28801 raft_consensus.cc:2804] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:39.093044 28801 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:39.093583 28801 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:10:39.096164 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:39.091157 23620 raft_consensus.cc:2410] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 22 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 13:10:39.092244 23619 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:39.108090 29183 raft_consensus.cc:2804] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:39.108748 29183 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:39.109305 29183 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:39.109982 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:39.110252 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:39.110841 22946 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 19.
I20260709 13:10:39.113292 29340 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:39.113075 29291 consensus_queue.cc:237] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 17.16, Last appended by leader: 16, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:39.118073 29237 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:39.119266 29237 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:39.125737 29240 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:10:39.132419 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 174228us
I20260709 13:10:39.133118 29275 raft_consensus.cc:2804] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:10:39.133702 29275 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:39.134276 29275 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:39.135090 29088 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:39.142263 29242 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:39.143306 29242 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:39.134290 29237 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:39.158213 29160 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:39.160117 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:39.160256 29374 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:39.161804 29374 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:39.170128 22919 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:39.174400 29160 raft_consensus.cc:2804] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:10:39.192737 29259 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:39.194571 29373 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:39.217504 23617 raft_consensus.cc:1275] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 14. (term mismatch)
I20260709 13:10:39.218056 23617 pending_rounds.cc:85] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2: Aborting all ops after (but not including) 13
I20260709 13:10:39.218330 23617 pending_rounds.cc:107] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2: Aborting uncommitted NO_OP operation due to leader change: 20.14
I20260709 13:10:39.218644 23617 raft_consensus.cc:2889] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:39.219812 22877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 2631) took 1468 ms. Trace:
I20260709 13:10:39.220264 22877 rpcz_store.cc:276] 0709 13:10:37.751626 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:37.839533 (+ 87907us) service_pool.cc:224] Handling call
0709 13:10:37.839947 (+   414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:38.400489 (+560542us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:39.219326 (+818837us) spinlock_profiling.cc:243] Waited 485 ms on lock 0x7b54002a1cd4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:39.219653 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1115716992}
W20260709 13:10:39.225914 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:39.230072 29374 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:39.231055 29374 consensus_queue.cc:1048] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:10:39.234210 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:39.234939 22949 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 17.
I20260709 13:10:39.235967 22951 raft_consensus.cc:1275] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:10:39.237283 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:39.238267 22955 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 15.
I20260709 13:10:39.207604 29160 raft_consensus.cc:697] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:39.241880 28868 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:39.242043 28869 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:39.242707 29404 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:39.242722 28868 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:39.243018 28869 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:39.249212 28869 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:39.251051 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:39.251950 23025 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
I20260709 13:10:39.252410 22993 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:39.252890 28868 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:39.253407 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:39.254382 22873 raft_consensus.cc:2393] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 16: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:39.254611 23590 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:39.255333 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:39.255867 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:39.207124 29242 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:39.255335 29191 raft_consensus.cc:2804] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:39.256024 22950 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:39.258004 29191 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:39.258535 29191 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:39.261420 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:39.261574 22955 raft_consensus.cc:1240] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:10:39.261664 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:39.262593 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:39.263247 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:39.263396 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:39.264143 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:39.264503 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:39.264730 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:39.265033 23026 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 19.
I20260709 13:10:39.265542 23029 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 18.
I20260709 13:10:39.264153 23020 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 17.
I20260709 13:10:39.271697 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:39.273358 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:39.274150 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:39.274297 22949 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 18.
I20260709 13:10:39.274739 22954 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 19.
I20260709 13:10:39.271277 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:39.273358 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:39.292537 23619 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18.
I20260709 13:10:39.296526 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:39.295317 29291 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:39.297533 22954 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 13.
I20260709 13:10:39.303505 29265 raft_consensus.cc:2764] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 13:10:39.315896 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:39.355187 22950 raft_consensus.cc:2410] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 17 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
W20260709 13:10:39.358047 29455 log.cc:927] Time spent T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:10:39.298619 29291 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:39.373304 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 69378.6us
W20260709 13:10:39.376813 23591 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:39.378948 29176 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:39.386130 29150 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:39.389097 23024 raft_consensus.cc:2393] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 20: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:39.390503 29183 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:39.392021 22949 raft_consensus.cc:1240] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16. Ops: []
I20260709 13:10:39.394929 29176 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:39.400184 22947 raft_consensus.cc:2468] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 16.
I20260709 13:10:39.401384 28813 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:39.402052 28813 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:39.406739 29237 consensus_queue.cc:1048] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:10:39.408595 29313 consensus_meta.cc:287] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.686s	user 0.014s	sys 0.007s
I20260709 13:10:39.409560 23588 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:39.409246 29313 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:39.418988 29183 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:39.421125 29346 raft_consensus.cc:2804] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:39.421604 29346 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:39.422031 29346 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:39.423318 29291 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:39.425199 29150 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:39.425320 23588 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:39.426313 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:39.427301 22950 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 20.
I20260709 13:10:39.428318 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:39.428932 22950 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 19.
I20260709 13:10:39.428984 29439 raft_consensus.cc:2749] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:39.417536 29377 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:39.408128 29349 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:39.437657 29389 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:39.438623 29349 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:39.438809 29389 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:39.439998 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:39.441009 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:39.441658 22950 raft_consensus.cc:1275] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:10:39.436389 29377 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:39.443140 22919 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:39.444125 29150 raft_consensus.cc:2804] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:10:39.444710 29150 raft_consensus.cc:697] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:39.445152 29378 consensus_queue.cc:1048] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:10:39.446439 29150 consensus_queue.cc:237] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:39.448361 23588 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:39.451045 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:39.452040 22876 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 17.
I20260709 13:10:39.453999 22874 raft_consensus.cc:1275] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 11. (term mismatch)
I20260709 13:10:39.454586 22874 pending_rounds.cc:85] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 10
I20260709 13:10:39.454841 22874 pending_rounds.cc:107] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 20.11
I20260709 13:10:39.455128 22874 raft_consensus.cc:2889] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:39.436254 29346 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:39.358829 29240 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:39.461179 29240 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:39.464454 29378 raft_consensus.cc:2804] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:10:39.465257 29378 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:39.465857 29378 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:39.467453 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:39.361865 22951 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 19.
I20260709 13:10:39.471094 23619 raft_consensus.cc:1240] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16. Ops: []
I20260709 13:10:39.481472 22873 raft_consensus.cc:1275] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:10:39.486066 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:39.487069 22950 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 18.
I20260709 13:10:39.503055 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:39.503954 22950 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:39.514052 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:39.517267 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:10:39.518006 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:39.481197 29313 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:39.524991 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:39.525837 22955 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 17.
I20260709 13:10:39.498641 29259 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:39.530982 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:39.543386 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:39.533380 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:39.546639 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:39.529186 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:39.551905 23616 raft_consensus.cc:2393] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 18: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:39.553375 29399 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:39.553493 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 252738us
I20260709 13:10:39.554081 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:39.554144 29399 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:39.554927 23022 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 14.
I20260709 13:10:39.556082 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:39.556927 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:39.557592 22951 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 21.
I20260709 13:10:39.557668 29373 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:39.558486 29373 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:39.559145 22951 raft_consensus.cc:1275] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:10:39.536686 29346 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:39.560288 28869 consensus_queue.cc:1048] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:10:39.560999 29280 consensus_queue.cc:1048] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:10:39.544313 28868 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:39.564973 22874 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:39.565335 28868 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:39.567437 29373 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:39.567660 28868 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:39.568423 29399 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:39.569235 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:39.570122 22955 raft_consensus.cc:2393] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 18: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:39.571889 22955 raft_consensus.cc:1275] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:10:39.574535 29291 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:39.575132 29221 consensus_queue.cc:1048] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:10:39.553570 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:39.578073 22918 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:39.577939 29275 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:39.580137 23616 raft_consensus.cc:1275] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:10:39.580320 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:39.581048 29160 consensus_queue.cc:237] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:39.594475 29291 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:39.581048 29183 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:39.531908 29242 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:39.556288 28801 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:39.593144 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:39.594734 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:39.587086 29389 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:39.584911 29127 consensus_meta.cc:287] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.691s	user 0.010s	sys 0.009s
W20260709 13:10:39.597541 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 76738.9us
I20260709 13:10:39.600447 29399 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:39.596899 29183 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:39.600720 22951 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:39.601616 23022 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 15.
I20260709 13:10:39.606679 22878 raft_consensus.cc:3060] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:10:39.603920 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:10:39.614095 29456 log.cc:927] Time spent T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.112s	user 0.005s	sys 0.008s
I20260709 13:10:39.614457 22948 raft_consensus.cc:2368] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16.
I20260709 13:10:39.618413 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:39.606061 29221 raft_consensus.cc:2804] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:10:39.621698 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:39.627339 23022 raft_consensus.cc:2393] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 20: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:39.587559 29378 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:39.629782 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:39.636233 22874 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 20.
I20260709 13:10:39.639637 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:39.622571 29221 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:39.632727 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:39.626040 29291 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:39.620867 29399 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:39.640144 29266 consensus_queue.cc:1048] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.020s
I20260709 13:10:39.673924 28801 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:39.649775 22950 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 18.
I20260709 13:10:39.642040 29275 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:39.635290 29454 log.cc:927] Time spent T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 13:10:39.648315 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:39.658500 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:39.658191 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:39.657570 22879 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 20.
W20260709 13:10:39.663446 29246 consensus_meta.cc:287] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.602s	user 0.009s	sys 0.011s
I20260709 13:10:39.611227 29259 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:39.665292 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 59109.2us
I20260709 13:10:39.652755 29221 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:39.666761 29191 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:39.668313 22843 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:39.670863 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:39.691069 23616 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:39.670994 29399 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:39.671792 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:39.672421 29422 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:39.672631 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:39.673517 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:39.679003 22947 raft_consensus.cc:2393] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 18: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:39.612566 29127 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:39.681432 22951 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 18.
I20260709 13:10:39.686368 29259 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:39.625873 29242 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:39.692999 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:39.700721 23625 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:39.694310 22918 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:39.705209 23022 raft_consensus.cc:2393] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 17: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:39.700979 29246 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
W20260709 13:10:39.712412 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 44348.5us
I20260709 13:10:39.719089 22843 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:39.720032 22843 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:39.720870 29373 raft_consensus.cc:2804] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:39.720973 22843 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:39.721369 29373 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:39.721820 29373 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:39.722272 28869 raft_consensus.cc:2804] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:39.722702 28869 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:39.723024 28869 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:39.724648 29280 raft_consensus.cc:2804] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:39.725078 29280 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:39.725427 29280 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:39.728891 29239 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:39.729440 29239 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:39.723218 22843 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:39.736855 28868 raft_consensus.cc:2804] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:10:39.737002 22843 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:39.737308 28868 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:39.737757 28868 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:39.738555 29373 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:39.740968 29388 raft_consensus.cc:2764] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 13:10:39.741161 29280 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:39.742130 22843 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:39.743001 29388 raft_consensus.cc:2804] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:10:39.743314 29280 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:39.743391 29388 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:39.743968 29388 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:39.710951 28898 consensus_queue.cc:1048] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.134s
I20260709 13:10:39.746181 28868 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:39.746774 29389 consensus_queue.cc:1059] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:10:39.747489 29259 raft_consensus.cc:2804] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:39.767362 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:39.750840 22871 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 15.
I20260709 13:10:39.709304 29266 raft_consensus.cc:2749] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:39.753710 29373 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:39.764640 23625 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 16.
I20260709 13:10:39.765347 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:39.753168 29240 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:39.771680 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:39.831884 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 73676.9us
W20260709 13:10:39.834617 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 77160.5us
I20260709 13:10:39.714089 29422 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:39.778051 29259 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
W20260709 13:10:39.878402 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 197876us
W20260709 13:10:39.882426 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 48803.7us
I20260709 13:10:39.854997 29259 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:39.845090 29266 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:39.779482 29246 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:39.921394 29242 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:39.879827 29183 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:39.903556 22844 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:39.934190 23616 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 20.
I20260709 13:10:39.881712 29280 consensus_queue.cc:1048] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 13:10:39.905216 28898 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
W20260709 13:10:39.939105 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 83533.6us
W20260709 13:10:39.985157 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 45900.4us
I20260709 13:10:39.986467 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:40.013427 22843 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:39.987898 29373 raft_consensus.cc:2804] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:39.984748 29127 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:40.024210 22844 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
W20260709 13:10:40.033839 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 100874us
I20260709 13:10:40.010711 28801 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:40.019490 29404 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:10:40.057392 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:40.031342 28813 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:40.022617 29266 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:40.025679 29373 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:40.082693 28813 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:40.076891 28868 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:40.100350 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 190711us
I20260709 13:10:40.071938 29389 consensus_queue.cc:1059] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.027s
W20260709 13:10:40.122602 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 68931.6us
I20260709 13:10:40.093972 29280 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:40.078357 28801 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:40.183564 23618 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 17.
I20260709 13:10:40.037506 28898 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:40.127565 29422 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:40.193660 22992 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:40.178421 29280 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:40.190874 29265 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:40.185830 28813 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:40.077759 29396 consensus_queue.cc:1048] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.211s
I20260709 13:10:40.120376 29408 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
W20260709 13:10:40.134456 29464 log.cc:927] Time spent T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 13:10:40.136943 29463 log.cc:927] Time spent T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
I20260709 13:10:40.137475 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:10:40.118078 29160 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.062s	sys 0.131s
I20260709 13:10:40.158838 29266 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:40.176574 29388 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:40.128446 29373 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:40.203759 29280 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:10:40.229274 29150 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.779s	user 0.027s	sys 0.073s
I20260709 13:10:40.209952 29403 raft_consensus.cc:2804] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:40.230208 29422 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 13:10:40.250151 29160 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.063s	sys 0.131s
I20260709 13:10:40.216845 29240 raft_consensus.cc:3055] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:10:40.126968 22843 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:40.108251 22878 raft_consensus.cc:2468] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 20.
W20260709 13:10:40.299461 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:40.250349 22955 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.019s	sys 0.106s
I20260709 13:10:40.256197 29403 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:10:40.255944 29399 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.047s	sys 0.062s
W20260709 13:10:40.331821 22955 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.019s	sys 0.106s
W20260709 13:10:40.256003 29150 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.806s	user 0.027s	sys 0.073s
I20260709 13:10:40.334367 29403 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:10:40.334453 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 299737us
I20260709 13:10:40.214399 29265 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:40.269389 29422 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:40.399499 22843 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:10:40.337153 29399 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.047s	sys 0.062s
I20260709 13:10:40.330363 29240 raft_consensus.cc:740] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:40.430411 22948 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58320 (request call id 3322) took 1165 ms. Trace:
W20260709 13:10:40.431998 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 233397us
I20260709 13:10:40.431777 22948 rpcz_store.cc:276] 0709 13:10:39.257674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:39.261127 (+  3453us) service_pool.cc:224] Handling call
0709 13:10:40.420988 (+1159861us) spinlock_profiling.cc:243] Waited 351 ms on lock 0x7b54000753d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:40.422149 (+  1161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":806476544}
I20260709 13:10:40.262312 29408 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:40.435858 22919 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:40.410609 29422 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:40.437570 22993 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:40.447139 29150 raft_consensus.cc:2804] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:10:40.447897 29150 raft_consensus.cc:697] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:40.449168 29150 consensus_queue.cc:237] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:40.453977 29398 raft_consensus.cc:2804] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election won for term 18
W20260709 13:10:40.425305 29221 consensus_meta.cc:287] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.726s	user 0.008s	sys 0.004s
W20260709 13:10:40.456012 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:40.457095 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 278435us
I20260709 13:10:40.444175 22993 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:40.454449 28898 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:40.462711 29127 raft_consensus.cc:2804] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:10:40.463052 22843 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:40.427008 28868 raft_consensus.cc:2804] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:10:40.458657 29389 raft_consensus.cc:2804] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:10:40.495817 22872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 3193) took 1133 ms. Trace:
I20260709 13:10:40.482884 22992 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:40.483517 29127 raft_consensus.cc:697] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 17 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:40.491079 22843 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:40.491757 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:40.483639 22946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 3315) took 1251 ms. Trace:
I20260709 13:10:40.477404 29340 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:40.478245 29237 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:10:40.499382 28869 consensus_meta.cc:287] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.776s	user 0.011s	sys 0.003s
I20260709 13:10:40.516032 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:40.462035 29221 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:40.502291 29466 log.cc:927] Time spent T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:10:40.521087 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:40.497889 22872 rpcz_store.cc:276] 0709 13:10:39.360998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:39.376431 (+ 15433us) service_pool.cc:224] Handling call
0709 13:10:39.377088 (+   657us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:39.377639 (+   551us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:39.377678 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:39.378369 (+   691us) log.cc:844] Serialized 26 byte log entry
0709 13:10:39.676884 (+298515us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:10:39.676922 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:39.676984 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:40.493627 (+816643us) raft_consensus.cc:1685] finished
0709 13:10:40.493670 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:40.493830 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":19543,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":153,"thread_start_us":298050,"threads_started":1,"wal-append.queue_time_us":605717}
I20260709 13:10:40.500802 22946 rpcz_store.cc:276] 0709 13:10:39.227937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:39.228233 (+   296us) service_pool.cc:224] Handling call
0709 13:10:39.228695 (+   462us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:39.229303 (+   608us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:39.229336 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:39.304629 (+ 75293us) spinlock_profiling.cc:243] Waited 48.9 ms on lock 0x7b2c0005beb8. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f61346dea11 00007f61346ee372 00007f6135e20a01 00007f6135de543a 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2
0709 13:10:39.305110 (+   481us) log.cc:844] Serialized 26 byte log entry
0709 13:10:39.690211 (+385101us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:10:39.690698 (+   487us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:39.690767 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:40.470698 (+779931us) raft_consensus.cc:1685] finished
0709 13:10:40.470728 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:40.478885 (+  8157us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2306,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":102,"spinlock_wait_cycles":112428416,"thread_start_us":384691,"threads_started":1,"wal-append.queue_time_us":722984}
I20260709 13:10:40.494208 28868 raft_consensus.cc:697] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 18 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:40.525827 23621 raft_consensus.cc:2393] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 19: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:40.526005 29388 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 22 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:40.526135 29399 consensus_queue.cc:1048] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:10:40.527011 29240 consensus_queue.cc:260] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:40.531793 23618 raft_consensus.cc:1240] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16. Ops: []
I20260709 13:10:40.498596 28801 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:40.498530 29389 raft_consensus.cc:697] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 20 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:40.543445 22994 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:40.500125 29396 raft_consensus.cc:2749] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:40.545676 29388 consensus_queue.cc:1048] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:10:40.546221 28868 consensus_queue.cc:237] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:40.528219 29373 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:40.544646 22918 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:40.522800 29237 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:40.532761 28869 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:40.554517 29176 raft_consensus.cc:2749] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:10:40.568801 29259 consensus_meta.cc:287] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.585s	user 0.012s	sys 0.000s
I20260709 13:10:40.517976 29340 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:40.590317 22952 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 3316) took 1361 ms. Trace:
W20260709 13:10:40.590940 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:40.590741 22952 rpcz_store.cc:276] 0709 13:10:39.228506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:39.228635 (+   129us) service_pool.cc:224] Handling call
0709 13:10:39.229138 (+   503us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:39.229949 (+   811us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 13:10:39.229979 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:39.700025 (+470046us) spinlock_profiling.cc:243] Waited 48.4 ms on lock 0x7b2c0005beb8. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f61346dea11 00007f61346ee372 00007f6135e20a01 00007f6135de543a 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2
0709 13:10:39.746254 (+ 46229us) log.cc:844] Serialized 26 byte log entry
0709 13:10:40.245399 (+499145us) raft_consensus.cc:1645] Marking committed up to 8
0709 13:10:40.245470 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:40.245761 (+   291us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:40.589989 (+344228us) raft_consensus.cc:1685] finished
0709 13:10:40.590011 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:40.590250 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2044,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":91,"spinlock_wait_cycles":111228288,"thread_start_us":485641,"threads_started":1,"wal-append.queue_time_us":792381}
I20260709 13:10:40.600805 29373 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:40.603725 29280 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:40.604760 29266 raft_consensus.cc:2804] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:40.604913 29176 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:40.574581 29259 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:40.610905 29246 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:40.614881 22875 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.004s	sys 0.032s
W20260709 13:10:40.625126 22875 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.004s	sys 0.033s
I20260709 13:10:40.573221 29240 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Advancing to term 16
W20260709 13:10:40.620771 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 158337us
I20260709 13:10:40.615435 29396 raft_consensus.cc:2749] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:40.580857 28813 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:40.595978 29403 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:40.640978 22844 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.015s	sys 0.033s
W20260709 13:10:40.655586 22844 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.015s	sys 0.033s
W20260709 13:10:40.656080 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 631908us
I20260709 13:10:40.628949 22875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 3192) took 1356 ms. Trace:
I20260709 13:10:40.656759 22875 rpcz_store.cc:276] 0709 13:10:39.276511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:39.290579 (+ 14068us) service_pool.cc:224] Handling call
0709 13:10:39.513788 (+223209us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:39.592946 (+ 79158us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:10:40.035097 (+442151us) log.cc:844] Serialized 14 byte log entry
0709 13:10:40.628502 (+593405us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:40.628570 (+    68us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:10:40.628627 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:40.628728 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:40.628873 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":63651,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":2256,"raft.queue_time_us":14129,"thread_start_us":1002934,"threads_started":2}
W20260709 13:10:40.659312 29465 log.cc:927] Time spent T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.647s	user 0.001s	sys 0.008s
I20260709 13:10:40.620339 29266 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:40.660485 29266 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:40.660933 29378 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:40.663386 29378 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:40.663877 29378 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:40.625152 22918 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:40.611904 29408 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:40.655344 22992 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:40.625800 29265 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:10:40.656565 29275 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.027s	sys 0.028s
I20260709 13:10:40.628135 29246 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:40.693017 29246 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:40.695818 29246 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:10:40.688117 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 47449.9us
W20260709 13:10:40.696889 22846 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.014s	sys 0.033s
W20260709 13:10:40.697364 22846 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.015s	sys 0.034s
I20260709 13:10:40.698087 22992 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:40.699234 29378 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:40.699589 29472 log.cc:927] Time spent T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 13:10:40.701475 29237 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:40.625382 29389 consensus_queue.cc:237] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:40.707269 29340 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:40.709731 29280 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:40.710896 29221 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:40.578011 29127 consensus_queue.cc:237] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:40.627664 29176 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:40.696499 29246 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:40.715157 29176 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:40.715888 29176 consensus_queue.cc:1059] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 13:10:40.690764 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 71231.6us
I20260709 13:10:40.691799 22844 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:40.712801 29403 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:40.746798 22955 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 1145) took 1112 ms. Trace:
W20260709 13:10:40.689427 29275 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.028s	sys 0.028s
W20260709 13:10:40.766752 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 75034.6us
I20260709 13:10:40.745657 28813 raft_consensus.cc:2749] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:10:40.791409 22918 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:40.747335 22955 rpcz_store.cc:276] 0709 13:10:39.634268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:39.634599 (+   331us) service_pool.cc:224] Handling call
0709 13:10:39.637284 (+  2685us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:10:39.637695 (+   411us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:39.637726 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:10:39.638351 (+   625us) log.cc:844] Serialized 52 byte log entry
0709 13:10:40.337359 (+699008us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:10:40.744152 (+406793us) log.cc:844] Serialized 14 byte log entry
0709 13:10:40.746304 (+  2152us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:40.746408 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:40.746461 (+    53us) raft_consensus.cc:1685] finished
0709 13:10:40.746479 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:40.746650 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1696,"raft.queue_time_us":5222,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":105,"thread_start_us":1052736,"threads_started":2,"wal-append.queue_time_us":920132}
I20260709 13:10:40.794309 29266 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:40.800189 29266 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:40.803499 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:40.804738 29280 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:10:40.805143 29280 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:40.805496 29280 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:40.805785 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:40.808651 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:40.809816 29388 consensus_queue.cc:1048] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:10:40.813031 29479 consensus_queue.cc:1048] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:10:40.813367 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:40.814127 22875 raft_consensus.cc:2468] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 12.
I20260709 13:10:40.814707 29479 raft_consensus.cc:2804] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:40.815243 29479 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:40.815733 29479 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:40.819954 29239 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:40.828459 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 76084.5us
I20260709 13:10:40.788106 29176 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:10:40.857327 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 310587us
I20260709 13:10:40.795382 29191 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:40.865466 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:40.866168 29471 log.cc:927] Time spent T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.350s	user 0.001s	sys 0.001s
I20260709 13:10:40.873157 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:10:40.883036 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:40.864429 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 234960us
I20260709 13:10:40.875646 22955 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 21.
I20260709 13:10:40.847512 29266 raft_consensus.cc:2804] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:40.848017 29176 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:40.888298 22918 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:40.901124 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:40.901837 22875 raft_consensus.cc:1240] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 18. Current term is 19. Ops: []
W20260709 13:10:40.904240 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 54054.7us
I20260709 13:10:40.903369 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:40.893848 29176 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:40.908222 23621 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 21.
I20260709 13:10:40.909516 22948 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 13.
I20260709 13:10:40.888151 29266 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:40.910797 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:10:40.914209 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:40.914285 23618 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 19.
I20260709 13:10:40.908258 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:40.913288 29266 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:40.924800 22918 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:40.927275 22918 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:40.928261 29221 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:40.929020 29221 raft_consensus.cc:2804] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:40.929396 29221 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:40.929769 29221 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:40.938043 29221 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:40.940294 29221 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:40.945822 23623 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60044 (request call id 1174) took 1261 ms. Trace:
I20260709 13:10:40.946326 23623 rpcz_store.cc:276] 0709 13:10:39.683926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:39.684780 (+   854us) service_pool.cc:224] Handling call
0709 13:10:39.685182 (+   402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:39.685597 (+   415us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:39.685630 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:39.686238 (+   608us) log.cc:844] Serialized 26 byte log entry
0709 13:10:40.083035 (+396797us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:10:40.083091 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:40.083182 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:40.865140 (+781958us) spinlock_profiling.cc:243] Waited 30.7 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:10:40.945321 (+ 80181us) raft_consensus.cc:1685] finished
0709 13:10:40.945359 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:40.945614 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":89,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":99,"spinlock_wait_cycles":70624512,"thread_start_us":396338,"threads_started":1,"wal-append.queue_time_us":755595}
I20260709 13:10:40.945847 29483 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:40.949270 29483 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:40.952070 29483 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:40.905517 29242 raft_consensus.cc:3055] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:10:40.907382 28813 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:10:40.963470 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:40.969682 22994 leader_election.cc:400] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 17. Current term is 18.
I20260709 13:10:40.970180 22994 leader_election.cc:403] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:10:40.974753 29266 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:40.975472 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:40.975924 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:40.976289 22873 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:40.978559 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:40.979363 22879 raft_consensus.cc:2393] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 19: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:40.980400 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:40.983822 22873 raft_consensus.cc:2468] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 20.
I20260709 13:10:40.980453 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:40.979156 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:10:40.991729 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 293715us
I20260709 13:10:40.998368 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:40.989908 22878 raft_consensus.cc:2376] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 20. Re-sending same reply.
I20260709 13:10:41.004041 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:40.962515 28813 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:40.959445 29242 raft_consensus.cc:740] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:41.009814 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.018690 22873 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 17.
I20260709 13:10:41.017931 23621 raft_consensus.cc:2468] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 13.
I20260709 13:10:41.038422 22877 raft_consensus.cc:2368] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 19. Current term is 20.
I20260709 13:10:41.038940 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:41.041633 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 330696us
I20260709 13:10:40.988867 29237 raft_consensus.cc:493] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:41.046218 22993 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:10:41.041859 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 48867.1us
I20260709 13:10:41.038632 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.042481 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.048251 22876 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 13.
W20260709 13:10:41.050843 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 140760us
I20260709 13:10:41.055383 22992 leader_election.cc:400] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16.
I20260709 13:10:41.055735 22992 leader_election.cc:403] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:10:41.031208 28813 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:41.039862 29275 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:10:41.056548 22994 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:41.058872 29191 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:41.071967 29242 consensus_queue.cc:260] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:41.073168 29242 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:41.077088 29266 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:41.077401 29280 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:41.078022 29127 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:10:41.080139 29280 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:41.070379 29246 raft_consensus.cc:2749] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:10:41.070447 29422 consensus_meta.cc:287] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.578s	user 0.009s	sys 0.006s
I20260709 13:10:41.084054 29422 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:41.052135 29237 raft_consensus.cc:515] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:41.087101 22875 raft_consensus.cc:2368] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 19. Current term is 20.
I20260709 13:10:41.055673 29404 raft_consensus.cc:2804] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:10:41.090507 29404 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:41.093407 29404 raft_consensus.cc:3060] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:41.093313 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.095011 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:41.095763 23621 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:41.100327 29479 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:41.102414 29479 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:41.104799 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.105675 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.103871 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.105829 22873 raft_consensus.cc:2393] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 19: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:41.106514 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:41.118424 22878 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:41.123255 29280 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:10:41.107712 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 59843us
W20260709 13:10:41.110145 22843 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.006s	sys 0.010s
W20260709 13:10:41.124780 22843 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.007s	sys 0.010s
I20260709 13:10:41.115370 29404 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:41.129304 23621 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 19.
I20260709 13:10:41.134897 22878 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 20.
I20260709 13:10:41.092592 29422 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:41.136467 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 645451us
I20260709 13:10:41.137250 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.137897 22876 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 13.
I20260709 13:10:41.070389 29191 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:41.152642 29191 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:10:41.148212 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 91770.2us
I20260709 13:10:41.149590 29237 leader_election.cc:290] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:41.129549 29280 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:41.134125 29240 raft_consensus.cc:2749] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:41.158305 23588 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:41.163763 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:41.161509 22919 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:41.173839 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:41.174628 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:41.175324 22955 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:41.179805 22843 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:41.185281 22949 raft_consensus.cc:2393] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 20: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:41.181456 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:41.189098 22948 raft_consensus.cc:2410] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 25 because replica has last-logged OpId of term: 24 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 16.
I20260709 13:10:41.183455 22919 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:41.192025 22952 raft_consensus.cc:2368] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16.
I20260709 13:10:41.193305 29266 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:10:41.193794 29266 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:41.194221 29266 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:41.194592 29404 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:41.195442 22994 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:41.196060 29242 consensus_queue.cc:1048] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:10:41.196877 29389 raft_consensus.cc:2804] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:10:41.197479 29389 raft_consensus.cc:697] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 20 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:41.196381 22994 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:41.202054 22994 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:41.177915 29340 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:10:41.198697 23588 leader_election.cc:336] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.579s (SENT)
I20260709 13:10:41.204380 23621 raft_consensus.cc:1275] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:10:41.204545 22994 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:41.203238 29389 consensus_queue.cc:237] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:41.207701 29469 log.cc:927] Time spent T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.704s	user 0.006s	sys 0.006s
I20260709 13:10:41.200932 29408 raft_consensus.cc:2804] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:41.188007 29160 raft_consensus.cc:2804] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:10:41.213779 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:41.217558 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:41.220435 22994 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:41.223366 23621 raft_consensus.cc:3060] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:41.187621 29176 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:41.225543 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 64580.7us
I20260709 13:10:41.226699 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:41.209672 29239 raft_consensus.cc:2804] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:10:41.213205 29340 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:41.235688 23621 raft_consensus.cc:2468] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 14.
I20260709 13:10:41.239862 22949 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 17.
I20260709 13:10:41.222760 29242 raft_consensus.cc:2749] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:10:41.227705 29313 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.136s	user 0.018s	sys 0.034s
I20260709 13:10:41.244004 22950 raft_consensus.cc:1240] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:10:41.222985 29408 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:41.254704 29240 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:41.255291 29240 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:41.256147 29176 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:41.257263 29240 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:41.257740 28869 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:41.194720 29488 consensus_queue.cc:1048] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 13:10:41.259032 29280 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:41.241211 29340 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:10:41.248625 29313 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.167s	user 0.018s	sys 0.034s
I20260709 13:10:41.261132 29408 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:41.239282 29239 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:41.228935 29160 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:41.250054 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:41.250577 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:41.270612 22948 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 18.
I20260709 13:10:41.231536 29404 raft_consensus.cc:2804] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:10:41.248479 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:41.245253 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.266991 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:41.239337 29493 raft_consensus.cc:2804] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:10:41.273514 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:41.277231 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 78549.6us
I20260709 13:10:41.277243 22874 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 15.
I20260709 13:10:41.275655 22876 raft_consensus.cc:2468] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 20.
I20260709 13:10:41.279913 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:41.278470 23588 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:41.269675 29239 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:41.272640 29160 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:41.308933 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:41.312078 22949 raft_consensus.cc:3060] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:10:41.317312 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 85423.6us
I20260709 13:10:41.317556 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:41.314368 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 76745us
I20260709 13:10:41.319849 22948 raft_consensus.cc:2393] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 18: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:41.321520 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:41.300213 29237 raft_consensus.cc:2804] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:41.326236 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.327049 22876 raft_consensus.cc:2393] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 22: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:41.301735 29404 raft_consensus.cc:697] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:41.330345 22993 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:41.310928 29493 raft_consensus.cc:697] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 20 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:41.359401 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:41.323160 29246 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:41.327920 29127 raft_consensus.cc:2749] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16.
I20260709 13:10:41.325765 29237 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:41.363131 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:41.363732 22874 raft_consensus.cc:2393] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 20: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:41.329581 29259 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:41.365752 22876 raft_consensus.cc:3060] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:10:41.367048 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:41.368091 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:41.365123 29237 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:41.377024 29191 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:41.377662 29191 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:41.380048 29191 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:41.380606 29237 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:41.381918 23624 raft_consensus.cc:1240] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 17. Current term is 18. Ops: []
I20260709 13:10:41.329370 29313 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:41.386433 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:41.376370 22955 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 20.
I20260709 13:10:41.398904 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:41.399272 23029 raft_consensus.cc:1275] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 24 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 16. (term mismatch)
I20260709 13:10:41.402576 23024 raft_consensus.cc:2410] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:10:41.403592 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:41.399825 23029 pending_rounds.cc:85] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75: Aborting all ops after (but not including) 15
I20260709 13:10:41.405894 23029 pending_rounds.cc:107] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75: Aborting uncommitted NO_OP operation due to leader change: 23.16
I20260709 13:10:41.406174 23029 raft_consensus.cc:2889] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:41.406487 23028 raft_consensus.cc:2393] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 19: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:41.410102 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.410758 22880 raft_consensus.cc:2468] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 12.
W20260709 13:10:41.411598 29502 log.cc:927] Time spent T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 13:10:41.412930 29503 raft_consensus.cc:2804] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:10:41.412961 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:41.413091 29313 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:41.413618 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:41.413249 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.414707 22877 raft_consensus.cc:2410] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 13:10:41.413709 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:41.415845 22871 raft_consensus.cc:2393] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 17: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:41.416002 22873 raft_consensus.cc:2393] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 19: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:41.413290 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.415115 29246 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:41.442016 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.414397 23624 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 18.
I20260709 13:10:41.418133 22872 raft_consensus.cc:1813] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Handling vote request from an unknown peer aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:41.445144 22872 raft_consensus.cc:2410] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 10.
I20260709 13:10:41.445662 29266 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:41.448366 22875 raft_consensus.cc:2393] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 18: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:41.332352 28813 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:41.484877 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 185704us
I20260709 13:10:41.485026 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.486320 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:41.489292 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 217028us
I20260709 13:10:41.447289 29239 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:41.449653 29473 log.cc:927] Time spent T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.909s	user 0.004s	sys 0.000s
I20260709 13:10:41.497191 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:41.497335 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:41.498492 23624 raft_consensus.cc:1275] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 17 index: 16. Preceding OpId from leader: term: 18 index: 17. (index mismatch)
I20260709 13:10:41.500263 23621 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:41.501976 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:41.503222 23024 raft_consensus.cc:2410] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 20 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:10:41.495247 23588 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:41.499956 22874 raft_consensus.cc:3060] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 12 FOLLOWER]: Advancing to term 14
I20260709 13:10:41.499523 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.448194 29266 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:41.500018 22993 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:41.484843 29221 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:41.507606 22947 raft_consensus.cc:2368] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 19. Current term is 20.
W20260709 13:10:41.495227 29464 log.cc:927] Time spent T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 13:10:41.509610 23621 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 22.
W20260709 13:10:41.494959 29398 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.031s	user 0.019s	sys 0.027s
I20260709 13:10:41.513635 22843 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:10:41.515825 29504 log.cc:927] Time spent T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
I20260709 13:10:41.515861 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:41.517405 23022 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 20.
I20260709 13:10:41.519846 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.520036 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:41.520846 23023 raft_consensus.cc:2393] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 21: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:41.490187 29389 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:41.492447 29240 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:41.527906 29266 consensus_queue.cc:1048] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:10:41.494371 29493 consensus_queue.cc:237] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:41.529655 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:41.564002 22874 raft_consensus.cc:2468] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 14.
I20260709 13:10:41.530037 29127 raft_consensus.cc:2804] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:41.530435 29503 raft_consensus.cc:697] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 14 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:41.514364 29221 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:41.533232 29513 log.cc:927] Time spent T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 13:10:41.534459 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:41.584717 22873 raft_consensus.cc:2393] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 18: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:41.534893 29506 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:41.535019 29275 raft_consensus.cc:2749] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 17. Current term is 18.
I20260709 13:10:41.536818 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:41.538550 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:41.538350 29160 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:41.541057 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:41.541431 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:41.499205 29518 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:41.543548 29239 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:41.511642 29404 consensus_queue.cc:237] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:41.596203 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 89543.9us
I20260709 13:10:41.544003 29508 consensus_queue.cc:1059] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 13:10:41.545894 22919 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:41.522060 29242 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
W20260709 13:10:41.511359 29398 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.048s	user 0.019s	sys 0.027s
I20260709 13:10:41.554152 28813 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:41.527886 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.561655 22994 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:41.562084 22949 raft_consensus.cc:2468] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 18.
I20260709 13:10:41.564460 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:41.564546 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 65352.7us
I20260709 13:10:41.565090 29259 raft_consensus.cc:2764] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
W20260709 13:10:41.565230 29519 log.cc:927] Time spent T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 13:10:41.577195 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:10:41.579623 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:41.583284 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:41.590926 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:41.573138 29127 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:41.508466 29485 raft_consensus.cc:2804] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:41.598121 22918 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:41.599314 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 53535.6us
I20260709 13:10:41.604705 23022 raft_consensus.cc:2468] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 13.
I20260709 13:10:41.603400 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:41.619680 23023 raft_consensus.cc:2410] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 13:10:41.608124 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:41.620321 22873 raft_consensus.cc:2376] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 20. Re-sending same reply.
I20260709 13:10:41.621143 22874 raft_consensus.cc:2368] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 19. Current term is 20.
W20260709 13:10:41.623729 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 62154.3us
I20260709 13:10:41.625262 22994 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:41.627141 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:41.630220 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:41.631227 22880 raft_consensus.cc:2368] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 19. Current term is 20.
I20260709 13:10:41.546125 28801 consensus_queue.cc:1048] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.055s
I20260709 13:10:41.596306 29237 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:41.639820 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:41.643288 23616 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:41.585652 29240 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:41.604501 29398 raft_consensus.cc:697] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 18 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:41.640645 22877 raft_consensus.cc:1240] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 13:10:41.587380 29389 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:41.646539 23024 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:41.647563 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:41.617377 29275 raft_consensus.cc:3055] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 13:10:41.650780 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 52768us
I20260709 13:10:41.656242 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:41.624068 29485 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:41.624010 29127 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:41.639842 29488 raft_consensus.cc:2749] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:10:41.665251 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:41.681295 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:41.599304 29506 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:41.684201 22872 raft_consensus.cc:2393] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 18: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:41.684163 23028 raft_consensus.cc:2393] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 19: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:41.665300 22994 leader_election.cc:400] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 19. Current term is 20.
I20260709 13:10:41.685978 22994 leader_election.cc:403] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
W20260709 13:10:41.684338 29512 log.cc:927] Time spent T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 13:10:41.688129 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:41.688233 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.651077 29422 raft_consensus.cc:2749] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:41.656632 29275 raft_consensus.cc:740] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:41.597317 29518 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:41.691819 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.661707 29485 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:41.648129 29242 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:41.700484 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:41.701558 29221 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:41.711066 23029 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 16.
I20260709 13:10:41.712128 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:41.713279 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:41.713894 22875 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:41.714615 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.715231 22876 raft_consensus.cc:2468] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18.
I20260709 13:10:41.717622 22872 raft_consensus.cc:2393] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 20: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:41.717759 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:41.690763 29508 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:41.723812 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.721957 23022 raft_consensus.cc:2393] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 17: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:41.689422 29520 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:10:41.725394 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:41.730388 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.732797 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:41.734416 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:41.735188 23022 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 17.
I20260709 13:10:41.737229 23616 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 20.
I20260709 13:10:41.736219 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:41.742245 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.742777 22875 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18.
I20260709 13:10:41.743325 22876 raft_consensus.cc:2393] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 19: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:41.744129 23022 raft_consensus.cc:2393] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 16: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:41.742785 23616 raft_consensus.cc:1240] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 19. Current term is 20. Ops: []
I20260709 13:10:41.732477 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.745441 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:41.744064 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 117358us
I20260709 13:10:41.746258 22878 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 13.
I20260709 13:10:41.748901 29408 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:41.751037 29408 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:41.751739 29398 consensus_queue.cc:237] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:41.758908 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:41.759589 22873 raft_consensus.cc:2376] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18. Re-sending same reply.
I20260709 13:10:41.764854 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:41.765611 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.755832 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:41.784423 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:41.785742 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.786350 22876 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 15.
I20260709 13:10:41.787192 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:41.787812 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.788619 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:41.789232 23028 raft_consensus.cc:2468] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 13.
I20260709 13:10:41.790074 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:41.790691 23028 raft_consensus.cc:2393] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 16: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:41.791394 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:41.792338 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:41.793673 29485 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:41.797605 29485 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:41.795164 29483 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:41.798022 29240 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:41.684949 29503 consensus_queue.cc:237] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:41.798449 29483 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:41.795408 22878 raft_consensus.cc:2393] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 17: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:41.795643 22876 raft_consensus.cc:2393] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 20: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:41.725710 29528 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:41.808427 23027 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 18.
I20260709 13:10:41.810276 23022 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:41.815966 23026 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 16.
I20260709 13:10:41.819276 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:41.816401 22872 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 18.
I20260709 13:10:41.819900 22876 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:41.755766 29520 raft_consensus.cc:697] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:41.820365 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.821532 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.821341 29520 consensus_queue.cc:237] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:41.824972 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:41.825695 23028 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 17.
I20260709 13:10:41.826665 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:41.827248 29518 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:41.800500 29483 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:41.827212 23028 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 17.
I20260709 13:10:41.829182 29506 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:41.829506 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.829879 22843 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:41.830675 29408 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:41.830714 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.831228 22843 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:41.831828 22876 raft_consensus.cc:2468] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 16.
I20260709 13:10:41.835021 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:41.835541 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.836621 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.831987 29408 raft_consensus.cc:2749] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:10:41.768949 29340 consensus_meta.cc:287] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.508s	user 0.005s	sys 0.005s
I20260709 13:10:41.829331 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:41.846985 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 189946us
I20260709 13:10:41.832473 22843 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:41.825591 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:41.842634 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.845598 23024 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18.
I20260709 13:10:41.849490 22873 raft_consensus.cc:2368] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 19. Current term is 20.
I20260709 13:10:41.849830 22878 raft_consensus.cc:1240] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 21. Current term is 22. Ops: [21.11-21.11]
I20260709 13:10:41.850940 22873 raft_consensus.cc:1275] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:10:41.844477 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:41.833624 29389 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:41.859151 22873 raft_consensus.cc:1275] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:10:41.861079 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.861753 22876 raft_consensus.cc:2376] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18. Re-sending same reply.
I20260709 13:10:41.863476 22872 raft_consensus.cc:1240] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 19. Current term is 20. Ops: []
I20260709 13:10:41.866947 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:41.870904 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:41.877097 29313 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:41.878325 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:41.880610 29408 raft_consensus.cc:2804] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:10:41.880815 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 48448.7us
I20260709 13:10:41.881146 29408 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:41.881568 29408 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:41.881616 22843 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:41.883167 29259 raft_consensus.cc:2804] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:10:41.883632 29259 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:41.884087 29259 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:41.880443 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:41.854882 29340 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:41.885455 23024 raft_consensus.cc:2468] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 16.
I20260709 13:10:41.886307 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec334666b7964c969125af95d9c797ed" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:41.886796 29340 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 22 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:41.887357 29340 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:41.887799 29340 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:41.888785 22994 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:41.758402 29275 consensus_queue.cc:260] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:41.890343 29275 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:41.891103 29389 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:41.891577 29389 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:41.891974 29389 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:41.865841 22918 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:41.884454 22843 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:41.878325 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:41.912707 23621 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 19.
W20260709 13:10:41.913293 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 47550.7us
I20260709 13:10:41.905241 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.910183 23022 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18.
I20260709 13:10:41.916797 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:41.917863 23022 raft_consensus.cc:2393] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 16: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:41.934854 22872 raft_consensus.cc:1240] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 18. Current term is 19. Ops: []
I20260709 13:10:41.935712 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.936481 22918 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:41.939527 22992 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:41.941325 29479 raft_consensus.cc:2749] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:10:41.947757 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:41.948606 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.952702 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.949579 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:41.948719 22949 raft_consensus.cc:2410] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 16 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 13:10:41.955156 22992 leader_election.cc:400] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 election: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16.
I20260709 13:10:41.956434 22992 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:41.962014 22992 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:41.963274 22992 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:41.963481 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:41.968699 22877 raft_consensus.cc:2393] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 20: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:41.970523 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:41.970908 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:41.971741 23027 raft_consensus.cc:2410] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 20 because replica has last-logged OpId of term: 17 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:10:41.974632 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:41.975695 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:41.975852 22992 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:41.976083 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:41.976831 22947 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:41.976923 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:41.973898 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.977725 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.979377 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:41.982087 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:41.982118 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:41.983225 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:41.984320 22993 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:41.935582 29528 raft_consensus.cc:2749] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:41.989367 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.990056 22992 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:41.991207 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.884853 29511 consensus_queue.cc:1059] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.097s
W20260709 13:10:41.989440 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 53054.6us
I20260709 13:10:41.994364 22875 raft_consensus.cc:2376] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 20. Re-sending same reply.
I20260709 13:10:41.995750 22918 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:41.997406 22880 raft_consensus.cc:2368] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 19. Current term is 20.
I20260709 13:10:41.998376 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:41.998835 22992 leader_election.cc:400] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 19. Current term is 20.
I20260709 13:10:41.999104 23024 raft_consensus.cc:2393] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 19: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:41.999683 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:42.000787 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:42.001674 23022 raft_consensus.cc:3060] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:42.003705 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:42.004616 22950 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:41.997529 23616 raft_consensus.cc:1275] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:10:42.006508 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:41.934408 29506 consensus_queue.cc:1059] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.059s
I20260709 13:10:41.971359 29237 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:42.013100 23025 raft_consensus.cc:2393] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 18: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:42.013525 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:42.010524 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:42.017072 23624 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:42.014420 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:42.015242 22918 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:42.017994 23029 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 15.
I20260709 13:10:41.983877 29479 raft_consensus.cc:2749] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:42.021669 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:41.989518 29503 raft_consensus.cc:2749] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:42.021912 22918 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:41.950363 29313 consensus_queue.cc:1048] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.056s
W20260709 13:10:42.024588 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 130839us
I20260709 13:10:41.980871 29526 raft_consensus.cc:2749] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:42.025561 23616 raft_consensus.cc:1275] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:10:41.991149 29191 raft_consensus.cc:2749] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:10:42.076802 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 54946.2us
I20260709 13:10:42.084127 23020 raft_consensus.cc:1240] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 13:10:42.020738 29404 raft_consensus.cc:2804] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:10:42.087888 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:42.011377 29533 raft_consensus.cc:2804] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:10:42.017148 29507 raft_consensus.cc:2749] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:42.089287 22918 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:42.104859 23022 raft_consensus.cc:2468] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 20.
I20260709 13:10:42.118216 22874 raft_consensus.cc:2393] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 14: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:42.123333 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:42.124444 22875 raft_consensus.cc:2393] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 16: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:42.130204 23022 raft_consensus.cc:1275] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:10:42.135080 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:42.141865 23588 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:42.143850 29160 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:42.145545 29515 raft_consensus.cc:2749] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:10:42.146683 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:42.147603 22875 raft_consensus.cc:2393] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 22: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:42.148761 29485 consensus_queue.cc:1059] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:10:42.151520 22873 raft_consensus.cc:1275] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 13:10:42.161273 29529 log.cc:927] Time spent T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.179s	user 0.000s	sys 0.000s
I20260709 13:10:42.145375 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:42.029160 29237 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:42.166232 29404 raft_consensus.cc:697] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 18 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:42.167378 29404 consensus_queue.cc:237] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:42.017697 29518 consensus_queue.cc:1059] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.081s
I20260709 13:10:42.036705 29259 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:42.171177 29242 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:42.172374 29191 consensus_queue.cc:1059] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:10:42.148540 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:42.176501 29246 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:42.183091 29240 raft_consensus.cc:3055] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:10:42.078645 29532 raft_consensus.cc:2804] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:42.146339 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:42.185117 22873 raft_consensus.cc:3055] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:10:42.237120 22873 raft_consensus.cc:740] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:42.088186 28813 raft_consensus.cc:2804] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:10:42.139036 29533 raft_consensus.cc:697] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:42.132005 29525 consensus_queue.cc:1048] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.056s
I20260709 13:10:42.187938 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:42.139400 28898 raft_consensus.cc:2804] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:42.139812 29398 raft_consensus.cc:2749] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 13:10:42.202171 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 102562us
I20260709 13:10:42.253229 22920 leader_election.cc:400] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 election: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 19. Current term is 20.
I20260709 13:10:42.253567 22920 leader_election.cc:403] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:10:42.203680 22950 raft_consensus.cc:2468] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 19.
W20260709 13:10:42.254043 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 369624us
W20260709 13:10:42.144244 29538 log.cc:927] Time spent T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 13:10:42.141341 23590 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:42.209285 23624 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 20.
W20260709 13:10:42.210776 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 78908.5us
W20260709 13:10:42.214355 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 214675us
I20260709 13:10:42.214444 29518 raft_consensus.cc:3055] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:10:42.256690 22920 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:42.258903 23588 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:42.232633 29484 raft_consensus.cc:2804] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:10:42.252833 28813 raft_consensus.cc:697] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:42.231401 29240 raft_consensus.cc:740] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:42.233110 29259 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:42.268879 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 287764us
W20260709 13:10:42.261116 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 119856us
I20260709 13:10:42.263628 22992 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:42.243781 29340 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:42.252286 28898 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:42.318578 23590 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:42.232478 29532 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
W20260709 13:10:42.338075 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 81459.6us
I20260709 13:10:42.327034 22993 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
W20260709 13:10:42.328022 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 238750us
W20260709 13:10:42.330132 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 66573.8us
I20260709 13:10:42.265695 22873 consensus_queue.cc:260] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:42.339504 22918 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:42.248644 29422 consensus_queue.cc:1048] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.019s
I20260709 13:10:42.342660 22918 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:42.343304 22992 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:42.260123 29518 raft_consensus.cc:740] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:42.340910 29523 raft_consensus.cc:2749] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:10:42.265712 29484 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:42.351787 22873 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:42.359018 22918 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:42.367519 22920 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:42.338376 28898 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:10:42.371570 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 112732us
I20260709 13:10:42.342764 29340 raft_consensus.cc:2749] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:10:42.395429 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 134347us
W20260709 13:10:42.396467 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 53238.3us
I20260709 13:10:42.351315 29160 raft_consensus.cc:2804] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:10:42.342031 29532 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:42.372576 23588 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:42.349247 29246 raft_consensus.cc:2749] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:42.400245 22992 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:42.431134 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:42.358057 29239 raft_consensus.cc:2804] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:10:42.357626 29484 raft_consensus.cc:3060] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:42.371881 29266 raft_consensus.cc:2804] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 13:10:42.404937 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 58945.1us
I20260709 13:10:42.395767 29528 consensus_queue.cc:1059] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 13:10:42.400075 29520 raft_consensus.cc:2749] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:10:42.398679 28801 consensus_queue.cc:1048] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 13:10:42.420495 29546 raft_consensus.cc:3055] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:10:42.420931 29522 raft_consensus.cc:2749] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 19. Current term is 20.
W20260709 13:10:42.421732 29540 log.cc:927] Time spent T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
I20260709 13:10:42.422134 28813 consensus_queue.cc:237] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:42.424742 29533 consensus_queue.cc:237] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:42.425982 29240 consensus_queue.cc:260] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:42.426712 22843 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:42.357668 29488 raft_consensus.cc:2749] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:42.445693 29266 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:42.431742 23029 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 14.
I20260709 13:10:42.439734 22947 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 21.
I20260709 13:10:42.396556 29237 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:42.449370 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 49215.6us
I20260709 13:10:42.472817 22992 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:42.474922 22992 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:42.450026 29553 raft_consensus.cc:2749] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:10:42.435345 29239 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:42.462447 22873 raft_consensus.cc:1275] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 14. (term mismatch)
W20260709 13:10:42.464095 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 96713.6us
W20260709 13:10:42.468462 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 95979.4us
I20260709 13:10:42.444917 29160 raft_consensus.cc:697] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
W20260709 13:10:42.463754 29127 consensus_meta.cc:287] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.780s	user 0.009s	sys 0.002s
I20260709 13:10:42.480523 22873 pending_rounds.cc:85] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 13
W20260709 13:10:42.471108 29508 consensus_meta.cc:287] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.732s	user 0.010s	sys 0.001s
I20260709 13:10:42.485219 22873 pending_rounds.cc:107] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 19.14
I20260709 13:10:42.451434 29546 raft_consensus.cc:740] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:42.499095 22873 raft_consensus.cc:2889] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:42.511399 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:42.512028 22947 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:42.499629 29242 raft_consensus.cc:3055] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:10:42.500836 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 53213.3us
W20260709 13:10:42.483949 29408 consensus_meta.cc:287] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.602s	user 0.010s	sys 0.000s
I20260709 13:10:42.480650 29266 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:42.476516 29525 raft_consensus.cc:2804] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 13:10:42.512872 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:42.523322 29536 log.cc:927] Time spent T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.181s	user 0.000s	sys 0.001s
I20260709 13:10:42.484097 29239 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:42.479897 29246 raft_consensus.cc:2749] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 13:10:42.485153 29275 consensus_meta.cc:287] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.594s	user 0.006s	sys 0.005s
I20260709 13:10:42.520398 23590 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:42.484340 29506 raft_consensus.cc:2749] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:42.486662 29237 raft_consensus.cc:2804] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:10:42.530953 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 50782.8us
I20260709 13:10:42.497402 29527 raft_consensus.cc:2804] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:10:42.536916 22992 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:42.445338 29518 consensus_queue.cc:260] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:42.518935 29508 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 19. Current term is 20.
W20260709 13:10:42.533622 29389 consensus_meta.cc:287] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.641s	user 0.012s	sys 0.000s
I20260709 13:10:42.519702 29242 raft_consensus.cc:740] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:42.527868 29493 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:42.550659 29493 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:42.552783 29493 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:42.553345 28898 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:42.547103 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 49113.2us
I20260709 13:10:42.554397 23021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 2634) took 1016 ms. Trace:
I20260709 13:10:42.554522 22920 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:42.555502 28898 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:42.555790 22920 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:42.554785 23021 rpcz_store.cc:276] 0709 13:10:41.537572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:41.539057 (+  1485us) service_pool.cc:224] Handling call
0709 13:10:41.539373 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:41.741206 (+201833us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:10:41.853397 (+112191us) log.cc:844] Serialized 14 byte log entry
0709 13:10:42.259089 (+405692us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:42.259137 (+    48us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:10:42.259178 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:42.554051 (+294873us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b5400359b54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:42.554096 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:42.554218 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14575,"prepare.queue_time_us":80982,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":6222,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":5128,"spinlock_wait_cycles":459840128,"thread_start_us":269469,"threads_started":1,"wal-append.queue_time_us":540146}
I20260709 13:10:42.571888 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:42.549440 23590 leader_election.cc:400] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 19. Current term is 20.
I20260709 13:10:42.549775 29389 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:42.528968 29525 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:42.538105 29237 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:42.498747 29127 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:42.549129 22775 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33488 (request call id 269) took 39314 ms (client timeout 15000 ms). Trace:
I20260709 13:10:42.557410 29520 raft_consensus.cc:2749] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:10:42.581825 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 60170.2us
I20260709 13:10:42.582649 23026 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 18.
I20260709 13:10:42.559464 22920 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:42.566572 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:10:42.569059 29535 log.cc:927] Time spent T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.301s	user 0.001s	sys 0.000s
I20260709 13:10:42.576886 23590 leader_election.cc:403] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:10:42.537211 22776 catalog_manager.cc:5697] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 11 to 16, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 16 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:42.598389 23029 raft_consensus.cc:3060] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:42.600636 22951 raft_consensus.cc:1240] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16. Ops: []
I20260709 13:10:42.531054 29408 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:42.546957 29530 raft_consensus.cc:2804] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:10:42.600625 22947 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 22.
I20260709 13:10:42.603291 22950 raft_consensus.cc:1275] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:10:42.547178 29559 raft_consensus.cc:2749] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:42.605400 23025 raft_consensus.cc:1240] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 18. Current term is 19. Ops: []
W20260709 13:10:42.607075 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 47695.2us
I20260709 13:10:42.604839 28898 raft_consensus.cc:2749] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:10:42.609988 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:42.603331 29522 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:42.611003 22776 catalog_manager.cc:5697] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 14 to 17, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 17 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:42.612486 29408 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:42.613168 29546 consensus_queue.cc:260] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:42.613361 22776 catalog_manager.cc:5697] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f reported cstate change: config changed from index 6 to 11, term changed from 12 to 14, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3), VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New cstate: current_term: 14 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:42.614238 29546 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:42.615160 22776 catalog_manager.cc:5697] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:42.618209 29389 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:42.579506 29527 raft_consensus.cc:697] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 20 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:42.619560 29527 consensus_queue.cc:237] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:42.583644 29237 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:42.579874 29525 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:42.625984 23028 raft_consensus.cc:2393] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 19: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:42.604259 23588 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:42.616426 22776 catalog_manager.cc:5697] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 14 to 19. New cstate: current_term: 19 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:42.622383 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:42.607825 29530 raft_consensus.cc:697] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:42.583529 29518 raft_consensus.cc:3060] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:42.624464 23023 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 20.
I20260709 13:10:42.580456 29503 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:10:42.630656 23588 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:42.634130 22776 catalog_manager.cc:5697] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:42.642266 29242 consensus_queue.cc:260] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:42.633792 29555 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 19. Current term is 20.
I20260709 13:10:42.641119 29563 raft_consensus.cc:2804] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:10:42.641705 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 92393.4us
I20260709 13:10:42.647558 22776 catalog_manager.cc:5697] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 13 to 16, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 16 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:42.649523 23621 raft_consensus.cc:1240] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 21. Current term is 22. Ops: []
I20260709 13:10:42.649971 23028 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32794 (request call id 1119) took 1052 ms. Trace:
I20260709 13:10:42.651084 23020 raft_consensus.cc:1240] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 18. Current term is 19. Ops: []
I20260709 13:10:42.650337 23028 rpcz_store.cc:276] 0709 13:10:41.599752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:41.976640 (+376888us) service_pool.cc:224] Handling call
0709 13:10:42.649774 (+673134us) spinlock_profiling.cc:243] Waited 625 ms on lock 0x7b54002da5d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:42.649916 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1438099328}
I20260709 13:10:42.655849 23029 raft_consensus.cc:2468] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 17.
I20260709 13:10:42.656960 23590 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:42.658860 23624 raft_consensus.cc:1240] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 19. Current term is 20. Ops: [19.14-19.14]
I20260709 13:10:42.657712 22920 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:42.649853 29555 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:10:42.661954 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:42.663082 29555 raft_consensus.cc:697] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 22 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:42.663506 29530 consensus_queue.cc:237] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:42.664048 29555 consensus_queue.cc:237] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:42.665704 23022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 1106) took 1110 ms. Trace:
I20260709 13:10:42.665917 23621 raft_consensus.cc:1240] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 19. Current term is 20. Ops: []
I20260709 13:10:42.645925 29503 raft_consensus.cc:697] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 21 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:42.666114 23022 rpcz_store.cc:276] 0709 13:10:41.555337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:42.140421 (+585084us) service_pool.cc:224] Handling call
0709 13:10:42.213534 (+ 73113us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 13:10:42.247191 (+ 33657us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:42.247237 (+    46us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 13:10:42.253281 (+  6044us) log.cc:844] Serialized 78 byte log entry
0709 13:10:42.493602 (+240321us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:10:42.493661 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:42.493755 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:42.665495 (+171740us) raft_consensus.cc:1685] finished
0709 13:10:42.665513 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:42.665655 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33171,"raft.queue_time_us":37292,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":5034,"thread_start_us":129735,"threads_started":1,"wal-append.queue_time_us":317592}
I20260709 13:10:42.587013 29160 consensus_queue.cc:237] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:42.682413 22919 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:42.670091 22920 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:42.675513 23616 raft_consensus.cc:1275] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:10:42.678056 29127 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:42.680230 29558 log.cc:927] Time spent T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.126s	user 0.004s	sys 0.000s
I20260709 13:10:42.647939 29563 raft_consensus.cc:697] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 19 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:42.668316 22776 catalog_manager.cc:5697] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 16 to 17, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 17 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:42.692103 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:42.694341 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:42.696231 22872 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 20.
I20260709 13:10:42.693306 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:42.670855 29485 raft_consensus.cc:2749] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:10:42.704515 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:42.669520 28898 raft_consensus.cc:2749] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:42.702904 22776 catalog_manager.cc:5697] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 11 to 19. New cstate: current_term: 19 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:42.689893 29221 raft_consensus.cc:2804] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:10:42.706782 22776 catalog_manager.cc:5697] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 17 to 18, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 18 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:42.732229 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:42.731940 22776 catalog_manager.cc:5697] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 12 to 18, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 18 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:42.733834 22776 catalog_manager.cc:5697] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 14 to 16, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 16 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:42.711359 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:42.735821 23028 raft_consensus.cc:2393] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 16: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:42.711928 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:42.712106 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:42.697321 29526 consensus_queue.cc:1048] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.017s
I20260709 13:10:42.719719 29239 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:42.729681 23020 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 2647) took 1186 ms. Trace:
I20260709 13:10:42.694042 29517 raft_consensus.cc:2804] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:10:42.732965 23624 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 17.
I20260709 13:10:42.707604 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:42.746588 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:10:42.739202 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:42.740420 22952 raft_consensus.cc:2410] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:10:42.742213 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:42.742978 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:42.743721 23020 rpcz_store.cc:276] 0709 13:10:41.543451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:42.118933 (+575482us) service_pool.cc:224] Handling call
0709 13:10:42.650929 (+531996us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:42.652183 (+  1254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:42.729428 (+ 77245us) spinlock_profiling.cc:243] Waited 498 ms on lock 0x7b54002da5d0. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:42.729604 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1145070464}
I20260709 13:10:42.713258 29127 consensus_queue.cc:1048] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 13:10:42.763803 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:42.719769 29483 raft_consensus.cc:3055] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:10:42.765270 23618 raft_consensus.cc:2393] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 19: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
W20260709 13:10:42.762207 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 74013.5us
I20260709 13:10:42.768276 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:42.767604 22920 leader_election.cc:400] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 19. Current term is 20.
I20260709 13:10:42.763685 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:42.766043 23590 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:42.776032 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:42.791181 23619 raft_consensus.cc:2393] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 20: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
W20260709 13:10:42.778961 22826 auto_rebalancer.cc:629] Failed to schedule move for tablet b41a09964104486c8e50b539ec2b16f6: Illegal state: Leader has not yet committed an operation in its own term
I20260709 13:10:42.779757 29503 consensus_queue.cc:237] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:42.792326 29485 raft_consensus.cc:2804] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:10:42.792816 29485 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:42.793150 29485 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:42.795549 29532 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:42.796386 23621 raft_consensus.cc:1275] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:10:42.752856 29506 consensus_queue.cc:1048] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
W20260709 13:10:42.580636 22775 rpcz_store.cc:269] 0709 13:10:03.163153 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:03.163400 (+   247us) service_pool.cc:224] Handling call
0709 13:10:42.468868 (+39305468us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:10:34.800870 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:10:34.801768 (+   898us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:10:34.801832 (+    64us) write_op.cc:435] Acquired schema lock
0709 13:10:34.801857 (+    25us) tablet.cc:662] Decoding operations
0709 13:10:34.802916 (+  1059us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:10:34.803024 (+   108us) write_op.cc:641] Partition lock acquired for write op
0709 13:10:34.803056 (+    32us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:10:34.803954 (+   898us) tablet.cc:701] Row locks acquired
0709 13:10:34.804007 (+    53us) write_op.cc:260] PREPARE: finished
0709 13:10:34.804183 (+   176us) write_op.cc:270] Start()
0709 13:10:34.804324 (+   141us) write_op.cc:276] Timestamp: P: 1783602634804123 usec, L: 0
0709 13:10:34.804345 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 13:10:34.935403 (+131058us) log.cc:844] Serialized 3934 byte log entry
0709 13:10:35.621101 (+685698us) op_driver.cc:464] REPLICATION: finished
0709 13:10:35.785122 (+164021us) write_op.cc:301] APPLY: starting
0709 13:10:35.789353 (+  4231us) tablet.cc:1367] starting BulkCheckPresence
0709 13:10:35.911999 (+122646us) tablet.cc:1370] finished BulkCheckPresence
0709 13:10:35.912106 (+   107us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:10:36.026612 (+114506us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:10:36.026852 (+   240us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=20,mrs_lookups=0
0709 13:10:36.026896 (+    44us) write_op.cc:312] APPLY: finished
0709 13:10:36.027196 (+   300us) log.cc:844] Serialized 97 byte log entry
0709 13:10:36.034835 (+  7639us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:36.034969 (+   134us) write_op.cc:454] Released schema lock
0709 13:10:36.035030 (+    61us) write_op.cc:341] FINISH: Updating metrics
0709 13:10:36.035585 (+   555us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:36.035615 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2004475,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":135005,"apply.run_cpu_time_us":18368,"apply.run_wall_time_us":271335,"cfile_cache_hit":42,"cfile_cache_hit_bytes":45947,"num_ops":10,"prepare.queue_time_us":75684,"prepare.run_cpu_time_us":38344,"prepare.run_wall_time_us":387920,"raft.queue_time_us":197673,"raft.run_cpu_time_us":33310,"raft.run_wall_time_us":60800,"replication_time_us":816605,"thread_start_us":440068,"threads_started":4,"wal-append.queue_time_us":472352,"wal-append.run_cpu_time_us":43228,"wal-append.run_wall_time_us":1616535}]]}
I20260709 13:10:42.767923 29483 raft_consensus.cc:740] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:42.781114 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:42.783771 23588 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:42.802726 23588 leader_election.cc:304] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:42.803488 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:42.769032 22776 catalog_manager.cc:5697] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 6 to 13, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 13 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:42.805436 22776 catalog_manager.cc:5697] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 12 to 14, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 14 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:42.807160 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:42.807651 22993 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:42.752678 29389 consensus_queue.cc:1059] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 13:10:42.817009 23588 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:42.807170 29340 raft_consensus.cc:2749] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:42.808847 29559 raft_consensus.cc:2749] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:10:42.829106 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:42.814937 22776 catalog_manager.cc:5697] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: config changed from index 5 to 9, term changed from 11 to 17, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New cstate: current_term: 17 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:42.845219 22776 catalog_manager.cc:5697] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:42.831903 23588 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:42.851814 23020 raft_consensus.cc:2393] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 18: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:42.818466 29506 raft_consensus.cc:2749] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:10:42.856159 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:42.824967 29422 consensus_queue.cc:1048] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:10:42.809957 29563 consensus_queue.cc:237] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:42.861800 29515 consensus_queue.cc:1048] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.005s
I20260709 13:10:42.855167 29483 consensus_queue.cc:260] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:42.867147 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:42.861984 29160 consensus_queue.cc:1048] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.006s
I20260709 13:10:42.787238 29525 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:42.860400 22776 catalog_manager.cc:5697] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 10 to 15, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 15 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:42.874017 22871 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 3218) took 1373 ms. Trace:
I20260709 13:10:42.874533 22871 rpcz_store.cc:276] 0709 13:10:41.500321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:41.753245 (+252924us) service_pool.cc:224] Handling call
0709 13:10:41.753728 (+   483us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:41.754177 (+   449us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:41.754222 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:41.754932 (+   710us) log.cc:844] Serialized 26 byte log entry
0709 13:10:42.058391 (+303459us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:10:42.099896 (+ 41505us) log.cc:844] Serialized 14 byte log entry
0709 13:10:42.356611 (+256715us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:42.356710 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:42.863542 (+506832us) spinlock_profiling.cc:243] Waited 22.7 ms on lock 0x7b24001210e8. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132b7e9c4 00007f6132b7cacb 00007f6132d37dc9 00007f6132d37a55 00007f6132d36fb0 00007f61359ad488 00007f61359adbe8 00007f6135985398 00007f613598076a 00007f613597fbb6
0709 13:10:42.869400 (+  5858us) raft_consensus.cc:1685] finished
0709 13:10:42.873758 (+  4358us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:42.873944 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":41682,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":1701,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":35165,"spinlock_wait_cycles":52183296,"thread_start_us":551174,"threads_started":2,"wal-append.queue_time_us":494241}
I20260709 13:10:42.853286 29557 raft_consensus.cc:2749] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 13:10:42.893556 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 122447us
I20260709 13:10:42.863057 29532 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:42.898391 23625 raft_consensus.cc:1240] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14. Ops: []
I20260709 13:10:42.899116 22776 catalog_manager.cc:5697] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 11 to 13, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 13 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:42.879376 29522 raft_consensus.cc:2749] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:42.905112 22776 catalog_manager.cc:5697] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 8 to 16. New cstate: current_term: 16 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:42.880379 29160 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:42.880564 29523 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:42.908856 22920 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:42.913020 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 106180us
I20260709 13:10:42.919157 22776 catalog_manager.cc:5697] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:42.901405 29221 raft_consensus.cc:697] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:42.908140 29517 raft_consensus.cc:697] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
W20260709 13:10:42.929637 29575 log.cc:927] Time spent T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:10:42.930850 29266 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:42.934556 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:42.935225 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:42.935868 29517 consensus_queue.cc:237] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:42.936883 29556 raft_consensus.cc:2749] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:10:42.937050 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 510328us
W20260709 13:10:42.938392 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 160662us
I20260709 13:10:42.939693 22843 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:42.940121 29259 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:42.940812 29408 consensus_queue.cc:1059] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:10:42.941072 29547 consensus_queue.cc:1059] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:10:42.923866 22776 catalog_manager.cc:5697] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: config changed from index 6 to 9, term changed from 10 to 12, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to <none>, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER, no longer has a pending config: VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1), VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2), VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3), VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 12 committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:42.941709 29408 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:42.942370 29408 raft_consensus.cc:3055] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:10:42.942440 29547 raft_consensus.cc:3055] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:10:42.942791 29408 raft_consensus.cc:740] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:42.943087 29547 raft_consensus.cc:740] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:42.943347 29221 consensus_queue.cc:237] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:42.943822 29408 consensus_queue.cc:260] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:42.942847 22776 catalog_manager.cc:5697] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 7 to 16, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 16 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:42.919659 29523 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:42.949218 22871 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
mode: GRACEFUL
new_leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:40244
I20260709 13:10:42.950004 22871 raft_consensus.cc:606] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Received request to transfer leadership to TS aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:42.950383 22871 raft_consensus.cc:614] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:10:42.940814 29259 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:42.950860 29523 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:42.951503 29266 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:42.952656 29545 consensus_queue.cc:1048] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:10:42.923151 29239 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:42.954468 22844 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:42.954671 29522 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:10:42.954800 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 45998us
I20260709 13:10:42.955626 22844 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:42.952704 29259 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:42.958261 29408 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:10:42.961412 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:42.955658 22920 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:42.968323 22776 catalog_manager.cc:5697] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 14 to 17, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 17 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:42.970110 29511 raft_consensus.cc:2749] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:10:42.987519 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 47918.4us
I20260709 13:10:42.995404 22920 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:42.971860 29556 raft_consensus.cc:2804] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:10:43.008301 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 175777us
I20260709 13:10:43.009720 22992 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:43.019344 23588 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:42.987130 29523 raft_consensus.cc:2749] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:43.020067 22776 catalog_manager.cc:5697] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 13 to 17, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 17 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
W20260709 13:10:43.006909 29484 consensus_meta.cc:287] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.555s	user 0.008s	sys 0.002s
W20260709 13:10:43.022073 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 66513.1us
I20260709 13:10:42.922240 29160 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:43.024911 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:43.027097 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:43.005184 29340 raft_consensus.cc:2749] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:10:43.027911 22878 raft_consensus.cc:1240] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 19. Current term is 20. Ops: []
I20260709 13:10:43.005518 29518 raft_consensus.cc:2749] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:43.029182 22843 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:43.029428 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:43.029162 23588 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:43.025144 22776 catalog_manager.cc:5697] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 11 to 17. New cstate: current_term: 17 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:43.031505 22873 raft_consensus.cc:2393] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 17: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:42.944182 29547 consensus_queue.cc:260] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:43.016992 29556 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:43.032770 22872 raft_consensus.cc:2393] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 20: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:43.033850 22776 catalog_manager.cc:5697] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 12 to 14, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 14 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:43.035280 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:43.039515 29543 raft_consensus.cc:2749] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:10:43.039678 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:43.038228 29557 raft_consensus.cc:2749] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:10:43.041649 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:43.041348 22776 catalog_manager.cc:5697] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:43.027253 29389 raft_consensus.cc:2749] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:43.047350 22872 raft_consensus.cc:1240] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:10:43.046602 22776 catalog_manager.cc:5697] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 12 to 15, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 15 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
W20260709 13:10:43.049161 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:43.028853 29563 raft_consensus.cc:2749] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:43.058435 22843 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:43.064291 23588 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:43.039162 29556 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:43.067164 23588 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:43.028836 29484 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:43.066331 29557 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:43.071964 29525 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:43.072651 29484 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:43.072870 23618 raft_consensus.cc:1240] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 13. Current term is 14. Ops: []
I20260709 13:10:43.080577 23024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 2635) took 1542 ms. Trace:
I20260709 13:10:43.081032 23024 rpcz_store.cc:276] 0709 13:10:41.537878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:42.019668 (+481790us) service_pool.cc:224] Handling call
0709 13:10:42.042507 (+ 22839us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:42.136763 (+ 94256us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:42.136793 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:42.140141 (+  3348us) log.cc:844] Serialized 26 byte log entry
0709 13:10:42.453627 (+313486us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:10:42.500039 (+ 46412us) log.cc:844] Serialized 14 byte log entry
0709 13:10:42.513702 (+ 13663us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:42.643110 (+129408us) spinlock_profiling.cc:243] Waited 76.7 ms on lock 0x7b5400358254. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:42.643163 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:43.074671 (+431508us) spinlock_profiling.cc:243] Waited 3.09 ms on lock 0x7b18003b9200. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cf83f3 00007f6132cfa7c1 00007f6132cfa779 00007f6132cfa709 00007f6132cfa6d1 00007f6132cf99cd 000000000038f354 000000000038f199 00007f6132cdb83a
0709 13:10:43.080312 (+  5641us) raft_consensus.cc:1685] finished
0709 13:10:43.080338 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:43.080487 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30399,"prepare.queue_time_us":159088,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":35068,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":11716,"spinlock_wait_cycles":183446272,"thread_start_us":294422,"threads_started":2,"wal-append.queue_time_us":350982}
I20260709 13:10:43.083449 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:10:43.083722 29544 log.cc:927] Time spent T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.574s	user 0.000s	sys 0.003s
I20260709 13:10:43.084041 23024 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:43.085986 29160 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:43.089828 22872 raft_consensus.cc:1275] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 13:10:43.090065 29150 raft_consensus.cc:3055] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:10:43.090504 29150 raft_consensus.cc:740] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:43.091403 29150 consensus_queue.cc:260] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:43.092111 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:43.092980 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:43.094611 22880 raft_consensus.cc:1240] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16. Ops: []
I20260709 13:10:43.098380 22918 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:43.101841 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:43.105113 22877 raft_consensus.cc:2410] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 13:10:43.105736 22878 raft_consensus.cc:1275] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:10:43.105800 23588 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:43.112564 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:43.065320 29547 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:10:43.123344 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 59661.4us
I20260709 13:10:43.130285 22994 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:43.110390 22918 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:43.092583 29150 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:43.173854 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:43.174405 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 68718.9us
I20260709 13:10:43.174641 23024 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18.
I20260709 13:10:43.175007 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:43.177307 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:43.179770 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:43.181119 22877 raft_consensus.cc:1240] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:43.123041 29559 raft_consensus.cc:2749] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:10:43.186600 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:43.147238 29160 raft_consensus.cc:2804] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:10:43.188834 23616 raft_consensus.cc:2393] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 18: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:43.152329 29389 raft_consensus.cc:2749] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:10:43.189136 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:43.190689 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:43.193670 23024 raft_consensus.cc:3060] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:43.190486 29577 raft_consensus.cc:2804] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 13:10:43.138072 29237 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:43.178658 29484 raft_consensus.cc:2749] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:43.181681 29564 consensus_queue.cc:1048] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.088s
I20260709 13:10:43.204702 29569 consensus_queue.cc:1059] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 13:10:43.213910 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:43.221477 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:43.222620 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:43.223384 23022 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 16.
I20260709 13:10:43.223979 23623 raft_consensus.cc:1240] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 17. Current term is 18. Ops: []
I20260709 13:10:43.185436 29517 consensus_queue.cc:1059] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.025s
I20260709 13:10:43.197310 29577 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:43.237269 23024 raft_consensus.cc:2468] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 14.
I20260709 13:10:43.202189 29160 raft_consensus.cc:697] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:43.241794 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:43.242341 29160 consensus_queue.cc:237] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:43.243090 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:43.243853 23024 raft_consensus.cc:2393] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 20: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:43.244591 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:43.252254 22875 raft_consensus.cc:1275] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 12. (term mismatch)
I20260709 13:10:43.252713 22875 pending_rounds.cc:85] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 11
I20260709 13:10:43.252974 22875 pending_rounds.cc:107] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 19.12
I20260709 13:10:43.253239 22875 raft_consensus.cc:2889] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:43.253571 22875 pending_rounds.cc:107] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 21.13
I20260709 13:10:43.253897 22875 raft_consensus.cc:2889] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:43.258404 29237 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:43.261063 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:43.266220 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:43.236902 29577 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:43.269613 29484 consensus_queue.cc:1059] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.003s
I20260709 13:10:43.274863 29522 raft_consensus.cc:2749] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:10:43.217031 29485 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:43.276883 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:43.277354 29485 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 21 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:43.277489 23029 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:43.289001 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:43.289964 23022 raft_consensus.cc:2393] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 19: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:43.291103 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:43.293283 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:43.293803 23624 raft_consensus.cc:1240] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 13:10:43.292567 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:43.299527 23022 raft_consensus.cc:2368] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 17. Current term is 18.
I20260709 13:10:43.306522 23029 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 16.
W20260709 13:10:43.310418 29589 log.cc:927] Time spent T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 13:10:43.310829 23025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 2642) took 1771 ms. Trace:
I20260709 13:10:43.312065 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:43.313359 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:43.314359 29557 raft_consensus.cc:2749] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:43.319370 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:43.324572 23625 raft_consensus.cc:1275] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 13:10:43.355602 23589 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:43.357631 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 157383us
I20260709 13:10:43.361258 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:43.377242 23029 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:43.367887 23025 rpcz_store.cc:276] 0709 13:10:41.539767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:42.022739 (+482972us) service_pool.cc:224] Handling call
0709 13:10:42.023105 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:42.610546 (+587441us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:43.310539 (+699993us) spinlock_profiling.cc:243] Waited 581 ms on lock 0x7b5400358754. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:43.310761 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1336978176}
I20260709 13:10:43.379112 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:43.392463 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:43.393234 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:43.394308 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:43.394955 23028 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 16
I20260709 13:10:43.396391 22843 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:43.406275 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:43.406708 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:43.407487 22994 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:43.408341 23622 raft_consensus.cc:1240] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 18. Current term is 19. Ops: []
I20260709 13:10:43.409224 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:43.415896 22843 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:43.418529 23022 raft_consensus.cc:2393] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 17: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
W20260709 13:10:43.424858 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 47557.3us
I20260709 13:10:43.367877 29259 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:43.427030 23029 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 20.
I20260709 13:10:43.429033 23027 raft_consensus.cc:1240] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 19. Current term is 20. Ops: []
W20260709 13:10:43.428573 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 243166us
W20260709 13:10:43.441462 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 109706us
I20260709 13:10:43.414232 29191 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:43.420470 29564 raft_consensus.cc:2749] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:43.420167 29543 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:43.454541 23588 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:43.420089 29511 raft_consensus.cc:2749] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:43.473212 23022 raft_consensus.cc:1240] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 13:10:43.474597 22951 raft_consensus.cc:1275] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 13:10:43.477298 22955 raft_consensus.cc:1275] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:10:43.477797 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:43.479261 23588 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:43.433465 29259 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:43.487179 29259 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:43.487783 29259 raft_consensus.cc:2804] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:10:43.488775 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:43.489259 29483 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:43.488209 29259 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:43.492581 29608 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 13:10:43.493007 29259 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:43.494714 29574 raft_consensus.cc:3055] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:10:43.495182 29574 raft_consensus.cc:740] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:43.500015 22948 raft_consensus.cc:1275] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 16. (term mismatch)
I20260709 13:10:43.428246 29389 consensus_queue.cc:1048] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 13:10:43.494602 22993 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:43.509690 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:10:43.500301 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 378293us
I20260709 13:10:43.519802 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 21 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:43.505376 22953 raft_consensus.cc:1275] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:10:43.500512 22948 pending_rounds.cc:85] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting all ops after (but not including) 15
W20260709 13:10:43.501955 23023 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33420 (request call id 2651) took 1865 ms (client timeout 1704 ms). Trace:
I20260709 13:10:43.522673 22948 pending_rounds.cc:107] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7: Aborting uncommitted NO_OP operation due to leader change: 19.16
I20260709 13:10:43.523229 22948 raft_consensus.cc:2889] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:10:43.522758 23023 rpcz_store.cc:269] 0709 13:10:41.636573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:41.690280 (+ 53707us) service_pool.cc:224] Handling call
0709 13:10:43.501679 (+1811399us) spinlock_profiling.cc:243] Waited 869 ms on lock 0x7b54001f72d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:43.501830 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1998665472}
I20260709 13:10:43.535593 23023 raft_consensus.cc:1275] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:10:43.531877 22872 raft_consensus.cc:3055] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:10:43.532411 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 14 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:43.549981 22872 raft_consensus.cc:740] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:43.536089 22920 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:43.537659 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 58517.4us
I20260709 13:10:43.545116 22952 raft_consensus.cc:1275] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:10:43.530673 29564 raft_consensus.cc:2749] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:10:43.529603 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:43.551779 23588 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:43.470947 29191 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:43.556713 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:43.547107 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 129968us
I20260709 13:10:43.542508 29563 raft_consensus.cc:2749] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:43.559348 23588 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:43.566308 23028 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 16.
I20260709 13:10:43.531524 29422 consensus_queue.cc:1059] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 13:10:43.577452 29259 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:43.577950 29191 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:43.578480 29191 raft_consensus.cc:3055] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:10:43.578817 29191 raft_consensus.cc:740] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:43.579517 29259 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:43.579599 29191 consensus_queue.cc:260] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:43.559317 22843 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:43.573803 22878 raft_consensus.cc:1275] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:10:43.564953 29483 raft_consensus.cc:2749] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:10:43.585853 23023 raft_consensus.cc:1240] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 15. Current term is 16. Ops: []
I20260709 13:10:43.590040 23588 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:43.596598 23588 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:43.597671 29599 raft_consensus.cc:2749] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:10:43.599143 23621 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab"
dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:34202
I20260709 13:10:43.599750 23621 raft_consensus.cc:606] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:43.600101 23621 raft_consensus.cc:614] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:10:43.537889 29503 consensus_queue.cc:1059] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.023s
W20260709 13:10:43.604099 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 44839.1us
I20260709 13:10:43.601994 23028 raft_consensus.cc:1240] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 13:10:43.607699 22843 leader_election.cc:400] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 17. Current term is 18.
I20260709 13:10:43.607671 23023 raft_consensus.cc:1240] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 13:10:43.608417 22843 leader_election.cc:403] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 13:10:43.587405 29275 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:43.630678 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:43.592262 29608 raft_consensus.cc:2749] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:10:43.644194 22952 raft_consensus.cc:1240] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:43.642830 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:43.637800 23023 raft_consensus.cc:1240] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 13. Current term is 14. Ops: []
I20260709 13:10:43.601929 29557 raft_consensus.cc:2804] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 13:10:43.604444 29191 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:43.634579 22872 consensus_queue.cc:260] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:43.587136 28813 consensus_queue.cc:1059] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 13:10:43.607417 29408 raft_consensus.cc:2749] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:10:43.656374 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:43.646785 29606 log.cc:927] Time spent T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 13:10:43.671594 22952 raft_consensus.cc:1240] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 20. Current term is 21. Ops: []
W20260709 13:10:43.663674 23588 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.861s (SENT) [suppressed 42 similar messages]
W20260709 13:10:43.665407 29605 log.cc:927] Time spent T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
W20260709 13:10:43.666236 29609 log.cc:927] Time spent T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 13:10:43.666916 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:43.683073 22953 raft_consensus.cc:1240] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 15. Current term is 16. Ops: []
I20260709 13:10:43.671001 22872 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:43.662863 23023 raft_consensus.cc:1275] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:10:43.676844 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:43.685897 22949 raft_consensus.cc:2393] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 18: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:43.687558 22947 raft_consensus.cc:1240] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 17. Current term is 18. Ops: []
I20260709 13:10:43.677750 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:43.689054 22948 raft_consensus.cc:2410] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 20 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:10:43.686497 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:43.693264 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:43.636396 29577 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:43.638536 29275 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:43.673645 29557 raft_consensus.cc:697] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:43.681082 28801 raft_consensus.cc:2749] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:10:43.706238 23026 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:32794 (request call id 1131) took 1871 ms (client timeout 1861 ms). Trace:
W20260709 13:10:43.706804 23026 rpcz_store.cc:269] 0709 13:10:41.834335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:41.854936 (+ 20601us) service_pool.cc:224] Handling call
0709 13:10:43.706065 (+1851129us) spinlock_profiling.cc:243] Waited 668 ms on lock 0x7b54001c84d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:43.706176 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1537258880}
W20260709 13:10:43.708433 29611 log.cc:927] Time spent T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 13:10:43.691255 29503 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:43.712184 22955 raft_consensus.cc:1240] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 LEADER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 19. Current term is 20. Ops: []
I20260709 13:10:43.715095 23021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33420 (request call id 2654) took 1348 ms. Trace:
W20260709 13:10:43.715350 29603 log.cc:927] Time spent T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 13:10:43.715533 23021 rpcz_store.cc:276] 0709 13:10:42.366885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:42.589104 (+222219us) service_pool.cc:224] Handling call
0709 13:10:43.714858 (+1125754us) spinlock_profiling.cc:243] Waited 264 ms on lock 0x7b54000bc7d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:43.715034 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":606520576}
I20260709 13:10:43.719115 23022 raft_consensus.cc:1240] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 21 LEADER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 20. Current term is 21. Ops: []
I20260709 13:10:43.730288 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:43.734930 29275 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:43.735477 22874 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f603724630e44ef78014e7719f096cb3"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
mode: GRACEFUL
new_leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:40244
I20260709 13:10:43.736115 22874 raft_consensus.cc:606] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 17 LEADER]: Received request to transfer leadership to TS aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:43.722239 29520 raft_consensus.cc:3055] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:10:43.744854 22946 raft_consensus.cc:1240] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 13:10:43.721765 29503 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:43.745611 29520 raft_consensus.cc:740] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:43.760229 22955 raft_consensus.cc:1240] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 LEADER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 21. Current term is 22. Ops: [19.12-21.13]
W20260709 13:10:43.760753 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 206266us
I20260709 13:10:43.761199 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:43.762986 22952 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 19.
I20260709 13:10:43.771548 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:43.770449 29275 consensus_queue.cc:1048] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.006s
I20260709 13:10:43.774362 22954 raft_consensus.cc:1240] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 13:10:43.779662 29569 consensus_queue.cc:1048] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:10:43.780818 23588 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:43.766776 29127 consensus_queue.cc:1048] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.014s
I20260709 13:10:43.780473 28813 consensus_queue.cc:1059] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:10:43.754715 29557 consensus_queue.cc:237] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:43.788234 28813 raft_consensus.cc:3055] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:10:43.788615 28813 raft_consensus.cc:740] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:43.789364 28813 consensus_queue.cc:260] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:43.790505 28813 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:43.791129 29563 consensus_queue.cc:1048] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:10:43.792625 29242 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:43.793355 29601 raft_consensus.cc:2749] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:10:43.785038 29527 consensus_queue.cc:1048] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 13:10:43.794423 23618 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 1212) took 1021 ms. Trace:
I20260709 13:10:43.800001 23029 raft_consensus.cc:1275] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 20 index: 10. (index mismatch)
I20260709 13:10:43.802474 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:10:43.804641 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:43.805231 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:43.800238 23618 rpcz_store.cc:276] 0709 13:10:42.772824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:42.776775 (+  3951us) service_pool.cc:224] Handling call
0709 13:10:42.777226 (+   451us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:43.073269 (+296043us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:43.794176 (+720907us) spinlock_profiling.cc:243] Waited 295 ms on lock 0x7b54003d90d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:43.794356 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":679394048}
I20260709 13:10:43.805838 22918 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:43.768739 29617 consensus_queue.cc:1048] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:10:43.827662 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:43.837046 28813 raft_consensus.cc:2804] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:10:43.838915 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:43.842343 22955 raft_consensus.cc:2393] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 16: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:43.831072 23021 raft_consensus.cc:2376] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 20. Re-sending same reply.
W20260709 13:10:43.820401 29556 consensus_meta.cc:287] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.696s	user 0.009s	sys 0.003s
W20260709 13:10:43.846841 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:43.829972 29621 log.cc:927] Time spent T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
W20260709 13:10:43.901309 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:43.906213 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:43.931792 29620 log.cc:927] Time spent T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.153s	user 0.000s	sys 0.000s
I20260709 13:10:43.933221 23021 raft_consensus.cc:1240] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 18. Current term is 19. Ops: [18.17-18.17]
I20260709 13:10:43.855983 29556 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:43.935693 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:10:43.946617 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:43.954427 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:43.970803 22954 raft_consensus.cc:2393] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 20: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:43.974150 22876 raft_consensus.cc:1240] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 17. Current term is 18. Ops: []
I20260709 13:10:43.971453 23028 raft_consensus.cc:1275] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:10:43.932967 29577 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:43.855237 28813 raft_consensus.cc:697] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
W20260709 13:10:43.985189 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 179438us
I20260709 13:10:43.985670 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:43.986218 22918 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:43.989830 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:43.990464 22873 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 18.
I20260709 13:10:43.815743 29520 consensus_queue.cc:260] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 18.17, Last appended by leader: 17, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:43.992466 28813 consensus_queue.cc:237] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:43.997736 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:44.000029 22918 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:43.996960 29622 log.cc:927] Time spent T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.250s	user 0.000s	sys 0.001s
I20260709 13:10:44.000180 29404 raft_consensus.cc:3055] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:10:44.008875 22918 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:44.001399 29520 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:44.012928 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:44.013300 29518 raft_consensus.cc:2804] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:10:44.014770 22880 raft_consensus.cc:1275] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:10:44.001873 29631 raft_consensus.cc:2749] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:44.021723 29569 consensus_queue.cc:1059] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:10:44.012302 29404 raft_consensus.cc:740] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:44.032346 29574 consensus_queue.cc:260] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:44.033730 29574 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:44.033401 29404 consensus_queue.cc:260] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:44.037786 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:44.042091 22880 raft_consensus.cc:1240] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 18. Current term is 19. Ops: []
I20260709 13:10:44.042127 22951 raft_consensus.cc:1275] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:10:44.042757 22918 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:44.043807 28898 raft_consensus.cc:2749] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:44.045828 22878 raft_consensus.cc:3055] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:10:44.046391 22878 raft_consensus.cc:740] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:44.022210 29630 consensus_queue.cc:1059] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.003s
I20260709 13:10:44.045921 22918 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:44.026451 29518 raft_consensus.cc:697] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:44.054018 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:44.048168 29625 log.cc:927] Time spent T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.196s	user 0.001s	sys 0.000s
I20260709 13:10:44.054687 22878 consensus_queue.cc:260] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:44.047888 29404 raft_consensus.cc:3060] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:44.062090 29630 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:10:44.062705 29630 raft_consensus.cc:697] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:44.063871 29581 raft_consensus.cc:3055] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:10:44.064445 29581 raft_consensus.cc:740] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:44.063779 29630 consensus_queue.cc:237] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:44.065171 29518 consensus_queue.cc:237] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:44.065394 29581 consensus_queue.cc:260] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:44.066720 29581 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:44.066792 29503 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:44.069131 22878 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:44.073879 29629 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:44.075911 23618 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f"
dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
mode: GRACEFUL
new_leader_uuid: "f3c799b1249e40119d2a5864231c6c75"
 from {username='slave'} at 127.0.0.1:34202
W20260709 13:10:44.068102 29569 consensus_queue.cc:1175] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "0a407504ed1741bf89a276558adf4c0f" responder_term: 18 status { last_received { term: 18 index: 13 } last_committed_idx: 13 last_received_current_leader { term: 18 index: 13 } }
I20260709 13:10:44.076539 23618 raft_consensus.cc:606] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Received request to transfer leadership to TS f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:44.077332 23618 raft_consensus.cc:614] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:10:44.078691 21756 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:10:44.047658 29631 consensus_queue.cc:1048] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 13:10:44.079222 29631 consensus_queue.cc:1243] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer f3c799b1249e40119d2a5864231c6c75 log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 13:10:44.080653 21756 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:10:44.057819 29533 consensus_queue.cc:1048] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.023s
I20260709 13:10:44.081346 29556 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:44.087379 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:44.048964 28898 consensus_queue.cc:1048] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 13:10:44.092952 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:44.092108 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:44.093807 23022 raft_consensus.cc:2376] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 18. Re-sending same reply.
I20260709 13:10:44.095188 23588 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:44.122924 22951 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 18.
I20260709 13:10:44.124181 22994 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:10:44.126281 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 80442.4us
I20260709 13:10:44.129174 29617 raft_consensus.cc:2804] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:44.129624 29617 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:44.130009 29617 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:10:44.093328 29628 log.cc:927] Time spent T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 13:10:44.139027 29631 consensus_queue.cc:1048] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:10:44.148023 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:44.149262 22878 raft_consensus.cc:1275] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 12. (term mismatch)
I20260709 13:10:44.151882 22878 pending_rounds.cc:85] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 11
I20260709 13:10:44.152266 22878 pending_rounds.cc:107] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260709 13:10:44.152601 22878 raft_consensus.cc:2889] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:44.154543 23022 raft_consensus.cc:2393] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 16: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:44.127089 29608 raft_consensus.cc:2804] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:10:44.171789 23029 pending_rounds.cc:85] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75: Aborting all ops after (but not including) 11
I20260709 13:10:44.172258 23029 pending_rounds.cc:107] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 13:10:44.172567 23029 raft_consensus.cc:2889] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:44.189574 29237 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:10:44.189855 29626 log.cc:927] Time spent T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.198s	user 0.003s	sys 0.001s
I20260709 13:10:44.190428 29237 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:44.179836 29633 log.cc:927] Time spent T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 13:10:44.196226 23025 raft_consensus.cc:1275] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:10:44.207350 23619 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 1356) took 1294 ms. Trace:
I20260709 13:10:44.209578 28813 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:44.210314 28813 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:44.163326 29493 consensus_queue.cc:1048] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 13:10:44.178154 28801 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:10:44.214687 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:44.207911 23619 rpcz_store.cc:276] 0709 13:10:42.913181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:43.777507 (+864326us) service_pool.cc:224] Handling call
0709 13:10:43.777832 (+   325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:43.778186 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:43.778216 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:43.779283 (+  1067us) log.cc:844] Serialized 26 byte log entry
0709 13:10:43.859521 (+ 80238us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:10:43.859578 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:43.859672 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:44.189048 (+329376us) spinlock_profiling.cc:243] Waited 3.28 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:10:44.198769 (+  9721us) raft_consensus.cc:1685] finished
0709 13:10:44.198802 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:44.207264 (+  8462us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":94,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":76,"spinlock_wait_cycles":7545728,"thread_start_us":79785,"threads_started":1,"wal-append.queue_time_us":183608}
I20260709 13:10:44.180409 29608 raft_consensus.cc:697] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 18 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:44.193334 29275 consensus_queue.cc:1048] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 13:10:44.235569 28801 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:44.239737 29543 consensus_meta.cc:287] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.763s	user 0.006s	sys 0.006s
I20260709 13:10:44.239885 29237 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:44.250439 22995 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:44.267158 22876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 3252) took 1166 ms. Trace:
I20260709 13:10:44.267627 22876 rpcz_store.cc:276] 0709 13:10:43.100953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:43.101121 (+   168us) service_pool.cc:224] Handling call
0709 13:10:43.138725 (+ 37604us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:43.975015 (+836290us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:44.266941 (+291926us) spinlock_profiling.cc:243] Waited 835 ms on lock 0x7b54003bc754. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:44.267097 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1920785664}
I20260709 13:10:44.269212 29548 raft_consensus.cc:3055] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:10:44.269649 29548 raft_consensus.cc:740] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:44.278648 22877 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 2709) took 1155 ms. Trace:
I20260709 13:10:44.287525 29176 raft_consensus.cc:3055] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:10:44.288053 29176 raft_consensus.cc:740] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:44.289012 29176 consensus_queue.cc:260] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:44.290629 29176 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 13 FOLLOWER]: Advancing to term 14
W20260709 13:10:44.292860 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 686992us
I20260709 13:10:44.293680 22843 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:44.295135 22843 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:44.296905 29408 raft_consensus.cc:2749] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 17. Current term is 18.
I20260709 13:10:44.297659 29408 raft_consensus.cc:2749] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:10:44.298586 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:44.324208 23020 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 19.
I20260709 13:10:44.314946 22843 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:44.337052 23616 raft_consensus.cc:1275] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:10:44.339133 28813 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:44.240450 29543 raft_consensus.cc:2749] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:44.279255 22877 rpcz_store.cc:276] 0709 13:10:43.122808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:43.196940 (+ 74132us) service_pool.cc:224] Handling call
0709 13:10:43.215282 (+ 18342us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:10:43.230406 (+ 15124us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:43.230439 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:10:43.248653 (+ 18214us) log.cc:844] Serialized 52 byte log entry
0709 13:10:43.390673 (+142020us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:10:43.415603 (+ 24930us) log.cc:844] Serialized 14 byte log entry
0709 13:10:43.602512 (+186909us) log.cc:844] Serialized 14 byte log entry
0709 13:10:44.277885 (+675373us) spinlock_profiling.cc:243] Waited 4.88 ms on lock 0x7b18004a0440. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132c72c0a 00007f613c3b2219 00007f6132cfaa40 00007f6132cf54e4 00007f6132cf5455 00007f6135ddd060 00007f6135def716 00007f6135e00287 00007f6135e00208
0709 13:10:44.278110 (+   225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:44.278219 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:44.278391 (+   172us) raft_consensus.cc:1685] finished
0709 13:10:44.278414 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:44.278570 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32150,"prepare.queue_time_us":29848,"prepare.run_cpu_time_us":125,"prepare.run_wall_time_us":2827,"raft.queue_time_us":31208,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":38059,"spinlock_wait_cycles":21201408,"thread_start_us":327920,"threads_started":2,"wal-append.queue_time_us":299222}
W20260709 13:10:44.343628 29633 log.cc:927] Time spent T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.122s	user 0.000s	sys 0.000s
W20260709 13:10:44.356853 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:44.347939 29398 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:44.363298 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 48439.2us
I20260709 13:10:44.372267 22843 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:44.343492 29612 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:10:44.378005 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:44.380275 23621 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 16.
W20260709 13:10:44.383188 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:44.384895 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:44.387663 29239 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:44.367213 29408 raft_consensus.cc:2804] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:10:44.388409 29239 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:44.372107 29240 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:44.378285 29493 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:44.366235 29398 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:44.400462 29546 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:44.404695 29551 raft_consensus.cc:2804] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader election won for term 16
I20260709 13:10:44.435464 23621 raft_consensus.cc:1240] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 13. Current term is 14. Ops: []
I20260709 13:10:44.435381 22993 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:44.439335 28813 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:44.461499 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:44.382033 29577 consensus_queue.cc:1048] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.033s
I20260709 13:10:44.439931 28813 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:44.481946 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 46624.3us
W20260709 13:10:44.485677 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:44.492329 22993 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:44.499071 29398 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:44.499914 29398 consensus_queue.cc:1059] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:10:44.501981 29634 log.cc:927] Time spent T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
I20260709 13:10:44.504119 22955 raft_consensus.cc:1240] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 18. Current term is 19. Ops: []
I20260709 13:10:44.406152 29612 consensus_queue.cc:1059] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.088s
I20260709 13:10:44.514209 28813 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:44.514750 28813 consensus_queue.cc:1048] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:10:44.516396 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:44.517444 23027 raft_consensus.cc:2410] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 19 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 13:10:44.402670 29608 consensus_queue.cc:237] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:44.481087 29546 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:44.407970 29613 consensus_queue.cc:1059] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.099s
I20260709 13:10:44.424037 29493 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:44.540635 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:44.517921 29246 raft_consensus.cc:2749] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:44.523111 29479 raft_consensus.cc:2804] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:10:44.547698 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 364812us
I20260709 13:10:44.571633 29408 raft_consensus.cc:697] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 20 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
W20260709 13:10:44.518833 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 172816us
I20260709 13:10:44.573154 29408 consensus_queue.cc:237] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:44.578178 29551 raft_consensus.cc:697] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 16 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
W20260709 13:10:44.544651 29574 consensus_meta.cc:287] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.506s	user 0.003s	sys 0.009s
I20260709 13:10:44.523718 29556 consensus_queue.cc:1048] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
I20260709 13:10:44.581444 29551 consensus_queue.cc:237] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:44.549754 29389 consensus_queue.cc:1059] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.075s
I20260709 13:10:44.608093 29613 raft_consensus.cc:3055] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:10:44.626766 29613 raft_consensus.cc:740] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:44.610739 29559 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:44.629269 29559 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:44.630023 29493 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:44.585304 29520 consensus_queue.cc:1048] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 13:10:44.636272 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:44.637100 22873 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 19.
W20260709 13:10:44.624809 23020 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33418 (request call id 2556) took 1407 ms (client timeout 1684 ms). Trace:
W20260709 13:10:44.630146 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:44.638466 23020 rpcz_store.cc:269] 0709 13:10:43.217770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:43.221193 (+  3423us) service_pool.cc:224] Handling call
0709 13:10:44.624638 (+1403445us) spinlock_profiling.cc:243] Waited 866 ms on lock 0x7b5400248e54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:44.624753 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1991831296}
W20260709 13:10:44.639017 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:44.597126 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 163510us
I20260709 13:10:44.590180 29612 raft_consensus.cc:3055] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 13 LEADER]: Stepping down as leader of term 13
I20260709 13:10:44.601970 29601 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:44.650717 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:44.661705 23616 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 19.
W20260709 13:10:44.620203 29629 consensus_meta.cc:287] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.546s	user 0.007s	sys 0.000s
W20260709 13:10:44.623308 29581 consensus_meta.cc:287] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.556s	user 0.010s	sys 0.001s
I20260709 13:10:44.616850 29649 raft_consensus.cc:3055] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 15 LEADER]: Stepping down as leader of term 15
I20260709 13:10:44.663754 22993 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:44.626710 28801 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:44.627895 29613 consensus_queue.cc:260] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:44.625860 29546 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:44.651518 29612 raft_consensus.cc:740] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:44.684788 22871 raft_consensus.cc:1275] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:10:44.686026 22873 raft_consensus.cc:3055] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:10:44.689132 22873 raft_consensus.cc:740] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:44.689532 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:44.690146 22873 consensus_queue.cc:260] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:44.691816 22873 raft_consensus.cc:3060] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:44.622334 29547 consensus_queue.cc:1059] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.055s
I20260709 13:10:44.673808 29629 raft_consensus.cc:2749] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:44.694481 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:44.695952 22871 raft_consensus.cc:1240] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:44.696652 23620 raft_consensus.cc:1240] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 22 LEADER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 13:10:44.700766 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:44.701462 22950 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 19.
I20260709 13:10:44.705945 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:44.706591 22952 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 17.
I20260709 13:10:44.673583 29649 raft_consensus.cc:740] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:44.680141 29398 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:44.716275 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:44.717824 22871 pending_rounds.cc:85] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 13
I20260709 13:10:44.718571 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:44.720543 22871 pending_rounds.cc:107] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 21.14
I20260709 13:10:44.721382 22871 raft_consensus.cc:2889] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:44.720425 22878 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 20.
I20260709 13:10:44.725288 22878 raft_consensus.cc:1240] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:44.696013 29493 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:10:44.667634 29601 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:44.730670 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:44.731947 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:44.734071 22875 raft_consensus.cc:2393] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 19: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:44.732429 22843 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:44.737033 22878 raft_consensus.cc:1240] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 20 LEADER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 19. Current term is 20. Ops: []
I20260709 13:10:44.710285 29613 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:10:44.740128 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:44.740036 29239 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:44.730868 29398 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:44.744156 29398 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:44.745471 29645 log.cc:927] Time spent T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
W20260709 13:10:44.747337 29640 log.cc:927] Time spent T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
W20260709 13:10:44.727790 29636 log.cc:927] Time spent T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.235s	user 0.000s	sys 0.001s
I20260709 13:10:44.748436 23024 raft_consensus.cc:2410] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 22 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
I20260709 13:10:44.766942 22875 raft_consensus.cc:1275] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 13:10:44.763162 22843 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:44.756175 29556 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:44.711445 29242 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:44.763736 29546 raft_consensus.cc:2749] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:44.695783 29656 consensus_queue.cc:1059] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.072s
I20260709 13:10:44.757516 22880 raft_consensus.cc:1275] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:10:44.733314 29493 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:44.774374 22873 raft_consensus.cc:1275] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:10:44.778326 22843 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:44.761677 29547 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:44.817384 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:44.833292 29398 raft_consensus.cc:3055] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:10:44.833751 29398 raft_consensus.cc:740] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:44.834620 29398 consensus_queue.cc:260] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:44.836004 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 57757us
I20260709 13:10:44.837105 22843 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:10:44.771852 29404 consensus_meta.cc:287] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.699s	user 0.012s	sys 0.000s
I20260709 13:10:44.838810 22843 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:44.839682 22873 raft_consensus.cc:1275] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:10:44.841609 22843 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:44.852306 23028 raft_consensus.cc:1275] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
I20260709 13:10:44.852841 23028 pending_rounds.cc:85] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75: Aborting all ops after (but not including) 10
I20260709 13:10:44.853157 23028 pending_rounds.cc:107] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 13:10:44.853451 23028 raft_consensus.cc:2889] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:44.780149 29493 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:44.856235 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:44.832185 29408 raft_consensus.cc:2749] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:44.839600 29564 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:44.862063 29638 log.cc:927] Time spent T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.290s	user 0.000s	sys 0.001s
I20260709 13:10:44.861799 29564 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:44.863229 29604 log.cc:927] Time spent T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.802s	user 0.000s	sys 0.003s
I20260709 13:10:44.863376 29657 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:44.864045 29564 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:44.863977 29657 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:44.850656 29239 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:44.865216 29239 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:44.866770 29657 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:44.840834 29547 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:44.869390 29547 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:44.869953 29547 raft_consensus.cc:2749] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:10:44.870671 29617 consensus_meta.cc:287] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.740s	user 0.012s	sys 0.000s
I20260709 13:10:44.871212 29617 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:44.777237 29242 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:44.854156 29546 raft_consensus.cc:2749] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:10:44.777738 29556 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:44.884702 22880 raft_consensus.cc:1240] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:44.872095 29612 consensus_queue.cc:260] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:44.885958 29664 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:44.888770 29653 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:44.889971 29601 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:44.863255 29654 raft_consensus.cc:2749] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:10:44.891685 29398 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:44.849490 29649 consensus_queue.cc:260] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:44.893229 29239 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:44.908805 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:44.924796 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:44.938829 22875 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 17.
I20260709 13:10:44.939640 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:44.940085 22875 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 22.
I20260709 13:10:44.870689 29547 raft_consensus.cc:2749] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:44.911731 22877 raft_consensus.cc:1275] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 10. (term mismatch)
I20260709 13:10:44.932184 22872 raft_consensus.cc:1275] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:10:44.906973 29237 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:44.935469 22920 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:44.942505 22877 pending_rounds.cc:85] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 9
I20260709 13:10:44.944195 22877 pending_rounds.cc:107] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 18.10
I20260709 13:10:44.944474 22877 raft_consensus.cc:2889] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 13:10:44.952085 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 110541us
I20260709 13:10:44.966684 22920 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:44.974514 29548 consensus_queue.cc:260] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:44.953346 29557 consensus_queue.cc:1059] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.023s
I20260709 13:10:44.967242 22843 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:44.938169 29656 consensus_queue.cc:1059] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.024s
I20260709 13:10:44.954830 29577 raft_consensus.cc:2749] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 13:10:44.941422 29520 consensus_queue.cc:1059] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.025s
I20260709 13:10:44.943288 29601 consensus_queue.cc:1048] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.025s
I20260709 13:10:44.993618 22843 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:44.964776 29581 consensus_queue.cc:1048] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 13:10:44.931196 29653 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:44.989850 28801 raft_consensus.cc:2749] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:10:44.994643 29522 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.934s	user 0.022s	sys 0.021s
I20260709 13:10:44.997210 22920 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:45.000232 29546 raft_consensus.cc:2804] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:10:44.990521 29679 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:44.930322 29664 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:44.995707 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:44.953675 29237 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:45.011878 22920 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:45.012977 29680 raft_consensus.cc:2804] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:10:45.013420 29680 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:45.013644 29237 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:45.013761 29680 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:10:44.966224 29631 consensus_queue.cc:1048] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 13:10:45.015833 29546 raft_consensus.cc:697] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 19 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:44.995214 29617 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:45.016935 29546 consensus_queue.cc:237] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:45.018633 29631 consensus_queue.cc:1048] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:10:45.001672 22843 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:44.995530 29543 raft_consensus.cc:2749] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 13:10:45.008148 22845 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:45.010902 29630 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:10:45.011387 29661 log.cc:927] Time spent T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 13:10:45.028581 29518 consensus_queue.cc:1048] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 13:10:45.027518 29630 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:45.030431 29630 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:45.027105 29547 raft_consensus.cc:2749] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 13:10:45.037636 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:45.023451 22843 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:45.002856 29548 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:10:45.004472 29522 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.943s	user 0.022s	sys 0.022s
I20260709 13:10:45.042353 22843 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:45.008015 29556 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:45.046073 22843 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:45.051806 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 108240us
I20260709 13:10:45.020844 29679 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:45.044605 29547 raft_consensus.cc:2749] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:45.058910 22843 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:45.064894 22843 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:45.047287 29654 raft_consensus.cc:2749] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:10:45.073216 23616 raft_consensus.cc:1275] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 13:10:45.075384 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:45.075747 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 23 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:45.003262 28898 consensus_queue.cc:1175] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "0a407504ed1741bf89a276558adf4c0f" responder_term: 17 status { last_received { term: 16 index: 9 } last_committed_idx: 9 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 16. Current term is 17. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 13:10:45.072836 29662 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:45.061039 29675 raft_consensus.cc:2749] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:45.063544 29522 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:45.062424 29547 consensus_queue.cc:1048] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 13:10:45.089370 29653 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:45.090974 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:45.091130 29574 raft_consensus.cc:3055] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:10:45.091611 29574 raft_consensus.cc:740] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:45.097872 29664 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:45.098569 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:45.102659 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 431820us
I20260709 13:10:45.070932 29408 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:45.106364 23616 raft_consensus.cc:1275] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:10:45.107439 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:45.123061 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:45.124768 23022 raft_consensus.cc:1275] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:10:45.069592 29557 consensus_queue.cc:1048] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:10:45.080744 29617 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:45.087765 29662 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:45.127748 23020 raft_consensus.cc:1275] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 13:10:45.092509 29574 consensus_queue.cc:260] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:45.133448 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:45.135427 22875 raft_consensus.cc:1275] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:10:45.135529 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:45.135958 23022 raft_consensus.cc:1275] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 13:10:45.143546 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:45.144418 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:10:45.144790 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 69768us
I20260709 13:10:45.114490 29275 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:45.153307 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:45.118242 29239 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:45.129555 29557 consensus_queue.cc:1243] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer 0a407504ed1741bf89a276558adf4c0f log is divergent from this leader: its last log entry 18.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:10:45.139101 29679 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:45.107292 29408 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:45.141891 23028 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 17.
I20260709 13:10:45.109344 29522 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:45.147840 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:45.149163 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:45.178488 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:45.150687 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:45.179222 22954 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 19.
I20260709 13:10:45.164902 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:45.168999 22844 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:45.179795 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:45.127342 29553 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:45.176836 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:45.119946 28801 consensus_queue.cc:1048] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.048s
I20260709 13:10:45.185992 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:45.187806 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:45.137038 29662 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:45.184165 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:45.194950 23029 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 14.
I20260709 13:10:45.182507 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 23 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:45.195721 23618 raft_consensus.cc:2393] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 14: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:45.119809 29493 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:45.186297 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:45.196631 22843 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:45.177974 29408 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:45.137230 29617 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:45.168288 29685 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:45.177510 29239 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:45.205377 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:45.224164 23028 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 17.
I20260709 13:10:45.183744 29574 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:45.226351 23028 raft_consensus.cc:1240] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 15. Current term is 16. Ops: [13.8-15.9]
I20260709 13:10:45.230806 29176 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:45.171695 29275 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:45.205353 29547 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:45.205832 29520 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:45.221427 29654 raft_consensus.cc:2749] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:45.198495 29679 raft_consensus.cc:2749] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:45.217242 29663 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:45.217163 29644 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:45.244057 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:45.243898 29630 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:45.207041 29518 consensus_queue.cc:1048] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 13:10:45.246352 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:45.195297 29680 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:45.246321 29559 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:45.254092 22879 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 22.
W20260709 13:10:45.256449 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 61719.8us
I20260709 13:10:45.237179 28801 consensus_queue.cc:1048] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.018s
W20260709 13:10:45.258179 29686 log.cc:927] Time spent T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 13:10:45.232386 29577 consensus_queue.cc:1048] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 13:10:45.193084 29553 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:45.259991 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:45.264587 22877 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:45.266106 22994 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:45.286137 23588 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:45.287032 29664 raft_consensus.cc:2804] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:10:45.289496 22879 raft_consensus.cc:1240] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:45.240923 29176 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:45.231030 29685 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:45.250545 29520 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:45.243765 29547 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:45.288975 29664 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:45.307947 29664 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:45.308074 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:45.310065 22879 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 21.
W20260709 13:10:45.312209 29694 log.cc:927] Time spent T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 13:10:45.255203 29644 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:45.297616 22844 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:45.314522 29644 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:45.314803 29543 raft_consensus.cc:2749] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:45.315567 22844 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:45.317070 29630 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:45.289706 29508 raft_consensus.cc:2804] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:10:45.303663 29275 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:45.305280 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:45.305801 29680 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 23 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:45.291226 29699 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:45.306682 29522 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:45.307327 29649 raft_consensus.cc:3060] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Advancing to term 16
I20260709 13:10:45.255529 29663 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:45.313613 29239 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:45.329432 22877 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 19.
I20260709 13:10:45.338795 23622 raft_consensus.cc:2410] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 20 because replica has last-logged OpId of term: 18 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 16.
I20260709 13:10:45.338689 22920 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:45.344846 22844 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:45.350896 23020 raft_consensus.cc:2393] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 19: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:45.354063 29663 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:45.354159 29654 raft_consensus.cc:2749] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:10:45.356462 23620 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60026 (request call id 1213) took 2583 ms. Trace:
I20260709 13:10:45.356984 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:45.357739 23619 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 21.
I20260709 13:10:45.357957 29685 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:45.358314 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:45.356851 23620 rpcz_store.cc:276] 0709 13:10:42.773214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:42.800315 (+ 27101us) service_pool.cc:224] Handling call
0709 13:10:42.801949 (+  1634us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:44.703222 (+1901273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:45.356185 (+652963us) spinlock_profiling.cc:243] Waited 1.89 s on lock 0x7b54004943d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:45.356380 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4350465024}
W20260709 13:10:45.364411 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 127956us
I20260709 13:10:45.341286 29613 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:45.336710 29508 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:10:45.374804 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:45.365866 29639 log.cc:927] Time spent T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.853s	user 0.004s	sys 0.000s
I20260709 13:10:45.377228 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:45.384094 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:45.384794 23023 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:45.387836 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:45.389108 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:45.389940 22947 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 18.
I20260709 13:10:45.384109 22994 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:45.395445 22946 raft_consensus.cc:2393] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 23: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:45.368170 29526 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:45.373503 29508 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:10:45.404541 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:45.405871 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:45.406450 23622 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:45.383632 29548 raft_consensus.cc:2804] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:10:45.384409 29493 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:45.348523 29699 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:45.421625 29176 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:45.422355 29176 raft_consensus.cc:2804] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 13:10:45.423427 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:45.425264 29520 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:45.427666 29479 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:45.428265 29479 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:45.409368 29526 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:45.439517 29526 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:45.445919 22951 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58320 (request call id 3385) took 1342 ms. Trace:
I20260709 13:10:45.446470 22951 rpcz_store.cc:276] 0709 13:10:44.103675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:44.123769 (+ 20094us) service_pool.cc:224] Handling call
0709 13:10:44.124076 (+   307us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:44.137395 (+ 13319us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:44.137433 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:44.140236 (+  2803us) log.cc:844] Serialized 26 byte log entry
0709 13:10:44.343161 (+202925us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:10:44.343216 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:44.343299 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:45.317452 (+974153us) spinlock_profiling.cc:243] Waited 3.08 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:10:45.445528 (+128076us) raft_consensus.cc:1685] finished
0709 13:10:45.445557 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:45.445833 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12907,"raft.queue_time_us":13771,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":91,"spinlock_wait_cycles":7078272,"thread_start_us":164364,"threads_started":1,"wal-append.queue_time_us":271847}
I20260709 13:10:45.414007 29548 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:45.402324 29617 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:45.449297 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:45.468387 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:45.485991 23023 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18.
I20260709 13:10:45.488636 23620 raft_consensus.cc:2393] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 18: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:45.493566 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:45.495581 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:45.497265 23023 raft_consensus.cc:2393] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 19: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:45.495738 29527 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:45.457728 29547 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:45.465076 29548 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:45.487973 29176 raft_consensus.cc:697] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 19 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:45.544684 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:45.545665 23023 raft_consensus.cc:2393] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 20: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:45.545471 29176 consensus_queue.cc:237] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:45.546082 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:45.546607 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:45.546918 23620 raft_consensus.cc:2393] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 21: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:45.547320 23023 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 20.
I20260709 13:10:45.542455 29527 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:45.490466 29398 consensus_meta.cc:287] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.552s	user 0.010s	sys 0.002s
I20260709 13:10:45.573521 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:45.574458 22951 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 19.
I20260709 13:10:45.579155 23588 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:45.548342 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:45.581926 23023 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 21.
I20260709 13:10:45.583240 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:45.584051 23023 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 20.
I20260709 13:10:45.603727 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:45.607918 23622 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 20.
I20260709 13:10:45.460273 29662 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:45.612879 29579 raft_consensus.cc:2804] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:10:45.613509 29579 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:45.613973 29579 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:45.550535 29527 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:45.616772 22878 raft_consensus.cc:1275] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 13:10:45.617167 22878 pending_rounds.cc:85] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 8
I20260709 13:10:45.617398 22878 pending_rounds.cc:107] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 13:10:45.448632 29664 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:45.617640 22878 raft_consensus.cc:2889] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:45.452152 29699 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:45.623580 22879 raft_consensus.cc:2393] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 18: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:45.619692 29664 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:45.634320 23028 raft_consensus.cc:1275] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:10:45.628166 29662 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:45.634855 29408 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:45.675391 29422 raft_consensus.cc:3055] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:10:45.675869 29422 raft_consensus.cc:740] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:10:45.679718 23024 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:33418 (request call id 2554) took 2477 ms (client timeout 2201 ms). Trace:
I20260709 13:10:45.674449 29663 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:45.683269 29663 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:45.683938 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:45.689816 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:45.680173 23024 rpcz_store.cc:269] 0709 13:10:43.202176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:43.244361 (+ 42185us) service_pool.cc:224] Handling call
0709 13:10:45.679472 (+2435111us) spinlock_profiling.cc:243] Waited 1.5 s on lock 0x7b540010bb54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:45.679619 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3456516992}
I20260709 13:10:45.691131 22992 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:45.693266 23588 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:45.694600 23588 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:45.694751 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:45.695330 23020 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 14.
I20260709 13:10:45.695627 23588 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:45.696740 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:45.697355 23020 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 19.
I20260709 13:10:45.697698 22872 raft_consensus.cc:2393] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 17: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:45.699877 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:45.712755 29685 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:10:45.738791 29685 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:45.739198 29685 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:45.743108 29553 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:45.699188 29642 raft_consensus.cc:2804] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:45.744885 29642 raft_consensus.cc:493] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:45.745262 29642 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:45.746958 29644 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:45.747479 29644 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:45.749620 29644 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:45.750171 29644 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:45.718343 29479 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:45.719333 29408 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:45.721457 29612 raft_consensus.cc:3060] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 13 FOLLOWER]: Advancing to term 14
I20260709 13:10:45.721467 29548 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:45.697115 29422 consensus_queue.cc:260] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:45.697487 29508 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:45.711608 29683 raft_consensus.cc:2804] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:10:45.727910 29613 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 13:10:45.729738 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:45.729981 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:45.696995 29555 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:45.733932 22993 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:45.769497 22872 raft_consensus.cc:2393] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 17: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:45.734633 29663 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:45.737355 29579 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:45.751430 22877 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 20.
W20260709 13:10:45.751557 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 52738.8us
W20260709 13:10:45.768124 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:45.772840 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:10:45.768568 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 68077us
I20260709 13:10:45.750648 29644 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:45.773475 22950 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:45.775018 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:45.775449 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:45.776849 29716 raft_consensus.cc:2804] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:10:45.787349 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:45.787750 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:45.787954 22877 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 19.
I20260709 13:10:45.788326 22875 raft_consensus.cc:3060] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:10:45.788856 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:45.789791 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:45.777321 29479 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:45.779744 22871 raft_consensus.cc:1240] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 17. Current term is 18. Ops: []
I20260709 13:10:45.791589 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:45.763983 29485 consensus_queue.cc:1048] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.016s
I20260709 13:10:45.781627 29508 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:45.794951 22871 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 14.
I20260709 13:10:45.792213 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:45.769861 29683 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:45.790952 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:45.785921 29548 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:45.791168 29716 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:45.797891 22993 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:45.798430 29642 raft_consensus.cc:515] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:45.807044 22993 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:45.807896 22919 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:45.801766 22872 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:45.810472 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:45.796262 22877 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 20.
I20260709 13:10:45.811033 23619 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:45.811642 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:45.811334 22994 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:45.805562 29683 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:45.817771 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:45.794703 29657 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:45.805994 22950 raft_consensus.cc:2468] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 20.
I20260709 13:10:45.827325 22872 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 20.
I20260709 13:10:45.828289 22875 raft_consensus.cc:2468] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 23.
I20260709 13:10:45.809399 29663 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:45.811868 29577 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:45.807188 29716 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:45.784368 29685 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:45.813529 29240 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 13:10:45.836119 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 421762us
I20260709 13:10:45.843050 23619 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 17.
I20260709 13:10:45.844413 22993 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:45.833518 29714 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:10:45.847859 22873 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 2720) took 1011 ms. Trace:
I20260709 13:10:45.848606 22873 rpcz_store.cc:276] 0709 13:10:44.836009 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:44.836291 (+   282us) service_pool.cc:224] Handling call
0709 13:10:44.836571 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:44.840852 (+  4281us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:45.847632 (+1006780us) spinlock_profiling.cc:243] Waited 2.08 ms on lock 0x7b54001bc454. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:45.847806 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4777728}
I20260709 13:10:45.852731 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:45.853416 23020 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 20.
I20260709 13:10:45.841450 29266 raft_consensus.cc:2749] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:10:45.856383 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:45.856801 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 17 index: 16 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:45.857475 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:45.857581 22950 raft_consensus.cc:2410] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 20 because replica has last-logged OpId of term: 18 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 16.
I20260709 13:10:45.857590 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:45.856935 22952 raft_consensus.cc:3060] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:45.858227 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:45.858958 22947 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 18.
I20260709 13:10:45.858336 22948 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 17.
I20260709 13:10:45.865108 22876 raft_consensus.cc:1275] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 13. (index mismatch)
I20260709 13:10:45.849727 29714 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:45.834496 29701 consensus_queue.cc:1048] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.021s
I20260709 13:10:45.849192 29643 raft_consensus.cc:2804] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:10:45.833431 29657 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:45.866169 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:45.841413 29577 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:45.853678 29240 raft_consensus.cc:697] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 20 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:45.873719 22946 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 19.
I20260709 13:10:45.875864 23588 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:45.878199 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:45.886948 22955 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58310 (request call id 2634) took 1916 ms. Trace:
I20260709 13:10:45.884397 22952 raft_consensus.cc:2468] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 19.
I20260709 13:10:45.889338 22955 rpcz_store.cc:276] 0709 13:10:43.970194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:43.970652 (+   458us) service_pool.cc:224] Handling call
0709 13:10:44.503894 (+533242us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:44.504611 (+   717us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:45.886671 (+1382060us) spinlock_profiling.cc:243] Waited 530 ms on lock 0x7b5400148550. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de48ad 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:45.886874 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1218761984}
I20260709 13:10:45.885106 29724 consensus_queue.cc:1048] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.019s
I20260709 13:10:45.888298 29643 raft_consensus.cc:697] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 17 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:45.895244 29643 consensus_queue.cc:237] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:45.900498 23622 raft_consensus.cc:1240] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 13:10:45.901427 29240 consensus_queue.cc:237] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:45.902825 29635 consensus_queue.cc:1059] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:10:45.904019 29266 raft_consensus.cc:3055] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:10:45.904438 29266 raft_consensus.cc:740] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
W20260709 13:10:45.904742 29720 log.cc:927] Time spent T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 13:10:45.905298 29266 consensus_queue.cc:260] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:45.906090 29577 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:45.907027 29612 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:45.907497 29612 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:45.908628 29657 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:45.909432 29612 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:45.909582 29642 leader_election.cc:290] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:45.901852 29644 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:45.911787 29685 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 21 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:45.885763 29707 log.cc:927] Time spent T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.331s	user 0.000s	sys 0.004s
I20260709 13:10:45.917783 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:45.918476 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:45.918651 22947 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 16.
I20260709 13:10:45.919667 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:45.871903 29714 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:45.926750 22954 raft_consensus.cc:2393] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 23: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:45.922744 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:45.924737 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:45.925163 23617 raft_consensus.cc:1240] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 13:10:45.931213 29242 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:45.887725 29597 raft_consensus.cc:2804] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:10:45.926622 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:10:45.935158 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 89870us
I20260709 13:10:45.942148 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:45.947698 22878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 2726) took 1050 ms. Trace:
I20260709 13:10:45.950510 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:45.948230 22878 rpcz_store.cc:276] 0709 13:10:44.897842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:44.898023 (+   181us) service_pool.cc:224] Handling call
0709 13:10:44.898432 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:45.620241 (+721809us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:45.947396 (+327155us) spinlock_profiling.cc:243] Waited 718 ms on lock 0x7b54001bba54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:45.947624 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1651076352}
I20260709 13:10:45.953976 22880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 2728) took 1055 ms. Trace:
I20260709 13:10:45.954454 22951 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:45.956434 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:45.961123 22952 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 20.
I20260709 13:10:45.938205 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:45.965488 22873 raft_consensus.cc:1240] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:45.937976 29635 consensus_queue.cc:1059] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.019s
W20260709 13:10:45.957494 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 113144us
I20260709 13:10:45.954421 22880 rpcz_store.cc:276] 0709 13:10:44.898523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:44.927822 (+ 29299us) service_pool.cc:224] Handling call
0709 13:10:44.931240 (+  3418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:44.945382 (+ 14142us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:44.945416 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:44.951734 (+  6318us) log.cc:844] Serialized 26 byte log entry
0709 13:10:44.999459 (+ 47725us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:10:44.999884 (+   425us) log.cc:844] Serialized 14 byte log entry
0709 13:10:45.000619 (+   735us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:45.947693 (+947074us) spinlock_profiling.cc:243] Waited 1.11 ms on lock 0x7b540022c4d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:45.947762 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:45.947836 (+    74us) raft_consensus.cc:1685] finished
0709 13:10:45.953776 (+  5940us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:45.953925 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19906,"prepare.queue_time_us":281,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":20331,"raft.run_cpu_time_us":354,"raft.run_wall_time_us":6573,"spinlock_wait_cycles":2557312,"thread_start_us":30222,"threads_started":1,"wal-append.queue_time_us":44896}
I20260709 13:10:45.977099 29635 raft_consensus.cc:3055] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:10:45.977505 29635 raft_consensus.cc:740] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:45.978763 22880 pending_rounds.cc:85] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 10
I20260709 13:10:45.979094 22880 pending_rounds.cc:107] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 13:10:45.944597 29612 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:45.979359 22880 raft_consensus.cc:2889] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:45.979689 29612 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:45.981910 29612 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:45.950052 29597 raft_consensus.cc:697] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 20 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:45.979265 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3047c357228240a2aa4d84e8fbf437fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:45.983306 29597 consensus_queue.cc:237] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:45.978313 29635 consensus_queue.cc:260] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:45.969899 29716 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:45.987147 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:45.970734 22951 raft_consensus.cc:2468] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 19.
I20260709 13:10:46.005795 22992 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:46.033346 22992 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:46.039016 22992 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:46.021207 29635 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:46.021034 29553 raft_consensus.cc:2749] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:10:45.979400 29683 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:46.050120 22992 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:46.036155 29663 raft_consensus.cc:2749] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:46.055030 22992 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:46.052526 22918 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:46.060865 22992 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:46.040853 29408 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:46.043779 29507 raft_consensus.cc:2749] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:10:46.068549 29563 raft_consensus.cc:3055] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:10:46.069006 29563 raft_consensus.cc:740] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:46.070005 29563 consensus_queue.cc:260] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:46.071234 29563 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:46.073480 29722 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:10:46.074034 29722 raft_consensus.cc:697] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:46.074863 29722 consensus_queue.cc:237] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:46.055944 29701 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:46.078593 29701 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:46.063397 29679 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:46.085165 29679 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:46.087139 29679 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:46.056062 29577 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:46.094062 29577 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:46.096118 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:46.065740 29526 raft_consensus.cc:2749] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:46.056831 29643 raft_consensus.cc:2804] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:46.060503 29127 raft_consensus.cc:2804] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:10:46.096875 23022 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:46.112267 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:46.112517 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:46.105031 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:46.114486 23027 raft_consensus.cc:2368] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 21. Current term is 22.
I20260709 13:10:46.110771 23026 raft_consensus.cc:1275] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:10:46.119769 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
W20260709 13:10:46.118425 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 57650.5us
I20260709 13:10:46.098541 22876 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53724 (request call id 1249) took 1384 ms. Trace:
I20260709 13:10:46.123957 23617 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:46.110808 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:46.125346 23023 raft_consensus.cc:2376] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 19. Re-sending same reply.
I20260709 13:10:46.126039 22876 rpcz_store.cc:276] 0709 13:10:44.715024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:44.715185 (+   161us) service_pool.cc:224] Handling call
0709 13:10:44.715465 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:45.865496 (+1150031us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:46.098240 (+232744us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54002d6c54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:46.098468 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2635368320}
I20260709 13:10:46.125985 22992 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:46.129541 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:46.133683 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:46.134277 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:46.135097 23024 raft_consensus.cc:2393] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 18: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:46.130859 22918 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:46.138896 22992 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:46.141022 23026 raft_consensus.cc:1275] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:10:46.157461 22992 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:46.108872 29555 raft_consensus.cc:2749] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:46.108318 29643 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:46.122736 29507 raft_consensus.cc:2749] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:46.108644 29127 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:46.168851 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:46.102701 29716 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:46.106909 29683 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 21 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:46.096616 29577 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:46.109992 29701 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:46.157436 29663 raft_consensus.cc:2804] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 13:10:46.174826 29667 log.cc:927] Time spent T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 1.248s	user 0.007s	sys 0.000s
I20260709 13:10:46.134562 28898 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:46.177053 23021 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 2680) took 2016 ms. Trace:
I20260709 13:10:46.176831 28898 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:46.177445 23021 rpcz_store.cc:276] 0709 13:10:44.160701 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:44.180955 (+ 20254us) service_pool.cc:224] Handling call
0709 13:10:44.181421 (+   466us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:44.424553 (+243132us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 13:10:44.485851 (+ 61298us) log.cc:844] Serialized 14 byte log entry
0709 13:10:44.704861 (+219010us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:44.705418 (+   557us) log.cc:844] Serialized 26 byte log entry
0709 13:10:44.705543 (+   125us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:10:44.705580 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:44.705650 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:46.174280 (+1468630us) spinlock_profiling.cc:243] Waited 1.85 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:10:46.176780 (+  2500us) raft_consensus.cc:1685] finished
0709 13:10:46.176870 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:46.176997 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19812,"prepare.queue_time_us":61065,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":9359,"raft.queue_time_us":43062,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":210,"spinlock_wait_cycles":4257536,"thread_start_us":391245,"threads_started":2,"wal-append.queue_time_us":326204}
I20260709 13:10:46.179275 28898 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:46.169019 29644 raft_consensus.cc:2749] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:10:46.158603 29581 raft_consensus.cc:2804] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:10:46.180706 29581 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:46.181128 29581 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:46.183863 23617 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 22.
I20260709 13:10:46.169168 29643 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:46.109117 29408 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:46.187124 29408 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:46.188341 22993 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:46.159659 29642 raft_consensus.cc:2749] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:46.196743 29663 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:46.197428 29663 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:46.198344 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:46.171478 29127 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:46.212826 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:46.217832 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:10:46.218312 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 111811us
W20260709 13:10:46.219048 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:46.219110 23590 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:46.221132 22843 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:46.223178 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:46.222451 23590 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:10:46.225229 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:46.228462 23022 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 22.
I20260709 13:10:46.228308 22843 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:46.208200 29581 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:46.224429 29653 raft_consensus.cc:2749] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 13:10:46.235097 23590 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:46.227797 29555 raft_consensus.cc:2749] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:46.231309 29683 raft_consensus.cc:2804] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:10:46.228287 29507 raft_consensus.cc:2804] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:10:46.232436 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:46.252377 22994 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:46.248648 29683 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:46.254242 29664 raft_consensus.cc:2804] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 13:10:46.260738 22843 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:46.257088 29683 raft_consensus.cc:3060] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:46.267089 29526 raft_consensus.cc:2749] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:46.255190 29557 consensus_queue.cc:1048] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 13:10:46.286230 29643 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:46.252513 29714 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:46.289779 23020 raft_consensus.cc:2368] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 19. Current term is 20.
I20260709 13:10:46.290773 29714 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:46.266173 29664 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:46.292855 29664 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:46.297941 29581 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 21 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:46.254433 29507 raft_consensus.cc:697] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 22 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:46.252396 29653 raft_consensus.cc:2749] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:10:46.299539 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:46.300148 23022 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:10:46.264631 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 104768us
I20260709 13:10:46.302011 22992 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:46.304091 22992 leader_election.cc:304] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:46.306557 23022 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 21.
I20260709 13:10:46.299547 29507 consensus_queue.cc:237] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:46.309492 29714 raft_consensus.cc:2804] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:10:46.319059 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:46.319801 23617 raft_consensus.cc:2393] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 18: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:46.307492 29275 raft_consensus.cc:2764] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:10:46.312528 29653 raft_consensus.cc:2749] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:46.332652 29714 raft_consensus.cc:697] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 19 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:46.337662 29714 consensus_queue.cc:237] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:46.334928 29663 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:46.344615 29663 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:46.321381 29643 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:46.309538 29127 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:46.365146 29191 consensus_queue.cc:1059] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 17.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:10:46.365804 29127 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:46.369959 29683 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:46.371223 22879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35360 (request call id 2738) took 1031 ms. Trace:
I20260709 13:10:46.372169 29683 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:46.387782 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:10:46.390621 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 123590us
I20260709 13:10:46.392625 23617 raft_consensus.cc:1240] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 20 LEADER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 19. Current term is 20. Ops: []
I20260709 13:10:46.371762 22879 rpcz_store.cc:276] 0709 13:10:45.340142 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:45.362601 (+ 22459us) service_pool.cc:224] Handling call
0709 13:10:46.370954 (+1008353us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b54003bd654. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:46.371145 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":551362304}
I20260709 13:10:46.394872 29408 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:46.416021 22876 raft_consensus.cc:2468] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 20.
I20260709 13:10:46.418810 23590 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:46.423413 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:46.425474 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:46.424625 29699 raft_consensus.cc:2804] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:10:46.419034 29408 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:46.438968 29699 raft_consensus.cc:697] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 20 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:46.445763 29699 consensus_queue.cc:237] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:46.449067 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:46.449684 22950 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:46.449414 29750 consensus_queue.cc:1059] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:10:46.450425 29750 consensus_queue.cc:1243] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer 5b346767a89745fa82928a5dac9cc0b2 log is divergent from this leader: its last log entry 20.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:10:46.450870 29750 raft_consensus.cc:3055] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:10:46.451162 29750 raft_consensus.cc:740] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:46.451897 29750 consensus_queue.cc:260] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:46.453312 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:46.454357 29750 raft_consensus.cc:3060] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:46.455325 22954 raft_consensus.cc:1240] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:10:46.455387 23025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 2688) took 1336 ms. Trace:
I20260709 13:10:46.456403 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:46.456025 23025 rpcz_store.cc:276] 0709 13:10:45.119140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:45.120692 (+  1552us) service_pool.cc:224] Handling call
0709 13:10:45.121040 (+   348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:45.237823 (+116783us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:45.237869 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:45.237925 (+    56us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:10:45.237985 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:46.455101 (+1217116us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b54001f7cd4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:46.455161 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:46.455317 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":268086144}
I20260709 13:10:46.463496 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:46.464095 22878 raft_consensus.cc:3060] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:46.466615 29579 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:46.468961 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:46.473827 22950 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 20.
I20260709 13:10:46.474406 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:46.475193 23023 raft_consensus.cc:2393] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 20: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:46.475359 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:46.475890 22949 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:46.477049 29664 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:46.480345 29664 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:46.486452 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:10:46.487670 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:46.489183 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:46.490123 22918 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:46.492828 29150 raft_consensus.cc:2749] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:46.498201 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:46.498795 22948 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 13 FOLLOWER]: Advancing to term 15
I20260709 13:10:46.502638 22949 raft_consensus.cc:2468] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 17.
I20260709 13:10:46.504352 22843 leader_election.cc:400] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 21. Current term is 22.
I20260709 13:10:46.508708 22843 leader_election.cc:403] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:10:46.506366 22878 raft_consensus.cc:2468] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 22.
I20260709 13:10:46.510430 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:46.511178 22878 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:46.515830 22843 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:46.517515 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:46.518158 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:46.518707 22946 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:46.519243 22879 raft_consensus.cc:2393] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 17: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:46.501313 22918 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:46.523794 22843 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:46.538280 22843 leader_election.cc:400] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 19. Current term is 20.
I20260709 13:10:46.538614 22843 leader_election.cc:403] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 13:10:46.540037 22878 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 15.
I20260709 13:10:46.541163 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:46.525473 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:46.542852 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:46.534207 22947 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 17.
W20260709 13:10:46.543831 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:46.544415 22920 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:46.539175 23588 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:46.541167 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:46.545394 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:46.548480 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:46.549096 22879 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 17.
I20260709 13:10:46.490168 29408 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:46.557902 22952 raft_consensus.cc:1240] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 20. Current term is 21. Ops: []
I20260709 13:10:46.558962 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:46.527436 29612 raft_consensus.cc:3060] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Advancing to term 22
W20260709 13:10:46.561863 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 100339us
W20260709 13:10:46.569830 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 92086.8us
I20260709 13:10:46.563288 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:46.564286 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:46.561264 23588 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:46.572283 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:46.573896 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:46.570201 22918 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:46.577035 29520 raft_consensus.cc:2804] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:10:46.577623 29520 raft_consensus.cc:697] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:46.577714 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:46.578181 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:46.578428 29752 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:46.578768 23026 raft_consensus.cc:3060] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Advancing to term 15
I20260709 13:10:46.578888 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:46.578627 29520 consensus_queue.cc:237] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:46.579270 29557 raft_consensus.cc:2804] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:10:46.579810 29557 raft_consensus.cc:697] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 20 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:46.580360 22953 raft_consensus.cc:1240] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 19. Current term is 20. Ops: []
I20260709 13:10:46.551389 29597 raft_consensus.cc:2749] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:10:46.580567 29557 consensus_queue.cc:237] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 18.17, Last appended by leader: 17, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:46.555188 29722 raft_consensus.cc:2804] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:10:46.526757 29748 raft_consensus.cc:2804] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:10:46.582182 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:46.583904 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 18 index: 17 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:46.602757 22950 raft_consensus.cc:1240] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 16. Current term is 17. Ops: []
I20260709 13:10:46.631726 23023 raft_consensus.cc:3060] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:46.602625 23617 raft_consensus.cc:1275] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:10:46.536518 29547 consensus_queue.cc:1048] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 13:10:46.536162 29613 consensus_queue.cc:1048] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 13:10:46.606251 22920 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:46.606817 29518 consensus_queue.cc:1048] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:10:46.554863 29608 consensus_queue.cc:1059] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.061s
I20260709 13:10:46.612907 23029 raft_consensus.cc:2368] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 22. Current term is 23.
I20260709 13:10:46.614243 22992 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:10:46.614799 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 90448.5us
I20260709 13:10:46.644249 22992 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:46.615234 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:46.617789 23026 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 15.
I20260709 13:10:46.612397 23620 raft_consensus.cc:1240] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 20. Current term is 21. Ops: []
I20260709 13:10:46.630134 22946 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 20.
I20260709 13:10:46.578910 29752 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:46.636948 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:10:46.629426 29734 log.cc:927] Time spent T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 13:10:46.654171 22992 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:46.659021 22920 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:46.659760 28813 raft_consensus.cc:2804] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:10:46.575837 29408 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:46.640029 29748 raft_consensus.cc:697] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 19 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:46.639415 29722 raft_consensus.cc:697] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:46.640276 29553 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:46.685516 23023 raft_consensus.cc:2468] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 20.
I20260709 13:10:46.665125 29191 raft_consensus.cc:2804] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:10:46.664145 29728 raft_consensus.cc:2804] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:10:46.678359 29714 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:10:46.680863 28898 consensus_queue.cc:1048] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.019s
I20260709 13:10:46.698729 29493 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:46.698983 29518 raft_consensus.cc:2749] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:46.656706 29564 consensus_queue.cc:1048] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 13:10:46.672840 29657 consensus_queue.cc:1048] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 13:10:46.689672 29629 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:46.718398 29747 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:46.718950 29747 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:46.690119 29749 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
W20260709 13:10:46.723381 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 162185us
I20260709 13:10:46.696244 29553 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:46.730758 29752 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:46.731519 29553 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:46.708242 29191 raft_consensus.cc:697] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 17 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:46.733142 29191 consensus_queue.cc:237] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:46.713337 29547 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:46.738039 29547 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:46.740192 29547 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:46.711061 29728 raft_consensus.cc:697] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 17 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:46.740782 29749 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:46.741698 29728 consensus_queue.cc:237] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:46.742986 29749 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:46.718359 29722 consensus_queue.cc:237] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:46.715811 29714 raft_consensus.cc:697] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 20 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:46.746907 29714 consensus_queue.cc:237] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:46.750208 29742 log.cc:927] Time spent T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.483s	user 0.005s	sys 0.000s
I20260709 13:10:46.726547 29629 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:46.751331 29747 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:46.752913 29629 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:46.753050 22948 raft_consensus.cc:2468] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 15.
W20260709 13:10:46.770023 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 231900us
I20260709 13:10:46.770668 23588 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:46.795732 23588 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:10:46.779141 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 120173us
I20260709 13:10:46.783905 22946 raft_consensus.cc:1275] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:10:46.787663 29685 raft_consensus.cc:3055] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:10:46.796908 29685 raft_consensus.cc:740] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
W20260709 13:10:46.797834 22843 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:46.714066 29493 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:46.770699 29679 raft_consensus.cc:2749] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:46.799552 22920 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:46.805510 22871 raft_consensus.cc:1275] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 15. (term mismatch)
I20260709 13:10:46.807173 22871 pending_rounds.cc:85] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 14
I20260709 13:10:46.810313 22871 pending_rounds.cc:107] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 19.15
I20260709 13:10:46.810700 22871 raft_consensus.cc:2889] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:46.813123 22843 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:46.799178 29485 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:10:46.816795 22843 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:46.798684 29747 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:46.807842 29608 raft_consensus.cc:2804] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 13:10:46.818843 22843 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:46.823788 22843 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:46.829409 23023 raft_consensus.cc:1240] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 19 LEADER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 18. Current term is 19. Ops: []
I20260709 13:10:46.797834 29685 consensus_queue.cc:260] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:46.843824 22954 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 18.
I20260709 13:10:46.845160 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:46.833989 29547 raft_consensus.cc:2749] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:46.798183 29748 consensus_queue.cc:237] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:46.817761 29763 raft_consensus.cc:2749] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:46.843950 29613 consensus_queue.cc:1059] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 13:10:46.827541 29773 raft_consensus.cc:2804] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:46.820405 29765 consensus_queue.cc:1059] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 13:10:46.847187 22992 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:46.820648 29653 raft_consensus.cc:2749] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:10:46.811789 29555 consensus_queue.cc:1048] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.028s
I20260709 13:10:46.849465 28813 raft_consensus.cc:697] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:46.834604 29749 raft_consensus.cc:2749] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:10:46.880867 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:46.869066 29756 log.cc:927] Time spent T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 13:10:46.853780 29773 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:46.859265 29408 raft_consensus.cc:2749] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 19. Current term is 20.
I20260709 13:10:46.854431 29608 raft_consensus.cc:697] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:46.888945 22873 raft_consensus.cc:1275] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:10:46.856829 29485 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:46.856508 29266 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:46.870430 29422 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:46.898006 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:46.878086 29507 raft_consensus.cc:2804] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:10:46.838079 29747 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:46.911832 29613 raft_consensus.cc:3055] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:10:46.966904 29613 raft_consensus.cc:740] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:46.967907 29613 consensus_queue.cc:260] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:46.929180 28813 consensus_queue.cc:237] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:46.969456 29747 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:46.890537 29763 raft_consensus.cc:3055] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 16 LEADER]: Stepping down as leader of term 16
I20260709 13:10:46.971027 29763 raft_consensus.cc:740] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
W20260709 13:10:46.971490 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:46.972036 29763 consensus_queue.cc:260] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:46.973771 29763 raft_consensus.cc:3060] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:10:46.969426 29613 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:46.993813 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:46.880887 22920 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:46.997507 29507 raft_consensus.cc:697] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 18 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:46.998620 29507 consensus_queue.cc:237] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:47.005823 29761 log.cc:927] Time spent T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.130s	user 0.000s	sys 0.000s
I20260709 13:10:47.021136 29715 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:47.021780 29715 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:46.962307 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 127339us
W20260709 13:10:46.963053 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 83771.9us
I20260709 13:10:47.034407 22879 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 20.
W20260709 13:10:47.037993 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 157077us
I20260709 13:10:47.038995 22920 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:47.039994 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:47.041047 23619 raft_consensus.cc:2393] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 21: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:46.899190 29773 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:47.035684 29547 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:47.048715 29547 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:47.050904 29547 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:47.052060 29569 consensus_queue.cc:1059] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:10:47.052683 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:47.053103 29493 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:47.053164 29569 raft_consensus.cc:3055] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:10:46.960830 29485 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:47.048048 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:47.024200 29608 consensus_queue.cc:237] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:47.054301 23617 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 19.
I20260709 13:10:47.055414 29485 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:47.025477 29699 consensus_queue.cc:1048] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 13:10:47.057022 22953 raft_consensus.cc:1275] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
W20260709 13:10:47.046042 22992 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:46.994601 29563 consensus_queue.cc:1059] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.063s
W20260709 13:10:47.058817 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:47.053707 29569 raft_consensus.cc:740] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:47.045188 22946 raft_consensus.cc:1275] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:10:47.062395 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:47.058363 29629 raft_consensus.cc:2749] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:47.064098 22948 raft_consensus.cc:1275] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:10:47.063061 23617 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 17.
W20260709 13:10:47.066418 22846 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:47.069151 22844 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:47.076509 29715 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:10:47.077711 29612 consensus_meta.cc:287] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.501s	user 0.005s	sys 0.007s
I20260709 13:10:47.077916 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:47.078284 29612 raft_consensus.cc:2749] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 21. Current term is 22.
I20260709 13:10:47.078624 22947 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 18.
I20260709 13:10:47.065805 29563 consensus_queue.cc:1243] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Peer f3c799b1249e40119d2a5864231c6c75 log is divergent from this leader: its last log entry 19.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 13:10:47.045719 29747 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:47.085557 29747 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:47.088137 29747 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:47.089210 23588 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:47.090018 29563 raft_consensus.cc:2804] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:47.089731 29569 consensus_queue.cc:260] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:47.090422 29563 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:47.090915 29563 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:47.091308 29569 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:47.092729 29408 raft_consensus.cc:2804] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:47.093222 29408 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:47.093590 29408 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:47.097393 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:47.100414 22874 raft_consensus.cc:2468] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 22.
I20260709 13:10:47.100729 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:47.104000 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:47.106487 22993 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:47.115180 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:47.116324 22843 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:47.118230 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:47.118212 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 43934.2us
I20260709 13:10:47.074460 29150 raft_consensus.cc:2749] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:47.065587 29547 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:47.135416 22843 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:47.146556 29563 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:47.125304 29747 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:10:47.150527 29740 log.cc:927] Time spent T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.885s	user 0.000s	sys 0.005s
I20260709 13:10:47.142416 29612 raft_consensus.cc:2749] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:47.153182 22880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53724 (request call id 1263) took 1368 ms. Trace:
I20260709 13:10:47.154062 29773 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:47.156837 22946 raft_consensus.cc:2368] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 17. Current term is 18.
I20260709 13:10:47.141705 29176 consensus_queue.cc:1048] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 13:10:47.157055 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:47.141484 29553 consensus_queue.cc:1048] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 13:10:47.143477 29479 consensus_queue.cc:1048] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 13:10:47.159060 29773 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:47.153560 22880 rpcz_store.cc:276] 0709 13:10:45.789467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:45.978283 (+188816us) service_pool.cc:224] Handling call
0709 13:10:45.978585 (+   302us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:10:45.980846 (+  2261us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:45.980883 (+    37us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:10:45.981494 (+   611us) log.cc:844] Serialized 52 byte log entry
0709 13:10:46.131511 (+150017us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:10:46.131575 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:46.131675 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:47.150020 (+1018345us) spinlock_profiling.cc:243] Waited 14.9 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:10:47.152883 (+  2863us) raft_consensus.cc:1685] finished
0709 13:10:47.152906 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:47.153126 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":160,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":90,"spinlock_wait_cycles":34287488,"thread_start_us":149657,"threads_started":1,"wal-append.queue_time_us":270343}
I20260709 13:10:47.163332 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:47.165402 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:47.166109 22947 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:47.132048 29714 raft_consensus.cc:2804] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:10:47.170006 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 523216us
I20260709 13:10:47.183949 22951 raft_consensus.cc:2410] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for term 21 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 13:10:47.188038 22947 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 19.
I20260709 13:10:47.190793 23026 raft_consensus.cc:1240] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 13:10:47.191448 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:47.193720 23024 raft_consensus.cc:1240] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 17 LEADER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 13:10:47.197160 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:47.199076 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:47.200527 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:47.201328 23021 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 17.
I20260709 13:10:47.203667 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:47.204515 23029 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 18.
I20260709 13:10:47.205690 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:47.212442 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:47.213158 22880 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:47.160303 29563 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:47.181892 29714 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:47.211710 23588 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:47.218331 22843 leader_election.cc:400] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 22 election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 22. Current term is 23.
I20260709 13:10:47.224143 22843 leader_election.cc:403] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 13:10:47.225894 22843 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:47.227732 22843 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:47.230175 22843 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:47.139364 29547 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:47.243088 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:47.244421 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:47.258044 23620 raft_consensus.cc:1275] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:10:47.260226 23622 raft_consensus.cc:1275] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:10:47.262496 29747 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:47.262542 29242 consensus_queue.cc:1048] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:10:47.232950 29613 raft_consensus.cc:2749] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 22. Current term is 23.
I20260709 13:10:47.263967 29553 consensus_queue.cc:1048] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:10:47.267357 29547 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:47.234635 29714 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:47.263052 22949 raft_consensus.cc:1275] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:10:47.257975 29408 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:47.272739 29563 raft_consensus.cc:2804] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:10:47.273471 29563 raft_consensus.cc:697] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 19 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:47.273626 22948 raft_consensus.cc:1275] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 18 index: 17. Preceding OpId from leader: term: 20 index: 18. (index mismatch)
I20260709 13:10:47.274525 29563 consensus_queue.cc:237] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:47.278292 29664 consensus_queue.cc:1048] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:10:47.279280 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:47.231535 29679 raft_consensus.cc:2749] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:47.280115 22947 raft_consensus.cc:1275] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:10:47.260864 22880 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18.
I20260709 13:10:47.274716 29408 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:47.282996 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:47.284884 23022 raft_consensus.cc:1275] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 13:10:47.285935 29553 consensus_queue.cc:1048] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:10:47.286192 29176 consensus_queue.cc:1048] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:10:47.289129 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:47.289475 22918 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:47.293715 22880 raft_consensus.cc:2410] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 21 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 14.
I20260709 13:10:47.294557 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:47.297019 22843 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:47.301390 23025 raft_consensus.cc:1275] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:10:47.314685 22875 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35370 (request call id 3282) took 2025 ms. Trace:
W20260709 13:10:47.337179 23588 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:47.317461 22994 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:10:47.375748 22921 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 13:10:47.386229 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 68888.5us
I20260709 13:10:47.389559 22994 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:47.398790 22918 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:47.347218 29679 raft_consensus.cc:2804] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:47.378216 29617 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:47.313279 29768 consensus_queue.cc:1048] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.084s
I20260709 13:10:47.385731 29748 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:47.396216 29191 raft_consensus.cc:2749] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:47.439922 23025 raft_consensus.cc:1275] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 17 index: 16. Preceding OpId from leader: term: 20 index: 18. (index mismatch)
I20260709 13:10:47.392980 29763 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:47.417034 29752 raft_consensus.cc:2749] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:47.442025 28813 consensus_queue.cc:1048] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 13:10:47.455212 22871 raft_consensus.cc:1275] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 11. (term mismatch)
I20260709 13:10:47.455817 22871 pending_rounds.cc:85] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 10
I20260709 13:10:47.456245 22871 pending_rounds.cc:107] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 20.11
I20260709 13:10:47.456581 22871 raft_consensus.cc:2889] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:47.432400 29630 raft_consensus.cc:2749] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:10:47.476564 29779 log.cc:927] Time spent T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
W20260709 13:10:47.476912 29785 log.cc:927] Time spent T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:10:47.450366 29748 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:47.482249 29564 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:47.482861 29564 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:47.437737 29617 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:47.438599 29612 raft_consensus.cc:2749] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:47.457118 29657 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:47.480512 29748 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:47.496304 29789 log.cc:927] Time spent T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 13:10:47.492422 29617 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:47.499423 29547 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:47.462045 29763 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:47.517607 29714 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:47.515379 29564 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:47.520392 29714 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:47.531991 22874 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:35370 (request call id 3299) took 1043 ms. Trace:
I20260709 13:10:47.532431 22874 rpcz_store.cc:276] 0709 13:10:46.488826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:46.496090 (+  7264us) service_pool.cc:224] Handling call
0709 13:10:47.531807 (+1035717us) spinlock_profiling.cc:243] Waited 537 ms on lock 0x7b5400265554. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:47.531931 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1235811328}
I20260709 13:10:47.558207 29613 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:47.558912 29613 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:47.566772 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:47.572299 22874 raft_consensus.cc:1275] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:10:47.507593 29657 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:47.517024 29547 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:47.611294 22874 raft_consensus.cc:1275] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:10:47.622589 22874 raft_consensus.cc:1275] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 13:10:47.643004 22874 raft_consensus.cc:1275] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:10:47.647465 29763 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:47.650681 29547 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:47.652212 22874 raft_consensus.cc:1275] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:10:47.616638 29613 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:47.659642 29657 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:47.319522 22875 rpcz_store.cc:276] 0709 13:10:45.289552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:45.833889 (+544337us) service_pool.cc:224] Handling call
0709 13:10:47.094061 (+1260172us) spinlock_profiling.cc:243] Waited 2.96 ms on lock 0x7b5800000c98. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc0eee 0000000000375939 00007f613ad341e0 00007f613adfd0c4 00007f613adfd131 00007f613adbefc0 00007f613ada9929 00007f613ada93d2 00007f6134320bd4
0709 13:10:47.094215 (+   154us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 13:10:47.094424 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 13:10:47.094692 (+   268us) log.cc:844] Serialized 14 byte log entry
0709 13:10:47.314366 (+219674us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 13:10:47.314414 (+    48us) raft_consensus.cc:1645] Marking committed up to 12
0709 13:10:47.314456 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:47.314521 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:47.314631 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":18863,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":600,"spinlock_wait_cycles":59762816,"thread_start_us":218704,"threads_started":1}
I20260709 13:10:47.674451 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:47.675226 22875 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 21.
I20260709 13:10:47.681244 22875 raft_consensus.cc:1275] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 23 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 13:10:47.685194 22875 raft_consensus.cc:1275] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:10:47.686570 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:47.694176 22951 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52222 (request call id 1202) took 1247 ms. Trace:
I20260709 13:10:47.694732 22951 rpcz_store.cc:276] 0709 13:10:46.446702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:46.453086 (+  6384us) service_pool.cc:224] Handling call
0709 13:10:47.693895 (+1240809us) spinlock_profiling.cc:243] Waited 730 ms on lock 0x7b5400283f54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:47.694100 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1678162304}
I20260709 13:10:47.689797 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:47.699154 23618 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:47.685773 29771 raft_consensus.cc:2749] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:47.703518 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:47.705529 23020 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:47.704304 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:47.707422 23022 raft_consensus.cc:1240] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 19. Current term is 20. Ops: []
I20260709 13:10:47.709664 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:47.710207 23025 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Advancing to term 18
I20260709 13:10:47.712442 22877 raft_consensus.cc:1275] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:10:47.717093 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:47.717535 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:47.720866 22875 raft_consensus.cc:1275] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 11. (term mismatch)
I20260709 13:10:47.721395 22875 pending_rounds.cc:85] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 10
I20260709 13:10:47.721657 22875 pending_rounds.cc:107] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 16.11
I20260709 13:10:47.721926 22875 raft_consensus.cc:2889] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:47.726684 23618 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 20.
I20260709 13:10:47.717814 22874 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 18.
I20260709 13:10:47.730311 23023 raft_consensus.cc:1275] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 13:10:47.732014 22951 raft_consensus.cc:1275] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:10:47.734424 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:47.735668 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:47.737360 23020 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 18.
I20260709 13:10:47.749991 22993 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:47.736907 29764 consensus_queue.cc:1048] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.021s
I20260709 13:10:47.777433 22874 raft_consensus.cc:1275] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 13:10:47.767385 29176 raft_consensus.cc:2804] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:10:47.802915 23618 raft_consensus.cc:1240] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 19 LEADER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 18. Current term is 19. Ops: []
I20260709 13:10:47.809321 29810 consensus_queue.cc:1059] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.004s
I20260709 13:10:47.813721 29810 consensus_queue.cc:1243] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer 5b346767a89745fa82928a5dac9cc0b2 log is divergent from this leader: its last log entry 19.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 13:10:47.796525 29176 raft_consensus.cc:697] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 20 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:47.848840 29176 consensus_queue.cc:237] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:47.863744 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 293767us
I20260709 13:10:47.853204 23025 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18.
I20260709 13:10:47.881072 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:47.881223 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:47.881445 22843 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:10:47.885116 23588 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:47.883634 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:47.891067 23024 raft_consensus.cc:1275] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:10:47.896008 22843 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:47.900112 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:47.910578 23023 raft_consensus.cc:1275] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 13:10:47.910689 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:47.916522 29722 consensus_queue.cc:1048] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:10:47.919137 22918 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:47.920066 29581 raft_consensus.cc:2804] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:10:47.920719 29581 raft_consensus.cc:697] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:47.921886 29581 consensus_queue.cc:237] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:47.922875 29522 consensus_queue.cc:1048] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:10:47.925711 29722 consensus_queue.cc:1048] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:10:47.902697 29750 raft_consensus.cc:2804] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:10:47.934446 29750 raft_consensus.cc:697] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 18 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:47.935614 29750 consensus_queue.cc:237] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:47.902570 29612 raft_consensus.cc:2749] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:10:48.106896 29679 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:48.117322 29679 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:10:48.118814 29803 log.cc:927] Time spent T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.375s	user 0.006s	sys 0.001s
I20260709 13:10:46.247378 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 13:10:48.109849 29788 log.cc:927] Time spent T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.632s	user 0.003s	sys 0.000s
I20260709 13:10:48.123327 22878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 2754) took 1284 ms. Trace:
I20260709 13:10:48.123709 22878 rpcz_store.cc:276] 0709 13:10:46.838844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:46.864917 (+ 26073us) service_pool.cc:224] Handling call
0709 13:10:46.865262 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:46.895852 (+ 30590us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:46.895880 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:46.896315 (+   435us) log.cc:844] Serialized 26 byte log entry
0709 13:10:47.233840 (+337525us) raft_consensus.cc:1645] Marking committed up to 9
0709 13:10:47.293919 (+ 60079us) log.cc:844] Serialized 14 byte log entry
0709 13:10:47.294107 (+   188us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:47.294196 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:47.528110 (+233914us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b1800460540. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cf83f3 00007f6132cfa7c1 00007f6132cfa779 00007f6132cfa709 00007f6132cfa6d1 00007f6132cf99cd 000000000038f354 000000000038f199 00007f6132cdb83a
0709 13:10:48.105401 (+577291us) spinlock_profiling.cc:243] Waited 8.24 ms on lock 0x7b4400022b4c. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132c1ec85 00007f6132c1f6a2 00007f61359911db 00007f613598caaf 00007f613598086e 00007f613597fbb6 00007f613597f7d8 00007f6135989221 00007f61359891d9 00007f6135989169
0709 13:10:48.123117 (+ 17716us) raft_consensus.cc:1685] finished
0709 13:10:48.123146 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:48.123272 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30212,"prepare.queue_time_us":77409,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":41643,"raft.run_cpu_time_us":278,"raft.run_wall_time_us":67769,"spinlock_wait_cycles":42755584,"thread_start_us":239834,"threads_started":1,"wal-append.queue_time_us":574302}
I20260709 13:10:48.109727 29685 raft_consensus.cc:3060] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:48.110281 22872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 2752) took 1272 ms. Trace:
I20260709 13:10:48.131037 22872 rpcz_store.cc:276] 0709 13:10:46.838332 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:46.859049 (+ 20717us) service_pool.cc:224] Handling call
0709 13:10:46.859441 (+   392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:46.900061 (+ 40620us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:46.900092 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:46.912890 (+ 12798us) log.cc:844] Serialized 26 byte log entry
0709 13:10:47.382628 (+469738us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:10:47.693083 (+310455us) log.cc:844] Serialized 14 byte log entry
0709 13:10:47.696411 (+  3328us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:48.017206 (+320795us) spinlock_profiling.cc:243] Waited 14.8 ms on lock 0x7b540022b5d4. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de5e01 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2 00007f613412048c 00007f613411f8c6
0709 13:10:48.017276 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:48.109997 (+ 92721us) raft_consensus.cc:1685] finished
0709 13:10:48.110086 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:48.110220 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":84773,"prepare.queue_time_us":147491,"prepare.run_cpu_time_us":24,"prepare.run_wall_time_us":24,"raft.queue_time_us":32800,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":4204,"spinlock_wait_cycles":33929088,"thread_start_us":667818,"threads_started":2,"wal-append.queue_time_us":594022}
I20260709 13:10:48.135578 29679 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:48.112145 29813 log.cc:927] Time spent T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.281s	user 0.010s	sys 0.001s
I20260709 13:10:48.113111 22946 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:58320 (request call id 3436) took 1038 ms. Trace:
I20260709 13:10:47.899102 29763 consensus_queue.cc:1059] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.041s
I20260709 13:10:48.141664 22946 rpcz_store.cc:276] 0709 13:10:47.075015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:47.100384 (+ 25369us) service_pool.cc:224] Handling call
0709 13:10:47.994146 (+893762us) spinlock_profiling.cc:243] Waited 55.2 ms on lock 0x7b5400285854. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4e571 00007f6135de92fa 00007f613adaab17 00007f61343256d4 00007f6134325662 00007f6134325591 00007f613432550c 00007f61343247c2 00007f613412048c 00007f613411f8c6 00007f613411f26f
0709 13:10:48.113037 (+118891us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":126896512}
I20260709 13:10:48.142066 29763 raft_consensus.cc:3055] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:10:48.143532 29763 raft_consensus.cc:740] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:48.144546 29763 consensus_queue.cc:260] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:48.147399 29816 log.cc:927] Time spent T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.307s	user 0.000s	sys 0.012s
I20260709 13:10:48.146473 29763 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:48.151499 29810 raft_consensus.cc:3055] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:10:48.152293 29810 raft_consensus.cc:740] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:48.153270 29810 consensus_queue.cc:260] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:48.154723 29810 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:48.138088 29679 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:48.179348 22946 raft_consensus.cc:1275] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:10:48.182658 22843 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:48.196024 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:48.196700 23622 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:48.196044 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:48.180968 29557 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:48.199158 29527 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:48.206310 29557 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:48.206760 29635 consensus_queue.cc:1048] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.003s
I20260709 13:10:48.208737 29763 raft_consensus.cc:2749] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:48.206660 29527 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:48.210214 23617 raft_consensus.cc:1275] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 13:10:48.210604 23617 pending_rounds.cc:85] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2: Aborting all ops after (but not including) 12
I20260709 13:10:48.210820 23617 pending_rounds.cc:107] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 13:10:48.211041 23617 raft_consensus.cc:2889] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:48.211184 29527 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:48.208510 29557 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:10:48.211949 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 241068us
I20260709 13:10:48.212950 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:48.214529 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:48.215214 29493 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:48.215667 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:48.215706 29493 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:48.217623 22949 raft_consensus.cc:1275] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:10:48.226224 22844 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:48.227710 22844 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:48.229772 22844 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:48.187220 29679 raft_consensus.cc:493] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:48.254659 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:48.263774 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:48.264602 23617 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 21.
I20260709 13:10:48.238112 22826 placement_policy_util.cc:248] tablet ac5cc94926a742e6a0c1340ec33c0ff0: not considering replicas for movement since the tablet's status is 'UNAVAILABLE'
I20260709 13:10:48.274501 23622 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 18.
W20260709 13:10:48.277099 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 47429.1us
I20260709 13:10:48.252434 29750 raft_consensus.cc:2749] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:10:48.230865 29763 raft_consensus.cc:2749] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:48.252461 29612 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:48.283833 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:48.284190 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:48.301460 22872 raft_consensus.cc:2393] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 18: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:48.286082 29520 consensus_queue.cc:1048] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 13:10:48.286825 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:48.266392 29266 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:48.294409 22880 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53724 (request call id 1274) took 1224 ms. Trace:
I20260709 13:10:48.280856 29771 raft_consensus.cc:2749] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:48.253680 29581 consensus_queue.cc:1048] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 13:10:48.252768 29557 consensus_queue.cc:1048] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
I20260709 13:10:48.306951 22880 rpcz_store.cc:276] 0709 13:10:47.070518 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:47.320497 (+249979us) service_pool.cc:224] Handling call
0709 13:10:47.320889 (+   392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:47.378157 (+ 57268us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 13:10:47.735293 (+357136us) log.cc:844] Serialized 14 byte log entry
0709 13:10:47.878140 (+142847us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:47.878570 (+   430us) log.cc:844] Serialized 26 byte log entry
0709 13:10:47.878696 (+   126us) raft_consensus.cc:1645] Marking committed up to 14
0709 13:10:47.878725 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:47.878806 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:48.294197 (+415391us) raft_consensus.cc:1685] finished
0709 13:10:48.294221 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:48.294350 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":94044,"prepare.queue_time_us":151768,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":60876,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":28006,"thread_start_us":435443,"threads_started":2,"wal-append.queue_time_us":373086}
I20260709 13:10:48.284475 29493 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:48.278100 29700 consensus_queue.cc:1048] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 13:10:48.279649 29679 raft_consensus.cc:515] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:48.309671 29728 consensus_queue.cc:1048] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.010s
I20260709 13:10:48.300808 29191 consensus_queue.cc:1048] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.020s
I20260709 13:10:48.306558 29715 consensus_queue.cc:1048] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 13:10:48.303534 29612 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:48.355298 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 72683.2us
I20260709 13:10:48.310324 29714 consensus_queue.cc:1048] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.038s
I20260709 13:10:48.310757 29266 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:48.313591 29527 consensus_queue.cc:1048] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.037s
W20260709 13:10:48.375625 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 157563us
I20260709 13:10:48.385597 29553 consensus_queue.cc:1048] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:10:48.395983 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 126796us
I20260709 13:10:48.375847 29763 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:48.398170 22992 leader_election.cc:400] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 17. Current term is 18.
I20260709 13:10:48.398648 22992 leader_election.cc:403] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:10:48.399014 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 139096us
I20260709 13:10:48.399533 22880 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
mode: GRACEFUL
new_leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:40244
I20260709 13:10:48.400164 22880 raft_consensus.cc:606] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Received request to transfer leadership to TS aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:48.400578 22880 raft_consensus.cc:614] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:10:48.402887 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 121602us
W20260709 13:10:48.398742 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 104479us
I20260709 13:10:48.423848 22878 raft_consensus.cc:1275] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:10:48.419916 22949 raft_consensus.cc:1240] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 18. Current term is 19. Ops: []
I20260709 13:10:48.428586 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:48.419705 23020 raft_consensus.cc:1275] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 13:10:48.441318 22919 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:48.405593 29763 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:48.450384 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:48.450701 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:48.451048 23028 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 18.
I20260709 13:10:48.468113 29685 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:48.468815 29685 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:48.470877 29685 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:48.494778 29612 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:48.495409 29612 raft_consensus.cc:493] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:48.497876 29266 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:48.495824 29612 raft_consensus.cc:515] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:48.402624 29750 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:48.402969 29768 consensus_queue.cc:1048] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.100s
I20260709 13:10:48.499105 29679 leader_election.cc:290] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:48.527623 23026 raft_consensus.cc:1275] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 15. (term mismatch)
I20260709 13:10:48.509613 29837 consensus_queue.cc:1048] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 13:10:48.537998 29612 leader_election.cc:290] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:48.519491 29750 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:48.548959 29750 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:48.551452 29828 log.cc:927] Time spent T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 13:10:48.559275 29763 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:48.559849 22776 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33466 (request call id 294) took 18434 ms (client timeout 15000 ms). Trace:
I20260709 13:10:48.567162 23026 pending_rounds.cc:85] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75: Aborting all ops after (but not including) 14
I20260709 13:10:48.569818 23026 pending_rounds.cc:107] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75: Aborting uncommitted NO_OP operation due to leader change: 18.15
I20260709 13:10:48.570152 23026 raft_consensus.cc:2889] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:48.574180 22871 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
mode: GRACEFUL
new_leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:40244
I20260709 13:10:48.575084 22871 raft_consensus.cc:606] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Received request to transfer leadership to TS aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:48.575562 22871 raft_consensus.cc:614] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:10:48.508961 29722 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:10:48.584867 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 184343us
I20260709 13:10:48.594875 23590 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:10:48.567498 22776 rpcz_store.cc:269] 0709 13:10:30.125396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:30.129015 (+  3619us) service_pool.cc:224] Handling call
0709 13:10:48.559717 (+18430702us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:10:43.647039 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:10:43.766801 (+119762us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:10:43.770548 (+  3747us) write_op.cc:435] Acquired schema lock
0709 13:10:43.771443 (+   895us) tablet.cc:662] Decoding operations
0709 13:10:44.126161 (+354718us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:10:44.126269 (+   108us) write_op.cc:641] Partition lock acquired for write op
0709 13:10:44.126295 (+    26us) tablet.cc:685] Acquiring locks for 28 operations
0709 13:10:44.131053 (+  4758us) tablet.cc:701] Row locks acquired
0709 13:10:44.131116 (+    63us) write_op.cc:260] PREPARE: finished
0709 13:10:44.131360 (+   244us) write_op.cc:270] Start()
0709 13:10:44.131547 (+   187us) write_op.cc:276] Timestamp: P: 1783602644131277 usec, L: 0
0709 13:10:44.131579 (+    32us) op_driver.cc:348] REPLICATION: starting
0709 13:10:44.145929 (+ 14350us) log.cc:844] Serialized 10858 byte log entry
0709 13:10:45.001088 (+855159us) op_driver.cc:464] REPLICATION: finished
0709 13:10:45.048778 (+ 47690us) write_op.cc:301] APPLY: starting
0709 13:10:45.048912 (+   134us) tablet.cc:1367] starting BulkCheckPresence
0709 13:10:45.219208 (+170296us) tablet.cc:1370] finished BulkCheckPresence
0709 13:10:45.219333 (+   125us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:10:45.736808 (+517475us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:10:45.737261 (+   453us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=56,key_file_lookups=56,delta_file_lookups=56,mrs_lookups=0
0709 13:10:45.737325 (+    64us) write_op.cc:312] APPLY: finished
0709 13:10:45.737906 (+   581us) log.cc:844] Serialized 244 byte log entry
0709 13:10:45.770141 (+ 32235us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:45.784873 (+ 14732us) write_op.cc:454] Released schema lock
0709 13:10:45.784988 (+   115us) write_op.cc:341] FINISH: Updating metrics
0709 13:10:45.785438 (+   450us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:45.785469 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":833442,"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":44166,"apply.run_cpu_time_us":40131,"apply.run_wall_time_us":736756,"cfile_cache_hit":114,"cfile_cache_hit_bytes":121223,"num_ops":28,"prepare.queue_time_us":295444,"prepare.run_cpu_time_us":73054,"prepare.run_wall_time_us":836718,"raft.queue_time_us":304887,"raft.run_cpu_time_us":6706,"raft.run_wall_time_us":94037,"replication_time_us":854006,"thread_start_us":525236,"threads_started":4,"wal-append.queue_time_us":326175,"wal-append.run_cpu_time_us":7736,"wal-append.run_wall_time_us":2226737}]]}
I20260709 13:10:48.570436 22772 catalog_manager.cc:5697] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:48.525879 29493 raft_consensus.cc:2749] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 17. Current term is 18.
I20260709 13:10:48.583988 29722 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:48.644533 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:48.647799 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:48.648257 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:48.648514 22877 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 20.
I20260709 13:10:48.647351 22946 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 21.
I20260709 13:10:48.656838 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:48.659183 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:48.659611 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:48.661927 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:48.662393 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:48.652232 29722 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:48.607577 29612 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:48.635998 29563 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:48.626925 29833 raft_consensus.cc:2749] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:48.655038 29773 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:48.710345 22880 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
mode: GRACEFUL
new_leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:40244
I20260709 13:10:48.673991 29851 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
W20260709 13:10:48.733516 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 305557us
W20260709 13:10:48.742587 22826 auto_rebalancer.cc:629] Failed to schedule move for tablet 104e2273f37843cbb5245981c817dfe0: Illegal state: Replica 0a407504ed1741bf89a276558adf4c0f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 20 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:48.677907 29612 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:48.745886 22764 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 7426119f4da44ffaa1513b481a866a01 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER failed with leader 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 0a407504ed1741bf89a276558adf4c0f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 21 committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
W20260709 13:10:48.746615 22764 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 72038us
I20260709 13:10:48.762693 22880 raft_consensus.cc:606] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Received request to transfer leadership to TS aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:48.763161 22880 raft_consensus.cc:614] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:10:48.699783 29563 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:48.770684 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 101739us
I20260709 13:10:48.714653 29844 consensus_queue.cc:1048] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 13:10:48.725066 29773 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:48.823309 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 158273us
W20260709 13:10:48.920827 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 207098us
W20260709 13:10:48.898770 29802 log.cc:927] Time spent T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 13:10:48.746726 29851 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:49.049324 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 342665us
W20260709 13:10:49.055387 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 317213us
I20260709 13:10:48.749565 29657 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:10:49.069643 29150 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.004s	sys 0.018s
W20260709 13:10:49.208492 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 421562us
I20260709 13:10:48.837144 29547 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
W20260709 13:10:49.208730 22921 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.000s	sys 0.023s
W20260709 13:10:49.246760 22921 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.000s	sys 0.024s
I20260709 13:10:48.752882 29847 consensus_queue.cc:1048] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.137s
W20260709 13:10:49.210125 29176 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.007s	sys 0.006s
I20260709 13:10:48.837412 29750 raft_consensus.cc:493] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:49.109077 29613 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:49.141170 29679 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
W20260709 13:10:49.259150 29150 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.004s	sys 0.018s
I20260709 13:10:49.220184 29522 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:10:49.311439 23024 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.019s	sys 0.047s
W20260709 13:10:49.312229 23024 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.020s	sys 0.047s
W20260709 13:10:49.269491 29176 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.008s	sys 0.006s
W20260709 13:10:49.336133 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 117768us
I20260709 13:10:49.338244 23588 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:49.319180 29507 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:49.269234 29657 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:49.285419 29679 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:49.362461 29657 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:49.269780 29547 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:49.362709 29679 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:49.363884 29574 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:49.364432 29547 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:49.365411 29849 log.cc:927] Time spent T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
W20260709 13:10:49.366667 29845 log.cc:927] Time spent T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 13:10:49.328055 29522 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:49.369602 29612 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:49.369760 29522 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:49.370249 29522 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:49.370919 29522 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:49.372552 29833 raft_consensus.cc:2804] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:49.373063 29833 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:49.373413 29833 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:49.370292 29877 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:49.374248 29877 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:49.365525 29850 log.cc:927] Time spent T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 13:10:49.375617 29851 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:49.376318 29768 consensus_queue.cc:1048] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:10:49.276286 29750 raft_consensus.cc:515] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:49.284909 29613 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:49.389092 29563 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:49.389664 29750 leader_election.cc:290] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:49.386135 29835 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:49.390663 29835 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:49.391031 29613 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:49.384244 29679 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:49.402464 29877 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:49.431252 29864 log.cc:927] Time spent T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.109s	user 0.002s	sys 0.000s
I20260709 13:10:49.435379 29835 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:49.486214 22772 catalog_manager.cc:5697] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:49.496132 22772 catalog_manager.cc:5697] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 6 to 10, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 10 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:10:49.511210 22920 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.840s	user 0.018s	sys 0.028s
W20260709 13:10:49.511844 22920 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.841s	user 0.018s	sys 0.028s
W20260709 13:10:49.512516 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 848538us
I20260709 13:10:49.513659 22879 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 2773) took 1088 ms. Trace:
W20260709 13:10:49.516821 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 276815us
I20260709 13:10:49.518065 23024 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 1173) took 1113 ms. Trace:
I20260709 13:10:49.518622 23024 rpcz_store.cc:276] 0709 13:10:48.404019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:48.404191 (+   172us) service_pool.cc:224] Handling call
0709 13:10:48.404621 (+   430us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:10:48.651377 (+246756us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:48.651413 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:10:48.670476 (+ 19063us) log.cc:844] Serialized 52 byte log entry
0709 13:10:49.312881 (+642405us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:10:49.345176 (+ 32295us) log.cc:844] Serialized 14 byte log entry
0709 13:10:49.517159 (+171983us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:49.517350 (+   191us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:49.517446 (+    96us) raft_consensus.cc:1685] finished
0709 13:10:49.517470 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:49.517716 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":211671,"prepare.queue_time_us":45372,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":114408,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":40907,"thread_start_us":833871,"threads_started":2,"wal-append.queue_time_us":701017}
I20260709 13:10:49.522411 22994 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:49.525913 22994 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:49.529697 23029 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:32794 (request call id 1176) took 1123 ms. Trace:
I20260709 13:10:49.511839 22872 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:35360 (request call id 2771) took 1089 ms. Trace:
I20260709 13:10:49.514225 22879 rpcz_store.cc:276] 0709 13:10:48.424820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:48.424960 (+   140us) service_pool.cc:224] Handling call
0709 13:10:48.425340 (+   380us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:10:48.438826 (+ 13486us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 13:10:48.578701 (+139875us) log.cc:844] Serialized 14 byte log entry
0709 13:10:48.755746 (+177045us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:10:48.756434 (+   688us) log.cc:844] Serialized 52 byte log entry
0709 13:10:48.756604 (+   170us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:10:48.773730 (+ 17126us) log.cc:844] Serialized 14 byte log entry
0709 13:10:48.774179 (+   449us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:48.774277 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:49.456111 (+681834us) spinlock_profiling.cc:243] Waited 8.08 ms on lock 0x7b18003efd40. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cf83f3 00007f6132cfa7c1 00007f6132cfa779 00007f6132cfa709 00007f6132cfa6d1 00007f6132cf99cd 000000000038f354 000000000038f199 00007f6132cdb83a
0709 13:10:49.513150 (+ 57039us) raft_consensus.cc:1685] finished
0709 13:10:49.513185 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:49.513467 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":287473,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":77,"raft.queue_time_us":4562,"raft.run_cpu_time_us":296,"raft.run_wall_time_us":32973,"spinlock_wait_cycles":37195392,"thread_start_us":241582,"threads_started":2,"wal-append.queue_time_us":799927}
I20260709 13:10:49.565692 22872 rpcz_store.cc:276] 0709 13:10:48.421860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:48.422000 (+   140us) service_pool.cc:224] Handling call
0709 13:10:48.422323 (+   323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:48.422649 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:48.422675 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:48.439890 (+ 17215us) log.cc:844] Serialized 26 byte log entry
0709 13:10:48.654585 (+214695us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:10:48.654626 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:48.654695 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:49.511240 (+856545us) raft_consensus.cc:1685] finished
0709 13:10:49.511302 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:49.511608 (+   306us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":449,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":18479,"thread_start_us":214354,"threads_started":1,"wal-append.queue_time_us":312440}
I20260709 13:10:49.537146 22920 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:49.537971 23029 rpcz_store.cc:276] 0709 13:10:48.406019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:48.406893 (+   874us) service_pool.cc:224] Handling call
0709 13:10:48.445687 (+ 38794us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:10:48.461561 (+ 15874us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:48.461592 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:10:48.466790 (+  5198us) log.cc:844] Serialized 52 byte log entry
0709 13:10:48.824927 (+358137us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:10:48.824991 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:48.825092 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:49.529037 (+703945us) raft_consensus.cc:1685] finished
0709 13:10:49.529094 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:49.529470 (+   376us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":104272,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":127,"thread_start_us":287781,"threads_started":1,"wal-append.queue_time_us":266918}
W20260709 13:10:49.413645 29722 consensus_meta.cc:287] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.747s	user 0.013s	sys 0.000s
I20260709 13:10:49.386439 29547 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:49.606225 29878 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:49.607537 29750 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:49.607398 29878 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:49.364445 29574 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:49.383025 29507 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:49.611801 29722 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:49.617846 29878 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:49.621114 29810 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:49.621398 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:49.622229 29806 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:49.637848 29613 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:49.638294 29630 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:49.638751 29613 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:49.639142 29630 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:49.637852 29612 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:49.641902 29612 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:49.641158 29700 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:49.642763 29493 consensus_queue.cc:1048] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:10:49.643755 29700 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:49.646596 29612 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:49.649477 29810 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:49.652086 29810 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:49.653002 29553 consensus_queue.cc:1048] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:10:49.655107 29613 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:49.413118 29679 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:49.656816 29877 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:49.657761 29877 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:49.658524 29679 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:49.665880 29507 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:49.665782 29773 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:49.666958 29853 log.cc:927] Time spent T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.409s	user 0.006s	sys 0.000s
W20260709 13:10:49.667580 29854 log.cc:927] Time spent T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.424s	user 0.006s	sys 0.000s
I20260709 13:10:49.668042 29547 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:49.670012 29806 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:49.659804 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 144203us
I20260709 13:10:49.684032 23022 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 2743) took 1465 ms. Trace:
I20260709 13:10:49.684161 23622 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 1430) took 1026 ms. Trace:
I20260709 13:10:49.711953 23622 rpcz_store.cc:276] 0709 13:10:48.635898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:48.636016 (+   118us) service_pool.cc:224] Handling call
0709 13:10:48.636324 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:48.636737 (+   413us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:48.636786 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:48.637415 (+   629us) log.cc:844] Serialized 26 byte log entry
0709 13:10:49.073294 (+435879us) raft_consensus.cc:1645] Marking committed up to 13
0709 13:10:49.324894 (+251600us) log.cc:844] Serialized 14 byte log entry
0709 13:10:49.325132 (+   238us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:49.325222 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:49.590612 (+265390us) raft_consensus.cc:1685] finished
0709 13:10:49.593357 (+  2745us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:49.662071 (+ 68714us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12978,"prepare.queue_time_us":326418,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":65045,"raft.run_cpu_time_us":281,"raft.run_wall_time_us":5967,"thread_start_us":543685,"threads_started":2,"wal-append.queue_time_us":752335}
I20260709 13:10:49.608546 29750 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:49.721020 29750 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:49.684516 23022 rpcz_store.cc:276] 0709 13:10:48.218494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:48.218607 (+   113us) service_pool.cc:224] Handling call
0709 13:10:48.218878 (+   271us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:48.397998 (+179120us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 13:10:48.436932 (+ 38934us) log.cc:844] Serialized 14 byte log entry
0709 13:10:48.682084 (+245152us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:49.528514 (+846430us) spinlock_profiling.cc:243] Waited 2.39 ms on lock 0x7b2c000bc418. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f61346dea11 00007f61346ee372 00007f6135e20a01 00007f6135de543a 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2
0709 13:10:49.641695 (+113181us) log.cc:844] Serialized 26 byte log entry
0709 13:10:49.642078 (+   383us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:10:49.642164 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:49.642286 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:49.671872 (+ 29586us) raft_consensus.cc:1685] finished
0709 13:10:49.677061 (+  5189us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:49.683839 (+  6778us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45207,"prepare.queue_time_us":42255,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":2588,"raft.queue_time_us":46685,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":15177,"spinlock_wait_cycles":5507456,"thread_start_us":355716,"threads_started":2,"wal-append.queue_time_us":314658}
I20260709 13:10:49.689054 29860 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:49.692811 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:49.693673 29574 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:49.696772 23618 raft_consensus.cc:1275] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:10:49.696987 29522 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:49.698402 29830 log.cc:927] Time spent T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 1.091s	user 0.002s	sys 0.015s
I20260709 13:10:49.699244 29889 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:10:49.699932 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 174290us
I20260709 13:10:49.705103 29883 raft_consensus.cc:2749] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:49.705406 22772 catalog_manager.cc:5697] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 12 to 16, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 16 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:49.708999 29851 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:49.674727 29612 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:49.677529 29613 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:10:49.711024 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 420045us
I20260709 13:10:49.708222 29176 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:49.738876 29773 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:49.673364 29878 raft_consensus.cc:493] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:49.729036 29889 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:49.752065 29851 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:49.761183 29889 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:49.762100 29773 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:49.764977 23617 raft_consensus.cc:2393] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 21: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:49.765223 29612 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:49.765874 29176 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:49.768131 29612 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:49.768383 29127 consensus_queue.cc:1048] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:10:49.768951 29613 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:49.769959 23025 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:33420 (request call id 2742) took 1554 ms. Trace:
I20260709 13:10:49.771445 29613 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:49.770346 23025 rpcz_store.cc:276] 0709 13:10:48.215616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:48.215735 (+   119us) service_pool.cc:224] Handling call
0709 13:10:48.216047 (+   312us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 13:10:48.216374 (+   327us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:48.216400 (+    26us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 13:10:48.217046 (+   646us) log.cc:844] Serialized 52 byte log entry
0709 13:10:48.378154 (+161108us) raft_consensus.cc:1645] Marking committed up to 11
0709 13:10:48.378214 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:48.378315 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:49.697673 (+1319358us) spinlock_profiling.cc:243] Waited 5.77 ms on lock 0x7b240025adc8. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132b7e9c4 00007f6132b7cacb 00007f6132d37dc9 00007f6132d37a55 00007f6132d3709d 00007f613596910e 00007f613598088c 00007f613597fbb6 00007f613597f7d8 00007f6135989221
0709 13:10:49.769485 (+ 71812us) raft_consensus.cc:1685] finished
0709 13:10:49.769506 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:49.769776 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18664,"raft.queue_time_us":14175,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":5090,"spinlock_wait_cycles":13270272,"thread_start_us":160647,"threads_started":1,"wal-append.queue_time_us":291883}
I20260709 13:10:49.773339 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:49.774150 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:49.774390 29522 raft_consensus.cc:493] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:49.774922 22955 raft_consensus.cc:2368] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 21. Current term is 22.
I20260709 13:10:49.775166 29522 raft_consensus.cc:515] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:49.775372 22772 catalog_manager.cc:5697] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 12 to 15, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 15 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:49.751978 29860 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:49.760349 29883 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:49.784401 29773 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:49.805294 29806 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:49.776865 29714 raft_consensus.cc:2804] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:49.804641 29833 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:49.810938 29176 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:49.812207 29700 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:49.813407 29722 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:49.815397 29522 leader_election.cc:290] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:49.815630 29630 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:49.779793 29878 raft_consensus.cc:515] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:49.845002 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 496429us
I20260709 13:10:49.845216 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:49.844511 29878 leader_election.cc:290] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:49.818699 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 53686.1us
I20260709 13:10:49.841315 29833 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:49.846746 29883 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:49.846558 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:49.849675 29883 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:49.849792 29851 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:49.851248 29613 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:49.852087 29860 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:49.852244 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:49.853241 29612 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:49.853994 29612 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:49.854683 29547 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:49.855262 29547 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:49.855873 29547 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:49.857124 29878 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:49.857860 29878 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:49.858187 22826 auto_rebalancer.cc:629] Failed to schedule move for tablet 7cd8492f63fb4193a47cf9fbc574ef58: Illegal state: Replica 0a407504ed1741bf89a276558adf4c0f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 21 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:49.860244 29878 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:49.852079 29613 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:49.864781 29613 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:49.856407 29612 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:49.869125 29852 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:49.870124 29852 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:49.870913 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 75299.7us
I20260709 13:10:49.871193 22950 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52222 (request call id 1237) took 1240 ms. Trace:
I20260709 13:10:49.871404 29176 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:49.871640 22950 rpcz_store.cc:276] 0709 13:10:48.630491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:48.630625 (+   134us) service_pool.cc:224] Handling call
0709 13:10:48.652979 (+ 22354us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:48.653369 (+   390us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:48.653399 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:49.517335 (+863936us) spinlock_profiling.cc:243] Waited 1.13 ms on lock 0x7b1800472d80. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cc62a6 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f61346dea11 00007f61346ee372 00007f6135e20a01 00007f6135de543a
0709 13:10:49.517951 (+   616us) spinlock_profiling.cc:243] Waited 14.8 ms on lock 0x7b2c0005beb8. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f61346dea11 00007f61346ee372 00007f6135e20a01 00007f6135de543a 00007f6135de483e 00007f613ada9419 00007f6134320bd4 00007f6134320b62 00007f6134320a91 00007f6134320a0c 00007f613431fcc2
0709 13:10:49.519098 (+  1147us) log.cc:844] Serialized 26 byte log entry
0709 13:10:49.604775 (+ 85677us) raft_consensus.cc:1645] Marking committed up to 10
0709 13:10:49.604821 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:49.636285 (+ 31464us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:49.870766 (+234481us) raft_consensus.cc:1685] finished
0709 13:10:49.870787 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:49.870974 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":148,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":18926,"spinlock_wait_cycles":36695808,"thread_start_us":85118,"threads_started":1,"wal-append.queue_time_us":308049}
I20260709 13:10:49.873829 29852 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:49.872169 29176 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:49.874657 29768 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:49.876065 29833 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:49.846653 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:49.818675 29877 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:49.835222 29714 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:10:49.896307 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 48056us
I20260709 13:10:49.898196 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:49.896654 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 50216.3us
I20260709 13:10:49.901859 22992 leader_election.cc:400] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 21. Current term is 22.
I20260709 13:10:49.902608 22992 leader_election.cc:403] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
W20260709 13:10:49.903621 23591 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.006s	sys 0.010s
I20260709 13:10:49.903827 29630 raft_consensus.cc:493] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
W20260709 13:10:49.904103 22764 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 7426119f4da44ffaa1513b481a866a01 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER failed with leader 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 0a407504ed1741bf89a276558adf4c0f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 21 committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
W20260709 13:10:49.904366 23591 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.006s	sys 0.010s
I20260709 13:10:49.904561 29630 raft_consensus.cc:515] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:49.907837 29630 leader_election.cc:290] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:49.908759 29685 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:49.909125 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:49.910084 22880 raft_consensus.cc:2393] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 18: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:49.909715 29685 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:49.912037 29685 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:49.911943 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:49.889027 29176 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:49.875559 29768 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:49.894863 29714 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:49.886930 29833 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:49.934870 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 71372.6us
I20260709 13:10:49.937945 29630 raft_consensus.cc:493] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:49.960269 29547 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:49.963691 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 44954.4us
I20260709 13:10:49.964802 29700 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:49.972102 22871 raft_consensus.cc:1275] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 15. (term mismatch)
I20260709 13:10:49.973062 29833 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:49.973518 22843 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:49.956946 29176 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:10:49.935765 29851 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:50.008584 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:50.008838 29564 raft_consensus.cc:493] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:49.989593 22772 catalog_manager.cc:5697] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 3 to 11, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 11 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:49.983008 22871 pending_rounds.cc:85] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 14
I20260709 13:10:50.021848 23618 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 18.
I20260709 13:10:50.029573 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:50.034528 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:50.035579 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:50.025461 22871 pending_rounds.cc:107] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 19.15
I20260709 13:10:50.047288 29883 consensus_queue.cc:1048] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:10:50.054658 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:50.056041 29889 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:50.060109 22953 raft_consensus.cc:1240] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 20. Current term is 21. Ops: []
I20260709 13:10:50.065181 29867 consensus_queue.cc:1059] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:10:50.066160 29867 raft_consensus.cc:3055] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:10:50.066552 29867 raft_consensus.cc:740] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:50.067729 29867 consensus_queue.cc:260] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:50.069159 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:50.069060 29630 raft_consensus.cc:515] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:50.070401 29728 raft_consensus.cc:493] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:50.070688 29752 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:50.071169 29728 raft_consensus.cc:515] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:50.071653 29630 leader_election.cc:290] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:50.071400 29752 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:50.065382 29806 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:50.073388 29728 leader_election.cc:290] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:50.073484 29806 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:50.074643 29867 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:50.074775 29882 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:50.075119 29838 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:50.075543 29882 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:50.076025 29806 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:50.075784 29838 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:50.070380 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:50.076768 29700 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:50.077987 29882 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:50.078979 29700 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:50.079931 29882 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:50.079629 29851 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:50.082701 29877 raft_consensus.cc:2749] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:10:50.080639 29882 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:50.083123 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 168276us
I20260709 13:10:50.082166 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:50.045408 22871 raft_consensus.cc:2889] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:50.084877 29882 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:50.082285 29851 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:50.087991 22993 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:50.073644 29752 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:10:50.090677 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:50.091457 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:50.091535 29564 raft_consensus.cc:515] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:50.092177 22871 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 21.
I20260709 13:10:50.033453 29547 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:50.076570 29806 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:50.094041 29547 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:50.094830 29806 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:50.098371 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:50.098527 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:50.099167 22871 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 17.
I20260709 13:10:50.073572 29768 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:50.100344 22874 raft_consensus.cc:1240] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 18. Current term is 19. Ops: []
I20260709 13:10:50.102166 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:50.103022 22877 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18.
I20260709 13:10:50.103180 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:50.104068 22874 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 17.
W20260709 13:10:50.086470 29896 log.cc:927] Time spent T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 13:10:50.105595 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:50.109112 22826 auto_rebalancer.cc:629] Failed to schedule move for tablet b7e4ee754b7a4408b5fc55259d431f39: Illegal state: Replica 0a407504ed1741bf89a276558adf4c0f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 20 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:50.102453 29806 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:50.096370 29547 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:50.120823 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:50.126670 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:50.128468 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:50.129518 22877 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 21.
I20260709 13:10:50.130738 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:50.139863 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:50.138019 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:50.143963 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:50.147270 29895 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:50.147782 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:50.148180 29895 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:50.149827 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:50.151714 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:50.152347 22874 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:50.155973 22878 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53724 (request call id 1283) took 1413 ms. Trace:
I20260709 13:10:50.156912 29878 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:50.157594 29878 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:50.158633 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:50.159276 23620 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:50.159759 29878 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:50.160310 29878 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:50.164036 29894 raft_consensus.cc:493] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:50.164283 29878 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:50.164724 29894 raft_consensus.cc:515] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:50.166121 29564 leader_election.cc:290] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:50.166785 29894 leader_election.cc:290] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:50.160912 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:50.161098 29851 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:50.167670 29851 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:50.168486 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:50.169242 22873 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18.
I20260709 13:10:50.170083 29851 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:50.161532 29877 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:50.161965 23022 raft_consensus.cc:2410] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for term 21 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 13:10:50.171283 29877 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:50.170069 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:50.173415 29877 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:50.174122 23588 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:50.156502 22878 rpcz_store.cc:276] 0709 13:10:48.620341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:48.621428 (+  1087us) service_pool.cc:224] Handling call
0709 13:10:48.621706 (+   278us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:48.622037 (+   331us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:48.622066 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:48.631706 (+  9640us) log.cc:844] Serialized 26 byte log entry
0709 13:10:49.691322 (+1059616us) spinlock_profiling.cc:243] Waited 2.29 ms on lock 0x7b18003d3840. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cdae07 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf7c02 00007f613597f20b 00007f6135984d23 00007f61359850d1 00007f6135da0a0f
0709 13:10:49.953112 (+261790us) raft_consensus.cc:1645] Marking committed up to 15
0709 13:10:49.957135 (+  4023us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:49.957359 (+   224us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:49.981211 (+ 23852us) raft_consensus.cc:1685] finished
0709 13:10:49.981280 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:50.033620 (+ 52340us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6188,"raft.run_cpu_time_us":166,"raft.run_wall_time_us":8313,"spinlock_wait_cycles":5256960,"thread_start_us":241698,"threads_started":1,"wal-append.queue_time_us":1329096}
I20260709 13:10:50.163467 29895 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:50.160452 29768 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:50.177016 29685 raft_consensus.cc:2804] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:10:50.176641 29768 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:50.177565 29685 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:50.178066 29685 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:50.179445 29768 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:10:50.187657 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 48607.3us
I20260709 13:10:50.192692 22874 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 22.
I20260709 13:10:50.198606 23620 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 22.
I20260709 13:10:50.202666 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:50.203599 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:50.212237 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:50.212911 22953 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:50.213492 22953 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 18.
I20260709 13:10:50.215039 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:50.216851 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:50.217073 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:50.218341 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:50.218976 22952 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 24.
I20260709 13:10:50.218994 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:50.190773 29852 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:50.222810 23026 raft_consensus.cc:1275] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 13:10:50.224360 29878 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:50.230342 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:50.238615 29547 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:50.252142 23588 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:10:50.256210 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 52474.7us
I20260709 13:10:50.261394 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:50.261349 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:50.260265 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:50.271297 22992 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:50.277235 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:50.252820 29630 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:50.284943 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:50.285321 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:50.244830 29547 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:50.296571 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:50.310710 22992 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:50.310022 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:50.314604 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:50.265614 29887 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:50.321553 29630 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:50.334014 29844 raft_consensus.cc:2749] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:50.309726 29889 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 21 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:50.335124 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 83095.9us
I20260709 13:10:50.339015 29714 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:50.299772 29752 raft_consensus.cc:2804] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:50.332248 29630 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:50.349905 22993 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:50.353780 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:50.356462 22993 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:50.305666 29878 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:50.357535 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:50.360291 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:50.328087 29852 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:10:50.363170 22947 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 22.
I20260709 13:10:50.369666 23029 raft_consensus.cc:2368] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 18. Current term is 19.
I20260709 13:10:50.327133 29895 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:50.367794 29809 raft_consensus.cc:2804] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:50.351723 29752 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:50.375028 29809 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:50.391053 29809 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:50.396091 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:50.385391 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:50.374806 29887 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:50.397868 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:50.399269 29887 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:50.376601 22826 auto_rebalancer.cc:629] Failed to schedule move for tablet 616e98b5f7534394ae8790b08aec10e6: Illegal state: Replica f3c799b1249e40119d2a5864231c6c75 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 23 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:50.381146 29867 raft_consensus.cc:2749] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:50.401708 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:50.402684 23620 raft_consensus.cc:1275] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:10:50.404840 29547 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:50.406198 29176 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 21. Current term is 22.
I20260709 13:10:50.376099 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:50.340549 29685 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:50.366653 29852 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:50.374578 22951 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 21.
I20260709 13:10:50.421023 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:50.392263 29878 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:50.375211 29752 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:50.430249 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:50.421322 23590 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:50.420316 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:50.433516 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:50.438385 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:50.419360 29852 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:10:50.445485 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a424231ef7b4470bbd575564909a365e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:50.413105 29838 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:50.455335 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:50.459254 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:50.458540 29844 raft_consensus.cc:2804] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 13:10:50.469961 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:50.466459 22843 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.21.63.3:42499 timed out after 1.807s (SENT) [suppressed 2 similar messages]
I20260709 13:10:50.436316 29895 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:50.476696 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cbf52b91ec054ac6a2f8503462b1b7a6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:50.479628 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:50.467941 29844 raft_consensus.cc:493] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:50.485461 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:50.489043 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:50.506405 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:50.481184 29844 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:10:50.512090 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 139536us
I20260709 13:10:50.508744 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:50.516204 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:50.516793 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6eba8b8b84ca4a9aa5d9eaca94670686" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:50.522737 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:50.525308 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:50.523520 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:50.557426 29837 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:50.558656 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 21 candidate_status { last_received { term: 20 index: 18 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:50.595125 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 71969.3us
I20260709 13:10:50.595417 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:50.598292 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:50.599035 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:50.598675 29837 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:50.605472 22992 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:50.520632 29714 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:50.600284 22876 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 20.
I20260709 13:10:50.613413 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:50.615182 29843 raft_consensus.cc:2804] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:10:50.615666 29843 raft_consensus.cc:493] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:50.616109 29843 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:50.619578 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:50.625130 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "579096ba88474cfbbe40e8bf769879ec" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:10:50.625344 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 268923us
W20260709 13:10:50.627790 22993 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:50.606002 29837 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:50.628401 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:50.630751 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:50.633972 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:50.620915 29685 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 22 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:50.638424 23619 raft_consensus.cc:2368] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 21. Current term is 22.
I20260709 13:10:50.640108 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:50.641532 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:50.642055 23620 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:50.646898 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:50.647902 23616 raft_consensus.cc:2410] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 21 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
W20260709 13:10:50.649747 29909 log.cc:927] Time spent T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
I20260709 13:10:50.658128 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:50.656418 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:50.652577 29728 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:50.667927 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:10:50.670938 29922 log.cc:927] Time spent T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 13:10:50.694988 29728 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:50.698611 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:50.700042 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:50.701290 23619 raft_consensus.cc:2410] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for term 23 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 14.
I20260709 13:10:50.699870 23590 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:50.671451 29878 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:50.714972 29685 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:10:50.715408 29685 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:50.715763 29685 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:50.729031 23616 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:50.733039 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:50.736860 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:50.737079 29809 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:50.739271 29809 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:50.745105 23026 raft_consensus.cc:1240] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 18. Current term is 19. Ops: []
I20260709 13:10:50.747706 23618 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:50.754496 23024 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 22.
I20260709 13:10:50.755394 23620 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 19.
I20260709 13:10:50.761830 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:50.767290 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:50.768460 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:50.769706 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:50.770954 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:50.771920 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:50.778241 29728 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:50.780402 23588 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:50.781388 29768 raft_consensus.cc:2749] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:50.764045 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:50.782258 23588 leader_election.cc:400] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 18. Current term is 19.
I20260709 13:10:50.783010 23588 leader_election.cc:403] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
I20260709 13:10:50.783669 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:50.784761 23029 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 21.
I20260709 13:10:50.720336 29843 raft_consensus.cc:515] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:50.792804 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:50.794293 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:50.796713 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:50.794749 29894 raft_consensus.cc:2749] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 18. Current term is 19.
I20260709 13:10:50.797959 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:50.799053 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:50.799211 29843 leader_election.cc:290] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:50.800279 23623 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:50.796633 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:50.802446 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:50.805423 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:50.806561 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:50.806000 23623 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 18.
I20260709 13:10:50.810302 23625 raft_consensus.cc:2368] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 21. Current term is 22.
I20260709 13:10:50.817647 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:50.820817 23027 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:10:50.824923 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 358466us
I20260709 13:10:50.804519 29895 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:50.804890 29887 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 23 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:50.835516 23588 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:50.839267 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:50.852579 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:50.863996 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:50.864974 23024 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 18.
I20260709 13:10:50.805138 29547 consensus_queue.cc:1048] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 13:10:50.868222 22843 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:50.869876 22843 leader_election.cc:304] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:50.871634 29877 raft_consensus.cc:2749] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:10:50.872407 29877 raft_consensus.cc:2749] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:50.876492 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:50.878113 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:50.878930 23023 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 18.
I20260709 13:10:50.879208 29752 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:50.880115 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aa6540758938493d98d7852fdbe3ffab" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 23 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:50.883706 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:50.884843 23588 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:50.848182 29612 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:50.891031 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:50.849277 29837 raft_consensus.cc:2749] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:10:50.898101 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:50.903793 23028 raft_consensus.cc:2393] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 19: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:50.906414 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:50.908208 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ced80ef275e04ef8bf4c071883b8284e" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:50.909615 22946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "67e7fdd613434ffaacb8a030d4693490" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:50.860006 29887 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:50.901754 22920 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:50.922173 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1120807254f5444997e16f6a7feece4d" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:50.922485 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:50.924126 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:50.901198 29895 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:50.886451 29685 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:50.933074 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:50.933000 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:50.919060 29612 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:50.892599 29630 consensus_queue.cc:1059] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 13:10:50.937698 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:50.945611 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "91c88873d54149c58a20f4cf6194e5fc" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:50.956842 23029 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 18.
W20260709 13:10:50.936290 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 48566.3us
I20260709 13:10:50.945737 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:50.950999 22954 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 21.
I20260709 13:10:50.952275 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:50.952400 22993 leader_election.cc:400] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 21. Current term is 22.
I20260709 13:10:50.953277 23027 raft_consensus.cc:2468] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 22.
I20260709 13:10:50.945053 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80ab3f0c16e747e3b1917985c734eed4" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:50.948781 23026 raft_consensus.cc:1275] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:10:50.981801 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:50.982295 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:50.936407 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 248835us
I20260709 13:10:50.936581 29889 raft_consensus.cc:2804] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:50.946235 29612 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:50.951061 29837 raft_consensus.cc:2749] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:51.021435 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:51.028191 23020 raft_consensus.cc:1240] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 18. Current term is 19. Ops: []
I20260709 13:10:51.024616 23023 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 20.
W20260709 13:10:51.028760 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 144295us
I20260709 13:10:50.936820 29895 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:51.030049 22920 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:51.012120 29889 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
W20260709 13:10:51.029264 29852 consensus_meta.cc:287] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75: Time spent flushing consensus metadata: real 0.569s	user 0.010s	sys 0.000s
W20260709 13:10:51.048431 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 178637us
I20260709 13:10:51.068755 22843 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:51.029688 29878 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:51.067814 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:51.075700 22880 raft_consensus.cc:2393] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 19: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:51.048679 29889 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:51.071184 29773 raft_consensus.cc:2804] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 13:10:51.098124 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 101747us
W20260709 13:10:51.107036 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 57858.1us
I20260709 13:10:51.108286 22843 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:51.080610 29612 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 13:10:51.108814 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 109285us
I20260709 13:10:51.068346 29852 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:51.119310 29852 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 25 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:51.114464 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:51.091274 29773 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:51.120792 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:51.121925 22993 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:51.105829 22992 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:10:51.114018 29844 consensus_meta.cc:287] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2: Time spent flushing consensus metadata: real 0.595s	user 0.009s	sys 0.002s
I20260709 13:10:51.077746 29878 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:51.132895 22993 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:51.113673 29612 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:51.172899 29612 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:10:51.140241 29844 raft_consensus.cc:515] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:51.179862 29844 leader_election.cc:290] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:51.180069 29685 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 21 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:51.182121 29895 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:51.125092 29877 raft_consensus.cc:2749] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:51.128769 29773 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:10:51.189472 29926 log.cc:927] Time spent T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
W20260709 13:10:51.140931 29923 log.cc:927] Time spent T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.233s	user 0.000s	sys 0.001s
I20260709 13:10:51.190105 23622 raft_consensus.cc:1240] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 17. Current term is 18. Ops: []
I20260709 13:10:51.192914 29887 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:51.193488 29928 log.cc:927] Time spent T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
I20260709 13:10:51.189487 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:51.195560 29927 log.cc:927] Time spent T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 13:10:51.195410 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "104e2273f37843cbb5245981c817dfe0" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:51.130638 29176 raft_consensus.cc:2804] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:10:51.197537 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:51.198714 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 66931.2us
I20260709 13:10:51.198911 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:51.140450 29852 raft_consensus.cc:2764] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader election decision vote started in defunct term 21: won
I20260709 13:10:51.199971 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e3b39c18a21f448783cdd998acfb2f80" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 15 candidate_status { last_received { term: 14 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:51.200981 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:51.201175 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7633b7b4c334b039db57d77774df345" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:51.201817 22994 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:51.202095 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:51.142859 22843 leader_election.cc:304] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:51.204689 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdebaf291e5d4ae9814a80ce81ee19fe" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:51.226210 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
W20260709 13:10:51.234526 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 198944us
I20260709 13:10:51.238924 22920 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:51.246045 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:51.247064 22873 raft_consensus.cc:3060] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:51.227972 29877 raft_consensus.cc:2749] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:10:51.324369 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:51.328488 29176 raft_consensus.cc:697] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 19 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:51.341800 29176 consensus_queue.cc:237] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:51.344796 22873 raft_consensus.cc:2468] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 19.
W20260709 13:10:51.345391 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 212550us
I20260709 13:10:51.346393 22993 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:51.347608 22993 leader_election.cc:400] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 election: Vote denied by peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043) with higher term. Message: Invalid argument: T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 21. Current term is 22.
I20260709 13:10:51.330502 29838 raft_consensus.cc:2749] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:10:51.361060 29851 raft_consensus.cc:2749] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:10:51.345023 29714 consensus_queue.cc:1048] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.025s
W20260709 13:10:51.381995 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 378373us
W20260709 13:10:51.387956 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 274566us
W20260709 13:10:51.392165 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 187860us
W20260709 13:10:51.393569 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 154750us
I20260709 13:10:51.395160 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:10:51.403384 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 333022us
I20260709 13:10:51.369509 29942 raft_consensus.cc:2804] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:51.408967 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:51.410606 22920 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:51.395474 22994 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:51.423342 29700 raft_consensus.cc:3055] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 13:10:51.436484 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 293679us
W20260709 13:10:51.406124 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 46749.2us
W20260709 13:10:51.441686 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 46335.5us
I20260709 13:10:51.442549 22843 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:51.446324 22843 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:51.483639 22994 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:10:51.493778 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 47544.5us
I20260709 13:10:51.494673 22843 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:10:51.548269 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 50101.4us
I20260709 13:10:51.448637 29942 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:51.442229 29700 raft_consensus.cc:740] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:51.591462 29877 raft_consensus.cc:493] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:51.600870 29877 raft_consensus.cc:515] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:51.602129 29941 consensus_queue.cc:1059] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.188s
I20260709 13:10:51.602921 29877 leader_election.cc:290] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:51.603407 29940 raft_consensus.cc:2749] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:51.498593 29547 raft_consensus.cc:2749] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:10:51.613972 29700 consensus_queue.cc:260] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:51.614722 29937 log.cc:927] Time spent T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.210s	user 0.000s	sys 0.000s
I20260709 13:10:51.615506 29949 raft_consensus.cc:2749] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:10:51.617069 29944 log.cc:927] Time spent T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.224s	user 0.000s	sys 0.001s
I20260709 13:10:51.617244 23588 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:51.620257 29941 raft_consensus.cc:3055] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:10:51.483742 29957 raft_consensus.cc:2749] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:51.624636 29941 raft_consensus.cc:740] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:51.625774 29941 consensus_queue.cc:260] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:51.627089 29941 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:10:51.623839 29889 consensus_meta.cc:287] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.524s	user 0.007s	sys 0.000s
I20260709 13:10:51.593205 29942 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:51.629251 29889 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:51.631985 22949 raft_consensus.cc:1240] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 18. Current term is 19. Ops: []
I20260709 13:10:51.633450 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:51.634393 23621 raft_consensus.cc:2393] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 19: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:51.640961 29936 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:51.624636 23588 leader_election.cc:304] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:10:51.643182 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 255764us
I20260709 13:10:51.649394 22994 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:10:51.646914 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 236375us
I20260709 13:10:51.653496 22920 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:51.654331 22994 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:51.659998 22994 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:51.582226 29895 raft_consensus.cc:2749] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:51.581758 29630 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:10:51.670990 22994 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:51.684808 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:51.641803 29936 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:51.690568 22994 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:10:51.691880 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 284187us
I20260709 13:10:51.698028 29960 raft_consensus.cc:2749] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:51.670215 29940 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:51.702095 29527 raft_consensus.cc:2804] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:10:51.702724 29527 raft_consensus.cc:697] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 22 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
W20260709 13:10:51.703259 29773 consensus_meta.cc:287] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.519s	user 0.012s	sys 0.000s
I20260709 13:10:51.703668 29527 consensus_queue.cc:237] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:51.705027 29961 raft_consensus.cc:2804] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:51.705241 29889 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:51.705530 29752 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:51.707034 29961 raft_consensus.cc:493] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:51.707491 29961 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:51.642652 29612 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:51.710785 29612 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 23 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:51.692483 29838 raft_consensus.cc:2749] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:51.717831 22918 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:51.718477 29878 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:51.719688 22918 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:51.686458 29630 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:51.721216 29630 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:51.722668 23625 raft_consensus.cc:1240] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 20. Current term is 21. Ops: []
I20260709 13:10:51.724344 29873 consensus_queue.cc:1048] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 13:10:51.726368 22994 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:51.636531 29952 raft_consensus.cc:2804] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:51.733629 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:51.735148 29953 raft_consensus.cc:2749] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:51.736083 29953 raft_consensus.cc:2749] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:51.726774 29957 raft_consensus.cc:2749] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:51.686151 29852 raft_consensus.cc:2749] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:10:51.750041 22994 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:51.751567 22994 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:51.758416 29940 raft_consensus.cc:2749] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:51.758772 29700 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:51.766002 29957 raft_consensus.cc:2749] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:51.703727 29773 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:51.772275 22950 raft_consensus.cc:1240] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 17. Current term is 18. Ops: []
I20260709 13:10:51.773308 29773 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:51.777550 29942 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:51.779820 29942 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:51.788883 22994 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:51.749112 29852 raft_consensus.cc:2749] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:51.792483 29936 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:51.792827 29806 raft_consensus.cc:2749] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:51.746622 29952 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:51.793524 29952 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:51.794981 29973 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:10:51.795218 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 141956us
W20260709 13:10:51.795958 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 44821.5us
I20260709 13:10:51.796593 22918 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:51.802498 22918 leader_election.cc:304] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:51.819258 29963 raft_consensus.cc:2749] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:10:51.820165 29964 raft_consensus.cc:2749] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:51.821348 22918 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:51.822372 29963 raft_consensus.cc:2749] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:10:51.827212 29959 raft_consensus.cc:493] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:51.828152 22918 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:51.831112 29948 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:10:51.832077 29948 raft_consensus.cc:697] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:51.833362 29948 consensus_queue.cc:237] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:51.839159 22918 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:51.748281 29961 raft_consensus.cc:515] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:51.805840 29852 raft_consensus.cc:2749] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:10:51.849203 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 60416.4us
I20260709 13:10:51.828068 29959 raft_consensus.cc:515] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:51.859333 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f35ebcf93ad423ea8bd0b777bee23ee" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:51.861786 23026 raft_consensus.cc:1240] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 25 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 24. Current term is 25. Ops: []
I20260709 13:10:51.865221 22920 leader_election.cc:304] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:51.851051 22918 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:51.874102 22920 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:51.853420 29943 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:10:51.869858 29889 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:51.889873 22994 leader_election.cc:304] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:51.897737 23622 raft_consensus.cc:1240] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 18. Current term is 19. Ops: []
I20260709 13:10:51.859880 29873 consensus_queue.cc:1059] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
W20260709 13:10:51.914232 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 43780.5us
I20260709 13:10:51.888247 29953 raft_consensus.cc:2749] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:10:51.930495 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 203815us
I20260709 13:10:51.931566 23590 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:51.935765 29936 consensus_queue.cc:1059] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:10:51.937309 29878 raft_consensus.cc:3055] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:10:51.937784 29878 raft_consensus.cc:740] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:51.938895 29878 consensus_queue.cc:260] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:51.940410 29878 raft_consensus.cc:3060] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:51.944197 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:51.945043 23021 raft_consensus.cc:2376] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 21. Re-sending same reply.
I20260709 13:10:51.947224 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:51.912004 29773 raft_consensus.cc:2749] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:51.960362 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:51.970933 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:51.977017 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:51.921114 29962 raft_consensus.cc:2749] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:51.929376 29685 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:52.006167 29685 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:52.007169 29852 consensus_queue.cc:1059] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:10:52.008635 29851 raft_consensus.cc:3055] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:10:52.009249 29851 raft_consensus.cc:740] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:52.010592 29851 consensus_queue.cc:260] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:51.944303 29943 raft_consensus.cc:697] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:52.015220 29943 consensus_queue.cc:237] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:52.016849 29833 raft_consensus.cc:2749] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:10:52.017508 29728 consensus_queue.cc:1059] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:10:52.018738 29728 raft_consensus.cc:3055] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:10:52.019132 29728 raft_consensus.cc:740] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:52.019737 29557 consensus_queue.cc:1059] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:10:52.020090 29728 consensus_queue.cc:260] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:52.026125 22878 raft_consensus.cc:2410] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 13:10:52.026379 29728 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:52.036402 23022 raft_consensus.cc:2468] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 18.
I20260709 13:10:52.016598 29851 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Advancing to term 18
W20260709 13:10:51.992496 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 61027.3us
I20260709 13:10:52.044533 29630 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:52.040350 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 166477us
I20260709 13:10:52.039564 29557 raft_consensus.cc:3055] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 13:10:52.052132 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:52.053303 23026 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 18.
I20260709 13:10:52.055524 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5cbb1e724f1d4e3f97c0d29d6b2ce944" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 18 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:52.060384 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fab4388fa7ea49bda9fa070cf542577f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:52.061240 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:52.056800 23588 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
W20260709 13:10:52.066458 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 431421us
W20260709 13:10:52.043378 29176 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.022s	sys 0.043s
I20260709 13:10:52.071516 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:52.038769 29685 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:52.040207 23025 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:10:52.073609 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:52.075739 23025 raft_consensus.cc:606] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 17 LEADER]: Received request to transfer leadership to TS aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:52.085686 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:52.086722 23021 raft_consensus.cc:2393] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 19: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:52.056207 29557 raft_consensus.cc:740] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
W20260709 13:10:52.073233 29176 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.702s	user 0.023s	sys 0.043s
I20260709 13:10:52.072110 29837 raft_consensus.cc:2804] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:10:52.070511 29959 leader_election.cc:290] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:52.093633 22843 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:52.096163 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 49424.3us
I20260709 13:10:52.090065 29952 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:52.102766 22920 leader_election.cc:304] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:52.105675 22952 raft_consensus.cc:1240] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 18. Current term is 19. Ops: []
I20260709 13:10:52.101861 29974 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:52.105643 29557 consensus_queue.cc:260] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:52.106559 29974 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:52.107182 29557 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:10:52.108850 29952 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:52.109997 29952 raft_consensus.cc:2749] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:52.105178 22920 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:52.112010 29952 raft_consensus.cc:2749] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:10:52.100387 29837 raft_consensus.cc:697] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 18 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:52.127105 29837 consensus_queue.cc:237] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:52.123212 22843 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:52.130061 29974 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:52.130607 29612 raft_consensus.cc:2749] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:10:52.130674 22843 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:52.133847 22843 leader_election.cc:304] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:52.135442 22843 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:52.137490 29895 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:52.137524 29877 raft_consensus.cc:2749] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:52.140883 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:52.144963 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
W20260709 13:10:52.149206 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 53454.4us
I20260709 13:10:52.150332 22843 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:52.104199 29878 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.116185 29889 raft_consensus.cc:2749] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:52.154171 22843 leader_election.cc:304] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:52.103799 29936 raft_consensus.cc:2749] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:52.163720 22843 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:52.164894 22843 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:52.166205 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:52.167095 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6bdc53dc56134e28bcc43e68e76055fc" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:52.178210 22843 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:52.147941 29630 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:52.179690 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "13e53d9e00624dfcb87aca9b055fdc9a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:52.157863 29877 raft_consensus.cc:2749] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:52.186390 22877 raft_consensus.cc:1240] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 23 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 22. Current term is 23. Ops: []
I20260709 13:10:52.156908 29895 raft_consensus.cc:2749] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:52.190594 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:52.193126 22843 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:52.196146 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:52.179252 29878 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:52.194248 29936 raft_consensus.cc:2749] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:52.162783 29685 raft_consensus.cc:493] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:52.205297 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:52.205587 23590 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:52.187628 29612 raft_consensus.cc:2749] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:52.212770 22873 raft_consensus.cc:2468] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 19.
I20260709 13:10:52.214494 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:52.214742 22943 tablet_service.cc:1558] Processing DeleteTablet for tablet 2959101199a44a4583a4ee456ecf044d with delete_type TABLET_DATA_TOMBSTONED (TS aed3f089ec8b49af9c8cf5cdee76e8e7 not found in new config with opid_index 11) from {username='slave'} at 127.0.0.1:53202
I20260709 13:10:52.189790 29949 raft_consensus.cc:2749] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:52.216226 22947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "616e98b5f7534394ae8790b08aec10e6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:52.200489 29878 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:52.194335 29547 raft_consensus.cc:2749] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:52.218641 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:52.189728 29956 raft_consensus.cc:2749] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:52.201767 29894 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:52.221123 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:52.225445 22920 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:52.226821 22955 raft_consensus.cc:1240] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 18. Current term is 19. Ops: []
I20260709 13:10:52.225667 23590 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:52.229375 22920 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:52.230819 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:52.228143 29833 consensus_queue.cc:1059] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.005s
I20260709 13:10:52.234035 29993 raft_consensus.cc:2749] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 13:10:52.206460 29685 raft_consensus.cc:515] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:52.240922 29685 leader_election.cc:290] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:52.232020 29894 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:52.242740 29833 raft_consensus.cc:3055] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:10:52.243120 29833 raft_consensus.cc:740] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:52.243288 29894 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:52.237732 29945 raft_consensus.cc:2804] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:10:52.246055 29945 raft_consensus.cc:697] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 19 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:52.233717 29630 raft_consensus.cc:2749] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 13:10:52.253279 22844 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:52.255599 29964 raft_consensus.cc:3055] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:10:52.256027 29964 raft_consensus.cc:740] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:52.256940 29964 consensus_queue.cc:260] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:52.262271 23622 raft_consensus.cc:2468] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 25.
I20260709 13:10:52.244068 29833 consensus_queue.cc:260] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:52.264376 29833 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:10:52.265535 29961 leader_election.cc:290] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:52.271292 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 169918us
I20260709 13:10:52.246963 29945 consensus_queue.cc:237] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:52.258287 29964 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:52.276757 29998 tablet_replica.cc:333] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:10:52.281013 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:52.280359 22872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:52.290926 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:52.291611 29959 consensus_queue.cc:1059] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:10:52.293118 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd740f20d2d74669b0fd10f1ec5b44f0" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:52.296865 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 119402us
I20260709 13:10:52.311573 22994 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:52.313982 22994 leader_election.cc:304] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:52.314693 29883 raft_consensus.cc:2749] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:10:52.315384 29960 raft_consensus.cc:2749] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:10:52.321712 22876 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a424231ef7b4470bbd575564909a365e"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
mode: GRACEFUL
new_leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:40244
I20260709 13:10:52.322343 22876 raft_consensus.cc:606] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Received request to transfer leadership to TS aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:52.322664 22876 raft_consensus.cc:614] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:10:52.325167 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:52.326359 22875 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
mode: GRACEFUL
new_leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:40244
I20260709 13:10:52.326907 22875 raft_consensus.cc:606] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Received request to transfer leadership to TS aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:52.327261 22875 raft_consensus.cc:614] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:10:52.327796 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:52.328759 21756 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:10:52.330770 23024 raft_consensus.cc:1275] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:10:52.336035 22918 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:52.338043 29958 raft_consensus.cc:2749] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:10:52.339838 22918 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:52.342696 29963 raft_consensus.cc:2749] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:10:52.344027 29963 consensus_queue.cc:1048] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:10:52.344429 23588 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:52.345594 29892 raft_consensus.cc:2749] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 13:10:52.353524 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 160435us
I20260709 13:10:52.376093 22843 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:52.377393 22949 raft_consensus.cc:1240] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 21. Current term is 22. Ops: []
I20260709 13:10:52.392093 22843 leader_election.cc:304] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:52.392995 29612 raft_consensus.cc:2749] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:10:52.393432 22843 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:52.394181 29612 raft_consensus.cc:2749] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:52.395556 22843 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:52.396420 22843 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:52.398057 22843 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:52.400092 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 49326us
I20260709 13:10:52.388653 29956 raft_consensus.cc:2749] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:52.400823 22993 leader_election.cc:304] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:52.394984 29974 raft_consensus.cc:2749] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:52.405920 22843 leader_election.cc:304] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:52.406323 22993 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:52.407599 29547 raft_consensus.cc:2749] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:52.407949 29752 raft_consensus.cc:2749] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:52.402778 30000 raft_consensus.cc:2749] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:52.397157 29949 raft_consensus.cc:2749] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:10:52.412098 29612 raft_consensus.cc:2749] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:10:52.413759 22843 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:52.414398 22871 raft_consensus.cc:1275] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:10:52.440646 29877 raft_consensus.cc:2804] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:52.448112 29920 raft_consensus.cc:2804] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:10:52.448731 29920 raft_consensus.cc:697] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 25 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:52.448570 29877 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:52.449524 29877 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:52.449714 29920 consensus_queue.cc:237] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:52.454429 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:52.470695 22843 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:52.476881 29949 raft_consensus.cc:2749] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:52.477150 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:52.498064 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 66481.3us
W20260709 13:10:52.503641 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 94364.3us
I20260709 13:10:52.505604 23621 raft_consensus.cc:1275] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:10:52.499682 29728 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:52.431764 29963 consensus_queue.cc:1059] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.073s
I20260709 13:10:52.507283 22843 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:52.515532 29728 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:52.516158 29998 raft_consensus.cc:2243] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:10:52.517176 29963 raft_consensus.cc:3055] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:10:52.517617 29963 raft_consensus.cc:740] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:52.517900 29728 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:52.511436 29942 consensus_queue.cc:1059] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.007s
I20260709 13:10:52.512141 23588 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:52.522781 29843 raft_consensus.cc:3055] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:10:52.523357 29843 raft_consensus.cc:740] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:52.519881 29612 raft_consensus.cc:2749] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:52.523646 29942 raft_consensus.cc:3055] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:10:52.524091 29942 raft_consensus.cc:740] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:52.524537 29998 raft_consensus.cc:2272] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:10:52.526583 29942 consensus_queue.cc:260] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:52.528029 29998 ts_tablet_manager.cc:1916] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:10:52.528204 29942 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:52.529513 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:52.530130 22871 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 20.
I20260709 13:10:52.518565 29963 consensus_queue.cc:260] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:52.539433 29963 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:52.542531 22843 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:52.544008 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:52.547616 23588 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:52.548979 29892 raft_consensus.cc:2749] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:52.524245 29843 consensus_queue.cc:260] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:52.556777 29843 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:52.557515 23024 raft_consensus.cc:1275] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:10:52.558084 29877 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:52.563593 29980 consensus_queue.cc:1048] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:10:52.569895 21756 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:10:52.583622 29974 raft_consensus.cc:2749] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:10:52.586508 22948 raft_consensus.cc:3060] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:52.594015 22948 raft_consensus.cc:1275] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:10:52.595230 22875 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:52.598096 29973 consensus_queue.cc:1048] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:10:52.604348 22994 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:52.609496 29949 raft_consensus.cc:2804] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:10:52.609921 29949 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:52.610283 29949 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:10:52.608443 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 65974.1us
I20260709 13:10:52.616360 29612 consensus_queue.cc:1059] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:10:52.617770 29612 raft_consensus.cc:3055] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:10:52.618209 29612 raft_consensus.cc:740] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:52.619267 29612 consensus_queue.cc:260] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:52.621284 29612 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:10:52.626173 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 77296.9us
I20260709 13:10:52.614347 22843 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:52.630681 22843 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:52.635417 22875 raft_consensus.cc:1275] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 13:10:52.639001 22843 leader_election.cc:304] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:52.627411 29942 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:10:52.641239 29547 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:52.660634 23023 raft_consensus.cc:1275] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:10:52.652395 29942 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:52.666745 29851 raft_consensus.cc:993] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75: : Instructing follower aed3f089ec8b49af9c8cf5cdee76e8e7 to start an election
I20260709 13:10:52.666945 29942 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:52.667402 29851 raft_consensus.cc:1081] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 17 LEADER]: Signalling peer aed3f089ec8b49af9c8cf5cdee76e8e7 to start an election
I20260709 13:10:52.652284 29878 raft_consensus.cc:2749] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:10:52.645962 29952 consensus_queue.cc:1048] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 13:10:52.675449 29981 consensus_queue.cc:1048] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:10:52.645591 29956 raft_consensus.cc:2749] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:52.624020 29892 consensus_queue.cc:1048] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.028s
I20260709 13:10:52.682510 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:52.684656 22764 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 7426119f4da44ffaa1513b481a866a01 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER failed with leader 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 0a407504ed1741bf89a276558adf4c0f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 22 committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:52.686084 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:52.686509 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:52.691250 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85d33f0fd80a49b8bd8cb73571ed3191" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:52.693710 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:52.664866 29547 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:52.694998 29547 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:52.696049 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:52.697361 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:52.710973 22876 raft_consensus.cc:3060] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:52.714545 22873 raft_consensus.cc:1240] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 18. Current term is 19. Ops: []
I20260709 13:10:52.716234 23023 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f3b08335f3564868a8160dc1b5ade501" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:52.719369 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:52.723229 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:52.726586 22843 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:52.727257 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:52.688032 29877 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:52.728055 23025 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 19.
I20260709 13:10:52.722146 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:52.733148 22843 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:52.731958 22951 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 19.
I20260709 13:10:52.677291 29949 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:52.752460 29942 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:52.760555 29942 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:52.761207 23022 raft_consensus.cc:1275] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 13:10:52.728077 29981 consensus_queue.cc:1059] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 13:10:52.777498 29981 raft_consensus.cc:3055] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:10:52.777920 29981 raft_consensus.cc:740] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:52.742676 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:52.778997 29981 consensus_queue.cc:260] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:52.781111 30001 log.cc:927] Time spent T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 13:10:52.785279 29961 consensus_queue.cc:1048] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:10:52.787149 29949 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:52.791076 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:52.793401 29892 consensus_queue.cc:1048] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:10:52.731395 29612 raft_consensus.cc:2749] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:10:52.794853 23020 raft_consensus.cc:2368] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 21. Current term is 22.
I20260709 13:10:52.800508 23026 raft_consensus.cc:1240] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 13:10:52.810343 22875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:52.816984 22876 raft_consensus.cc:1275] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 13:10:52.823508 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 90450.8us
I20260709 13:10:52.824805 22843 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:52.828765 22843 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:52.830346 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:52.830505 29980 consensus_queue.cc:1059] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.005s
I20260709 13:10:52.831686 29980 raft_consensus.cc:3055] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:10:52.832103 29980 raft_consensus.cc:740] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:52.833146 29980 consensus_queue.cc:260] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:52.834683 29980 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:52.839799 22994 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:52.844977 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:52.846920 22871 raft_consensus.cc:1275] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:10:52.848611 29892 consensus_queue.cc:1048] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:10:52.819802 22951 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:58320
I20260709 13:10:52.850193 22951 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:52.850737 22951 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:52.853786 29920 raft_consensus.cc:2749] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:10:52.822105 29981 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:52.861459 22843 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:52.838075 29877 raft_consensus.cc:2749] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:10:52.838271 29612 raft_consensus.cc:2749] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 13:10:52.887930 22843 leader_election.cc:304] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:52.841279 29949 raft_consensus.cc:2804] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:10:52.889312 22951 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:52.898957 23026 raft_consensus.cc:1275] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:10:52.826949 29833 consensus_queue.cc:1048] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.051s
I20260709 13:10:52.902428 22843 leader_election.cc:304] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:52.908305 22951 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:52.909516 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:52.910282 23026 raft_consensus.cc:3055] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 13:10:52.895898 29878 raft_consensus.cc:2749] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:10:52.891744 29949 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:52.898908 30000 raft_consensus.cc:2749] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:10:52.910703 23026 raft_consensus.cc:740] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
W20260709 13:10:52.950541 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 48210.6us
I20260709 13:10:52.951624 22843 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:52.957942 22949 raft_consensus.cc:1240] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 17. Current term is 18. Ops: []
I20260709 13:10:52.941078 29949 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:52.940857 29956 raft_consensus.cc:2749] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:10:52.963336 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: true dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:52.975229 23620 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:52.977936 22843 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:52.979228 22843 leader_election.cc:400] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 21. Current term is 22.
I20260709 13:10:52.979642 22843 leader_election.cc:403] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:10:52.981016 29934 consensus_queue.cc:1048] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.016s
I20260709 13:10:52.992158 29612 raft_consensus.cc:2749] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:10:52.990038 29921 consensus_queue.cc:1059] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 13:10:52.993671 29878 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:53.039427 29942 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:53.040009 29942 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:53.050923 23026 consensus_queue.cc:260] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:53.052191 23026 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Advancing to term 18
I20260709 13:10:53.057360 23624 raft_consensus.cc:1275] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 25 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 13:10:53.063024 23026 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18.
I20260709 13:10:52.975129 29547 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:53.069262 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:53.072878 22873 raft_consensus.cc:1240] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 18. Current term is 19. Ops: []
I20260709 13:10:53.073099 22955 raft_consensus.cc:3060] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:10:53.074822 29921 consensus_queue.cc:1048] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 13:10:53.078365 29547 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:53.083578 22955 raft_consensus.cc:1275] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 25 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
W20260709 13:10:53.086514 22919 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 13:10:52.985970 22843 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:53.099772 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 113912us
I20260709 13:10:53.094094 22949 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 19.
I20260709 13:10:52.980046 30000 raft_consensus.cc:2749] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:10:53.107563 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:53.112010 23620 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18.
I20260709 13:10:53.126425 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:53.132203 22919 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:53.144361 29963 consensus_queue.cc:1059] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:10:53.146871 29992 raft_consensus.cc:3055] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:10:53.147344 29992 raft_consensus.cc:740] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:53.157742 22992 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:52.979139 29998 ts_tablet_manager.cc:1929] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 12.10
I20260709 13:10:53.118979 29547 raft_consensus.cc:2749] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 13:10:53.160780 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:53.163267 22843 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
W20260709 13:10:53.207958 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 44773us
W20260709 13:10:53.221905 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 64236.2us
I20260709 13:10:53.150830 29942 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:53.212689 29952 raft_consensus.cc:2804] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 13:10:53.214970 30000 raft_consensus.cc:2749] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:10:53.207141 29843 consensus_queue.cc:1048] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.069s
I20260709 13:10:53.148366 29992 consensus_queue.cc:260] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:53.230191 29714 raft_consensus.cc:2804] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:10:53.242127 29998 log.cc:1199] T 2959101199a44a4583a4ee456ecf044d P aed3f089ec8b49af9c8cf5cdee76e8e7: Deleting WAL directory at /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-1-root/wals/2959101199a44a4583a4ee456ecf044d
I20260709 13:10:53.222740 22826 placement_policy_util.cc:248] tablet ac5cc94926a742e6a0c1340ec33c0ff0: not considering replicas for movement since the tablet's status is 'UNAVAILABLE'
I20260709 13:10:53.283908 29714 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:10:53.343777 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 210142us
I20260709 13:10:53.343132 29714 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:10:53.412621 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 189495us
I20260709 13:10:53.339250 29949 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:53.398180 29992 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:10:53.441836 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 209670us
W20260709 13:10:53.409880 30009 log.cc:927] Time spent T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
W20260709 13:10:53.459818 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 237052us
W20260709 13:10:53.468917 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 133977us
W20260709 13:10:53.495625 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 47304.7us
I20260709 13:10:53.531596 29176 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:53.532195 29176 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:53.496424 29952 raft_consensus.cc:697] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
W20260709 13:10:53.546327 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 47472.8us
I20260709 13:10:53.551043 29176 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:53.562974 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:53.572259 22952 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Advancing to term 20
W20260709 13:10:53.563820 30013 log.cc:927] Time spent T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.159s	user 0.000s	sys 0.000s
W20260709 13:10:53.567055 22772 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33478 (request call id 279) took 47709 ms (client timeout 15000 ms). Trace:
W20260709 13:10:53.569242 30018 log.cc:927] Time spent T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.069s	user 0.001s	sys 0.000s
W20260709 13:10:53.570523 30014 log.cc:927] Time spent T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 13:10:53.571262 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:53.557722 29974 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:53.575863 29998 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53202 (request call id 0) took 1405 ms. Trace:
I20260709 13:10:53.571508 29952 consensus_queue.cc:237] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:53.587812 29949 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:10:53.577808 22772 rpcz_store.cc:269] 0709 13:10:05.872410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:05.883748 (+ 11338us) service_pool.cc:224] Handling call
0709 13:10:53.566925 (+47683177us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:10:50.475433 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:10:50.598381 (+122948us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:10:50.598430 (+    49us) write_op.cc:435] Acquired schema lock
0709 13:10:50.598462 (+    32us) tablet.cc:662] Decoding operations
0709 13:10:50.599267 (+   805us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:10:50.599392 (+   125us) write_op.cc:641] Partition lock acquired for write op
0709 13:10:50.602427 (+  3035us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:10:50.657486 (+ 55059us) tablet.cc:701] Row locks acquired
0709 13:10:50.671636 (+ 14150us) write_op.cc:260] PREPARE: finished
0709 13:10:50.706278 (+ 34642us) write_op.cc:270] Start()
0709 13:10:50.740089 (+ 33811us) write_op.cc:276] Timestamp: P: 1783602650699300 usec, L: 0
0709 13:10:50.740130 (+    41us) op_driver.cc:348] REPLICATION: starting
0709 13:10:50.746882 (+  6752us) log.cc:844] Serialized 2413 byte log entry
0709 13:10:51.491724 (+744842us) op_driver.cc:464] REPLICATION: finished
0709 13:10:51.961174 (+469450us) write_op.cc:301] APPLY: starting
0709 13:10:51.961288 (+   114us) tablet.cc:1367] starting BulkCheckPresence
0709 13:10:51.968944 (+  7656us) tablet.cc:1370] finished BulkCheckPresence
0709 13:10:51.969076 (+   132us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:10:52.095845 (+126769us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:10:52.110611 (+ 14766us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=12,key_file_lookups=12,delta_file_lookups=12,mrs_lookups=0
0709 13:10:52.110677 (+    66us) write_op.cc:312] APPLY: finished
0709 13:10:52.111044 (+   367us) log.cc:844] Serialized 65 byte log entry
0709 13:10:52.111228 (+   184us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:52.111340 (+   112us) write_op.cc:454] Released schema lock
0709 13:10:52.111414 (+    74us) write_op.cc:341] FINISH: Updating metrics
0709 13:10:52.111874 (+   460us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:52.111917 (+    43us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":364614,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":378325,"apply.run_cpu_time_us":13264,"apply.run_wall_time_us":150695,"cfile_cache_hit":26,"cfile_cache_hit_bytes":29219,"num_ops":6,"prepare.queue_time_us":250585,"prepare.run_cpu_time_us":30752,"prepare.run_wall_time_us":534011,"raft.queue_time_us":296894,"raft.run_cpu_time_us":67565,"raft.run_wall_time_us":495283,"replication_time_us":694134,"thread_start_us":872979,"threads_started":4,"wal-append.queue_time_us":301390,"wal-append.run_cpu_time_us":78416,"wal-append.run_wall_time_us":2073685}]]}
W20260709 13:10:53.598484 29980 consensus_meta.cc:287] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.763s	user 0.012s	sys 0.000s
W20260709 13:10:53.600549 30011 log.cc:927] Time spent T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.346s	user 0.000s	sys 0.004s
I20260709 13:10:53.584201 29974 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:53.604038 29949 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:53.605412 29949 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:53.605906 29974 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:53.609208 29949 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:53.584578 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 84614.9us
I20260709 13:10:53.580504 29998 rpcz_store.cc:276] 0709 13:10:52.203195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:52.214213 (+ 11018us) service_pool.cc:224] Handling call
0709 13:10:52.276335 (+ 62122us) ts_tablet_manager.cc:1151] Deleting tablet 2959101199a44a4583a4ee456ecf044d
0709 13:10:52.276428 (+    93us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 13:10:52.768131 (+491703us) tablet_metadata.cc:665] Metadata flushed
0709 13:10:52.960126 (+191995us) tablet_metadata.cc:665] Metadata flushed
0709 13:10:53.575736 (+615610us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":159584,"tablet-delete.queue_time_us":60702,"thread_start_us":7912,"threads_started":1}
I20260709 13:10:53.634127 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:53.635044 23025 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:53.635407 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:53.636053 22951 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 21.
I20260709 13:10:53.649034 22952 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 20.
W20260709 13:10:53.625792 29878 consensus_meta.cc:287] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f: Time spent flushing consensus metadata: real 0.591s	user 0.012s	sys 0.000s
I20260709 13:10:53.651211 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:53.653206 22876 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 18.
I20260709 13:10:53.657943 22843 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:53.658671 29949 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:10:53.658849 22843 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:53.659076 29949 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:53.659463 29949 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:53.659643 30000 raft_consensus.cc:2804] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:10:53.660291 30000 raft_consensus.cc:697] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 20 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:53.661477 30000 consensus_queue.cc:237] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:53.656975 29878 raft_consensus.cc:2749] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499) with higher term. Message: Invalid argument: T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f for earlier term 21. Current term is 22.
W20260709 13:10:53.660365 30034 log.cc:927] Time spent T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 13:10:53.771908 29700 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:53.772547 29700 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:53.774752 29700 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:53.775681 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:53.776427 22876 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 19.
I20260709 13:10:53.777881 22920 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:53.781073 29981 raft_consensus.cc:2804] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:10:53.781162 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 108326us
I20260709 13:10:53.781574 29981 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:53.788138 23025 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 20.
I20260709 13:10:53.782044 29981 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:53.767875 29867 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:53.801791 29867 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:53.803963 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:53.804641 23020 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 19.
I20260709 13:10:53.813287 30035 raft_consensus.cc:493] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:53.813908 30035 raft_consensus.cc:515] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:53.816198 30035 leader_election.cc:290] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:53.819065 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:53.811658 30025 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:53.820779 29956 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:53.821285 29956 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:53.822320 29949 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:53.824983 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:53.825726 22952 raft_consensus.cc:2393] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 22: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:53.828308 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:53.828907 23021 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 22.
I20260709 13:10:53.823378 29956 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:53.834293 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:53.834508 29809 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:53.835129 23625 raft_consensus.cc:2393] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 21: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:53.835184 29809 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:53.837664 29809 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:53.840801 30025 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:53.841552 30033 log.cc:927] Time spent T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.271s	user 0.001s	sys 0.006s
I20260709 13:10:53.841888 29973 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:53.854277 29973 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:53.856257 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:53.857088 23625 raft_consensus.cc:2393] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 21: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:53.857162 29973 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:53.857414 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:53.860234 23022 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:53.860893 23022 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18.
I20260709 13:10:53.862413 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:53.863111 22878 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 17.
I20260709 13:10:53.863194 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:53.863921 29867 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:53.864619 22920 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:53.865546 29952 raft_consensus.cc:2804] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:53.849320 29949 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:53.850301 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:53.873538 29714 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:53.866009 29952 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:53.877171 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:53.877405 22994 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:53.878041 22843 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:53.880201 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:53.639845 22762 catalog_manager.cc:5028] TS aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): tablet 2959101199a44a4583a4ee456ecf044d (table test-workload [id=a42f2ccfbe484baca3019323a490c64a]) successfully deleted
I20260709 13:10:53.906597 22878 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 17.
W20260709 13:10:53.906710 22762 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 269731us
I20260709 13:10:53.888754 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:53.896265 22880 raft_consensus.cc:1240] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 20. Current term is 21. Ops: []
I20260709 13:10:53.904141 22952 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 21.
I20260709 13:10:53.907958 23620 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0c4ecd16880043f4a75006a3da4717bf" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:53.870167 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:53.906278 22876 raft_consensus.cc:2393] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 19: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:53.907727 22877 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 23.
I20260709 13:10:53.909114 23025 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 19.
I20260709 13:10:53.879010 30025 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:53.913836 22843 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:53.914124 30038 raft_consensus.cc:2804] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:10:53.914597 30038 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:53.914960 29956 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:10:53.922613 22994 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:53.922119 22918 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:53.915472 29956 raft_consensus.cc:697] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 21 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:53.911247 29952 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:53.914942 30038 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:53.915865 29949 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:10:53.946334 30029 log.cc:927] Time spent T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.382s	user 0.003s	sys 0.006s
I20260709 13:10:53.949163 22919 leader_election.cc:304] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:53.936830 29878 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:53.952544 29878 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:53.954541 29993 raft_consensus.cc:2749] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 13:10:53.954713 29878 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:53.936554 29873 consensus_queue.cc:1059] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 20.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.019s
I20260709 13:10:53.941612 29867 raft_consensus.cc:2804] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:53.956522 29873 raft_consensus.cc:3055] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:10:53.956749 29867 raft_consensus.cc:493] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:53.957067 29873 raft_consensus.cc:740] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:53.957554 29867 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:53.958214 29873 consensus_queue.cc:260] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:53.959612 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:53.985432 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:53.986196 23026 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 19.
I20260709 13:10:53.988612 30025 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:53.989207 30025 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:53.991261 30025 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:53.943423 29949 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:53.992969 29949 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:53.997485 29873 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:53.977046 29700 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:53.998147 29700 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:53.998934 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:53.999658 22880 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 18.
I20260709 13:10:54.000213 29700 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:10:54.002286 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 50887.1us
I20260709 13:10:53.937510 29973 raft_consensus.cc:2804] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:10:53.915745 29981 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:54.018368 22949 raft_consensus.cc:2393] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 19: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:54.015940 22920 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:54.021147 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
W20260709 13:10:54.026260 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 50203.5us
I20260709 13:10:54.027168 22880 raft_consensus.cc:2393] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 23: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:54.028653 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:54.031847 22843 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:54.038758 29878 raft_consensus.cc:2804] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:54.039175 29878 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting leader election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:54.039552 29878 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Advancing to term 19
W20260709 13:10:54.022750 30029 log.cc:927] Time spent T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 13:10:54.043042 29949 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:54.044147 22952 raft_consensus.cc:1275] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:10:54.045336 30036 raft_consensus.cc:493] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:54.045658 30000 consensus_queue.cc:1048] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:10:54.045959 30036 raft_consensus.cc:515] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:54.048138 30036 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:54.023120 29973 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:54.057053 29878 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:54.057626 29973 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:54.059273 29878 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:54.060297 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:54.064023 29714 leader_election.cc:290] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:54.064905 29956 consensus_queue.cc:237] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:54.065711 22764 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 7426119f4da44ffaa1513b481a866a01 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER failed with leader 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 0a407504ed1741bf89a276558adf4c0f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 22 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:54.067469 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:54.068064 22877 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:54.075685 23619 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:54.076363 23619 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 23.
W20260709 13:10:54.077142 22826 auto_rebalancer.cc:629] Failed to schedule move for tablet 0f076a0e47a14c2b933d3058063c2bcd: Illegal state: Replica aed3f089ec8b49af9c8cf5cdee76e8e7 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 22 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
W20260709 13:10:54.081983 22826 auto_rebalancer.cc:629] Failed to schedule move for tablet ec7dd25342084943aacb4b9830f5f1ea: Illegal state: Replica f3c799b1249e40119d2a5864231c6c75 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 19 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:54.057235 29993 raft_consensus.cc:2804] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 13:10:54.100807 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 72904us
I20260709 13:10:54.100184 22877 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 20.
I20260709 13:10:54.086117 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:54.086334 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:54.086992 23027 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:54.088646 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:54.094172 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:54.102890 23025 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:54.102684 22949 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 19.
I20260709 13:10:54.095927 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:54.104970 22950 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:54.083909 29981 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:54.114679 23025 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 20.
I20260709 13:10:54.099663 29993 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:54.117533 22918 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:54.125572 30028 raft_consensus.cc:2804] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:10:54.126324 30028 raft_consensus.cc:697] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:54.127631 30028 consensus_queue.cc:237] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:54.132149 23617 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60032 (request call id 1498) took 1035 ms. Trace:
I20260709 13:10:54.134667 22950 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 20.
I20260709 13:10:54.142259 29964 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.100143 29867 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:54.162024 29964 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:54.203619 29964 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:54.174507 29949 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 22 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:54.179582 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:54.206095 22769 catalog_manager.cc:5697] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 16 to 19, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 19 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:54.183911 30038 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:54.185148 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:54.188176 23024 raft_consensus.cc:1275] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:10:54.208400 22769 catalog_manager.cc:5697] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:54.192792 22777 catalog_manager.cc:5244] ChangeConfig:REMOVE_PEER RPC for tablet 2959101199a44a4583a4ee456ecf044d with cas_config_opid_index 6: aborting the task: latest config opid_index 11; task opid_index 6
I20260709 13:10:54.148046 23617 rpcz_store.cc:276] 0709 13:10:53.097087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:53.097232 (+   145us) service_pool.cc:224] Handling call
0709 13:10:53.098521 (+  1289us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 13:10:53.110774 (+ 12253us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 13:10:53.110816 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 13:10:53.124089 (+ 13273us) log.cc:844] Serialized 26 byte log entry
0709 13:10:53.849063 (+724974us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b1800270f40. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f6132cdae07 00007f613bb35ab9 00007f6132cf769b 00007f6132cf593f 00007f6132cf7c02 00007f613597f20b 00007f6135984d23 00007f61359850d1 00007f6135da0a0f
0709 13:10:53.935941 (+ 86878us) raft_consensus.cc:1645] Marking committed up to 16
0709 13:10:53.935969 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 13:10:53.936019 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 13:10:54.131883 (+195864us) raft_consensus.cc:1685] finished
0709 13:10:54.131902 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 13:10:54.132095 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11839,"raft.queue_time_us":12426,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":12397,"spinlock_wait_cycles":34534144,"thread_start_us":86717,"threads_started":1,"wal-append.queue_time_us":965632}
I20260709 13:10:54.120170 29993 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:54.211781 22955 raft_consensus.cc:2393] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 0a407504ed1741bf89a276558adf4c0f in current term 22: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:54.216722 23026 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 19.
I20260709 13:10:54.218919 23020 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 22.
I20260709 13:10:54.221057 23027 raft_consensus.cc:2393] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 20: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
W20260709 13:10:54.231612 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 115047us
I20260709 13:10:54.232690 22992 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:54.234102 30044 raft_consensus.cc:2804] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:54.234541 30044 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:54.235002 30044 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:54.236732 22992 leader_election.cc:304] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:10:54.237238 22777 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48906 (request call id 202) took 52972 ms (client timeout 15000 ms). Trace:
I20260709 13:10:54.237680 29714 raft_consensus.cc:2804] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:10:54.238324 29714 raft_consensus.cc:697] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 20 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:54.239251 29714 consensus_queue.cc:237] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:10:54.237658 22777 rpcz_store.cc:269] 0709 13:10:01.271918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:01.308216 (+ 36298us) service_pool.cc:224] Handling call
0709 13:10:01.731664 (+423448us) spinlock_profiling.cc:243] Waited 7.62 ms on lock 0x7b5400002210. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0b71 00007f6132cc10b2 00007f613bc3aa89 00007f613bbd2da6 00007f613bba4088 00007f613bca237e 00007f613a1632a4 00007f613a163232 00007f613a163161 00007f613a1630dc
0709 13:10:02.914839 (+1183175us) spinlock_profiling.cc:243] Waited 98.9 ms on lock 0x7b5400002210. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc0e9c 00007f6132cc10f6 00007f613bc3aa89 00007f613bbd2da6 00007f613bba4088 00007f613bca237e 00007f613a1632a4 00007f613a163232 00007f613a163161 00007f613a1630dc
0709 13:10:03.101638 (+186799us) spinlock_profiling.cc:243] Waited 85 ms on lock 0x7b54002b6e54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4f090 00007f6135dedf3a 00007f613bba41cc 00007f613bca237e 00007f613a1632a4 00007f613a163232 00007f613a163161 00007f613a1630dc 00007f613a162392 00007f613412048c 00007f613411f8c6
0709 13:10:54.237146 (+51135508us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":956780,"reported_tablets":33,"spinlock_wait_cycles":440358784}
I20260709 13:10:54.247172 29973 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:54.248791 22769 catalog_manager.cc:5697] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:54.251142 22769 catalog_manager.cc:5697] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 17 to 20, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 20 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:54.270216 23590 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:54.288534 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:54.289259 22871 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:54.291775 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:54.292279 30023 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:10:54.292824 30023 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:54.293229 30023 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:10:54.300493 22843 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:54.304949 22843 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:54.307461 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:54.308110 22878 raft_consensus.cc:3060] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Advancing to term 19
I20260709 13:10:54.308800 29956 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:10:54.219609 29952 raft_consensus.cc:515] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:54.310477 22948 raft_consensus.cc:2393] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 19: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:54.312152 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:54.312482 29952 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:54.309386 29956 raft_consensus.cc:697] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 22 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:54.313923 29956 consensus_queue.cc:237] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:54.314769 29981 consensus_queue.cc:1048] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.002s
I20260709 13:10:54.312317 29867 leader_election.cc:290] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:54.325824 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:54.326800 23026 raft_consensus.cc:2393] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 19: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:54.329888 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:54.295854 23025 consensus_queue.cc:237] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:10:54.330524 22877 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:10:54.331911 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:54.331672 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:54.332572 23024 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 22.
I20260709 13:10:54.340116 23625 raft_consensus.cc:3060] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Advancing to term 20
I20260709 13:10:54.328743 30044 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:54.347453 23020 raft_consensus.cc:1275] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:10:54.355036 22918 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:54.356359 29964 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:10:54.363442 29964 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:54.359421 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:54.363847 29964 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:10:54.364267 22955 raft_consensus.cc:2393] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 19: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:54.369354 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f1c9e3f9b244b0695775e91f28d67d7" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:54.370389 22952 raft_consensus.cc:2368] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 19. Current term is 20.
I20260709 13:10:54.376159 23588 leader_election.cc:400] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 19. Current term is 20.
I20260709 13:10:54.376618 23588 leader_election.cc:403] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:10:54.391494 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8a996de6f02a4723aef91074b3d56714" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:54.393275 22919 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:54.394768 22920 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:54.397882 22877 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 20.
I20260709 13:10:54.340832 29557 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.423480 23624 raft_consensus.cc:1275] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 13:10:54.427666 22878 raft_consensus.cc:2468] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 19.
I20260709 13:10:54.394078 29973 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:54.411535 29948 raft_consensus.cc:2749] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 13:10:54.413822 29557 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:54.443416 29980 consensus_queue.cc:1048] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.004s
I20260709 13:10:54.418969 29700 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:10:54.465553 30038 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:54.409087 29993 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:54.477717 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:54.481187 23625 raft_consensus.cc:2468] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 20.
I20260709 13:10:54.481815 29934 raft_consensus.cc:2749] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309) with higher term. Message: Invalid argument: T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 for earlier term 19. Current term is 20.
I20260709 13:10:54.485044 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "612dc49c81064f4faa79f65fcf1cfc02" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:54.485657 23020 raft_consensus.cc:3060] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:54.484452 22994 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:54.453228 29700 raft_consensus.cc:493] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:10:54.491513 22994 leader_election.cc:304] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:54.493973 29993 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:54.494088 22951 raft_consensus.cc:2393] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 20: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:54.492239 30041 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.495764 30041 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:54.449652 30023 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:54.497460 22871 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 20.
I20260709 13:10:54.500160 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:54.503551 23020 raft_consensus.cc:2468] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 20.
I20260709 13:10:54.504551 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:54.505760 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:54.505957 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:54.505266 22878 raft_consensus.cc:2393] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 20: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:10:54.506466 23026 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 22.
I20260709 13:10:54.499382 22920 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:54.497725 30041 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:54.510586 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:54.420964 22769 catalog_manager.cc:5697] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 14 to 18, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 18 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:54.518447 22877 raft_consensus.cc:2393] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 19: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:54.505151 30038 raft_consensus.cc:2804] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:10:54.513742 29867 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:54.519102 30038 raft_consensus.cc:697] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 20 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:54.519394 29867 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:54.498054 29949 raft_consensus.cc:2764] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader election decision vote started in defunct term 19: won
I20260709 13:10:54.511346 30052 raft_consensus.cc:2804] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:10:54.521507 29867 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:54.522282 30052 raft_consensus.cc:697] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 19 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:54.523479 30052 consensus_queue.cc:237] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:54.508587 30000 consensus_queue.cc:1048] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 13:10:54.518978 22769 catalog_manager.cc:5697] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 18 to 20, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 20 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:54.498298 29700 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Advancing to term 23
W20260709 13:10:54.533247 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 82149.7us
I20260709 13:10:54.534746 22918 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:54.537209 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:54.541920 22955 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 21.
I20260709 13:10:54.540386 30025 raft_consensus.cc:2804] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:10:54.543145 30025 raft_consensus.cc:697] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:54.544142 30025 consensus_queue.cc:237] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:54.548442 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:54.520228 30038 consensus_queue.cc:237] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:54.549582 23588 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:54.534454 30023 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 24 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:54.564254 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:54.564951 22871 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:10:54.568183 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 57810.4us
I20260709 13:10:54.567484 29993 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:10:54.569773 29993 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:54.570158 29993 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:10:54.577796 29557 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:54.577060 22769 catalog_manager.cc:5697] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 19 to 22, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 22 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:54.578131 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:54.578884 22769 catalog_manager.cc:5697] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 17 to 20, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 20 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:54.583616 30044 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:54.583015 30024 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:10:54.584169 30024 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:54.584507 30024 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:54.575519 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:54.589128 22951 raft_consensus.cc:2393] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 20: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:10:54.593152 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:54.593750 22951 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 19.
I20260709 13:10:54.605698 22948 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 19.
I20260709 13:10:54.607800 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:54.608810 22877 raft_consensus.cc:2393] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 19: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:54.613947 22871 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 24.
I20260709 13:10:54.589473 29964 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:54.617776 22948 raft_consensus.cc:1275] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:10:54.615499 29956 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.620045 29956 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:54.623252 29956 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:54.625701 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:54.620350 23590 leader_election.cc:304] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:54.626662 22948 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 19.
I20260709 13:10:54.627717 22843 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:54.629055 30037 raft_consensus.cc:2804] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:10:54.629637 30037 raft_consensus.cc:697] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 24 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:54.630568 30037 consensus_queue.cc:237] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:54.657827 29964 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 23 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:54.580528 22769 catalog_manager.cc:5697] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:54.660166 22769 catalog_manager.cc:5697] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 17 to 20, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 20 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:54.666829 29700 raft_consensus.cc:515] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:54.668805 22955 raft_consensus.cc:3060] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:10:54.633176 30000 consensus_queue.cc:1048] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.042s
W20260709 13:10:54.671584 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 53132.3us
I20260709 13:10:54.675400 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
W20260709 13:10:54.685791 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 72154.9us
W20260709 13:10:54.685791 22993 consensus_peers.cc:597] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 -> Peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Couldn't send request to peer 5b346767a89745fa82928a5dac9cc0b2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 80ab3f0c16e747e3b1917985c734eed4. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:54.690470 22992 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:54.642853 29956 raft_consensus.cc:2804] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:10:54.695741 22955 raft_consensus.cc:2468] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 24.
I20260709 13:10:54.702286 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7cd8492f63fb4193a47cf9fbc574ef58" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:54.702873 22992 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:54.702888 23025 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:10:54.705204 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:54.705775 23024 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 19.
I20260709 13:10:54.700901 29956 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:54.715030 22950 raft_consensus.cc:1275] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:10:54.716454 22951 raft_consensus.cc:1275] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:10:54.718357 23028 raft_consensus.cc:1275] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:10:54.690698 30024 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:54.757421 23025 raft_consensus.cc:2468] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 23.
I20260709 13:10:54.712850 29867 raft_consensus.cc:2804] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:10:54.712445 29921 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:10:54.718392 29956 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:54.763243 23625 raft_consensus.cc:1240] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 21. Current term is 22. Ops: []
I20260709 13:10:54.768786 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:54.770787 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ab4700268fc4c67b9930087c5da71c8" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:54.769423 29843 consensus_queue.cc:1048] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.003s
I20260709 13:10:54.771526 22877 raft_consensus.cc:2368] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 23. Current term is 24.
I20260709 13:10:54.769212 29867 raft_consensus.cc:697] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 19 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:54.662428 22769 catalog_manager.cc:5697] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 11 to 14, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 14 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:54.778198 29867 consensus_queue.cc:237] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:54.778853 22769 catalog_manager.cc:5697] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:54.790272 30024 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 22 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:54.761822 29921 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:54.803010 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:54.805917 22879 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:54.760213 29883 consensus_queue.cc:1048] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 13:10:54.809729 22951 raft_consensus.cc:1240] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Rejecting Update request from peer 0a407504ed1741bf89a276558adf4c0f for earlier term 22. Current term is 23. Ops: []
I20260709 13:10:54.770879 29700 leader_election.cc:290] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 23 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:54.815634 22920 leader_election.cc:400] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 23 election: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 23. Current term is 24.
I20260709 13:10:54.816149 22920 leader_election.cc:403] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
I20260709 13:10:54.792471 29993 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:54.824323 29993 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 23 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:54.805627 29921 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:10:54.830174 29952 raft_consensus.cc:2749] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 23. Current term is 24.
I20260709 13:10:54.841356 23590 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:54.841279 30000 consensus_queue.cc:1059] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:10:54.843948 30000 raft_consensus.cc:3055] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:10:54.844321 30000 raft_consensus.cc:740] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:54.845134 30000 consensus_queue.cc:260] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:54.846524 30050 raft_consensus.cc:2749] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:10:54.848187 30000 raft_consensus.cc:3060] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:10:54.859524 22877 raft_consensus.cc:1275] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:10:54.860718 22871 raft_consensus.cc:1275] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 13:10:54.863868 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:54.873271 22918 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:54.879141 29873 raft_consensus.cc:2804] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:10:54.879662 29873 raft_consensus.cc:697] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:54.880447 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:54.881055 23024 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:10:54.884425 29873 consensus_queue.cc:237] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:54.887693 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:54.893709 29878 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:54.894388 29878 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:54.897990 22918 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:54.780422 22769 catalog_manager.cc:5697] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:54.900087 30028 raft_consensus.cc:2749] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:10:54.900058 22769 catalog_manager.cc:5697] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 16 to 18, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 18 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:54.909293 23024 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 23.
I20260709 13:10:54.917423 29993 consensus_queue.cc:1048] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:10:54.920631 29714 consensus_queue.cc:1048] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:10:54.922398 29878 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:54.923391 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:54.924141 22948 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 21.
I20260709 13:10:54.925171 22843 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:54.905558 22769 catalog_manager.cc:5697] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 15 to 18. New cstate: current_term: 18 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:54.926280 29878 raft_consensus.cc:2804] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:10:54.926694 22769 catalog_manager.cc:5697] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 13 to 16, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 16 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:54.926788 29878 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:54.927975 23617 raft_consensus.cc:1275] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
W20260709 13:10:54.930807 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 54325us
I20260709 13:10:54.908898 29956 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:54.968207 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:54.956151 29878 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:54.928197 29921 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:54.979916 29955 consensus_queue.cc:1048] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.014s
I20260709 13:10:54.977872 29980 consensus_queue.cc:1048] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 13:10:55.015795 29956 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:55.018374 29921 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:55.021065 22769 catalog_manager.cc:5697] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 15 to 18, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 18 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:55.026166 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:55.023255 22769 catalog_manager.cc:5697] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 13 to 18, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 18 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:55.007874 22918 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:55.027626 22769 catalog_manager.cc:5697] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 13 to 14, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 14 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:55.032729 22871 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "421f71937d5e4cef9084068ce92d12b7"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
mode: GRACEFUL
new_leader_uuid: "f3c799b1249e40119d2a5864231c6c75"
 from {username='slave'} at 127.0.0.1:40244
I20260709 13:10:55.033360 22871 raft_consensus.cc:606] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Received request to transfer leadership to TS f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:55.033720 22871 raft_consensus.cc:614] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 13:10:55.033228 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 101692us
I20260709 13:10:54.988906 29949 consensus_queue.cc:1059] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.022s
I20260709 13:10:55.035944 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:55.036398 22871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:55.036545 22955 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:55.039722 22871 raft_consensus.cc:2393] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 19: Already voted for candidate 0a407504ed1741bf89a276558adf4c0f in this term.
I20260709 13:10:55.041599 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:55.050225 22955 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 20.
I20260709 13:10:55.051873 22843 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:55.056573 30079 raft_consensus.cc:2804] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:10:55.057127 30079 raft_consensus.cc:697] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 20 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:55.061631 22948 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 19.
I20260709 13:10:55.039284 29948 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:10:55.067016 29948 raft_consensus.cc:697] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:55.068094 29948 consensus_queue.cc:237] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:55.063288 22992 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
W20260709 13:10:55.073696 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 131203us
I20260709 13:10:55.075663 29920 raft_consensus.cc:2955] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 20 LEADER]: Committing config change with OpId 20.13: config changed from index -1 to 13, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:10:55.079077 29921 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:10:55.079644 29921 raft_consensus.cc:697] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 19 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:55.089754 29921 consensus_queue.cc:237] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:55.098142 29956 raft_consensus.cc:3055] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:10:55.098598 29956 raft_consensus.cc:740] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:55.099483 29956 consensus_queue.cc:260] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:55.101076 30079 consensus_queue.cc:237] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:55.103478 29956 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:55.102744 22769 catalog_manager.cc:5697] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 12 to 18, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 18 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:10:55.106288 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 56708.1us
I20260709 13:10:55.107542 22871 raft_consensus.cc:1275] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 24 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:10:55.108692 22880 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a424231ef7b4470bbd575564909a365e"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
mode: GRACEFUL
new_leader_uuid: "5b346767a89745fa82928a5dac9cc0b2"
 from {username='slave'} at 127.0.0.1:40244
I20260709 13:10:55.112918 22880 raft_consensus.cc:606] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Received request to transfer leadership to TS 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:55.113215 22880 raft_consensus.cc:614] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:10:55.122031 22871 raft_consensus.cc:1275] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:10:55.133811 30050 consensus_queue.cc:1048] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:10:55.142884 29714 consensus_queue.cc:1048] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:10:55.143401 22879 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
mode: GRACEFUL
new_leader_uuid: "5b346767a89745fa82928a5dac9cc0b2"
 from {username='slave'} at 127.0.0.1:40244
I20260709 13:10:55.143942 22879 raft_consensus.cc:606] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Received request to transfer leadership to TS 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:55.144276 22879 raft_consensus.cc:614] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:10:55.152832 23028 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c26347a647b240d7a3bc5cd449b7993b"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "5b346767a89745fa82928a5dac9cc0b2"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:10:55.153563 23028 raft_consensus.cc:606] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Received request to transfer leadership to TS 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:55.153975 23028 raft_consensus.cc:614] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:10:55.121973 22948 raft_consensus.cc:2955] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Committing config change with OpId 20.13: config changed from index -1 to 13, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:10:55.165655 22949 raft_consensus.cc:1275] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
W20260709 13:10:55.162782 22764 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 7426119f4da44ffaa1513b481a866a01 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER failed with leader 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 0a407504ed1741bf89a276558adf4c0f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 22 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:55.169555 22871 raft_consensus.cc:2955] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Committing config change with OpId 20.13: config changed from index -1 to 13, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:10:55.160379 30044 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:10:55.156605 29878 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:55.217109 22878 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
mode: GRACEFUL
new_leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:40244
I20260709 13:10:55.217736 22878 raft_consensus.cc:606] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Received request to transfer leadership to TS aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:55.218061 22878 raft_consensus.cc:614] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:10:55.220311 29957 consensus_queue.cc:1048] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:10:55.226776 22871 raft_consensus.cc:1275] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:10:55.227385 22877 raft_consensus.cc:1275] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:10:55.226212 30044 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:55.251668 30044 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:55.104640 22769 catalog_manager.cc:5697] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 15 to 18. New cstate: current_term: 18 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:55.254776 22769 catalog_manager.cc:5697] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 17 to 20, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 20 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:55.256027 30038 consensus_queue.cc:1048] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:10:55.257342 22950 raft_consensus.cc:1275] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:10:55.257484 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:55.258389 22955 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 21.
I20260709 13:10:55.262719 29878 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 22 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:55.264241 22878 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
mode: GRACEFUL
new_leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:40244
I20260709 13:10:55.265020 22878 raft_consensus.cc:606] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Received request to transfer leadership to TS aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:55.265410 22878 raft_consensus.cc:614] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:10:55.266206 22769 catalog_manager.cc:5697] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 14 to 20, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 20 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:55.270550 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:55.271209 22950 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:55.274782 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 22 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:55.275432 23028 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Advancing to term 22
I20260709 13:10:55.276365 22876 raft_consensus.cc:1240] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 20 LEADER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 19. Current term is 20. Ops: []
I20260709 13:10:55.262567 22880 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 20 index: 10 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:55.279358 22950 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 22.
I20260709 13:10:55.283215 22880 raft_consensus.cc:2368] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 21. Current term is 22.
I20260709 13:10:55.282974 22843 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:55.274324 22948 consensus_queue.cc:237] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:10:55.290740 21756 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:10:55.285180 30028 consensus_queue.cc:1048] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 13:10:55.293217 29728 consensus_queue.cc:1059] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:10:55.294031 29728 consensus_queue.cc:1243] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer 0a407504ed1741bf89a276558adf4c0f log is divergent from this leader: its last log entry 20.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 13:10:55.294560 29809 raft_consensus.cc:3055] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:10:55.295051 29809 raft_consensus.cc:740] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:55.292496 21756 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:10:55.296154 29809 consensus_queue.cc:260] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:55.297586 29809 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:55.301887 29920 consensus_queue.cc:1048] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:10:55.303086 22992 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:55.308854 30083 raft_consensus.cc:2804] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:10:55.313707 23025 raft_consensus.cc:1275] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:10:55.315382 22877 raft_consensus.cc:1275] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 23 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 13. (term mismatch)
I20260709 13:10:55.315949 22877 pending_rounds.cc:85] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f: Aborting all ops after (but not including) 12
I20260709 13:10:55.316263 22877 pending_rounds.cc:107] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f: Aborting uncommitted NO_OP operation due to leader change: 22.13
I20260709 13:10:55.316586 22877 raft_consensus.cc:2889] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:55.319344 23028 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 22.
I20260709 13:10:55.326948 22994 leader_election.cc:400] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 for earlier term 21. Current term is 22.
I20260709 13:10:55.309648 30083 raft_consensus.cc:697] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 22 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:55.337105 22769 catalog_manager.cc:5697] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 14 to 21, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 21 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:55.322952 30044 raft_consensus.cc:2764] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 13:10:55.357553 29873 consensus_queue.cc:1048] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:10:55.357257 22769 catalog_manager.cc:5697] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 14 to 16, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 16 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:55.359186 22769 catalog_manager.cc:5697] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 19 to 21, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 21 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:55.370126 22769 catalog_manager.cc:5697] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 17 to 20, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 20 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:55.345069 29981 consensus_queue.cc:1048] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 13:10:55.372249 22769 catalog_manager.cc:5697] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 15 to 17. New cstate: current_term: 17 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:55.389564 22950 raft_consensus.cc:1275] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 24 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 13:10:55.391306 30083 consensus_queue.cc:237] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:55.391120 22769 catalog_manager.cc:5697] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 15 to 22. New cstate: current_term: 22 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:55.395134 23026 raft_consensus.cc:1275] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:10:55.409855 29956 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:55.407369 29873 consensus_queue.cc:1048] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 13:10:55.408979 30050 consensus_queue.cc:1048] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.027s
I20260709 13:10:55.443168 29956 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:55.501556 30094 log.cc:927] Time spent T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 13:10:55.508586 22878 raft_consensus.cc:1275] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:10:55.522426 29956 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:55.525866 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:55.526650 22948 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 19.
I20260709 13:10:55.530239 30040 consensus_queue.cc:1048] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:10:55.532827 22843 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
W20260709 13:10:55.539307 22919 consensus_peers.cc:597] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Couldn't send request to peer 5b346767a89745fa82928a5dac9cc0b2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f11454b577ab42d7bfe7c09154b00817. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:10:55.543859 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:55.545709 23028 raft_consensus.cc:1275] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 13:10:55.540836 29956 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:10:55.573906 22878 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:10:55.559128 30095 ts_tablet_manager.cc:933] T 80ab3f0c16e747e3b1917985c734eed4 P 5b346767a89745fa82928a5dac9cc0b2: Initiating tablet copy from peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:55.558821 30041 consensus_queue.cc:1048] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 13:10:55.563970 29956 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:55.600484 29956 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:55.643083 22949 raft_consensus.cc:1275] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:10:55.654217 23025 raft_consensus.cc:1275] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:10:55.667039 30052 consensus_queue.cc:1048] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:10:55.686022 22878 raft_consensus.cc:1275] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 23 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:10:55.687116 30083 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:55.714892 30041 consensus_queue.cc:1048] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.013s
I20260709 13:10:55.746742 29964 consensus_queue.cc:1048] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:10:55.715799 30095 tablet_copy_client.cc:323] T 80ab3f0c16e747e3b1917985c734eed4 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.63.3:42499
I20260709 13:10:55.722369 30083 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:55.781596 30101 log.cc:927] Time spent T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 13:10:55.788128 29956 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:55.789286 22880 raft_consensus.cc:1240] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 19. Current term is 20. Ops: []
I20260709 13:10:55.790469 29956 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:55.787279 29980 raft_consensus.cc:2955] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 LEADER]: Committing config change with OpId 17.13: config changed from index -1 to 13, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:10:55.799984 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:55.802096 29955 consensus_queue.cc:1059] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:10:55.803469 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:10:55.801756 30083 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:55.803144 29955 raft_consensus.cc:3055] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:10:55.807760 29955 raft_consensus.cc:740] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:55.808776 29955 consensus_queue.cc:260] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:55.810241 29955 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:55.810351 23028 raft_consensus.cc:1275] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 13:10:55.810873 23028 pending_rounds.cc:85] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75: Aborting all ops after (but not including) 10
I20260709 13:10:55.811139 23028 pending_rounds.cc:107] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 13:10:55.811408 23028 raft_consensus.cc:2889] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:10:55.818300 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:55.820039 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:55.820381 22948 raft_consensus.cc:1275] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 13:10:55.800907 22879 raft_consensus.cc:2955] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Committing config change with OpId 17.13: config changed from index -1 to 13, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:10:55.842823 22843 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:55.846567 22843 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:10:55.834283 23028 raft_consensus.cc:2955] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Committing config change with OpId 17.13: config changed from index -1 to 13, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:10:55.849974 29956 raft_consensus.cc:2749] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 13:10:55.860208 23039 tablet_copy_service.cc:140] P f3c799b1249e40119d2a5864231c6c75: Received BeginTabletCopySession request for tablet 80ab3f0c16e747e3b1917985c734eed4 from peer 5b346767a89745fa82928a5dac9cc0b2 ({username='slave'} at 127.0.0.1:32794)
I20260709 13:10:55.860958 23039 tablet_copy_service.cc:161] P f3c799b1249e40119d2a5864231c6c75: Beginning new tablet copy session on tablet 80ab3f0c16e747e3b1917985c734eed4 from peer 5b346767a89745fa82928a5dac9cc0b2 at {username='slave'} at 127.0.0.1:32794: session id = 5b346767a89745fa82928a5dac9cc0b2-80ab3f0c16e747e3b1917985c734eed4
I20260709 13:10:55.876411 23039 tablet_copy_source_session.cc:215] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 13:10:55.900535 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 44338.3us
I20260709 13:10:55.912081 29878 raft_consensus.cc:2749] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:10:55.909289 30083 consensus_queue.cc:1048] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.012s
I20260709 13:10:55.923755 29956 consensus_queue.cc:1048] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 13:10:55.926868 22949 raft_consensus.cc:1275] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:10:55.935225 30095 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80ab3f0c16e747e3b1917985c734eed4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:55.930599 29956 consensus_queue.cc:1048] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.009s
I20260709 13:10:56.005033 23026 raft_consensus.cc:1275] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 20 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 13:10:56.011353 29878 consensus_queue.cc:1048] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:10:56.124881 30117 ts_tablet_manager.cc:933] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2: Initiating tablet copy from peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:56.131150 30095 tablet_copy_client.cc:806] T 80ab3f0c16e747e3b1917985c734eed4 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 0 data blocks...
I20260709 13:10:56.131870 30095 tablet_copy_client.cc:670] T 80ab3f0c16e747e3b1917985c734eed4 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:10:56.231606 30117 tablet_copy_client.cc:323] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.63.2:33309
I20260709 13:10:56.244482 30095 tablet_copy_client.cc:538] T 80ab3f0c16e747e3b1917985c734eed4 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:10:56.285895 22965 tablet_copy_service.cc:140] P aed3f089ec8b49af9c8cf5cdee76e8e7: Received BeginTabletCopySession request for tablet f11454b577ab42d7bfe7c09154b00817 from peer 5b346767a89745fa82928a5dac9cc0b2 ({username='slave'} at 127.0.0.1:52222)
I20260709 13:10:56.286578 22965 tablet_copy_service.cc:161] P aed3f089ec8b49af9c8cf5cdee76e8e7: Beginning new tablet copy session on tablet f11454b577ab42d7bfe7c09154b00817 from peer 5b346767a89745fa82928a5dac9cc0b2 at {username='slave'} at 127.0.0.1:52222: session id = 5b346767a89745fa82928a5dac9cc0b2-f11454b577ab42d7bfe7c09154b00817
I20260709 13:10:56.325400 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:56.373958 30095 tablet_bootstrap.cc:492] T 80ab3f0c16e747e3b1917985c734eed4 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap starting.
I20260709 13:10:56.398733 29878 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:56.399282 29878 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:56.402424 29878 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:56.414358 22965 tablet_copy_source_session.cc:215] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:10:56.418852 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:10:56.433135 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 71468us
I20260709 13:10:56.433282 30040 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:56.435412 30040 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:56.438287 30040 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:56.443140 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:56.443547 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:56.444211 23625 raft_consensus.cc:2393] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 22: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:10:56.444267 22877 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 22.
I20260709 13:10:56.449575 22920 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:56.451210 30040 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:10:56.451610 30040 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:10:56.451983 30040 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:10:56.466853 30117 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f11454b577ab42d7bfe7c09154b00817. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:10:56.470592 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:10:56.471320 23625 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 22.
I20260709 13:10:56.462538 30040 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:56.481698 22844 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:10:56.482815 29878 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:10:56.483184 29878 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:56.483510 29878 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:10:56.490806 29878 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:56.515525 29878 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 23 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:56.517421 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:56.518122 23625 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:10:56.521312 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:56.517534 30040 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:56.527448 22877 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:56.528128 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:56.528143 23625 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 23.
I20260709 13:10:56.528245 22877 raft_consensus.cc:2368] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 22. Current term is 23.
I20260709 13:10:56.528803 22951 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:10:56.529698 22844 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:10:56.542379 29956 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:10:56.542915 29956 raft_consensus.cc:697] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 23 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:56.543744 29956 consensus_queue.cc:237] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:56.545341 22920 leader_election.cc:400] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 election: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 22. Current term is 23.
I20260709 13:10:56.545683 22920 leader_election.cc:403] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 13:10:56.551468 22951 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 23.
I20260709 13:10:56.552472 30040 raft_consensus.cc:2749] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 22. Current term is 23.
I20260709 13:10:56.637151 30117 tablet_copy_client.cc:806] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 0 data blocks...
I20260709 13:10:56.653368 30117 tablet_copy_client.cc:670] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:10:56.673424 30117 tablet_copy_client.cc:538] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:10:56.700546 30095 tablet_bootstrap.cc:492] T 80ab3f0c16e747e3b1917985c734eed4 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=1 applied=13 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:56.712468 30095 tablet_bootstrap.cc:492] T 80ab3f0c16e747e3b1917985c734eed4 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap complete.
I20260709 13:10:56.713384 30095 ts_tablet_manager.cc:1403] T 80ab3f0c16e747e3b1917985c734eed4 P 5b346767a89745fa82928a5dac9cc0b2: Time spent bootstrapping tablet: real 0.340s	user 0.045s	sys 0.006s
I20260709 13:10:56.715441 30117 tablet_bootstrap.cc:492] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap starting.
I20260709 13:10:56.777374 30095 raft_consensus.cc:359] T 80ab3f0c16e747e3b1917985c734eed4 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:10:56.780835 30095 raft_consensus.cc:740] T 80ab3f0c16e747e3b1917985c734eed4 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 LEARNER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Initialized, Role: LEARNER
I20260709 13:10:56.781872 30095 consensus_queue.cc:260] T 80ab3f0c16e747e3b1917985c734eed4 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:10:56.796085 30095 ts_tablet_manager.cc:1434] T 80ab3f0c16e747e3b1917985c734eed4 P 5b346767a89745fa82928a5dac9cc0b2: Time spent starting tablet: real 0.082s	user 0.008s	sys 0.000s
I20260709 13:10:56.798980 23039 tablet_copy_service.cc:342] P f3c799b1249e40119d2a5864231c6c75: Request end of tablet copy session 5b346767a89745fa82928a5dac9cc0b2-80ab3f0c16e747e3b1917985c734eed4 received from {username='slave'} at 127.0.0.1:32794
I20260709 13:10:56.800686 23039 tablet_copy_service.cc:434] P f3c799b1249e40119d2a5864231c6c75: ending tablet copy session 5b346767a89745fa82928a5dac9cc0b2-80ab3f0c16e747e3b1917985c734eed4 on tablet 80ab3f0c16e747e3b1917985c734eed4 with peer 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:56.819619 22773 catalog_manager.cc:5697] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 14 to 19. New cstate: current_term: 19 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:56.822273 22773 catalog_manager.cc:5697] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:56.826399 22773 catalog_manager.cc:5697] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 16 to 24, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 24 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:10:56.828195 22778 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48906 (request call id 201) took 80020 ms (client timeout 15000 ms). Trace:
W20260709 13:10:56.828823 22778 rpcz_store.cc:269] 0709 13:09:36.807482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:09:36.808790 (+  1308us) service_pool.cc:224] Handling call
0709 13:10:56.827979 (+80019189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":468944,"reported_tablets":33}
I20260709 13:10:56.828261 22773 catalog_manager.cc:5697] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 13 to 15, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 15 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:56.833374 22773 catalog_manager.cc:5697] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 12 to 13, leader changed from <none> to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 13 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:56.837662 22773 catalog_manager.cc:5697] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 14 to 19, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 19 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:56.840502 22773 catalog_manager.cc:5697] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 13 to 17. New cstate: current_term: 17 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:56.852212 22773 catalog_manager.cc:5697] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 16 to 20, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 20 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:56.854540 22773 catalog_manager.cc:5697] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:56.856976 22773 catalog_manager.cc:5697] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 17 to 18, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 18 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:56.861725 22769 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33466 (request call id 295) took 8195 ms. Trace:
I20260709 13:10:56.862206 22769 rpcz_store.cc:276] 0709 13:10:48.666096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:48.666864 (+   768us) service_pool.cc:224] Handling call
0709 13:10:56.861631 (+8194767us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:10:55.757995 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:10:55.805905 (+ 47910us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:10:55.806057 (+   152us) write_op.cc:435] Acquired schema lock
0709 13:10:55.806089 (+    32us) tablet.cc:662] Decoding operations
0709 13:10:55.876890 (+ 70801us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:10:55.876956 (+    66us) write_op.cc:641] Partition lock acquired for write op
0709 13:10:55.876996 (+    40us) tablet.cc:685] Acquiring locks for 29 operations
0709 13:10:55.879753 (+  2757us) tablet.cc:701] Row locks acquired
0709 13:10:55.879772 (+    19us) write_op.cc:260] PREPARE: finished
0709 13:10:55.879926 (+   154us) write_op.cc:270] Start()
0709 13:10:55.880046 (+   120us) write_op.cc:276] Timestamp: P: 1783602655879876 usec, L: 0
0709 13:10:55.880068 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:10:55.916586 (+ 36518us) log.cc:844] Serialized 11158 byte log entry
0709 13:10:56.121977 (+205391us) op_driver.cc:464] REPLICATION: finished
0709 13:10:56.244897 (+122920us) write_op.cc:301] APPLY: starting
0709 13:10:56.259948 (+ 15051us) tablet.cc:1367] starting BulkCheckPresence
0709 13:10:56.476432 (+216484us) tablet.cc:1370] finished BulkCheckPresence
0709 13:10:56.476533 (+   101us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:10:56.653979 (+177446us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:10:56.654407 (+   428us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=58,key_file_lookups=58,delta_file_lookups=58,mrs_lookups=0
0709 13:10:56.654458 (+    51us) write_op.cc:312] APPLY: finished
0709 13:10:56.655248 (+   790us) log.cc:844] Serialized 252 byte log entry
0709 13:10:56.655457 (+   209us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:56.655706 (+   249us) write_op.cc:454] Released schema lock
0709 13:10:56.655779 (+    73us) write_op.cc:341] FINISH: Updating metrics
0709 13:10:56.656207 (+   428us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:56.656233 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":48,"child_traces":[["op",{"apply.queue_time_us":111777,"apply.run_cpu_time_us":38710,"apply.run_wall_time_us":416579,"cfile_cache_hit":118,"cfile_cache_hit_bytes":125405,"num_ops":29,"prepare.queue_time_us":45956,"prepare.run_cpu_time_us":48461,"prepare.run_wall_time_us":306823,"raft.queue_time_us":5353,"raft.run_cpu_time_us":10727,"raft.run_wall_time_us":72145,"replication_time_us":241736,"spinlock_wait_cycles":57216,"thread_start_us":176704,"threads_started":4,"wal-append.queue_time_us":174574}]]}
I20260709 13:10:56.944211 22951 raft_consensus.cc:1275] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:10:56.947343 29956 consensus_queue.cc:1048] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:10:56.961323 23622 raft_consensus.cc:1217] T 80ab3f0c16e747e3b1917985c734eed4 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 LEARNER]: Deduplicated request from leader. Original: 20.12->[20.13-20.13]   Dedup: 20.13->[]
W20260709 13:10:57.171978 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 59468.8us
W20260709 13:10:57.178746 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 100597us
W20260709 13:10:57.169859 30133 log.cc:927] Time spent T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 13:10:57.169926 30117 log.cc:927] Time spent T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 13:10:57.274307 30044 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:10:57.276376 30044 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:57.279182 30044 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:57.293341 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:10:57.294095 22951 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 19.
I20260709 13:10:57.321477 22992 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:57.339635 30044 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 13:10:57.342985 22764 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 7426119f4da44ffaa1513b481a866a01 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER failed with leader 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 0a407504ed1741bf89a276558adf4c0f is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 22 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:57.336411 29873 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:10:57.344487 30044 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:10:57.355391 29873 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:57.358959 30044 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:57.391374 29873 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:57.408310 30044 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:10:57.409961 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 307610us
I20260709 13:10:57.412276 30117 tablet_bootstrap.cc:492] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=15 overwritten=2 applied=13 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:10:57.412880 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:57.413235 30117 tablet_bootstrap.cc:492] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap complete.
I20260709 13:10:57.413497 22948 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Advancing to term 21
I20260709 13:10:57.413985 30117 ts_tablet_manager.cc:1403] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2: Time spent bootstrapping tablet: real 0.699s	user 0.066s	sys 0.003s
I20260709 13:10:57.432284 22878 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:57.433449 22878 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 20.
I20260709 13:10:57.439244 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:10:57.439898 22876 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:10:57.449304 30117 raft_consensus.cc:359] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:10:57.450227 30117 raft_consensus.cc:740] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 LEARNER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Initialized, Role: LEARNER
I20260709 13:10:57.450882 30117 consensus_queue.cc:260] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:10:57.458838 30117 ts_tablet_manager.cc:1434] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2: Time spent starting tablet: real 0.044s	user 0.008s	sys 0.000s
W20260709 13:10:57.464946 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 92759.2us
I20260709 13:10:57.472431 22948 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 21.
I20260709 13:10:57.476987 22876 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 21.
I20260709 13:10:57.480218 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:57.488174 22965 tablet_copy_service.cc:342] P aed3f089ec8b49af9c8cf5cdee76e8e7: Request end of tablet copy session 5b346767a89745fa82928a5dac9cc0b2-f11454b577ab42d7bfe7c09154b00817 received from {username='slave'} at 127.0.0.1:52222
I20260709 13:10:57.488634 22965 tablet_copy_service.cc:434] P aed3f089ec8b49af9c8cf5cdee76e8e7: ending tablet copy session 5b346767a89745fa82928a5dac9cc0b2-f11454b577ab42d7bfe7c09154b00817 on tablet f11454b577ab42d7bfe7c09154b00817 with peer 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:57.491003 22992 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:57.493876 30139 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:10:57.494390 30139 raft_consensus.cc:697] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 21 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:57.495203 30139 consensus_queue.cc:237] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:57.499995 30044 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:10:57.512298 23625 raft_consensus.cc:1275] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 23 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:10:57.524832 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:10:57.525238 22773 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48906 (request call id 204) took 17601 ms (client timeout 15000 ms). Trace:
I20260709 13:10:57.519788 22769 catalog_manager.cc:5697] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:57.538973 22769 catalog_manager.cc:5697] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 17 to 20, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 20 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:57.540472 22769 catalog_manager.cc:5697] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 20 to 22, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 22 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
W20260709 13:10:57.525652 22773 rpcz_store.cc:269] 0709 13:10:39.913116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:39.915804 (+  2688us) service_pool.cc:224] Handling call
0709 13:10:57.513984 (+17598180us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:10:56.878538 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:10:56.879085 (+   547us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:10:56.879119 (+    34us) write_op.cc:435] Acquired schema lock
0709 13:10:56.879137 (+    18us) tablet.cc:662] Decoding operations
0709 13:10:56.931052 (+ 51915us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:10:56.931127 (+    75us) write_op.cc:641] Partition lock acquired for write op
0709 13:10:56.931161 (+    34us) tablet.cc:685] Acquiring locks for 10 operations
0709 13:10:56.932112 (+   951us) tablet.cc:701] Row locks acquired
0709 13:10:56.932137 (+    25us) write_op.cc:260] PREPARE: finished
0709 13:10:56.932296 (+   159us) write_op.cc:270] Start()
0709 13:10:56.932428 (+   132us) write_op.cc:276] Timestamp: P: 1783602656932244 usec, L: 0
0709 13:10:56.932455 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 13:10:56.933220 (+   765us) log.cc:844] Serialized 4042 byte log entry
0709 13:10:57.010316 (+ 77096us) op_driver.cc:464] REPLICATION: finished
0709 13:10:57.012277 (+  1961us) write_op.cc:301] APPLY: starting
0709 13:10:57.012361 (+    84us) tablet.cc:1367] starting BulkCheckPresence
0709 13:10:57.019644 (+  7283us) tablet.cc:1370] finished BulkCheckPresence
0709 13:10:57.019748 (+   104us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:10:57.248734 (+228986us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:10:57.249222 (+   488us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=20,mrs_lookups=0
0709 13:10:57.249304 (+    82us) write_op.cc:312] APPLY: finished
0709 13:10:57.276652 (+ 27348us) log.cc:844] Serialized 97 byte log entry
0709 13:10:57.285771 (+  9119us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:57.286078 (+   307us) write_op.cc:454] Released schema lock
0709 13:10:57.286129 (+    51us) write_op.cc:341] FINISH: Updating metrics
0709 13:10:57.286463 (+   334us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:57.286492 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":152845,"reported_tablets":34,"child_traces":[["op",{"apply.queue_time_us":1708,"apply.run_cpu_time_us":16878,"apply.run_wall_time_us":274162,"cfile_cache_hit":40,"cfile_cache_hit_bytes":41820,"num_ops":10,"prepare.queue_time_us":16268,"prepare.run_cpu_time_us":4726,"prepare.run_wall_time_us":64514,"replication_time_us":76297,"thread_start_us":17241,"threads_started":1}]]}
I20260709 13:10:57.546247 22874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:10:57.547255 22874 raft_consensus.cc:2368] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 20. Current term is 21.
I20260709 13:10:57.555353 22769 catalog_manager.cc:5697] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 18 to 20, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 20 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:57.557973 22769 catalog_manager.cc:5697] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 10 to 19. New cstate: current_term: 19 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:57.559676 22769 catalog_manager.cc:5697] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 19 to 23, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 23 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:57.561524 22769 catalog_manager.cc:5697] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 14 to 20, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 20 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:57.572522 22769 catalog_manager.cc:5697] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 18 to 22. New cstate: current_term: 22 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:57.574054 22769 catalog_manager.cc:5697] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 12 to 17. New cstate: current_term: 17 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:57.578678 22769 catalog_manager.cc:5697] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:57.604715 23617 raft_consensus.cc:1217] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 LEARNER]: Deduplicated request from leader. Original: 17.12->[17.13-17.13]   Dedup: 17.13->[]
I20260709 13:10:57.602910 22769 catalog_manager.cc:5697] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 18 to 20, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 20 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:57.606232 22769 catalog_manager.cc:5697] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 12 to 19, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 19 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
W20260709 13:10:57.651355 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 43844.2us
W20260709 13:10:57.665479 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 85637.1us
I20260709 13:10:57.665568 30141 consensus_queue.cc:1048] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:10:57.666136 22769 catalog_manager.cc:5697] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 21 to 23, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 23 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:57.667516 22769 catalog_manager.cc:5697] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 14 to 23. New cstate: current_term: 23 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:57.668645 22769 catalog_manager.cc:5697] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 17 to 22, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 22 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:57.669876 22769 catalog_manager.cc:5697] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: config changed from index -1 to 13, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New cstate: current_term: 17 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:10:57.671384 22769 catalog_manager.cc:5697] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 13 to 15, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 15 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
W20260709 13:10:57.676851 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 70218us
W20260709 13:10:57.679134 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 69740.7us
W20260709 13:10:57.680331 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 136916us
I20260709 13:10:57.692415 22920 leader_election.cc:400] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 20. Current term is 21.
I20260709 13:10:57.692830 22920 leader_election.cc:403] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 13:10:57.689124 22769 catalog_manager.cc:5697] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 16 to 19, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 19 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:57.695305 22769 catalog_manager.cc:5697] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 18 to 19, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 19 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:57.696554 22769 catalog_manager.cc:5697] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 20 to 23, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 23 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:57.698246 22769 catalog_manager.cc:5697] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 17 to 22, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 22 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:57.699533 22769 catalog_manager.cc:5697] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: config changed from index -1 to 13, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New cstate: current_term: 20 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:10:57.701359 30151 raft_consensus.cc:2749] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929) with higher term. Message: Invalid argument: T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 20. Current term is 21.
I20260709 13:10:57.711774 30139 raft_consensus.cc:1064] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
I20260709 13:10:57.726351 30139 consensus_queue.cc:237] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 11, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:57.700942 22769 catalog_manager.cc:5697] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 17 to 18, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 18 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:57.733170 22769 catalog_manager.cc:5697] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:57.734542 22769 catalog_manager.cc:5697] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 17 to 20. New cstate: current_term: 20 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:57.735623 22948 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:39742
I20260709 13:10:57.736577 22948 raft_consensus.cc:606] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 LEADER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:57.741974 30025 raft_consensus.cc:993] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7: : Instructing follower 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:10:57.742455 30025 raft_consensus.cc:1081] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 LEADER]: Signalling peer 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:10:57.745308 22873 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:35360
I20260709 13:10:57.745846 22873 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 23 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:57.745573 22948 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:39742
I20260709 13:10:57.746416 22873 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:10:57.747643 22769 catalog_manager.cc:5697] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 15 to 21, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 21 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:57.748986 22769 catalog_manager.cc:5697] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 13 to 20, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 20 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:57.746632 22948 raft_consensus.cc:606] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 LEADER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:57.761978 22948 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:39742
I20260709 13:10:57.762548 22948 raft_consensus.cc:606] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:57.762981 22948 raft_consensus.cc:614] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 13:10:57.767220 22949 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:39742
I20260709 13:10:57.767786 22949 raft_consensus.cc:606] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 LEADER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:10:57.771341 23026 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "5b346767a89745fa82928a5dac9cc0b2"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:10:57.771930 23026 raft_consensus.cc:606] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 19 LEADER]: Received request to transfer leadership to TS 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:10:57.774555 21756 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:10:57.776401 21756 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:10:57.792699 30025 raft_consensus.cc:993] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7: : Instructing follower 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:10:57.793238 30151 raft_consensus.cc:1081] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 LEADER]: Signalling peer 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:10:57.795413 22871 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:35360
I20260709 13:10:57.795980 22871 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:57.796356 22871 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:10:57.810165 22873 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 24 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:57.817608 22871 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 23 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:57.827598 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:57.828300 23028 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:10:57.833564 22949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: true dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:57.834054 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: true dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:57.834172 22949 raft_consensus.cc:3055] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 13:10:57.834592 22949 raft_consensus.cc:740] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:57.835300 22949 consensus_queue.cc:260] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:57.836371 22949 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:10:57.838469 22873 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 24 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:57.846031 22948 raft_consensus.cc:3055] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:10:57.848832 22948 raft_consensus.cc:740] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:57.851452 22951 raft_consensus.cc:1275] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:10:57.851497 22873 raft_consensus.cc:1275] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:10:57.853343 30139 consensus_queue.cc:1048] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:10:57.854209 30136 consensus_queue.cc:1048] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:10:57.855486 23625 raft_consensus.cc:1275] T 80ab3f0c16e747e3b1917985c734eed4 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 LEARNER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:10:57.856134 22948 consensus_queue.cc:260] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:57.857051 30147 consensus_queue.cc:1048] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:10:57.836388 23028 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 24.
I20260709 13:10:57.870795 22843 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:10:57.871727 30141 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:10:57.872383 30141 raft_consensus.cc:697] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 24 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:57.873497 30141 consensus_queue.cc:237] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:10:57.878222 22871 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 23 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:57.858615 22949 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 24.
I20260709 13:10:57.874310 22948 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:10:57.907743 22948 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 23.
I20260709 13:10:57.909448 22843 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:57.912385 30141 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:10:57.912948 30141 raft_consensus.cc:697] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 23 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:57.913868 30141 consensus_queue.cc:237] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:57.918066 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: true dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:57.918812 23625 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:10:57.935380 23625 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 23.
I20260709 13:10:57.991698 30004 raft_consensus.cc:2955] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 20 LEADER]: Committing config change with OpId 20.14: config changed from index 13 to 14, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
W20260709 13:10:57.995157 30158 log.cc:927] Time spent T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 13:10:57.997823 22948 raft_consensus.cc:1275] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:10:58.000092 30139 consensus_queue.cc:1048] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 13:10:58.009346 30159 log.cc:927] Time spent T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 13:10:57.999181 22873 raft_consensus.cc:2955] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Committing config change with OpId 20.14: config changed from index 13 to 14, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:58.030079 22948 raft_consensus.cc:2955] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Committing config change with OpId 20.14: config changed from index 13 to 14, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:58.047632 23622 raft_consensus.cc:2955] T 80ab3f0c16e747e3b1917985c734eed4 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Committing config change with OpId 20.14: config changed from index 13 to 14, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:58.120944 30143 raft_consensus.cc:993] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7: : Instructing follower 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:10:58.121793 30143 raft_consensus.cc:1064] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
I20260709 13:10:58.122102 30025 raft_consensus.cc:1081] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 LEADER]: Signalling peer 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:10:58.125720 22872 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:35360
I20260709 13:10:58.126346 22872 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:58.126754 22872 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:10:58.125640 30143 consensus_queue.cc:237] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 11, Current term: 17, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:58.141186 22873 raft_consensus.cc:1275] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:10:58.157644 30147 consensus_queue.cc:1048] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:10:58.155489 22872 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 23 FOLLOWER]: Starting forced leader election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:58.169011 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: true dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:58.169602 22950 raft_consensus.cc:3055] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 13:10:58.169893 22950 raft_consensus.cc:740] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:10:58.170859 22950 consensus_queue.cc:260] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:58.173770 22950 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:10:58.195662 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:58.196300 23028 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:10:58.197322 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: true dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:10:58.197861 23622 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:10:58.198293 22872 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 23 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:10:58.180399 22950 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 23.
I20260709 13:10:58.214497 23622 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 23.
I20260709 13:10:58.216014 22844 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:58.230141 23028 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 23.
I20260709 13:10:58.225085 30141 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:10:58.336145 23617 raft_consensus.cc:1275] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 LEARNER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:10:58.339707 23026 raft_consensus.cc:1275] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:10:58.342732 29993 consensus_queue.cc:1048] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:10:58.351687 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 79728.7us
I20260709 13:10:58.355172 30152 consensus_queue.cc:1048] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.009s
I20260709 13:10:58.350924 30141 raft_consensus.cc:697] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 23 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:10:58.370630 23024 raft_consensus.cc:1275] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 24 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
W20260709 13:10:58.451347 22844 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 59640.7us
I20260709 13:10:58.452962 22874 raft_consensus.cc:1275] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 13:10:58.457403 29964 consensus_queue.cc:1048] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:10:58.464658 30141 consensus_queue.cc:237] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:58.502307 30004 raft_consensus.cc:993] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75: : Instructing follower 5b346767a89745fa82928a5dac9cc0b2 to start an election
I20260709 13:10:58.504004 30004 raft_consensus.cc:1081] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 19 LEADER]: Signalling peer 5b346767a89745fa82928a5dac9cc0b2 to start an election
I20260709 13:10:58.519794 30182 raft_consensus.cc:2955] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 LEADER]: Committing config change with OpId 17.14: config changed from index 13 to 14, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:58.522842 23026 raft_consensus.cc:2955] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Committing config change with OpId 17.14: config changed from index 13 to 14, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
W20260709 13:10:58.525039 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 67893.3us
I20260709 13:10:58.543893 23617 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589"
dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
 from {username='slave'} at 127.0.0.1:60032
I20260709 13:10:58.544574 23617 raft_consensus.cc:493] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:10:58.545024 23617 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:58.547369 23624 raft_consensus.cc:2955] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Committing config change with OpId 17.14: config changed from index 13 to 14, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
W20260709 13:10:58.556321 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 177781us
I20260709 13:10:58.553802 23617 raft_consensus.cc:515] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:58.564999 30177 consensus_queue.cc:1048] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:10:58.567891 22948 raft_consensus.cc:1275] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 24 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
W20260709 13:10:58.570199 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 113377us
I20260709 13:10:58.577117 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: true dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:10:58.577893 22950 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:58.584226 23617 leader_election.cc:290] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:10:58.593434 22950 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 20.
I20260709 13:10:58.595688 23588 leader_election.cc:304] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:10:58.597175 23624 raft_consensus.cc:1275] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 23 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:10:58.599948 22951 raft_consensus.cc:1275] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:10:58.591930 30173 consensus_queue.cc:1048] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.023s
W20260709 13:10:58.620527 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 50583.8us
I20260709 13:10:58.631089 22873 raft_consensus.cc:2955] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Committing config change with OpId 17.14: config changed from index 13 to 14, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:58.608104 30185 consensus_queue.cc:1048] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 13:10:58.635802 30138 consensus_queue.cc:1048] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:10:58.639770 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "abd5e7d435ca4afd9727e37b2ab31589" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:10:58.640353 23024 raft_consensus.cc:3055] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:10:58.640640 23024 raft_consensus.cc:740] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:10:58.641256 30175 raft_consensus.cc:2804] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:10:58.641307 23024 consensus_queue.cc:260] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:58.641912 30175 raft_consensus.cc:697] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:10:58.642565 23024 raft_consensus.cc:3060] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:10:58.642771 30175 consensus_queue.cc:237] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:10:58.670574 23024 raft_consensus.cc:2468] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 20.
W20260709 13:10:58.673198 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 82772.6us
I20260709 13:10:58.781363 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:10:58.788784 22769 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33466 (request call id 296) took 1731 ms. Trace:
I20260709 13:10:58.791049 22769 rpcz_store.cc:276] 0709 13:10:57.029041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:57.029264 (+   223us) service_pool.cc:224] Handling call
0709 13:10:58.759873 (+1730609us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:10:57.916705 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:10:57.917283 (+   578us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:10:57.917315 (+    32us) write_op.cc:435] Acquired schema lock
0709 13:10:57.917332 (+    17us) tablet.cc:662] Decoding operations
0709 13:10:57.936295 (+ 18963us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:10:57.936386 (+    91us) write_op.cc:641] Partition lock acquired for write op
0709 13:10:57.936416 (+    30us) tablet.cc:685] Acquiring locks for 27 operations
0709 13:10:57.938631 (+  2215us) tablet.cc:701] Row locks acquired
0709 13:10:57.938653 (+    22us) write_op.cc:260] PREPARE: finished
0709 13:10:57.938805 (+   152us) write_op.cc:270] Start()
0709 13:10:57.938927 (+   122us) write_op.cc:276] Timestamp: P: 1783602657938759 usec, L: 0
0709 13:10:57.938953 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 13:10:57.939695 (+   742us) log.cc:844] Serialized 10647 byte log entry
0709 13:10:58.091866 (+152171us) op_driver.cc:464] REPLICATION: finished
0709 13:10:58.161233 (+ 69367us) write_op.cc:301] APPLY: starting
0709 13:10:58.161319 (+    86us) tablet.cc:1367] starting BulkCheckPresence
0709 13:10:58.378942 (+217623us) tablet.cc:1370] finished BulkCheckPresence
0709 13:10:58.379069 (+   127us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:10:58.620813 (+241744us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:10:58.621208 (+   395us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=54,key_file_lookups=54,delta_file_lookups=54,mrs_lookups=0
0709 13:10:58.621255 (+    47us) write_op.cc:312] APPLY: finished
0709 13:10:58.621764 (+   509us) log.cc:844] Serialized 236 byte log entry
0709 13:10:58.621930 (+   166us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:58.622136 (+   206us) write_op.cc:454] Released schema lock
0709 13:10:58.622201 (+    65us) write_op.cc:341] FINISH: Updating metrics
0709 13:10:58.622566 (+   365us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:58.622592 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":29,"child_traces":[["op",{"apply.queue_time_us":67907,"apply.run_cpu_time_us":38003,"apply.run_wall_time_us":461315,"cfile_cache_hit":110,"cfile_cache_hit_bytes":117041,"num_ops":27,"prepare.queue_time_us":131851,"prepare.run_cpu_time_us":6728,"prepare.run_wall_time_us":75157,"replication_time_us":152748,"thread_start_us":196446,"threads_started":2}]]}
I20260709 13:10:58.828770 22776 catalog_manager.cc:5697] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 11 to 20, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 20 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:58.848459 22776 catalog_manager.cc:5697] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 12 to 18. New cstate: current_term: 18 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:58.860025 22776 catalog_manager.cc:5697] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 10 to 17, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 17 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:58.864404 22776 catalog_manager.cc:5697] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:58.867316 22776 catalog_manager.cc:5697] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 15 to 18, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 18 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:58.871455 22776 catalog_manager.cc:5697] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 11 to 16. New cstate: current_term: 16 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:58.945380 23624 raft_consensus.cc:1275] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 23 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:10:58.947347 30185 consensus_queue.cc:1048] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:10:59.029734 23028 raft_consensus.cc:1275] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:10:59.031771 30185 consensus_queue.cc:1048] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:10:59.064348 22950 raft_consensus.cc:1275] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 13:10:59.079383 30185 consensus_queue.cc:1048] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:10:59.171838 22769 catalog_manager.cc:5697] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: config changed from index 13 to 14, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New cstate: current_term: 20 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:59.174415 22769 catalog_manager.cc:5697] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 22 to 23, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 23 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:10:59.180701 22769 catalog_manager.cc:5697] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 18 to 21, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 21 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } }
I20260709 13:10:59.212875 22769 catalog_manager.cc:5697] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 22 to 23, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 23 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:10:59.214828 22769 catalog_manager.cc:5697] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 23 to 24, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 24 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:10:59.204944 22943 tablet_service.cc:1558] Processing DeleteTablet for tablet 2959101199a44a4583a4ee456ecf044d with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 6 (current committed config index is 11)) from {username='slave'} at 127.0.0.1:53202
W20260709 13:10:59.217281 22774 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33488 (request call id 270) took 38288 ms (client timeout 15000 ms). Trace:
W20260709 13:10:59.220167 22774 rpcz_store.cc:269] 0709 13:10:20.929114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:20.959215 (+ 30101us) service_pool.cc:224] Handling call
0709 13:10:59.217096 (+38257881us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1477230,"reported_tablets":49}
I20260709 13:10:59.203550 22776 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33478 (request call id 281) took 9055 ms. Trace:
I20260709 13:10:59.238466 23024 raft_consensus.cc:1275] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:10:59.238776 22950 raft_consensus.cc:1275] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:10:59.239048 22769 catalog_manager.cc:5697] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: config changed from index 13 to 14, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New cstate: current_term: 17 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:10:59.240247 30155 consensus_queue.cc:1048] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:10:59.241567 30155 consensus_queue.cc:1048] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:10:59.235730 22776 rpcz_store.cc:276] 0709 13:10:50.147900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:50.149420 (+  1520us) service_pool.cc:224] Handling call
0709 13:10:59.203461 (+9054041us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:10:58.889747 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:10:58.890629 (+   882us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:10:58.890668 (+    39us) write_op.cc:435] Acquired schema lock
0709 13:10:58.890690 (+    22us) tablet.cc:662] Decoding operations
0709 13:10:58.895731 (+  5041us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:10:58.895815 (+    84us) write_op.cc:641] Partition lock acquired for write op
0709 13:10:58.895842 (+    27us) tablet.cc:685] Acquiring locks for 6 operations
0709 13:10:58.896319 (+   477us) tablet.cc:701] Row locks acquired
0709 13:10:58.896336 (+    17us) write_op.cc:260] PREPARE: finished
0709 13:10:58.896466 (+   130us) write_op.cc:270] Start()
0709 13:10:58.896582 (+   116us) write_op.cc:276] Timestamp: P: 1783602658896418 usec, L: 0
0709 13:10:58.896604 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 13:10:58.897247 (+   643us) log.cc:844] Serialized 2421 byte log entry
0709 13:10:58.922691 (+ 25444us) spinlock_profiling.cc:243] Waited 5.93 ms on lock 0x7b1800267760. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f613bbc0de1 00007f613bbcd5b1 00007f613981c41d 00007f613981e71a 00007f613981e6a8 00007f613981e628 00007f613981e5e3 00007f613981d7fc 00007f613bc50626 00007f613bc4add8 00007f6135df1310
0709 13:10:58.922742 (+    51us) op_driver.cc:464] REPLICATION: finished
0709 13:10:58.923280 (+   538us) write_op.cc:301] APPLY: starting
0709 13:10:58.923367 (+    87us) tablet.cc:1367] starting BulkCheckPresence
0709 13:10:58.926758 (+  3391us) tablet.cc:1370] finished BulkCheckPresence
0709 13:10:58.926781 (+    23us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:10:58.932875 (+  6094us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:10:58.964712 (+ 31837us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=12,key_file_lookups=12,delta_file_lookups=12,mrs_lookups=0
0709 13:10:58.964767 (+    55us) write_op.cc:312] APPLY: finished
0709 13:10:58.965038 (+   271us) log.cc:844] Serialized 65 byte log entry
0709 13:10:58.966022 (+   984us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:58.966132 (+   110us) write_op.cc:454] Released schema lock
0709 13:10:58.966175 (+    43us) write_op.cc:341] FINISH: Updating metrics
0709 13:10:58.966718 (+   543us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:10:58.966750 (+    32us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":62053,"reported_tablets":49,"child_traces":[["op",{"apply.queue_time_us":286,"apply.run_cpu_time_us":8849,"apply.run_wall_time_us":43260,"cfile_cache_hit":24,"cfile_cache_hit_bytes":25092,"num_ops":6,"prepare.queue_time_us":13350,"prepare.run_cpu_time_us":3559,"prepare.run_wall_time_us":10268,"replication_time_us":13394,"spinlock_wait_cycles":13642624,"thread_start_us":12671,"threads_started":1}]]}
I20260709 13:10:59.266968 22762 catalog_manager.cc:5028] TS aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): tablet 2959101199a44a4583a4ee456ecf044d (table test-workload [id=a42f2ccfbe484baca3019323a490c64a]) successfully deleted
W20260709 13:10:59.600996 22775 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33488 (request call id 271) took 13283 ms (client timeout 15000 ms). Trace:
W20260709 13:10:59.609503 22775 rpcz_store.cc:269] 0709 13:10:46.317839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:46.320930 (+  3091us) service_pool.cc:224] Handling call
0709 13:10:59.600779 (+13279849us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1721934,"reported_tablets":49}
W20260709 13:10:59.714082 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 52871.4us
I20260709 13:10:59.887231 22943 tablet_service.cc:1558] Processing DeleteTablet for tablet 2959101199a44a4583a4ee456ecf044d with delete_type TABLET_DATA_TOMBSTONED (Replica with old config index 6 (current committed config index is 11)) from {username='slave'} at 127.0.0.1:53202
W20260709 13:10:59.958326 22770 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33478 (request call id 280) took 28082 ms (client timeout 15000 ms). Trace:
W20260709 13:10:59.959764 22770 rpcz_store.cc:269] 0709 13:10:31.833058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:31.833385 (+   327us) service_pool.cc:224] Handling call
0709 13:10:59.912997 (+28079612us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":171771,"reported_tablets":49}
W20260709 13:10:59.959051 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 48813.9us
I20260709 13:11:00.088488 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:00.103318 22762 catalog_manager.cc:5028] TS aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): tablet 2959101199a44a4583a4ee456ecf044d (table test-workload [id=a42f2ccfbe484baca3019323a490c64a]) successfully deleted
W20260709 13:11:00.113646 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 56675us
I20260709 13:11:00.125429 22777 catalog_manager.cc:5697] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 24 to 25, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 25 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
W20260709 13:11:00.127480 22771 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48906 (request call id 203) took 38948 ms (client timeout 15000 ms). Trace:
W20260709 13:11:00.145023 22771 rpcz_store.cc:269] 0709 13:10:21.178690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:21.178918 (+   228us) service_pool.cc:224] Handling call
0709 13:10:21.981249 (+802331us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b54002b6e54. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6135d4f090 00007f6135dedf3a 00007f613bba41cc 00007f613bca237e 00007f613a1632a4 00007f613a163232 00007f613a163161 00007f613a1630dc 00007f613a162392 00007f613412048c 00007f613411f8c6
0709 13:11:00.127268 (+38146019us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":312012,"reported_tablets":34,"spinlock_wait_cycles":27945216}
I20260709 13:11:00.164211 22777 catalog_manager.cc:5697] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 17 to 18, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 18 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:11:00.239166 23028 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:11:00.240187 23028 raft_consensus.cc:606] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 20 LEADER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:11:00.247638 22777 catalog_manager.cc:5697] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 20 to 22, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 22 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:00.254565 22777 catalog_manager.cc:5697] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 17 to 19, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 19 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:00.256632 22777 catalog_manager.cc:5697] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 18 to 20, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 20 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:11:00.277182 23026 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:11:00.277967 23026 raft_consensus.cc:606] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 20 LEADER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:11:00.285382 22950 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:39742
I20260709 13:11:00.286021 22950 raft_consensus.cc:606] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 LEADER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:11:00.300618 22950 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:39742
I20260709 13:11:00.310133 30222 raft_consensus.cc:993] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7: : Instructing follower 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:11:00.310740 30222 raft_consensus.cc:1081] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 LEADER]: Signalling peer 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:11:00.301281 22950 raft_consensus.cc:606] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 LEADER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:11:00.324347 22876 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:35360
I20260709 13:11:00.325148 22876 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:00.325635 22876 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:11:00.329493 22950 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f603724630e44ef78014e7719f096cb3"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:39742
I20260709 13:11:00.330173 22950 raft_consensus.cc:606] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 LEADER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:11:00.337896 21756 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:11:00.339720 21756 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:11:00.361771 22876 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:00.392171 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: true dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:11:00.393047 22951 raft_consensus.cc:3055] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:11:00.393371 22951 raft_consensus.cc:740] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:11:00.394053 22876 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:11:00.394367 22951 consensus_queue.cc:260] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:00.395645 22951 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:11:00.408507 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: true dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:11:00.409307 23622 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:11:00.433465 22951 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 21.
I20260709 13:11:00.435072 22843 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:11:00.436168 30215 raft_consensus.cc:2804] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:11:00.436681 30215 raft_consensus.cc:697] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 21 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:11:00.437760 30215 consensus_queue.cc:237] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:00.441742 23622 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 21.
I20260709 13:11:00.446923 30213 raft_consensus.cc:993] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7: : Instructing follower 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:11:00.447422 30213 raft_consensus.cc:1081] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 LEADER]: Signalling peer 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:11:00.449410 22874 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f603724630e44ef78014e7719f096cb3"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:35360
I20260709 13:11:00.450104 22874 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:00.450474 22874 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:11:00.472921 30139 raft_consensus.cc:993] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75: : Instructing follower 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:11:00.473505 30139 raft_consensus.cc:1081] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 20 LEADER]: Signalling peer 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:11:00.488765 22874 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:00.496656 22876 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:35370
I20260709 13:11:00.524892 22876 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:00.525377 22876 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:11:00.526036 22874 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:11:00.527279 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:11:00.527885 23026 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:11:00.504895 22771 catalog_manager.cc:5697] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 20 to 21, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 21 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:00.521373 22777 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:48906 (request call id 205) took 4875 ms. Trace:
I20260709 13:11:00.523451 23622 consensus_queue.cc:237] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:00.534387 22777 rpcz_store.cc:276] 0709 13:10:55.646290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:55.646451 (+   161us) service_pool.cc:224] Handling call
0709 13:11:00.521300 (+4874849us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:11:00.280508 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:11:00.281369 (+   861us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:11:00.281428 (+    59us) write_op.cc:435] Acquired schema lock
0709 13:11:00.281472 (+    44us) tablet.cc:662] Decoding operations
0709 13:11:00.282392 (+   920us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:11:00.282516 (+   124us) write_op.cc:641] Partition lock acquired for write op
0709 13:11:00.282543 (+    27us) tablet.cc:685] Acquiring locks for 5 operations
0709 13:11:00.283286 (+   743us) tablet.cc:701] Row locks acquired
0709 13:11:00.283333 (+    47us) write_op.cc:260] PREPARE: finished
0709 13:11:00.283587 (+   254us) write_op.cc:270] Start()
0709 13:11:00.283748 (+   161us) write_op.cc:276] Timestamp: P: 1783602660283515 usec, L: 0
0709 13:11:00.283784 (+    36us) op_driver.cc:348] REPLICATION: starting
0709 13:11:00.296799 (+ 13015us) log.cc:844] Serialized 2035 byte log entry
0709 13:11:00.308072 (+ 11273us) op_driver.cc:464] REPLICATION: finished
0709 13:11:00.408802 (+100730us) write_op.cc:301] APPLY: starting
0709 13:11:00.408899 (+    97us) tablet.cc:1367] starting BulkCheckPresence
0709 13:11:00.465609 (+ 56710us) tablet.cc:1370] finished BulkCheckPresence
0709 13:11:00.465733 (+   124us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:11:00.477424 (+ 11691us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:11:00.477767 (+   343us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=10,mrs_lookups=0
0709 13:11:00.477825 (+    58us) write_op.cc:312] APPLY: finished
0709 13:11:00.478166 (+   341us) log.cc:844] Serialized 57 byte log entry
0709 13:11:00.478373 (+   207us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:00.480762 (+  2389us) write_op.cc:454] Released schema lock
0709 13:11:00.484851 (+  4089us) write_op.cc:341] FINISH: Updating metrics
0709 13:11:00.524106 (+ 39255us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:00.524137 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":28436,"reported_tablets":34,"child_traces":[["op",{"apply.queue_time_us":100436,"apply.run_cpu_time_us":10624,"apply.run_wall_time_us":115276,"cfile_cache_hit":22,"cfile_cache_hit_bytes":25037,"num_ops":5,"prepare.queue_time_us":18687,"prepare.run_cpu_time_us":4681,"prepare.run_wall_time_us":22771,"replication_time_us":24122,"thread_start_us":96506,"threads_started":2}]]}
I20260709 13:11:00.537516 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: true dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:11:00.550606 22951 raft_consensus.cc:3055] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 13:11:00.551746 22951 raft_consensus.cc:740] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:11:00.553606 22951 consensus_queue.cc:260] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:00.556854 30221 raft_consensus.cc:993] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7: : Instructing follower 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:11:00.565074 30221 raft_consensus.cc:1081] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 LEADER]: Signalling peer 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:11:00.568809 22873 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:35360
I20260709 13:11:00.569413 22873 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:00.569921 22873 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:11:00.571471 22876 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:00.587080 22876 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:11:00.587280 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:11:00.588272 23028 raft_consensus.cc:3055] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:11:00.588980 23028 raft_consensus.cc:740] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:11:00.590001 23028 consensus_queue.cc:260] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:00.593206 23024 raft_consensus.cc:1275] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:11:00.595585 22950 raft_consensus.cc:1275] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 13:11:00.597298 30192 consensus_queue.cc:1048] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:11:00.599380 30192 consensus_queue.cc:1048] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:11:00.610985 30219 raft_consensus.cc:993] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75: : Instructing follower 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:11:00.611575 30219 raft_consensus.cc:1081] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 20 LEADER]: Signalling peer 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:11:00.614005 22876 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:35370
I20260709 13:11:00.614718 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: true dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:11:00.615445 23624 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:11:00.614706 22876 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:00.621306 22876 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:11:00.627967 23624 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 21.
I20260709 13:11:00.661136 22876 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:00.661849 22844 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2; no voters: 
I20260709 13:11:00.663971 22876 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:11:00.663640 30215 raft_consensus.cc:2804] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:11:00.665720 30215 raft_consensus.cc:697] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 21 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:11:00.669239 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:11:00.670284 23024 raft_consensus.cc:3055] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:11:00.670647 23024 raft_consensus.cc:740] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:11:00.671671 23024 consensus_queue.cc:260] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:00.673427 23024 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:11:00.678215 23028 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:11:00.679234 22951 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:11:00.681661 23026 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 19.
I20260709 13:11:00.692140 30192 raft_consensus.cc:2955] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 LEADER]: Committing config change with OpId 15.12: config changed from index 9 to 12, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:11:00.764016 22843 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:11:00.765385 30217 raft_consensus.cc:2804] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:11:00.693315 30215 consensus_queue.cc:237] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:00.705919 23025 raft_consensus.cc:2955] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Committing config change with OpId 15.12: config changed from index 9 to 12, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:11:00.774039 30217 raft_consensus.cc:697] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 19 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:11:00.824007 30217 consensus_queue.cc:237] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:00.830540 22763 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ac5cc94926a742e6a0c1340ec33c0ff0 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:00.838881 22951 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 19.
I20260709 13:11:00.855067 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: true dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:11:00.855734 23624 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:11:00.853912 22873 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:00.861923 23024 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 21.
I20260709 13:11:00.871253 23028 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 21.
I20260709 13:11:00.921169 22843 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:11:00.922683 30234 log.cc:927] Time spent T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 13:11:00.924353 30233 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:11:00.925352 30233 raft_consensus.cc:697] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 21 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:11:00.925623 23624 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 21.
I20260709 13:11:00.926401 30233 consensus_queue.cc:237] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:00.981730 22873 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 22 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:11:00.986819 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
W20260709 13:11:00.989639 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 124210us
I20260709 13:11:00.987627 23025 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Advancing to term 22
W20260709 13:11:01.009115 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 76322.3us
W20260709 13:11:01.025684 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 128552us
W20260709 13:11:01.046911 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 181481us
I20260709 13:11:01.055637 23025 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 22.
I20260709 13:11:01.044003 22777 catalog_manager.cc:5697] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 20 to 21, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 21 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:01.070065 22776 catalog_manager.cc:5697] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 19 to 20, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 20 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
W20260709 13:11:01.089401 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 58097.5us
I20260709 13:11:01.138509 22843 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:11:01.153807 22951 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: true dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:11:01.154734 22951 raft_consensus.cc:3055] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:11:01.155139 22951 raft_consensus.cc:740] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:11:01.156551 22949 raft_consensus.cc:1275] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:11:01.169592 22874 raft_consensus.cc:1240] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 21. Current term is 22. Ops: []
I20260709 13:11:01.176928 22951 consensus_queue.cc:260] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:11:01.181157 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 79875.5us
W20260709 13:11:01.182507 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 44081.9us
I20260709 13:11:01.185165 30217 raft_consensus.cc:2804] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:11:01.185691 30217 raft_consensus.cc:697] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 22 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:11:01.186712 30217 consensus_queue.cc:237] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:01.190940 30233 consensus_queue.cc:1048] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 13:11:01.205804 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 333882us
W20260709 13:11:01.224777 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 64621us
I20260709 13:11:01.229055 23024 raft_consensus.cc:1240] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Rejecting Update request from peer aed3f089ec8b49af9c8cf5cdee76e8e7 for earlier term 21. Current term is 22. Ops: []
I20260709 13:11:01.250325 22951 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:11:01.254670 23024 raft_consensus.cc:1275] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:11:01.259065 23625 raft_consensus.cc:1275] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:11:01.267388 30235 consensus_queue.cc:1048] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:11:01.268137 22950 raft_consensus.cc:2955] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Committing config change with OpId 15.12: config changed from index 9 to 12, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New config: { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
W20260709 13:11:01.272641 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 110277us
I20260709 13:11:01.282058 30248 consensus_queue.cc:1048] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:11:01.280432 22951 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 22.
I20260709 13:11:01.325958 23625 raft_consensus.cc:1275] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 13:11:01.327471 30217 consensus_queue.cc:1048] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:11:01.346506 23024 raft_consensus.cc:1275] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:11:01.391783 30233 raft_consensus.cc:493] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:11:01.392537 30233 raft_consensus.cc:515] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:01.397023 30233 leader_election.cc:290] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:11:01.397581 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:11:01.398240 30233 consensus_queue.cc:1048] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:11:01.381026 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:01.405447 22777 catalog_manager.cc:5697] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 20 to 21, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 21 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:01.407505 22777 catalog_manager.cc:5697] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 18 to 19, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 19 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:01.408850 22950 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:11:01.409262 22777 catalog_manager.cc:5697] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 21 to 22, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 22 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:01.439879 22843 leader_election.cc:304] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:01.440698 30235 raft_consensus.cc:2749] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 13:11:01.442148 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac5cc94926a742e6a0c1340ec33c0ff0" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:11:01.447515 22876 consensus_queue.cc:237] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:01.452699 22776 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33478 (request call id 282) took 1953 ms. Trace:
I20260709 13:11:01.453249 22776 rpcz_store.cc:276] 0709 13:10:59.499040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:59.499168 (+   128us) service_pool.cc:224] Handling call
0709 13:10:59.529879 (+ 30711us) spinlock_profiling.cc:243] Waited 3.51 ms on lock 0x7b5400002210. stack: 00007f6132cc6343 00007f6132cc6240 00007f61326551d4 00007f613c393304 00007f613c393299 00007f6132b80db1 00007f6132cc1640 00007f613bb98b76 00007f613bca2c3e 00007f613a1632a4 00007f613a163232 00007f613a163161 00007f613a1630dc 00007f613a162392 00007f613412048c 00007f613411f8c6
0709 13:11:01.452603 (+1922724us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 13:11:01.230503 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 13:11:01.231116 (+   613us) write_op.cc:432] Acquiring schema lock in shared mode
0709 13:11:01.231143 (+    27us) write_op.cc:435] Acquired schema lock
0709 13:11:01.231161 (+    18us) tablet.cc:662] Decoding operations
0709 13:11:01.231494 (+   333us) write_op.cc:620] Acquiring the partition lock for write op
0709 13:11:01.231543 (+    49us) write_op.cc:641] Partition lock acquired for write op
0709 13:11:01.231570 (+    27us) tablet.cc:685] Acquiring locks for 1 operations
0709 13:11:01.231759 (+   189us) tablet.cc:701] Row locks acquired
0709 13:11:01.231793 (+    34us) write_op.cc:260] PREPARE: finished
0709 13:11:01.231947 (+   154us) write_op.cc:270] Start()
0709 13:11:01.232062 (+   115us) write_op.cc:276] Timestamp: P: 1783602661231892 usec, L: 0
0709 13:11:01.232098 (+    36us) op_driver.cc:348] REPLICATION: starting
0709 13:11:01.246704 (+ 14606us) log.cc:844] Serialized 530 byte log entry
0709 13:11:01.345457 (+ 98753us) op_driver.cc:464] REPLICATION: finished
0709 13:11:01.346081 (+   624us) write_op.cc:301] APPLY: starting
0709 13:11:01.346160 (+    79us) tablet.cc:1367] starting BulkCheckPresence
0709 13:11:01.346998 (+   838us) tablet.cc:1370] finished BulkCheckPresence
0709 13:11:01.347092 (+    94us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 13:11:01.347750 (+   658us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 13:11:01.347916 (+   166us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=2,key_file_lookups=2,delta_file_lookups=2,mrs_lookups=0
0709 13:11:01.347970 (+    54us) write_op.cc:312] APPLY: finished
0709 13:11:01.348156 (+   186us) log.cc:844] Serialized 25 byte log entry
0709 13:11:01.348307 (+   151us) write_op.cc:489] Releasing partition, row and schema locks
0709 13:11:01.348376 (+    69us) write_op.cc:454] Released schema lock
0709 13:11:01.348409 (+    33us) write_op.cc:341] FINISH: Updating metrics
Metrics: {"mutex_wait_us":20511,"reported_tablets":34,"spinlock_wait_cycles":8068480,"child_traces":[["op",{"apply.queue_time_us":404,"apply.run_cpu_time_us":2659,"apply.run_wall_time_us":12994,"cfile_cache_hit":4,"cfile_cache_hit_bytes":4182,"num_ops":1,"prepare.queue_time_us":95239,"prepare.run_cpu_time_us":2631,"prepare.run_wall_time_us":16522,"replication_time_us":113225}]]}
I20260709 13:11:01.483218 22951 consensus_queue.cc:237] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:01.495540 23024 raft_consensus.cc:1275] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:11:01.502542 23624 raft_consensus.cc:1275] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 13:11:01.503970 30233 consensus_queue.cc:1048] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:11:01.526217 30233 consensus_queue.cc:1048] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:11:01.526901 22769 catalog_manager.cc:5697] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: config changed from index 9 to 12, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New cstate: current_term: 15 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 12 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:11:01.531821 22950 raft_consensus.cc:1275] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 13:11:01.533305 23625 raft_consensus.cc:1275] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 23 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:11:01.535094 30233 consensus_queue.cc:1048] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:11:01.561762 22867 tablet_service.cc:1558] Processing DeleteTablet for tablet ac5cc94926a742e6a0c1340ec33c0ff0 with delete_type TABLET_DATA_TOMBSTONED (TS 0a407504ed1741bf89a276558adf4c0f not found in new config with opid_index 12) from {username='slave'} at 127.0.0.1:43122
I20260709 13:11:01.604187 30233 consensus_queue.cc:1048] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 13:11:01.615473 30279 tablet_replica.cc:333] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:01.624737 30279 raft_consensus.cc:2243] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:01.625555 30279 raft_consensus.cc:2272] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:01.635478 30279 ts_tablet_manager.cc:1916] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:01.657549 23024 raft_consensus.cc:1275] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 13:11:01.643307 30217 raft_consensus.cc:2955] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 23 LEADER]: Committing config change with OpId 23.16: config changed from index 9 to 16, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:11:01.668475 23622 raft_consensus.cc:1275] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:11:01.665709 23625 raft_consensus.cc:2955] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 23 FOLLOWER]: Committing config change with OpId 23.16: config changed from index 9 to 16, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:11:01.675509 23024 raft_consensus.cc:1275] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 13:11:01.685640 30213 consensus_queue.cc:1048] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:11:01.689404 22876 raft_consensus.cc:1275] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 13:11:01.695106 22775 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33488 (request call id 272) took 1858 ms. Trace:
I20260709 13:11:01.695525 22775 rpcz_store.cc:276] 0709 13:10:59.822573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:10:59.825252 (+  2679us) service_pool.cc:224] Handling call
0709 13:11:01.680098 (+1854846us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1040,"reported_tablets":45}
I20260709 13:11:01.696977 30245 consensus_queue.cc:1048] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.003s
W20260709 13:11:01.713917 30277 log.cc:927] Time spent T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
W20260709 13:11:01.820794 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 98096.5us
I20260709 13:11:01.822589 30215 consensus_queue.cc:1048] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:11:01.875478 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 49864.1us
I20260709 13:11:01.886385 22950 raft_consensus.cc:1275] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
W20260709 13:11:01.898571 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 100407us
I20260709 13:11:01.935031 30284 consensus_queue.cc:1048] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.033s
I20260709 13:11:01.940960 30253 raft_consensus.cc:2955] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 LEADER]: Committing config change with OpId 17.15: config changed from index 14 to 15, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:11:01.960372 23624 raft_consensus.cc:2955] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Committing config change with OpId 17.15: config changed from index 14 to 15, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:11:02.001345 30215 consensus_queue.cc:1048] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.008s
W20260709 13:11:02.021903 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 57640us
W20260709 13:11:02.027894 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 55780.5us
I20260709 13:11:02.038740 22764 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 7426119f4da44ffaa1513b481a866a01 with cas_config_opid_index 9: ChangeConfig:REMOVE_PEER succeeded (attempt 10)
I20260709 13:11:02.032496 23024 raft_consensus.cc:2955] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Committing config change with OpId 23.16: config changed from index 9 to 16, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:11:02.060878 30279 ts_tablet_manager.cc:1929] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 15.11
I20260709 13:11:02.111671 30279 log.cc:1199] T ac5cc94926a742e6a0c1340ec33c0ff0 P 0a407504ed1741bf89a276558adf4c0f: Deleting WAL directory at /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-0-root/wals/ac5cc94926a742e6a0c1340ec33c0ff0
I20260709 13:11:02.262291 22771 catalog_manager.cc:5697] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: config changed from index 9 to 16, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New cstate: current_term: 23 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
W20260709 13:11:02.300700 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 239010us
I20260709 13:11:02.343632 22764 catalog_manager.cc:5028] TS 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): tablet ac5cc94926a742e6a0c1340ec33c0ff0 (table test-workload [id=a42f2ccfbe484baca3019323a490c64a]) successfully deleted
W20260709 13:11:02.349843 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 44651.3us
W20260709 13:11:02.440296 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 53969us
W20260709 13:11:02.460893 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 106116us
W20260709 13:11:02.456280 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 57435.1us
W20260709 13:11:02.507292 30253 consensus_meta.cc:287] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7: Time spent flushing consensus metadata: real 0.564s	user 0.014s	sys 0.000s
W20260709 13:11:02.515492 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 49688.1us
I20260709 13:11:02.519141 30253 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:53202 (request call id 3) took 1038 ms. Trace:
I20260709 13:11:02.501073 23029 consensus_queue.cc:237] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:11:02.521952 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 56025.4us
I20260709 13:11:02.541250 22876 raft_consensus.cc:2955] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Committing config change with OpId 17.15: config changed from index 14 to 15, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:11:02.519711 30253 rpcz_store.cc:276] 0709 13:11:01.480391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 13:11:01.480507 (+   116us) service_pool.cc:224] Handling call
0709 13:11:01.510154 (+ 29647us) log.cc:844] Serialized 487 byte log entry
0709 13:11:02.519010 (+1008856us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8221,"raft.run_cpu_time_us":989,"raft.run_wall_time_us":989,"thread_start_us":143778,"threads_started":1,"wal-append.queue_time_us":138321}
I20260709 13:11:02.566494 22762 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet f11454b577ab42d7bfe7c09154b00817 with cas_config_opid_index 14: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:02.689625 30258 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:02.701587 30258 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:02.706869 22873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:11:02.708750 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:11:02.711230 22992 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:02.712316 30311 raft_consensus.cc:2749] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:11:02.739231 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 121599us
I20260709 13:11:02.739977 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:02.779812 30258 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:11:02.782716 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11454b577ab42d7bfe7c09154b00817" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:11:02.830391 22943 tablet_service.cc:1558] Processing DeleteTablet for tablet 7426119f4da44ffaa1513b481a866a01 with delete_type TABLET_DATA_TOMBSTONED (TS aed3f089ec8b49af9c8cf5cdee76e8e7 not found in new config with opid_index 16) from {username='slave'} at 127.0.0.1:53202
I20260709 13:11:02.843873 30305 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:11:02.844594 30305 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } attrs { replace: true } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:02.848136 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "0a407504ed1741bf89a276558adf4c0f" is_pre_election: true
I20260709 13:11:02.849416 30305 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:11:02.851226 23625 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:11:02.857200 22920 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:11:02.870574 22873 raft_consensus.cc:1275] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:11:02.863034 30305 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:11:02.882922 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:11:02.897046 30179 consensus_queue.cc:1048] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.009s
I20260709 13:11:02.903393 23025 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:11:02.903910 23025 raft_consensus.cc:606] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 20 LEADER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:11:02.925414 23625 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a"
dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
mode: GRACEFUL
new_leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:34202
I20260709 13:11:02.926041 23625 raft_consensus.cc:606] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 19 LEADER]: Received request to transfer leadership to TS aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:02.937107 23625 raft_consensus.cc:1275] T 80ab3f0c16e747e3b1917985c734eed4 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:11:02.941927 23026 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:11:02.942606 23026 raft_consensus.cc:606] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 20 LEADER]: Received request to transfer leadership to TS 0a407504ed1741bf89a276558adf4c0f
I20260709 13:11:02.963972 30244 consensus_queue.cc:1048] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.008s
I20260709 13:11:03.017488 30326 tablet_replica.cc:333] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:03.014185 22771 catalog_manager.cc:5697] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: config changed from index 14 to 15, VOTER f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) evicted. New cstate: current_term: 17 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:11:03.038877 23025 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b"
dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
mode: GRACEFUL
new_leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:60044
I20260709 13:11:03.039427 23025 raft_consensus.cc:606] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 21 LEADER]: Received request to transfer leadership to TS aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:03.014361 30179 raft_consensus.cc:2955] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 20 LEADER]: Committing config change with OpId 20.15: config changed from index 14 to 15, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:11:03.074833 21756 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 13:11:03.076619 21756 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:11:03.079563 22875 raft_consensus.cc:2955] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Committing config change with OpId 20.15: config changed from index 14 to 15, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:11:03.098326 30300 raft_consensus.cc:993] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75: : Instructing follower 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:11:03.098851 30300 raft_consensus.cc:1081] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 20 LEADER]: Signalling peer 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:11:03.112166 22880 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:35370
I20260709 13:11:03.112869 22880 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:03.113472 22880 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:11:03.126520 22880 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:03.132844 22880 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:11:03.142158 30326 raft_consensus.cc:2243] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:03.143061 30326 raft_consensus.cc:2272] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.154045 23019 tablet_service.cc:1558] Processing DeleteTablet for tablet f11454b577ab42d7bfe7c09154b00817 with delete_type TABLET_DATA_TOMBSTONED (TS f3c799b1249e40119d2a5864231c6c75 not found in new config with opid_index 15) from {username='slave'} at 127.0.0.1:39246
I20260709 13:11:03.155579 30326 ts_tablet_manager.cc:1916] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:03.160316 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: true dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:11:03.160988 22948 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:11:03.173833 22948 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 21.
I20260709 13:11:03.184391 22843 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:11:03.185221 30248 raft_consensus.cc:2804] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:11:03.185424 22952 raft_consensus.cc:1240] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Rejecting Update request from peer f3c799b1249e40119d2a5864231c6c75 for earlier term 20. Current term is 21. Ops: []
I20260709 13:11:03.185796 30248 raft_consensus.cc:697] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 21 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:11:03.186143 30311 raft_consensus.cc:993] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75: : Instructing follower aed3f089ec8b49af9c8cf5cdee76e8e7 to start an election
I20260709 13:11:03.186713 30311 raft_consensus.cc:1081] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 21 LEADER]: Signalling peer aed3f089ec8b49af9c8cf5cdee76e8e7 to start an election
I20260709 13:11:03.186741 30248 consensus_queue.cc:237] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:03.193248 30218 raft_consensus.cc:993] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2: : Instructing follower aed3f089ec8b49af9c8cf5cdee76e8e7 to start an election
I20260709 13:11:03.193950 30289 raft_consensus.cc:1081] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 19 LEADER]: Signalling peer aed3f089ec8b49af9c8cf5cdee76e8e7 to start an election
W20260709 13:11:03.192562 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 69641.7us
W20260709 13:11:03.200915 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 60063.1us
I20260709 13:11:03.206311 23617 raft_consensus.cc:2955] T 80ab3f0c16e747e3b1917985c734eed4 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Committing config change with OpId 20.15: config changed from index 14 to 15, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:11:03.211252 22952 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:52222
I20260709 13:11:03.212051 22952 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:03.212456 22952 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:11:03.215070 30331 tablet_replica.cc:333] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:03.217795 30244 consensus_queue.cc:1059] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:11:03.218967 30299 raft_consensus.cc:3055] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:11:03.219331 30299 raft_consensus.cc:740] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:11:03.220075 30299 consensus_queue.cc:260] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:03.221853 30299 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:11:03.228384 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:11:03.237980 22762 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 80ab3f0c16e747e3b1917985c734eed4 with cas_config_opid_index 14: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:03.237823 22948 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b"
dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
 from {username='slave'} at 127.0.0.1:58320
I20260709 13:11:03.238896 22948 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:03.239241 22948 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:11:03.241510 22952 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:03.244966 22952 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 election: Requested vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:11:03.246747 22948 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:03.248960 22948 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:11:03.249543 22876 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: true dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
I20260709 13:11:03.250303 22876 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:11:03.255954 23026 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 21.
I20260709 13:11:03.257803 22876 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 22.
I20260709 13:11:03.258136 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:11:03.258754 23025 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:11:03.259192 22920 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:11:03.260145 30253 raft_consensus.cc:2804] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:11:03.260689 30253 raft_consensus.cc:697] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:11:03.261665 30253 consensus_queue.cc:237] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:03.266372 23025 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 20.
I20260709 13:11:03.268149 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "edc3e3e3969d4c17b76da48ab861a54b" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:11:03.268890 23029 raft_consensus.cc:3055] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 13:11:03.269186 23029 raft_consensus.cc:740] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:11:03.269897 23029 consensus_queue.cc:260] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:03.271639 23029 raft_consensus.cc:3060] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:11:03.286355 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: true dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:11:03.287024 23617 raft_consensus.cc:3055] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 13:11:03.287310 23617 raft_consensus.cc:740] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:11:03.288115 23617 consensus_queue.cc:260] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:03.289163 23617 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:11:03.297170 23617 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 20.
I20260709 13:11:03.302837 30331 raft_consensus.cc:2243] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:03.303941 30331 raft_consensus.cc:2272] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:03.305156 22918 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:11:03.306166 30310 raft_consensus.cc:2804] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:11:03.306736 30310 raft_consensus.cc:697] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:11:03.307706 30310 consensus_queue.cc:237] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:03.308161 22775 catalog_manager.cc:5697] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: config changed from index 14 to 15, VOTER aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) evicted. New cstate: current_term: 20 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:03.316272 23026 raft_consensus.cc:1240] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Rejecting Update request from peer 5b346767a89745fa82928a5dac9cc0b2 for earlier term 19. Current term is 20. Ops: []
I20260709 13:11:03.325232 23029 raft_consensus.cc:2468] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 22.
I20260709 13:11:03.338205 22769 catalog_manager.cc:5697] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 20 to 21, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 21 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:03.355348 30331 ts_tablet_manager.cc:1916] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:03.392297 22943 tablet_service.cc:1558] Processing DeleteTablet for tablet 80ab3f0c16e747e3b1917985c734eed4 with delete_type TABLET_DATA_TOMBSTONED (TS aed3f089ec8b49af9c8cf5cdee76e8e7 not found in new config with opid_index 15) from {username='slave'} at 127.0.0.1:53202
I20260709 13:11:03.398267 22775 catalog_manager.cc:5697] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 21 to 22, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 22 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:03.418864 30258 raft_consensus.cc:993] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75: : Instructing follower 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:11:03.429809 30311 raft_consensus.cc:1081] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 20 LEADER]: Signalling peer 0a407504ed1741bf89a276558adf4c0f to start an election
I20260709 13:11:03.449548 22871 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39"
dest_uuid: "0a407504ed1741bf89a276558adf4c0f"
 from {username='slave'} at 127.0.0.1:35370
I20260709 13:11:03.450105 22871 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:11:03.450421 22871 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:11:03.471354 22871 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:03.484553 22871 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:11:03.486680 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: true dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:11:03.487317 22948 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:11:03.490337 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: true dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:11:03.499315 22948 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 21.
I20260709 13:11:03.500371 22843 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:11:03.501369 30248 raft_consensus.cc:2804] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:11:03.501905 30248 raft_consensus.cc:697] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 21 LEADER]: Becoming Leader. State: Replica: 0a407504ed1741bf89a276558adf4c0f, State: Running, Role: LEADER
I20260709 13:11:03.503021 30248 consensus_queue.cc:237] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:03.506129 23029 raft_consensus.cc:3055] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 13:11:03.507865 23029 raft_consensus.cc:740] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:11:03.508459 23029 consensus_queue.cc:260] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:11:03.512362 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 75121.4us
I20260709 13:11:03.515368 23029 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:11:03.518502 30326 ts_tablet_manager.cc:1929] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 23.15
I20260709 13:11:03.518998 30326 log.cc:1199] T 7426119f4da44ffaa1513b481a866a01 P aed3f089ec8b49af9c8cf5cdee76e8e7: Deleting WAL directory at /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-1-root/wals/7426119f4da44ffaa1513b481a866a01
I20260709 13:11:03.520720 22762 catalog_manager.cc:5028] TS aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): tablet 7426119f4da44ffaa1513b481a866a01 (table test-workload [id=a42f2ccfbe484baca3019323a490c64a]) successfully deleted
I20260709 13:11:03.542308 30326 tablet_replica.cc:333] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:03.544435 30326 raft_consensus.cc:2243] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:03.546417 30326 raft_consensus.cc:2272] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:03.557158 22920 raft_consensus.cc:687] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7: failed to submit failure detected task: Service unavailable: Thread pool token was shut down
I20260709 13:11:03.555694 22775 catalog_manager.cc:5697] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 19 to 20, leader changed from 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 20 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:03.572117 30326 ts_tablet_manager.cc:1916] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 13:11:03.582296 30331 ts_tablet_manager.cc:1929] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 17.14
I20260709 13:11:03.582836 30331 log.cc:1199] T f11454b577ab42d7bfe7c09154b00817 P f3c799b1249e40119d2a5864231c6c75: Deleting WAL directory at /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-2-root/wals/f11454b577ab42d7bfe7c09154b00817
I20260709 13:11:03.584601 22762 catalog_manager.cc:5028] TS f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): tablet f11454b577ab42d7bfe7c09154b00817 (table test-workload [id=a42f2ccfbe484baca3019323a490c64a]) successfully deleted
I20260709 13:11:03.585180 22769 catalog_manager.cc:5697] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f reported cstate change: term changed from 20 to 21, leader changed from f3c799b1249e40119d2a5864231c6c75 (127.21.63.3) to 0a407504ed1741bf89a276558adf4c0f (127.21.63.1). New cstate: current_term: 21 leader_uuid: "0a407504ed1741bf89a276558adf4c0f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:03.550938 23029 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a407504ed1741bf89a276558adf4c0f in term 21.
I20260709 13:11:03.613170 30326 ts_tablet_manager.cc:1929] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 20.14
I20260709 13:11:03.613678 30326 log.cc:1199] T 80ab3f0c16e747e3b1917985c734eed4 P aed3f089ec8b49af9c8cf5cdee76e8e7: Deleting WAL directory at /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783602429420501-21756-0/minicluster-data/ts-1-root/wals/80ab3f0c16e747e3b1917985c734eed4
I20260709 13:11:03.615389 22762 catalog_manager.cc:5028] TS aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): tablet 80ab3f0c16e747e3b1917985c734eed4 (table test-workload [id=a42f2ccfbe484baca3019323a490c64a]) successfully deleted
I20260709 13:11:03.622972 23029 raft_consensus.cc:1275] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:11:03.625432 30248 consensus_queue.cc:1048] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:11:03.666394 22948 raft_consensus.cc:1275] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 13:11:03.667894 30318 consensus_queue.cc:1048] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 13:11:03.731187 23622 raft_consensus.cc:1275] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:11:03.741422 30310 consensus_queue.cc:1048] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:11:03.763799 23028 raft_consensus.cc:1275] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:11:03.766198 30213 consensus_queue.cc:1048] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:11:03.769064 23028 raft_consensus.cc:1275] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:11:03.773046 30213 consensus_queue.cc:1048] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:11:03.803778 22871 raft_consensus.cc:1275] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:11:03.806648 30305 consensus_queue.cc:1048] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:11:03.867228 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 56311us
I20260709 13:11:03.964923 22952 raft_consensus.cc:1275] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:11:03.966527 30292 consensus_queue.cc:1048] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:11:04.029503 23028 raft_consensus.cc:1275] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Refusing update from remote peer 0a407504ed1741bf89a276558adf4c0f: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 13:11:04.035995 30318 consensus_queue.cc:1048] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:11:04.081259 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:11:04.231287 21756 tablet_server.cc:179] TabletServer@127.21.63.1:0 shutting down...
W20260709 13:11:04.233397 22994 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 3 similar messages]
W20260709 13:11:04.285588 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 46459.9us
W20260709 13:11:04.294548 22920 consensus_peers.cc:597] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:04.305877 22994 consensus_peers.cc:597] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:04.308203 22994 consensus_peers.cc:597] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:04.315737 23590 consensus_peers.cc:597] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:04.316428 22994 consensus_peers.cc:597] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:04.320623 23590 consensus_peers.cc:597] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:04.321659 23590 consensus_peers.cc:597] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:04.345011 22920 consensus_peers.cc:597] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:04.345808 22920 consensus_peers.cc:597] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:04.347153 23590 consensus_peers.cc:597] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:04.354949 23590 consensus_peers.cc:597] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:04.371243 22994 consensus_peers.cc:597] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:04.392194 22920 consensus_peers.cc:597] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:04.413189 22920 consensus_peers.cc:597] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:04.460762 23590 consensus_peers.cc:597] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:04.470067 22994 consensus_peers.cc:597] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:04.476126 22994 consensus_peers.cc:597] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:04.488258 22920 consensus_peers.cc:597] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:04.506955 22920 consensus_peers.cc:597] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:04.516412 22994 consensus_peers.cc:597] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:04.530838 22994 consensus_peers.cc:597] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:04.531932 23590 consensus_peers.cc:597] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:04.540123 21756 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:11:04.542492 22994 consensus_peers.cc:597] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:04.545904 21756 tablet_replica.cc:333] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:04.547060 21756 raft_consensus.cc:2243] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:11:04.548228 21756 raft_consensus.cc:2272] T 61adbc0c57034ac28a0b50af8a59ed07 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:04.559561 22920 consensus_peers.cc:597] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:04.565737 21756 tablet_replica.cc:333] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:04.567173 21756 raft_consensus.cc:2243] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:04.576076 21756 raft_consensus.cc:2272] T ced80ef275e04ef8bf4c071883b8284e P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:04.585521 23590 consensus_peers.cc:597] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:04.586426 21756 tablet_replica.cc:333] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:04.587872 21756 raft_consensus.cc:2243] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:11:04.589061 21756 raft_consensus.cc:2272] T 0fa29d11409a4d209411116a7ef8af04 P 0a407504ed1741bf89a276558adf4c0f [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:04.593981 21756 tablet_replica.cc:333] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:04.595182 21756 raft_consensus.cc:2243] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:04.596279 21756 raft_consensus.cc:2272] T f11454b577ab42d7bfe7c09154b00817 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:04.596506 22920 consensus_peers.cc:597] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:04.616408 21756 tablet_replica.cc:333] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:04.617403 21756 raft_consensus.cc:2243] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:11:04.618605 21756 raft_consensus.cc:2272] T 3ec25afe61174e01a75a21ab33afe2e1 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:04.629165 23590 consensus_peers.cc:597] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:04.629980 22920 consensus_peers.cc:597] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:04.633942 21756 tablet_replica.cc:333] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:04.635190 21756 raft_consensus.cc:2243] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:04.638324 21756 raft_consensus.cc:2272] T c3d2a83be5024e50818057526cbaf822 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:04.640239 22948 consensus_queue.cc:237] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
W20260709 13:11:04.651702 22920 consensus_peers.cc:597] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:04.653973 21756 tablet_replica.cc:333] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:04.655002 21756 raft_consensus.cc:2243] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:04.655691 21756 raft_consensus.cc:2272] T edc3e3e3969d4c17b76da48ab861a54b P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:04.663724 21756 tablet_replica.cc:333] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:04.664574 21756 raft_consensus.cc:2243] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:11:04.677114 21756 raft_consensus.cc:2272] T f603724630e44ef78014e7719f096cb3 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:04.684576 22919 consensus_peers.cc:597] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Couldn't send request to peer 5b346767a89745fa82928a5dac9cc0b2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c3d2a83be5024e50818057526cbaf822. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:04.685535 23028 raft_consensus.cc:1275] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 13:11:04.695266 23590 consensus_peers.cc:597] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:04.698680 21756 tablet_replica.cc:333] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:04.699693 21756 raft_consensus.cc:2243] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:04.700361 21756 raft_consensus.cc:2272] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P 0a407504ed1741bf89a276558adf4c0f [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:04.700120 30253 consensus_queue.cc:1048] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 13:11:04.720341 22920 consensus_peers.cc:597] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:04.726899 21756 tablet_replica.cc:333] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:04.728075 21756 raft_consensus.cc:2243] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 21 LEADER]: Raft consensus shutting down.
W20260709 13:11:04.729857 23590 consensus_peers.cc:597] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:04.732329 22920 consensus_peers.cc:597] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:04.737442 21756 raft_consensus.cc:2272] T 4e689c91ad7343b5b6ba36fa648a06ac P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:04.740640 21756 tablet_replica.cc:333] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:04.741622 21756 raft_consensus.cc:2243] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:04.742268 21756 raft_consensus.cc:2272] T 421f71937d5e4cef9084068ce92d12b7 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:04.754606 21756 tablet_replica.cc:333] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:04.755478 21756 raft_consensus.cc:2243] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:04.762375 21756 raft_consensus.cc:2272] T 7cd8492f63fb4193a47cf9fbc574ef58 P 0a407504ed1741bf89a276558adf4c0f [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:04.765442 21756 tablet_replica.cc:333] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:04.766706 21756 raft_consensus.cc:2243] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:11:04.768074 21756 raft_consensus.cc:2272] T f511a9ca0d9b4f54833aa7d0c092ccc1 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:04.771132 21756 tablet_replica.cc:333] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:04.772341 21756 raft_consensus.cc:2243] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:04.773094 21756 raft_consensus.cc:2272] T aa6540758938493d98d7852fdbe3ffab P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:04.778513 21756 tablet_replica.cc:333] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:04.779521 21756 raft_consensus.cc:2243] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:04.780360 21756 raft_consensus.cc:2272] T cdebaf291e5d4ae9814a80ce81ee19fe P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:04.783403 21756 tablet_replica.cc:333] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:04.784898 21756 raft_consensus.cc:2243] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:04.785840 21756 raft_consensus.cc:2272] T e3b39c18a21f448783cdd998acfb2f80 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:04.799558 23025 raft_consensus.cc:2955] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Committing config change with OpId 16.10: config changed from index -1 to 10, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:11:04.796245 30253 raft_consensus.cc:2955] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Committing config change with OpId 16.10: config changed from index -1 to 10, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:11:04.831475 21756 tablet_replica.cc:333] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:04.846383 21756 raft_consensus.cc:2243] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:04.845484 22770 catalog_manager.cc:5697] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: config changed from index -1 to 10, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New cstate: current_term: 16 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:11:04.853451 21756 raft_consensus.cc:2272] T fab4388fa7ea49bda9fa070cf542577f P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:04.856631 21756 tablet_replica.cc:333] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:04.861373 21756 raft_consensus.cc:2243] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:04.865352 21756 raft_consensus.cc:2272] T 6bdc53dc56134e28bcc43e68e76055fc P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:04.875605 22994 consensus_peers.cc:597] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:04.877697 21756 tablet_replica.cc:333] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
W20260709 13:11:04.881119 22826 auto_rebalancer.cc:629] Failed to schedule move for tablet 3ec25afe61174e01a75a21ab33afe2e1: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:11:04.887176 21756 raft_consensus.cc:2243] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:04.888236 21756 raft_consensus.cc:2272] T ec7dd25342084943aacb4b9830f5f1ea P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:04.892475 23029 consensus_queue.cc:237] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:11:04.901345 21756 tablet_replica.cc:333] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:04.902913 21756 raft_consensus.cc:2243] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:04.903960 21756 raft_consensus.cc:2272] T 579096ba88474cfbbe40e8bf769879ec P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:04.907265 21756 tablet_replica.cc:333] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:04.908216 21756 raft_consensus.cc:2243] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:11:04.909754 21756 raft_consensus.cc:2272] T 392f5a2324d74a499e6baad745c941fd P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:04.913251 21756 tablet_replica.cc:333] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:04.914484 21756 raft_consensus.cc:2243] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:11:04.947163 21756 raft_consensus.cc:2272] T 0f076a0e47a14c2b933d3058063c2bcd P 0a407504ed1741bf89a276558adf4c0f [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:04.972972 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 64213.8us
I20260709 13:11:04.978021 21756 tablet_replica.cc:333] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:04.981702 21756 raft_consensus.cc:2243] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:04.994750 21756 raft_consensus.cc:2272] T b7633b7b4c334b039db57d77774df345 P 0a407504ed1741bf89a276558adf4c0f [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:04.997793 21756 tablet_replica.cc:333] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:05.002532 21756 raft_consensus.cc:2243] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:05.003347 21756 raft_consensus.cc:2272] T 85d33f0fd80a49b8bd8cb73571ed3191 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:05.003715 22993 consensus_peers.cc:597] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 -> Peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Couldn't send request to peer 5b346767a89745fa82928a5dac9cc0b2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8a996de6f02a4723aef91074b3d56714. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:05.008093 21756 tablet_replica.cc:333] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:05.015982 21756 raft_consensus.cc:2243] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:05.020612 22952 raft_consensus.cc:1275] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 13:11:05.025553 21756 raft_consensus.cc:2272] T 1120807254f5444997e16f6a7feece4d P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:05.037395 21756 tablet_replica.cc:333] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:05.046604 21756 raft_consensus.cc:2243] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:05.048460 21756 raft_consensus.cc:2272] T 8a996de6f02a4723aef91074b3d56714 P 0a407504ed1741bf89a276558adf4c0f [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:05.040269 30370 consensus_queue.cc:1048] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 13:11:05.057160 22994 consensus_peers.cc:597] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:05.088950 21756 tablet_replica.cc:333] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:05.089838 21756 raft_consensus.cc:2243] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:05.090479 21756 raft_consensus.cc:2272] T 5ab4700268fc4c67b9930087c5da71c8 P 0a407504ed1741bf89a276558adf4c0f [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:05.093992 21756 tablet_replica.cc:333] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:05.095485 21756 raft_consensus.cc:2243] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:11:05.098455 21756 raft_consensus.cc:2272] T 7426119f4da44ffaa1513b481a866a01 P 0a407504ed1741bf89a276558adf4c0f [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:05.104318 21756 tablet_replica.cc:333] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:05.106060 21756 raft_consensus.cc:2243] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:05.108067 21756 raft_consensus.cc:2272] T 13e53d9e00624dfcb87aca9b055fdc9a P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:05.112380 21756 tablet_replica.cc:333] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:05.115737 21756 raft_consensus.cc:2243] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:05.116727 21756 raft_consensus.cc:2272] T 80ab3f0c16e747e3b1917985c734eed4 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:05.120174 21756 tablet_replica.cc:333] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:05.126930 21756 raft_consensus.cc:2243] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 22 LEADER]: Raft consensus shutting down.
W20260709 13:11:05.127077 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 102949us
I20260709 13:11:05.161116 21756 raft_consensus.cc:2272] T 778fe3ade80844c48f8408313fc86a26 P 0a407504ed1741bf89a276558adf4c0f [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:05.172899 21756 tablet_replica.cc:333] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:05.175148 21756 raft_consensus.cc:2243] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:05.175784 21756 raft_consensus.cc:2272] T 7f35ebcf93ad423ea8bd0b777bee23ee P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:05.178776 21756 tablet_replica.cc:333] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:05.179750 21756 raft_consensus.cc:2243] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:11:05.181152 21756 raft_consensus.cc:2272] T 2af6865189dc47369064cfad3fd137e1 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:05.199353 22843 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 67547.2us
I20260709 13:11:05.220690 30375 raft_consensus.cc:493] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:11:05.221361 30375 raft_consensus.cc:515] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:05.223681 30375 leader_election.cc:290] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:11:05.228277 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:11:05.241559 30376 ts_tablet_manager.cc:933] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2: Initiating tablet copy from peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:11:05.241974 30370 raft_consensus.cc:2955] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 19 LEADER]: Committing config change with OpId 19.13: config changed from index -1 to 13, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:11:05.243662 22952 raft_consensus.cc:2955] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Committing config change with OpId 19.13: config changed from index -1 to 13, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:11:05.248777 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2959101199a44a4583a4ee456ecf044d" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:11:05.255702 22843 leader_election.cc:304] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75
I20260709 13:11:05.269986 21756 tablet_replica.cc:333] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:05.272315 21756 raft_consensus.cc:2243] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:11:05.270752 30374 raft_consensus.cc:2749] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 13:11:05.288302 22948 consensus_queue.cc:237] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:11:05.295446 30374 raft_consensus.cc:493] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 5b346767a89745fa82928a5dac9cc0b2)
I20260709 13:11:05.296653 30374 raft_consensus.cc:515] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:05.296977 21756 raft_consensus.cc:2272] T 4c0bea78fb2a40ef90cff914c5c6e250 P 0a407504ed1741bf89a276558adf4c0f [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:05.299381 30374 leader_election.cc:290] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:11:05.300711 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:11:05.306401 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65d9dbf257d434d86d4656dabfd1b70" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:11:05.308354 22844 leader_election.cc:304] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:05.309128 30374 raft_consensus.cc:2749] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:11:05.313335 30376 tablet_copy_client.cc:323] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.63.2:33309
I20260709 13:11:05.317360 21756 tablet_replica.cc:333] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:05.318244 21756 raft_consensus.cc:2243] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:05.319064 21756 raft_consensus.cc:2272] T 616e98b5f7534394ae8790b08aec10e6 P 0a407504ed1741bf89a276558adf4c0f [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:05.319527 22965 tablet_copy_service.cc:140] P aed3f089ec8b49af9c8cf5cdee76e8e7: Received BeginTabletCopySession request for tablet c3d2a83be5024e50818057526cbaf822 from peer 5b346767a89745fa82928a5dac9cc0b2 ({username='slave'} at 127.0.0.1:52222)
I20260709 13:11:05.320137 22965 tablet_copy_service.cc:161] P aed3f089ec8b49af9c8cf5cdee76e8e7: Beginning new tablet copy session on tablet c3d2a83be5024e50818057526cbaf822 from peer 5b346767a89745fa82928a5dac9cc0b2 at {username='slave'} at 127.0.0.1:52222: session id = 5b346767a89745fa82928a5dac9cc0b2-c3d2a83be5024e50818057526cbaf822
I20260709 13:11:05.321722 21756 tablet_replica.cc:333] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:05.322587 21756 raft_consensus.cc:2243] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:05.323135 21756 raft_consensus.cc:2272] T dd740f20d2d74669b0fd10f1ec5b44f0 P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:05.326171 21756 tablet_replica.cc:333] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:05.326933 21756 raft_consensus.cc:2243] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:05.327878 21756 raft_consensus.cc:2272] T f3b08335f3564868a8160dc1b5ade501 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:05.330760 21756 tablet_replica.cc:333] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:05.335093 21756 raft_consensus.cc:2243] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:05.335816 21756 raft_consensus.cc:2272] T 104e2273f37843cbb5245981c817dfe0 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:05.336264 30374 raft_consensus.cc:493] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:11:05.336804 30374 raft_consensus.cc:515] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:05.344722 23028 raft_consensus.cc:1275] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 13:11:05.347453 30253 consensus_queue.cc:1048] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 13:11:05.352584 21756 tablet_replica.cc:333] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:05.353452 21756 raft_consensus.cc:2243] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:05.354008 21756 raft_consensus.cc:2272] T b65d9dbf257d434d86d4656dabfd1b70 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:05.356503 21756 tablet_replica.cc:333] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:05.357227 21756 raft_consensus.cc:2243] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Raft consensus shutting down.
W20260709 13:11:05.357426 22919 consensus_peers.cc:597] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043): Couldn't send request to peer 5b346767a89745fa82928a5dac9cc0b2. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ec7dd25342084943aacb4b9830f5f1ea. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:05.357780 21756 raft_consensus.cc:2272] T 91c88873d54149c58a20f4cf6194e5fc P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:05.359495 30374 leader_election.cc:290] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:11:05.360147 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:11:05.360342 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b41a09964104486c8e50b539ec2b16f6" candidate_uuid: "0a407504ed1741bf89a276558adf4c0f" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:11:05.360625 21756 tablet_replica.cc:333] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:05.363415 22843 leader_election.cc:304] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a407504ed1741bf89a276558adf4c0f; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:11:05.377895 23029 raft_consensus.cc:2955] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Committing config change with OpId 20.14: config changed from index -1 to 14, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:11:05.382290 30374 raft_consensus.cc:2749] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:11:05.376483 30367 raft_consensus.cc:2955] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 LEADER]: Committing config change with OpId 20.14: config changed from index -1 to 14, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:11:05.386633 21756 raft_consensus.cc:2243] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:05.387372 21756 raft_consensus.cc:2272] T b41a09964104486c8e50b539ec2b16f6 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:05.390188 21756 tablet_replica.cc:333] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:05.390889 21756 raft_consensus.cc:2243] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:05.391459 21756 raft_consensus.cc:2272] T a424231ef7b4470bbd575564909a365e P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:05.400941 21756 tablet_replica.cc:333] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:05.402145 21756 raft_consensus.cc:2243] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:11:05.403347 21756 raft_consensus.cc:2272] T b7e4ee754b7a4408b5fc55259d431f39 P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:05.406003 21756 tablet_replica.cc:333] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:05.406716 21756 raft_consensus.cc:2243] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:05.407369 21756 raft_consensus.cc:2272] T cbf52b91ec054ac6a2f8503462b1b7a6 P 0a407504ed1741bf89a276558adf4c0f [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:05.411320 21756 tablet_replica.cc:333] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:05.412086 21756 raft_consensus.cc:2243] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:05.412803 21756 raft_consensus.cc:2272] T 2959101199a44a4583a4ee456ecf044d P 0a407504ed1741bf89a276558adf4c0f [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:05.415597 21756 tablet_replica.cc:333] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:05.416527 21756 raft_consensus.cc:2243] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:11:05.417613 21756 raft_consensus.cc:2272] T 698093275c264d4d99fb8c1a49e3b28f P 0a407504ed1741bf89a276558adf4c0f [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:05.420189 21756 tablet_replica.cc:333] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f: stopping tablet replica
I20260709 13:11:05.420795 21756 raft_consensus.cc:2243] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:05.421818 21756 raft_consensus.cc:2272] T 612dc49c81064f4faa79f65fcf1cfc02 P 0a407504ed1741bf89a276558adf4c0f [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:05.429021 22920 consensus_peers.cc:597] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:05.440815 22965 tablet_copy_source_session.cc:215] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:05.444367 30376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c3d2a83be5024e50818057526cbaf822. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:05.450685 22775 catalog_manager.cc:5697] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: config changed from index -1 to 13, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New cstate: current_term: 19 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } } }
I20260709 13:11:05.501279 30376 tablet_copy_client.cc:806] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:05.502067 30376 tablet_copy_client.cc:670] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:05.510152 30376 tablet_copy_client.cc:538] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:05.516018 30383 ts_tablet_manager.cc:933] T 8a996de6f02a4723aef91074b3d56714 P 5b346767a89745fa82928a5dac9cc0b2: Initiating tablet copy from peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:11:05.540012 30376 tablet_bootstrap.cc:492] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap starting.
I20260709 13:11:05.539265 22775 catalog_manager.cc:5697] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: config changed from index -1 to 14, NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) added. New cstate: current_term: 20 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:05.548771 30383 tablet_copy_client.cc:323] T 8a996de6f02a4723aef91074b3d56714 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.63.3:42499
I20260709 13:11:05.550980 23039 tablet_copy_service.cc:140] P f3c799b1249e40119d2a5864231c6c75: Received BeginTabletCopySession request for tablet 8a996de6f02a4723aef91074b3d56714 from peer 5b346767a89745fa82928a5dac9cc0b2 ({username='slave'} at 127.0.0.1:32794)
I20260709 13:11:05.551561 23039 tablet_copy_service.cc:161] P f3c799b1249e40119d2a5864231c6c75: Beginning new tablet copy session on tablet 8a996de6f02a4723aef91074b3d56714 from peer 5b346767a89745fa82928a5dac9cc0b2 at {username='slave'} at 127.0.0.1:32794: session id = 5b346767a89745fa82928a5dac9cc0b2-8a996de6f02a4723aef91074b3d56714
I20260709 13:11:05.567601 23039 tablet_copy_source_session.cc:215] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:05.581586 30383 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8a996de6f02a4723aef91074b3d56714. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:05.664561 30361 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:11:05.665200 30361 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:05.667263 30361 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:11:05.675559 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:11:05.694573 30192 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:11:05.695122 30192 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:05.698666 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:11:05.699497 22948 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 23.
I20260709 13:11:05.700973 23588 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:11:05.702114 30371 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:11:05.702571 30371 raft_consensus.cc:493] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:11:05.703006 30371 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:11:05.705070 30370 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:11:05.705679 30370 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:11:05.711045 22994 leader_election.cc:336] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:11:05.713272 22920 leader_election.cc:336] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:11:05.713768 22920 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:11:05.714946 30253 raft_consensus.cc:2749] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:11:05.743391 30358 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:11:05.744011 30358 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:05.746060 30358 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:11:05.727288 30192 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:11:05.749652 30370 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:11:05.751658 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:11:05.754514 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:11:05.756014 22992 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:05.758970 30370 raft_consensus.cc:2749] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:11:05.762799 30376 tablet_bootstrap.cc:492] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=10 overwritten=0 applied=10 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:05.763916 30376 tablet_bootstrap.cc:492] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap complete.
I20260709 13:11:05.764786 30376 ts_tablet_manager.cc:1403] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2: Time spent bootstrapping tablet: real 0.225s	user 0.035s	sys 0.012s
I20260709 13:11:05.775549 30371 raft_consensus.cc:515] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:05.784488 30376 raft_consensus.cc:359] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:11:05.785553 30376 raft_consensus.cc:740] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEARNER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Initialized, Role: LEARNER
I20260709 13:11:05.786312 30376 consensus_queue.cc:260] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 10 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:11:05.787635 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0fa29d11409a4d209411116a7ef8af04" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:11:05.788241 22955 raft_consensus.cc:3060] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:11:05.789353 23590 leader_election.cc:336] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:11:05.790287 23590 leader_election.cc:336] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:11:05.790653 23590 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:11:05.795486 22955 raft_consensus.cc:2468] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 24.
I20260709 13:11:05.797030 23588 leader_election.cc:304] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:11:05.797441 30192 raft_consensus.cc:2749] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:11:05.797806 30358 raft_consensus.cc:2804] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:11:05.798337 30358 raft_consensus.cc:697] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 24 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:11:05.799358 30358 consensus_queue.cc:237] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:05.801858 30376 ts_tablet_manager.cc:1434] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.000s
I20260709 13:11:05.805204 30383 tablet_copy_client.cc:806] T 8a996de6f02a4723aef91074b3d56714 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:05.809725 30383 tablet_copy_client.cc:670] T 8a996de6f02a4723aef91074b3d56714 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:05.805876 22965 tablet_copy_service.cc:342] P aed3f089ec8b49af9c8cf5cdee76e8e7: Request end of tablet copy session 5b346767a89745fa82928a5dac9cc0b2-c3d2a83be5024e50818057526cbaf822 received from {username='slave'} at 127.0.0.1:52222
I20260709 13:11:05.805624 30371 leader_election.cc:290] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 24 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:11:05.810361 22965 tablet_copy_service.cc:434] P aed3f089ec8b49af9c8cf5cdee76e8e7: ending tablet copy session 5b346767a89745fa82928a5dac9cc0b2-c3d2a83be5024e50818057526cbaf822 on tablet c3d2a83be5024e50818057526cbaf822 with peer 5b346767a89745fa82928a5dac9cc0b2
W20260709 13:11:05.815254 23590 leader_election.cc:336] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:11:05.832785 30383 tablet_copy_client.cc:538] T 8a996de6f02a4723aef91074b3d56714 P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:05.853448 30253 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:11:05.854095 30253 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:05.856222 30253 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:11:05.857718 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:11:05.860735 22920 leader_election.cc:336] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:11:05.861171 22920 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:11:05.862223 30253 raft_consensus.cc:2749] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:11:05.893793 30383 tablet_bootstrap.cc:492] T 8a996de6f02a4723aef91074b3d56714 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap starting.
I20260709 13:11:05.889071 22775 catalog_manager.cc:5697] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 23 to 24, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 24 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:11:05.908030 23590 connection.cc:573] client connection to 127.21.63.1:33929 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260709 13:11:05.922079 30376 ts_tablet_manager.cc:933] T ec7dd25342084943aacb4b9830f5f1ea P 5b346767a89745fa82928a5dac9cc0b2: Initiating tablet copy from peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:11:05.949590 30376 tablet_copy_client.cc:323] T ec7dd25342084943aacb4b9830f5f1ea P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Beginning tablet copy session from remote peer at address 127.21.63.2:33309
I20260709 13:11:05.951434 22965 tablet_copy_service.cc:140] P aed3f089ec8b49af9c8cf5cdee76e8e7: Received BeginTabletCopySession request for tablet ec7dd25342084943aacb4b9830f5f1ea from peer 5b346767a89745fa82928a5dac9cc0b2 ({username='slave'} at 127.0.0.1:52222)
I20260709 13:11:05.952114 22965 tablet_copy_service.cc:161] P aed3f089ec8b49af9c8cf5cdee76e8e7: Beginning new tablet copy session on tablet ec7dd25342084943aacb4b9830f5f1ea from peer 5b346767a89745fa82928a5dac9cc0b2 at {username='slave'} at 127.0.0.1:52222: session id = 5b346767a89745fa82928a5dac9cc0b2-ec7dd25342084943aacb4b9830f5f1ea
I20260709 13:11:05.960901 22965 tablet_copy_source_session.cc:215] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 13:11:05.964572 30361 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:11:05.965137 30361 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:05.967206 30361 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:11:05.969224 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:11:05.972251 30370 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:11:05.972854 30370 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:05.976447 30370 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:11:05.977835 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:11:05.983080 22920 leader_election.cc:336] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:05.983577 22920 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:11:05.984321 30361 raft_consensus.cc:2749] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 13:11:05.992976 30370 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:11:05.993578 30370 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:05.996615 30370 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:11:05.996610 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:11:05.997660 30361 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:11:05.998085 22955 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 20.
I20260709 13:11:05.998220 30361 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:05.999437 22992 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:11:06.000788 30361 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:11:06.002599 30370 raft_consensus.cc:2804] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:11:06.003059 30370 raft_consensus.cc:493] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:11:06.004269 30370 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:11:06.003916 30376 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec7dd25342084943aacb4b9830f5f1ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:06.021344 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:11:06.022040 23028 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 20.
I20260709 13:11:06.027225 22918 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:11:06.028405 30361 raft_consensus.cc:2804] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:11:06.028810 30361 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:11:06.029177 30361 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:11:06.039678 30370 raft_consensus.cc:515] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:06.041918 30361 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:06.042176 30371 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:11:06.048555 30371 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:06.050889 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "61adbc0c57034ac28a0b50af8a59ed07" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:11:06.051462 22955 raft_consensus.cc:3060] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:11:06.051613 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:11:06.053188 30371 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:11:06.055212 23590 leader_election.cc:336] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:06.055670 23590 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:11:06.056851 30371 raft_consensus.cc:2749] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:11:06.057657 22920 leader_election.cc:336] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:06.058699 22955 raft_consensus.cc:2468] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 21.
I20260709 13:11:06.059918 22992 leader_election.cc:304] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:11:06.060858 30368 raft_consensus.cc:2804] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:11:06.061337 30368 raft_consensus.cc:697] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 21 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:11:06.062175 30368 consensus_queue.cc:237] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:06.066509 30372 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:11:06.067142 30372 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:06.069447 30372 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:06.070673 22920 leader_election.cc:336] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:06.074163 23622 raft_consensus.cc:1217] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEARNER]: Deduplicated request from leader. Original: 16.9->[16.10-16.10]   Dedup: 16.10->[]
I20260709 13:11:06.079054 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:11:06.063485 30361 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 21 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:11:06.065913 22994 leader_election.cc:336] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:06.081097 22994 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:06.081876 30372 raft_consensus.cc:2749] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:11:06.082947 22994 leader_election.cc:336] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:06.086079 30372 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:11:06.086802 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3ec25afe61174e01a75a21ab33afe2e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:11:06.086629 30372 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:06.087347 23028 raft_consensus.cc:3060] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:11:06.089548 30372 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:11:06.090683 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:11:06.091284 23622 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 21.
I20260709 13:11:06.092403 22993 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:11:06.093261 30372 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:11:06.093626 30372 raft_consensus.cc:493] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:11:06.074618 30370 leader_election.cc:290] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:11:06.095691 23028 raft_consensus.cc:2468] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 21.
I20260709 13:11:06.097733 30383 tablet_bootstrap.cc:492] T 8a996de6f02a4723aef91074b3d56714 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=1 applied=13 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:06.096982 30253 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:11:06.098719 30383 tablet_bootstrap.cc:492] T 8a996de6f02a4723aef91074b3d56714 P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap complete.
I20260709 13:11:06.098415 30253 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:06.099560 30383 ts_tablet_manager.cc:1403] T 8a996de6f02a4723aef91074b3d56714 P 5b346767a89745fa82928a5dac9cc0b2: Time spent bootstrapping tablet: real 0.206s	user 0.044s	sys 0.007s
I20260709 13:11:06.106932 30253 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:11:06.093981 30372 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:11:06.113114 22918 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:11:06.113420 30253 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:11:06.114245 30361 raft_consensus.cc:2804] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:11:06.114173 30253 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:06.114821 30361 raft_consensus.cc:697] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:11:06.115742 30361 consensus_queue.cc:237] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:11:06.123821 22920 leader_election.cc:336] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:06.128371 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:11:06.129015 23029 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 19.
I20260709 13:11:06.128096 22770 catalog_manager.cc:5697] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 20 to 21, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 21 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
W20260709 13:11:06.133034 22920 leader_election.cc:336] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:06.139874 22994 leader_election.cc:336] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:06.140542 22994 leader_election.cc:336] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:06.140972 22994 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:06.148628 30372 raft_consensus.cc:515] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:06.152319 30370 raft_consensus.cc:2749] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:11:06.152258 22775 catalog_manager.cc:5697] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 20 to 21, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 21 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:06.153265 30370 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:11:06.153861 30370 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:06.152786 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4e689c91ad7343b5b6ba36fa648a06ac" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:11:06.136112 30368 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:11:06.155992 23622 raft_consensus.cc:3060] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:11:06.140321 30253 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:11:06.166103 23622 raft_consensus.cc:2468] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 22.
I20260709 13:11:06.157435 30368 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:06.169443 30383 raft_consensus.cc:359] T 8a996de6f02a4723aef91074b3d56714 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:11:06.190738 22993 leader_election.cc:304] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:11:06.158607 30372 leader_election.cc:290] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:11:06.185935 30383 raft_consensus.cc:740] T 8a996de6f02a4723aef91074b3d56714 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 LEARNER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Initialized, Role: LEARNER
I20260709 13:11:06.189668 30253 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
W20260709 13:11:06.210222 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 75000.1us
I20260709 13:11:06.212379 22918 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
W20260709 13:11:06.215296 22994 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 74772.1us
I20260709 13:11:06.216732 30394 raft_consensus.cc:2804] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:11:06.217294 30394 raft_consensus.cc:697] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 22 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:11:06.218118 30394 consensus_queue.cc:237] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:06.219606 30361 raft_consensus.cc:2804] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 13:11:06.220055 30361 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
W20260709 13:11:06.216185 22994 leader_election.cc:336] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:06.224793 30368 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:11:06.211035 30253 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:06.229426 30370 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:11:06.220453 30361 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:11:06.231967 30383 consensus_queue.cc:260] T 8a996de6f02a4723aef91074b3d56714 P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 13 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:11:06.246157 30361 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:06.247613 30383 ts_tablet_manager.cc:1434] T 8a996de6f02a4723aef91074b3d56714 P 5b346767a89745fa82928a5dac9cc0b2: Time spent starting tablet: real 0.148s	user 0.007s	sys 0.001s
I20260709 13:11:06.253682 30361 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:11:06.258149 30253 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:11:06.255276 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 64649.4us
I20260709 13:11:06.268075 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:11:06.270466 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:11:06.269011 23621 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 21.
I20260709 13:11:06.285409 30376 tablet_copy_client.cc:806] T ec7dd25342084943aacb4b9830f5f1ea P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 0 data blocks...
I20260709 13:11:06.286083 30376 tablet_copy_client.cc:670] T ec7dd25342084943aacb4b9830f5f1ea P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Starting download of 1 WAL segments...
I20260709 13:11:06.265028 30396 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:11:06.286885 30396 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:06.289239 30396 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:11:06.289915 22948 raft_consensus.cc:1275] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 24 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 13:11:06.291988 30376 tablet_copy_client.cc:538] T ec7dd25342084943aacb4b9830f5f1ea P 5b346767a89745fa82928a5dac9cc0b2: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 13:11:06.292070 30358 consensus_queue.cc:1048] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 13:11:06.301239 30376 tablet_bootstrap.cc:492] T ec7dd25342084943aacb4b9830f5f1ea P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap starting.
W20260709 13:11:06.313138 22920 leader_election.cc:336] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:06.313727 22920 leader_election.cc:336] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:06.317121 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 104839us
I20260709 13:11:06.320645 23039 tablet_copy_service.cc:342] P f3c799b1249e40119d2a5864231c6c75: Request end of tablet copy session 5b346767a89745fa82928a5dac9cc0b2-8a996de6f02a4723aef91074b3d56714 received from {username='slave'} at 127.0.0.1:32794
I20260709 13:11:06.321120 23039 tablet_copy_service.cc:434] P f3c799b1249e40119d2a5864231c6c75: ending tablet copy session 5b346767a89745fa82928a5dac9cc0b2-8a996de6f02a4723aef91074b3d56714 on tablet 8a996de6f02a4723aef91074b3d56714 with peer 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:11:06.323272 22955 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:11:06.323911 22955 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 24.
I20260709 13:11:06.326581 22992 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:11:06.327100 22993 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:11:06.346276 30402 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:11:06.347347 30368 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 13:11:06.349901 30368 raft_consensus.cc:493] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:11:06.350284 30368 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:11:06.348448 30402 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:06.353051 30400 raft_consensus.cc:2804] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:11:06.353502 30400 raft_consensus.cc:493] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:11:06.353850 30400 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:11:06.354555 30402 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:11:06.371176 30372 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:11:06.371726 30372 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:06.373643 30372 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:11:06.386664 30371 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:11:06.405711 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:11:06.406491 22952 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 21.
I20260709 13:11:06.404852 30371 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:06.415829 30371 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:11:06.422160 30368 raft_consensus.cc:515] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:11:06.424227 23590 consensus_peers.cc:597] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:06.424903 23590 leader_election.cc:336] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:06.428620 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f603724630e44ef78014e7719f096cb3" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:11:06.429472 23025 raft_consensus.cc:3060] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Advancing to term 20
I20260709 13:11:06.429628 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:11:06.433370 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:11:06.434171 23028 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 22.
I20260709 13:11:06.435469 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:11:06.436273 23029 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 21.
I20260709 13:11:06.424510 30368 leader_election.cc:290] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 25 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:11:06.453052 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f076a0e47a14c2b933d3058063c2bcd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:11:06.453886 22948 raft_consensus.cc:3060] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 24 FOLLOWER]: Advancing to term 25
I20260709 13:11:06.455987 23025 raft_consensus.cc:2468] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 20.
I20260709 13:11:06.462822 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:11:06.467648 22918 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:11:06.467200 23020 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 23.
I20260709 13:11:06.474900 22918 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:11:06.475580 30253 raft_consensus.cc:2804] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:11:06.476027 30253 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:11:06.476392 30253 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:11:06.497859 22992 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:11:06.502712 30406 raft_consensus.cc:2804] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:11:06.503180 30406 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:11:06.503548 30406 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:11:06.508224 30404 raft_consensus.cc:2804] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:11:06.508714 30404 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:11:06.509105 30404 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:11:06.510833 30406 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:06.513563 30406 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:11:06.521294 30400 raft_consensus.cc:515] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:11:06.523301 22994 leader_election.cc:336] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:06.523674 30400 leader_election.cc:290] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:11:06.523790 22994 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:11:06.530052 22948 raft_consensus.cc:2468] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 25.
I20260709 13:11:06.530812 22770 catalog_manager.cc:5697] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 21 to 22, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 22 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:06.536921 30253 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:11:06.545195 22994 leader_election.cc:336] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:06.546240 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:11:06.546758 23588 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:11:06.528005 30404 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:06.547591 30394 raft_consensus.cc:2749] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:11:06.549232 30404 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:11:06.549965 22948 raft_consensus.cc:2393] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 22: Already voted for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in this term.
I20260709 13:11:06.545583 30371 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:11:06.553256 22992 leader_election.cc:304] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:11:06.553067 30371 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:06.555462 30371 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:11:06.558532 30406 raft_consensus.cc:2804] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 13:11:06.559656 30406 raft_consensus.cc:697] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 25 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:11:06.560564 30406 consensus_queue.cc:237] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:11:06.561090 23590 leader_election.cc:336] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:06.554797 22994 leader_election.cc:336] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:06.569432 22955 raft_consensus.cc:1275] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 13:11:06.570521 22994 leader_election.cc:336] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:06.571125 30413 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:11:06.571012 30400 consensus_queue.cc:1048] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 13:11:06.571061 22920 leader_election.cc:336] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:06.581614 30253 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 23 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:11:06.576377 30413 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:06.587311 30413 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:06.578994 22994 leader_election.cc:336] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:06.593031 22994 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:06.598433 30400 raft_consensus.cc:2749] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:11:06.600190 22994 leader_election.cc:336] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:06.604579 22994 leader_election.cc:336] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:06.605706 23590 leader_election.cc:336] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:06.601348 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 54665.2us
I20260709 13:11:06.593757 30398 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:11:06.609347 30398 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:06.609766 30371 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:11:06.610390 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
W20260709 13:11:06.610908 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 114101us
I20260709 13:11:06.611486 30398 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:11:06.611810 22918 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:11:06.612568 30397 raft_consensus.cc:1064] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
I20260709 13:11:06.613616 30361 raft_consensus.cc:2804] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 13:11:06.610250 30371 raft_consensus.cc:493] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:11:06.614130 30361 raft_consensus.cc:697] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:11:06.614231 30371 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:11:06.615175 30361 consensus_queue.cc:237] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:06.615252 30397 consensus_queue.cc:237] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 8, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:06.616217 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:11:06.616847 22992 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:06.618188 30410 raft_consensus.cc:2749] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 13:11:06.618366 22948 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 23.
W20260709 13:11:06.619470 22994 leader_election.cc:336] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:06.625841 23588 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:06.630198 23588 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f
W20260709 13:11:06.638113 22920 consensus_peers.cc:597] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:06.638650 22920 leader_election.cc:336] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:06.639181 22920 consensus_peers.cc:597] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:06.639887 30192 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 13:11:06.640417 30192 raft_consensus.cc:493] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:11:06.640887 30192 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:11:06.644641 23029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:11:06.646029 23029 raft_consensus.cc:2393] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 22: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:11:06.646950 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f511a9ca0d9b4f54833aa7d0c092ccc1" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:11:06.647549 23025 raft_consensus.cc:3060] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:11:06.655269 23025 raft_consensus.cc:2468] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 23.
W20260709 13:11:06.660614 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 333539us
I20260709 13:11:06.683838 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:11:06.686776 30192 raft_consensus.cc:515] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:06.688936 30192 leader_election.cc:290] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 24 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:11:06.667727 30405 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:11:06.690047 23622 raft_consensus.cc:2393] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 24: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
W20260709 13:11:06.693027 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 53084.4us
I20260709 13:11:06.695320 22775 catalog_manager.cc:5697] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 24 to 25, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 25 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:06.672674 30416 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:11:06.702842 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "392f5a2324d74a499e6baad745c941fd" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:11:06.703446 23622 raft_consensus.cc:3060] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:11:06.708189 22918 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:11:06.705116 30416 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:06.714061 30415 raft_consensus.cc:2749] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:11:06.714184 30416 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:11:06.719485 22770 catalog_manager.cc:5697] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 19 to 20, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 20 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:06.720520 23622 raft_consensus.cc:2468] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 22.
I20260709 13:11:06.727727 30410 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:11:06.728355 30410 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:06.730648 30410 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:06.734865 22920 leader_election.cc:336] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:06.732487 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:11:06.744046 22948 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 21.
W20260709 13:11:06.745539 22994 leader_election.cc:336] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:06.749306 30371 raft_consensus.cc:515] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:06.751389 30371 leader_election.cc:290] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 24 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:11:06.695175 30405 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:06.752250 22992 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
W20260709 13:11:06.754562 22994 consensus_peers.cc:597] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:06.755113 22994 leader_election.cc:336] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:06.759070 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 50968.7us
I20260709 13:11:06.760087 22918 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:11:06.763475 23020 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:11:06.767432 30367 raft_consensus.cc:2804] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:11:06.768030 30367 raft_consensus.cc:697] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:11:06.768241 30394 raft_consensus.cc:2804] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 13:11:06.768724 30394 raft_consensus.cc:493] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:11:06.769100 30394 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:11:06.768898 30367 consensus_queue.cc:237] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:11:06.772433 22994 consensus_peers.cc:597] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:06.780519 23590 consensus_peers.cc:597] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:06.782755 22993 leader_election.cc:304] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:11:06.783981 30396 raft_consensus.cc:2804] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 13:11:06.784138 30417 log.cc:927] Time spent T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 13:11:06.784543 30396 raft_consensus.cc:697] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 22 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:11:06.785485 30396 consensus_queue.cc:237] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:06.765530 30404 raft_consensus.cc:2749] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
W20260709 13:11:06.811302 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 171639us
I20260709 13:11:06.812062 23622 raft_consensus.cc:1217] T 8a996de6f02a4723aef91074b3d56714 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 LEARNER]: Deduplicated request from leader. Original: 19.12->[19.13-19.13]   Dedup: 19.13->[]
W20260709 13:11:06.816500 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 122637us
I20260709 13:11:06.822521 30405 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:11:06.824421 23590 leader_election.cc:336] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:06.831552 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 57916.7us
I20260709 13:11:06.833104 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:11:06.826093 23590 leader_election.cc:336] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:06.833701 23028 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 22.
I20260709 13:11:06.837833 23622 raft_consensus.cc:1275] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 13:11:06.841001 30405 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:11:06.841579 30405 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:11:06.850589 22994 consensus_peers.cc:597] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:06.860527 22919 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:11:06.865543 22918 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:11:06.867424 23028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7426119f4da44ffaa1513b481a866a01" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:11:06.868022 23028 raft_consensus.cc:3060] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Advancing to term 24
I20260709 13:11:06.876752 23029 raft_consensus.cc:1275] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 13:11:06.878304 22920 leader_election.cc:336] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:06.878806 22920 consensus_peers.cc:597] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:06.880746 22948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c0bea78fb2a40ef90cff914c5c6e250" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:11:06.881461 23025 raft_consensus.cc:1275] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:11:06.844132 30400 consensus_queue.cc:1048] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.028s
I20260709 13:11:06.904553 30424 raft_consensus.cc:2804] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:11:06.905298 30424 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
I20260709 13:11:06.905812 30424 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:11:06.884023 30428 raft_consensus.cc:3060] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Advancing to term 24
W20260709 13:11:06.920706 21756 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 22901)
I20260709 13:11:06.937451 23617 raft_consensus.cc:1275] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEARNER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 13:11:06.941728 23028 raft_consensus.cc:2468] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 24.
W20260709 13:11:06.968863 30373 log.cc:927] Time spent T 00000000000000000000000000000000 P 8d175b4ae72f4fc48971c2f31668971c: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
W20260709 13:11:06.979864 23590 consensus_peers.cc:597] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:06.996505 30428 raft_consensus.cc:2749] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:11:06.999251 30424 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:06.999768 30422 consensus_queue.cc:1048] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 13:11:07.011147 22948 raft_consensus.cc:2468] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 24.
I20260709 13:11:07.005491 22769 catalog_manager.cc:5697] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 21 to 22, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 22 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:07.022439 23622 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:11:07.023347 23622 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 21.
I20260709 13:11:07.034988 22919 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 
I20260709 13:11:07.038120 22770 catalog_manager.cc:5697] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 22 to 23, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 23 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
W20260709 13:11:07.047039 22920 consensus_peers.cc:597] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:07.049743 22920 leader_election.cc:336] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:07.049935 30433 raft_consensus.cc:2804] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 13:11:07.050397 22920 consensus_peers.cc:597] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:07.050482 30433 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 0a407504ed1741bf89a276558adf4c0f)
W20260709 13:11:07.050949 22920 leader_election.cc:336] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:07.051035 30433 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:11:07.003047 30405 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:11:06.946347 30430 consensus_queue.cc:1048] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.089s
I20260709 13:11:07.063038 30394 raft_consensus.cc:515] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:07.063453 30404 consensus_queue.cc:1048] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 13:11:07.064532 23020 raft_consensus.cc:2393] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in current term 22: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
W20260709 13:11:07.075889 22920 consensus_peers.cc:597] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:07.087824 23026 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "778fe3ade80844c48f8408313fc86a26" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:11:07.089804 23026 raft_consensus.cc:3060] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:11:07.097836 23588 leader_election.cc:304] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:11:07.098284 30394 leader_election.cc:290] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:11:07.099676 23588 leader_election.cc:304] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:11:07.100334 30415 raft_consensus.cc:2804] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:11:07.100945 30415 raft_consensus.cc:697] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 24 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:11:07.101816 30415 consensus_queue.cc:237] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:07.107577 22952 raft_consensus.cc:1275] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 25 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 13:11:07.106501 22954 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7e4ee754b7a4408b5fc55259d431f39" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:11:07.110242 22954 raft_consensus.cc:3060] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:11:07.114199 30434 raft_consensus.cc:2804] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 13:11:07.114773 30434 raft_consensus.cc:697] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 24 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:11:07.115605 30434 consensus_queue.cc:237] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:11:07.127007 23590 consensus_peers.cc:597] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:07.127707 23590 consensus_peers.cc:597] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:07.128110 23590 consensus_peers.cc:597] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:07.128459 23590 consensus_peers.cc:597] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:07.128914 23590 consensus_peers.cc:597] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:07.139076 23026 raft_consensus.cc:2468] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 23.
I20260709 13:11:07.085665 30424 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 23 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:11:07.157255 30437 log.cc:927] Time spent T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 13:11:07.141922 22918 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
W20260709 13:11:07.143285 22920 leader_election.cc:336] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:07.149290 30400 consensus_queue.cc:1048] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 13:11:07.166760 22920 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:11:07.171710 30404 raft_consensus.cc:2804] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:11:07.203765 22954 raft_consensus.cc:2468] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 22.
W20260709 13:11:07.210217 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 51105.8us
I20260709 13:11:07.210922 30424 raft_consensus.cc:2749] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 13:11:07.215480 30404 raft_consensus.cc:697] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
I20260709 13:11:07.257889 30404 consensus_queue.cc:237] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:07.232071 30433 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:11:07.238662 22920 consensus_peers.cc:597] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:07.241513 22994 leader_election.cc:336] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:07.269220 30433 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:11:07.271858 23590 consensus_peers.cc:597] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:07.279389 23621 raft_consensus.cc:1275] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:11:07.279551 23020 raft_consensus.cc:1275] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 13:11:07.281409 22994 consensus_peers.cc:597] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:07.281875 22994 consensus_peers.cc:597] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:07.282330 22994 consensus_peers.cc:597] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:07.282815 22994 consensus_peers.cc:597] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:07.284201 22920 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 45579.1us
W20260709 13:11:07.284742 22920 consensus_peers.cc:597] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:07.285167 22920 consensus_peers.cc:597] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:07.285615 22920 consensus_peers.cc:597] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:07.305704 22992 leader_election.cc:304] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
W20260709 13:11:07.308547 22994 consensus_peers.cc:597] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:07.310112 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "698093275c264d4d99fb8c1a49e3b28f" candidate_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:11:07.310694 23621 raft_consensus.cc:3060] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Advancing to term 22
I20260709 13:11:07.313206 30376 tablet_bootstrap.cc:492] T ec7dd25342084943aacb4b9830f5f1ea P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:07.324206 30253 consensus_queue.cc:1048] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 13:11:07.330404 30407 consensus_queue.cc:1048] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.004s
I20260709 13:11:07.331147 30401 raft_consensus.cc:2804] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:11:07.331717 30401 raft_consensus.cc:697] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 22 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:11:07.332602 30401 consensus_queue.cc:237] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:11:07.350638 22920 consensus_peers.cc:597] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:07.351282 22920 consensus_peers.cc:597] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:07.362421 23621 raft_consensus.cc:2468] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aed3f089ec8b49af9c8cf5cdee76e8e7 in term 22.
W20260709 13:11:07.364826 22920 consensus_peers.cc:597] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:07.365492 22920 leader_election.cc:336] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:07.374749 23026 raft_consensus.cc:1275] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:11:07.388422 30376 tablet_bootstrap.cc:492] T ec7dd25342084943aacb4b9830f5f1ea P 5b346767a89745fa82928a5dac9cc0b2: Bootstrap complete.
I20260709 13:11:07.389971 30376 ts_tablet_manager.cc:1403] T ec7dd25342084943aacb4b9830f5f1ea P 5b346767a89745fa82928a5dac9cc0b2: Time spent bootstrapping tablet: real 1.089s	user 0.067s	sys 0.034s
I20260709 13:11:07.406028 30376 raft_consensus.cc:359] T ec7dd25342084943aacb4b9830f5f1ea P 5b346767a89745fa82928a5dac9cc0b2 [term 20 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:11:07.413738 30376 raft_consensus.cc:740] T ec7dd25342084943aacb4b9830f5f1ea P 5b346767a89745fa82928a5dac9cc0b2 [term 20 LEARNER]: Becoming Follower/Learner. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Initialized, Role: LEARNER
I20260709 13:11:07.414919 30376 consensus_queue.cc:260] T ec7dd25342084943aacb4b9830f5f1ea P 5b346767a89745fa82928a5dac9cc0b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: NON_VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: true } }
I20260709 13:11:07.417902 30376 ts_tablet_manager.cc:1434] T ec7dd25342084943aacb4b9830f5f1ea P 5b346767a89745fa82928a5dac9cc0b2: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
I20260709 13:11:07.419540 22965 tablet_copy_service.cc:342] P aed3f089ec8b49af9c8cf5cdee76e8e7: Request end of tablet copy session 5b346767a89745fa82928a5dac9cc0b2-ec7dd25342084943aacb4b9830f5f1ea received from {username='slave'} at 127.0.0.1:52222
I20260709 13:11:07.419974 22965 tablet_copy_service.cc:434] P aed3f089ec8b49af9c8cf5cdee76e8e7: ending tablet copy session 5b346767a89745fa82928a5dac9cc0b2-ec7dd25342084943aacb4b9830f5f1ea on tablet ec7dd25342084943aacb4b9830f5f1ea with peer 5b346767a89745fa82928a5dac9cc0b2
I20260709 13:11:07.423501 22769 catalog_manager.cc:5697] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 21 to 22, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 22 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:07.429313 22770 catalog_manager.cc:5697] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 23 to 24, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 24 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:07.432628 30430 consensus_queue.cc:1048] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 13:11:07.434302 22770 catalog_manager.cc:5697] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 23 to 24, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 24 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:07.436314 22775 catalog_manager.cc:5697] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 22 to 23, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 23 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } }
W20260709 13:11:07.438905 23590 consensus_peers.cc:597] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:07.448114 22919 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:11:07.468779 30410 raft_consensus.cc:1064] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
I20260709 13:11:07.485028 23622 raft_consensus.cc:1217] T ec7dd25342084943aacb4b9830f5f1ea P 5b346767a89745fa82928a5dac9cc0b2 [term 20 LEARNER]: Deduplicated request from leader. Original: 20.13->[20.14-20.14]   Dedup: 20.14->[]
I20260709 13:11:07.456079 30422 raft_consensus.cc:2804] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 13:11:07.505975 30422 raft_consensus.cc:697] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 LEADER]: Becoming Leader. State: Replica: aed3f089ec8b49af9c8cf5cdee76e8e7, State: Running, Role: LEADER
W20260709 13:11:07.516917 22920 consensus_peers.cc:597] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:07.520962 22994 consensus_peers.cc:597] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:07.534906 30433 raft_consensus.cc:2955] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Committing config change with OpId 16.11: config changed from index 10 to 11, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:11:07.541721 30422 consensus_queue.cc:237] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:07.550607 30410 consensus_queue.cc:237] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 11, Current term: 19, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:11:07.562498 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 54793us
I20260709 13:11:07.564759 23024 raft_consensus.cc:2955] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index 10 to 11, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
W20260709 13:11:07.643658 23590 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 61710.7us
W20260709 13:11:07.693615 23590 consensus_peers.cc:597] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:07.703925 23590 consensus_peers.cc:597] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:07.709860 23590 consensus_peers.cc:597] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:07.719280 30444 log.cc:927] Time spent T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
W20260709 13:11:07.756210 30450 log.cc:927] Time spent T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
W20260709 13:11:07.767719 23590 consensus_peers.cc:597] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:07.804527 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 140796us
I20260709 13:11:07.806103 23624 raft_consensus.cc:2955] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Committing config change with OpId 16.11: config changed from index 10 to 11, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:11:07.841351 22954 raft_consensus.cc:1275] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 24 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 13:11:07.834018 22770 catalog_manager.cc:5697] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: term changed from 21 to 22, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2). New cstate: current_term: 22 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:11:07.865641 22920 consensus_peers.cc:597] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:07.885667 30427 consensus_queue.cc:1048] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 13:11:07.889626 23029 raft_consensus.cc:1275] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 24 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 13:11:07.901528 30371 consensus_queue.cc:1048] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:11:07.918651 22946 raft_consensus.cc:1275] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 13:11:07.922971 22946 raft_consensus.cc:1275] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:11:07.923715 30399 consensus_queue.cc:1048] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 13:11:07.925897 22994 consensus_peers.cc:597] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:07.929397 30424 raft_consensus.cc:1064] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7: attempting to promote NON_VOTER 5b346767a89745fa82928a5dac9cc0b2 to VOTER
W20260709 13:11:07.934278 21756 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 22901)
I20260709 13:11:07.932219 22769 catalog_manager.cc:5697] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: config changed from index 10 to 11, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New cstate: current_term: 16 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:11:07.931986 30424 consensus_queue.cc:237] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 12, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:07.947314 30372 consensus_queue.cc:1048] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 13:11:07.987015 23622 raft_consensus.cc:1275] T 8a996de6f02a4723aef91074b3d56714 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 LEARNER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 13:11:08.022348 23028 raft_consensus.cc:1275] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
W20260709 13:11:08.034444 22994 consensus_peers.cc:597] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:08.035073 22994 consensus_peers.cc:597] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:08.092414 30372 consensus_queue.cc:1048] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.018s
W20260709 13:11:08.110873 22920 consensus_peers.cc:597] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:08.120762 30361 consensus_queue.cc:1048] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
W20260709 13:11:08.122517 22994 consensus_peers.cc:597] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:08.125211 22994 consensus_peers.cc:597] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:08.126155 22994 consensus_peers.cc:597] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:08.161701 22920 consensus_peers.cc:597] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:08.178015 23020 raft_consensus.cc:1275] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:11:08.183765 30361 consensus_queue.cc:1048] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:11:08.215643 23624 raft_consensus.cc:1275] T ec7dd25342084943aacb4b9830f5f1ea P 5b346767a89745fa82928a5dac9cc0b2 [term 20 LEARNER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 13:11:08.217319 23624 raft_consensus.cc:1275] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 13:11:08.223702 30458 raft_consensus.cc:2955] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 19 LEADER]: Committing config change with OpId 19.14: config changed from index 13 to 14, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:11:08.240000 22946 raft_consensus.cc:2955] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Committing config change with OpId 19.14: config changed from index 13 to 14, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:11:08.248001 30424 consensus_queue.cc:1048] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:11:08.249177 30398 consensus_queue.cc:1048] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:11:08.271885 23617 raft_consensus.cc:2955] T 8a996de6f02a4723aef91074b3d56714 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Committing config change with OpId 19.14: config changed from index 13 to 14, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:11:08.268908 22775 catalog_manager.cc:5697] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: config changed from index 13 to 14, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New cstate: current_term: 19 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 14 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:08.348397 23029 consensus_queue.cc:237] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:11:08.397249 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 44952.1us
I20260709 13:11:08.400975 23621 raft_consensus.cc:1275] T 8a996de6f02a4723aef91074b3d56714 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:11:08.404537 22952 raft_consensus.cc:1275] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:11:08.411060 30394 consensus_queue.cc:1048] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 13:11:08.412583 30368 consensus_queue.cc:1048] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 13:11:08.429499 30459 raft_consensus.cc:2955] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 19 LEADER]: Committing config change with OpId 19.15: config changed from index 14 to 15, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:11:08.437129 22952 raft_consensus.cc:2955] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Committing config change with OpId 19.15: config changed from index 14 to 15, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:11:08.445747 23621 raft_consensus.cc:2955] T 8a996de6f02a4723aef91074b3d56714 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Committing config change with OpId 19.15: config changed from index 14 to 15, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:11:08.489852 30424 raft_consensus.cc:2955] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 LEADER]: Committing config change with OpId 20.15: config changed from index 14 to 15, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:11:08.517185 22762 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8a996de6f02a4723aef91074b3d56714 with cas_config_opid_index 14: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:08.520459 23621 raft_consensus.cc:2955] T ec7dd25342084943aacb4b9830f5f1ea P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Committing config change with OpId 20.15: config changed from index 14 to 15, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:11:08.527609 22770 catalog_manager.cc:5697] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: config changed from index 14 to 15, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New cstate: current_term: 19 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:11:08.626322 23029 raft_consensus.cc:2955] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Committing config change with OpId 20.15: config changed from index 14 to 15, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New config: { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:11:08.632494 22775 catalog_manager.cc:5697] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: config changed from index 14 to 15, 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4) changed from NON_VOTER to VOTER. New cstate: current_term: 20 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 13:11:08.649097 22952 consensus_queue.cc:237] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:08.664527 23029 raft_consensus.cc:1275] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:11:08.664714 23624 raft_consensus.cc:1275] T ec7dd25342084943aacb4b9830f5f1ea P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Refusing update from remote peer aed3f089ec8b49af9c8cf5cdee76e8e7: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 20 index: 16. (index mismatch)
I20260709 13:11:08.669641 30361 consensus_queue.cc:1048] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 13:11:08.685817 22764 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.21.63.1:33929, user_credentials={real_user=slave}} blocked reactor thread for 62553.5us
I20260709 13:11:08.686877 30361 consensus_queue.cc:1048] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 13:11:08.686324 30433 raft_consensus.cc:2955] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 LEADER]: Committing config change with OpId 20.16: config changed from index 15 to 16, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:11:08.699877 23617 raft_consensus.cc:2955] T ec7dd25342084943aacb4b9830f5f1ea P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Committing config change with OpId 20.16: config changed from index 15 to 16, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
W20260709 13:11:08.706166 22764 catalog_manager.cc:4755] TS 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 8a996de6f02a4723aef91074b3d56714: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:08.709709 22762 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ec7dd25342084943aacb4b9830f5f1ea with cas_config_opid_index 15: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 13:11:08.715445 22770 catalog_manager.cc:5697] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 reported cstate change: config changed from index 15 to 16, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New cstate: current_term: 20 leader_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:08.731832 23029 raft_consensus.cc:2955] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Committing config change with OpId 20.16: config changed from index 15 to 16, VOTER 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) evicted. New config: { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } } }
I20260709 13:11:08.824602 30368 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:08.825202 30368 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:11:08.834939 22994 leader_election.cc:336] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:08.837984 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" is_pre_election: true
I20260709 13:11:08.838649 22952 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 22.
I20260709 13:11:08.840024 22992 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:11:08.841002 30458 raft_consensus.cc:2804] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 13:11:08.841408 30458 raft_consensus.cc:493] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:08.841746 30458 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:11:08.844841 30368 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:11:08.866317 30458 raft_consensus.cc:515] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:08.908470 30458 leader_election.cc:290] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 election: Requested vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:11:08.911027 22952 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2af6865189dc47369064cfad3fd137e1" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7"
I20260709 13:11:08.913309 22952 raft_consensus.cc:3060] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Advancing to term 23
I20260709 13:11:08.923712 22952 raft_consensus.cc:2468] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 23.
W20260709 13:11:08.912626 22994 leader_election.cc:336] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:08.925360 22992 leader_election.cc:304] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f
I20260709 13:11:08.926143 30368 raft_consensus.cc:2804] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 13:11:08.927538 30368 raft_consensus.cc:697] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 23 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:11:08.928606 30368 consensus_queue.cc:237] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:11:08.935604 21756 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 22901)
W20260709 13:11:08.978490 22994 consensus_peers.cc:597] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:09.029582 22775 catalog_manager.cc:5697] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 21 to 23, leader changed from 0a407504ed1741bf89a276558adf4c0f (127.21.63.1) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 23 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } }
W20260709 13:11:09.035145 23590 consensus_peers.cc:597] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:09.118666 22920 consensus_peers.cc:597] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:09.214579 23590 consensus_peers.cc:597] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:09.234267 22994 consensus_peers.cc:597] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:09.240338 22920 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111) [suppressed 428 similar messages]
W20260709 13:11:09.252751 23590 consensus_peers.cc:597] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:09.335901 22994 consensus_peers.cc:597] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:09.338749 23590 consensus_peers.cc:597] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:09.340430 22920 consensus_peers.cc:597] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:09.343362 23590 consensus_peers.cc:597] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:09.395551 22920 consensus_peers.cc:597] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:11:09.489916 22952 raft_consensus.cc:1275] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 13:11:09.492072 30406 consensus_queue.cc:1048] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:11:09.503340 22920 consensus_peers.cc:597] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:09.551134 22994 consensus_peers.cc:597] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:09.578132 22920 consensus_peers.cc:597] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:09.590742 22994 consensus_peers.cc:597] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:09.640098 22994 consensus_peers.cc:597] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:09.656507 22994 consensus_peers.cc:597] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:09.670753 22920 consensus_peers.cc:597] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:09.699738 22920 consensus_peers.cc:597] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:09.723229 23590 consensus_peers.cc:597] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:09.727254 23590 consensus_peers.cc:597] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:09.732077 22920 consensus_peers.cc:597] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:09.744359 22994 consensus_peers.cc:597] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:09.762990 23590 consensus_peers.cc:597] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:09.772285 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 50788.7us
W20260709 13:11:09.781237 23590 consensus_peers.cc:597] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:09.806130 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 62467us
W20260709 13:11:09.811563 22764 catalog_manager.cc:4755] TS 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 8a996de6f02a4723aef91074b3d56714: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:09.812851 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 62511us
W20260709 13:11:09.813603 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 74823.4us
W20260709 13:11:09.815270 23590 consensus_peers.cc:597] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:09.871270 22920 consensus_peers.cc:597] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:09.881492 22920 consensus_peers.cc:597] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:09.895596 22920 consensus_peers.cc:597] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:09.898092 23590 consensus_peers.cc:597] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:09.901966 22920 consensus_peers.cc:597] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:09.932277 23590 consensus_peers.cc:597] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:09.935259 22920 consensus_peers.cc:597] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:09.936105 21756 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 22901)
W20260709 13:11:09.949440 22994 consensus_peers.cc:597] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:09.955420 23590 consensus_peers.cc:597] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:10.084117 22920 consensus_peers.cc:597] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:10.100823 22994 consensus_peers.cc:597] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:10.119062 22994 consensus_peers.cc:597] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:10.141397 22994 consensus_peers.cc:597] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:10.178913 22920 consensus_peers.cc:597] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:10.233502 22994 consensus_peers.cc:597] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:10.313534 23590 consensus_peers.cc:597] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:10.388653 22994 consensus_peers.cc:597] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:10.589850 22994 consensus_peers.cc:597] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:10.792975 22920 consensus_peers.cc:597] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:10.936760 21756 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 22901)
W20260709 13:11:10.980741 22764 catalog_manager.cc:4755] TS 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 8a996de6f02a4723aef91074b3d56714: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:11.278051 23590 consensus_peers.cc:597] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:11.474117 22920 consensus_peers.cc:597] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:11.498977 22994 consensus_peers.cc:597] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:11.612442 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 67247.3us
W20260709 13:11:11.757790 23590 consensus_peers.cc:597] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:11.791805 22994 consensus_peers.cc:597] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:11.806493 22920 consensus_peers.cc:597] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:11.815349 22994 consensus_peers.cc:597] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:11.837815 23590 consensus_peers.cc:597] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:11.889053 23590 consensus_peers.cc:597] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:11.912853 22920 consensus_peers.cc:597] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:11.915850 22920 consensus_peers.cc:597] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:11.937261 21756 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 22901)
W20260709 13:11:12.013213 23590 consensus_peers.cc:597] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:12.047940 22920 consensus_peers.cc:597] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:12.048600 23590 consensus_peers.cc:597] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:12.057674 22994 consensus_peers.cc:597] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:12.102891 22920 consensus_peers.cc:597] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:12.117547 22920 consensus_peers.cc:597] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:12.135835 23590 consensus_peers.cc:597] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:12.149891 23590 consensus_peers.cc:597] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:12.153631 23590 consensus_peers.cc:597] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:12.171046 22920 consensus_peers.cc:597] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:12.189102 22920 consensus_peers.cc:597] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:12.192845 22994 consensus_peers.cc:597] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:12.235646 22920 consensus_peers.cc:597] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:12.288888 23590 consensus_peers.cc:597] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:12.291605 22994 consensus_peers.cc:597] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:12.300153 22920 consensus_peers.cc:597] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:12.316394 22994 consensus_peers.cc:597] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:12.341753 22994 consensus_peers.cc:597] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:12.342669 23590 consensus_peers.cc:597] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:12.371484 22920 consensus_peers.cc:597] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:12.430104 22920 consensus_peers.cc:597] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:12.460186 22994 consensus_peers.cc:597] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:12.478202 23590 consensus_peers.cc:597] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:12.520315 23590 consensus_peers.cc:597] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:12.536680 22994 consensus_peers.cc:597] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:12.553460 22994 consensus_peers.cc:597] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:12.667290 22994 consensus_peers.cc:597] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:12.690809 22920 consensus_peers.cc:597] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:12.714731 23590 consensus_peers.cc:597] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:12.751017 22920 consensus_peers.cc:597] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:12.949822 21756 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 22901)
W20260709 13:11:13.051240 22994 consensus_peers.cc:597] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:13.066756 22994 consensus_peers.cc:597] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:13.118572 22764 catalog_manager.cc:4755] TS 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet ec7dd25342084943aacb4b9830f5f1ea: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:13.191606 22994 consensus_peers.cc:597] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:13.210577 22920 consensus_peers.cc:597] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:13.699681 22994 consensus_peers.cc:597] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:13.891960 23590 consensus_peers.cc:597] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:13.950417 21756 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 22901)
W20260709 13:11:14.074213 22994 consensus_peers.cc:597] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:14.120460 22994 consensus_peers.cc:597] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:14.121578 23590 consensus_peers.cc:597] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:14.141677 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 86037.7us
W20260709 13:11:14.214886 22920 consensus_peers.cc:597] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:14.254712 22994 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111) [suppressed 448 similar messages]
W20260709 13:11:14.330348 23590 consensus_peers.cc:597] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:14.361079 23590 consensus_peers.cc:597] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:14.395170 23590 consensus_peers.cc:597] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:14.395936 22920 consensus_peers.cc:597] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:14.422595 22920 consensus_peers.cc:597] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:14.445055 22920 consensus_peers.cc:597] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:14.467329 22994 consensus_peers.cc:597] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:14.475993 23590 consensus_peers.cc:597] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:14.536087 22920 consensus_peers.cc:597] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:14.543058 22920 consensus_peers.cc:597] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:14.674449 22994 consensus_peers.cc:597] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:14.717870 23590 consensus_peers.cc:597] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:14.731242 23590 consensus_peers.cc:597] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:14.758495 22920 consensus_peers.cc:597] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:14.763725 22920 consensus_peers.cc:597] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:14.799650 23590 consensus_peers.cc:597] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:14.805430 22920 consensus_peers.cc:597] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:14.816393 22920 consensus_peers.cc:597] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:14.857439 23590 consensus_peers.cc:597] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:14.866204 22994 consensus_peers.cc:597] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:14.913157 22920 consensus_peers.cc:597] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:14.944747 23590 consensus_peers.cc:597] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:14.950922 21756 thread.cc:527] Waited for 9000ms trying to join with diag-logger (tid 22901)
W20260709 13:11:14.962544 22920 consensus_peers.cc:597] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:14.980337 22994 consensus_peers.cc:597] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:14.989403 22994 consensus_peers.cc:597] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:14.994633 23590 consensus_peers.cc:597] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:15.069959 22994 consensus_peers.cc:597] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:15.087280 23590 consensus_peers.cc:597] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:15.105984 22920 consensus_peers.cc:597] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:15.108561 22994 consensus_peers.cc:597] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:15.141467 22920 consensus_peers.cc:597] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:15.175176 22920 consensus_peers.cc:597] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:15.279541 22994 consensus_peers.cc:597] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:15.384960 23590 consensus_peers.cc:597] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:15.476202 22994 consensus_peers.cc:597] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:15.507135 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 64519.8us
W20260709 13:11:15.510036 22994 consensus_peers.cc:597] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:15.510792 22994 consensus_peers.cc:597] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:15.653599 22920 consensus_peers.cc:597] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:15.681284 22994 consensus_peers.cc:597] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:15.952157 21756 thread.cc:527] Waited for 10000ms trying to join with diag-logger (tid 22901)
W20260709 13:11:16.554596 23590 consensus_peers.cc:597] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:16.551713 22994 consensus_peers.cc:597] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:16.566244 22920 consensus_peers.cc:597] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:16.639106 22994 consensus_peers.cc:597] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:16.688329 22920 consensus_peers.cc:597] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:16.699996 23590 consensus_peers.cc:597] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:16.700842 22920 consensus_peers.cc:597] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:16.805146 22994 consensus_peers.cc:597] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:16.843355 22809 debug-util.cc:398] Leaking SignalData structure 0x7b080034c1a0 after lost signal to thread 21759
W20260709 13:11:16.844658 22809 debug-util.cc:398] Leaking SignalData structure 0x7b080034c3e0 after lost signal to thread 22812
W20260709 13:11:16.846105 22809 debug-util.cc:398] Leaking SignalData structure 0x7b08000354a0 after lost signal to thread 22979
W20260709 13:11:16.847275 22809 debug-util.cc:398] Leaking SignalData structure 0x7b080014eba0 after lost signal to thread 23053
W20260709 13:11:16.848481 22809 debug-util.cc:398] Leaking SignalData structure 0x7b0800145920 after lost signal to thread 23649
W20260709 13:11:16.892593 23590 consensus_peers.cc:597] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:16.952690 21756 thread.cc:527] Waited for 11000ms trying to join with diag-logger (tid 22901)
W20260709 13:11:16.973347 22920 consensus_peers.cc:597] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:16.996230 23590 consensus_peers.cc:597] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:17.030413 22994 consensus_peers.cc:597] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:17.039899 22920 consensus_peers.cc:597] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:17.092509 23590 consensus_peers.cc:597] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:17.144562 23590 consensus_peers.cc:597] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:17.148638 22920 consensus_peers.cc:597] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:17.165132 23590 consensus_peers.cc:597] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:17.199527 22920 consensus_peers.cc:597] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:17.200359 22994 consensus_peers.cc:597] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:17.235265 22920 consensus_peers.cc:597] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:17.270933 22920 consensus_peers.cc:597] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:17.317955 22920 consensus_peers.cc:597] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:17.331918 22764 catalog_manager.cc:4755] TS 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet ec7dd25342084943aacb4b9830f5f1ea: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:17.363610 22994 consensus_peers.cc:597] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:17.418838 22920 consensus_peers.cc:597] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:17.423069 22920 consensus_peers.cc:597] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:17.425828 23590 consensus_peers.cc:597] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:17.446983 22994 consensus_peers.cc:597] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:17.480046 23590 consensus_peers.cc:597] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:17.569566 22994 consensus_peers.cc:597] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:17.585824 23590 consensus_peers.cc:597] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:17.615823 23590 consensus_peers.cc:597] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:17.644968 22994 consensus_peers.cc:597] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:17.653743 22994 consensus_peers.cc:597] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:17.682358 22920 consensus_peers.cc:597] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:17.737725 22994 consensus_peers.cc:597] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:17.781385 23590 consensus_peers.cc:597] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:17.790807 23590 consensus_peers.cc:597] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:17.822032 23590 consensus_peers.cc:597] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:17.869284 22920 consensus_peers.cc:597] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:17.895527 22994 consensus_peers.cc:597] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:17.911310 22994 consensus_peers.cc:597] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:17.953199 21756 thread.cc:527] Waited for 12000ms trying to join with diag-logger (tid 22901)
W20260709 13:11:18.003759 22920 consensus_peers.cc:597] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:18.018885 22994 consensus_peers.cc:597] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:18.178454 22994 consensus_peers.cc:597] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:18.232836 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 71359.8us
W20260709 13:11:18.350847 22920 consensus_peers.cc:597] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:11:18.703011 21756 tablet_server.cc:196] TabletServer@127.21.63.1:0 shutdown complete.
W20260709 13:11:18.938100 22920 consensus_peers.cc:597] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:11:19.131999 23590 consensus_peers.cc:597] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:11:19.161913 22994 consensus_peers.cc:597] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:19.192806 22994 consensus_peers.cc:597] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:19.196837 23590 consensus_peers.cc:597] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:19.243086 23590 consensus_peers.cc:597] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:11:19.243651 22920 consensus_peers.cc:597] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:11:19.270586 22920 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111) [suppressed 443 similar messages]
W20260709 13:11:19.472708 23590 consensus_peers.cc:597] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:11:19.501704 22920 consensus_peers.cc:597] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:11:19.578258 22920 consensus_peers.cc:597] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:19.591027 22994 consensus_peers.cc:597] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:19.608491 22994 consensus_peers.cc:597] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:11:19.599951 23590 consensus_peers.cc:597] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:11:19.610289 23590 consensus_peers.cc:597] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:11:19.613070 22920 consensus_peers.cc:597] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:11:19.686756 23590 consensus_peers.cc:597] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:11:19.700285 22920 consensus_peers.cc:597] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:11:19.750150 22920 consensus_peers.cc:597] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:11:20.003499 22918 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 75098.7us
W20260709 13:11:20.050433 22919 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 52973.7us
W20260709 13:11:20.112547 23590 consensus_peers.cc:597] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:11:20.121451 23590 consensus_peers.cc:597] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:11:20.130780 22920 consensus_peers.cc:597] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:11:20.141748 22920 consensus_peers.cc:597] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:20.142529 22920 consensus_peers.cc:597] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:20.149111 22993 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:44043, user_credentials={real_user=slave}} blocked reactor thread for 92462.2us
W20260709 13:11:20.183221 22920 consensus_peers.cc:597] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:11:20.240167 23590 consensus_peers.cc:597] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:11:20.329594 22920 consensus_peers.cc:597] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:20.340281 22920 consensus_peers.cc:597] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:11:20.341542 22994 consensus_peers.cc:597] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:11:20.343541 22994 consensus_peers.cc:597] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:20.344240 22994 consensus_peers.cc:597] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:11:20.344888 22994 consensus_peers.cc:597] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:20.359548 22994 consensus_peers.cc:597] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:11:20.364316 22994 consensus_peers.cc:597] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:11:20.381397 22994 consensus_peers.cc:597] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:20.381986 22994 consensus_peers.cc:597] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:11:20.442446 22994 consensus_peers.cc:597] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:11:20.446637 21756 tablet_server.cc:179] TabletServer@127.21.63.2:0 shutting down...
W20260709 13:11:20.455489 22920 consensus_peers.cc:597] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:20.458096 22992 consensus_peers.cc:597] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:20.511268 22994 consensus_peers.cc:597] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:11:20.513175 23588 consensus_peers.cc:597] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:20.515352 22992 consensus_peers.cc:597] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:20.516232 22920 consensus_peers.cc:597] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:11:20.529307 23590 consensus_peers.cc:597] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:20.554979 23588 consensus_peers.cc:597] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:20.565239 23590 consensus_peers.cc:597] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:11:20.597396 22994 consensus_peers.cc:597] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:11:20.600656 22992 consensus_peers.cc:597] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:20.603020 22992 consensus_peers.cc:597] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:20.610790 23590 consensus_peers.cc:597] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:11:20.625195 23590 consensus_peers.cc:597] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:20.631047 22992 consensus_peers.cc:597] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:20.635749 23588 consensus_peers.cc:597] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:20.645118 23588 consensus_peers.cc:597] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:20.656818 22992 consensus_peers.cc:597] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:20.657979 23588 consensus_peers.cc:597] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:20.665640 22992 consensus_peers.cc:597] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:20.671276 23588 consensus_peers.cc:597] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:20.702435 22992 consensus_peers.cc:597] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:20.757321 21756 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:11:20.757382 22992 consensus_peers.cc:597] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:20.764011 22992 consensus_peers.cc:597] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:20.764822 21756 tablet_replica.cc:333] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:20.766202 21756 raft_consensus.cc:2243] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:20.767374 21756 raft_consensus.cc:2272] T 0fa29d11409a4d209411116a7ef8af04 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:20.772578 21756 tablet_replica.cc:333] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:20.777014 21756 raft_consensus.cc:2243] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:11:20.778615 21756 raft_consensus.cc:2272] T f11454b577ab42d7bfe7c09154b00817 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:20.794598 21756 tablet_replica.cc:333] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:20.795768 21756 raft_consensus.cc:2243] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Raft consensus shutting down.
W20260709 13:11:20.802802 22992 consensus_peers.cc:597] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:20.803759 23588 consensus_peers.cc:597] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:20.816227 22992 consensus_peers.cc:597] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:20.818980 23588 consensus_peers.cc:597] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:20.822316 21756 raft_consensus.cc:2272] T 6eba8b8b84ca4a9aa5d9eaca94670686 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:20.826011 21756 tablet_replica.cc:333] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:20.827735 21756 raft_consensus.cc:2243] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Raft consensus shutting down.
W20260709 13:11:20.829639 23588 consensus_peers.cc:597] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:20.829780 21756 raft_consensus.cc:2272] T 3f1c9e3f9b244b0695775e91f28d67d7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:20.834745 21756 tablet_replica.cc:333] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:20.836639 21756 raft_consensus.cc:2243] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:20.837809 21756 raft_consensus.cc:2272] T 61adbc0c57034ac28a0b50af8a59ed07 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:20.841308 21756 tablet_replica.cc:333] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
W20260709 13:11:20.841548 22992 consensus_peers.cc:597] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:20.843487 21756 raft_consensus.cc:2243] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:11:20.845037 21756 raft_consensus.cc:2272] T ced80ef275e04ef8bf4c071883b8284e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:20.848825 21756 tablet_replica.cc:333] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:20.850030 21756 raft_consensus.cc:2243] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:20.854085 21756 raft_consensus.cc:2272] T b65d9dbf257d434d86d4656dabfd1b70 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:20.854400 22992 consensus_peers.cc:597] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:20.859135 21756 tablet_replica.cc:333] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:20.861048 21756 raft_consensus.cc:2243] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:11:20.862567 21756 raft_consensus.cc:2272] T 616e98b5f7534394ae8790b08aec10e6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:20.868819 21756 tablet_replica.cc:333] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:20.871232 21756 raft_consensus.cc:2243] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:20.872349 21756 raft_consensus.cc:2272] T 4c0bea78fb2a40ef90cff914c5c6e250 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:20.880285 21756 tablet_replica.cc:333] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:20.881642 21756 raft_consensus.cc:2243] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:11:20.883503 21756 raft_consensus.cc:2272] T edc3e3e3969d4c17b76da48ab861a54b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:20.887336 21756 tablet_replica.cc:333] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:20.888258 21756 raft_consensus.cc:2243] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:11:20.889931 21756 raft_consensus.cc:2272] T f603724630e44ef78014e7719f096cb3 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:20.895231 21756 tablet_replica.cc:333] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:20.896790 21756 raft_consensus.cc:2243] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:20.897542 21756 raft_consensus.cc:2272] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 17 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:20.898497 23588 consensus_peers.cc:597] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:20.900532 21756 tablet_replica.cc:333] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:20.901779 21756 raft_consensus.cc:2243] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:11:20.903337 21756 raft_consensus.cc:2272] T 421f71937d5e4cef9084068ce92d12b7 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:20.906509 21756 tablet_replica.cc:333] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:20.907717 21756 raft_consensus.cc:2243] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:11:20.910149 21756 raft_consensus.cc:2272] T 7cd8492f63fb4193a47cf9fbc574ef58 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:20.910732 23588 consensus_peers.cc:597] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:20.920431 21756 tablet_replica.cc:333] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:20.921520 21756 raft_consensus.cc:2243] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:11:20.923029 21756 raft_consensus.cc:2272] T 7ca8b9a08f4146c386bc06d03c497038 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:20.930275 21756 tablet_replica.cc:333] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:20.931311 21756 raft_consensus.cc:2243] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:11:20.932716 21756 raft_consensus.cc:2272] T f511a9ca0d9b4f54833aa7d0c092ccc1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:20.936349 21756 tablet_replica.cc:333] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:20.938051 21756 raft_consensus.cc:2243] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:11:20.939568 21756 raft_consensus.cc:2272] T 698093275c264d4d99fb8c1a49e3b28f P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:20.943060 21756 tablet_replica.cc:333] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:20.943958 21756 raft_consensus.cc:2243] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:11:20.946849 21756 raft_consensus.cc:2272] T 612dc49c81064f4faa79f65fcf1cfc02 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:20.954682 21756 tablet_replica.cc:333] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:20.957515 21756 raft_consensus.cc:2243] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:20.959537 21756 raft_consensus.cc:2272] T c26347a647b240d7a3bc5cd449b7993b P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:20.963243 21756 tablet_replica.cc:333] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:20.964155 21756 raft_consensus.cc:2243] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:20.965286 21756 raft_consensus.cc:2272] T 3047c357228240a2aa4d84e8fbf437fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:20.968568 21756 tablet_replica.cc:333] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:20.969515 21756 raft_consensus.cc:2243] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:11:20.970851 21756 raft_consensus.cc:2272] T ec7dd25342084943aacb4b9830f5f1ea P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:20.973454 23588 consensus_peers.cc:597] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:20.974679 21756 tablet_replica.cc:333] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:20.975636 21756 raft_consensus.cc:2243] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:20.976428 21756 raft_consensus.cc:2272] T 579096ba88474cfbbe40e8bf769879ec P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:20.981616 21756 tablet_replica.cc:333] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:20.983069 21756 raft_consensus.cc:2243] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:20.989176 21756 raft_consensus.cc:2272] T ec334666b7964c969125af95d9c797ed P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:21.002354 21756 tablet_replica.cc:333] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:21.003317 21756 raft_consensus.cc:2243] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:21.004043 21756 raft_consensus.cc:2272] T 0f076a0e47a14c2b933d3058063c2bcd P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:21.027519 21756 tablet_replica.cc:333] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:21.028461 21756 raft_consensus.cc:2243] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:11:21.030680 21756 raft_consensus.cc:2272] T 3ec25afe61174e01a75a21ab33afe2e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:21.034281 21756 tablet_replica.cc:333] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:21.051163 21756 raft_consensus.cc:2243] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:11:21.054014 21756 raft_consensus.cc:2272] T c3d2a83be5024e50818057526cbaf822 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:21.056901 21756 tablet_replica.cc:333] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:21.057940 21756 raft_consensus.cc:2243] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:21.058955 21756 raft_consensus.cc:2272] T 85d33f0fd80a49b8bd8cb73571ed3191 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:21.062000 21756 tablet_replica.cc:333] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:21.062911 21756 raft_consensus.cc:2243] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:21.064009 21756 raft_consensus.cc:2272] T 1120807254f5444997e16f6a7feece4d P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:21.066988 21756 tablet_replica.cc:333] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:21.073557 21756 raft_consensus.cc:2243] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:21.075397 21756 raft_consensus.cc:2272] T 8a996de6f02a4723aef91074b3d56714 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:21.081604 21756 tablet_replica.cc:333] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:21.082548 21756 raft_consensus.cc:2243] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:11:21.083875 21756 raft_consensus.cc:2272] T 8217046fd4e14cc4a0194da4f6ad3887 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:21.087155 21756 tablet_replica.cc:333] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:21.087838 21756 raft_consensus.cc:2243] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:21.088759 21756 raft_consensus.cc:2272] T 91c88873d54149c58a20f4cf6194e5fc P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:21.091307 21756 tablet_replica.cc:333] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:21.092038 21756 raft_consensus.cc:2243] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:11:21.095927 21756 raft_consensus.cc:2272] T 778fe3ade80844c48f8408313fc86a26 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:21.098656 21756 tablet_replica.cc:333] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:21.100170 21756 raft_consensus.cc:2243] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:21.102864 21756 raft_consensus.cc:2272] T f3b08335f3564868a8160dc1b5ade501 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:21.106288 21756 tablet_replica.cc:333] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:21.112654 21756 raft_consensus.cc:2243] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:21.113354 21756 raft_consensus.cc:2272] T 2af6865189dc47369064cfad3fd137e1 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:21.116171 21756 tablet_replica.cc:333] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:21.116926 21756 raft_consensus.cc:2243] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:21.117820 21756 raft_consensus.cc:2272] T abd5e7d435ca4afd9727e37b2ab31589 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:21.121155 21756 tablet_replica.cc:333] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:21.121868 21756 raft_consensus.cc:2243] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:11:21.123306 21756 raft_consensus.cc:2272] T 104e2273f37843cbb5245981c817dfe0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:21.126291 21756 tablet_replica.cc:333] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:21.126992 21756 raft_consensus.cc:2243] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:11:21.141733 21756 raft_consensus.cc:2272] T a0b6723ad2db4fccb7a1c219ff04e99a P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:21.156239 21756 tablet_replica.cc:333] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:21.157065 21756 raft_consensus.cc:2243] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:11:21.158218 21756 raft_consensus.cc:2272] T b7633b7b4c334b039db57d77774df345 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:21.160962 21756 tablet_replica.cc:333] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:21.161671 21756 raft_consensus.cc:2243] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:21.162971 21756 raft_consensus.cc:2272] T b7e4ee754b7a4408b5fc55259d431f39 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:21.176153 21756 tablet_replica.cc:333] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:21.176888 21756 raft_consensus.cc:2243] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:11:21.179656 21756 raft_consensus.cc:2272] T cbf52b91ec054ac6a2f8503462b1b7a6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:21.186301 21756 tablet_replica.cc:333] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:21.187019 21756 raft_consensus.cc:2243] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:21.187605 21756 raft_consensus.cc:2272] T 67e7fdd613434ffaacb8a030d4693490 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:21.190383 21756 tablet_replica.cc:333] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:21.191061 21756 raft_consensus.cc:2243] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:21.192133 21756 raft_consensus.cc:2272] T b41a09964104486c8e50b539ec2b16f6 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:21.202244 21756 tablet_replica.cc:333] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:21.202919 21756 raft_consensus.cc:2243] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:21.217923 21756 raft_consensus.cc:2272] T a424231ef7b4470bbd575564909a365e P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:21.221629 21756 tablet_replica.cc:333] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:21.222335 21756 raft_consensus.cc:2243] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:21.223008 21756 raft_consensus.cc:2272] T 0c4ecd16880043f4a75006a3da4717bf P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:21.225890 21756 tablet_replica.cc:333] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:21.237174 21756 raft_consensus.cc:2243] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:21.238687 21756 raft_consensus.cc:2272] T ac5cc94926a742e6a0c1340ec33c0ff0 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:21.241623 21756 tablet_replica.cc:333] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7: stopping tablet replica
I20260709 13:11:21.242201 21756 raft_consensus.cc:2243] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:21.242911 21756 raft_consensus.cc:2272] T 5ab4700268fc4c67b9930087c5da71c8 P aed3f089ec8b49af9c8cf5cdee76e8e7 [term 24 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:21.519867 23590 consensus_peers.cc:597] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:11:21.636236 22994 consensus_peers.cc:597] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:11:21.645412 23590 consensus_peers.cc:597] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:11:21.682585 23590 consensus_peers.cc:597] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:11:21.730297 22994 consensus_peers.cc:597] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 13:11:21.812582 23590 consensus_peers.cc:597] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:11:21.813143 23590 consensus_peers.cc:597] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:11:21.913575 30520 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:21.914363 30520 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:21.929560 30520 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:21.935566 22994 leader_election.cc:336] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:21.956931 23590 consensus_peers.cc:597] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:11:21.976593 22994 consensus_peers.cc:597] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:11:21.977725 22992 leader_election.cc:336] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:21.980432 22992 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:21.991927 30520 raft_consensus.cc:2749] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:11:22.035396 30536 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:22.036172 30536 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:22.040441 30520 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:22.045033 30520 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:22.069643 30520 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:11:22.080681 30524 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:22.097460 30543 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:22.104966 30543 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:22.107582 30543 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:11:22.101970 30530 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:22.103637 30546 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:22.103816 30536 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:11:22.110299 30530 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:11:22.117292 22994 leader_election.cc:336] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:22.119247 30530 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:11:22.122493 22994 leader_election.cc:336] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:22.135982 30543 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:22.126271 30524 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:11:22.139844 22992 leader_election.cc:336] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:22.140532 22992 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:22.138345 30543 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:22.141644 30536 raft_consensus.cc:2749] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:11:22.142833 30543 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:22.142863 22992 leader_election.cc:336] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:22.143652 22992 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:11:22.144734 22992 leader_election.cc:336] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:22.127914 30546 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:22.146430 30543 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:22.147130 30543 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:22.148245 30546 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:11:22.149467 30543 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:11:22.152005 30543 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:22.152545 30543 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:22.155244 30536 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:22.155602 30524 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:11:22.155772 30536 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:22.157783 30536 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:11:22.159919 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:11:22.144780 30544 raft_consensus.cc:2749] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:11:22.165050 30547 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:22.165607 30547 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:22.168252 30547 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:11:22.170747 30543 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:22.171294 23590 leader_election.cc:336] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:22.175845 30543 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:22.176384 30543 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:22.178395 30543 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:22.178788 23590 leader_election.cc:336] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:22.179267 22994 consensus_peers.cc:597] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:11:22.182658 22994 leader_election.cc:336] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:22.183173 22994 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:11:22.187613 23588 leader_election.cc:336] T ec7dd25342084943aacb4b9830f5f1ea P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:22.188221 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:11:22.192253 30546 raft_consensus.cc:2749] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:11:22.198565 23588 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:11:22.201972 30548 raft_consensus.cc:2749] T ec7dd25342084943aacb4b9830f5f1ea P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:11:22.207870 23588 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:11:22.208767 30548 raft_consensus.cc:2749] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:11:22.192324 22994 leader_election.cc:336] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:22.211639 22992 leader_election.cc:336] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
W20260709 13:11:22.211695 22994 leader_election.cc:336] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:22.212394 22994 leader_election.cc:336] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:22.212548 22992 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:11:22.213366 22994 leader_election.cc:336] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:22.213304 30546 raft_consensus.cc:2749] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:11:22.214071 22994 leader_election.cc:336] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:22.218703 22994 leader_election.cc:336] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:22.220912 23588 leader_election.cc:336] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
W20260709 13:11:22.213435 22992 leader_election.cc:336] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:22.226629 30546 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:22.228037 22992 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:22.228266 30546 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:22.228891 30543 raft_consensus.cc:2749] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:11:22.229091 22992 leader_election.cc:336] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:22.229606 22992 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:22.230311 30543 raft_consensus.cc:2749] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:11:22.227304 23590 consensus_peers.cc:597] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:11:22.231719 30546 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:11:22.234617 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:11:22.235440 23624 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 16.
W20260709 13:11:22.239413 22992 leader_election.cc:336] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:22.239848 22992 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:22.240602 30546 raft_consensus.cc:2749] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:11:22.241318 23588 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:22.244469 30548 raft_consensus.cc:2749] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:11:22.245564 23588 leader_election.cc:336] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
W20260709 13:11:22.240765 22992 leader_election.cc:336] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:22.251078 22992 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:22.251945 30546 raft_consensus.cc:2749] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:11:22.256007 30546 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:22.256543 30546 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:11:22.258251 22994 leader_election.cc:336] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:22.258648 30546 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:11:22.262084 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:11:22.262805 23624 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 20.
I20260709 13:11:22.264688 22993 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:11:22.264827 30524 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:22.265385 30524 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:22.265734 30543 raft_consensus.cc:2804] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:11:22.266244 30543 raft_consensus.cc:493] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:22.266650 30543 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:11:22.267819 30524 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:11:22.277612 23588 leader_election.cc:336] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:22.279592 30524 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:22.283727 30524 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:22.286057 30524 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:22.290102 23590 leader_election.cc:336] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:22.282254 22994 consensus_peers.cc:597] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:11:22.283077 22992 leader_election.cc:336] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:22.293781 22992 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:22.294658 30544 raft_consensus.cc:2749] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:11:22.296339 22992 leader_election.cc:336] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:22.305274 22992 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:22.308486 30544 raft_consensus.cc:2749] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:11:22.307624 23588 leader_election.cc:336] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:22.310823 23588 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:22.312348 30524 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:22.314070 30524 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:22.316254 30524 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:11:22.317121 30544 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:22.317673 30544 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:22.319945 30544 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:11:22.321336 30524 raft_consensus.cc:2749] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:11:22.321570 23621 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:11:22.322283 23621 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 20.
I20260709 13:11:22.299649 30543 raft_consensus.cc:515] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:22.323787 22993 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:11:22.325562 30544 raft_consensus.cc:2804] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 13:11:22.325956 30544 raft_consensus.cc:493] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:22.326363 30544 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:11:22.327870 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:11:22.332353 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec7dd25342084943aacb4b9830f5f1ea" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:11:22.332406 30543 leader_election.cc:290] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:11:22.332985 23624 raft_consensus.cc:3060] T ec7dd25342084943aacb4b9830f5f1ea P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Advancing to term 21
W20260709 13:11:22.339568 22992 leader_election.cc:336] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:22.342796 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
W20260709 13:11:22.344372 23588 leader_election.cc:336] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:22.345628 23588 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:11:22.346756 30548 raft_consensus.cc:2749] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 13:11:22.350376 30548 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:22.350860 30548 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:11:22.352119 22992 leader_election.cc:336] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:22.353029 30548 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:11:22.357743 23624 raft_consensus.cc:2468] T ec7dd25342084943aacb4b9830f5f1ea P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 21.
W20260709 13:11:22.358723 23588 leader_election.cc:336] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
W20260709 13:11:22.359350 22992 leader_election.cc:336] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:22.366230 22993 leader_election.cc:304] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:22.367408 30543 raft_consensus.cc:2804] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:11:22.368157 30543 raft_consensus.cc:697] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 21 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:11:22.369521 30543 consensus_queue.cc:237] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:22.375728 30546 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:22.376309 30546 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:22.378706 30546 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:11:22.383916 23024 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:11:22.384640 23024 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 16.
I20260709 13:11:22.387323 23588 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:22.388185 30548 raft_consensus.cc:2804] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 13:11:22.388775 30548 raft_consensus.cc:493] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:22.389236 30548 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Advancing to term 17
I20260709 13:11:22.388470 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
W20260709 13:11:22.398272 22992 leader_election.cc:336] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:22.362040 30544 raft_consensus.cc:515] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:22.401455 23021 raft_consensus.cc:2393] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5b346767a89745fa82928a5dac9cc0b2 in current term 21: Already voted for candidate f3c799b1249e40119d2a5864231c6c75 in this term.
I20260709 13:11:22.403756 30548 raft_consensus.cc:515] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:22.405303 23624 raft_consensus.cc:2393] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f3c799b1249e40119d2a5864231c6c75 in current term 17: Already voted for candidate 5b346767a89745fa82928a5dac9cc0b2 in this term.
I20260709 13:11:22.407677 22993 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 5b346767a89745fa82928a5dac9cc0b2, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:22.408697 30546 raft_consensus.cc:3060] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Advancing to term 17
W20260709 13:11:22.412760 23588 leader_election.cc:336] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:22.416204 30546 raft_consensus.cc:2749] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:11:22.441380 22770 catalog_manager.cc:5697] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 20 to 21, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 21 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 16 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:22.443923 30548 leader_election.cc:290] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:11:22.451702 30520 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:22.452313 30520 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:22.455366 30520 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:11:22.454355 30543 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
W20260709 13:11:22.458070 22994 leader_election.cc:336] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:22.457772 30543 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:22.466933 30543 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:11:22.478555 23588 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:11:22.479872 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:11:22.480654 23617 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 18.
I20260709 13:11:22.481763 30524 raft_consensus.cc:3060] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Advancing to term 21
I20260709 13:11:22.482884 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a0b6723ad2db4fccb7a1c219ff04e99a" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 21 candidate_status { last_received { term: 20 index: 16 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:11:22.490970 22993 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:11:22.492130 30520 raft_consensus.cc:2804] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 13:11:22.492574 30520 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Starting leader election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:22.493016 30520 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:11:22.497449 30524 raft_consensus.cc:2749] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:11:22.501847 21756 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 22976)
I20260709 13:11:22.511281 23617 raft_consensus.cc:2468] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 21.
I20260709 13:11:22.513283 30544 leader_election.cc:290] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:11:22.526196 22992 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:33309, user_credentials={real_user=slave}} blocked reactor thread for 50731.7us
I20260709 13:11:22.530920 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ca8b9a08f4146c386bc06d03c497038" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75"
I20260709 13:11:22.541666 30520 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:11:22.552256 22994 consensus_peers.cc:597] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:11:22.553750 23021 raft_consensus.cc:2468] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 17.
I20260709 13:11:22.568078 22993 leader_election.cc:304] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:11:22.569131 30546 raft_consensus.cc:2804] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 13:11:22.569905 30546 raft_consensus.cc:697] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 21 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:11:22.571034 30546 consensus_queue.cc:237] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 20.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:22.584026 30520 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Requested vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
I20260709 13:11:22.585884 23588 leader_election.cc:304] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:22.587370 30524 raft_consensus.cc:2804] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 13:11:22.587845 30524 raft_consensus.cc:697] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 LEADER]: Becoming Leader. State: Replica: 5b346767a89745fa82928a5dac9cc0b2, State: Running, Role: LEADER
I20260709 13:11:22.588769 30524 consensus_queue.cc:237] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:11:22.593667 22992 leader_election.cc:336] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:22.594040 22992 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:11:22.594815 22992 leader_election.cc:336] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
W20260709 13:11:22.595469 22992 leader_election.cc:336] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:22.596043 30544 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:22.596527 30544 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:22.598240 30548 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:22.598904 30544 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:11:22.598762 30548 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:22.601171 23624 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8217046fd4e14cc4a0194da4f6ad3887" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2"
I20260709 13:11:22.601745 23624 raft_consensus.cc:3060] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Advancing to term 19
I20260709 13:11:22.606565 30548 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:22.610073 22994 leader_election.cc:336] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:22.604341 30520 raft_consensus.cc:2749] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:11:22.619964 22994 consensus_peers.cc:597] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:11:22.621481 23588 leader_election.cc:336] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:22.618840 23624 raft_consensus.cc:2468] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 19.
I20260709 13:11:22.622722 22993 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 
I20260709 13:11:22.625379 30544 raft_consensus.cc:2804] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 13:11:22.626080 30544 raft_consensus.cc:697] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 19 LEADER]: Becoming Leader. State: Replica: f3c799b1249e40119d2a5864231c6c75, State: Running, Role: LEADER
I20260709 13:11:22.627058 30544 consensus_queue.cc:237] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:22.624168 22775 catalog_manager.cc:5697] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 reported cstate change: term changed from 16 to 17, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4). New cstate: current_term: 17 leader_uuid: "5b346767a89745fa82928a5dac9cc0b2" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 13:11:22.628244 23590 leader_election.cc:336] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:22.631021 23590 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:22.631726 30548 raft_consensus.cc:2749] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:11:22.634784 22992 leader_election.cc:336] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 19 election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
W20260709 13:11:22.636554 22994 consensus_peers.cc:597] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:11:22.645437 22992 leader_election.cc:336] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:22.645872 22992 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:11:22.649796 22994 consensus_peers.cc:597] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:11:22.650885 30546 raft_consensus.cc:2749] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:11:22.680568 22994 consensus_peers.cc:597] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:11:22.695679 23590 consensus_peers.cc:597] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:11:22.745992 22994 consensus_peers.cc:597] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:11:22.783651 23617 raft_consensus.cc:1275] T ec7dd25342084943aacb4b9830f5f1ea P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:11:22.785588 30536 consensus_queue.cc:1048] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 13:11:22.791360 22770 catalog_manager.cc:5697] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 20 to 21, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 21 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:22.800884 22770 catalog_manager.cc:5697] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 reported cstate change: term changed from 18 to 19, leader changed from aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2) to f3c799b1249e40119d2a5864231c6c75 (127.21.63.3). New cstate: current_term: 19 leader_uuid: "f3c799b1249e40119d2a5864231c6c75" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260709 13:11:22.867519 22992 consensus_peers.cc:597] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:22.922395 23590 consensus_peers.cc:597] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:11:22.923619 22994 consensus_peers.cc:597] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:11:22.925757 22992 consensus_peers.cc:597] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:22.943822 23590 consensus_peers.cc:597] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:11:22.985894 23590 consensus_peers.cc:597] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:11:23.002225 23590 consensus_peers.cc:597] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:11:23.041843 22994 consensus_peers.cc:597] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:11:23.043411 23588 consensus_peers.cc:597] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:23.044917 23024 raft_consensus.cc:1275] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Refusing update from remote peer 5b346767a89745fa82928a5dac9cc0b2: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 13:11:23.057837 23590 consensus_peers.cc:597] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:11:23.063983 23588 consensus_peers.cc:597] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:23.070329 22992 consensus_peers.cc:597] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:23.091229 30548 consensus_queue.cc:1048] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 13:11:23.096051 22994 consensus_peers.cc:597] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:11:23.099184 22992 consensus_peers.cc:597] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:23.106753 23588 consensus_peers.cc:597] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:23.118008 23588 consensus_peers.cc:597] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:23.140202 23621 raft_consensus.cc:1275] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 13:11:23.143002 30520 consensus_queue.cc:1048] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 13:11:23.150784 22992 consensus_peers.cc:597] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:23.156538 23588 consensus_peers.cc:597] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:23.157153 23588 consensus_peers.cc:597] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:23.196333 23590 consensus_peers.cc:597] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:11:23.204871 22992 consensus_peers.cc:597] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:23.228698 23621 raft_consensus.cc:1275] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Refusing update from remote peer f3c799b1249e40119d2a5864231c6c75: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 13:11:23.230172 30536 consensus_queue.cc:1048] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 13:11:23.243072 22994 consensus_peers.cc:597] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 13:11:23.245155 22992 consensus_peers.cc:597] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:23.257613 22992 consensus_peers.cc:597] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:23.286891 23588 consensus_peers.cc:597] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:23.292035 22992 consensus_peers.cc:597] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:23.333616 22992 consensus_peers.cc:597] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:23.348919 23588 consensus_peers.cc:597] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:23.349874 22992 consensus_peers.cc:597] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:23.381202 23588 consensus_peers.cc:597] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:23.401492 22992 consensus_peers.cc:597] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:23.427671 23588 consensus_peers.cc:597] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:23.459946 23588 consensus_peers.cc:597] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:23.463200 22992 consensus_peers.cc:597] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:23.481647 22992 consensus_peers.cc:597] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:23.502107 22992 consensus_peers.cc:597] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:23.502424 21756 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 22976)
W20260709 13:11:23.541226 22992 consensus_peers.cc:597] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:23.541843 22992 consensus_peers.cc:597] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:23.696455 30536 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:23.697176 30536 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:11:23.730896 22992 leader_election.cc:336] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:23.750598 30520 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:23.751189 30520 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:23.757886 30520 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:11:23.771452 30536 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:23.809039 23588 consensus_peers.cc:597] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:23.829329 22992 leader_election.cc:336] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
W20260709 13:11:23.856333 22994 leader_election.cc:336] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:23.873629 22994 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:23.877820 30536 raft_consensus.cc:2749] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:11:23.879266 22994 leader_election.cc:336] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:23.881453 22994 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:23.884424 30543 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:23.885512 30543 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:23.888494 30543 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:11:23.884128 30536 raft_consensus.cc:2749] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:11:23.912966 22992 leader_election.cc:336] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
W20260709 13:11:23.922887 23588 consensus_peers.cc:597] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:23.930658 22994 leader_election.cc:336] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:23.938381 22994 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:23.940718 30536 raft_consensus.cc:2749] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:11:24.058157 30543 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:24.058749 30543 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:24.063957 30543 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:24.067070 22994 leader_election.cc:336] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:24.068897 22992 leader_election.cc:336] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:24.069365 22992 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:24.070148 30543 raft_consensus.cc:2749] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:11:24.143117 30566 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:24.143726 30566 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:11:24.143657 22994 consensus_peers.cc:597] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:11:24.150180 30566 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:11:24.150967 22994 consensus_peers.cc:597] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:11:24.154130 23025 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:11:24.154950 23025 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 16.
W20260709 13:11:24.159318 23588 leader_election.cc:336] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
W20260709 13:11:24.164026 23590 leader_election.cc:336] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:24.167316 23590 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:11:24.169917 22994 consensus_peers.cc:597] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 13:11:24.170819 30543 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:24.171375 30543 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:24.171754 30545 raft_consensus.cc:2749] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:11:24.173764 30543 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:24.183421 22992 leader_election.cc:336] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:24.184785 30543 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:24.185333 30543 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:11:24.186412 22994 leader_election.cc:336] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:24.186987 22994 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:24.187706 30543 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:11:24.187737 30536 raft_consensus.cc:2749] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:11:24.196424 22992 leader_election.cc:336] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
W20260709 13:11:24.217288 22994 leader_election.cc:336] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:24.217770 22994 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:24.221012 30543 raft_consensus.cc:2749] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:11:24.231345 30536 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:24.231987 30536 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:24.234373 30536 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:24.255710 22992 leader_election.cc:336] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:24.256265 30536 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:24.257171 30536 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:24.259325 30536 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:24.260870 22994 leader_election.cc:336] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:24.261950 22994 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:24.267099 30536 raft_consensus.cc:2749] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:11:24.272102 23588 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111) [suppressed 602 similar messages]
W20260709 13:11:24.275622 23590 consensus_peers.cc:597] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:11:24.278802 23590 consensus_peers.cc:597] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:11:24.284358 22992 leader_election.cc:336] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
W20260709 13:11:24.289269 22994 leader_election.cc:336] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:24.290184 22994 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:24.292833 30536 raft_consensus.cc:2749] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:11:24.307015 30543 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:24.307770 30566 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:24.307535 30543 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:24.308284 30566 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:24.309805 30543 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:11:24.312726 30566 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:24.316172 22992 leader_election.cc:336] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
W20260709 13:11:24.318529 23588 leader_election.cc:336] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:24.319367 30520 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:24.319895 30520 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:24.322266 30520 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:24.328177 22992 leader_election.cc:336] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
W20260709 13:11:24.329180 22994 leader_election.cc:336] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:24.330243 22994 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:24.331812 30543 raft_consensus.cc:2749] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:11:24.337322 23590 leader_election.cc:336] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:24.340956 30520 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
W20260709 13:11:24.339272 22994 leader_election.cc:336] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:24.341620 22994 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:24.342425 30543 raft_consensus.cc:2749] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:11:24.340399 23590 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:24.356521 30566 raft_consensus.cc:2749] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 13:11:24.341532 30520 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:24.359148 30520 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:24.367422 22994 leader_election.cc:336] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:24.369421 22992 leader_election.cc:336] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:24.369867 22992 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:24.371727 30536 raft_consensus.cc:2749] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:11:24.399791 23590 consensus_peers.cc:597] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:11:24.440567 23590 consensus_peers.cc:597] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:11:24.441828 30545 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:24.442438 30545 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:24.444716 30545 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:24.449477 23590 leader_election.cc:336] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:24.450912 23588 leader_election.cc:336] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:24.451314 23588 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:24.457935 30545 raft_consensus.cc:2749] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:11:24.477478 23590 consensus_peers.cc:597] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:11:24.502939 21756 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 22976)
W20260709 13:11:24.529855 23590 consensus_peers.cc:597] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:11:24.602211 22994 consensus_peers.cc:597] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 13:11:24.721485 30520 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:24.722052 30520 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:24.731031 30520 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:24.733052 22994 leader_election.cc:336] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:24.733899 22992 leader_election.cc:336] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:24.734287 22992 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:24.735447 30520 raft_consensus.cc:2749] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:11:24.758630 23590 consensus_peers.cc:597] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 13:11:24.795287 30545 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:24.795903 30545 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:11:24.814352 23590 leader_election.cc:336] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:24.820303 30545 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:24.822496 23588 leader_election.cc:336] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:24.823187 23588 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:24.824484 30545 raft_consensus.cc:2749] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:11:24.871400 22994 consensus_peers.cc:597] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:11:24.909430 22994 consensus_peers.cc:597] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:11:25.015523 22994 consensus_peers.cc:597] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:11:25.041458 22994 consensus_peers.cc:597] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:11:25.113277 23590 consensus_peers.cc:597] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:11:25.171253 22994 consensus_peers.cc:597] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:11:25.177318 22994 consensus_peers.cc:597] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:11:25.215698 22992 consensus_peers.cc:597] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:25.248690 22994 consensus_peers.cc:597] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:11:25.296567 23590 consensus_peers.cc:597] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:11:25.383824 23590 consensus_peers.cc:597] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:11:25.398051 23588 consensus_peers.cc:597] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:25.411923 22994 consensus_peers.cc:597] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:11:25.413841 22992 consensus_peers.cc:597] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:25.420365 23590 consensus_peers.cc:597] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:11:25.435567 22992 consensus_peers.cc:597] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:25.479123 22994 consensus_peers.cc:597] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:11:25.503556 21756 thread.cc:527] Waited for 4000ms trying to join with diag-logger (tid 22976)
W20260709 13:11:25.521665 23590 consensus_peers.cc:597] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:11:25.524371 23588 consensus_peers.cc:597] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:25.558013 23588 consensus_peers.cc:597] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:25.596084 22764 catalog_manager.cc:4755] TS 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet 8a996de6f02a4723aef91074b3d56714: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:25.620153 22992 consensus_peers.cc:597] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:25.639487 23588 consensus_peers.cc:597] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:25.644464 23590 consensus_peers.cc:597] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:11:25.691882 22992 consensus_peers.cc:597] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:25.693739 22994 consensus_peers.cc:597] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:11:25.703595 23588 consensus_peers.cc:597] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:25.702337 23590 consensus_peers.cc:597] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:11:25.716393 23588 consensus_peers.cc:597] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:25.719687 22992 consensus_peers.cc:597] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:25.737375 22992 consensus_peers.cc:597] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:25.744731 23588 consensus_peers.cc:597] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:25.780004 22992 consensus_peers.cc:597] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:25.973627 22992 consensus_peers.cc:597] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:25.974107 22992 consensus_peers.cc:597] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:25.974498 22992 consensus_peers.cc:597] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:26.000505 22994 consensus_peers.cc:597] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:11:26.004244 22992 consensus_peers.cc:597] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:26.012225 23588 consensus_peers.cc:597] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:11:26.017854 30545 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:26.018450 30545 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:11:26.026479 23590 leader_election.cc:336] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:26.027987 30545 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:26.039419 23588 leader_election.cc:336] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:26.041467 23588 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:26.042378 30545 raft_consensus.cc:2749] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:11:26.063589 22992 consensus_peers.cc:597] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:26.089802 30573 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:26.090421 30573 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:26.093472 30573 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:26.101541 23590 leader_election.cc:336] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:26.109985 22992 consensus_peers.cc:597] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:26.125497 22992 consensus_peers.cc:597] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:26.130581 23588 leader_election.cc:336] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:26.136891 23588 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:26.137845 30573 raft_consensus.cc:2749] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 13:11:26.142074 22992 consensus_peers.cc:597] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:26.157051 22992 consensus_peers.cc:597] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:26.188786 23588 consensus_peers.cc:597] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:26.196349 23588 consensus_peers.cc:597] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:26.197067 23588 consensus_peers.cc:597] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:11:26.198145 30568 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:26.198734 30568 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:26.200888 30568 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:26.214969 22994 leader_election.cc:336] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:26.230909 22992 leader_election.cc:336] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:26.231491 22992 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:26.232295 30568 raft_consensus.cc:2749] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:11:26.248803 30545 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:26.249404 30545 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:26.252142 30545 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:11:26.257400 23590 leader_election.cc:336] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:26.262187 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:11:26.262892 23021 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 16.
W20260709 13:11:26.263219 23588 leader_election.cc:336] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:26.263672 23588 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:26.270723 30545 raft_consensus.cc:2749] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:11:26.331486 23588 consensus_peers.cc:597] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:11:26.384986 30568 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:26.385634 30568 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:26.387862 30568 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:26.399447 22994 leader_election.cc:336] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:26.409636 23588 consensus_peers.cc:597] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:26.418803 22992 leader_election.cc:336] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:26.424638 22992 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:26.429157 30543 raft_consensus.cc:2749] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:11:26.482985 30568 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:26.483527 30568 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:11:26.495287 22994 leader_election.cc:336] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:26.496096 30568 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:26.504079 21756 thread.cc:527] Waited for 5000ms trying to join with diag-logger (tid 22976)
W20260709 13:11:26.514173 22992 leader_election.cc:336] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:26.514670 22992 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:26.516352 30568 raft_consensus.cc:2749] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:11:26.534008 23590 consensus_peers.cc:597] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:11:26.701896 22994 consensus_peers.cc:597] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:11:26.764868 22994 consensus_peers.cc:597] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 13:11:26.794210 22994 consensus_peers.cc:597] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:11:26.848971 23050 debug-util.cc:398] Leaking SignalData structure 0x7b080035b5e0 after lost signal to thread 21759
W20260709 13:11:26.850244 23050 debug-util.cc:398] Leaking SignalData structure 0x7b08002e6de0 after lost signal to thread 22812
W20260709 13:11:26.851709 23050 debug-util.cc:398] Leaking SignalData structure 0x7b0800193240 after lost signal to thread 23053
W20260709 13:11:26.853199 23050 debug-util.cc:398] Leaking SignalData structure 0x7b0800346380 after lost signal to thread 23649
I20260709 13:11:26.941059 30568 raft_consensus.cc:493] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:26.941646 30568 raft_consensus.cc:515] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:26.943851 30568 leader_election.cc:290] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:26.949975 22994 leader_election.cc:336] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:26.951848 23590 consensus_peers.cc:597] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:11:26.960278 23590 consensus_peers.cc:597] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:11:26.962184 22992 leader_election.cc:336] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:26.962816 22992 leader_election.cc:304] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:26.970922 30568 raft_consensus.cc:2749] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:11:26.978271 23590 consensus_peers.cc:597] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:11:27.082590 30568 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:27.083268 30568 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:27.086956 30568 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:27.089917 22994 leader_election.cc:336] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:27.155308 30543 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:27.155897 30543 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:27.158341 30543 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:27.162837 22994 leader_election.cc:336] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:27.177038 22992 leader_election.cc:336] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:27.190725 22992 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:27.177909 30568 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:27.193212 30568 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:27.195716 30568 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:11:27.198350 30575 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:27.199128 30575 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:27.200533 30543 raft_consensus.cc:2749] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:11:27.201139 30575 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:27.205826 22994 leader_election.cc:336] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:27.206415 22994 leader_election.cc:336] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:27.208343 30543 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:27.208952 30543 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:27.211252 30543 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:27.215749 23590 consensus_peers.cc:597] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:11:27.217622 22994 leader_election.cc:336] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:27.223783 22994 consensus_peers.cc:597] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:11:27.265760 23590 consensus_peers.cc:597] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:11:27.288275 30543 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:27.288939 30543 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:27.291141 30543 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:27.298025 22994 leader_election.cc:336] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:27.272318 22992 leader_election.cc:336] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:27.307089 22992 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:11:27.309316 22992 leader_election.cc:336] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:27.315572 22992 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:27.309893 30543 raft_consensus.cc:2749] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:11:27.324641 22992 leader_election.cc:336] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:27.325330 30543 raft_consensus.cc:2749] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 13:11:27.330978 22992 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:27.333004 30543 raft_consensus.cc:2749] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:11:27.344004 22992 leader_election.cc:336] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:27.344592 22992 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:27.360685 30543 raft_consensus.cc:2749] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:11:27.372102 22992 leader_election.cc:336] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:27.372709 22992 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:27.373483 30543 raft_consensus.cc:2749] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:11:27.424654 30568 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:27.425246 30568 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:11:27.433442 22994 consensus_peers.cc:597] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:11:27.461689 30568 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:27.472652 22994 leader_election.cc:336] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:27.503450 22992 leader_election.cc:336] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
W20260709 13:11:27.504567 21756 thread.cc:527] Waited for 6000ms trying to join with diag-logger (tid 22976)
I20260709 13:11:27.506628 22992 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:27.516556 30543 raft_consensus.cc:2749] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:11:27.511790 23590 consensus_peers.cc:597] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:11:27.574414 22994 consensus_peers.cc:597] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:11:27.577893 22994 consensus_peers.cc:597] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:11:27.587505 23590 consensus_peers.cc:597] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:11:27.735131 30543 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:27.735709 30543 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:27.737737 30543 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:27.753847 22992 leader_election.cc:336] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
W20260709 13:11:27.760746 22994 leader_election.cc:336] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:27.761260 22994 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:27.769456 30577 raft_consensus.cc:2749] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:11:27.778504 23590 consensus_peers.cc:597] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:11:27.795555 22994 consensus_peers.cc:597] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:11:27.813283 22994 consensus_peers.cc:597] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:11:27.831022 23590 consensus_peers.cc:597] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:11:27.839835 22994 consensus_peers.cc:597] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:11:27.857963 22994 consensus_peers.cc:597] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:11:27.866276 23588 consensus_peers.cc:597] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:11:27.905659 30578 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:27.906230 30578 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:27.908981 30578 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:11:27.909442 30578 raft_consensus.cc:493] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:27.909891 30578 raft_consensus.cc:515] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:27.911875 30578 leader_election.cc:290] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:11:27.912940 23021 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "5b346767a89745fa82928a5dac9cc0b2" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f3c799b1249e40119d2a5864231c6c75" is_pre_election: true
I20260709 13:11:27.913628 23021 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b346767a89745fa82928a5dac9cc0b2 in term 16.
W20260709 13:11:27.917472 23590 leader_election.cc:336] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:27.931638 22992 consensus_peers.cc:597] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:27.932190 23588 leader_election.cc:336] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:27.934705 23588 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2, f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:11:27.935262 23590 leader_election.cc:336] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:27.936004 30567 raft_consensus.cc:2749] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:11:27.936712 23588 leader_election.cc:336] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:27.937487 23588 leader_election.cc:304] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:27.938282 30567 raft_consensus.cc:2749] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 13:11:27.973234 22992 consensus_peers.cc:597] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:28.013549 23588 consensus_peers.cc:597] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:11:28.025209 30545 raft_consensus.cc:493] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:28.025765 30545 raft_consensus.cc:515] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:28.027580 30545 leader_election.cc:290] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:28.034564 23588 consensus_peers.cc:597] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:28.047864 23588 leader_election.cc:336] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
W20260709 13:11:28.063239 22994 consensus_peers.cc:597] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:11:28.064535 22992 consensus_peers.cc:597] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:28.076431 23590 leader_election.cc:336] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:28.077023 23590 leader_election.cc:304] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:28.078382 30567 raft_consensus.cc:2749] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:11:28.077626 22992 consensus_peers.cc:597] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:28.089098 23590 consensus_peers.cc:597] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:11:28.103426 23590 consensus_peers.cc:597] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:11:28.110244 23588 consensus_peers.cc:597] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:28.122231 22994 consensus_peers.cc:597] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:11:28.134712 22992 consensus_peers.cc:597] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:28.138664 22994 consensus_peers.cc:597] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:11:28.165150 22992 consensus_peers.cc:597] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:28.196076 23588 consensus_peers.cc:597] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:28.196951 22992 consensus_peers.cc:597] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:28.202152 23590 consensus_peers.cc:597] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:11:28.212796 22992 consensus_peers.cc:597] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:28.383067 23588 consensus_peers.cc:597] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:28.431211 23590 consensus_peers.cc:597] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:11:28.433271 22994 consensus_peers.cc:597] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:11:28.447122 23588 consensus_peers.cc:597] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:28.498721 22992 consensus_peers.cc:597] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:28.499145 22992 consensus_peers.cc:597] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:28.499698 22992 consensus_peers.cc:597] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:28.506531 21756 thread.cc:527] Waited for 7000ms trying to join with diag-logger (tid 22976)
W20260709 13:11:28.522774 22992 consensus_peers.cc:597] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:28.536687 22992 consensus_peers.cc:597] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:28.574361 22992 consensus_peers.cc:597] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:11:28.590960 30582 raft_consensus.cc:493] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
W20260709 13:11:28.591454 23588 consensus_peers.cc:597] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:11:28.603128 30582 raft_consensus.cc:515] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:28.613437 30582 leader_election.cc:290] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:28.614023 22994 leader_election.cc:336] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:28.647305 22992 leader_election.cc:336] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:28.647912 22992 leader_election.cc:304] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:28.649149 30582 raft_consensus.cc:2749] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:11:28.720901 22992 consensus_peers.cc:597] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:11:28.774168 23588 consensus_peers.cc:597] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:28.774793 23588 consensus_peers.cc:597] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:28.811477 22992 consensus_peers.cc:597] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:28.843961 23588 consensus_peers.cc:597] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:28.937492 22992 consensus_peers.cc:597] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:28.939751 23588 consensus_peers.cc:597] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:28.960048 23590 consensus_peers.cc:597] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:11:29.004366 23588 consensus_peers.cc:597] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:29.016717 22994 consensus_peers.cc:597] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:11:29.129542 30568 raft_consensus.cc:493] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:29.130138 30568 raft_consensus.cc:515] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:11:29.143040 22994 leader_election.cc:336] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:29.149721 30568 leader_election.cc:290] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:11:29.183187 30582 raft_consensus.cc:493] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:29.183790 30582 raft_consensus.cc:515] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:29.186040 30582 leader_election.cc:290] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:29.207808 22994 leader_election.cc:336] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:29.207335 22992 leader_election.cc:336] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:29.209085 22992 leader_election.cc:304] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:11:29.208515 22994 consensus_peers.cc:597] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:11:29.210588 30568 raft_consensus.cc:2749] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:11:29.229467 22992 leader_election.cc:336] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:29.231968 22992 leader_election.cc:304] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:29.232831 30568 raft_consensus.cc:2749] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:11:29.277035 23590 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111) [suppressed 635 similar messages]
I20260709 13:11:29.287905 30568 raft_consensus.cc:493] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:29.288475 30568 raft_consensus.cc:515] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:29.290436 30568 leader_election.cc:290] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:29.295639 22992 leader_election.cc:336] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
W20260709 13:11:29.296344 22994 leader_election.cc:336] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:29.296802 22994 leader_election.cc:304] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:29.297524 30568 raft_consensus.cc:2749] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:11:29.400935 23646 debug-util.cc:398] Leaking SignalData structure 0x7b08003647e0 after lost signal to thread 21759
W20260709 13:11:29.402055 23646 debug-util.cc:398] Leaking SignalData structure 0x7b0800343680 after lost signal to thread 22812
W20260709 13:11:29.403294 23646 debug-util.cc:398] Leaking SignalData structure 0x7b0800158140 after lost signal to thread 23053
W20260709 13:11:29.404541 23646 debug-util.cc:398] Leaking SignalData structure 0x7b08003a24a0 after lost signal to thread 23649
W20260709 13:11:29.464995 23590 consensus_peers.cc:597] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:11:29.490808 22994 consensus_peers.cc:597] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 13:11:29.510875 21756 thread.cc:527] Waited for 8000ms trying to join with diag-logger (tid 22976)
W20260709 13:11:29.600142 23590 consensus_peers.cc:597] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:11:29.604620 23590 consensus_peers.cc:597] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:11:29.709599 30582 raft_consensus.cc:493] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:29.710170 30582 raft_consensus.cc:515] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:11:29.720419 23590 consensus_peers.cc:597] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:11:29.724810 22994 leader_election.cc:336] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:29.712391 30582 leader_election.cc:290] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:29.739017 22992 leader_election.cc:336] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:29.745919 22992 leader_election.cc:304] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:29.746675 30568 raft_consensus.cc:2749] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 13:11:29.828397 22994 consensus_peers.cc:597] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:11:29.831363 23590 consensus_peers.cc:597] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:11:29.849627 23588 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42499, user_credentials={real_user=slave}} blocked reactor thread for 47299.1us
W20260709 13:11:29.858191 23590 consensus_peers.cc:597] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:11:29.907119 22994 consensus_peers.cc:597] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:11:30.083575 23590 consensus_peers.cc:597] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:11:30.136412 23590 consensus_peers.cc:597] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:11:30.186391 23588 consensus_peers.cc:597] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 13:11:30.210052 22994 consensus_peers.cc:597] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:11:30.234318 23590 consensus_peers.cc:597] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:11:30.244835 22992 consensus_peers.cc:597] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:11:30.276747 30584 raft_consensus.cc:493] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:30.277439 30584 raft_consensus.cc:515] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } attrs { replace: true } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
W20260709 13:11:30.289166 22994 leader_election.cc:336] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:30.291024 30584 leader_election.cc:290] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), 5b346767a89745fa82928a5dac9cc0b2 (127.21.63.4:44043)
W20260709 13:11:30.323064 22994 consensus_peers.cc:597] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:11:30.326146 22992 leader_election.cc:336] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:30.326179 23617 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c3d2a83be5024e50818057526cbaf822" candidate_uuid: "f3c799b1249e40119d2a5864231c6c75" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "5b346767a89745fa82928a5dac9cc0b2" is_pre_election: true
I20260709 13:11:30.339361 22992 leader_election.cc:304] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:30.339581 23617 raft_consensus.cc:2468] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3c799b1249e40119d2a5864231c6c75 in term 16.
I20260709 13:11:30.340965 30582 raft_consensus.cc:2749] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 13:11:30.371536 22994 consensus_peers.cc:597] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:11:30.408298 22994 consensus_peers.cc:597] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:11:30.424185 23588 consensus_peers.cc:597] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:30.510919 23588 consensus_peers.cc:597] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:11:30.517584 30584 raft_consensus.cc:493] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:30.518844 30584 raft_consensus.cc:515] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:30.522687 30584 leader_election.cc:290] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:30.559242 22994 consensus_peers.cc:597] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:11:30.561965 22994 consensus_peers.cc:597] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:11:30.562549 22994 leader_election.cc:336] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:30.580603 22992 leader_election.cc:336] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
W20260709 13:11:30.591267 23588 consensus_peers.cc:597] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:11:30.593636 22992 leader_election.cc:304] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
W20260709 13:11:30.594658 22992 consensus_peers.cc:597] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:11:30.596853 30595 raft_consensus.cc:2749] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 13:11:30.599233 22994 consensus_peers.cc:597] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 13:11:30.602964 23588 consensus_peers.cc:597] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:11:30.612381 30595 raft_consensus.cc:493] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:30.613190 30595 raft_consensus.cc:515] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:30.616044 30595 leader_election.cc:290] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:30.618309 22992 consensus_peers.cc:597] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:30.618822 22992 consensus_peers.cc:597] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:30.622895 23590 consensus_peers.cc:597] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:11:30.630494 22994 leader_election.cc:336] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:30.634732 22992 leader_election.cc:336] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:30.635218 22992 leader_election.cc:304] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:30.636473 30595 raft_consensus.cc:2749] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 13:11:30.650686 30595 raft_consensus.cc:493] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:30.651365 30595 raft_consensus.cc:515] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:30.653839 30595 leader_election.cc:290] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:30.659878 22994 leader_election.cc:336] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:30.666512 22992 leader_election.cc:336] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:30.667002 22992 leader_election.cc:304] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:30.667768 30595 raft_consensus.cc:2749] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 13:11:30.671186 22992 consensus_peers.cc:597] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:11:30.676107 21756 tablet_server.cc:196] TabletServer@127.21.63.2:0 shutdown complete.
W20260709 13:11:30.679982 23590 consensus_peers.cc:597] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:11:30.706676 22992 consensus_peers.cc:597] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 13:11:30.724578 30594 raft_consensus.cc:493] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:30.725636 30594 raft_consensus.cc:515] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:11:30.735380 22994 leader_election.cc:336] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:30.742790 30594 leader_election.cc:290] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:30.746857 22994 consensus_peers.cc:597] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:11:30.750770 22992 leader_election.cc:336] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:30.751379 22992 leader_election.cc:304] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:30.752242 30594 raft_consensus.cc:2749] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:11:30.759755 23588 consensus_peers.cc:597] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:30.793639 23588 consensus_peers.cc:597] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:11:30.853016 30593 raft_consensus.cc:493] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:30.853888 30593 raft_consensus.cc:515] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
I20260709 13:11:30.858428 30593 leader_election.cc:290] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:30.866339 22994 leader_election.cc:336] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:30.867221 22992 leader_election.cc:336] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:30.871944 21756 tablet_server.cc:179] TabletServer@127.21.63.3:0 shutting down...
I20260709 13:11:30.873265 22992 leader_election.cc:304] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:30.876101 30595 raft_consensus.cc:2749] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:11:30.883155 23590 consensus_peers.cc:597] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:11:30.892325 30595 raft_consensus.cc:493] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:30.893179 30595 raft_consensus.cc:515] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } }
W20260709 13:11:30.897398 23588 consensus_peers.cc:597] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 -> Peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Couldn't send request to peer f3c799b1249e40119d2a5864231c6c75. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:30.895936 30595 leader_election.cc:290] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
W20260709 13:11:30.901167 22992 consensus_peers.cc:597] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:30.901731 22992 consensus_peers.cc:597] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:30.904016 23588 consensus_peers.cc:597] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 -> Peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Couldn't send request to peer f3c799b1249e40119d2a5864231c6c75. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:30.904876 23588 consensus_peers.cc:597] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Couldn't send request to peer f3c799b1249e40119d2a5864231c6c75. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:30.905587 22992 leader_election.cc:336] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
W20260709 13:11:30.913062 22994 consensus_peers.cc:597] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
W20260709 13:11:30.913818 22994 leader_election.cc:336] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:30.914278 22994 leader_election.cc:304] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f3c799b1249e40119d2a5864231c6c75; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:30.915139 30594 raft_consensus.cc:2749] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 13:11:30.938864 23588 consensus_peers.cc:597] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:30.949486 22992 consensus_peers.cc:597] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:11:30.959362 21756 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:11:30.961198 21756 tablet_replica.cc:333] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:30.963471 21756 raft_consensus.cc:2243] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:30.966230 21756 raft_consensus.cc:2272] T ced80ef275e04ef8bf4c071883b8284e P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:30.970405 23588 consensus_peers.cc:597] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 -> Peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Couldn't send request to peer f3c799b1249e40119d2a5864231c6c75. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:30.973001 22992 consensus_peers.cc:597] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:11:30.973102 21756 tablet_replica.cc:333] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:30.974429 21756 raft_consensus.cc:2243] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 21 LEADER]: Raft consensus shutting down.
W20260709 13:11:30.980881 22992 consensus_peers.cc:597] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:11:30.982016 21756 raft_consensus.cc:2272] T 61adbc0c57034ac28a0b50af8a59ed07 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:30.985687 21756 tablet_replica.cc:333] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:30.987196 21756 raft_consensus.cc:2243] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:11:30.988533 21756 raft_consensus.cc:2272] T 3f1c9e3f9b244b0695775e91f28d67d7 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:30.991499 21756 tablet_replica.cc:333] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:30.992453 21756 raft_consensus.cc:2243] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:11:30.993784 21756 raft_consensus.cc:2272] T 80ab3f0c16e747e3b1917985c734eed4 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:30.996696 23590 consensus_peers.cc:597] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 13:11:30.997767 21756 tablet_replica.cc:333] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:30.998719 21756 raft_consensus.cc:2243] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:11:30.999949 21756 raft_consensus.cc:2272] T 6eba8b8b84ca4a9aa5d9eaca94670686 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.002771 21756 tablet_replica.cc:333] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:31.003575 21756 raft_consensus.cc:2243] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.005139 21756 raft_consensus.cc:2272] T c3d2a83be5024e50818057526cbaf822 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:31.005416 23588 consensus_peers.cc:597] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Couldn't send request to peer f3c799b1249e40119d2a5864231c6c75. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:31.007395 21756 tablet_replica.cc:333] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:31.008153 21756 raft_consensus.cc:2243] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.008961 21756 raft_consensus.cc:2272] T 3ec25afe61174e01a75a21ab33afe2e1 P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.011431 21756 tablet_replica.cc:333] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:31.012212 21756 raft_consensus.cc:2243] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.013191 21756 raft_consensus.cc:2272] T 104e2273f37843cbb5245981c817dfe0 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.015957 21756 tablet_replica.cc:333] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:31.016815 21756 raft_consensus.cc:2243] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.017618 21756 raft_consensus.cc:2272] T abd5e7d435ca4afd9727e37b2ab31589 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.019934 21756 tablet_replica.cc:333] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:31.020723 21756 raft_consensus.cc:2243] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.021268 21756 raft_consensus.cc:2272] T edc3e3e3969d4c17b76da48ab861a54b P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.023478 21756 tablet_replica.cc:333] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:31.024107 21756 raft_consensus.cc:2243] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.025240 21756 raft_consensus.cc:2272] T f603724630e44ef78014e7719f096cb3 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.027874 21756 tablet_replica.cc:333] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:31.028579 21756 raft_consensus.cc:2243] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:11:31.029608 21756 raft_consensus.cc:2272] T 5cbb1e724f1d4e3f97c0d29d6b2ce944 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.031592 21756 tablet_replica.cc:333] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:31.032343 21756 raft_consensus.cc:2243] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:11:31.033521 21756 raft_consensus.cc:2272] T 4e689c91ad7343b5b6ba36fa648a06ac P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.035610 21756 tablet_replica.cc:333] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:31.036152 21756 raft_consensus.cc:2243] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.036726 21756 raft_consensus.cc:2272] T 421f71937d5e4cef9084068ce92d12b7 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.038686 21756 tablet_replica.cc:333] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:31.039245 21756 raft_consensus.cc:2243] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.039829 21756 raft_consensus.cc:2272] T 7cd8492f63fb4193a47cf9fbc574ef58 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.041805 21756 tablet_replica.cc:333] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:31.042323 21756 raft_consensus.cc:2243] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.042892 21756 raft_consensus.cc:2272] T f511a9ca0d9b4f54833aa7d0c092ccc1 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.045303 21756 tablet_replica.cc:333] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:31.046211 21756 raft_consensus.cc:2243] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:11:31.047524 21756 raft_consensus.cc:2272] T aa6540758938493d98d7852fdbe3ffab P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.049916 21756 tablet_replica.cc:333] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:31.050690 21756 raft_consensus.cc:2243] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.051368 21756 raft_consensus.cc:2272] T cdebaf291e5d4ae9814a80ce81ee19fe P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.054039 21756 tablet_replica.cc:333] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:31.054793 21756 raft_consensus.cc:2243] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.055493 21756 raft_consensus.cc:2272] T c26347a647b240d7a3bc5cd449b7993b P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:31.056214 23588 consensus_peers.cc:597] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Couldn't send request to peer f3c799b1249e40119d2a5864231c6c75. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:31.058177 21756 tablet_replica.cc:333] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:31.058980 21756 raft_consensus.cc:2243] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 19 LEADER]: Raft consensus shutting down.
W20260709 13:11:31.059619 23588 consensus_peers.cc:597] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Couldn't send request to peer f3c799b1249e40119d2a5864231c6c75. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:31.060240 21756 raft_consensus.cc:2272] T 3047c357228240a2aa4d84e8fbf437fc P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.062480 21756 tablet_replica.cc:333] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:31.063048 21756 raft_consensus.cc:2243] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:11:31.064836 21756 raft_consensus.cc:2272] T ec7dd25342084943aacb4b9830f5f1ea P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.067348 21756 tablet_replica.cc:333] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:31.068130 21756 raft_consensus.cc:2243] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.069128 21756 raft_consensus.cc:2272] T 6bdc53dc56134e28bcc43e68e76055fc P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:31.070429 22994 consensus_peers.cc:597] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:11:31.071564 21756 tablet_replica.cc:333] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:31.072207 21756 raft_consensus.cc:2243] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.072970 21756 raft_consensus.cc:2272] T e3b39c18a21f448783cdd998acfb2f80 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.074628 30592 raft_consensus.cc:493] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader aed3f089ec8b49af9c8cf5cdee76e8e7)
I20260709 13:11:31.075664 21756 tablet_replica.cc:333] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:31.075307 30592 raft_consensus.cc:515] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:31.076375 21756 raft_consensus.cc:2243] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:11:31.077412 21756 raft_consensus.cc:2272] T fab4388fa7ea49bda9fa070cf542577f P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.077627 30592 leader_election.cc:290] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309)
I20260709 13:11:31.079867 21756 tablet_replica.cc:333] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:31.080788 21756 raft_consensus.cc:2243] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.081645 21756 raft_consensus.cc:2272] T 7ca8b9a08f4146c386bc06d03c497038 P f3c799b1249e40119d2a5864231c6c75 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.084496 21756 tablet_replica.cc:333] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:31.085474 21756 raft_consensus.cc:2243] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 14 LEADER]: Raft consensus shutting down.
W20260709 13:11:31.085541 23588 leader_election.cc:336] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
W20260709 13:11:31.086496 23590 leader_election.cc:336] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:31.086740 21756 raft_consensus.cc:2272] T 2959101199a44a4583a4ee456ecf044d P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.086939 23590 leader_election.cc:304] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, aed3f089ec8b49af9c8cf5cdee76e8e7
I20260709 13:11:31.087754 30592 raft_consensus.cc:2749] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 13:11:31.089395 21756 tablet_replica.cc:333] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:31.089968 21756 raft_consensus.cc:2243] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.090503 21756 raft_consensus.cc:2272] T 612dc49c81064f4faa79f65fcf1cfc02 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.092453 21756 tablet_replica.cc:333] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:31.093034 21756 raft_consensus.cc:2243] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:11:31.094027 21756 raft_consensus.cc:2272] T 85d33f0fd80a49b8bd8cb73571ed3191 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.096346 21756 tablet_replica.cc:333] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:31.097144 21756 raft_consensus.cc:2243] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:11:31.098431 21756 raft_consensus.cc:2272] T 8a996de6f02a4723aef91074b3d56714 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.100708 21756 tablet_replica.cc:333] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:31.101279 21756 raft_consensus.cc:2243] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:11:31.102257 21756 raft_consensus.cc:2272] T 8217046fd4e14cc4a0194da4f6ad3887 P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.104249 21756 tablet_replica.cc:333] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:31.104787 21756 raft_consensus.cc:2243] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:11:31.105664 21756 raft_consensus.cc:2272] T 392f5a2324d74a499e6baad745c941fd P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.107915 21756 tablet_replica.cc:333] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:31.108669 21756 raft_consensus.cc:2243] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:11:31.109941 21756 raft_consensus.cc:2272] T f3b08335f3564868a8160dc1b5ade501 P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:31.110637 23588 consensus_peers.cc:597] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 -> Peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Couldn't send request to peer f3c799b1249e40119d2a5864231c6c75. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:31.112134 21756 tablet_replica.cc:333] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:31.112751 21756 raft_consensus.cc:2243] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.113302 21756 raft_consensus.cc:2272] T 616e98b5f7534394ae8790b08aec10e6 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.115231 21756 tablet_replica.cc:333] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:31.115777 21756 raft_consensus.cc:2243] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:11:31.116921 21756 raft_consensus.cc:2272] T dd740f20d2d74669b0fd10f1ec5b44f0 P f3c799b1249e40119d2a5864231c6c75 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.119158 21756 tablet_replica.cc:333] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:31.119716 21756 raft_consensus.cc:2243] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 23 LEADER]: Raft consensus shutting down.
I20260709 13:11:31.120702 21756 raft_consensus.cc:2272] T 2af6865189dc47369064cfad3fd137e1 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.122718 21756 tablet_replica.cc:333] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:31.123239 21756 raft_consensus.cc:2243] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.124153 21756 raft_consensus.cc:2272] T 778fe3ade80844c48f8408313fc86a26 P f3c799b1249e40119d2a5864231c6c75 [term 23 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.126582 21756 tablet_replica.cc:333] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:31.127246 21756 raft_consensus.cc:2243] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.127805 21756 raft_consensus.cc:2272] T 7f35ebcf93ad423ea8bd0b777bee23ee P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.129841 21756 tablet_replica.cc:333] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:31.130354 21756 raft_consensus.cc:2243] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:11:31.131222 21756 raft_consensus.cc:2272] T ec334666b7964c969125af95d9c797ed P f3c799b1249e40119d2a5864231c6c75 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.133266 21756 tablet_replica.cc:333] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:31.133775 21756 raft_consensus.cc:2243] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:11:31.134948 21756 raft_consensus.cc:2272] T 0f076a0e47a14c2b933d3058063c2bcd P f3c799b1249e40119d2a5864231c6c75 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.136878 21756 tablet_replica.cc:333] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:31.137390 21756 raft_consensus.cc:2243] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.137985 21756 raft_consensus.cc:2272] T 67e7fdd613434ffaacb8a030d4693490 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.139933 21756 tablet_replica.cc:333] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:31.140475 21756 raft_consensus.cc:2243] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.141131 21756 raft_consensus.cc:2272] T b7633b7b4c334b039db57d77774df345 P f3c799b1249e40119d2a5864231c6c75 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.143152 21756 tablet_replica.cc:333] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:31.143816 21756 raft_consensus.cc:2243] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.144622 21756 raft_consensus.cc:2272] T 13e53d9e00624dfcb87aca9b055fdc9a P f3c799b1249e40119d2a5864231c6c75 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.147228 21756 tablet_replica.cc:333] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:31.147917 21756 raft_consensus.cc:2243] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 24 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.149964 21756 raft_consensus.cc:2272] T 7426119f4da44ffaa1513b481a866a01 P f3c799b1249e40119d2a5864231c6c75 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.152247 21756 tablet_replica.cc:333] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:31.152870 21756 raft_consensus.cc:2243] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:11:31.153944 21756 raft_consensus.cc:2272] T a0b6723ad2db4fccb7a1c219ff04e99a P f3c799b1249e40119d2a5864231c6c75 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.156133 21756 tablet_replica.cc:333] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:31.156725 21756 raft_consensus.cc:2243] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:11:31.157601 21756 raft_consensus.cc:2272] T b7e4ee754b7a4408b5fc55259d431f39 P f3c799b1249e40119d2a5864231c6c75 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.159571 21756 tablet_replica.cc:333] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:31.160085 21756 raft_consensus.cc:2243] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 25 LEADER]: Raft consensus shutting down.
I20260709 13:11:31.161190 21756 raft_consensus.cc:2272] T 0c4ecd16880043f4a75006a3da4717bf P f3c799b1249e40119d2a5864231c6c75 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.163563 21756 tablet_replica.cc:333] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
I20260709 13:11:31.164188 21756 raft_consensus.cc:2243] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:31.165007 21756 raft_consensus.cc:2272] T ac5cc94926a742e6a0c1340ec33c0ff0 P f3c799b1249e40119d2a5864231c6c75 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:31.167490 21756 tablet_replica.cc:333] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75: stopping tablet replica
W20260709 13:11:31.167943 23588 consensus_peers.cc:597] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Couldn't send request to peer f3c799b1249e40119d2a5864231c6c75. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:31.168031 21756 raft_consensus.cc:2243] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:11:31.169217 21756 raft_consensus.cc:2272] T b41a09964104486c8e50b539ec2b16f6 P f3c799b1249e40119d2a5864231c6c75 [term 18 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:31.217435 23588 consensus_peers.cc:597] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Couldn't send request to peer f3c799b1249e40119d2a5864231c6c75. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:31.262217 23588 consensus_peers.cc:597] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:31.305212 23588 consensus_peers.cc:597] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 -> Peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Couldn't send request to peer f3c799b1249e40119d2a5864231c6c75. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.3:42499: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:31.341364 23588 consensus_peers.cc:597] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:31.379277 23588 consensus_peers.cc:597] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:31.469897 23588 consensus_peers.cc:597] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 13:11:31.647046 23590 consensus_peers.cc:597] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
W20260709 13:11:31.758759 23588 consensus_peers.cc:597] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Couldn't send request to peer aed3f089ec8b49af9c8cf5cdee76e8e7. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 13:11:31.953259 21756 tablet_server.cc:196] TabletServer@127.21.63.3:0 shutdown complete.
W20260709 13:11:31.991202 23590 consensus_peers.cc:597] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:11:32.012993 30590 raft_consensus.cc:493] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:11:32.013664 30590 raft_consensus.cc:515] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:32.016124 30590 leader_election.cc:290] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:11:32.024525 23588 leader_election.cc:336] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Network error: Client connection negotiation failed: client connection to 127.21.63.3:42499: connect: Connection refused (error 111)
W20260709 13:11:32.025382 23588 leader_election.cc:336] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
I20260709 13:11:32.025964 23588 leader_election.cc:304] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:11:32.026865 30590 raft_consensus.cc:2749] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 13:11:32.057705 23590 consensus_peers.cc:597] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 13:11:32.074187 21756 tablet_server.cc:179] TabletServer@127.21.63.4:0 shutting down...
I20260709 13:11:32.083328 30590 raft_consensus.cc:493] T 80ab3f0c16e747e3b1917985c734eed4 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:11:32.083901 30590 raft_consensus.cc:515] T 80ab3f0c16e747e3b1917985c734eed4 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 15 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:32.086094 30590 leader_election.cc:290] T 80ab3f0c16e747e3b1917985c734eed4 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:11:32.093060 23590 leader_election.cc:336] T 80ab3f0c16e747e3b1917985c734eed4 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
W20260709 13:11:32.100317 23588 leader_election.cc:336] T 80ab3f0c16e747e3b1917985c734eed4 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Network error: Client connection negotiation failed: client connection to 127.21.63.3:42499: connect: Connection refused (error 111)
I20260709 13:11:32.100840 23588 leader_election.cc:304] T 80ab3f0c16e747e3b1917985c734eed4 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:11:32.101609 30590 raft_consensus.cc:2749] T 80ab3f0c16e747e3b1917985c734eed4 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 13:11:32.127493 30590 raft_consensus.cc:493] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:11:32.128082 30590 raft_consensus.cc:515] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "aed3f089ec8b49af9c8cf5cdee76e8e7" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 33309 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:32.130409 30590 leader_election.cc:290] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
W20260709 13:11:32.135596 23588 leader_election.cc:336] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer aed3f089ec8b49af9c8cf5cdee76e8e7 (127.21.63.2:33309): Network error: Client connection negotiation failed: client connection to 127.21.63.2:33309: connect: Connection refused (error 111)
W20260709 13:11:32.136327 23588 leader_election.cc:336] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Network error: Client connection negotiation failed: client connection to 127.21.63.3:42499: connect: Connection refused (error 111)
I20260709 13:11:32.136766 21756 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:11:32.136762 23588 leader_election.cc:304] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: aed3f089ec8b49af9c8cf5cdee76e8e7, f3c799b1249e40119d2a5864231c6c75
I20260709 13:11:32.137897 30592 raft_consensus.cc:2749] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:11:32.139789 21756 tablet_replica.cc:333] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2: stopping tablet replica
I20260709 13:11:32.141163 21756 raft_consensus.cc:2243] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Raft consensus shutting down.
W20260709 13:11:32.141265 23590 consensus_peers.cc:597] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:11:32.142190 21756 raft_consensus.cc:2272] T f11454b577ab42d7bfe7c09154b00817 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:32.145738 21756 tablet_replica.cc:333] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2: stopping tablet replica
I20260709 13:11:32.146701 21756 raft_consensus.cc:2243] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:11:32.147984 21756 raft_consensus.cc:2272] T 7426119f4da44ffaa1513b481a866a01 P 5b346767a89745fa82928a5dac9cc0b2 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:32.150920 21756 tablet_replica.cc:333] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2: stopping tablet replica
I20260709 13:11:32.151737 21756 raft_consensus.cc:2243] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:32.152329 21756 raft_consensus.cc:2272] T ec334666b7964c969125af95d9c797ed P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:32.155357 21756 tablet_replica.cc:333] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2: stopping tablet replica
I20260709 13:11:32.156175 21756 raft_consensus.cc:2243] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:11:32.157328 21756 raft_consensus.cc:2272] T 5ab4700268fc4c67b9930087c5da71c8 P 5b346767a89745fa82928a5dac9cc0b2 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:32.160089 21756 tablet_replica.cc:333] T 8a996de6f02a4723aef91074b3d56714 P 5b346767a89745fa82928a5dac9cc0b2: stopping tablet replica
I20260709 13:11:32.161115 21756 raft_consensus.cc:2243] T 8a996de6f02a4723aef91074b3d56714 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:32.162060 21756 raft_consensus.cc:2272] T 8a996de6f02a4723aef91074b3d56714 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:32.165120 21756 tablet_replica.cc:333] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2: stopping tablet replica
I20260709 13:11:32.165980 21756 raft_consensus.cc:2243] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:32.166759 21756 raft_consensus.cc:2272] T c3d2a83be5024e50818057526cbaf822 P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:32.169845 21756 tablet_replica.cc:333] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2: stopping tablet replica
I20260709 13:11:32.170674 21756 raft_consensus.cc:2243] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:32.171372 21756 raft_consensus.cc:2272] T 8217046fd4e14cc4a0194da4f6ad3887 P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:32.174172 21756 tablet_replica.cc:333] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2: stopping tablet replica
I20260709 13:11:32.175022 21756 raft_consensus.cc:2243] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:11:32.176249 21756 raft_consensus.cc:2272] T 4c0bea78fb2a40ef90cff914c5c6e250 P 5b346767a89745fa82928a5dac9cc0b2 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:32.179006 21756 tablet_replica.cc:333] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2: stopping tablet replica
I20260709 13:11:32.179769 21756 raft_consensus.cc:2243] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:32.180414 21756 raft_consensus.cc:2272] T aa6540758938493d98d7852fdbe3ffab P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:32.183283 21756 tablet_replica.cc:333] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2: stopping tablet replica
I20260709 13:11:32.184141 21756 raft_consensus.cc:2243] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:11:32.185264 21756 raft_consensus.cc:2272] T c26347a647b240d7a3bc5cd449b7993b P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:32.188019 21756 tablet_replica.cc:333] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2: stopping tablet replica
I20260709 13:11:32.188771 21756 raft_consensus.cc:2243] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 22 LEADER]: Raft consensus shutting down.
I20260709 13:11:32.189911 21756 raft_consensus.cc:2272] T 67e7fdd613434ffaacb8a030d4693490 P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:32.192691 21756 tablet_replica.cc:333] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2: stopping tablet replica
I20260709 13:11:32.193595 21756 raft_consensus.cc:2243] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:32.194233 21756 raft_consensus.cc:2272] T 2959101199a44a4583a4ee456ecf044d P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:32.197055 21756 tablet_replica.cc:333] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2: stopping tablet replica
I20260709 13:11:32.197793 21756 raft_consensus.cc:2243] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:32.198581 21756 raft_consensus.cc:2272] T 698093275c264d4d99fb8c1a49e3b28f P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:32.201336 21756 tablet_replica.cc:333] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2: stopping tablet replica
I20260709 13:11:32.202234 21756 raft_consensus.cc:2243] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 LEADER]: Raft consensus shutting down.
I20260709 13:11:32.203439 21756 raft_consensus.cc:2272] T b65d9dbf257d434d86d4656dabfd1b70 P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:32.205355 23590 consensus_peers.cc:597] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 -> Peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Couldn't send request to peer 0a407504ed1741bf89a276558adf4c0f. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111). This is attempt 56: this message will repeat every 5th retry.
I20260709 13:11:32.206388 21756 tablet_replica.cc:333] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2: stopping tablet replica
I20260709 13:11:32.207249 21756 raft_consensus.cc:2243] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:11:32.208318 21756 raft_consensus.cc:2272] T 91c88873d54149c58a20f4cf6194e5fc P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:32.211144 21756 tablet_replica.cc:333] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2: stopping tablet replica
I20260709 13:11:32.211901 21756 raft_consensus.cc:2243] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:32.212496 21756 raft_consensus.cc:2272] T 6eba8b8b84ca4a9aa5d9eaca94670686 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:32.215195 21756 tablet_replica.cc:333] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2: stopping tablet replica
I20260709 13:11:32.215939 21756 raft_consensus.cc:2243] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:32.216784 21756 raft_consensus.cc:2272] T 392f5a2324d74a499e6baad745c941fd P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:32.219353 21756 tablet_replica.cc:333] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2: stopping tablet replica
I20260709 13:11:32.220103 21756 raft_consensus.cc:2243] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:11:32.221210 21756 raft_consensus.cc:2272] T 579096ba88474cfbbe40e8bf769879ec P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:32.223763 21756 tablet_replica.cc:333] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2: stopping tablet replica
I20260709 13:11:32.224470 21756 raft_consensus.cc:2243] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:32.225241 21756 raft_consensus.cc:2272] T 4e689c91ad7343b5b6ba36fa648a06ac P 5b346767a89745fa82928a5dac9cc0b2 [term 22 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:32.227919 21756 tablet_replica.cc:333] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2: stopping tablet replica
I20260709 13:11:32.228711 21756 raft_consensus.cc:2243] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:11:32.229676 21756 raft_consensus.cc:2272] T abd5e7d435ca4afd9727e37b2ab31589 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:32.232334 21756 tablet_replica.cc:333] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2: stopping tablet replica
I20260709 13:11:32.233057 21756 raft_consensus.cc:2243] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 LEADER]: Raft consensus shutting down.
I20260709 13:11:32.234095 21756 raft_consensus.cc:2272] T 7ca8b9a08f4146c386bc06d03c497038 P 5b346767a89745fa82928a5dac9cc0b2 [term 17 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:32.236930 21756 tablet_replica.cc:333] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2: stopping tablet replica
I20260709 13:11:32.237727 21756 raft_consensus.cc:2243] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:32.238423 21756 raft_consensus.cc:2272] T cbf52b91ec054ac6a2f8503462b1b7a6 P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:32.241212 21756 tablet_replica.cc:333] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2: stopping tablet replica
I20260709 13:11:32.241881 21756 raft_consensus.cc:2243] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 16 LEADER]: Raft consensus shutting down.
I20260709 13:11:32.242972 21756 raft_consensus.cc:2272] T cdebaf291e5d4ae9814a80ce81ee19fe P 5b346767a89745fa82928a5dac9cc0b2 [term 16 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:32.245467 21756 tablet_replica.cc:333] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2: stopping tablet replica
I20260709 13:11:32.246160 21756 raft_consensus.cc:2243] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:32.246829 21756 raft_consensus.cc:2272] T a0b6723ad2db4fccb7a1c219ff04e99a P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:32.249567 21756 tablet_replica.cc:333] T ec7dd25342084943aacb4b9830f5f1ea P 5b346767a89745fa82928a5dac9cc0b2: stopping tablet replica
I20260709 13:11:32.250331 21756 raft_consensus.cc:2243] T ec7dd25342084943aacb4b9830f5f1ea P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:32.250854 21756 raft_consensus.cc:2272] T ec7dd25342084943aacb4b9830f5f1ea P 5b346767a89745fa82928a5dac9cc0b2 [term 21 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:32.253911 21756 tablet_replica.cc:333] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2: stopping tablet replica
I20260709 13:11:32.254635 21756 raft_consensus.cc:2243] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:32.255254 21756 raft_consensus.cc:2272] T 3047c357228240a2aa4d84e8fbf437fc P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:32.257536 30590 raft_consensus.cc:493] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f3c799b1249e40119d2a5864231c6c75)
I20260709 13:11:32.257877 21756 tablet_replica.cc:333] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2: stopping tablet replica
I20260709 13:11:32.258054 30590 raft_consensus.cc:515] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "0a407504ed1741bf89a276558adf4c0f" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 33929 } } peers { permanent_uuid: "f3c799b1249e40119d2a5864231c6c75" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42499 } } peers { permanent_uuid: "5b346767a89745fa82928a5dac9cc0b2" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 44043 } attrs { promote: false } }
I20260709 13:11:32.258692 21756 raft_consensus.cc:2243] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:32.259294 21756 raft_consensus.cc:2272] T 0c4ecd16880043f4a75006a3da4717bf P 5b346767a89745fa82928a5dac9cc0b2 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:32.260277 30590 leader_election.cc:290] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929), f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499)
I20260709 13:11:32.262164 21756 tablet_replica.cc:333] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2: stopping tablet replica
I20260709 13:11:32.262966 21756 raft_consensus.cc:2243] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 20 LEADER]: Raft consensus shutting down.
W20260709 13:11:32.264012 23590 leader_election.cc:336] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:32.264169 21756 raft_consensus.cc:2272] T a424231ef7b4470bbd575564909a365e P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:32.265762 23588 leader_election.cc:336] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer f3c799b1249e40119d2a5864231c6c75 (127.21.63.3:42499): Network error: Client connection negotiation failed: client connection to 127.21.63.3:42499: connect: Connection refused (error 111)
I20260709 13:11:32.266121 23588 leader_election.cc:304] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5b346767a89745fa82928a5dac9cc0b2; no voters: 0a407504ed1741bf89a276558adf4c0f, f3c799b1249e40119d2a5864231c6c75
I20260709 13:11:32.266680 30590 raft_consensus.cc:2749] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 13:11:32.267102 21756 tablet_replica.cc:333] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2: stopping tablet replica
I20260709 13:11:32.267899 21756 raft_consensus.cc:2243] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:32.268515 21756 raft_consensus.cc:2272] T 3f1c9e3f9b244b0695775e91f28d67d7 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:32.271015 21756 tablet_replica.cc:333] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2: stopping tablet replica
I20260709 13:11:32.271703 21756 raft_consensus.cc:2243] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 LEADER]: Raft consensus shutting down.
I20260709 13:11:32.272712 21756 raft_consensus.cc:2272] T ac5cc94926a742e6a0c1340ec33c0ff0 P 5b346767a89745fa82928a5dac9cc0b2 [term 15 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:32.275614 21756 tablet_replica.cc:333] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2: stopping tablet replica
I20260709 13:11:32.276297 21756 raft_consensus.cc:2243] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 LEADER]: Raft consensus shutting down.
I20260709 13:11:32.277393 21756 raft_consensus.cc:2272] T e3b39c18a21f448783cdd998acfb2f80 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:32.280164 21756 tablet_replica.cc:333] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2: stopping tablet replica
I20260709 13:11:32.280813 21756 raft_consensus.cc:2243] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 24 LEADER]: Raft consensus shutting down.
I20260709 13:11:32.281860 21756 raft_consensus.cc:2272] T 0fa29d11409a4d209411116a7ef8af04 P 5b346767a89745fa82928a5dac9cc0b2 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:32.284650 21756 tablet_replica.cc:333] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2: stopping tablet replica
I20260709 13:11:32.285358 21756 raft_consensus.cc:2243] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 19 LEADER]: Raft consensus shutting down.
I20260709 13:11:32.286512 21756 raft_consensus.cc:2272] T 6bdc53dc56134e28bcc43e68e76055fc P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:32.289168 21756 tablet_replica.cc:333] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2: stopping tablet replica
I20260709 13:11:32.289819 21756 raft_consensus.cc:2243] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:32.290359 21756 raft_consensus.cc:2272] T fab4388fa7ea49bda9fa070cf542577f P 5b346767a89745fa82928a5dac9cc0b2 [term 19 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:32.292807 21756 tablet_replica.cc:333] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2: stopping tablet replica
I20260709 13:11:32.293416 21756 raft_consensus.cc:2243] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:11:32.294436 21756 raft_consensus.cc:2272] T 1120807254f5444997e16f6a7feece4d P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:32.296775 21756 tablet_replica.cc:333] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2: stopping tablet replica
I20260709 13:11:32.297389 21756 raft_consensus.cc:2243] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 20 LEADER]: Raft consensus shutting down.
I20260709 13:11:32.298388 21756 raft_consensus.cc:2272] T 13e53d9e00624dfcb87aca9b055fdc9a P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:32.300860 21756 tablet_replica.cc:333] T 80ab3f0c16e747e3b1917985c734eed4 P 5b346767a89745fa82928a5dac9cc0b2: stopping tablet replica
I20260709 13:11:32.301538 21756 raft_consensus.cc:2243] T 80ab3f0c16e747e3b1917985c734eed4 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:32.302601 21756 raft_consensus.cc:2272] T 80ab3f0c16e747e3b1917985c734eed4 P 5b346767a89745fa82928a5dac9cc0b2 [term 20 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:32.305636 21756 tablet_replica.cc:333] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2: stopping tablet replica
I20260709 13:11:32.306237 21756 raft_consensus.cc:2243] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 18 LEADER]: Raft consensus shutting down.
I20260709 13:11:32.307253 21756 raft_consensus.cc:2272] T 7f35ebcf93ad423ea8bd0b777bee23ee P 5b346767a89745fa82928a5dac9cc0b2 [term 18 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:32.309837 21756 tablet_replica.cc:333] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2: stopping tablet replica
I20260709 13:11:32.310423 21756 raft_consensus.cc:2243] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:32.311077 21756 raft_consensus.cc:2272] T dd740f20d2d74669b0fd10f1ec5b44f0 P 5b346767a89745fa82928a5dac9cc0b2 [term 14 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:32.808697 21756 tablet_server.cc:196] TabletServer@127.21.63.4:0 shutdown complete.
I20260709 13:11:32.878439 21756 master.cc:562] Master@127.21.63.62:33399 shutting down...
W20260709 13:11:32.923801 22826 auto_rebalancer.cc:873] Removing replace marker failed: Catalog manager is not initialized. State: Closing
W20260709 13:11:32.924178 22826 auto_rebalancer.cc:901] Could not move replica: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:11:32.924387 22826 auto_rebalancer.cc:330] scheduled replica move failed to complete: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 13:11:41.992780 22764 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111) [suppressed 334 similar messages]
W20260709 13:11:41.996465 22764 catalog_manager.cc:4755] TS 0a407504ed1741bf89a276558adf4c0f (127.21.63.1:33929): DeleteTablet:TABLET_DATA_TOMBSTONED RPC failed for tablet ec7dd25342084943aacb4b9830f5f1ea: Network error: Client connection negotiation failed: client connection to 127.21.63.1:33929: connect: Connection refused (error 111)
I20260709 13:11:43.826205 21756 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 8d175b4ae72f4fc48971c2f31668971c [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:43.827265 21756 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 8d175b4ae72f4fc48971c2f31668971c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:43.827664 21756 tablet_replica.cc:333] T 00000000000000000000000000000000 P 8d175b4ae72f4fc48971c2f31668971c: stopping tablet replica
I20260709 13:11:43.995117 21756 master.cc:584] Master@127.21.63.62:33399 shutdown complete.
[       OK ] LeaderRebalancerTest.AddTserver (251420 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 13:11:44.109241 21756 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.63.62:40095
I20260709 13:11:44.110314 21756 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:11:44.116240 30599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:44.116335 30600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:44.118006 30602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:44.119065 21756 server_base.cc:1061] running on GCE node
I20260709 13:11:44.122399 21756 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:11:44.122592 21756 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:11:44.122797 21756 hybrid_clock.cc:648] HybridClock initialized: now 1783602704122763 us; error 0 us; skew 500 ppm
I20260709 13:11:44.123383 21756 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:11:44.125847 21756 webserver.cc:533] Webserver started at http://127.21.63.62:40395/ using document root <none> and password file <none>
I20260709 13:11:44.126338 21756 fs_manager.cc:362] Metadata directory not provided
I20260709 13:11:44.126531 21756 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:11:44.126819 21756 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:11:44.128101 21756 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602429420501-21756-0/minicluster-data/master-0-root/instance:
uuid: "35038aac9dd04872a98f269c76b1b5d3"
format_stamp: "Formatted at 2026-07-09 13:11:44 on dist-test-slave-c6mt"
I20260709 13:11:44.132959 21756 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.000s
I20260709 13:11:44.136375 30607 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:44.137545 21756 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 13:11:44.137873 21756 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602429420501-21756-0/minicluster-data/master-0-root
uuid: "35038aac9dd04872a98f269c76b1b5d3"
format_stamp: "Formatted at 2026-07-09 13:11:44 on dist-test-slave-c6mt"
I20260709 13:11:44.138454 21756 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602429420501-21756-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602429420501-21756-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602429420501-21756-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:11:44.154388 21756 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:11:44.155663 21756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:11:44.195941 21756 rpc_server.cc:307] RPC server started. Bound to: 127.21.63.62:40095
I20260709 13:11:44.196054 30658 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.63.62:40095 every 8 connection(s)
I20260709 13:11:44.199997 30659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:44.211402 30659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35038aac9dd04872a98f269c76b1b5d3: Bootstrap starting.
I20260709 13:11:44.216202 30659 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35038aac9dd04872a98f269c76b1b5d3: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:44.220475 30659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35038aac9dd04872a98f269c76b1b5d3: No bootstrap required, opened a new log
I20260709 13:11:44.222766 30659 raft_consensus.cc:359] T 00000000000000000000000000000000 P 35038aac9dd04872a98f269c76b1b5d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35038aac9dd04872a98f269c76b1b5d3" member_type: VOTER }
I20260709 13:11:44.223165 30659 raft_consensus.cc:385] T 00000000000000000000000000000000 P 35038aac9dd04872a98f269c76b1b5d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:44.223431 30659 raft_consensus.cc:740] T 00000000000000000000000000000000 P 35038aac9dd04872a98f269c76b1b5d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35038aac9dd04872a98f269c76b1b5d3, State: Initialized, Role: FOLLOWER
I20260709 13:11:44.224078 30659 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35038aac9dd04872a98f269c76b1b5d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35038aac9dd04872a98f269c76b1b5d3" member_type: VOTER }
I20260709 13:11:44.224545 30659 raft_consensus.cc:399] T 00000000000000000000000000000000 P 35038aac9dd04872a98f269c76b1b5d3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:11:44.224790 30659 raft_consensus.cc:493] T 00000000000000000000000000000000 P 35038aac9dd04872a98f269c76b1b5d3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:11:44.225042 30659 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 35038aac9dd04872a98f269c76b1b5d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:44.229607 30659 raft_consensus.cc:515] T 00000000000000000000000000000000 P 35038aac9dd04872a98f269c76b1b5d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35038aac9dd04872a98f269c76b1b5d3" member_type: VOTER }
I20260709 13:11:44.230105 30659 leader_election.cc:304] T 00000000000000000000000000000000 P 35038aac9dd04872a98f269c76b1b5d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 35038aac9dd04872a98f269c76b1b5d3; no voters: 
I20260709 13:11:44.231240 30659 leader_election.cc:290] T 00000000000000000000000000000000 P 35038aac9dd04872a98f269c76b1b5d3 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:11:44.231777 30662 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 35038aac9dd04872a98f269c76b1b5d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:44.233098 30662 raft_consensus.cc:697] T 00000000000000000000000000000000 P 35038aac9dd04872a98f269c76b1b5d3 [term 1 LEADER]: Becoming Leader. State: Replica: 35038aac9dd04872a98f269c76b1b5d3, State: Running, Role: LEADER
I20260709 13:11:44.233783 30662 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35038aac9dd04872a98f269c76b1b5d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35038aac9dd04872a98f269c76b1b5d3" member_type: VOTER }
I20260709 13:11:44.234138 30659 sys_catalog.cc:565] T 00000000000000000000000000000000 P 35038aac9dd04872a98f269c76b1b5d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:11:44.240948 30664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35038aac9dd04872a98f269c76b1b5d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35038aac9dd04872a98f269c76b1b5d3. Latest consensus state: current_term: 1 leader_uuid: "35038aac9dd04872a98f269c76b1b5d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35038aac9dd04872a98f269c76b1b5d3" member_type: VOTER } }
I20260709 13:11:44.241137 30663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35038aac9dd04872a98f269c76b1b5d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "35038aac9dd04872a98f269c76b1b5d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35038aac9dd04872a98f269c76b1b5d3" member_type: VOTER } }
I20260709 13:11:44.241662 30664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35038aac9dd04872a98f269c76b1b5d3 [sys.catalog]: This master's current role is: LEADER
I20260709 13:11:44.241681 30663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35038aac9dd04872a98f269c76b1b5d3 [sys.catalog]: This master's current role is: LEADER
I20260709 13:11:44.244012 30672 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:11:44.250113 30672 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:11:44.251127 21756 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:11:44.259377 30672 catalog_manager.cc:1383] Generated new cluster ID: 327eaabdb2594fb390a7c3ce0e678202
I20260709 13:11:44.259696 30672 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:11:44.278054 30672 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:11:44.279999 30672 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:11:44.300307 30672 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 35038aac9dd04872a98f269c76b1b5d3: Generated new TSK 0
I20260709 13:11:44.300961 30672 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:11:44.318449 21756 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:11:44.325125 30680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:44.325642 30681 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:44.327224 30683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:44.328765 21756 server_base.cc:1061] running on GCE node
I20260709 13:11:44.330965 21756 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:11:44.331144 21756 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:11:44.331338 21756 hybrid_clock.cc:648] HybridClock initialized: now 1783602704331323 us; error 0 us; skew 500 ppm
I20260709 13:11:44.331923 21756 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:11:44.334249 21756 webserver.cc:533] Webserver started at http://127.21.63.1:33247/ using document root <none> and password file <none>
I20260709 13:11:44.334734 21756 fs_manager.cc:362] Metadata directory not provided
I20260709 13:11:44.334925 21756 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:11:44.335227 21756 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:11:44.336452 21756 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602429420501-21756-0/minicluster-data/ts-0-root/instance:
uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
format_stamp: "Formatted at 2026-07-09 13:11:44 on dist-test-slave-c6mt"
I20260709 13:11:44.341100 21756 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:11:44.344455 30688 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:44.345259 21756 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.001s
I20260709 13:11:44.345557 21756 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602429420501-21756-0/minicluster-data/ts-0-root
uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
format_stamp: "Formatted at 2026-07-09 13:11:44 on dist-test-slave-c6mt"
I20260709 13:11:44.345873 21756 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602429420501-21756-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602429420501-21756-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602429420501-21756-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:11:44.357908 21756 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:11:44.359082 21756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:11:44.360498 21756 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:11:44.362890 21756 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:11:44.363108 21756 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:44.363338 21756 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:11:44.363516 21756 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:44.404968 21756 rpc_server.cc:307] RPC server started. Bound to: 127.21.63.1:37325
I20260709 13:11:44.405069 30750 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.63.1:37325 every 8 connection(s)
I20260709 13:11:44.411355 21756 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:11:44.418427 30755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:44.420569 30756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:44.423163 21756 server_base.cc:1061] running on GCE node
W20260709 13:11:44.423223 30758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:44.426303 30751 heartbeater.cc:344] Connected to a master server at 127.21.63.62:40095
I20260709 13:11:44.426783 30751 heartbeater.cc:461] Registering TS with master...
I20260709 13:11:44.426937 21756 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:11:44.427261 21756 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:11:44.427508 21756 hybrid_clock.cc:648] HybridClock initialized: now 1783602704427487 us; error 0 us; skew 500 ppm
I20260709 13:11:44.427979 30751 heartbeater.cc:507] Master 127.21.63.62:40095 requested a full tablet report, sending...
I20260709 13:11:44.428473 21756 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:11:44.430927 30624 ts_manager.cc:194] Registered new tserver with Master: d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:44.431572 21756 webserver.cc:533] Webserver started at http://127.21.63.2:46625/ using document root <none> and password file <none>
I20260709 13:11:44.432121 21756 fs_manager.cc:362] Metadata directory not provided
I20260709 13:11:44.432349 21756 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:11:44.432636 21756 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:11:44.433221 30624 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34512
I20260709 13:11:44.434170 21756 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602429420501-21756-0/minicluster-data/ts-1-root/instance:
uuid: "05213ac836744a53b9c089a5c052f4be"
format_stamp: "Formatted at 2026-07-09 13:11:44 on dist-test-slave-c6mt"
I20260709 13:11:44.439870 21756 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:11:44.443675 30763 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:44.444942 21756 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:11:44.445288 21756 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602429420501-21756-0/minicluster-data/ts-1-root
uuid: "05213ac836744a53b9c089a5c052f4be"
format_stamp: "Formatted at 2026-07-09 13:11:44 on dist-test-slave-c6mt"
I20260709 13:11:44.445572 21756 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602429420501-21756-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602429420501-21756-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602429420501-21756-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:11:44.462729 21756 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:11:44.464154 21756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:11:44.465863 21756 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:11:44.468202 21756 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:11:44.468465 21756 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:44.468779 21756 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:11:44.468983 21756 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:44.515121 21756 rpc_server.cc:307] RPC server started. Bound to: 127.21.63.2:43771
I20260709 13:11:44.515228 30825 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.63.2:43771 every 8 connection(s)
I20260709 13:11:44.520721 21756 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:11:44.528726 30829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:44.530705 30830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:44.532189 30832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:44.533877 21756 server_base.cc:1061] running on GCE node
I20260709 13:11:44.534461 30826 heartbeater.cc:344] Connected to a master server at 127.21.63.62:40095
I20260709 13:11:44.534904 30826 heartbeater.cc:461] Registering TS with master...
I20260709 13:11:44.535125 21756 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:11:44.535401 21756 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:11:44.535609 21756 hybrid_clock.cc:648] HybridClock initialized: now 1783602704535589 us; error 0 us; skew 500 ppm
I20260709 13:11:44.535893 30826 heartbeater.cc:507] Master 127.21.63.62:40095 requested a full tablet report, sending...
I20260709 13:11:44.536450 21756 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:11:44.538491 30624 ts_manager.cc:194] Registered new tserver with Master: 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:44.539479 21756 webserver.cc:533] Webserver started at http://127.21.63.3:43715/ using document root <none> and password file <none>
I20260709 13:11:44.540126 21756 fs_manager.cc:362] Metadata directory not provided
I20260709 13:11:44.540175 30624 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34518
I20260709 13:11:44.540401 21756 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:11:44.540867 21756 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:11:44.542392 21756 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602429420501-21756-0/minicluster-data/ts-2-root/instance:
uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
format_stamp: "Formatted at 2026-07-09 13:11:44 on dist-test-slave-c6mt"
I20260709 13:11:44.547546 21756 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:11:44.551142 30837 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:44.551949 21756 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:11:44.552325 21756 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602429420501-21756-0/minicluster-data/ts-2-root
uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
format_stamp: "Formatted at 2026-07-09 13:11:44 on dist-test-slave-c6mt"
I20260709 13:11:44.552649 21756 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602429420501-21756-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602429420501-21756-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602429420501-21756-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:11:44.566948 21756 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:11:44.568239 21756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:11:44.569970 21756 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:11:44.572299 21756 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:11:44.572542 21756 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:44.572810 21756 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:11:44.572994 21756 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:44.615797 21756 rpc_server.cc:307] RPC server started. Bound to: 127.21.63.3:43993
I20260709 13:11:44.615921 30899 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.63.3:43993 every 8 connection(s)
I20260709 13:11:44.620930 21756 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:11:44.629033 30903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:44.629714 30904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:44.631978 30906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:44.633065 21756 server_base.cc:1061] running on GCE node
I20260709 13:11:44.634227 21756 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:11:44.634490 21756 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:11:44.634704 21756 hybrid_clock.cc:648] HybridClock initialized: now 1783602704634685 us; error 0 us; skew 500 ppm
I20260709 13:11:44.635149 30900 heartbeater.cc:344] Connected to a master server at 127.21.63.62:40095
I20260709 13:11:44.635406 21756 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:11:44.635722 30900 heartbeater.cc:461] Registering TS with master...
I20260709 13:11:44.637039 30900 heartbeater.cc:507] Master 127.21.63.62:40095 requested a full tablet report, sending...
I20260709 13:11:44.638513 21756 webserver.cc:533] Webserver started at http://127.21.63.4:41151/ using document root <none> and password file <none>
I20260709 13:11:44.639130 21756 fs_manager.cc:362] Metadata directory not provided
I20260709 13:11:44.639376 21756 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:11:44.639365 30624 ts_manager.cc:194] Registered new tserver with Master: 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:44.639990 21756 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:11:44.641350 30624 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34526
I20260709 13:11:44.641628 21756 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602429420501-21756-0/minicluster-data/ts-3-root/instance:
uuid: "262942f67dfa4b8fa42234b18ac31ffd"
format_stamp: "Formatted at 2026-07-09 13:11:44 on dist-test-slave-c6mt"
I20260709 13:11:44.646349 21756 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:11:44.650493 30911 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:44.651399 21756 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.000s
I20260709 13:11:44.651671 21756 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602429420501-21756-0/minicluster-data/ts-3-root
uuid: "262942f67dfa4b8fa42234b18ac31ffd"
format_stamp: "Formatted at 2026-07-09 13:11:44 on dist-test-slave-c6mt"
I20260709 13:11:44.651964 21756 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602429420501-21756-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602429420501-21756-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602429420501-21756-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:11:44.692840 21756 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:11:44.694167 21756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:11:44.695729 21756 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:11:44.698208 21756 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:11:44.698405 21756 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:44.698606 21756 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:11:44.698743 21756 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:44.743088 21756 rpc_server.cc:307] RPC server started. Bound to: 127.21.63.4:41313
I20260709 13:11:44.743177 30973 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.63.4:41313 every 8 connection(s)
I20260709 13:11:44.758455 30974 heartbeater.cc:344] Connected to a master server at 127.21.63.62:40095
I20260709 13:11:44.758821 30974 heartbeater.cc:461] Registering TS with master...
I20260709 13:11:44.759562 30974 heartbeater.cc:507] Master 127.21.63.62:40095 requested a full tablet report, sending...
I20260709 13:11:44.761762 30624 ts_manager.cc:194] Registered new tserver with Master: 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:44.762678 21756 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.016231343s
I20260709 13:11:44.763304 30624 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34534
I20260709 13:11:44.789827 30624 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34548:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 13:11:45.033985 30865 tablet_service.cc:1511] Processing CreateTablet for tablet b226c44412c74106a0feb2ca411b1661 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:11:45.035466 30865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b226c44412c74106a0feb2ca411b1661. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.037910 30864 tablet_service.cc:1511] Processing CreateTablet for tablet e28f28c3f05f40e68531e20dd7c1a1a0 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:11:45.039418 30864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e28f28c3f05f40e68531e20dd7c1a1a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.039048 30791 tablet_service.cc:1511] Processing CreateTablet for tablet b226c44412c74106a0feb2ca411b1661 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:11:45.042086 30790 tablet_service.cc:1511] Processing CreateTablet for tablet e28f28c3f05f40e68531e20dd7c1a1a0 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:11:45.043622 30791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b226c44412c74106a0feb2ca411b1661. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.044044 30789 tablet_service.cc:1511] Processing CreateTablet for tablet 7b988f0495c34b7591ad2fb0d5f65693 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:11:45.044965 30790 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e28f28c3f05f40e68531e20dd7c1a1a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.045689 30789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b988f0495c34b7591ad2fb0d5f65693. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.052502 30863 tablet_service.cc:1511] Processing CreateTablet for tablet bcac31d5b7ab48ec9c071a8f77750a88 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:11:45.053925 30863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcac31d5b7ab48ec9c071a8f77750a88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.056692 30788 tablet_service.cc:1511] Processing CreateTablet for tablet bc2a86f27b0f4fb59d7cfa586a23fef1 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:11:45.058220 30788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc2a86f27b0f4fb59d7cfa586a23fef1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.059185 30715 tablet_service.cc:1511] Processing CreateTablet for tablet 7b988f0495c34b7591ad2fb0d5f65693 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:11:45.060796 30715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b988f0495c34b7591ad2fb0d5f65693. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.062309 30862 tablet_service.cc:1511] Processing CreateTablet for tablet bc2a86f27b0f4fb59d7cfa586a23fef1 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:11:45.063683 30862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc2a86f27b0f4fb59d7cfa586a23fef1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.066560 30787 tablet_service.cc:1511] Processing CreateTablet for tablet 39980eee51514ba493958931f406a145 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:11:45.067931 30787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39980eee51514ba493958931f406a145. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.069865 30786 tablet_service.cc:1511] Processing CreateTablet for tablet 11c7485c290740bba00c8b9eaf9b917f (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:11:45.071246 30786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11c7485c290740bba00c8b9eaf9b917f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.074476 30714 tablet_service.cc:1511] Processing CreateTablet for tablet bcac31d5b7ab48ec9c071a8f77750a88 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:11:45.075899 30714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcac31d5b7ab48ec9c071a8f77750a88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.056602 30716 tablet_service.cc:1511] Processing CreateTablet for tablet e28f28c3f05f40e68531e20dd7c1a1a0 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 13:11:45.078474 30785 tablet_service.cc:1511] Processing CreateTablet for tablet efddea050a344563877c3e01ee2a4c81 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:11:45.079854 30785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efddea050a344563877c3e01ee2a4c81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.083087 30861 tablet_service.cc:1511] Processing CreateTablet for tablet 437c20f3d53f43048eca57834248cc0d (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:11:45.087124 30713 tablet_service.cc:1511] Processing CreateTablet for tablet bc2a86f27b0f4fb59d7cfa586a23fef1 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 13:11:45.087626 30784 tablet_service.cc:1511] Processing CreateTablet for tablet 7ed3eac558b34457aa26b1c40652f887 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:11:45.089027 30784 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ed3eac558b34457aa26b1c40652f887. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.089984 30783 tablet_service.cc:1511] Processing CreateTablet for tablet 283eb8311aeb4b5599dd844da98ee0b7 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:11:45.091392 30783 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 283eb8311aeb4b5599dd844da98ee0b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.091734 30712 tablet_service.cc:1511] Processing CreateTablet for tablet 39980eee51514ba493958931f406a145 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:11:45.085841 30860 tablet_service.cc:1511] Processing CreateTablet for tablet 11c7485c290740bba00c8b9eaf9b917f (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:11:45.092309 30859 tablet_service.cc:1511] Processing CreateTablet for tablet efddea050a344563877c3e01ee2a4c81 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:11:45.092747 30858 tablet_service.cc:1511] Processing CreateTablet for tablet 0ac93952b93645b1ad1d7d96cb60c3e9 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:11:45.093715 30716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e28f28c3f05f40e68531e20dd7c1a1a0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.094476 30712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39980eee51514ba493958931f406a145. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.095830 30857 tablet_service.cc:1511] Processing CreateTablet for tablet 283eb8311aeb4b5599dd844da98ee0b7 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:11:45.097036 30713 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc2a86f27b0f4fb59d7cfa586a23fef1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.084524 30861 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 437c20f3d53f43048eca57834248cc0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.097898 30859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efddea050a344563877c3e01ee2a4c81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.098522 30858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ac93952b93645b1ad1d7d96cb60c3e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.099176 30860 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11c7485c290740bba00c8b9eaf9b917f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.099840 30857 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 283eb8311aeb4b5599dd844da98ee0b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.115412 30711 tablet_service.cc:1511] Processing CreateTablet for tablet 437c20f3d53f43048eca57834248cc0d (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:11:45.122215 30710 tablet_service.cc:1511] Processing CreateTablet for tablet efddea050a344563877c3e01ee2a4c81 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 13:11:45.123179 30711 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 437c20f3d53f43048eca57834248cc0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.123878 30710 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet efddea050a344563877c3e01ee2a4c81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.138582 30939 tablet_service.cc:1511] Processing CreateTablet for tablet b226c44412c74106a0feb2ca411b1661 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 13:11:45.140112 30939 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b226c44412c74106a0feb2ca411b1661. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.141819 30938 tablet_service.cc:1511] Processing CreateTablet for tablet 7b988f0495c34b7591ad2fb0d5f65693 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 13:11:45.143224 30938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b988f0495c34b7591ad2fb0d5f65693. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.145903 30937 tablet_service.cc:1511] Processing CreateTablet for tablet bcac31d5b7ab48ec9c071a8f77750a88 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 13:11:45.147330 30937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcac31d5b7ab48ec9c071a8f77750a88. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.150939 30936 tablet_service.cc:1511] Processing CreateTablet for tablet 39980eee51514ba493958931f406a145 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 13:11:45.152288 30936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 39980eee51514ba493958931f406a145. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.157733 30934 tablet_service.cc:1511] Processing CreateTablet for tablet 11c7485c290740bba00c8b9eaf9b917f (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 13:11:45.159173 30934 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11c7485c290740bba00c8b9eaf9b917f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.159731 30709 tablet_service.cc:1511] Processing CreateTablet for tablet 0ac93952b93645b1ad1d7d96cb60c3e9 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:11:45.161464 30933 tablet_service.cc:1511] Processing CreateTablet for tablet 7ed3eac558b34457aa26b1c40652f887 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:11:45.163059 30933 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ed3eac558b34457aa26b1c40652f887. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.167387 30932 tablet_service.cc:1511] Processing CreateTablet for tablet 0ac93952b93645b1ad1d7d96cb60c3e9 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 13:11:45.156502 30935 tablet_service.cc:1511] Processing CreateTablet for tablet 437c20f3d53f43048eca57834248cc0d (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 13:11:45.182399 30932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ac93952b93645b1ad1d7d96cb60c3e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.182925 30931 tablet_service.cc:1511] Processing CreateTablet for tablet 283eb8311aeb4b5599dd844da98ee0b7 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 13:11:45.185345 30709 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0ac93952b93645b1ad1d7d96cb60c3e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.187060 30931 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 283eb8311aeb4b5599dd844da98ee0b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.193064 30935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 437c20f3d53f43048eca57834248cc0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.193375 30789 tablet_service.cc:1511] Processing CreateTablet for tablet 1474b6fc151c4a3994a3aef89ea936a2 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:11:45.194790 30789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1474b6fc151c4a3994a3aef89ea936a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.194643 30708 tablet_service.cc:1511] Processing CreateTablet for tablet 1474b6fc151c4a3994a3aef89ea936a2 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:11:45.196089 30708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1474b6fc151c4a3994a3aef89ea936a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.197574 30716 tablet_service.cc:1511] Processing CreateTablet for tablet 7ed3eac558b34457aa26b1c40652f887 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 13:11:45.197693 30856 tablet_service.cc:1511] Processing CreateTablet for tablet 1474b6fc151c4a3994a3aef89ea936a2 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 13:11:45.199259 30856 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1474b6fc151c4a3994a3aef89ea936a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.201818 30716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ed3eac558b34457aa26b1c40652f887. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.213819 30930 tablet_service.cc:1511] Processing CreateTablet for tablet 4fb3f8d627b04c8a8971f7e4d3a5c010 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:11:45.215279 30930 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fb3f8d627b04c8a8971f7e4d3a5c010. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.217911 30994 tablet_bootstrap.cc:492] T b226c44412c74106a0feb2ca411b1661 P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
I20260709 13:11:45.233372 30995 tablet_bootstrap.cc:492] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
W20260709 13:11:45.245554 30671 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:11:45.248371 30712 tablet_service.cc:1511] Processing CreateTablet for tablet 4fb3f8d627b04c8a8971f7e4d3a5c010 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:11:45.249786 30712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fb3f8d627b04c8a8971f7e4d3a5c010. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.272145 30865 tablet_service.cc:1511] Processing CreateTablet for tablet 27f26e218737414893d5c4bfa4b2cd94 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:11:45.273921 30865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27f26e218737414893d5c4bfa4b2cd94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.275928 30714 tablet_service.cc:1511] Processing CreateTablet for tablet 27f26e218737414893d5c4bfa4b2cd94 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:11:45.277248 30714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27f26e218737414893d5c4bfa4b2cd94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.270094 30786 tablet_service.cc:1511] Processing CreateTablet for tablet 4fb3f8d627b04c8a8971f7e4d3a5c010 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 13:11:45.296468 30996 tablet_bootstrap.cc:492] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
I20260709 13:11:45.297410 30786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4fb3f8d627b04c8a8971f7e4d3a5c010. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.299960 30864 tablet_service.cc:1511] Processing CreateTablet for tablet d4dbe5b2d724412a909fd0641d1a213a (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:11:45.297111 30788 tablet_service.cc:1511] Processing CreateTablet for tablet d4dbe5b2d724412a909fd0641d1a213a (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:11:45.302088 30788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4dbe5b2d724412a909fd0641d1a213a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.304982 30994 tablet_bootstrap.cc:654] T b226c44412c74106a0feb2ca411b1661 P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:45.313223 30864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4dbe5b2d724412a909fd0641d1a213a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.324016 30715 tablet_service.cc:1511] Processing CreateTablet for tablet d05da516ed9040c988b1fdfe2fdde9ed (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:11:45.325399 30715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d05da516ed9040c988b1fdfe2fdde9ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.332706 30936 tablet_service.cc:1511] Processing CreateTablet for tablet 27f26e218737414893d5c4bfa4b2cd94 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 13:11:45.335108 30935 tablet_service.cc:1511] Processing CreateTablet for tablet d4dbe5b2d724412a909fd0641d1a213a (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 13:11:45.337262 30787 tablet_service.cc:1511] Processing CreateTablet for tablet d05da516ed9040c988b1fdfe2fdde9ed (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:11:45.338505 30787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d05da516ed9040c988b1fdfe2fdde9ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.338920 30936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 27f26e218737414893d5c4bfa4b2cd94. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.349792 30996 tablet_bootstrap.cc:654] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:45.357321 30862 tablet_service.cc:1511] Processing CreateTablet for tablet d05da516ed9040c988b1fdfe2fdde9ed (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 13:11:45.358676 30862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d05da516ed9040c988b1fdfe2fdde9ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.361610 30995 tablet_bootstrap.cc:654] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:45.367952 30933 tablet_service.cc:1511] Processing CreateTablet for tablet bbb2acd63baa48a3a2ace94016522e92 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:11:45.380156 30935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d4dbe5b2d724412a909fd0641d1a213a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.384253 30997 tablet_bootstrap.cc:492] T bcac31d5b7ab48ec9c071a8f77750a88 P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
I20260709 13:11:45.399039 30933 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbb2acd63baa48a3a2ace94016522e92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.415560 30995 tablet_bootstrap.cc:492] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:45.415990 30996 tablet_bootstrap.cc:492] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
I20260709 13:11:45.435662 30996 ts_tablet_manager.cc:1403] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.140s	user 0.010s	sys 0.035s
I20260709 13:11:45.436810 30751 heartbeater.cc:499] Master 127.21.63.62:40095 was elected leader, sending a full tablet report...
I20260709 13:11:45.417917 30994 tablet_bootstrap.cc:492] T b226c44412c74106a0feb2ca411b1661 P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
I20260709 13:11:45.417623 30716 tablet_service.cc:1511] Processing CreateTablet for tablet bbb2acd63baa48a3a2ace94016522e92 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:11:45.419374 30997 tablet_bootstrap.cc:654] T bcac31d5b7ab48ec9c071a8f77750a88 P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:45.439355 30996 raft_consensus.cc:359] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:45.440817 30996 raft_consensus.cc:385] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:45.440958 30716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbb2acd63baa48a3a2ace94016522e92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.441151 30996 raft_consensus.cc:740] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
I20260709 13:11:45.442041 30996 consensus_queue.cc:260] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:45.421996 30786 tablet_service.cc:1511] Processing CreateTablet for tablet bbb2acd63baa48a3a2ace94016522e92 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 13:11:45.425593 30938 tablet_service.cc:1511] Processing CreateTablet for tablet 8770fcfd98154b2095c2c9f74135593b (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:11:45.444115 30786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bbb2acd63baa48a3a2ace94016522e92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.444223 30938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8770fcfd98154b2095c2c9f74135593b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.426789 30714 tablet_service.cc:1511] Processing CreateTablet for tablet 8770fcfd98154b2095c2c9f74135593b (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:11:45.447335 30714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8770fcfd98154b2095c2c9f74135593b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.447733 30995 ts_tablet_manager.cc:1403] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.215s	user 0.004s	sys 0.025s
I20260709 13:11:45.424021 30859 tablet_service.cc:1511] Processing CreateTablet for tablet 8770fcfd98154b2095c2c9f74135593b (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 13:11:45.449710 30859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8770fcfd98154b2095c2c9f74135593b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.450913 30995 raft_consensus.cc:359] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:45.451700 30995 raft_consensus.cc:385] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:45.451992 30995 raft_consensus.cc:740] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:45.428031 30788 tablet_service.cc:1511] Processing CreateTablet for tablet c02f992ce0974544a4112672d8792c5f (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:11:45.458580 30933 tablet_service.cc:1511] Processing CreateTablet for tablet c02f992ce0974544a4112672d8792c5f (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:11:45.459604 30788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c02f992ce0974544a4112672d8792c5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.440233 30994 ts_tablet_manager.cc:1403] T b226c44412c74106a0feb2ca411b1661 P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.223s	user 0.022s	sys 0.016s
I20260709 13:11:45.469614 30715 tablet_service.cc:1511] Processing CreateTablet for tablet 64a95fa1266848048486722c02f0d4cf (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:11:45.471122 30715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64a95fa1266848048486722c02f0d4cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.462173 30865 tablet_service.cc:1511] Processing CreateTablet for tablet c02f992ce0974544a4112672d8792c5f (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 13:11:45.471266 30995 consensus_queue.cc:260] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:45.472959 30865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c02f992ce0974544a4112672d8792c5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.478154 30787 tablet_service.cc:1511] Processing CreateTablet for tablet 64a95fa1266848048486722c02f0d4cf (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:11:45.479487 30787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64a95fa1266848048486722c02f0d4cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.480088 30994 raft_consensus.cc:359] T b226c44412c74106a0feb2ca411b1661 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:45.485473 30994 raft_consensus.cc:385] T b226c44412c74106a0feb2ca411b1661 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:45.485846 30994 raft_consensus.cc:740] T b226c44412c74106a0feb2ca411b1661 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
I20260709 13:11:45.486693 30994 consensus_queue.cc:260] T b226c44412c74106a0feb2ca411b1661 P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:45.492579 30708 tablet_service.cc:1511] Processing CreateTablet for tablet d7edce5d08f24852b7afca9c57ba9c19 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:11:45.500615 30789 tablet_service.cc:1511] Processing CreateTablet for tablet d7edce5d08f24852b7afca9c57ba9c19 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:11:45.501350 30994 ts_tablet_manager.cc:1434] T b226c44412c74106a0feb2ca411b1661 P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.033s	user 0.005s	sys 0.005s
I20260709 13:11:45.506143 30997 tablet_bootstrap.cc:492] T bcac31d5b7ab48ec9c071a8f77750a88 P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
I20260709 13:11:45.506682 30789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7edce5d08f24852b7afca9c57ba9c19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.506966 30708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7edce5d08f24852b7afca9c57ba9c19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.507030 30933 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c02f992ce0974544a4112672d8792c5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.508723 30936 tablet_service.cc:1511] Processing CreateTablet for tablet d7edce5d08f24852b7afca9c57ba9c19 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 13:11:45.509661 30937 tablet_service.cc:1511] Processing CreateTablet for tablet 726b808ed64848dd89da4340d4d891fb (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:11:45.513731 30862 tablet_service.cc:1511] Processing CreateTablet for tablet 726b808ed64848dd89da4340d4d891fb (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:11:45.515146 30862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 726b808ed64848dd89da4340d4d891fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.519389 30826 heartbeater.cc:499] Master 127.21.63.62:40095 was elected leader, sending a full tablet report...
W20260709 13:11:45.520802 30827 tablet.cc:2406] T b226c44412c74106a0feb2ca411b1661 P 05213ac836744a53b9c089a5c052f4be: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:11:45.522097 30712 tablet_service.cc:1511] Processing CreateTablet for tablet 726b808ed64848dd89da4340d4d891fb (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 13:11:45.523365 30712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 726b808ed64848dd89da4340d4d891fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.527609 30936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7edce5d08f24852b7afca9c57ba9c19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.529024 30937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 726b808ed64848dd89da4340d4d891fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.532248 30994 tablet_bootstrap.cc:492] T e28f28c3f05f40e68531e20dd7c1a1a0 P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
I20260709 13:11:45.533806 30997 ts_tablet_manager.cc:1403] T bcac31d5b7ab48ec9c071a8f77750a88 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.150s	user 0.017s	sys 0.016s
I20260709 13:11:45.548841 30900 heartbeater.cc:499] Master 127.21.63.62:40095 was elected leader, sending a full tablet report...
I20260709 13:11:45.554121 30791 tablet_service.cc:1511] Processing CreateTablet for tablet ccc68dd4c56548cba0624daafb5ae23c (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:11:45.555540 30791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccc68dd4c56548cba0624daafb5ae23c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.556399 30935 tablet_service.cc:1511] Processing CreateTablet for tablet ccc68dd4c56548cba0624daafb5ae23c (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:11:45.561096 31004 raft_consensus.cc:493] T b226c44412c74106a0feb2ca411b1661 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.561719 31004 raft_consensus.cc:515] T b226c44412c74106a0feb2ca411b1661 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:45.561877 30856 tablet_service.cc:1511] Processing CreateTablet for tablet ccc68dd4c56548cba0624daafb5ae23c (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 13:11:45.563323 30856 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccc68dd4c56548cba0624daafb5ae23c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.570888 30935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ccc68dd4c56548cba0624daafb5ae23c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.571005 31004 leader_election.cc:290] T b226c44412c74106a0feb2ca411b1661 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:45.586131 30714 tablet_service.cc:1511] Processing CreateTablet for tablet 5028253a91e74a73afca005c77e8b565 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:11:45.587545 30714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5028253a91e74a73afca005c77e8b565. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.594278 30995 ts_tablet_manager.cc:1434] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.146s	user 0.013s	sys 0.006s
I20260709 13:11:45.595449 30995 tablet_bootstrap.cc:492] T 39980eee51514ba493958931f406a145 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:45.596887 30864 tablet_service.cc:1511] Processing CreateTablet for tablet 64a95fa1266848048486722c02f0d4cf (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 13:11:45.598250 30864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 64a95fa1266848048486722c02f0d4cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.597411 30996 ts_tablet_manager.cc:1434] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.161s	user 0.016s	sys 0.011s
I20260709 13:11:45.597904 30997 raft_consensus.cc:359] T bcac31d5b7ab48ec9c071a8f77750a88 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:45.601537 30997 raft_consensus.cc:385] T bcac31d5b7ab48ec9c071a8f77750a88 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:45.601943 30997 raft_consensus.cc:740] T bcac31d5b7ab48ec9c071a8f77750a88 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
I20260709 13:11:45.610394 30995 tablet_bootstrap.cc:654] T 39980eee51514ba493958931f406a145 P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:45.610110 30858 tablet_service.cc:1511] Processing CreateTablet for tablet 5028253a91e74a73afca005c77e8b565 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:11:45.611464 30858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5028253a91e74a73afca005c77e8b565. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.615149 30938 tablet_service.cc:1511] Processing CreateTablet for tablet 26172e2c14cd4d178764ed13012c21f7 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:11:45.616405 30938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26172e2c14cd4d178764ed13012c21f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.617632 30715 tablet_service.cc:1511] Processing CreateTablet for tablet 26172e2c14cd4d178764ed13012c21f7 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:11:45.618887 30715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26172e2c14cd4d178764ed13012c21f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.639035 30994 tablet_bootstrap.cc:654] T e28f28c3f05f40e68531e20dd7c1a1a0 P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:45.602800 30997 consensus_queue.cc:260] T bcac31d5b7ab48ec9c071a8f77750a88 P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:45.660161 30932 tablet_service.cc:1511] Processing CreateTablet for tablet 01cceb88bca348958b59f3697979dff1 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:11:45.663676 30932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01cceb88bca348958b59f3697979dff1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.660858 30996 tablet_bootstrap.cc:492] T b226c44412c74106a0feb2ca411b1661 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
I20260709 13:11:45.669629 30949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b226c44412c74106a0feb2ca411b1661" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:45.670462 30788 tablet_service.cc:1511] Processing CreateTablet for tablet 26172e2c14cd4d178764ed13012c21f7 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 13:11:45.662075 30712 tablet_service.cc:1511] Processing CreateTablet for tablet 01cceb88bca348958b59f3697979dff1 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:11:45.672067 30788 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 26172e2c14cd4d178764ed13012c21f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.672137 30712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01cceb88bca348958b59f3697979dff1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:11:45.684899 30765 leader_election.cc:343] T b226c44412c74106a0feb2ca411b1661 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:45.691690 30787 tablet_service.cc:1511] Processing CreateTablet for tablet c1efa319af774b23bd4ea782611f5252 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:11:45.693163 30787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1efa319af774b23bd4ea782611f5252. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.699440 30995 tablet_bootstrap.cc:492] T 39980eee51514ba493958931f406a145 P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:45.699548 30875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b226c44412c74106a0feb2ca411b1661" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:45.700120 30995 ts_tablet_manager.cc:1403] T 39980eee51514ba493958931f406a145 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.105s	user 0.009s	sys 0.004s
W20260709 13:11:45.701268 30766 leader_election.cc:343] T b226c44412c74106a0feb2ca411b1661 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:45.701792 30766 leader_election.cc:304] T b226c44412c74106a0feb2ca411b1661 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 262942f67dfa4b8fa42234b18ac31ffd
I20260709 13:11:45.702630 31004 raft_consensus.cc:2749] T b226c44412c74106a0feb2ca411b1661 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:45.705446 30995 raft_consensus.cc:359] T 39980eee51514ba493958931f406a145 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:45.706496 30995 raft_consensus.cc:385] T 39980eee51514ba493958931f406a145 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:45.706874 30995 raft_consensus.cc:740] T 39980eee51514ba493958931f406a145 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:45.707324 30789 tablet_service.cc:1511] Processing CreateTablet for tablet 5028253a91e74a73afca005c77e8b565 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 13:11:45.708736 30789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5028253a91e74a73afca005c77e8b565. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.708045 30995 consensus_queue.cc:260] T 39980eee51514ba493958931f406a145 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:45.711177 31003 raft_consensus.cc:493] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.659072 30858 tablet_service.cc:1511] Processing CreateTablet for tablet 01cceb88bca348958b59f3697979dff1 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 13:11:45.712182 31003 raft_consensus.cc:515] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:45.716146 30858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 01cceb88bca348958b59f3697979dff1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.719528 30974 heartbeater.cc:499] Master 127.21.63.62:40095 was elected leader, sending a full tablet report...
I20260709 13:11:45.719844 30864 tablet_service.cc:1511] Processing CreateTablet for tablet c1efa319af774b23bd4ea782611f5252 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:11:45.721580 31003 leader_election.cc:290] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:45.741734 30937 tablet_service.cc:1511] Processing CreateTablet for tablet c1efa319af774b23bd4ea782611f5252 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 13:11:45.744052 30937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1efa319af774b23bd4ea782611f5252. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.730394 30864 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c1efa319af774b23bd4ea782611f5252. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.753230 30712 tablet_service.cc:1511] Processing CreateTablet for tablet 892368b6886842febffc849d73f76c10 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:11:45.747366 30994 tablet_bootstrap.cc:492] T e28f28c3f05f40e68531e20dd7c1a1a0 P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
I20260709 13:11:45.761430 30994 ts_tablet_manager.cc:1403] T e28f28c3f05f40e68531e20dd7c1a1a0 P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.229s	user 0.005s	sys 0.010s
I20260709 13:11:45.764887 30994 raft_consensus.cc:359] T e28f28c3f05f40e68531e20dd7c1a1a0 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:45.766263 30994 raft_consensus.cc:385] T e28f28c3f05f40e68531e20dd7c1a1a0 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:45.756584 30712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 892368b6886842febffc849d73f76c10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.760208 30862 tablet_service.cc:1511] Processing CreateTablet for tablet 892368b6886842febffc849d73f76c10 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:11:45.765175 30933 tablet_service.cc:1511] Processing CreateTablet for tablet a38ff14550f74527a62591df59fb2c51 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:11:45.768720 30708 tablet_service.cc:1511] Processing CreateTablet for tablet a38ff14550f74527a62591df59fb2c51 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:11:45.750764 30996 tablet_bootstrap.cc:654] T b226c44412c74106a0feb2ca411b1661 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:45.776918 30856 tablet_service.cc:1511] Processing CreateTablet for tablet 93a75acc8893414180a15797033a2d5a (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:11:45.749979 30995 ts_tablet_manager.cc:1434] T 39980eee51514ba493958931f406a145 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.049s	user 0.009s	sys 0.002s
I20260709 13:11:45.782680 30708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a38ff14550f74527a62591df59fb2c51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.785413 30715 tablet_service.cc:1511] Processing CreateTablet for tablet 93a75acc8893414180a15797033a2d5a (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:11:45.785979 30862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 892368b6886842febffc849d73f76c10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.786717 30715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93a75acc8893414180a15797033a2d5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.789706 30865 tablet_service.cc:1511] Processing CreateTablet for tablet 054fc96e257a4dc6b2ef90db5c1ad499 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:11:45.790086 30791 tablet_service.cc:1511] Processing CreateTablet for tablet a38ff14550f74527a62591df59fb2c51 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 13:11:45.791071 30865 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 054fc96e257a4dc6b2ef90db5c1ad499. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.790717 30935 tablet_service.cc:1511] Processing CreateTablet for tablet 054fc96e257a4dc6b2ef90db5c1ad499 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:11:45.791321 30791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a38ff14550f74527a62591df59fb2c51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.786417 30936 tablet_service.cc:1511] Processing CreateTablet for tablet 93a75acc8893414180a15797033a2d5a (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 13:11:45.797403 30995 tablet_bootstrap.cc:492] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:45.774099 30994 raft_consensus.cc:740] T e28f28c3f05f40e68531e20dd7c1a1a0 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
I20260709 13:11:45.799731 30994 consensus_queue.cc:260] T e28f28c3f05f40e68531e20dd7c1a1a0 P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:45.804513 30994 ts_tablet_manager.cc:1434] T e28f28c3f05f40e68531e20dd7c1a1a0 P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.043s	user 0.006s	sys 0.000s
I20260709 13:11:45.805577 30994 tablet_bootstrap.cc:492] T 7b988f0495c34b7591ad2fb0d5f65693 P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
I20260709 13:11:45.805610 30856 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93a75acc8893414180a15797033a2d5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.809263 30933 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a38ff14550f74527a62591df59fb2c51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.781265 30997 ts_tablet_manager.cc:1434] T bcac31d5b7ab48ec9c071a8f77750a88 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.247s	user 0.012s	sys 0.025s
I20260709 13:11:45.814040 30997 tablet_bootstrap.cc:492] T 39980eee51514ba493958931f406a145 P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
I20260709 13:11:45.814093 30786 tablet_service.cc:1511] Processing CreateTablet for tablet 054fc96e257a4dc6b2ef90db5c1ad499 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 13:11:45.815253 31009 raft_consensus.cc:493] T bcac31d5b7ab48ec9c071a8f77750a88 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.815680 30786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 054fc96e257a4dc6b2ef90db5c1ad499. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.815790 31009 raft_consensus.cc:515] T bcac31d5b7ab48ec9c071a8f77750a88 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:45.820411 31009 leader_election.cc:290] T bcac31d5b7ab48ec9c071a8f77750a88 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:45.841897 30997 tablet_bootstrap.cc:654] T 39980eee51514ba493958931f406a145 P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:45.845933 30714 tablet_service.cc:1511] Processing CreateTablet for tablet 0d8c1da1f7254fde94f7deaad5f41709 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:11:45.848351 30996 tablet_bootstrap.cc:492] T b226c44412c74106a0feb2ca411b1661 P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
I20260709 13:11:45.849182 30996 ts_tablet_manager.cc:1403] T b226c44412c74106a0feb2ca411b1661 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.205s	user 0.014s	sys 0.018s
I20260709 13:11:45.853135 30935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 054fc96e257a4dc6b2ef90db5c1ad499. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.860340 30996 raft_consensus.cc:359] T b226c44412c74106a0feb2ca411b1661 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:45.862776 30996 raft_consensus.cc:385] T b226c44412c74106a0feb2ca411b1661 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:45.863794 30996 raft_consensus.cc:740] T b226c44412c74106a0feb2ca411b1661 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
I20260709 13:11:45.867597 30996 consensus_queue.cc:260] T b226c44412c74106a0feb2ca411b1661 P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:45.869508 30785 tablet_service.cc:1511] Processing CreateTablet for tablet 0d8c1da1f7254fde94f7deaad5f41709 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:11:45.870237 30714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d8c1da1f7254fde94f7deaad5f41709. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.870548 30994 tablet_bootstrap.cc:654] T 7b988f0495c34b7591ad2fb0d5f65693 P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:45.870831 30785 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d8c1da1f7254fde94f7deaad5f41709. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.872862 30936 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93a75acc8893414180a15797033a2d5a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.883888 31002 raft_consensus.cc:493] T b226c44412c74106a0feb2ca411b1661 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.884477 31002 raft_consensus.cc:515] T b226c44412c74106a0feb2ca411b1661 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:45.886552 30859 tablet_service.cc:1511] Processing CreateTablet for tablet 0d8c1da1f7254fde94f7deaad5f41709 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 13:11:45.887933 30859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d8c1da1f7254fde94f7deaad5f41709. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.891144 31003 raft_consensus.cc:493] T 39980eee51514ba493958931f406a145 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.858111 30789 tablet_service.cc:1511] Processing CreateTablet for tablet 892368b6886842febffc849d73f76c10 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 13:11:45.889696 31002 leader_election.cc:290] T b226c44412c74106a0feb2ca411b1661 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:45.897821 30801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e28f28c3f05f40e68531e20dd7c1a1a0" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:45.911095 30801 raft_consensus.cc:2468] T e28f28c3f05f40e68531e20dd7c1a1a0 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 0.
I20260709 13:11:45.912585 30689 leader_election.cc:304] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:45.913578 30995 tablet_bootstrap.cc:654] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:45.909406 31002 raft_consensus.cc:493] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:45.895515 31003 raft_consensus.cc:515] T 39980eee51514ba493958931f406a145 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:45.937882 30716 tablet_service.cc:1511] Processing CreateTablet for tablet b3754a670e0b4ed99d37283bf1647bf5 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:11:45.939352 30716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3754a670e0b4ed99d37283bf1647bf5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.943390 30938 tablet_service.cc:1511] Processing CreateTablet for tablet 92a4b4a5a07442f49f518b479db8c94b (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:11:45.944904 30996 ts_tablet_manager.cc:1434] T b226c44412c74106a0feb2ca411b1661 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.095s	user 0.006s	sys 0.004s
I20260709 13:11:45.924396 30932 tablet_service.cc:1511] Processing CreateTablet for tablet b3754a670e0b4ed99d37283bf1647bf5 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:11:45.945966 30996 tablet_bootstrap.cc:492] T e28f28c3f05f40e68531e20dd7c1a1a0 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
I20260709 13:11:45.949196 30938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92a4b4a5a07442f49f518b479db8c94b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.952620 30791 tablet_service.cc:1511] Processing CreateTablet for tablet b3754a670e0b4ed99d37283bf1647bf5 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 13:11:45.954649 31003 leader_election.cc:290] T 39980eee51514ba493958931f406a145 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:45.955250 31003 raft_consensus.cc:2804] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:45.955608 31003 raft_consensus.cc:493] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:45.955979 31003 raft_consensus.cc:3060] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:45.932598 31002 raft_consensus.cc:515] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:45.960382 31002 leader_election.cc:290] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:45.973342 30875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e28f28c3f05f40e68531e20dd7c1a1a0" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:45.960521 30712 tablet_service.cc:1511] Processing CreateTablet for tablet 92a4b4a5a07442f49f518b479db8c94b (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:11:45.964135 30932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3754a670e0b4ed99d37283bf1647bf5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.974504 30787 tablet_service.cc:1511] Processing CreateTablet for tablet b9fcfade988746a2967e115dbbcfe0f1 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:11:45.966989 30789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 892368b6886842febffc849d73f76c10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.977259 30787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9fcfade988746a2967e115dbbcfe0f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:45.995919 30791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3754a670e0b4ed99d37283bf1647bf5. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:11:45.989236 30689 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:43771, user_credentials={real_user=slave}} blocked reactor thread for 76856us
I20260709 13:11:46.006407 30801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39980eee51514ba493958931f406a145" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:46.009142 30937 tablet_service.cc:1511] Processing CreateTablet for tablet b9fcfade988746a2967e115dbbcfe0f1 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:11:46.010787 30937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9fcfade988746a2967e115dbbcfe0f1. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:11:46.017972 30691 leader_election.cc:343] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:46.018149 30863 tablet_service.cc:1511] Processing CreateTablet for tablet 92a4b4a5a07442f49f518b479db8c94b (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 13:11:46.019191 30995 tablet_bootstrap.cc:492] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:46.020220 30712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92a4b4a5a07442f49f518b479db8c94b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:11:46.023339 30689 leader_election.cc:343] T 39980eee51514ba493958931f406a145 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:46.031690 30863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 92a4b4a5a07442f49f518b479db8c94b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.033536 30726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcac31d5b7ab48ec9c071a8f77750a88" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
W20260709 13:11:46.035207 30915 leader_election.cc:343] T bcac31d5b7ab48ec9c071a8f77750a88 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:46.035881 30995 ts_tablet_manager.cc:1403] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.239s	user 0.009s	sys 0.013s
I20260709 13:11:46.046310 30715 tablet_service.cc:1511] Processing CreateTablet for tablet 5532e197e7de48f28c034ba1fc490e19 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:11:46.051362 30786 tablet_service.cc:1511] Processing CreateTablet for tablet 5532e197e7de48f28c034ba1fc490e19 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:11:46.053436 30786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5532e197e7de48f28c034ba1fc490e19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.053962 30949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39980eee51514ba493958931f406a145" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:46.054553 30948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b226c44412c74106a0feb2ca411b1661" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:46.039160 30995 raft_consensus.cc:359] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
W20260709 13:11:46.056169 30690 leader_election.cc:343] T 39980eee51514ba493958931f406a145 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:46.056623 30995 raft_consensus.cc:385] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:46.056882 30690 leader_election.cc:304] T 39980eee51514ba493958931f406a145 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22c9d3d15904327a78a0e548a8ff6cd; no voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd
I20260709 13:11:46.058400 31022 raft_consensus.cc:2749] T 39980eee51514ba493958931f406a145 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:46.059182 30997 tablet_bootstrap.cc:492] T 39980eee51514ba493958931f406a145 P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
I20260709 13:11:46.059765 30997 ts_tablet_manager.cc:1403] T 39980eee51514ba493958931f406a145 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.246s	user 0.009s	sys 0.013s
I20260709 13:11:46.063019 30997 raft_consensus.cc:359] T 39980eee51514ba493958931f406a145 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:46.063818 30997 raft_consensus.cc:385] T 39980eee51514ba493958931f406a145 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:46.064224 30715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5532e197e7de48f28c034ba1fc490e19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.057135 30995 raft_consensus.cc:740] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:46.065011 30996 tablet_bootstrap.cc:654] T e28f28c3f05f40e68531e20dd7c1a1a0 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:46.065410 30995 consensus_queue.cc:260] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:46.067915 30995 ts_tablet_manager.cc:1434] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.032s	user 0.005s	sys 0.000s
I20260709 13:11:46.069041 30995 tablet_bootstrap.cc:492] T 7b988f0495c34b7591ad2fb0d5f65693 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:46.073997 30875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcac31d5b7ab48ec9c071a8f77750a88" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
W20260709 13:11:46.075548 30914 leader_election.cc:343] T bcac31d5b7ab48ec9c071a8f77750a88 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:46.076045 30914 leader_election.cc:304] T bcac31d5b7ab48ec9c071a8f77750a88 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:46.076936 31009 raft_consensus.cc:2749] T bcac31d5b7ab48ec9c071a8f77750a88 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:46.079808 30994 tablet_bootstrap.cc:492] T 7b988f0495c34b7591ad2fb0d5f65693 P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
I20260709 13:11:46.080425 30994 ts_tablet_manager.cc:1403] T 7b988f0495c34b7591ad2fb0d5f65693 P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.275s	user 0.018s	sys 0.001s
I20260709 13:11:46.064147 30997 raft_consensus.cc:740] T 39980eee51514ba493958931f406a145 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
I20260709 13:11:46.082504 30801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b226c44412c74106a0feb2ca411b1661" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:46.086045 30801 raft_consensus.cc:2468] T b226c44412c74106a0feb2ca411b1661 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 0.
I20260709 13:11:46.083487 30800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2a86f27b0f4fb59d7cfa586a23fef1" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
W20260709 13:11:46.056511 30839 leader_election.cc:343] T b226c44412c74106a0feb2ca411b1661 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:11:46.089864 30838 leader_election.cc:343] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:46.090777 30838 leader_election.cc:304] T b226c44412c74106a0feb2ca411b1661 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 05213ac836744a53b9c089a5c052f4be; no voters: 262942f67dfa4b8fa42234b18ac31ffd
I20260709 13:11:46.092269 31027 raft_consensus.cc:2804] T b226c44412c74106a0feb2ca411b1661 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:46.092818 31027 raft_consensus.cc:493] T b226c44412c74106a0feb2ca411b1661 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:46.072192 31003 raft_consensus.cc:515] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:46.093200 31027 raft_consensus.cc:3060] T b226c44412c74106a0feb2ca411b1661 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:46.095826 30801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e28f28c3f05f40e68531e20dd7c1a1a0" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:46.096534 30801 raft_consensus.cc:3060] T e28f28c3f05f40e68531e20dd7c1a1a0 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:46.086087 30997 consensus_queue.cc:260] T 39980eee51514ba493958931f406a145 P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:46.099036 30997 ts_tablet_manager.cc:1434] T 39980eee51514ba493958931f406a145 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.039s	user 0.007s	sys 0.003s
I20260709 13:11:46.105005 30997 tablet_bootstrap.cc:492] T 437c20f3d53f43048eca57834248cc0d P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
I20260709 13:11:46.107683 30801 raft_consensus.cc:2468] T e28f28c3f05f40e68531e20dd7c1a1a0 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 1.
I20260709 13:11:46.109165 30689 leader_election.cc:304] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:46.109990 31022 raft_consensus.cc:2804] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:46.111114 30994 raft_consensus.cc:359] T 7b988f0495c34b7591ad2fb0d5f65693 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:46.111927 30994 raft_consensus.cc:385] T 7b988f0495c34b7591ad2fb0d5f65693 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:46.112239 30994 raft_consensus.cc:740] T 7b988f0495c34b7591ad2fb0d5f65693 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
I20260709 13:11:46.118427 30726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2a86f27b0f4fb59d7cfa586a23fef1" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:46.118770 30875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e28f28c3f05f40e68531e20dd7c1a1a0" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
W20260709 13:11:46.120003 30691 leader_election.cc:343] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:11:46.120429 30841 leader_election.cc:343] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:46.121171 30841 leader_election.cc:304] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a; no voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:46.122099 31002 raft_consensus.cc:2749] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:46.125304 31003 leader_election.cc:290] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 election: Requested vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:46.120438 30995 tablet_bootstrap.cc:654] T 7b988f0495c34b7591ad2fb0d5f65693 P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:46.131369 30856 tablet_service.cc:1511] Processing CreateTablet for tablet b9fcfade988746a2967e115dbbcfe0f1 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 13:11:46.134886 30858 tablet_service.cc:1511] Processing CreateTablet for tablet 5532e197e7de48f28c034ba1fc490e19 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 13:11:46.137599 30996 tablet_bootstrap.cc:492] T e28f28c3f05f40e68531e20dd7c1a1a0 P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
I20260709 13:11:46.143705 30997 tablet_bootstrap.cc:654] T 437c20f3d53f43048eca57834248cc0d P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:46.141091 30994 consensus_queue.cc:260] T 7b988f0495c34b7591ad2fb0d5f65693 P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:46.132925 30856 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9fcfade988746a2967e115dbbcfe0f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.141444 30862 tablet_service.cc:1511] Processing CreateTablet for tablet 784deee1ea9543df840c58ecaa6f911a (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:11:46.161839 30994 ts_tablet_manager.cc:1434] T 7b988f0495c34b7591ad2fb0d5f65693 P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.075s	user 0.005s	sys 0.006s
I20260709 13:11:46.163031 30994 tablet_bootstrap.cc:492] T 7ed3eac558b34457aa26b1c40652f887 P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
I20260709 13:11:46.153682 31022 raft_consensus.cc:697] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 LEADER]: Becoming Leader. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Running, Role: LEADER
I20260709 13:11:46.155251 30938 tablet_service.cc:1511] Processing CreateTablet for tablet 784deee1ea9543df840c58ecaa6f911a (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:11:46.167501 30938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 784deee1ea9543df840c58ecaa6f911a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.168371 31031 raft_consensus.cc:493] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:46.168807 30858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5532e197e7de48f28c034ba1fc490e19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.168948 31031 raft_consensus.cc:515] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:46.172060 30875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcac31d5b7ab48ec9c071a8f77750a88" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:46.158890 30996 ts_tablet_manager.cc:1403] T e28f28c3f05f40e68531e20dd7c1a1a0 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.213s	user 0.013s	sys 0.006s
I20260709 13:11:46.140494 30789 tablet_service.cc:1511] Processing CreateTablet for tablet 8fb24a20af104a7486d0f1b2ae0e3843 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
W20260709 13:11:46.173549 30691 leader_election.cc:343] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:46.173885 30789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fb24a20af104a7486d0f1b2ae0e3843. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.175251 30862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 784deee1ea9543df840c58ecaa6f911a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.167315 31022 consensus_queue.cc:237] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:46.136308 30935 tablet_service.cc:1511] Processing CreateTablet for tablet 8fb24a20af104a7486d0f1b2ae0e3843 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:11:46.132615 31027 raft_consensus.cc:515] T b226c44412c74106a0feb2ca411b1661 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:46.175709 30996 raft_consensus.cc:359] T e28f28c3f05f40e68531e20dd7c1a1a0 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:46.140275 30716 tablet_service.cc:1511] Processing CreateTablet for tablet 8fb24a20af104a7486d0f1b2ae0e3843 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 13:11:46.175359 30712 tablet_service.cc:1511] Processing CreateTablet for tablet 784deee1ea9543df840c58ecaa6f911a (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 13:11:46.188802 30716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fb24a20af104a7486d0f1b2ae0e3843. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.177412 30948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcac31d5b7ab48ec9c071a8f77750a88" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:46.189873 30948 raft_consensus.cc:2468] T bcac31d5b7ab48ec9c071a8f77750a88 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 0.
I20260709 13:11:46.191363 30690 leader_election.cc:304] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a
I20260709 13:11:46.192010 30801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b226c44412c74106a0feb2ca411b1661" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:46.192281 30949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b226c44412c74106a0feb2ca411b1661" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:11:46.192647 30801 raft_consensus.cc:3060] T b226c44412c74106a0feb2ca411b1661 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:46.192770 30712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 784deee1ea9543df840c58ecaa6f911a. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:11:46.193678 30839 leader_election.cc:343] T b226c44412c74106a0feb2ca411b1661 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:46.194356 31003 raft_consensus.cc:2804] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:46.194733 31003 raft_consensus.cc:493] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:46.195062 31003 raft_consensus.cc:3060] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:46.199047 30863 tablet_service.cc:1511] Processing CreateTablet for tablet 6f8c0b2b0b11472ca76e15fd3ce2e6f1 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:11:46.209522 30863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f8c0b2b0b11472ca76e15fd3ce2e6f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.210657 30935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8fb24a20af104a7486d0f1b2ae0e3843. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.211642 30995 tablet_bootstrap.cc:492] T 7b988f0495c34b7591ad2fb0d5f65693 P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:46.200441 30937 tablet_service.cc:1511] Processing CreateTablet for tablet 6f8c0b2b0b11472ca76e15fd3ce2e6f1 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:11:46.215412 30995 ts_tablet_manager.cc:1403] T 7b988f0495c34b7591ad2fb0d5f65693 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.147s	user 0.008s	sys 0.012s
I20260709 13:11:46.215893 30937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f8c0b2b0b11472ca76e15fd3ce2e6f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.215663 30791 tablet_service.cc:1511] Processing CreateTablet for tablet 8155b3fc02b348e1adb290ecfe4592c5 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:11:46.217142 30791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8155b3fc02b348e1adb290ecfe4592c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.218930 30995 raft_consensus.cc:359] T 7b988f0495c34b7591ad2fb0d5f65693 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:46.219537 30801 raft_consensus.cc:2468] T b226c44412c74106a0feb2ca411b1661 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 1.
I20260709 13:11:46.219782 30995 raft_consensus.cc:385] T 7b988f0495c34b7591ad2fb0d5f65693 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:46.220185 30995 raft_consensus.cc:740] T 7b988f0495c34b7591ad2fb0d5f65693 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:46.201879 31027 leader_election.cc:290] T b226c44412c74106a0feb2ca411b1661 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Requested vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:46.202160 30714 tablet_service.cc:1511] Processing CreateTablet for tablet 8155b3fc02b348e1adb290ecfe4592c5 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:11:46.221019 30838 leader_election.cc:304] T b226c44412c74106a0feb2ca411b1661 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 05213ac836744a53b9c089a5c052f4be; no voters: 262942f67dfa4b8fa42234b18ac31ffd
I20260709 13:11:46.221694 30714 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8155b3fc02b348e1adb290ecfe4592c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.222430 30997 tablet_bootstrap.cc:492] T 437c20f3d53f43048eca57834248cc0d P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
I20260709 13:11:46.222965 30997 ts_tablet_manager.cc:1403] T 437c20f3d53f43048eca57834248cc0d P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.118s	user 0.006s	sys 0.009s
I20260709 13:11:46.224866 31027 raft_consensus.cc:2804] T b226c44412c74106a0feb2ca411b1661 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:46.226322 31027 raft_consensus.cc:697] T b226c44412c74106a0feb2ca411b1661 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Becoming Leader. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Running, Role: LEADER
I20260709 13:11:46.227309 31027 consensus_queue.cc:237] T b226c44412c74106a0feb2ca411b1661 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:46.176060 31031 leader_election.cc:290] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:46.184569 30786 tablet_service.cc:1511] Processing CreateTablet for tablet 6f8c0b2b0b11472ca76e15fd3ce2e6f1 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 13:11:46.187510 30996 raft_consensus.cc:385] T e28f28c3f05f40e68531e20dd7c1a1a0 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:46.230697 30996 raft_consensus.cc:740] T e28f28c3f05f40e68531e20dd7c1a1a0 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
I20260709 13:11:46.231527 30996 consensus_queue.cc:260] T e28f28c3f05f40e68531e20dd7c1a1a0 P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:46.232259 30859 tablet_service.cc:1511] Processing CreateTablet for tablet 8155b3fc02b348e1adb290ecfe4592c5 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 13:11:46.200420 30994 tablet_bootstrap.cc:654] T 7ed3eac558b34457aa26b1c40652f887 P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:46.230409 30786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f8c0b2b0b11472ca76e15fd3ce2e6f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.233697 30859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8155b3fc02b348e1adb290ecfe4592c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.233494 31003 raft_consensus.cc:515] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:46.233946 30996 ts_tablet_manager.cc:1434] T e28f28c3f05f40e68531e20dd7c1a1a0 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.061s	user 0.008s	sys 0.003s
I20260709 13:11:46.239055 30875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcac31d5b7ab48ec9c071a8f77750a88" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:11:46.241750 30997 raft_consensus.cc:359] T 437c20f3d53f43048eca57834248cc0d P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:46.242628 30997 raft_consensus.cc:385] T 437c20f3d53f43048eca57834248cc0d P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:46.242959 30997 raft_consensus.cc:740] T 437c20f3d53f43048eca57834248cc0d P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
I20260709 13:11:46.244479 30997 consensus_queue.cc:260] T 437c20f3d53f43048eca57834248cc0d P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
W20260709 13:11:46.249850 30671 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 13:11:46.249890 30691 leader_election.cc:343] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:46.246420 30932 tablet_service.cc:1511] Processing CreateTablet for tablet c0100ec8bd1e4a45b3c0ea52281f9204 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:11:46.250771 30996 tablet_bootstrap.cc:492] T bcac31d5b7ab48ec9c071a8f77750a88 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
I20260709 13:11:46.251334 30932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0100ec8bd1e4a45b3c0ea52281f9204. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.256997 30997 ts_tablet_manager.cc:1434] T 437c20f3d53f43048eca57834248cc0d P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.034s	user 0.010s	sys 0.000s
I20260709 13:11:46.265074 30997 tablet_bootstrap.cc:492] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
I20260709 13:11:46.274160 30715 tablet_service.cc:1511] Processing CreateTablet for tablet c0100ec8bd1e4a45b3c0ea52281f9204 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:11:46.276938 30715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0100ec8bd1e4a45b3c0ea52281f9204. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.285113 30932 tablet_service.cc:1511] Processing CreateTablet for tablet ac85f05e63ed463681b3ec790b39c4d8 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:11:46.286438 30932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac85f05e63ed463681b3ec790b39c4d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.286281 30708 tablet_service.cc:1511] Processing CreateTablet for tablet ac85f05e63ed463681b3ec790b39c4d8 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:11:46.287565 30708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac85f05e63ed463681b3ec790b39c4d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.296810 30995 consensus_queue.cc:260] T 7b988f0495c34b7591ad2fb0d5f65693 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:46.301090 30995 ts_tablet_manager.cc:1434] T 7b988f0495c34b7591ad2fb0d5f65693 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.085s	user 0.008s	sys 0.000s
I20260709 13:11:46.302183 30995 tablet_bootstrap.cc:492] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:46.298105 30789 tablet_service.cc:1511] Processing CreateTablet for tablet c0100ec8bd1e4a45b3c0ea52281f9204 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 13:11:46.305461 30789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c0100ec8bd1e4a45b3c0ea52281f9204. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.309469 30996 tablet_bootstrap.cc:654] T bcac31d5b7ab48ec9c071a8f77750a88 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:46.325512 30949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcac31d5b7ab48ec9c071a8f77750a88" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:11:46.326164 30949 raft_consensus.cc:3060] T bcac31d5b7ab48ec9c071a8f77750a88 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:46.325855 30624 catalog_manager.cc:5697] T b226c44412c74106a0feb2ca411b1661 P 02c5705b45c04706bbb2c6ff3d7b0b8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3). New cstate: current_term: 1 leader_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:46.330000 30791 tablet_service.cc:1511] Processing CreateTablet for tablet 65776989d6674454816411c251aff5c1 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:11:46.331393 30791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65776989d6674454816411c251aff5c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.333629 30859 tablet_service.cc:1511] Processing CreateTablet for tablet ac85f05e63ed463681b3ec790b39c4d8 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 13:11:46.334056 31003 leader_election.cc:290] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 election: Requested vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:46.335076 30859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac85f05e63ed463681b3ec790b39c4d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.338157 30997 tablet_bootstrap.cc:654] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:46.345000 30994 tablet_bootstrap.cc:492] T 7ed3eac558b34457aa26b1c40652f887 P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
I20260709 13:11:46.345732 30949 raft_consensus.cc:2468] T bcac31d5b7ab48ec9c071a8f77750a88 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 1.
I20260709 13:11:46.349471 30996 tablet_bootstrap.cc:492] T bcac31d5b7ab48ec9c071a8f77750a88 P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
I20260709 13:11:46.349998 30996 ts_tablet_manager.cc:1403] T bcac31d5b7ab48ec9c071a8f77750a88 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.111s	user 0.010s	sys 0.006s
I20260709 13:11:46.351227 30690 leader_election.cc:304] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a
I20260709 13:11:46.351892 30994 ts_tablet_manager.cc:1403] T 7ed3eac558b34457aa26b1c40652f887 P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.189s	user 0.023s	sys 0.001s
I20260709 13:11:46.352377 31022 raft_consensus.cc:2804] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:46.352947 31022 raft_consensus.cc:697] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 LEADER]: Becoming Leader. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Running, Role: LEADER
I20260709 13:11:46.353983 31022 consensus_queue.cc:237] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:46.358920 30856 tablet_service.cc:1511] Processing CreateTablet for tablet 65776989d6674454816411c251aff5c1 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:11:46.368534 30715 tablet_service.cc:1511] Processing CreateTablet for tablet af695b428b974773ba476dce0d31c5a1 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:11:46.363600 30996 raft_consensus.cc:359] T bcac31d5b7ab48ec9c071a8f77750a88 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:46.365094 30937 tablet_service.cc:1511] Processing CreateTablet for tablet 65776989d6674454816411c251aff5c1 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 13:11:46.365451 30994 raft_consensus.cc:359] T 7ed3eac558b34457aa26b1c40652f887 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:46.367635 30787 tablet_service.cc:1511] Processing CreateTablet for tablet af695b428b974773ba476dce0d31c5a1 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:11:46.369966 30996 raft_consensus.cc:385] T bcac31d5b7ab48ec9c071a8f77750a88 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:46.371058 30715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af695b428b974773ba476dce0d31c5a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.371424 30937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65776989d6674454816411c251aff5c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.371579 31009 raft_consensus.cc:493] T 39980eee51514ba493958931f406a145 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:46.344964 30623 catalog_manager.cc:5697] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd reported cstate change: term changed from 0 to 1, leader changed from <none> to d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1). New cstate: current_term: 1 leader_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:46.376899 30787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af695b428b974773ba476dce0d31c5a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.378273 31009 raft_consensus.cc:515] T 39980eee51514ba493958931f406a145 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:46.380964 30997 tablet_bootstrap.cc:492] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
I20260709 13:11:46.381568 31009 leader_election.cc:290] T 39980eee51514ba493958931f406a145 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:46.381842 30997 ts_tablet_manager.cc:1403] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.117s	user 0.013s	sys 0.000s
I20260709 13:11:46.385763 30726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39980eee51514ba493958931f406a145" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:46.386432 30726 raft_consensus.cc:2468] T 39980eee51514ba493958931f406a145 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 0.
I20260709 13:11:46.387773 30915 leader_election.cc:304] T 39980eee51514ba493958931f406a145 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:46.388739 31009 raft_consensus.cc:2804] T 39980eee51514ba493958931f406a145 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:46.389153 31009 raft_consensus.cc:493] T 39980eee51514ba493958931f406a145 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:46.389503 31009 raft_consensus.cc:3060] T 39980eee51514ba493958931f406a145 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:46.397327 30801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39980eee51514ba493958931f406a145" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:46.387864 30997 raft_consensus.cc:359] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:46.398365 30997 raft_consensus.cc:385] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:46.398778 30997 raft_consensus.cc:740] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
W20260709 13:11:46.399216 30912 leader_election.cc:343] T 39980eee51514ba493958931f406a145 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:46.373328 30994 raft_consensus.cc:385] T 7ed3eac558b34457aa26b1c40652f887 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:46.374266 30856 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 65776989d6674454816411c251aff5c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.400987 30994 raft_consensus.cc:740] T 7ed3eac558b34457aa26b1c40652f887 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
I20260709 13:11:46.401995 30994 consensus_queue.cc:260] T 7ed3eac558b34457aa26b1c40652f887 P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:46.407130 30994 ts_tablet_manager.cc:1434] T 7ed3eac558b34457aa26b1c40652f887 P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.055s	user 0.007s	sys 0.000s
I20260709 13:11:46.408212 30994 tablet_bootstrap.cc:492] T efddea050a344563877c3e01ee2a4c81 P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
I20260709 13:11:46.409578 30995 tablet_bootstrap.cc:654] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:46.399588 31009 raft_consensus.cc:515] T 39980eee51514ba493958931f406a145 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:46.374544 30996 raft_consensus.cc:740] T bcac31d5b7ab48ec9c071a8f77750a88 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
I20260709 13:11:46.420260 30932 tablet_service.cc:1511] Processing CreateTablet for tablet 8b6052cddae64c5c9d16f60272735788 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:11:46.421726 30932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b6052cddae64c5c9d16f60272735788. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.421828 30801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39980eee51514ba493958931f406a145" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
W20260709 13:11:46.423381 30912 leader_election.cc:343] T 39980eee51514ba493958931f406a145 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:46.423969 31009 leader_election.cc:290] T 39980eee51514ba493958931f406a145 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Requested vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:46.425927 30726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "39980eee51514ba493958931f406a145" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:46.426551 30726 raft_consensus.cc:3060] T 39980eee51514ba493958931f406a145 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:46.425946 30862 tablet_service.cc:1511] Processing CreateTablet for tablet af695b428b974773ba476dce0d31c5a1 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 13:11:46.428961 30862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af695b428b974773ba476dce0d31c5a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.440079 30996 consensus_queue.cc:260] T bcac31d5b7ab48ec9c071a8f77750a88 P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:46.441068 30726 raft_consensus.cc:2468] T 39980eee51514ba493958931f406a145 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 1.
I20260709 13:11:46.442541 30996 ts_tablet_manager.cc:1434] T bcac31d5b7ab48ec9c071a8f77750a88 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.092s	user 0.007s	sys 0.000s
I20260709 13:11:46.442359 30915 leader_election.cc:304] T 39980eee51514ba493958931f406a145 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 05213ac836744a53b9c089a5c052f4be
I20260709 13:11:46.443608 30996 tablet_bootstrap.cc:492] T 437c20f3d53f43048eca57834248cc0d P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
I20260709 13:11:46.443866 31022 raft_consensus.cc:493] T 7b988f0495c34b7591ad2fb0d5f65693 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:46.444317 31022 raft_consensus.cc:515] T 7b988f0495c34b7591ad2fb0d5f65693 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:46.446449 31022 leader_election.cc:290] T 7b988f0495c34b7591ad2fb0d5f65693 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:46.448206 30715 tablet_service.cc:1511] Processing CreateTablet for tablet 8b6052cddae64c5c9d16f60272735788 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:11:46.449188 30715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b6052cddae64c5c9d16f60272735788. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.447436 30949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b988f0495c34b7591ad2fb0d5f65693" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
W20260709 13:11:46.452845 30690 leader_election.cc:343] T 7b988f0495c34b7591ad2fb0d5f65693 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:46.453740 30801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b988f0495c34b7591ad2fb0d5f65693" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:46.454366 30801 raft_consensus.cc:2468] T 7b988f0495c34b7591ad2fb0d5f65693 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 0.
I20260709 13:11:46.456094 30689 leader_election.cc:304] T 7b988f0495c34b7591ad2fb0d5f65693 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 262942f67dfa4b8fa42234b18ac31ffd
I20260709 13:11:46.457664 31022 raft_consensus.cc:2804] T 7b988f0495c34b7591ad2fb0d5f65693 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:46.458137 31022 raft_consensus.cc:493] T 7b988f0495c34b7591ad2fb0d5f65693 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:46.458464 31022 raft_consensus.cc:3060] T 7b988f0495c34b7591ad2fb0d5f65693 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:46.450626 31009 raft_consensus.cc:2804] T 39980eee51514ba493958931f406a145 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:46.433084 30997 consensus_queue.cc:260] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:46.465678 30859 tablet_service.cc:1511] Processing CreateTablet for tablet ea563bd67a9849d18c9c3dfb21d58a93 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:11:46.471668 30859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea563bd67a9849d18c9c3dfb21d58a93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.474102 30937 tablet_service.cc:1511] Processing CreateTablet for tablet ea563bd67a9849d18c9c3dfb21d58a93 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:11:46.475876 30937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea563bd67a9849d18c9c3dfb21d58a93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.482089 31027 raft_consensus.cc:493] T e28f28c3f05f40e68531e20dd7c1a1a0 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:46.482609 31027 raft_consensus.cc:515] T e28f28c3f05f40e68531e20dd7c1a1a0 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:46.485213 31027 leader_election.cc:290] T e28f28c3f05f40e68531e20dd7c1a1a0 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:46.487470 30726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e28f28c3f05f40e68531e20dd7c1a1a0" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:46.490554 30801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e28f28c3f05f40e68531e20dd7c1a1a0" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:46.491627 30801 raft_consensus.cc:2393] T e28f28c3f05f40e68531e20dd7c1a1a0 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in current term 1: Already voted for candidate d22c9d3d15904327a78a0e548a8ff6cd in this term.
I20260709 13:11:46.493872 30791 tablet_service.cc:1511] Processing CreateTablet for tablet 8b6052cddae64c5c9d16f60272735788 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 13:11:46.494098 30838 leader_election.cc:304] T e28f28c3f05f40e68531e20dd7c1a1a0 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a; no voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:46.495322 30791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b6052cddae64c5c9d16f60272735788. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.497256 31027 raft_consensus.cc:3060] T e28f28c3f05f40e68531e20dd7c1a1a0 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:46.512585 30715 tablet_service.cc:1511] Processing CreateTablet for tablet ea563bd67a9849d18c9c3dfb21d58a93 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 13:11:46.514595 30715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea563bd67a9849d18c9c3dfb21d58a93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.517555 31044 raft_consensus.cc:493] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:46.518172 31044 raft_consensus.cc:515] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:46.521441 30789 tablet_service.cc:1511] Processing CreateTablet for tablet f8f19b152a0f46ac896478f3f05c8822 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:11:46.521801 30726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ed3eac558b34457aa26b1c40652f887" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:46.522328 30801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ed3eac558b34457aa26b1c40652f887" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:46.522933 30801 raft_consensus.cc:2468] T 7ed3eac558b34457aa26b1c40652f887 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 0.
W20260709 13:11:46.523320 30915 leader_election.cc:343] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:46.524084 30863 tablet_service.cc:1511] Processing CreateTablet for tablet f8f19b152a0f46ac896478f3f05c8822 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:11:46.525319 30912 leader_election.cc:304] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:46.522944 30789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8f19b152a0f46ac896478f3f05c8822. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.526924 30995 tablet_bootstrap.cc:492] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:46.527415 30995 ts_tablet_manager.cc:1403] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.225s	user 0.013s	sys 0.017s
I20260709 13:11:46.525566 30863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8f19b152a0f46ac896478f3f05c8822. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.531306 31027 raft_consensus.cc:2749] T e28f28c3f05f40e68531e20dd7c1a1a0 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:46.499912 31022 raft_consensus.cc:515] T 7b988f0495c34b7591ad2fb0d5f65693 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:46.539011 31046 raft_consensus.cc:2804] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:46.539485 31046 raft_consensus.cc:493] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:46.539734 31022 leader_election.cc:290] T 7b988f0495c34b7591ad2fb0d5f65693 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 election: Requested vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:46.539839 31046 raft_consensus.cc:3060] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:46.541374 30801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b988f0495c34b7591ad2fb0d5f65693" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:46.541957 30801 raft_consensus.cc:3060] T 7b988f0495c34b7591ad2fb0d5f65693 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:46.544013 30996 tablet_bootstrap.cc:654] T 437c20f3d53f43048eca57834248cc0d P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:46.546020 30997 ts_tablet_manager.cc:1434] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.161s	user 0.011s	sys 0.004s
I20260709 13:11:46.547089 30997 tablet_bootstrap.cc:492] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
I20260709 13:11:46.546967 30949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b988f0495c34b7591ad2fb0d5f65693" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:11:46.549352 31009 raft_consensus.cc:697] T 39980eee51514ba493958931f406a145 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Becoming Leader. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Running, Role: LEADER
I20260709 13:11:46.550343 31009 consensus_queue.cc:237] T 39980eee51514ba493958931f406a145 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:46.550851 30708 tablet_service.cc:1511] Processing CreateTablet for tablet d5175e0d898c453b8a7641d94dc1ac58 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:11:46.551690 30994 tablet_bootstrap.cc:654] T efddea050a344563877c3e01ee2a4c81 P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
W20260709 13:11:46.552639 30690 leader_election.cc:343] T 7b988f0495c34b7591ad2fb0d5f65693 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:46.553040 31045 raft_consensus.cc:493] T 437c20f3d53f43048eca57834248cc0d P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:46.553574 31045 raft_consensus.cc:515] T 437c20f3d53f43048eca57834248cc0d P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:46.556788 30995 raft_consensus.cc:359] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:46.557904 30875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "437c20f3d53f43048eca57834248cc0d" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:46.558203 31044 leader_election.cc:290] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:46.558615 30997 tablet_bootstrap.cc:654] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
W20260709 13:11:46.559489 30914 leader_election.cc:343] T 437c20f3d53f43048eca57834248cc0d P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:46.557351 31045 leader_election.cc:290] T 437c20f3d53f43048eca57834248cc0d P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:46.561868 30726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "437c20f3d53f43048eca57834248cc0d" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
W20260709 13:11:46.563313 30915 leader_election.cc:343] T 437c20f3d53f43048eca57834248cc0d P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:46.567210 30996 tablet_bootstrap.cc:492] T 437c20f3d53f43048eca57834248cc0d P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
I20260709 13:11:46.568089 30996 ts_tablet_manager.cc:1403] T 437c20f3d53f43048eca57834248cc0d P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.125s	user 0.013s	sys 0.000s
I20260709 13:11:46.571151 30996 raft_consensus.cc:359] T 437c20f3d53f43048eca57834248cc0d P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:46.576858 30996 raft_consensus.cc:385] T 437c20f3d53f43048eca57834248cc0d P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:46.566962 31046 raft_consensus.cc:515] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:46.575183 30708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5175e0d898c453b8a7641d94dc1ac58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.578759 30994 tablet_bootstrap.cc:492] T efddea050a344563877c3e01ee2a4c81 P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
I20260709 13:11:46.579319 30994 ts_tablet_manager.cc:1403] T efddea050a344563877c3e01ee2a4c81 P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.171s	user 0.008s	sys 0.004s
I20260709 13:11:46.558029 30995 raft_consensus.cc:385] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:46.579710 30996 raft_consensus.cc:740] T 437c20f3d53f43048eca57834248cc0d P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
I20260709 13:11:46.579855 30995 raft_consensus.cc:740] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:46.580708 30996 consensus_queue.cc:260] T 437c20f3d53f43048eca57834248cc0d P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:46.581638 30800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ed3eac558b34457aa26b1c40652f887" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:46.585724 30800 raft_consensus.cc:3060] T 7ed3eac558b34457aa26b1c40652f887 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:46.586031 30994 raft_consensus.cc:359] T efddea050a344563877c3e01ee2a4c81 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:46.586825 30994 raft_consensus.cc:385] T efddea050a344563877c3e01ee2a4c81 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:46.587117 30996 ts_tablet_manager.cc:1434] T 437c20f3d53f43048eca57834248cc0d P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.001s
I20260709 13:11:46.587105 30994 raft_consensus.cc:740] T efddea050a344563877c3e01ee2a4c81 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
I20260709 13:11:46.588282 30996 tablet_bootstrap.cc:492] T 283eb8311aeb4b5599dd844da98ee0b7 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
I20260709 13:11:46.587988 30994 consensus_queue.cc:260] T efddea050a344563877c3e01ee2a4c81 P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:46.594419 30789 tablet_service.cc:1511] Processing CreateTablet for tablet d5175e0d898c453b8a7641d94dc1ac58 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:11:46.595078 30624 catalog_manager.cc:5697] T 39980eee51514ba493958931f406a145 P 262942f67dfa4b8fa42234b18ac31ffd reported cstate change: term changed from 0 to 1, leader changed from <none> to 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4). New cstate: current_term: 1 leader_uuid: "262942f67dfa4b8fa42234b18ac31ffd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:46.579236 31046 leader_election.cc:290] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Requested vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:46.596895 30789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5175e0d898c453b8a7641d94dc1ac58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.598294 30994 ts_tablet_manager.cc:1434] T efddea050a344563877c3e01ee2a4c81 P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:11:46.599298 30994 tablet_bootstrap.cc:492] T 283eb8311aeb4b5599dd844da98ee0b7 P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
I20260709 13:11:46.605394 30994 tablet_bootstrap.cc:654] T 283eb8311aeb4b5599dd844da98ee0b7 P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:46.610141 30859 tablet_service.cc:1511] Processing CreateTablet for tablet d5175e0d898c453b8a7641d94dc1ac58 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 13:11:46.612443 30859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5175e0d898c453b8a7641d94dc1ac58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.615274 30715 tablet_service.cc:1511] Processing CreateTablet for tablet 6fe93136933f4d2880ec355e1bbbf90d (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:11:46.617974 30715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fe93136933f4d2880ec355e1bbbf90d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.563803 30915 leader_election.cc:304] T 437c20f3d53f43048eca57834248cc0d P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:46.596284 30996 tablet_bootstrap.cc:654] T 283eb8311aeb4b5599dd844da98ee0b7 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:46.623401 31046 raft_consensus.cc:2749] T 437c20f3d53f43048eca57834248cc0d P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:11:46.623548 30915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:37325, user_credentials={real_user=slave}} blocked reactor thread for 60269.2us
I20260709 13:11:46.626279 30937 tablet_service.cc:1511] Processing CreateTablet for tablet 6fe93136933f4d2880ec355e1bbbf90d (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:11:46.627696 30937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fe93136933f4d2880ec355e1bbbf90d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.625586 30726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ed3eac558b34457aa26b1c40652f887" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:46.580650 30995 consensus_queue.cc:260] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:46.650628 30995 ts_tablet_manager.cc:1434] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.123s	user 0.008s	sys 0.000s
I20260709 13:11:46.651990 30995 tablet_bootstrap.cc:492] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
W20260709 13:11:46.644357 30915 leader_election.cc:343] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:46.657238 30801 raft_consensus.cc:2468] T 7b988f0495c34b7591ad2fb0d5f65693 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 1.
I20260709 13:11:46.659209 30791 tablet_service.cc:1511] Processing CreateTablet for tablet 6fe93136933f4d2880ec355e1bbbf90d (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 13:11:46.661499 30689 leader_election.cc:304] T 7b988f0495c34b7591ad2fb0d5f65693 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 262942f67dfa4b8fa42234b18ac31ffd
I20260709 13:11:46.663103 30995 tablet_bootstrap.cc:654] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:46.663182 31022 raft_consensus.cc:2804] T 7b988f0495c34b7591ad2fb0d5f65693 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:46.665158 31022 raft_consensus.cc:697] T 7b988f0495c34b7591ad2fb0d5f65693 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 LEADER]: Becoming Leader. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Running, Role: LEADER
I20260709 13:11:46.666311 31022 consensus_queue.cc:237] T 7b988f0495c34b7591ad2fb0d5f65693 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:46.675567 30791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fe93136933f4d2880ec355e1bbbf90d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.678056 31003 consensus_queue.cc:1048] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:46.668555 30932 tablet_service.cc:1511] Processing CreateTablet for tablet f8f19b152a0f46ac896478f3f05c8822 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 13:11:46.682255 30932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f8f19b152a0f46ac896478f3f05c8822. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.687520 30996 tablet_bootstrap.cc:492] T 283eb8311aeb4b5599dd844da98ee0b7 P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
I20260709 13:11:46.688268 30996 ts_tablet_manager.cc:1403] T 283eb8311aeb4b5599dd844da98ee0b7 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.100s	user 0.005s	sys 0.029s
I20260709 13:11:46.694077 30996 raft_consensus.cc:359] T 283eb8311aeb4b5599dd844da98ee0b7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:46.695124 30996 raft_consensus.cc:385] T 283eb8311aeb4b5599dd844da98ee0b7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:46.695569 30996 raft_consensus.cc:740] T 283eb8311aeb4b5599dd844da98ee0b7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
I20260709 13:11:46.696740 30996 consensus_queue.cc:260] T 283eb8311aeb4b5599dd844da98ee0b7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:46.698671 30862 tablet_service.cc:1511] Processing CreateTablet for tablet 94220be3f55a4acbbd6bd5723b9f32e9 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:11:46.699687 30996 ts_tablet_manager.cc:1434] T 283eb8311aeb4b5599dd844da98ee0b7 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.002s
I20260709 13:11:46.700181 30862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94220be3f55a4acbbd6bd5723b9f32e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.704582 30937 tablet_service.cc:1511] Processing CreateTablet for tablet 94220be3f55a4acbbd6bd5723b9f32e9 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:11:46.706195 30937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94220be3f55a4acbbd6bd5723b9f32e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.706254 30996 tablet_bootstrap.cc:492] T 11c7485c290740bba00c8b9eaf9b917f P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
I20260709 13:11:46.707086 30856 tablet_service.cc:1511] Processing CreateTablet for tablet 80a61882ef804a85a31e90d67930c5e9 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:11:46.708307 30856 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80a61882ef804a85a31e90d67930c5e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.710501 30716 tablet_service.cc:1511] Processing CreateTablet for tablet 94220be3f55a4acbbd6bd5723b9f32e9 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 13:11:46.711740 30716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94220be3f55a4acbbd6bd5723b9f32e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.717417 30622 catalog_manager.cc:5697] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd reported cstate change: term changed from 0 to 1, leader changed from <none> to d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1). New cstate: current_term: 1 leader_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:46.704830 30800 raft_consensus.cc:2468] T 7ed3eac558b34457aa26b1c40652f887 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 1.
I20260709 13:11:46.720299 30996 tablet_bootstrap.cc:654] T 11c7485c290740bba00c8b9eaf9b917f P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:46.724987 30912 leader_election.cc:304] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:46.726159 31046 raft_consensus.cc:2804] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:46.726706 31046 raft_consensus.cc:697] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Becoming Leader. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Running, Role: LEADER
I20260709 13:11:46.727600 31046 consensus_queue.cc:237] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:46.731875 30789 tablet_service.cc:1511] Processing CreateTablet for tablet 80a61882ef804a85a31e90d67930c5e9 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:11:46.733552 30789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80a61882ef804a85a31e90d67930c5e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.733668 31003 consensus_queue.cc:1048] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:46.736953 31002 consensus_queue.cc:1048] T b226c44412c74106a0feb2ca411b1661 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:11:46.740097 30839 consensus_peers.cc:597] T b226c44412c74106a0feb2ca411b1661 P 02c5705b45c04706bbb2c6ff3d7b0b8a -> Peer 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313): Couldn't send request to peer 262942f67dfa4b8fa42234b18ac31ffd. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:46.741396 31027 raft_consensus.cc:493] T 283eb8311aeb4b5599dd844da98ee0b7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:46.741933 31027 raft_consensus.cc:515] T 283eb8311aeb4b5599dd844da98ee0b7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:46.744213 31027 leader_election.cc:290] T 283eb8311aeb4b5599dd844da98ee0b7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:46.745074 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "283eb8311aeb4b5599dd844da98ee0b7" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
W20260709 13:11:46.746536 30838 leader_election.cc:343] T 283eb8311aeb4b5599dd844da98ee0b7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:46.750422 31027 raft_consensus.cc:493] T bcac31d5b7ab48ec9c071a8f77750a88 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:46.750916 31027 raft_consensus.cc:515] T bcac31d5b7ab48ec9c071a8f77750a88 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:46.753687 31027 leader_election.cc:290] T bcac31d5b7ab48ec9c071a8f77750a88 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:46.768332 30935 tablet_service.cc:1511] Processing CreateTablet for tablet 80a61882ef804a85a31e90d67930c5e9 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 13:11:46.769312 30726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcac31d5b7ab48ec9c071a8f77750a88" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:46.769734 30935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 80a61882ef804a85a31e90d67930c5e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.773561 30949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "283eb8311aeb4b5599dd844da98ee0b7" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:46.779285 30712 tablet_service.cc:1511] Processing CreateTablet for tablet cae2b1cb8db54cbb812489cc1b4b1213 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:11:46.780949 30712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cae2b1cb8db54cbb812489cc1b4b1213. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:11:46.787762 30839 leader_election.cc:343] T 283eb8311aeb4b5599dd844da98ee0b7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:46.788249 30949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcac31d5b7ab48ec9c071a8f77750a88" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:46.788354 30839 leader_election.cc:304] T 283eb8311aeb4b5599dd844da98ee0b7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a; no voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd
I20260709 13:11:46.789203 30949 raft_consensus.cc:2393] T bcac31d5b7ab48ec9c071a8f77750a88 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in current term 1: Already voted for candidate d22c9d3d15904327a78a0e548a8ff6cd in this term.
I20260709 13:11:46.789680 31027 raft_consensus.cc:2749] T 283eb8311aeb4b5599dd844da98ee0b7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:46.790457 30839 leader_election.cc:304] T bcac31d5b7ab48ec9c071a8f77750a88 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a; no voters: 262942f67dfa4b8fa42234b18ac31ffd, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:46.791666 31027 raft_consensus.cc:3060] T bcac31d5b7ab48ec9c071a8f77750a88 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:46.792894 30997 tablet_bootstrap.cc:492] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
I20260709 13:11:46.793442 30997 ts_tablet_manager.cc:1403] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.247s	user 0.036s	sys 0.074s
I20260709 13:11:46.795085 30787 tablet_service.cc:1511] Processing CreateTablet for tablet cae2b1cb8db54cbb812489cc1b4b1213 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:11:46.796384 30787 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cae2b1cb8db54cbb812489cc1b4b1213. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.820365 30937 tablet_service.cc:1511] Processing CreateTablet for tablet d16c44f56c42466497ba4f7f1b2b5344 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:11:46.822029 30937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d16c44f56c42466497ba4f7f1b2b5344. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.818066 30858 tablet_service.cc:1511] Processing CreateTablet for tablet cae2b1cb8db54cbb812489cc1b4b1213 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 13:11:46.822880 30621 catalog_manager.cc:5697] T 7b988f0495c34b7591ad2fb0d5f65693 P d22c9d3d15904327a78a0e548a8ff6cd reported cstate change: term changed from 0 to 1, leader changed from <none> to d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1). New cstate: current_term: 1 leader_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:46.846442 30791 tablet_service.cc:1511] Processing CreateTablet for tablet d16c44f56c42466497ba4f7f1b2b5344 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:11:46.847520 30791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d16c44f56c42466497ba4f7f1b2b5344. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:11:46.840693 31056 log.cc:927] Time spent T e28f28c3f05f40e68531e20dd7c1a1a0 P 05213ac836744a53b9c089a5c052f4be: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:11:46.855710 31027 raft_consensus.cc:2749] T bcac31d5b7ab48ec9c071a8f77750a88 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:46.862587 30994 tablet_bootstrap.cc:492] T 283eb8311aeb4b5599dd844da98ee0b7 P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
I20260709 13:11:46.865213 30996 tablet_bootstrap.cc:492] T 11c7485c290740bba00c8b9eaf9b917f P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
I20260709 13:11:46.890532 30996 ts_tablet_manager.cc:1403] T 11c7485c290740bba00c8b9eaf9b917f P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.185s	user 0.005s	sys 0.034s
I20260709 13:11:46.867679 30995 tablet_bootstrap.cc:492] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:46.867306 30997 raft_consensus.cc:359] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:46.868809 30859 tablet_service.cc:1511] Processing CreateTablet for tablet 8b3e41a728f1457ebf97274b95c7b23a (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:11:46.849888 30858 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cae2b1cb8db54cbb812489cc1b4b1213. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.876216 31003 consensus_queue.cc:1048] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:46.877732 30708 tablet_service.cc:1511] Processing CreateTablet for tablet d16c44f56c42466497ba4f7f1b2b5344 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 13:11:46.893016 30994 ts_tablet_manager.cc:1403] T 283eb8311aeb4b5599dd844da98ee0b7 P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.294s	user 0.042s	sys 0.077s
I20260709 13:11:46.899981 30859 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b3e41a728f1457ebf97274b95c7b23a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.910781 30937 tablet_service.cc:1511] Processing CreateTablet for tablet 8b3e41a728f1457ebf97274b95c7b23a (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:11:46.912366 30937 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b3e41a728f1457ebf97274b95c7b23a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.900311 30997 raft_consensus.cc:385] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:46.914749 30708 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d16c44f56c42466497ba4f7f1b2b5344. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.920470 30716 tablet_service.cc:1511] Processing CreateTablet for tablet 8b3e41a728f1457ebf97274b95c7b23a (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 13:11:46.921276 30791 tablet_service.cc:1511] Processing CreateTablet for tablet ea4dcd4ecc9840d2b8785d14dd4537b8 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:11:46.921813 30716 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b3e41a728f1457ebf97274b95c7b23a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.922389 30791 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea4dcd4ecc9840d2b8785d14dd4537b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.913894 30997 raft_consensus.cc:740] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
I20260709 13:11:46.923990 30997 consensus_queue.cc:260] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:46.925621 30712 tablet_service.cc:1511] Processing CreateTablet for tablet e2376eaea6bc4f209a21a110370da7b2 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:11:46.926479 30997 ts_tablet_manager.cc:1434] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.133s	user 0.006s	sys 0.002s
I20260709 13:11:46.926741 30712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2376eaea6bc4f209a21a110370da7b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.927515 30997 tablet_bootstrap.cc:492] T 283eb8311aeb4b5599dd844da98ee0b7 P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
I20260709 13:11:46.908951 30995 ts_tablet_manager.cc:1403] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.257s	user 0.027s	sys 0.027s
I20260709 13:11:46.946115 30789 tablet_service.cc:1511] Processing CreateTablet for tablet e2376eaea6bc4f209a21a110370da7b2 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:11:46.947392 30789 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2376eaea6bc4f209a21a110370da7b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.946991 30863 tablet_service.cc:1511] Processing CreateTablet for tablet e2376eaea6bc4f209a21a110370da7b2 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 13:11:46.948751 30863 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2376eaea6bc4f209a21a110370da7b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.951347 31066 raft_consensus.cc:493] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:46.951859 31066 raft_consensus.cc:515] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:46.952271 30622 catalog_manager.cc:5697] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd reported cstate change: term changed from 0 to 1, leader changed from <none> to 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4). New cstate: current_term: 1 leader_uuid: "262942f67dfa4b8fa42234b18ac31ffd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:46.958474 30995 raft_consensus.cc:359] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:46.961158 30996 raft_consensus.cc:359] T 11c7485c290740bba00c8b9eaf9b917f P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:46.946276 31003 consensus_queue.cc:1048] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:46.966928 31066 leader_election.cc:290] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:46.967875 30801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efddea050a344563877c3e01ee2a4c81" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:46.972193 30938 tablet_service.cc:1511] Processing CreateTablet for tablet f63e902c7c7a4d0fa5d2107620ce851e (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:11:46.988430 30875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efddea050a344563877c3e01ee2a4c81" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:46.983929 30995 raft_consensus.cc:385] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:47.000602 30938 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f63e902c7c7a4d0fa5d2107620ce851e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:46.996337 30801 raft_consensus.cc:2468] T efddea050a344563877c3e01ee2a4c81 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 0.
I20260709 13:11:47.002394 30786 tablet_service.cc:1511] Processing CreateTablet for tablet f63e902c7c7a4d0fa5d2107620ce851e (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:11:46.989629 30996 raft_consensus.cc:385] T 11c7485c290740bba00c8b9eaf9b917f P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:46.964432 30994 raft_consensus.cc:359] T 283eb8311aeb4b5599dd844da98ee0b7 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:47.005340 30856 tablet_service.cc:1511] Processing CreateTablet for tablet ea4dcd4ecc9840d2b8785d14dd4537b8 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:11:47.005571 30715 tablet_service.cc:1511] Processing CreateTablet for tablet f63e902c7c7a4d0fa5d2107620ce851e (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 13:11:47.024384 30715 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f63e902c7c7a4d0fa5d2107620ce851e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:47.006676 30935 tablet_service.cc:1511] Processing CreateTablet for tablet ea4dcd4ecc9840d2b8785d14dd4537b8 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 13:11:47.029917 30935 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea4dcd4ecc9840d2b8785d14dd4537b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:47.023375 30996 raft_consensus.cc:740] T 11c7485c290740bba00c8b9eaf9b917f P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
W20260709 13:11:47.018496 30691 leader_election.cc:343] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:47.034770 30996 consensus_queue.cc:260] T 11c7485c290740bba00c8b9eaf9b917f P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:47.037940 30996 ts_tablet_manager.cc:1434] T 11c7485c290740bba00c8b9eaf9b917f P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.147s	user 0.006s	sys 0.002s
I20260709 13:11:47.039669 30856 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ea4dcd4ecc9840d2b8785d14dd4537b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:47.034164 30786 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f63e902c7c7a4d0fa5d2107620ce851e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:47.040475 30932 tablet_service.cc:1511] Processing CreateTablet for tablet 99e98c1324084d85b8a97aaf0c248df7 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:11:47.041913 30932 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99e98c1324084d85b8a97aaf0c248df7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:47.045538 30862 tablet_service.cc:1511] Processing CreateTablet for tablet 99e98c1324084d85b8a97aaf0c248df7 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:11:47.046846 30862 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99e98c1324084d85b8a97aaf0c248df7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:47.023550 30994 raft_consensus.cc:385] T 283eb8311aeb4b5599dd844da98ee0b7 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:47.048950 30994 raft_consensus.cc:740] T 283eb8311aeb4b5599dd844da98ee0b7 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
I20260709 13:11:47.049751 30994 consensus_queue.cc:260] T 283eb8311aeb4b5599dd844da98ee0b7 P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:47.052073 30994 ts_tablet_manager.cc:1434] T 283eb8311aeb4b5599dd844da98ee0b7 P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.140s	user 0.006s	sys 0.000s
I20260709 13:11:47.056047 30689 leader_election.cc:304] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a
I20260709 13:11:47.060432 30712 tablet_service.cc:1511] Processing CreateTablet for tablet 99e98c1324084d85b8a97aaf0c248df7 (DEFAULT_TABLE table=test-workload [id=20be961e431b424081ca24f922e822ba]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 13:11:47.062479 30712 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 99e98c1324084d85b8a97aaf0c248df7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:11:47.021124 30995 raft_consensus.cc:740] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:47.089041 31002 raft_consensus.cc:493] T 437c20f3d53f43048eca57834248cc0d P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.089843 31002 raft_consensus.cc:515] T 437c20f3d53f43048eca57834248cc0d P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
W20260709 13:11:47.089068 30689 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:43771, user_credentials={real_user=slave}} blocked reactor thread for 56086us
I20260709 13:11:47.095845 31002 leader_election.cc:290] T 437c20f3d53f43048eca57834248cc0d P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:47.072232 31068 raft_consensus.cc:2804] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:47.098443 30994 tablet_bootstrap.cc:492] T 11c7485c290740bba00c8b9eaf9b917f P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
I20260709 13:11:47.105000 31046 consensus_queue.cc:1048] T 39980eee51514ba493958931f406a145 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:47.102487 31068 raft_consensus.cc:493] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:47.108469 30690 consensus_peers.cc:597] T 7b988f0495c34b7591ad2fb0d5f65693 P d22c9d3d15904327a78a0e548a8ff6cd -> Peer 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313): Couldn't send request to peer 262942f67dfa4b8fa42234b18ac31ffd. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:47.109014 30947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "437c20f3d53f43048eca57834248cc0d" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:47.109264 31068 raft_consensus.cc:3060] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:47.109742 30947 raft_consensus.cc:2468] T 437c20f3d53f43048eca57834248cc0d P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 0.
I20260709 13:11:47.113765 30997 tablet_bootstrap.cc:654] T 283eb8311aeb4b5599dd844da98ee0b7 P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:47.117312 30839 leader_election.cc:304] T 437c20f3d53f43048eca57834248cc0d P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
I20260709 13:11:47.101287 30995 consensus_queue.cc:260] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:47.122654 30995 ts_tablet_manager.cc:1434] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.185s	user 0.008s	sys 0.000s
I20260709 13:11:47.123792 30995 tablet_bootstrap.cc:492] T 437c20f3d53f43048eca57834248cc0d P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:47.142560 31068 raft_consensus.cc:515] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
W20260709 13:11:47.143507 30912 consensus_peers.cc:597] T 39980eee51514ba493958931f406a145 P 262942f67dfa4b8fa42234b18ac31ffd -> Peer 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771): Couldn't send request to peer 05213ac836744a53b9c089a5c052f4be. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:47.144325 30725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "437c20f3d53f43048eca57834248cc0d" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:47.147881 31068 leader_election.cc:290] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 election: Requested vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:47.149230 30801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efddea050a344563877c3e01ee2a4c81" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:47.150003 30801 raft_consensus.cc:3060] T efddea050a344563877c3e01ee2a4c81 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:47.151743 31027 raft_consensus.cc:2804] T 437c20f3d53f43048eca57834248cc0d P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:47.152227 31027 raft_consensus.cc:493] T 437c20f3d53f43048eca57834248cc0d P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:47.152554 31027 raft_consensus.cc:3060] T 437c20f3d53f43048eca57834248cc0d P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:11:47.149273 30841 leader_election.cc:343] T 437c20f3d53f43048eca57834248cc0d P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:47.149528 30996 tablet_bootstrap.cc:492] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
I20260709 13:11:47.162607 30875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efddea050a344563877c3e01ee2a4c81" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
W20260709 13:11:47.165592 30691 leader_election.cc:343] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:11:47.168066 30915 consensus_peers.cc:597] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:47.171275 30997 tablet_bootstrap.cc:492] T 283eb8311aeb4b5599dd844da98ee0b7 P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
I20260709 13:11:47.171885 30997 ts_tablet_manager.cc:1403] T 283eb8311aeb4b5599dd844da98ee0b7 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.245s	user 0.008s	sys 0.008s
I20260709 13:11:47.175043 30997 raft_consensus.cc:359] T 283eb8311aeb4b5599dd844da98ee0b7 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:47.154089 30994 tablet_bootstrap.cc:654] T 11c7485c290740bba00c8b9eaf9b917f P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:47.163812 30995 tablet_bootstrap.cc:654] T 437c20f3d53f43048eca57834248cc0d P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:47.182634 30997 raft_consensus.cc:385] T 283eb8311aeb4b5599dd844da98ee0b7 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:47.183161 30997 raft_consensus.cc:740] T 283eb8311aeb4b5599dd844da98ee0b7 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
I20260709 13:11:47.184736 31027 raft_consensus.cc:515] T 437c20f3d53f43048eca57834248cc0d P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:47.189302 30801 raft_consensus.cc:2468] T efddea050a344563877c3e01ee2a4c81 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 1.
I20260709 13:11:47.191092 30947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "437c20f3d53f43048eca57834248cc0d" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:11:47.191896 30947 raft_consensus.cc:3060] T 437c20f3d53f43048eca57834248cc0d P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:47.185670 30997 consensus_queue.cc:260] T 283eb8311aeb4b5599dd844da98ee0b7 P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:47.201596 30996 tablet_bootstrap.cc:654] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:47.215479 30689 leader_election.cc:304] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a
I20260709 13:11:47.216958 31069 raft_consensus.cc:2804] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:47.217443 31069 raft_consensus.cc:697] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 LEADER]: Becoming Leader. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Running, Role: LEADER
I20260709 13:11:47.218225 31069 consensus_queue.cc:237] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:47.218690 30997 ts_tablet_manager.cc:1434] T 283eb8311aeb4b5599dd844da98ee0b7 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.046s	user 0.001s	sys 0.005s
I20260709 13:11:47.218999 30725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "437c20f3d53f43048eca57834248cc0d" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:47.219477 30947 raft_consensus.cc:2468] T 437c20f3d53f43048eca57834248cc0d P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 1.
I20260709 13:11:47.219756 30997 tablet_bootstrap.cc:492] T 11c7485c290740bba00c8b9eaf9b917f P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
W20260709 13:11:47.220407 30841 leader_election.cc:343] T 437c20f3d53f43048eca57834248cc0d P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:47.223451 30839 leader_election.cc:304] T 437c20f3d53f43048eca57834248cc0d P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 262942f67dfa4b8fa42234b18ac31ffd; no voters: d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:47.224373 31002 raft_consensus.cc:2804] T 437c20f3d53f43048eca57834248cc0d P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:47.224927 31002 raft_consensus.cc:697] T 437c20f3d53f43048eca57834248cc0d P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Becoming Leader. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Running, Role: LEADER
I20260709 13:11:47.225736 31002 consensus_queue.cc:237] T 437c20f3d53f43048eca57834248cc0d P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:47.226627 30994 tablet_bootstrap.cc:492] T 11c7485c290740bba00c8b9eaf9b917f P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
I20260709 13:11:47.227164 30994 ts_tablet_manager.cc:1403] T 11c7485c290740bba00c8b9eaf9b917f P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.160s	user 0.016s	sys 0.005s
I20260709 13:11:47.221323 31027 leader_election.cc:290] T 437c20f3d53f43048eca57834248cc0d P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Requested vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:47.233531 30995 tablet_bootstrap.cc:492] T 437c20f3d53f43048eca57834248cc0d P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:47.235762 30995 ts_tablet_manager.cc:1403] T 437c20f3d53f43048eca57834248cc0d P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.112s	user 0.007s	sys 0.017s
I20260709 13:11:47.242288 30997 tablet_bootstrap.cc:654] T 11c7485c290740bba00c8b9eaf9b917f P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:47.247943 31046 raft_consensus.cc:493] T 283eb8311aeb4b5599dd844da98ee0b7 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.263469 30995 raft_consensus.cc:359] T 437c20f3d53f43048eca57834248cc0d P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:47.264357 30995 raft_consensus.cc:385] T 437c20f3d53f43048eca57834248cc0d P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:47.264711 30995 raft_consensus.cc:740] T 437c20f3d53f43048eca57834248cc0d P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:47.253567 31046 raft_consensus.cc:515] T 283eb8311aeb4b5599dd844da98ee0b7 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:47.266597 31046 leader_election.cc:290] T 283eb8311aeb4b5599dd844da98ee0b7 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:47.249838 30994 raft_consensus.cc:359] T 11c7485c290740bba00c8b9eaf9b917f P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:47.270033 30994 raft_consensus.cc:385] T 11c7485c290740bba00c8b9eaf9b917f P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:47.270422 30994 raft_consensus.cc:740] T 11c7485c290740bba00c8b9eaf9b917f P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
I20260709 13:11:47.270532 30995 consensus_queue.cc:260] T 437c20f3d53f43048eca57834248cc0d P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:47.271662 30994 consensus_queue.cc:260] T 11c7485c290740bba00c8b9eaf9b917f P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:47.274255 30994 ts_tablet_manager.cc:1434] T 11c7485c290740bba00c8b9eaf9b917f P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.047s	user 0.006s	sys 0.001s
I20260709 13:11:47.275290 30994 tablet_bootstrap.cc:492] T 39980eee51514ba493958931f406a145 P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
I20260709 13:11:47.275874 30801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "283eb8311aeb4b5599dd844da98ee0b7" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:47.270385 30875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "283eb8311aeb4b5599dd844da98ee0b7" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:47.276926 30875 raft_consensus.cc:2468] T 283eb8311aeb4b5599dd844da98ee0b7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 0.
I20260709 13:11:47.278218 30914 leader_election.cc:304] T 283eb8311aeb4b5599dd844da98ee0b7 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
I20260709 13:11:47.275174 30622 catalog_manager.cc:5697] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd reported cstate change: term changed from 0 to 1, leader changed from <none> to d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1). New cstate: current_term: 1 leader_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:47.281198 30995 ts_tablet_manager.cc:1434] T 437c20f3d53f43048eca57834248cc0d P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.000s
I20260709 13:11:47.276623 30801 raft_consensus.cc:2468] T 283eb8311aeb4b5599dd844da98ee0b7 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 0.
I20260709 13:11:47.282372 30995 tablet_bootstrap.cc:492] T 0ac93952b93645b1ad1d7d96cb60c3e9 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:47.283013 31082 raft_consensus.cc:2804] T 283eb8311aeb4b5599dd844da98ee0b7 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:47.283442 31082 raft_consensus.cc:493] T 283eb8311aeb4b5599dd844da98ee0b7 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:47.283915 30994 tablet_bootstrap.cc:654] T 39980eee51514ba493958931f406a145 P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:47.283887 31082 raft_consensus.cc:3060] T 283eb8311aeb4b5599dd844da98ee0b7 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:47.296622 30995 tablet_bootstrap.cc:654] T 0ac93952b93645b1ad1d7d96cb60c3e9 P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:47.298915 30996 tablet_bootstrap.cc:492] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
I20260709 13:11:47.299431 30996 ts_tablet_manager.cc:1403] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.150s	user 0.016s	sys 0.020s
I20260709 13:11:47.302196 30994 tablet_bootstrap.cc:492] T 39980eee51514ba493958931f406a145 P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
I20260709 13:11:47.302575 30994 ts_tablet_manager.cc:1403] T 39980eee51514ba493958931f406a145 P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.027s	user 0.007s	sys 0.012s
I20260709 13:11:47.304654 30994 raft_consensus.cc:359] T 39980eee51514ba493958931f406a145 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:47.305467 30994 raft_consensus.cc:385] T 39980eee51514ba493958931f406a145 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:47.305801 30994 raft_consensus.cc:740] T 39980eee51514ba493958931f406a145 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
W20260709 13:11:47.306066 30671 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:11:47.279688 31069 consensus_queue.cc:1048] T 7b988f0495c34b7591ad2fb0d5f65693 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260709 13:11:47.307785 30994 consensus_queue.cc:260] T 39980eee51514ba493958931f406a145 P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:47.307301 30621 catalog_manager.cc:5697] T 437c20f3d53f43048eca57834248cc0d P 02c5705b45c04706bbb2c6ff3d7b0b8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3). New cstate: current_term: 1 leader_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:47.310738 30996 raft_consensus.cc:359] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:47.311578 30996 raft_consensus.cc:385] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:47.311921 30996 raft_consensus.cc:740] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
I20260709 13:11:47.312860 30996 consensus_queue.cc:260] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:47.315085 30996 ts_tablet_manager.cc:1434] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.001s
I20260709 13:11:47.318854 30994 ts_tablet_manager.cc:1434] T 39980eee51514ba493958931f406a145 P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.001s
I20260709 13:11:47.320254 30994 tablet_bootstrap.cc:492] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
I20260709 13:11:47.323146 31085 consensus_queue.cc:1048] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:47.331990 30994 tablet_bootstrap.cc:654] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:47.339039 30996 tablet_bootstrap.cc:492] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
I20260709 13:11:47.343935 31085 raft_consensus.cc:493] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.346535 31085 raft_consensus.cc:515] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:47.350915 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fb3f8d627b04c8a8971f7e4d3a5c010" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:47.352458 31085 leader_election.cc:290] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:47.352157 30726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fb3f8d627b04c8a8971f7e4d3a5c010" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:47.352231 31082 raft_consensus.cc:515] T 283eb8311aeb4b5599dd844da98ee0b7 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
W20260709 13:11:47.352824 30912 leader_election.cc:343] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:11:47.364152 30915 leader_election.cc:343] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:47.365455 30915 leader_election.cc:304] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd; no voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:47.367012 31085 raft_consensus.cc:2749] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:47.367519 31082 leader_election.cc:290] T 283eb8311aeb4b5599dd844da98ee0b7 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Requested vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:47.369133 30875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "283eb8311aeb4b5599dd844da98ee0b7" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:11:47.369899 30875 raft_consensus.cc:3060] T 283eb8311aeb4b5599dd844da98ee0b7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:47.367688 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "283eb8311aeb4b5599dd844da98ee0b7" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:47.373533 30799 raft_consensus.cc:3060] T 283eb8311aeb4b5599dd844da98ee0b7 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:47.377156 30996 tablet_bootstrap.cc:654] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:47.388008 30875 raft_consensus.cc:2468] T 283eb8311aeb4b5599dd844da98ee0b7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 1.
I20260709 13:11:47.394609 30994 tablet_bootstrap.cc:492] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
I20260709 13:11:47.395295 30994 ts_tablet_manager.cc:1403] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.075s	user 0.015s	sys 0.028s
I20260709 13:11:47.396035 30995 tablet_bootstrap.cc:492] T 0ac93952b93645b1ad1d7d96cb60c3e9 P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:47.396458 30995 ts_tablet_manager.cc:1403] T 0ac93952b93645b1ad1d7d96cb60c3e9 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.114s	user 0.015s	sys 0.013s
I20260709 13:11:47.398041 30997 tablet_bootstrap.cc:492] T 11c7485c290740bba00c8b9eaf9b917f P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
I20260709 13:11:47.398592 30997 ts_tablet_manager.cc:1403] T 11c7485c290740bba00c8b9eaf9b917f P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.179s	user 0.023s	sys 0.018s
I20260709 13:11:47.399204 30995 raft_consensus.cc:359] T 0ac93952b93645b1ad1d7d96cb60c3e9 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:47.398872 30994 raft_consensus.cc:359] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:47.400009 30995 raft_consensus.cc:385] T 0ac93952b93645b1ad1d7d96cb60c3e9 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:47.400345 30994 raft_consensus.cc:385] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:47.400454 30995 raft_consensus.cc:740] T 0ac93952b93645b1ad1d7d96cb60c3e9 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:47.404497 30995 consensus_queue.cc:260] T 0ac93952b93645b1ad1d7d96cb60c3e9 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:47.409494 30799 raft_consensus.cc:2468] T 283eb8311aeb4b5599dd844da98ee0b7 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 1.
I20260709 13:11:47.411099 30997 raft_consensus.cc:359] T 11c7485c290740bba00c8b9eaf9b917f P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:47.412142 30997 raft_consensus.cc:385] T 11c7485c290740bba00c8b9eaf9b917f P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:47.412278 30912 leader_election.cc:304] T 283eb8311aeb4b5599dd844da98ee0b7 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
I20260709 13:11:47.412554 30997 raft_consensus.cc:740] T 11c7485c290740bba00c8b9eaf9b917f P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
I20260709 13:11:47.414705 30997 consensus_queue.cc:260] T 11c7485c290740bba00c8b9eaf9b917f P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:47.400929 30994 raft_consensus.cc:740] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
I20260709 13:11:47.416229 30994 consensus_queue.cc:260] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:47.417892 31066 raft_consensus.cc:493] T 437c20f3d53f43048eca57834248cc0d P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.418226 30994 ts_tablet_manager.cc:1434] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.023s	user 0.009s	sys 0.000s
I20260709 13:11:47.418452 31066 raft_consensus.cc:515] T 437c20f3d53f43048eca57834248cc0d P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:47.419154 30994 tablet_bootstrap.cc:492] T 1474b6fc151c4a3994a3aef89ea936a2 P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
I20260709 13:11:47.420437 31066 leader_election.cc:290] T 437c20f3d53f43048eca57834248cc0d P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:47.421691 30874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "437c20f3d53f43048eca57834248cc0d" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:47.423880 30996 tablet_bootstrap.cc:492] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
I20260709 13:11:47.424369 30996 ts_tablet_manager.cc:1403] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.100s	user 0.006s	sys 0.017s
I20260709 13:11:47.425544 30994 tablet_bootstrap.cc:654] T 1474b6fc151c4a3994a3aef89ea936a2 P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:47.427242 30997 ts_tablet_manager.cc:1434] T 11c7485c290740bba00c8b9eaf9b917f P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.000s
I20260709 13:11:47.428310 30997 tablet_bootstrap.cc:492] T b226c44412c74106a0feb2ca411b1661 P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
I20260709 13:11:47.407250 30995 ts_tablet_manager.cc:1434] T 0ac93952b93645b1ad1d7d96cb60c3e9 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:11:47.429100 30996 raft_consensus.cc:359] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:47.432945 30996 raft_consensus.cc:385] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:47.433324 30996 raft_consensus.cc:740] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
I20260709 13:11:47.434535 31046 raft_consensus.cc:2804] T 283eb8311aeb4b5599dd844da98ee0b7 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:47.434592 30949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "437c20f3d53f43048eca57834248cc0d" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:47.434415 30996 consensus_queue.cc:260] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:47.435256 31046 raft_consensus.cc:697] T 283eb8311aeb4b5599dd844da98ee0b7 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Becoming Leader. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Running, Role: LEADER
I20260709 13:11:47.435505 30949 raft_consensus.cc:2393] T 437c20f3d53f43048eca57834248cc0d P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d22c9d3d15904327a78a0e548a8ff6cd in current term 1: Already voted for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in this term.
I20260709 13:11:47.433153 31069 raft_consensus.cc:493] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.436100 31069 raft_consensus.cc:515] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:47.436825 30690 leader_election.cc:304] T 437c20f3d53f43048eca57834248cc0d P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22c9d3d15904327a78a0e548a8ff6cd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 262942f67dfa4b8fa42234b18ac31ffd
I20260709 13:11:47.436414 31046 consensus_queue.cc:237] T 283eb8311aeb4b5599dd844da98ee0b7 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:47.437927 31066 raft_consensus.cc:3060] T 437c20f3d53f43048eca57834248cc0d P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:47.438874 30800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1474b6fc151c4a3994a3aef89ea936a2" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
W20260709 13:11:47.440305 30689 leader_election.cc:343] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:47.441442 31069 leader_election.cc:290] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:47.441601 31002 raft_consensus.cc:493] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.443548 31002 raft_consensus.cc:515] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:47.442900 30875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1474b6fc151c4a3994a3aef89ea936a2" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:47.445101 30875 raft_consensus.cc:2468] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 0.
I20260709 13:11:47.445573 31066 raft_consensus.cc:2749] T 437c20f3d53f43048eca57834248cc0d P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:47.446354 30691 leader_election.cc:304] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 05213ac836744a53b9c089a5c052f4be
I20260709 13:11:47.448266 31002 leader_election.cc:290] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:47.449360 30726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1474b6fc151c4a3994a3aef89ea936a2" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:47.450001 30726 raft_consensus.cc:2468] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 0.
I20260709 13:11:47.451391 30997 tablet_bootstrap.cc:654] T b226c44412c74106a0feb2ca411b1661 P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:47.451717 30841 leader_election.cc:304] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:47.452723 31066 raft_consensus.cc:2804] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:47.452970 30800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1474b6fc151c4a3994a3aef89ea936a2" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
W20260709 13:11:47.454643 30838 leader_election.cc:343] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:47.457221 31027 raft_consensus.cc:2804] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:47.457792 31027 raft_consensus.cc:493] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:47.458231 31027 raft_consensus.cc:3060] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:47.461304 30995 tablet_bootstrap.cc:492] T 27f26e218737414893d5c4bfa4b2cd94 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:47.453156 31066 raft_consensus.cc:493] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:47.464547 31066 raft_consensus.cc:3060] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:47.468369 31027 raft_consensus.cc:515] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:47.471633 30996 ts_tablet_manager.cc:1434] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.047s	user 0.010s	sys 0.000s
I20260709 13:11:47.476468 30995 tablet_bootstrap.cc:654] T 27f26e218737414893d5c4bfa4b2cd94 P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:47.477211 31066 raft_consensus.cc:515] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:47.480295 30800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1474b6fc151c4a3994a3aef89ea936a2" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:47.479797 30997 tablet_bootstrap.cc:492] T b226c44412c74106a0feb2ca411b1661 P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
I20260709 13:11:47.481700 30875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1474b6fc151c4a3994a3aef89ea936a2" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:11:47.482162 30997 ts_tablet_manager.cc:1403] T b226c44412c74106a0feb2ca411b1661 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.054s	user 0.015s	sys 0.003s
I20260709 13:11:47.482486 30875 raft_consensus.cc:2393] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d22c9d3d15904327a78a0e548a8ff6cd in current term 1: Already voted for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in this term.
W20260709 13:11:47.483407 30689 leader_election.cc:343] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:47.484679 30691 leader_election.cc:304] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22c9d3d15904327a78a0e548a8ff6cd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 05213ac836744a53b9c089a5c052f4be
I20260709 13:11:47.485082 30997 raft_consensus.cc:359] T b226c44412c74106a0feb2ca411b1661 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:47.485714 31069 raft_consensus.cc:2749] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:11:47.485968 30997 raft_consensus.cc:385] T b226c44412c74106a0feb2ca411b1661 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:47.486346 30997 raft_consensus.cc:740] T b226c44412c74106a0feb2ca411b1661 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
I20260709 13:11:47.487305 30997 consensus_queue.cc:260] T b226c44412c74106a0feb2ca411b1661 P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:47.489722 30997 ts_tablet_manager.cc:1434] T b226c44412c74106a0feb2ca411b1661 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:11:47.490685 30997 tablet_bootstrap.cc:492] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
I20260709 13:11:47.490869 30726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1474b6fc151c4a3994a3aef89ea936a2" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:47.491986 30726 raft_consensus.cc:2393] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in current term 1: Already voted for candidate d22c9d3d15904327a78a0e548a8ff6cd in this term.
I20260709 13:11:47.480532 31066 leader_election.cc:290] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 election: Requested vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:47.495710 31027 leader_election.cc:290] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Requested vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:47.497308 30996 tablet_bootstrap.cc:492] T efddea050a344563877c3e01ee2a4c81 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
I20260709 13:11:47.497118 30800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1474b6fc151c4a3994a3aef89ea936a2" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
W20260709 13:11:47.498672 30838 leader_election.cc:343] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:47.499033 30994 tablet_bootstrap.cc:492] T 1474b6fc151c4a3994a3aef89ea936a2 P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
I20260709 13:11:47.499121 30838 leader_election.cc:304] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a; no voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:47.499516 30997 tablet_bootstrap.cc:654] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:47.499589 30994 ts_tablet_manager.cc:1403] T 1474b6fc151c4a3994a3aef89ea936a2 P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.081s	user 0.049s	sys 0.027s
I20260709 13:11:47.502899 30994 raft_consensus.cc:359] T 1474b6fc151c4a3994a3aef89ea936a2 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:47.503559 30994 raft_consensus.cc:385] T 1474b6fc151c4a3994a3aef89ea936a2 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:47.503862 30994 raft_consensus.cc:740] T 1474b6fc151c4a3994a3aef89ea936a2 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
I20260709 13:11:47.504689 30994 consensus_queue.cc:260] T 1474b6fc151c4a3994a3aef89ea936a2 P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:47.506623 31002 raft_consensus.cc:493] T 11c7485c290740bba00c8b9eaf9b917f P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.507375 31002 raft_consensus.cc:515] T 11c7485c290740bba00c8b9eaf9b917f P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:47.512588 30800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c7485c290740bba00c8b9eaf9b917f" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:47.513543 30800 raft_consensus.cc:2468] T 11c7485c290740bba00c8b9eaf9b917f P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 0.
I20260709 13:11:47.515699 31002 leader_election.cc:290] T 11c7485c290740bba00c8b9eaf9b917f P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:47.515538 30621 catalog_manager.cc:5697] T 283eb8311aeb4b5599dd844da98ee0b7 P 262942f67dfa4b8fa42234b18ac31ffd reported cstate change: term changed from 0 to 1, leader changed from <none> to 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4). New cstate: current_term: 1 leader_uuid: "262942f67dfa4b8fa42234b18ac31ffd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:47.517113 30949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c7485c290740bba00c8b9eaf9b917f" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:47.517907 31027 raft_consensus.cc:2749] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:11:47.518959 30994 ts_tablet_manager.cc:1434] T 1474b6fc151c4a3994a3aef89ea936a2 P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.019s	user 0.003s	sys 0.003s
I20260709 13:11:47.516764 30838 leader_election.cc:304] T 11c7485c290740bba00c8b9eaf9b917f P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 05213ac836744a53b9c089a5c052f4be; no voters: 
I20260709 13:11:47.518006 30949 raft_consensus.cc:2468] T 11c7485c290740bba00c8b9eaf9b917f P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 0.
I20260709 13:11:47.523478 30994 tablet_bootstrap.cc:492] T d05da516ed9040c988b1fdfe2fdde9ed P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
I20260709 13:11:47.516918 30996 tablet_bootstrap.cc:654] T efddea050a344563877c3e01ee2a4c81 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:47.530593 30995 tablet_bootstrap.cc:492] T 27f26e218737414893d5c4bfa4b2cd94 P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:47.531523 30995 ts_tablet_manager.cc:1403] T 27f26e218737414893d5c4bfa4b2cd94 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.070s	user 0.022s	sys 0.012s
I20260709 13:11:47.532984 30994 tablet_bootstrap.cc:654] T d05da516ed9040c988b1fdfe2fdde9ed P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:47.532882 31027 raft_consensus.cc:2804] T 11c7485c290740bba00c8b9eaf9b917f P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:47.533987 31027 raft_consensus.cc:493] T 11c7485c290740bba00c8b9eaf9b917f P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:47.534444 31027 raft_consensus.cc:3060] T 11c7485c290740bba00c8b9eaf9b917f P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:47.534504 30995 raft_consensus.cc:359] T 27f26e218737414893d5c4bfa4b2cd94 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:47.535475 30995 raft_consensus.cc:385] T 27f26e218737414893d5c4bfa4b2cd94 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:47.535884 30995 raft_consensus.cc:740] T 27f26e218737414893d5c4bfa4b2cd94 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:47.536834 30995 consensus_queue.cc:260] T 27f26e218737414893d5c4bfa4b2cd94 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:47.537242 30997 tablet_bootstrap.cc:492] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
I20260709 13:11:47.538266 30997 ts_tablet_manager.cc:1403] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.048s	user 0.018s	sys 0.002s
I20260709 13:11:47.541851 30997 raft_consensus.cc:359] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:47.542727 30997 raft_consensus.cc:385] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:47.543063 30997 raft_consensus.cc:740] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
I20260709 13:11:47.544314 30800 raft_consensus.cc:3060] T 39980eee51514ba493958931f406a145 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:47.544083 30997 consensus_queue.cc:260] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:47.551172 30995 ts_tablet_manager.cc:1434] T 27f26e218737414893d5c4bfa4b2cd94 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.000s
I20260709 13:11:47.554214 30997 ts_tablet_manager.cc:1434] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 13:11:47.565986 30997 tablet_bootstrap.cc:492] T 7b988f0495c34b7591ad2fb0d5f65693 P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
I20260709 13:11:47.574366 30994 tablet_bootstrap.cc:492] T d05da516ed9040c988b1fdfe2fdde9ed P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
I20260709 13:11:47.574290 31027 raft_consensus.cc:515] T 11c7485c290740bba00c8b9eaf9b917f P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:47.575028 30994 ts_tablet_manager.cc:1403] T d05da516ed9040c988b1fdfe2fdde9ed P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.052s	user 0.018s	sys 0.018s
I20260709 13:11:47.576597 31027 leader_election.cc:290] T 11c7485c290740bba00c8b9eaf9b917f P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Requested vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:47.577682 30995 tablet_bootstrap.cc:492] T bc2a86f27b0f4fb59d7cfa586a23fef1 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:47.577973 30994 raft_consensus.cc:359] T d05da516ed9040c988b1fdfe2fdde9ed P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:47.578037 30949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c7485c290740bba00c8b9eaf9b917f" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:11:47.578883 30994 raft_consensus.cc:385] T d05da516ed9040c988b1fdfe2fdde9ed P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:47.579093 30949 raft_consensus.cc:3060] T 11c7485c290740bba00c8b9eaf9b917f P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:47.579248 30994 raft_consensus.cc:740] T d05da516ed9040c988b1fdfe2fdde9ed P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
I20260709 13:11:47.580112 30994 consensus_queue.cc:260] T d05da516ed9040c988b1fdfe2fdde9ed P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:47.580843 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c7485c290740bba00c8b9eaf9b917f" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:47.581532 30799 raft_consensus.cc:3060] T 11c7485c290740bba00c8b9eaf9b917f P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:47.584528 30995 tablet_bootstrap.cc:654] T bc2a86f27b0f4fb59d7cfa586a23fef1 P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:47.585135 30949 raft_consensus.cc:2468] T 11c7485c290740bba00c8b9eaf9b917f P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 1.
I20260709 13:11:47.586467 30839 leader_election.cc:304] T 11c7485c290740bba00c8b9eaf9b917f P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
I20260709 13:11:47.587438 31027 raft_consensus.cc:2804] T 11c7485c290740bba00c8b9eaf9b917f P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:47.588836 31027 raft_consensus.cc:697] T 11c7485c290740bba00c8b9eaf9b917f P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Becoming Leader. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Running, Role: LEADER
I20260709 13:11:47.590456 31027 consensus_queue.cc:237] T 11c7485c290740bba00c8b9eaf9b917f P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:47.591408 30996 tablet_bootstrap.cc:492] T efddea050a344563877c3e01ee2a4c81 P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
I20260709 13:11:47.592026 30996 ts_tablet_manager.cc:1403] T efddea050a344563877c3e01ee2a4c81 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.095s	user 0.024s	sys 0.014s
I20260709 13:11:47.590916 30994 ts_tablet_manager.cc:1434] T d05da516ed9040c988b1fdfe2fdde9ed P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:11:47.593592 30994 tablet_bootstrap.cc:492] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
I20260709 13:11:47.597326 30996 raft_consensus.cc:359] T efddea050a344563877c3e01ee2a4c81 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:47.597833 30996 raft_consensus.cc:385] T efddea050a344563877c3e01ee2a4c81 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:47.598095 30996 raft_consensus.cc:740] T efddea050a344563877c3e01ee2a4c81 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
I20260709 13:11:47.598798 31004 raft_consensus.cc:493] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.599151 30995 tablet_bootstrap.cc:492] T bc2a86f27b0f4fb59d7cfa586a23fef1 P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:47.599625 30995 ts_tablet_manager.cc:1403] T bc2a86f27b0f4fb59d7cfa586a23fef1 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.008s
I20260709 13:11:47.599311 31004 raft_consensus.cc:515] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:47.602507 30995 raft_consensus.cc:359] T bc2a86f27b0f4fb59d7cfa586a23fef1 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:47.603186 30995 raft_consensus.cc:385] T bc2a86f27b0f4fb59d7cfa586a23fef1 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:47.603497 30995 raft_consensus.cc:740] T bc2a86f27b0f4fb59d7cfa586a23fef1 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:47.604290 30995 consensus_queue.cc:260] T bc2a86f27b0f4fb59d7cfa586a23fef1 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:47.608618 30994 tablet_bootstrap.cc:654] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:47.609926 30875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2a86f27b0f4fb59d7cfa586a23fef1" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:47.610605 30875 raft_consensus.cc:2468] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05213ac836744a53b9c089a5c052f4be in term 0.
I20260709 13:11:47.611356 31004 leader_election.cc:290] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:47.612438 30766 leader_election.cc:304] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 05213ac836744a53b9c089a5c052f4be; no voters: 
I20260709 13:11:47.615285 30995 ts_tablet_manager.cc:1434] T bc2a86f27b0f4fb59d7cfa586a23fef1 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 13:11:47.615793 31004 raft_consensus.cc:2804] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:47.616214 31004 raft_consensus.cc:493] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:47.616549 31004 raft_consensus.cc:3060] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:47.624457 30995 tablet_bootstrap.cc:492] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:47.627391 30799 raft_consensus.cc:2468] T 11c7485c290740bba00c8b9eaf9b917f P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 1.
I20260709 13:11:47.629575 31063 consensus_queue.cc:1048] T 437c20f3d53f43048eca57834248cc0d P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:47.630481 30996 consensus_queue.cc:260] T efddea050a344563877c3e01ee2a4c81 P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:47.631659 31002 raft_consensus.cc:493] T efddea050a344563877c3e01ee2a4c81 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.632375 31002 raft_consensus.cc:515] T efddea050a344563877c3e01ee2a4c81 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:47.631669 30995 tablet_bootstrap.cc:654] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:47.636070 31002 leader_election.cc:290] T efddea050a344563877c3e01ee2a4c81 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:47.642817 30996 ts_tablet_manager.cc:1434] T efddea050a344563877c3e01ee2a4c81 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.050s	user 0.009s	sys 0.000s
I20260709 13:11:47.643865 30996 tablet_bootstrap.cc:492] T 27f26e218737414893d5c4bfa4b2cd94 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
I20260709 13:11:47.645282 30995 tablet_bootstrap.cc:492] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:47.645857 30995 ts_tablet_manager.cc:1403] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.022s	user 0.006s	sys 0.011s
I20260709 13:11:47.649019 30995 raft_consensus.cc:359] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:47.649668 30995 raft_consensus.cc:385] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:47.649938 30995 raft_consensus.cc:740] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:47.621606 30994 tablet_bootstrap.cc:492] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
I20260709 13:11:47.650624 30995 consensus_queue.cc:260] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:47.652871 30994 ts_tablet_manager.cc:1403] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.060s	user 0.015s	sys 0.001s
I20260709 13:11:47.606889 30997 tablet_bootstrap.cc:654] T 7b988f0495c34b7591ad2fb0d5f65693 P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:47.651170 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efddea050a344563877c3e01ee2a4c81" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:47.665706 31004 raft_consensus.cc:515] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:47.670660 30875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2a86f27b0f4fb59d7cfa586a23fef1" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:11:47.671291 30875 raft_consensus.cc:3060] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:47.674439 30994 raft_consensus.cc:359] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:47.675458 30994 raft_consensus.cc:385] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:47.675500 31046 raft_consensus.cc:493] T b226c44412c74106a0feb2ca411b1661 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.675940 30994 raft_consensus.cc:740] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
I20260709 13:11:47.676180 31046 raft_consensus.cc:515] T b226c44412c74106a0feb2ca411b1661 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:47.676838 30994 consensus_queue.cc:260] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:47.678011 30949 raft_consensus.cc:3060] T b226c44412c74106a0feb2ca411b1661 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:47.678501 31046 leader_election.cc:290] T b226c44412c74106a0feb2ca411b1661 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:47.679742 30874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b226c44412c74106a0feb2ca411b1661" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:47.680256 30801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b226c44412c74106a0feb2ca411b1661" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:47.681267 30997 tablet_bootstrap.cc:492] T 7b988f0495c34b7591ad2fb0d5f65693 P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
I20260709 13:11:47.681819 30997 ts_tablet_manager.cc:1403] T 7b988f0495c34b7591ad2fb0d5f65693 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.116s	user 0.014s	sys 0.000s
I20260709 13:11:47.681955 30912 leader_election.cc:304] T b226c44412c74106a0feb2ca411b1661 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 05213ac836744a53b9c089a5c052f4be
I20260709 13:11:47.686414 30799 raft_consensus.cc:2393] T efddea050a344563877c3e01ee2a4c81 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in current term 1: Already voted for candidate d22c9d3d15904327a78a0e548a8ff6cd in this term.
I20260709 13:11:47.696767 30995 ts_tablet_manager.cc:1434] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.051s	user 0.008s	sys 0.001s
I20260709 13:11:47.697647 30726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efddea050a344563877c3e01ee2a4c81" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:47.698311 30995 tablet_bootstrap.cc:492] T 7ed3eac558b34457aa26b1c40652f887 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:47.699801 31066 raft_consensus.cc:493] T 0ac93952b93645b1ad1d7d96cb60c3e9 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.700492 31066 raft_consensus.cc:515] T 0ac93952b93645b1ad1d7d96cb60c3e9 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:47.708380 30874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ac93952b93645b1ad1d7d96cb60c3e9" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:47.704633 30838 leader_election.cc:304] T efddea050a344563877c3e01ee2a4c81 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a; no voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:47.710137 30996 tablet_bootstrap.cc:654] T 27f26e218737414893d5c4bfa4b2cd94 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:47.690008 31004 leader_election.cc:290] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 election: Requested vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:47.708981 30947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ac93952b93645b1ad1d7d96cb60c3e9" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:47.700393 30875 raft_consensus.cc:2468] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05213ac836744a53b9c089a5c052f4be in term 1.
I20260709 13:11:47.709834 30874 raft_consensus.cc:2468] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 0.
I20260709 13:11:47.720329 30995 tablet_bootstrap.cc:654] T 7ed3eac558b34457aa26b1c40652f887 P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:47.721131 30691 leader_election.cc:304] T 0ac93952b93645b1ad1d7d96cb60c3e9 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:47.722055 31069 raft_consensus.cc:2804] T 0ac93952b93645b1ad1d7d96cb60c3e9 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:47.722447 31069 raft_consensus.cc:493] T 0ac93952b93645b1ad1d7d96cb60c3e9 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:47.721699 30623 catalog_manager.cc:5697] T 11c7485c290740bba00c8b9eaf9b917f P 02c5705b45c04706bbb2c6ff3d7b0b8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3). New cstate: current_term: 1 leader_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:47.722788 31069 raft_consensus.cc:3060] T 0ac93952b93645b1ad1d7d96cb60c3e9 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:47.723017 30766 leader_election.cc:304] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 05213ac836744a53b9c089a5c052f4be; no voters: 
I20260709 13:11:47.716763 31027 raft_consensus.cc:3060] T efddea050a344563877c3e01ee2a4c81 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:47.717419 30947 raft_consensus.cc:2468] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 0.
I20260709 13:11:47.706782 31066 leader_election.cc:290] T 0ac93952b93645b1ad1d7d96cb60c3e9 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:47.719893 30997 raft_consensus.cc:359] T 7b988f0495c34b7591ad2fb0d5f65693 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:47.725328 30997 raft_consensus.cc:385] T 7b988f0495c34b7591ad2fb0d5f65693 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:47.724062 31004 raft_consensus.cc:2804] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:47.725755 30997 raft_consensus.cc:740] T 7b988f0495c34b7591ad2fb0d5f65693 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
I20260709 13:11:47.729560 30994 ts_tablet_manager.cc:1434] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.075s	user 0.013s	sys 0.007s
I20260709 13:11:47.729239 30997 consensus_queue.cc:260] T 7b988f0495c34b7591ad2fb0d5f65693 P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:47.730799 31004 raft_consensus.cc:697] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 05213ac836744a53b9c089a5c052f4be [term 1 LEADER]: Becoming Leader. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Running, Role: LEADER
I20260709 13:11:47.734256 31004 consensus_queue.cc:237] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 05213ac836744a53b9c089a5c052f4be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:47.736888 31109 raft_consensus.cc:493] T d05da516ed9040c988b1fdfe2fdde9ed P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.737259 31069 raft_consensus.cc:515] T 0ac93952b93645b1ad1d7d96cb60c3e9 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:47.737411 31109 raft_consensus.cc:515] T d05da516ed9040c988b1fdfe2fdde9ed P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:47.739619 31109 leader_election.cc:290] T d05da516ed9040c988b1fdfe2fdde9ed P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:47.739952 31069 leader_election.cc:290] T 0ac93952b93645b1ad1d7d96cb60c3e9 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 election: Requested vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:47.741029 31069 raft_consensus.cc:493] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.741703 30874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ac93952b93645b1ad1d7d96cb60c3e9" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:11:47.741631 31069 raft_consensus.cc:515] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:47.744426 30875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d05da516ed9040c988b1fdfe2fdde9ed" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:47.744522 30726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2a86f27b0f4fb59d7cfa586a23fef1" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:47.744767 30724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d05da516ed9040c988b1fdfe2fdde9ed" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:47.745383 30726 raft_consensus.cc:2468] T bc2a86f27b0f4fb59d7cfa586a23fef1 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05213ac836744a53b9c089a5c052f4be in term 0.
W20260709 13:11:47.746058 30766 leader_election.cc:343] T d05da516ed9040c988b1fdfe2fdde9ed P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:47.746501 30725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc2a86f27b0f4fb59d7cfa586a23fef1" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:47.742465 30874 raft_consensus.cc:3060] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:47.747177 30725 raft_consensus.cc:3060] T bc2a86f27b0f4fb59d7cfa586a23fef1 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:47.750602 31063 consensus_queue.cc:1048] T 437c20f3d53f43048eca57834248cc0d P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:11:47.753396 30767 leader_election.cc:343] T d05da516ed9040c988b1fdfe2fdde9ed P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:47.754019 30767 leader_election.cc:304] T d05da516ed9040c988b1fdfe2fdde9ed P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:47.755285 31109 raft_consensus.cc:2749] T d05da516ed9040c988b1fdfe2fdde9ed P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:47.757820 30994 tablet_bootstrap.cc:492] T d4dbe5b2d724412a909fd0641d1a213a P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
I20260709 13:11:47.761534 30997 ts_tablet_manager.cc:1434] T 7b988f0495c34b7591ad2fb0d5f65693 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.079s	user 0.007s	sys 0.004s
I20260709 13:11:47.761824 30948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fb3f8d627b04c8a8971f7e4d3a5c010" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:47.762519 30948 raft_consensus.cc:2468] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 0.
I20260709 13:11:47.762826 30947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ac93952b93645b1ad1d7d96cb60c3e9" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:11:47.763430 30947 raft_consensus.cc:3060] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:47.763756 31069 leader_election.cc:290] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:47.766328 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fb3f8d627b04c8a8971f7e4d3a5c010" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:47.766958 30799 raft_consensus.cc:2468] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 0.
I20260709 13:11:47.768647 30689 leader_election.cc:304] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:47.770943 31066 raft_consensus.cc:2804] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:47.771447 31066 raft_consensus.cc:493] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:47.771890 31066 raft_consensus.cc:3060] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:47.762948 30997 tablet_bootstrap.cc:492] T 27f26e218737414893d5c4bfa4b2cd94 P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
I20260709 13:11:47.779701 31070 consensus_queue.cc:1048] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:47.782171 30947 raft_consensus.cc:2468] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 1.
I20260709 13:11:47.799372 30994 tablet_bootstrap.cc:654] T d4dbe5b2d724412a909fd0641d1a213a P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:47.786026 30874 raft_consensus.cc:2468] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 1.
I20260709 13:11:47.784865 30725 raft_consensus.cc:2468] T bc2a86f27b0f4fb59d7cfa586a23fef1 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05213ac836744a53b9c089a5c052f4be in term 1.
I20260709 13:11:47.792274 31109 raft_consensus.cc:493] T 1474b6fc151c4a3994a3aef89ea936a2 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.808048 30690 leader_election.cc:304] T 0ac93952b93645b1ad1d7d96cb60c3e9 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:47.809724 31070 raft_consensus.cc:2804] T 0ac93952b93645b1ad1d7d96cb60c3e9 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:47.810259 31070 raft_consensus.cc:697] T 0ac93952b93645b1ad1d7d96cb60c3e9 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 LEADER]: Becoming Leader. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Running, Role: LEADER
I20260709 13:11:47.811254 31070 consensus_queue.cc:237] T 0ac93952b93645b1ad1d7d96cb60c3e9 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:47.816442 31046 raft_consensus.cc:2749] T b226c44412c74106a0feb2ca411b1661 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:47.817500 31109 raft_consensus.cc:515] T 1474b6fc151c4a3994a3aef89ea936a2 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:47.820763 31069 raft_consensus.cc:493] T 27f26e218737414893d5c4bfa4b2cd94 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.820883 30724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1474b6fc151c4a3994a3aef89ea936a2" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:47.821724 30724 raft_consensus.cc:2393] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05213ac836744a53b9c089a5c052f4be in current term 1: Already voted for candidate d22c9d3d15904327a78a0e548a8ff6cd in this term.
I20260709 13:11:47.821352 31069 raft_consensus.cc:515] T 27f26e218737414893d5c4bfa4b2cd94 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:47.830215 31046 consensus_queue.cc:1048] T 283eb8311aeb4b5599dd844da98ee0b7 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 13:11:47.842918 31069 leader_election.cc:290] T 27f26e218737414893d5c4bfa4b2cd94 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:47.862313 31089 raft_consensus.cc:493] T 7b988f0495c34b7591ad2fb0d5f65693 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:47.862807 30873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1474b6fc151c4a3994a3aef89ea936a2" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:47.862885 31089 raft_consensus.cc:515] T 7b988f0495c34b7591ad2fb0d5f65693 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:47.864287 30873 raft_consensus.cc:2393] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05213ac836744a53b9c089a5c052f4be in current term 1: Already voted for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in this term.
I20260709 13:11:47.865833 30766 leader_election.cc:304] T 1474b6fc151c4a3994a3aef89ea936a2 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:47.820983 31109 leader_election.cc:290] T 1474b6fc151c4a3994a3aef89ea936a2 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:47.876830 31113 raft_consensus.cc:3060] T 1474b6fc151c4a3994a3aef89ea936a2 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:47.878129 30996 tablet_bootstrap.cc:492] T 27f26e218737414893d5c4bfa4b2cd94 P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
I20260709 13:11:47.878633 30996 ts_tablet_manager.cc:1403] T 27f26e218737414893d5c4bfa4b2cd94 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.235s	user 0.006s	sys 0.009s
I20260709 13:11:47.882598 31089 leader_election.cc:290] T 7b988f0495c34b7591ad2fb0d5f65693 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:47.883133 30801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b988f0495c34b7591ad2fb0d5f65693" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:47.883347 31066 raft_consensus.cc:515] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:47.884491 31113 raft_consensus.cc:2749] T 1474b6fc151c4a3994a3aef89ea936a2 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:47.886003 31066 leader_election.cc:290] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 election: Requested vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:47.886603 30947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27f26e218737414893d5c4bfa4b2cd94" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:47.886828 30949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fb3f8d627b04c8a8971f7e4d3a5c010" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:11:47.887425 30949 raft_consensus.cc:3060] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:47.888211 30801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fb3f8d627b04c8a8971f7e4d3a5c010" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:47.888857 30801 raft_consensus.cc:3060] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:47.890182 31027 raft_consensus.cc:2749] T efddea050a344563877c3e01ee2a4c81 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:11:47.892976 30690 leader_election.cc:343] T 27f26e218737414893d5c4bfa4b2cd94 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:47.893911 30725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b988f0495c34b7591ad2fb0d5f65693" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:47.895047 30997 tablet_bootstrap.cc:654] T 27f26e218737414893d5c4bfa4b2cd94 P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:47.895390 30915 leader_election.cc:304] T 7b988f0495c34b7591ad2fb0d5f65693 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd; no voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:47.896471 30996 raft_consensus.cc:359] T 27f26e218737414893d5c4bfa4b2cd94 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:47.900427 30996 raft_consensus.cc:385] T 27f26e218737414893d5c4bfa4b2cd94 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:47.901052 30996 raft_consensus.cc:740] T 27f26e218737414893d5c4bfa4b2cd94 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
I20260709 13:11:47.897712 30873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27f26e218737414893d5c4bfa4b2cd94" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:47.905232 30996 consensus_queue.cc:260] T 27f26e218737414893d5c4bfa4b2cd94 P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:47.903440 31089 raft_consensus.cc:2749] T 7b988f0495c34b7591ad2fb0d5f65693 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:47.899067 30621 catalog_manager.cc:5697] T 0ac93952b93645b1ad1d7d96cb60c3e9 P d22c9d3d15904327a78a0e548a8ff6cd reported cstate change: term changed from 0 to 1, leader changed from <none> to d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1). New cstate: current_term: 1 leader_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:47.921327 30996 ts_tablet_manager.cc:1434] T 27f26e218737414893d5c4bfa4b2cd94 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.042s	user 0.009s	sys 0.000s
I20260709 13:11:47.927287 30801 raft_consensus.cc:2468] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 1.
I20260709 13:11:47.921103 30873 raft_consensus.cc:2468] T 27f26e218737414893d5c4bfa4b2cd94 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 0.
I20260709 13:11:47.932765 30996 tablet_bootstrap.cc:492] T d05da516ed9040c988b1fdfe2fdde9ed P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
I20260709 13:11:47.934425 30995 tablet_bootstrap.cc:492] T 7ed3eac558b34457aa26b1c40652f887 P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:47.934993 30995 ts_tablet_manager.cc:1403] T 7ed3eac558b34457aa26b1c40652f887 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.237s	user 0.023s	sys 0.043s
I20260709 13:11:47.928656 30689 leader_election.cc:304] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:47.938361 31070 raft_consensus.cc:2804] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:47.938902 31070 raft_consensus.cc:697] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 LEADER]: Becoming Leader. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Running, Role: LEADER
I20260709 13:11:47.940430 30995 raft_consensus.cc:359] T 7ed3eac558b34457aa26b1c40652f887 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:47.941327 30995 raft_consensus.cc:385] T 7ed3eac558b34457aa26b1c40652f887 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:47.941609 30995 raft_consensus.cc:740] T 7ed3eac558b34457aa26b1c40652f887 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:47.939828 31070 consensus_queue.cc:237] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:47.942302 30995 consensus_queue.cc:260] T 7ed3eac558b34457aa26b1c40652f887 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:47.942751 30949 raft_consensus.cc:2468] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 1.
I20260709 13:11:47.935245 31066 consensus_queue.cc:1048] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 13:11:47.944813 30995 ts_tablet_manager.cc:1434] T 7ed3eac558b34457aa26b1c40652f887 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:11:47.945823 30995 tablet_bootstrap.cc:492] T d05da516ed9040c988b1fdfe2fdde9ed P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:47.949031 30996 tablet_bootstrap.cc:654] T d05da516ed9040c988b1fdfe2fdde9ed P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:47.959156 30994 tablet_bootstrap.cc:492] T d4dbe5b2d724412a909fd0641d1a213a P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
I20260709 13:11:47.959951 30994 ts_tablet_manager.cc:1403] T d4dbe5b2d724412a909fd0641d1a213a P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.229s	user 0.013s	sys 0.024s
I20260709 13:11:47.963979 30994 raft_consensus.cc:359] T d4dbe5b2d724412a909fd0641d1a213a P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:47.965051 30994 raft_consensus.cc:385] T d4dbe5b2d724412a909fd0641d1a213a P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:47.965981 30994 raft_consensus.cc:740] T d4dbe5b2d724412a909fd0641d1a213a P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
I20260709 13:11:47.971055 30995 tablet_bootstrap.cc:654] T d05da516ed9040c988b1fdfe2fdde9ed P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:47.967151 30994 consensus_queue.cc:260] T d4dbe5b2d724412a909fd0641d1a213a P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:47.974390 30994 ts_tablet_manager.cc:1434] T d4dbe5b2d724412a909fd0641d1a213a P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.004s
I20260709 13:11:47.975864 30994 tablet_bootstrap.cc:492] T bbb2acd63baa48a3a2ace94016522e92 P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
I20260709 13:11:47.978490 31046 consensus_queue.cc:1048] T 283eb8311aeb4b5599dd844da98ee0b7 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:47.982178 30994 tablet_bootstrap.cc:654] T bbb2acd63baa48a3a2ace94016522e92 P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:47.998080 30691 leader_election.cc:304] T 27f26e218737414893d5c4bfa4b2cd94 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 262942f67dfa4b8fa42234b18ac31ffd
I20260709 13:11:47.999331 31070 raft_consensus.cc:2804] T 27f26e218737414893d5c4bfa4b2cd94 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:47.999730 31070 raft_consensus.cc:493] T 27f26e218737414893d5c4bfa4b2cd94 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.000083 31070 raft_consensus.cc:3060] T 27f26e218737414893d5c4bfa4b2cd94 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:48.009845 30624 catalog_manager.cc:5697] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd reported cstate change: term changed from 0 to 1, leader changed from <none> to d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1). New cstate: current_term: 1 leader_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:48.017053 30996 tablet_bootstrap.cc:492] T d05da516ed9040c988b1fdfe2fdde9ed P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
I20260709 13:11:48.019004 31070 raft_consensus.cc:515] T 27f26e218737414893d5c4bfa4b2cd94 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:48.020784 30997 tablet_bootstrap.cc:492] T 27f26e218737414893d5c4bfa4b2cd94 P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
I20260709 13:11:48.029628 30997 ts_tablet_manager.cc:1403] T 27f26e218737414893d5c4bfa4b2cd94 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.267s	user 0.023s	sys 0.004s
I20260709 13:11:48.032801 30997 raft_consensus.cc:359] T 27f26e218737414893d5c4bfa4b2cd94 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:48.021920 30996 ts_tablet_manager.cc:1403] T d05da516ed9040c988b1fdfe2fdde9ed P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.090s	user 0.020s	sys 0.008s
I20260709 13:11:48.024740 30995 tablet_bootstrap.cc:492] T d05da516ed9040c988b1fdfe2fdde9ed P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:48.033704 31027 consensus_queue.cc:1048] T 11c7485c290740bba00c8b9eaf9b917f P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:48.034583 30995 ts_tablet_manager.cc:1403] T d05da516ed9040c988b1fdfe2fdde9ed P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.089s	user 0.010s	sys 0.013s
I20260709 13:11:48.038694 30996 raft_consensus.cc:359] T d05da516ed9040c988b1fdfe2fdde9ed P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:48.039700 30996 raft_consensus.cc:385] T d05da516ed9040c988b1fdfe2fdde9ed P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.039989 30996 raft_consensus.cc:740] T d05da516ed9040c988b1fdfe2fdde9ed P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.040386 30997 raft_consensus.cc:385] T 27f26e218737414893d5c4bfa4b2cd94 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.038757 30622 catalog_manager.cc:5697] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 05213ac836744a53b9c089a5c052f4be reported cstate change: term changed from 0 to 1, leader changed from <none> to 05213ac836744a53b9c089a5c052f4be (127.21.63.2). New cstate: current_term: 1 leader_uuid: "05213ac836744a53b9c089a5c052f4be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:48.040750 30997 raft_consensus.cc:740] T 27f26e218737414893d5c4bfa4b2cd94 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.042627 30996 consensus_queue.cc:260] T d05da516ed9040c988b1fdfe2fdde9ed P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:48.044764 30949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27f26e218737414893d5c4bfa4b2cd94" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:11:48.046042 30996 ts_tablet_manager.cc:1434] T d05da516ed9040c988b1fdfe2fdde9ed P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.002s
I20260709 13:11:48.046912 30996 tablet_bootstrap.cc:492] T d4dbe5b2d724412a909fd0641d1a213a P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
I20260709 13:11:48.047986 30997 consensus_queue.cc:260] T 27f26e218737414893d5c4bfa4b2cd94 P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:48.048974 30949 raft_consensus.cc:3060] T 27f26e218737414893d5c4bfa4b2cd94 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:48.049229 30873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27f26e218737414893d5c4bfa4b2cd94" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:11:48.049795 30873 raft_consensus.cc:3060] T 27f26e218737414893d5c4bfa4b2cd94 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:48.050279 30997 ts_tablet_manager.cc:1434] T 27f26e218737414893d5c4bfa4b2cd94 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.000s
I20260709 13:11:48.051540 30997 tablet_bootstrap.cc:492] T bbb2acd63baa48a3a2ace94016522e92 P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
I20260709 13:11:48.053354 31070 leader_election.cc:290] T 27f26e218737414893d5c4bfa4b2cd94 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 election: Requested vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:48.047243 30995 raft_consensus.cc:359] T d05da516ed9040c988b1fdfe2fdde9ed P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:48.054600 30995 raft_consensus.cc:385] T d05da516ed9040c988b1fdfe2fdde9ed P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.054936 30995 raft_consensus.cc:740] T d05da516ed9040c988b1fdfe2fdde9ed P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.056107 30949 raft_consensus.cc:2468] T 27f26e218737414893d5c4bfa4b2cd94 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 1.
I20260709 13:11:48.055811 30995 consensus_queue.cc:260] T d05da516ed9040c988b1fdfe2fdde9ed P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:48.058401 30995 ts_tablet_manager.cc:1434] T d05da516ed9040c988b1fdfe2fdde9ed P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.000s
I20260709 13:11:48.059321 30995 tablet_bootstrap.cc:492] T 64a95fa1266848048486722c02f0d4cf P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:48.060262 30996 tablet_bootstrap.cc:654] T d4dbe5b2d724412a909fd0641d1a213a P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.061585 30997 tablet_bootstrap.cc:654] T bbb2acd63baa48a3a2ace94016522e92 P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.064492 30994 tablet_bootstrap.cc:492] T bbb2acd63baa48a3a2ace94016522e92 P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
I20260709 13:11:48.064958 30994 ts_tablet_manager.cc:1403] T bbb2acd63baa48a3a2ace94016522e92 P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.089s	user 0.017s	sys 0.014s
I20260709 13:11:48.065021 30690 leader_election.cc:304] T 27f26e218737414893d5c4bfa4b2cd94 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:48.065997 31070 raft_consensus.cc:2804] T 27f26e218737414893d5c4bfa4b2cd94 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:48.066457 31070 raft_consensus.cc:697] T 27f26e218737414893d5c4bfa4b2cd94 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 LEADER]: Becoming Leader. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Running, Role: LEADER
I20260709 13:11:48.067251 31070 consensus_queue.cc:237] T 27f26e218737414893d5c4bfa4b2cd94 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:48.071168 30994 raft_consensus.cc:359] T bbb2acd63baa48a3a2ace94016522e92 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:48.071841 30994 raft_consensus.cc:385] T bbb2acd63baa48a3a2ace94016522e92 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.072110 30994 raft_consensus.cc:740] T bbb2acd63baa48a3a2ace94016522e92 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.072782 30994 consensus_queue.cc:260] T bbb2acd63baa48a3a2ace94016522e92 P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:48.074203 30995 tablet_bootstrap.cc:654] T 64a95fa1266848048486722c02f0d4cf P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.077025 30873 raft_consensus.cc:2468] T 27f26e218737414893d5c4bfa4b2cd94 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 1.
I20260709 13:11:48.077806 30994 ts_tablet_manager.cc:1434] T bbb2acd63baa48a3a2ace94016522e92 P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.013s	user 0.004s	sys 0.002s
I20260709 13:11:48.078761 30994 tablet_bootstrap.cc:492] T d7edce5d08f24852b7afca9c57ba9c19 P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
I20260709 13:11:48.084354 30994 tablet_bootstrap.cc:654] T d7edce5d08f24852b7afca9c57ba9c19 P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.088155 31115 consensus_queue.cc:1048] T 11c7485c290740bba00c8b9eaf9b917f P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:48.098564 30997 tablet_bootstrap.cc:492] T bbb2acd63baa48a3a2ace94016522e92 P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
I20260709 13:11:48.099067 30997 ts_tablet_manager.cc:1403] T bbb2acd63baa48a3a2ace94016522e92 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.048s	user 0.015s	sys 0.019s
I20260709 13:11:48.101886 30997 raft_consensus.cc:359] T bbb2acd63baa48a3a2ace94016522e92 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:48.102978 30997 raft_consensus.cc:385] T bbb2acd63baa48a3a2ace94016522e92 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.103291 30997 raft_consensus.cc:740] T bbb2acd63baa48a3a2ace94016522e92 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.104283 30997 consensus_queue.cc:260] T bbb2acd63baa48a3a2ace94016522e92 P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:48.105695 30996 tablet_bootstrap.cc:492] T d4dbe5b2d724412a909fd0641d1a213a P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
I20260709 13:11:48.106189 30996 ts_tablet_manager.cc:1403] T d4dbe5b2d724412a909fd0641d1a213a P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.059s	user 0.013s	sys 0.001s
I20260709 13:11:48.107215 30997 ts_tablet_manager.cc:1434] T bbb2acd63baa48a3a2ace94016522e92 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:11:48.108402 30997 tablet_bootstrap.cc:492] T d4dbe5b2d724412a909fd0641d1a213a P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
I20260709 13:11:48.112241 30949 raft_consensus.cc:3060] T 7b988f0495c34b7591ad2fb0d5f65693 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:48.115036 31115 raft_consensus.cc:493] T d05da516ed9040c988b1fdfe2fdde9ed P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.115527 31115 raft_consensus.cc:515] T d05da516ed9040c988b1fdfe2fdde9ed P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:48.119457 30801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d05da516ed9040c988b1fdfe2fdde9ed" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:48.120141 30801 raft_consensus.cc:2468] T d05da516ed9040c988b1fdfe2fdde9ed P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 0.
I20260709 13:11:48.121172 30726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d05da516ed9040c988b1fdfe2fdde9ed" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:48.121850 30726 raft_consensus.cc:2468] T d05da516ed9040c988b1fdfe2fdde9ed P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 0.
I20260709 13:11:48.123610 30841 leader_election.cc:304] T d05da516ed9040c988b1fdfe2fdde9ed P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:48.124486 31002 raft_consensus.cc:2804] T d05da516ed9040c988b1fdfe2fdde9ed P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:48.124871 31002 raft_consensus.cc:493] T d05da516ed9040c988b1fdfe2fdde9ed P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.125195 31002 raft_consensus.cc:3060] T d05da516ed9040c988b1fdfe2fdde9ed P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:48.129463 31115 leader_election.cc:290] T d05da516ed9040c988b1fdfe2fdde9ed P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:48.132118 30994 tablet_bootstrap.cc:492] T d7edce5d08f24852b7afca9c57ba9c19 P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
I20260709 13:11:48.131397 30621 catalog_manager.cc:5697] T 27f26e218737414893d5c4bfa4b2cd94 P d22c9d3d15904327a78a0e548a8ff6cd reported cstate change: term changed from 0 to 1, leader changed from <none> to d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1). New cstate: current_term: 1 leader_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:48.132625 30994 ts_tablet_manager.cc:1403] T d7edce5d08f24852b7afca9c57ba9c19 P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.054s	user 0.017s	sys 0.009s
I20260709 13:11:48.135418 30994 raft_consensus.cc:359] T d7edce5d08f24852b7afca9c57ba9c19 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:48.136312 30994 raft_consensus.cc:385] T d7edce5d08f24852b7afca9c57ba9c19 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.136564 30994 raft_consensus.cc:740] T d7edce5d08f24852b7afca9c57ba9c19 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.137512 30995 tablet_bootstrap.cc:492] T 64a95fa1266848048486722c02f0d4cf P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:48.137245 30994 consensus_queue.cc:260] T d7edce5d08f24852b7afca9c57ba9c19 P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:48.137997 30995 ts_tablet_manager.cc:1403] T 64a95fa1266848048486722c02f0d4cf P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.079s	user 0.011s	sys 0.000s
I20260709 13:11:48.140848 30997 tablet_bootstrap.cc:654] T d4dbe5b2d724412a909fd0641d1a213a P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.142736 30995 raft_consensus.cc:359] T 64a95fa1266848048486722c02f0d4cf P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:48.143246 30995 raft_consensus.cc:385] T 64a95fa1266848048486722c02f0d4cf P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.143503 30995 raft_consensus.cc:740] T 64a95fa1266848048486722c02f0d4cf P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.144137 30995 consensus_queue.cc:260] T 64a95fa1266848048486722c02f0d4cf P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:48.145779 31002 raft_consensus.cc:515] T d05da516ed9040c988b1fdfe2fdde9ed P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:48.108845 30996 raft_consensus.cc:359] T d4dbe5b2d724412a909fd0641d1a213a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:48.147740 30996 raft_consensus.cc:385] T d4dbe5b2d724412a909fd0641d1a213a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.148047 30996 raft_consensus.cc:740] T d4dbe5b2d724412a909fd0641d1a213a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.148742 30996 consensus_queue.cc:260] T d4dbe5b2d724412a909fd0641d1a213a P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:48.139356 30994 ts_tablet_manager.cc:1434] T d7edce5d08f24852b7afca9c57ba9c19 P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:11:48.149433 30726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d05da516ed9040c988b1fdfe2fdde9ed" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:48.149989 30726 raft_consensus.cc:3060] T d05da516ed9040c988b1fdfe2fdde9ed P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:48.150315 30994 tablet_bootstrap.cc:492] T 64a95fa1266848048486722c02f0d4cf P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
I20260709 13:11:48.146260 30995 ts_tablet_manager.cc:1434] T 64a95fa1266848048486722c02f0d4cf P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:11:48.154860 30995 tablet_bootstrap.cc:492] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:48.162092 30726 raft_consensus.cc:2468] T d05da516ed9040c988b1fdfe2fdde9ed P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 1.
I20260709 13:11:48.163507 30841 leader_election.cc:304] T d05da516ed9040c988b1fdfe2fdde9ed P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:48.164433 31115 raft_consensus.cc:2804] T d05da516ed9040c988b1fdfe2fdde9ed P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:48.164909 31115 raft_consensus.cc:697] T d05da516ed9040c988b1fdfe2fdde9ed P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Becoming Leader. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Running, Role: LEADER
I20260709 13:11:48.165859 30994 tablet_bootstrap.cc:654] T 64a95fa1266848048486722c02f0d4cf P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.165730 31115 consensus_queue.cc:237] T d05da516ed9040c988b1fdfe2fdde9ed P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:48.160540 30995 tablet_bootstrap.cc:654] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.171311 31002 leader_election.cc:290] T d05da516ed9040c988b1fdfe2fdde9ed P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Requested vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:48.172520 30801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d05da516ed9040c988b1fdfe2fdde9ed" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:48.173264 30801 raft_consensus.cc:3060] T d05da516ed9040c988b1fdfe2fdde9ed P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:48.162302 30996 ts_tablet_manager.cc:1434] T d4dbe5b2d724412a909fd0641d1a213a P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.056s	user 0.006s	sys 0.000s
I20260709 13:11:48.179556 31113 consensus_queue.cc:1048] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 05213ac836744a53b9c089a5c052f4be [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:48.177053 30726 raft_consensus.cc:3060] T 7ed3eac558b34457aa26b1c40652f887 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:48.181954 30801 raft_consensus.cc:2468] T d05da516ed9040c988b1fdfe2fdde9ed P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 1.
I20260709 13:11:48.184218 30996 tablet_bootstrap.cc:492] T 8770fcfd98154b2095c2c9f74135593b P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
I20260709 13:11:48.195403 31109 consensus_queue.cc:1048] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 05213ac836744a53b9c089a5c052f4be [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:48.211836 30997 tablet_bootstrap.cc:492] T d4dbe5b2d724412a909fd0641d1a213a P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
I20260709 13:11:48.222494 30997 ts_tablet_manager.cc:1403] T d4dbe5b2d724412a909fd0641d1a213a P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.114s	user 0.030s	sys 0.035s
I20260709 13:11:48.225919 30997 raft_consensus.cc:359] T d4dbe5b2d724412a909fd0641d1a213a P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:48.226686 30997 raft_consensus.cc:385] T d4dbe5b2d724412a909fd0641d1a213a P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.227012 30997 raft_consensus.cc:740] T d4dbe5b2d724412a909fd0641d1a213a P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.227873 30997 consensus_queue.cc:260] T d4dbe5b2d724412a909fd0641d1a213a P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:48.214341 30624 catalog_manager.cc:5697] T d05da516ed9040c988b1fdfe2fdde9ed P 02c5705b45c04706bbb2c6ff3d7b0b8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3). New cstate: current_term: 1 leader_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:48.233036 30997 ts_tablet_manager.cc:1434] T d4dbe5b2d724412a909fd0641d1a213a P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:11:48.234371 30997 tablet_bootstrap.cc:492] T 8770fcfd98154b2095c2c9f74135593b P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
I20260709 13:11:48.235077 30994 tablet_bootstrap.cc:492] T 64a95fa1266848048486722c02f0d4cf P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
I20260709 13:11:48.237632 30995 tablet_bootstrap.cc:492] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:48.238214 30995 ts_tablet_manager.cc:1403] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.084s	user 0.018s	sys 0.008s
I20260709 13:11:48.240574 30997 tablet_bootstrap.cc:654] T 8770fcfd98154b2095c2c9f74135593b P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.242441 30996 tablet_bootstrap.cc:654] T 8770fcfd98154b2095c2c9f74135593b P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.247231 30994 ts_tablet_manager.cc:1403] T 64a95fa1266848048486722c02f0d4cf P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.086s	user 0.015s	sys 0.010s
I20260709 13:11:48.250713 30995 raft_consensus.cc:359] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:48.251418 30995 raft_consensus.cc:385] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.251681 30995 raft_consensus.cc:740] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.252348 30995 consensus_queue.cc:260] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:48.254004 30997 tablet_bootstrap.cc:492] T 8770fcfd98154b2095c2c9f74135593b P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
I20260709 13:11:48.254528 30997 ts_tablet_manager.cc:1403] T 8770fcfd98154b2095c2c9f74135593b P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.000s
I20260709 13:11:48.254436 30994 raft_consensus.cc:359] T 64a95fa1266848048486722c02f0d4cf P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:48.255177 30994 raft_consensus.cc:385] T 64a95fa1266848048486722c02f0d4cf P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.255435 30994 raft_consensus.cc:740] T 64a95fa1266848048486722c02f0d4cf P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.256075 30994 consensus_queue.cc:260] T 64a95fa1266848048486722c02f0d4cf P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:48.257421 30997 raft_consensus.cc:359] T 8770fcfd98154b2095c2c9f74135593b P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:48.257918 30994 ts_tablet_manager.cc:1434] T 64a95fa1266848048486722c02f0d4cf P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 13:11:48.258090 30997 raft_consensus.cc:385] T 8770fcfd98154b2095c2c9f74135593b P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.258433 30997 raft_consensus.cc:740] T 8770fcfd98154b2095c2c9f74135593b P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.258561 31046 raft_consensus.cc:493] T d4dbe5b2d724412a909fd0641d1a213a P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.259045 31046 raft_consensus.cc:515] T d4dbe5b2d724412a909fd0641d1a213a P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:48.259130 30997 consensus_queue.cc:260] T 8770fcfd98154b2095c2c9f74135593b P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:48.260133 30995 ts_tablet_manager.cc:1434] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 13:11:48.261126 30994 tablet_bootstrap.cc:492] T c02f992ce0974544a4112672d8792c5f P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
I20260709 13:11:48.261273 30997 ts_tablet_manager.cc:1434] T 8770fcfd98154b2095c2c9f74135593b P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:11:48.262669 31070 raft_consensus.cc:493] T 7ed3eac558b34457aa26b1c40652f887 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 262942f67dfa4b8fa42234b18ac31ffd)
I20260709 13:11:48.262939 30801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4dbe5b2d724412a909fd0641d1a213a" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:48.263062 30997 tablet_bootstrap.cc:492] T 726b808ed64848dd89da4340d4d891fb P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
I20260709 13:11:48.263151 31070 raft_consensus.cc:515] T 7ed3eac558b34457aa26b1c40652f887 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:48.263664 30801 raft_consensus.cc:2468] T d4dbe5b2d724412a909fd0641d1a213a P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 0.
I20260709 13:11:48.265118 30912 leader_election.cc:304] T d4dbe5b2d724412a909fd0641d1a213a P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
I20260709 13:11:48.265928 31089 raft_consensus.cc:2804] T d4dbe5b2d724412a909fd0641d1a213a P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:48.266273 31089 raft_consensus.cc:493] T d4dbe5b2d724412a909fd0641d1a213a P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.266607 31089 raft_consensus.cc:3060] T d4dbe5b2d724412a909fd0641d1a213a P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:48.267880 30949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ed3eac558b34457aa26b1c40652f887" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:48.269548 30997 tablet_bootstrap.cc:654] T 726b808ed64848dd89da4340d4d891fb P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.270891 31070 leader_election.cc:290] T 7ed3eac558b34457aa26b1c40652f887 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:48.271847 30801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ed3eac558b34457aa26b1c40652f887" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:48.272528 30873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4dbe5b2d724412a909fd0641d1a213a" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:48.273190 30873 raft_consensus.cc:2468] T d4dbe5b2d724412a909fd0641d1a213a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 0.
I20260709 13:11:48.273414 30689 leader_election.cc:304] T 7ed3eac558b34457aa26b1c40652f887 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22c9d3d15904327a78a0e548a8ff6cd; no voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd
I20260709 13:11:48.274573 31070 raft_consensus.cc:2749] T 7ed3eac558b34457aa26b1c40652f887 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:48.261379 30996 tablet_bootstrap.cc:492] T 8770fcfd98154b2095c2c9f74135593b P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
I20260709 13:11:48.276484 30996 ts_tablet_manager.cc:1403] T 8770fcfd98154b2095c2c9f74135593b P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.092s	user 0.012s	sys 0.000s
I20260709 13:11:48.278271 31070 raft_consensus.cc:493] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.278822 31070 raft_consensus.cc:515] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:48.282377 30949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8770fcfd98154b2095c2c9f74135593b" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:48.283016 30949 raft_consensus.cc:2468] T 8770fcfd98154b2095c2c9f74135593b P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 0.
I20260709 13:11:48.286679 31089 raft_consensus.cc:515] T d4dbe5b2d724412a909fd0641d1a213a P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:48.283020 30873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8770fcfd98154b2095c2c9f74135593b" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:48.287891 30690 leader_election.cc:304] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:48.288381 31070 leader_election.cc:290] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:48.261088 30995 tablet_bootstrap.cc:492] T bbb2acd63baa48a3a2ace94016522e92 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:48.292549 31068 raft_consensus.cc:2804] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:48.278277 30994 tablet_bootstrap.cc:654] T c02f992ce0974544a4112672d8792c5f P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.292953 31068 raft_consensus.cc:493] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.294844 31068 raft_consensus.cc:3060] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:11:48.296543 30691 leader_election.cc:343] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:48.261063 31046 leader_election.cc:290] T d4dbe5b2d724412a909fd0641d1a213a P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:48.297894 30995 tablet_bootstrap.cc:654] T bbb2acd63baa48a3a2ace94016522e92 P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.299700 30994 tablet_bootstrap.cc:492] T c02f992ce0974544a4112672d8792c5f P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
I20260709 13:11:48.300220 30994 ts_tablet_manager.cc:1403] T c02f992ce0974544a4112672d8792c5f P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.042s	user 0.005s	sys 0.006s
I20260709 13:11:48.300611 31089 leader_election.cc:290] T d4dbe5b2d724412a909fd0641d1a213a P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Requested vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:48.293845 30801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4dbe5b2d724412a909fd0641d1a213a" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:48.301431 30801 raft_consensus.cc:3060] T d4dbe5b2d724412a909fd0641d1a213a P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:48.302816 30873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d4dbe5b2d724412a909fd0641d1a213a" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:11:48.303021 30994 raft_consensus.cc:359] T c02f992ce0974544a4112672d8792c5f P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:48.303591 30873 raft_consensus.cc:3060] T d4dbe5b2d724412a909fd0641d1a213a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:48.303725 30994 raft_consensus.cc:385] T c02f992ce0974544a4112672d8792c5f P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.304111 30994 raft_consensus.cc:740] T c02f992ce0974544a4112672d8792c5f P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.304792 30994 consensus_queue.cc:260] T c02f992ce0974544a4112672d8792c5f P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:48.305552 31004 raft_consensus.cc:493] T c02f992ce0974544a4112672d8792c5f P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.306167 31004 raft_consensus.cc:515] T c02f992ce0974544a4112672d8792c5f P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:48.307089 30994 ts_tablet_manager.cc:1434] T c02f992ce0974544a4112672d8792c5f P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:11:48.307996 30994 tablet_bootstrap.cc:492] T ccc68dd4c56548cba0624daafb5ae23c P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
I20260709 13:11:48.308254 31068 raft_consensus.cc:515] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:48.309461 30874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c02f992ce0974544a4112672d8792c5f" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:48.310348 31004 leader_election.cc:290] T c02f992ce0974544a4112672d8792c5f P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:48.308002 30801 raft_consensus.cc:2468] T d4dbe5b2d724412a909fd0641d1a213a P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 1.
I20260709 13:11:48.294137 30997 tablet_bootstrap.cc:492] T 726b808ed64848dd89da4340d4d891fb P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
W20260709 13:11:48.311359 30766 leader_election.cc:343] T c02f992ce0974544a4112672d8792c5f P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:48.311772 30997 ts_tablet_manager.cc:1403] T 726b808ed64848dd89da4340d4d891fb P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.049s	user 0.016s	sys 0.005s
I20260709 13:11:48.312142 30912 leader_election.cc:304] T d4dbe5b2d724412a909fd0641d1a213a P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
I20260709 13:11:48.313126 31089 raft_consensus.cc:2804] T d4dbe5b2d724412a909fd0641d1a213a P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:48.313604 31089 raft_consensus.cc:697] T d4dbe5b2d724412a909fd0641d1a213a P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Becoming Leader. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Running, Role: LEADER
I20260709 13:11:48.314287 30994 tablet_bootstrap.cc:654] T ccc68dd4c56548cba0624daafb5ae23c P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.314477 30997 raft_consensus.cc:359] T 726b808ed64848dd89da4340d4d891fb P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:48.315207 30997 raft_consensus.cc:385] T 726b808ed64848dd89da4340d4d891fb P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.315681 30997 raft_consensus.cc:740] T 726b808ed64848dd89da4340d4d891fb P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.314435 31089 consensus_queue.cc:237] T d4dbe5b2d724412a909fd0641d1a213a P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:48.317344 31068 leader_election.cc:290] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 election: Requested vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:48.317716 30873 raft_consensus.cc:2468] T d4dbe5b2d724412a909fd0641d1a213a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 1.
I20260709 13:11:48.292621 30996 raft_consensus.cc:359] T 8770fcfd98154b2095c2c9f74135593b P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:48.318885 30996 raft_consensus.cc:385] T 8770fcfd98154b2095c2c9f74135593b P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.319118 30996 raft_consensus.cc:740] T 8770fcfd98154b2095c2c9f74135593b P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.319783 30949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8770fcfd98154b2095c2c9f74135593b" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:11:48.319867 30996 consensus_queue.cc:260] T 8770fcfd98154b2095c2c9f74135593b P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:48.320483 30949 raft_consensus.cc:3060] T 8770fcfd98154b2095c2c9f74135593b P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:48.322786 30996 ts_tablet_manager.cc:1434] T 8770fcfd98154b2095c2c9f74135593b P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.046s	user 0.005s	sys 0.000s
I20260709 13:11:48.323994 30996 tablet_bootstrap.cc:492] T 5028253a91e74a73afca005c77e8b565 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
I20260709 13:11:48.316435 30997 consensus_queue.cc:260] T 726b808ed64848dd89da4340d4d891fb P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:48.327109 30997 ts_tablet_manager.cc:1434] T 726b808ed64848dd89da4340d4d891fb P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.000s
I20260709 13:11:48.328480 30949 raft_consensus.cc:2468] T 8770fcfd98154b2095c2c9f74135593b P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 1.
I20260709 13:11:48.315208 30874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8770fcfd98154b2095c2c9f74135593b" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:11:48.330013 30690 leader_election.cc:304] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:48.330765 30874 raft_consensus.cc:3060] T 8770fcfd98154b2095c2c9f74135593b P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:48.331063 31070 raft_consensus.cc:2804] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:48.331535 31070 raft_consensus.cc:697] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd [term 1 LEADER]: Becoming Leader. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Running, Role: LEADER
I20260709 13:11:48.332324 31070 consensus_queue.cc:237] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:48.334497 30947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c02f992ce0974544a4112672d8792c5f" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:48.328168 30997 tablet_bootstrap.cc:492] T 26172e2c14cd4d178764ed13012c21f7 P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
W20260709 13:11:48.335793 30765 leader_election.cc:343] T c02f992ce0974544a4112672d8792c5f P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:48.336267 30765 leader_election.cc:304] T c02f992ce0974544a4112672d8792c5f P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 262942f67dfa4b8fa42234b18ac31ffd
I20260709 13:11:48.337946 31004 raft_consensus.cc:2749] T c02f992ce0974544a4112672d8792c5f P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:48.347144 30996 tablet_bootstrap.cc:654] T 5028253a91e74a73afca005c77e8b565 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.350564 30997 tablet_bootstrap.cc:654] T 26172e2c14cd4d178764ed13012c21f7 P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.359737 30619 catalog_manager.cc:5697] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd reported cstate change: term changed from 0 to 1, leader changed from <none> to d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1). New cstate: current_term: 1 leader_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:48.360811 31046 raft_consensus.cc:493] T bbb2acd63baa48a3a2ace94016522e92 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.364544 31046 raft_consensus.cc:515] T bbb2acd63baa48a3a2ace94016522e92 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:48.374881 30725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbb2acd63baa48a3a2ace94016522e92" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:48.377158 31004 raft_consensus.cc:493] T bbb2acd63baa48a3a2ace94016522e92 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.377844 31004 raft_consensus.cc:515] T bbb2acd63baa48a3a2ace94016522e92 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:48.378917 30874 raft_consensus.cc:2468] T 8770fcfd98154b2095c2c9f74135593b P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 1.
I20260709 13:11:48.383764 31004 leader_election.cc:290] T bbb2acd63baa48a3a2ace94016522e92 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
W20260709 13:11:48.376459 30915 leader_election.cc:343] T bbb2acd63baa48a3a2ace94016522e92 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:48.384684 31109 raft_consensus.cc:493] T d7edce5d08f24852b7afca9c57ba9c19 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.385151 31109 raft_consensus.cc:515] T d7edce5d08f24852b7afca9c57ba9c19 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:48.387108 31109 leader_election.cc:290] T d7edce5d08f24852b7afca9c57ba9c19 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:48.389684 30948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbb2acd63baa48a3a2ace94016522e92" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:48.390317 30948 raft_consensus.cc:2468] T bbb2acd63baa48a3a2ace94016522e92 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05213ac836744a53b9c089a5c052f4be in term 0.
I20260709 13:11:48.391608 30725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7edce5d08f24852b7afca9c57ba9c19" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:48.391776 31046 leader_election.cc:290] T bbb2acd63baa48a3a2ace94016522e92 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
W20260709 13:11:48.393546 30767 leader_election.cc:343] T d7edce5d08f24852b7afca9c57ba9c19 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:48.394629 30765 leader_election.cc:304] T bbb2acd63baa48a3a2ace94016522e92 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
I20260709 13:11:48.398015 30801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbb2acd63baa48a3a2ace94016522e92" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:48.398674 30801 raft_consensus.cc:2468] T bbb2acd63baa48a3a2ace94016522e92 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 0.
I20260709 13:11:48.399983 31069 consensus_queue.cc:1048] T 0ac93952b93645b1ad1d7d96cb60c3e9 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:48.400367 30912 leader_election.cc:304] T bbb2acd63baa48a3a2ace94016522e92 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:48.401345 31069 consensus_queue.cc:1048] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:48.401759 31082 raft_consensus.cc:2804] T bbb2acd63baa48a3a2ace94016522e92 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:48.402140 31082 raft_consensus.cc:493] T bbb2acd63baa48a3a2ace94016522e92 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.402472 31082 raft_consensus.cc:3060] T bbb2acd63baa48a3a2ace94016522e92 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:48.406926 30947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7edce5d08f24852b7afca9c57ba9c19" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:48.399847 31109 raft_consensus.cc:2804] T bbb2acd63baa48a3a2ace94016522e92 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:48.408396 31109 raft_consensus.cc:493] T bbb2acd63baa48a3a2ace94016522e92 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.408878 31109 raft_consensus.cc:3060] T bbb2acd63baa48a3a2ace94016522e92 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:48.415786 30726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbb2acd63baa48a3a2ace94016522e92" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
W20260709 13:11:48.417173 30765 leader_election.cc:343] T d7edce5d08f24852b7afca9c57ba9c19 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:48.417850 30765 leader_election.cc:304] T d7edce5d08f24852b7afca9c57ba9c19 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be; no voters: 262942f67dfa4b8fa42234b18ac31ffd, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:48.418959 31004 raft_consensus.cc:2749] T d7edce5d08f24852b7afca9c57ba9c19 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:11:48.420755 30767 leader_election.cc:343] T bbb2acd63baa48a3a2ace94016522e92 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:48.416553 31082 raft_consensus.cc:515] T bbb2acd63baa48a3a2ace94016522e92 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
W20260709 13:11:48.429004 31151 log.cc:927] Time spent T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 13:11:48.434913 30726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbb2acd63baa48a3a2ace94016522e92" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
W20260709 13:11:48.436461 30915 leader_election.cc:343] T bbb2acd63baa48a3a2ace94016522e92 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:11:48.438321 30671 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:11:48.441656 31109 raft_consensus.cc:515] T bbb2acd63baa48a3a2ace94016522e92 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:48.442376 30997 tablet_bootstrap.cc:492] T 26172e2c14cd4d178764ed13012c21f7 P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
I20260709 13:11:48.464725 30997 ts_tablet_manager.cc:1403] T 26172e2c14cd4d178764ed13012c21f7 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.137s	user 0.023s	sys 0.025s
I20260709 13:11:48.465617 30996 tablet_bootstrap.cc:492] T 5028253a91e74a73afca005c77e8b565 P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
I20260709 13:11:48.466146 30996 ts_tablet_manager.cc:1403] T 5028253a91e74a73afca005c77e8b565 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.142s	user 0.066s	sys 0.057s
I20260709 13:11:48.465637 30994 tablet_bootstrap.cc:492] T ccc68dd4c56548cba0624daafb5ae23c P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
I20260709 13:11:48.448288 31068 consensus_queue.cc:1048] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:48.467374 30997 raft_consensus.cc:359] T 26172e2c14cd4d178764ed13012c21f7 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:48.467952 30997 raft_consensus.cc:385] T 26172e2c14cd4d178764ed13012c21f7 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.468223 30997 raft_consensus.cc:740] T 26172e2c14cd4d178764ed13012c21f7 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.458092 31082 leader_election.cc:290] T bbb2acd63baa48a3a2ace94016522e92 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Requested vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:48.468925 30996 raft_consensus.cc:359] T 5028253a91e74a73afca005c77e8b565 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:48.469058 30997 consensus_queue.cc:260] T 26172e2c14cd4d178764ed13012c21f7 P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:48.469828 30996 raft_consensus.cc:385] T 5028253a91e74a73afca005c77e8b565 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.470161 30996 raft_consensus.cc:740] T 5028253a91e74a73afca005c77e8b565 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.470906 30996 consensus_queue.cc:260] T 5028253a91e74a73afca005c77e8b565 P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:48.471316 30997 ts_tablet_manager.cc:1434] T 26172e2c14cd4d178764ed13012c21f7 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:11:48.472295 30997 tablet_bootstrap.cc:492] T c02f992ce0974544a4112672d8792c5f P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
I20260709 13:11:48.472949 30996 ts_tablet_manager.cc:1434] T 5028253a91e74a73afca005c77e8b565 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:11:48.473680 31109 leader_election.cc:290] T bbb2acd63baa48a3a2ace94016522e92 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 election: Requested vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:48.462531 30995 tablet_bootstrap.cc:492] T bbb2acd63baa48a3a2ace94016522e92 P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:48.474352 30995 ts_tablet_manager.cc:1403] T bbb2acd63baa48a3a2ace94016522e92 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.213s	user 0.023s	sys 0.000s
I20260709 13:11:48.468978 30994 ts_tablet_manager.cc:1403] T ccc68dd4c56548cba0624daafb5ae23c P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.161s	user 0.052s	sys 0.051s
I20260709 13:11:48.477252 30995 raft_consensus.cc:359] T bbb2acd63baa48a3a2ace94016522e92 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:48.477993 30997 tablet_bootstrap.cc:654] T c02f992ce0974544a4112672d8792c5f P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.478011 30995 raft_consensus.cc:385] T bbb2acd63baa48a3a2ace94016522e92 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.478595 30995 raft_consensus.cc:740] T bbb2acd63baa48a3a2ace94016522e92 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.479319 30995 consensus_queue.cc:260] T bbb2acd63baa48a3a2ace94016522e92 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:48.479696 30994 raft_consensus.cc:359] T ccc68dd4c56548cba0624daafb5ae23c P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:48.480412 30994 raft_consensus.cc:385] T ccc68dd4c56548cba0624daafb5ae23c P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.480726 30994 raft_consensus.cc:740] T ccc68dd4c56548cba0624daafb5ae23c P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.481807 30995 ts_tablet_manager.cc:1434] T bbb2acd63baa48a3a2ace94016522e92 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 13:11:48.443761 31069 consensus_queue.cc:1048] T 0ac93952b93645b1ad1d7d96cb60c3e9 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:48.473845 30996 tablet_bootstrap.cc:492] T c02f992ce0974544a4112672d8792c5f P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
I20260709 13:11:48.487203 30948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbb2acd63baa48a3a2ace94016522e92" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:11:48.488140 30948 raft_consensus.cc:2393] T bbb2acd63baa48a3a2ace94016522e92 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 05213ac836744a53b9c089a5c052f4be in current term 1: Already voted for candidate 262942f67dfa4b8fa42234b18ac31ffd in this term.
I20260709 13:11:48.460212 30801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbb2acd63baa48a3a2ace94016522e92" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:48.489976 30996 tablet_bootstrap.cc:654] T c02f992ce0974544a4112672d8792c5f P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.490206 30801 raft_consensus.cc:2393] T bbb2acd63baa48a3a2ace94016522e92 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 262942f67dfa4b8fa42234b18ac31ffd in current term 1: Already voted for candidate 05213ac836744a53b9c089a5c052f4be in this term.
I20260709 13:11:48.490267 30995 tablet_bootstrap.cc:492] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:48.491773 30912 leader_election.cc:304] T bbb2acd63baa48a3a2ace94016522e92 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd; no voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:48.481375 30994 consensus_queue.cc:260] T ccc68dd4c56548cba0624daafb5ae23c P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:48.493062 31082 raft_consensus.cc:2749] T bbb2acd63baa48a3a2ace94016522e92 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:11:48.494390 30994 ts_tablet_manager.cc:1434] T ccc68dd4c56548cba0624daafb5ae23c P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.000s
I20260709 13:11:48.495397 30994 tablet_bootstrap.cc:492] T 5028253a91e74a73afca005c77e8b565 P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
I20260709 13:11:48.501552 30994 tablet_bootstrap.cc:654] T 5028253a91e74a73afca005c77e8b565 P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.465821 30619 catalog_manager.cc:5697] T d4dbe5b2d724412a909fd0641d1a213a P 262942f67dfa4b8fa42234b18ac31ffd reported cstate change: term changed from 0 to 1, leader changed from <none> to 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4). New cstate: current_term: 1 leader_uuid: "262942f67dfa4b8fa42234b18ac31ffd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:48.508477 30726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bbb2acd63baa48a3a2ace94016522e92" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:48.518895 30726 raft_consensus.cc:3060] T bbb2acd63baa48a3a2ace94016522e92 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:48.517482 31109 raft_consensus.cc:493] T 64a95fa1266848048486722c02f0d4cf P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.522941 31109 raft_consensus.cc:515] T 64a95fa1266848048486722c02f0d4cf P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:48.524422 31069 raft_consensus.cc:493] T 64a95fa1266848048486722c02f0d4cf P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.525132 31069 raft_consensus.cc:515] T 64a95fa1266848048486722c02f0d4cf P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:48.526700 30725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64a95fa1266848048486722c02f0d4cf" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:48.527551 30725 raft_consensus.cc:2468] T 64a95fa1266848048486722c02f0d4cf P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05213ac836744a53b9c089a5c052f4be in term 0.
I20260709 13:11:48.529119 30767 leader_election.cc:304] T 64a95fa1266848048486722c02f0d4cf P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:48.529727 30873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64a95fa1266848048486722c02f0d4cf" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:48.530166 31004 raft_consensus.cc:2804] T 64a95fa1266848048486722c02f0d4cf P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:48.530705 31004 raft_consensus.cc:493] T 64a95fa1266848048486722c02f0d4cf P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.531097 31004 raft_consensus.cc:3060] T 64a95fa1266848048486722c02f0d4cf P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:48.531358 31069 leader_election.cc:290] T 64a95fa1266848048486722c02f0d4cf P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
W20260709 13:11:48.532321 30691 leader_election.cc:343] T 64a95fa1266848048486722c02f0d4cf P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:48.533802 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64a95fa1266848048486722c02f0d4cf" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:48.550104 31109 leader_election.cc:290] T 64a95fa1266848048486722c02f0d4cf P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:48.551702 30873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64a95fa1266848048486722c02f0d4cf" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:48.553113 31004 raft_consensus.cc:515] T 64a95fa1266848048486722c02f0d4cf P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
W20260709 13:11:48.555423 30766 leader_election.cc:343] T 64a95fa1266848048486722c02f0d4cf P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:48.557140 30799 raft_consensus.cc:2393] T 64a95fa1266848048486722c02f0d4cf P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d22c9d3d15904327a78a0e548a8ff6cd in current term 1: Already voted for candidate 05213ac836744a53b9c089a5c052f4be in this term.
I20260709 13:11:48.559553 30689 leader_election.cc:304] T 64a95fa1266848048486722c02f0d4cf P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22c9d3d15904327a78a0e548a8ff6cd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 05213ac836744a53b9c089a5c052f4be
I20260709 13:11:48.560892 31069 raft_consensus.cc:3060] T 64a95fa1266848048486722c02f0d4cf P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:48.563841 30725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64a95fa1266848048486722c02f0d4cf" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:48.564512 31004 leader_election.cc:290] T 64a95fa1266848048486722c02f0d4cf P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 election: Requested vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:48.572139 30873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64a95fa1266848048486722c02f0d4cf" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
W20260709 13:11:48.574980 30766 leader_election.cc:343] T 64a95fa1266848048486722c02f0d4cf P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:48.576201 30995 tablet_bootstrap.cc:654] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.581600 30996 tablet_bootstrap.cc:492] T c02f992ce0974544a4112672d8792c5f P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
I20260709 13:11:48.581830 30997 tablet_bootstrap.cc:492] T c02f992ce0974544a4112672d8792c5f P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
I20260709 13:11:48.582238 30996 ts_tablet_manager.cc:1403] T c02f992ce0974544a4112672d8792c5f P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.109s	user 0.022s	sys 0.026s
I20260709 13:11:48.582381 30997 ts_tablet_manager.cc:1403] T c02f992ce0974544a4112672d8792c5f P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.110s	user 0.038s	sys 0.062s
I20260709 13:11:48.581489 30726 raft_consensus.cc:2468] T bbb2acd63baa48a3a2ace94016522e92 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05213ac836744a53b9c089a5c052f4be in term 1.
I20260709 13:11:48.585242 30996 raft_consensus.cc:359] T c02f992ce0974544a4112672d8792c5f P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:48.585932 30996 raft_consensus.cc:385] T c02f992ce0974544a4112672d8792c5f P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.586194 30996 raft_consensus.cc:740] T c02f992ce0974544a4112672d8792c5f P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.586056 30997 raft_consensus.cc:359] T c02f992ce0974544a4112672d8792c5f P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:48.587183 30996 consensus_queue.cc:260] T c02f992ce0974544a4112672d8792c5f P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:48.587605 30997 raft_consensus.cc:385] T c02f992ce0974544a4112672d8792c5f P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.588042 30997 raft_consensus.cc:740] T c02f992ce0974544a4112672d8792c5f P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.589368 30997 consensus_queue.cc:260] T c02f992ce0974544a4112672d8792c5f P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:48.593526 30997 ts_tablet_manager.cc:1434] T c02f992ce0974544a4112672d8792c5f P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.002s
I20260709 13:11:48.597728 30994 tablet_bootstrap.cc:492] T 5028253a91e74a73afca005c77e8b565 P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
I20260709 13:11:48.598348 30994 ts_tablet_manager.cc:1403] T 5028253a91e74a73afca005c77e8b565 P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.103s	user 0.012s	sys 0.030s
I20260709 13:11:48.599848 30997 tablet_bootstrap.cc:492] T ccc68dd4c56548cba0624daafb5ae23c P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
I20260709 13:11:48.602146 30994 raft_consensus.cc:359] T 5028253a91e74a73afca005c77e8b565 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:48.603042 30994 raft_consensus.cc:385] T 5028253a91e74a73afca005c77e8b565 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.603375 30994 raft_consensus.cc:740] T 5028253a91e74a73afca005c77e8b565 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.605266 30996 ts_tablet_manager.cc:1434] T c02f992ce0974544a4112672d8792c5f P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 13:11:48.606564 30996 tablet_bootstrap.cc:492] T ccc68dd4c56548cba0624daafb5ae23c P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
I20260709 13:11:48.607795 30994 consensus_queue.cc:260] T 5028253a91e74a73afca005c77e8b565 P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:48.609371 30997 tablet_bootstrap.cc:654] T ccc68dd4c56548cba0624daafb5ae23c P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.614696 30994 ts_tablet_manager.cc:1434] T 5028253a91e74a73afca005c77e8b565 P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.002s
I20260709 13:11:48.611043 31069 raft_consensus.cc:2749] T 64a95fa1266848048486722c02f0d4cf P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:48.611238 30767 leader_election.cc:304] T bbb2acd63baa48a3a2ace94016522e92 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 262942f67dfa4b8fa42234b18ac31ffd
I20260709 13:11:48.615483 31004 raft_consensus.cc:2804] T bbb2acd63baa48a3a2ace94016522e92 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:48.615988 31004 raft_consensus.cc:697] T bbb2acd63baa48a3a2ace94016522e92 P 05213ac836744a53b9c089a5c052f4be [term 1 LEADER]: Becoming Leader. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Running, Role: LEADER
I20260709 13:11:48.616835 31004 consensus_queue.cc:237] T bbb2acd63baa48a3a2ace94016522e92 P 05213ac836744a53b9c089a5c052f4be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:48.621677 31115 raft_consensus.cc:493] T c02f992ce0974544a4112672d8792c5f P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.622220 31115 raft_consensus.cc:515] T c02f992ce0974544a4112672d8792c5f P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:48.624382 31115 leader_election.cc:290] T c02f992ce0974544a4112672d8792c5f P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:48.613042 30995 tablet_bootstrap.cc:492] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:48.631734 30995 ts_tablet_manager.cc:1403] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.142s	user 0.025s	sys 0.001s
I20260709 13:11:48.632239 30948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c02f992ce0974544a4112672d8792c5f" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:48.633049 30948 raft_consensus.cc:2468] T c02f992ce0974544a4112672d8792c5f P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 0.
I20260709 13:11:48.634478 30839 leader_election.cc:304] T c02f992ce0974544a4112672d8792c5f P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
I20260709 13:11:48.635461 31063 raft_consensus.cc:2804] T c02f992ce0974544a4112672d8792c5f P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:48.635315 30995 raft_consensus.cc:359] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:48.635973 31063 raft_consensus.cc:493] T c02f992ce0974544a4112672d8792c5f P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.625027 30725 raft_consensus.cc:2468] T 64a95fa1266848048486722c02f0d4cf P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05213ac836744a53b9c089a5c052f4be in term 1.
I20260709 13:11:48.625922 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c02f992ce0974544a4112672d8792c5f" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:48.637245 30799 raft_consensus.cc:2468] T c02f992ce0974544a4112672d8792c5f P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 0.
I20260709 13:11:48.636334 30995 raft_consensus.cc:385] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.612511 30996 tablet_bootstrap.cc:654] T ccc68dd4c56548cba0624daafb5ae23c P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.629005 30994 tablet_bootstrap.cc:492] T 26172e2c14cd4d178764ed13012c21f7 P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
I20260709 13:11:48.640990 31069 consensus_queue.cc:1048] T 27f26e218737414893d5c4bfa4b2cd94 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:48.641650 30995 raft_consensus.cc:740] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.642338 30995 consensus_queue.cc:260] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:48.644362 30995 ts_tablet_manager.cc:1434] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 13:11:48.646270 30767 leader_election.cc:304] T 64a95fa1266848048486722c02f0d4cf P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a
I20260709 13:11:48.636412 31063 raft_consensus.cc:3060] T c02f992ce0974544a4112672d8792c5f P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:48.648764 30995 tablet_bootstrap.cc:492] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:48.649300 31069 raft_consensus.cc:493] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.649780 31069 raft_consensus.cc:515] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:48.652823 31069 leader_election.cc:290] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:48.652384 31027 consensus_queue.cc:1048] T d05da516ed9040c988b1fdfe2fdde9ed P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:11:48.655769 30997 tablet_bootstrap.cc:492] T ccc68dd4c56548cba0624daafb5ae23c P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
I20260709 13:11:48.656410 30997 ts_tablet_manager.cc:1403] T ccc68dd4c56548cba0624daafb5ae23c P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.060s	user 0.012s	sys 0.000s
I20260709 13:11:48.656886 30994 tablet_bootstrap.cc:654] T 26172e2c14cd4d178764ed13012c21f7 P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.656980 31004 raft_consensus.cc:2804] T 64a95fa1266848048486722c02f0d4cf P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:48.657960 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5028253a91e74a73afca005c77e8b565" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:48.658607 31004 raft_consensus.cc:697] T 64a95fa1266848048486722c02f0d4cf P 05213ac836744a53b9c089a5c052f4be [term 1 LEADER]: Becoming Leader. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Running, Role: LEADER
I20260709 13:11:48.658710 30799 raft_consensus.cc:2468] T 5028253a91e74a73afca005c77e8b565 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 0.
I20260709 13:11:48.659895 30997 raft_consensus.cc:359] T ccc68dd4c56548cba0624daafb5ae23c P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:48.665716 30997 raft_consensus.cc:385] T ccc68dd4c56548cba0624daafb5ae23c P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.665998 30997 raft_consensus.cc:740] T ccc68dd4c56548cba0624daafb5ae23c P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.667128 30997 consensus_queue.cc:260] T ccc68dd4c56548cba0624daafb5ae23c P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:48.668084 30874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5028253a91e74a73afca005c77e8b565" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:48.668836 30874 raft_consensus.cc:2468] T 5028253a91e74a73afca005c77e8b565 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 0.
I20260709 13:11:48.669049 30689 leader_election.cc:304] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:48.673182 30997 ts_tablet_manager.cc:1434] T ccc68dd4c56548cba0624daafb5ae23c P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.001s
I20260709 13:11:48.674640 30997 tablet_bootstrap.cc:492] T d7edce5d08f24852b7afca9c57ba9c19 P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
I20260709 13:11:48.685657 30997 tablet_bootstrap.cc:654] T d7edce5d08f24852b7afca9c57ba9c19 P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.687819 30995 tablet_bootstrap.cc:654] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.674830 31069 raft_consensus.cc:2804] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:48.690151 31069 raft_consensus.cc:493] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.692054 30996 tablet_bootstrap.cc:492] T ccc68dd4c56548cba0624daafb5ae23c P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
I20260709 13:11:48.693468 30996 ts_tablet_manager.cc:1403] T ccc68dd4c56548cba0624daafb5ae23c P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.087s	user 0.009s	sys 0.011s
I20260709 13:11:48.694706 31069 raft_consensus.cc:3060] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:48.699501 31063 raft_consensus.cc:515] T c02f992ce0974544a4112672d8792c5f P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:48.696245 31004 consensus_queue.cc:237] T 64a95fa1266848048486722c02f0d4cf P 05213ac836744a53b9c089a5c052f4be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:48.702548 31063 leader_election.cc:290] T c02f992ce0974544a4112672d8792c5f P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Requested vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:48.704737 31068 consensus_queue.cc:1048] T 27f26e218737414893d5c4bfa4b2cd94 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:48.704941 30949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c02f992ce0974544a4112672d8792c5f" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:11:48.705839 30949 raft_consensus.cc:3060] T c02f992ce0974544a4112672d8792c5f P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:48.710510 31063 consensus_queue.cc:1048] T d05da516ed9040c988b1fdfe2fdde9ed P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:48.709033 30996 raft_consensus.cc:359] T ccc68dd4c56548cba0624daafb5ae23c P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:48.711699 30996 raft_consensus.cc:385] T ccc68dd4c56548cba0624daafb5ae23c P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.712107 30996 raft_consensus.cc:740] T ccc68dd4c56548cba0624daafb5ae23c P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.717679 30994 tablet_bootstrap.cc:492] T 26172e2c14cd4d178764ed13012c21f7 P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
I20260709 13:11:48.718200 30994 ts_tablet_manager.cc:1403] T 26172e2c14cd4d178764ed13012c21f7 P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.089s	user 0.012s	sys 0.021s
I20260709 13:11:48.725576 30996 consensus_queue.cc:260] T ccc68dd4c56548cba0624daafb5ae23c P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:48.716462 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c02f992ce0974544a4112672d8792c5f" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:48.737677 30799 raft_consensus.cc:3060] T c02f992ce0974544a4112672d8792c5f P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:48.746322 30996 ts_tablet_manager.cc:1434] T ccc68dd4c56548cba0624daafb5ae23c P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.052s	user 0.008s	sys 0.000s
I20260709 13:11:48.749200 30996 tablet_bootstrap.cc:492] T 726b808ed64848dd89da4340d4d891fb P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
I20260709 13:11:48.750804 31069 raft_consensus.cc:515] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:48.750875 30994 raft_consensus.cc:359] T 26172e2c14cd4d178764ed13012c21f7 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:48.751755 30994 raft_consensus.cc:385] T 26172e2c14cd4d178764ed13012c21f7 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.752046 30994 raft_consensus.cc:740] T 26172e2c14cd4d178764ed13012c21f7 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.752871 30994 consensus_queue.cc:260] T 26172e2c14cd4d178764ed13012c21f7 P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:48.755694 30994 ts_tablet_manager.cc:1434] T 26172e2c14cd4d178764ed13012c21f7 P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.000s
I20260709 13:11:48.756285 31089 raft_consensus.cc:493] T 726b808ed64848dd89da4340d4d891fb P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.756820 31089 raft_consensus.cc:515] T 726b808ed64848dd89da4340d4d891fb P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:48.756982 30994 tablet_bootstrap.cc:492] T a38ff14550f74527a62591df59fb2c51 P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
I20260709 13:11:48.758936 31089 leader_election.cc:290] T 726b808ed64848dd89da4340d4d891fb P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:48.766842 30873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "726b808ed64848dd89da4340d4d891fb" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:48.775671 31069 leader_election.cc:290] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 election: Requested vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:48.778540 30725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "726b808ed64848dd89da4340d4d891fb" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:48.797667 30873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5028253a91e74a73afca005c77e8b565" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:11:48.783154 30996 tablet_bootstrap.cc:654] T 726b808ed64848dd89da4340d4d891fb P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.798322 30873 raft_consensus.cc:3060] T 5028253a91e74a73afca005c77e8b565 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:48.770177 31089 raft_consensus.cc:493] T ccc68dd4c56548cba0624daafb5ae23c P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.801100 30800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5028253a91e74a73afca005c77e8b565" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:48.801726 30800 raft_consensus.cc:3060] T 5028253a91e74a73afca005c77e8b565 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:48.802085 30997 tablet_bootstrap.cc:492] T d7edce5d08f24852b7afca9c57ba9c19 P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
I20260709 13:11:48.802774 30997 ts_tablet_manager.cc:1403] T d7edce5d08f24852b7afca9c57ba9c19 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.129s	user 0.014s	sys 0.008s
I20260709 13:11:48.803409 30949 raft_consensus.cc:2468] T c02f992ce0974544a4112672d8792c5f P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 1.
I20260709 13:11:48.806298 30839 leader_election.cc:304] T c02f992ce0974544a4112672d8792c5f P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
W20260709 13:11:48.796464 30914 leader_election.cc:343] T 726b808ed64848dd89da4340d4d891fb P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:48.800951 31089 raft_consensus.cc:515] T ccc68dd4c56548cba0624daafb5ae23c P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:48.797005 31063 raft_consensus.cc:493] T ccc68dd4c56548cba0624daafb5ae23c P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.817181 31089 leader_election.cc:290] T ccc68dd4c56548cba0624daafb5ae23c P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:48.823909 30994 tablet_bootstrap.cc:654] T a38ff14550f74527a62591df59fb2c51 P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.825199 30801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccc68dd4c56548cba0624daafb5ae23c" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:48.826205 30801 raft_consensus.cc:2468] T ccc68dd4c56548cba0624daafb5ae23c P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 0.
I20260709 13:11:48.827193 30799 raft_consensus.cc:2468] T c02f992ce0974544a4112672d8792c5f P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 1.
I20260709 13:11:48.827775 30912 leader_election.cc:304] T ccc68dd4c56548cba0624daafb5ae23c P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
I20260709 13:11:48.830334 30995 tablet_bootstrap.cc:492] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:48.823961 30997 raft_consensus.cc:359] T d7edce5d08f24852b7afca9c57ba9c19 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:48.840842 31063 raft_consensus.cc:515] T ccc68dd4c56548cba0624daafb5ae23c P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:48.855224 30995 ts_tablet_manager.cc:1403] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.207s	user 0.021s	sys 0.025s
I20260709 13:11:48.857594 31063 leader_election.cc:290] T ccc68dd4c56548cba0624daafb5ae23c P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:48.855209 30997 raft_consensus.cc:385] T d7edce5d08f24852b7afca9c57ba9c19 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.858796 30997 raft_consensus.cc:740] T d7edce5d08f24852b7afca9c57ba9c19 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.859591 30949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccc68dd4c56548cba0624daafb5ae23c" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:48.860308 30949 raft_consensus.cc:2468] T ccc68dd4c56548cba0624daafb5ae23c P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 0.
I20260709 13:11:48.859782 30997 consensus_queue.cc:260] T d7edce5d08f24852b7afca9c57ba9c19 P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
W20260709 13:11:48.861344 30915 leader_election.cc:343] T 726b808ed64848dd89da4340d4d891fb P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:48.861661 30839 leader_election.cc:304] T ccc68dd4c56548cba0624daafb5ae23c P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
I20260709 13:11:48.861865 30915 leader_election.cc:304] T 726b808ed64848dd89da4340d4d891fb P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:48.843374 30873 raft_consensus.cc:2468] T 5028253a91e74a73afca005c77e8b565 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 1.
I20260709 13:11:48.858282 30801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccc68dd4c56548cba0624daafb5ae23c" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:48.865578 31082 raft_consensus.cc:2804] T ccc68dd4c56548cba0624daafb5ae23c P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:48.865685 30801 raft_consensus.cc:2468] T ccc68dd4c56548cba0624daafb5ae23c P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 0.
I20260709 13:11:48.865978 31089 raft_consensus.cc:493] T 26172e2c14cd4d178764ed13012c21f7 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:48.866122 31082 raft_consensus.cc:493] T ccc68dd4c56548cba0624daafb5ae23c P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.866814 31082 raft_consensus.cc:3060] T ccc68dd4c56548cba0624daafb5ae23c P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:48.867936 30691 leader_election.cc:304] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:48.868657 31070 consensus_queue.cc:1048] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:48.815361 31115 raft_consensus.cc:2804] T c02f992ce0974544a4112672d8792c5f P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:48.873611 30997 ts_tablet_manager.cc:1434] T d7edce5d08f24852b7afca9c57ba9c19 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.070s	user 0.010s	sys 0.001s
I20260709 13:11:48.853216 30621 catalog_manager.cc:5697] T bbb2acd63baa48a3a2ace94016522e92 P 05213ac836744a53b9c089a5c052f4be reported cstate change: term changed from 0 to 1, leader changed from <none> to 05213ac836744a53b9c089a5c052f4be (127.21.63.2). New cstate: current_term: 1 leader_uuid: "05213ac836744a53b9c089a5c052f4be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:48.887219 30621 catalog_manager.cc:5697] T 64a95fa1266848048486722c02f0d4cf P 05213ac836744a53b9c089a5c052f4be reported cstate change: term changed from 0 to 1, leader changed from <none> to 05213ac836744a53b9c089a5c052f4be (127.21.63.2). New cstate: current_term: 1 leader_uuid: "05213ac836744a53b9c089a5c052f4be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:48.895423 30997 tablet_bootstrap.cc:492] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
I20260709 13:11:48.870365 31063 raft_consensus.cc:2804] T ccc68dd4c56548cba0624daafb5ae23c P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:48.869580 31089 raft_consensus.cc:515] T 26172e2c14cd4d178764ed13012c21f7 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:48.844601 30800 raft_consensus.cc:2468] T 5028253a91e74a73afca005c77e8b565 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 1.
I20260709 13:11:48.897601 31168 raft_consensus.cc:2804] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:48.903018 31046 raft_consensus.cc:2749] T 726b808ed64848dd89da4340d4d891fb P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:48.897785 31115 raft_consensus.cc:697] T c02f992ce0974544a4112672d8792c5f P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Becoming Leader. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Running, Role: LEADER
I20260709 13:11:48.901280 31063 raft_consensus.cc:493] T ccc68dd4c56548cba0624daafb5ae23c P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.904376 30873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccc68dd4c56548cba0624daafb5ae23c" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:48.926345 31089 leader_election.cc:290] T 26172e2c14cd4d178764ed13012c21f7 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:48.912177 31063 raft_consensus.cc:3060] T ccc68dd4c56548cba0624daafb5ae23c P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:48.930338 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26172e2c14cd4d178764ed13012c21f7" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:48.930965 30799 raft_consensus.cc:2468] T 26172e2c14cd4d178764ed13012c21f7 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 0.
W20260709 13:11:48.912593 30915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:37325, user_credentials={real_user=slave}} blocked reactor thread for 51300.7us
I20260709 13:11:48.918071 31115 consensus_queue.cc:237] T c02f992ce0974544a4112672d8792c5f P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:48.936264 30996 tablet_bootstrap.cc:492] T 726b808ed64848dd89da4340d4d891fb P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
I20260709 13:11:48.936517 30994 tablet_bootstrap.cc:492] T a38ff14550f74527a62591df59fb2c51 P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
I20260709 13:11:48.936787 30996 ts_tablet_manager.cc:1403] T 726b808ed64848dd89da4340d4d891fb P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.188s	user 0.017s	sys 0.021s
I20260709 13:11:48.937079 30994 ts_tablet_manager.cc:1403] T a38ff14550f74527a62591df59fb2c51 P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.180s	user 0.029s	sys 0.028s
I20260709 13:11:48.937318 31068 consensus_queue.cc:1048] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:48.939627 30996 raft_consensus.cc:359] T 726b808ed64848dd89da4340d4d891fb P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:48.940609 30996 raft_consensus.cc:385] T 726b808ed64848dd89da4340d4d891fb P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.940953 30996 raft_consensus.cc:740] T 726b808ed64848dd89da4340d4d891fb P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.941592 30996 consensus_queue.cc:260] T 726b808ed64848dd89da4340d4d891fb P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:48.945552 30726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26172e2c14cd4d178764ed13012c21f7" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:48.921967 31168 raft_consensus.cc:697] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 LEADER]: Becoming Leader. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Running, Role: LEADER
I20260709 13:11:48.922364 30997 tablet_bootstrap.cc:654] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:48.923372 31178 consensus_queue.cc:1048] T d4dbe5b2d724412a909fd0641d1a213a P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:48.946724 30912 leader_election.cc:304] T 26172e2c14cd4d178764ed13012c21f7 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
I20260709 13:11:48.939905 30994 raft_consensus.cc:359] T a38ff14550f74527a62591df59fb2c51 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:48.947822 30994 raft_consensus.cc:385] T a38ff14550f74527a62591df59fb2c51 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.947937 31174 raft_consensus.cc:2804] T 26172e2c14cd4d178764ed13012c21f7 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:48.948174 30994 raft_consensus.cc:740] T a38ff14550f74527a62591df59fb2c51 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.948462 31174 raft_consensus.cc:493] T 26172e2c14cd4d178764ed13012c21f7 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:48.948905 31174 raft_consensus.cc:3060] T 26172e2c14cd4d178764ed13012c21f7 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:48.912487 30995 raft_consensus.cc:359] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:48.945801 30996 ts_tablet_manager.cc:1434] T 726b808ed64848dd89da4340d4d891fb P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:11:48.949997 30995 raft_consensus.cc:385] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:48.950531 30995 raft_consensus.cc:740] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:48.951174 30996 tablet_bootstrap.cc:492] T 64a95fa1266848048486722c02f0d4cf P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
I20260709 13:11:48.920051 31082 raft_consensus.cc:515] T ccc68dd4c56548cba0624daafb5ae23c P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:48.949146 30994 consensus_queue.cc:260] T a38ff14550f74527a62591df59fb2c51 P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
W20260709 13:11:48.953712 30915 leader_election.cc:343] T 26172e2c14cd4d178764ed13012c21f7 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:48.954597 30995 consensus_queue.cc:260] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:48.949286 31168 consensus_queue.cc:237] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:48.956866 31082 leader_election.cc:290] T ccc68dd4c56548cba0624daafb5ae23c P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Requested vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:48.961061 30995 ts_tablet_manager.cc:1434] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.105s	user 0.008s	sys 0.000s
I20260709 13:11:48.969303 30994 ts_tablet_manager.cc:1434] T a38ff14550f74527a62591df59fb2c51 P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.000s
I20260709 13:11:48.970026 30872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccc68dd4c56548cba0624daafb5ae23c" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:11:48.970307 30994 tablet_bootstrap.cc:492] T c1efa319af774b23bd4ea782611f5252 P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
I20260709 13:11:48.977705 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccc68dd4c56548cba0624daafb5ae23c" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:48.978340 30799 raft_consensus.cc:3060] T ccc68dd4c56548cba0624daafb5ae23c P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:48.981875 31063 raft_consensus.cc:515] T ccc68dd4c56548cba0624daafb5ae23c P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:48.983188 30873 raft_consensus.cc:2393] T ccc68dd4c56548cba0624daafb5ae23c P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 262942f67dfa4b8fa42234b18ac31ffd in current term 1: Already voted for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in this term.
I20260709 13:11:48.986765 30799 raft_consensus.cc:2468] T ccc68dd4c56548cba0624daafb5ae23c P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 1.
I20260709 13:11:48.986866 31174 raft_consensus.cc:515] T 26172e2c14cd4d178764ed13012c21f7 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:48.987506 30800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccc68dd4c56548cba0624daafb5ae23c" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:48.990209 30995 tablet_bootstrap.cc:492] T 26172e2c14cd4d178764ed13012c21f7 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:48.994933 30948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ccc68dd4c56548cba0624daafb5ae23c" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:11:48.996114 30948 raft_consensus.cc:2393] T ccc68dd4c56548cba0624daafb5ae23c P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in current term 1: Already voted for candidate 262942f67dfa4b8fa42234b18ac31ffd in this term.
I20260709 13:11:48.987494 31063 leader_election.cc:290] T ccc68dd4c56548cba0624daafb5ae23c P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Requested vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:49.007879 31174 leader_election.cc:290] T 26172e2c14cd4d178764ed13012c21f7 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Requested vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:49.001277 30800 raft_consensus.cc:2393] T ccc68dd4c56548cba0624daafb5ae23c P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in current term 1: Already voted for candidate 262942f67dfa4b8fa42234b18ac31ffd in this term.
I20260709 13:11:49.008642 30725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26172e2c14cd4d178764ed13012c21f7" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
W20260709 13:11:49.010833 30915 leader_election.cc:343] T 26172e2c14cd4d178764ed13012c21f7 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:49.013811 30994 tablet_bootstrap.cc:654] T c1efa319af774b23bd4ea782611f5252 P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:49.014601 30839 leader_election.cc:304] T ccc68dd4c56548cba0624daafb5ae23c P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a; no voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd
I20260709 13:11:49.015805 31063 raft_consensus.cc:2749] T ccc68dd4c56548cba0624daafb5ae23c P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:11:49.025789 30912 leader_election.cc:304] T ccc68dd4c56548cba0624daafb5ae23c P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a
I20260709 13:11:49.030833 31082 raft_consensus.cc:2804] T ccc68dd4c56548cba0624daafb5ae23c P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:49.031523 31082 raft_consensus.cc:697] T ccc68dd4c56548cba0624daafb5ae23c P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Becoming Leader. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Running, Role: LEADER
I20260709 13:11:49.032284 30800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26172e2c14cd4d178764ed13012c21f7" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:49.033077 30800 raft_consensus.cc:3060] T 26172e2c14cd4d178764ed13012c21f7 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:49.032932 31082 consensus_queue.cc:237] T ccc68dd4c56548cba0624daafb5ae23c P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:49.036373 31109 consensus_queue.cc:1048] T bbb2acd63baa48a3a2ace94016522e92 P 05213ac836744a53b9c089a5c052f4be [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:11:49.037978 31181 log.cc:927] Time spent T 8770fcfd98154b2095c2c9f74135593b P 262942f67dfa4b8fa42234b18ac31ffd: Append to log took a long time: real 0.061s	user 0.005s	sys 0.000s
I20260709 13:11:49.057178 31089 consensus_queue.cc:1048] T d4dbe5b2d724412a909fd0641d1a213a P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:49.060738 30621 catalog_manager.cc:5697] T c02f992ce0974544a4112672d8792c5f P 02c5705b45c04706bbb2c6ff3d7b0b8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3). New cstate: current_term: 1 leader_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:49.062041 30800 raft_consensus.cc:2468] T 26172e2c14cd4d178764ed13012c21f7 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 1.
I20260709 13:11:49.064603 31068 raft_consensus.cc:493] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:49.065253 31068 raft_consensus.cc:515] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:49.070179 30912 leader_election.cc:304] T 26172e2c14cd4d178764ed13012c21f7 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:49.070685 30873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1474b6fc151c4a3994a3aef89ea936a2" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:49.071533 30873 raft_consensus.cc:2468] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 1.
I20260709 13:11:49.073055 30691 leader_election.cc:304] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:49.076181 31069 raft_consensus.cc:2804] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:49.079087 30996 tablet_bootstrap.cc:654] T 64a95fa1266848048486722c02f0d4cf P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:49.077044 31068 leader_election.cc:290] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:49.076809 31069 raft_consensus.cc:493] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:49.077632 31089 raft_consensus.cc:2804] T 26172e2c14cd4d178764ed13012c21f7 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:49.076442 30800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1474b6fc151c4a3994a3aef89ea936a2" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:49.083169 31069 raft_consensus.cc:3060] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:49.085740 31089 raft_consensus.cc:697] T 26172e2c14cd4d178764ed13012c21f7 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Becoming Leader. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Running, Role: LEADER
I20260709 13:11:49.098441 31089 consensus_queue.cc:237] T 26172e2c14cd4d178764ed13012c21f7 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:49.104737 30800 raft_consensus.cc:2468] T 1474b6fc151c4a3994a3aef89ea936a2 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 1.
I20260709 13:11:49.129516 30997 tablet_bootstrap.cc:492] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
I20260709 13:11:49.130241 30997 ts_tablet_manager.cc:1403] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.252s	user 0.029s	sys 0.024s
I20260709 13:11:49.125126 31004 consensus_queue.cc:1048] T bbb2acd63baa48a3a2ace94016522e92 P 05213ac836744a53b9c089a5c052f4be [LEADER]: Connected to new peer: Peer: permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 13:11:49.142745 30624 catalog_manager.cc:5697] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd reported cstate change: term changed from 0 to 1, leader changed from <none> to d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1). New cstate: current_term: 1 leader_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } }
W20260709 13:11:49.150125 30766 consensus_peers.cc:597] T 64a95fa1266848048486722c02f0d4cf P 05213ac836744a53b9c089a5c052f4be -> Peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Couldn't send request to peer 02c5705b45c04706bbb2c6ff3d7b0b8a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:49.150414 30620 catalog_manager.cc:5697] T ccc68dd4c56548cba0624daafb5ae23c P 262942f67dfa4b8fa42234b18ac31ffd reported cstate change: term changed from 0 to 1, leader changed from <none> to 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4). New cstate: current_term: 1 leader_uuid: "262942f67dfa4b8fa42234b18ac31ffd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:49.154523 30995 tablet_bootstrap.cc:654] T 26172e2c14cd4d178764ed13012c21f7 P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:49.164037 30997 raft_consensus.cc:359] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:49.170679 30997 raft_consensus.cc:385] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:49.171775 30997 raft_consensus.cc:740] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
I20260709 13:11:49.175225 30994 tablet_bootstrap.cc:492] T c1efa319af774b23bd4ea782611f5252 P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
I20260709 13:11:49.175755 30994 ts_tablet_manager.cc:1403] T c1efa319af774b23bd4ea782611f5252 P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.206s	user 0.017s	sys 0.009s
I20260709 13:11:49.178244 30997 consensus_queue.cc:260] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:49.180169 31177 raft_consensus.cc:493] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:49.180682 31177 raft_consensus.cc:515] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:49.168741 31069 raft_consensus.cc:515] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:49.191349 30997 ts_tablet_manager.cc:1434] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.061s	user 0.006s	sys 0.001s
I20260709 13:11:49.192348 30997 tablet_bootstrap.cc:492] T c1efa319af774b23bd4ea782611f5252 P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
I20260709 13:11:49.197413 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1474b6fc151c4a3994a3aef89ea936a2" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:49.198463 30799 raft_consensus.cc:3060] T 1474b6fc151c4a3994a3aef89ea936a2 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:49.201740 31177 leader_election.cc:290] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:49.203423 30997 tablet_bootstrap.cc:654] T c1efa319af774b23bd4ea782611f5252 P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:49.210641 30874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01cceb88bca348958b59f3697979dff1" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
W20260709 13:11:49.212338 30914 leader_election.cc:343] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:49.210219 30994 raft_consensus.cc:359] T c1efa319af774b23bd4ea782611f5252 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:49.214535 30994 raft_consensus.cc:385] T c1efa319af774b23bd4ea782611f5252 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:49.214975 30994 raft_consensus.cc:740] T c1efa319af774b23bd4ea782611f5252 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
I20260709 13:11:49.216176 30994 consensus_queue.cc:260] T c1efa319af774b23bd4ea782611f5252 P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:49.219305 30725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01cceb88bca348958b59f3697979dff1" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:49.222136 30996 tablet_bootstrap.cc:492] T 64a95fa1266848048486722c02f0d4cf P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
I20260709 13:11:49.222862 30996 ts_tablet_manager.cc:1403] T 64a95fa1266848048486722c02f0d4cf P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.272s	user 0.012s	sys 0.003s
W20260709 13:11:49.224090 30915 leader_election.cc:343] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:49.224740 30915 leader_election.cc:304] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:49.225615 30994 ts_tablet_manager.cc:1434] T c1efa319af774b23bd4ea782611f5252 P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.050s	user 0.010s	sys 0.000s
I20260709 13:11:49.226619 31178 raft_consensus.cc:2749] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:49.223964 31069 leader_election.cc:290] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 2 election: Requested vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:49.231365 30994 tablet_bootstrap.cc:492] T 054fc96e257a4dc6b2ef90db5c1ad499 P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
I20260709 13:11:49.252799 30799 raft_consensus.cc:2468] T 1474b6fc151c4a3994a3aef89ea936a2 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 2.
I20260709 13:11:49.254065 30995 tablet_bootstrap.cc:492] T 26172e2c14cd4d178764ed13012c21f7 P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:49.256006 30995 ts_tablet_manager.cc:1403] T 26172e2c14cd4d178764ed13012c21f7 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.266s	user 0.025s	sys 0.024s
I20260709 13:11:49.248467 30873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1474b6fc151c4a3994a3aef89ea936a2" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:11:49.257539 30873 raft_consensus.cc:3060] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:49.267794 30873 raft_consensus.cc:2468] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 2.
I20260709 13:11:49.267895 30689 leader_election.cc:304] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:49.268836 31069 raft_consensus.cc:2804] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:49.269322 31069 raft_consensus.cc:697] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 LEADER]: Becoming Leader. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Running, Role: LEADER
I20260709 13:11:49.270071 30994 tablet_bootstrap.cc:654] T 054fc96e257a4dc6b2ef90db5c1ad499 P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:49.269953 30996 raft_consensus.cc:359] T 64a95fa1266848048486722c02f0d4cf P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:49.270117 31069 consensus_queue.cc:237] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:49.274158 30995 raft_consensus.cc:359] T 26172e2c14cd4d178764ed13012c21f7 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:49.274637 30997 tablet_bootstrap.cc:492] T c1efa319af774b23bd4ea782611f5252 P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
I20260709 13:11:49.274960 30995 raft_consensus.cc:385] T 26172e2c14cd4d178764ed13012c21f7 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:49.275290 30997 ts_tablet_manager.cc:1403] T c1efa319af774b23bd4ea782611f5252 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.083s	user 0.014s	sys 0.005s
I20260709 13:11:49.270850 30996 raft_consensus.cc:385] T 64a95fa1266848048486722c02f0d4cf P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:49.275331 30995 raft_consensus.cc:740] T 26172e2c14cd4d178764ed13012c21f7 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:49.275807 30996 raft_consensus.cc:740] T 64a95fa1266848048486722c02f0d4cf P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
I20260709 13:11:49.276312 30995 consensus_queue.cc:260] T 26172e2c14cd4d178764ed13012c21f7 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:49.269956 30621 catalog_manager.cc:5697] T 26172e2c14cd4d178764ed13012c21f7 P 262942f67dfa4b8fa42234b18ac31ffd reported cstate change: term changed from 0 to 1, leader changed from <none> to 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4). New cstate: current_term: 1 leader_uuid: "262942f67dfa4b8fa42234b18ac31ffd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:49.277242 30996 consensus_queue.cc:260] T 64a95fa1266848048486722c02f0d4cf P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:49.278532 30997 raft_consensus.cc:359] T c1efa319af774b23bd4ea782611f5252 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:49.280200 30997 raft_consensus.cc:385] T c1efa319af774b23bd4ea782611f5252 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:49.280608 30997 raft_consensus.cc:740] T c1efa319af774b23bd4ea782611f5252 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
I20260709 13:11:49.281430 30997 consensus_queue.cc:260] T c1efa319af774b23bd4ea782611f5252 P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:49.283392 30994 tablet_bootstrap.cc:492] T 054fc96e257a4dc6b2ef90db5c1ad499 P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
I20260709 13:11:49.283443 30997 ts_tablet_manager.cc:1434] T c1efa319af774b23bd4ea782611f5252 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:11:49.284080 30994 ts_tablet_manager.cc:1403] T 054fc96e257a4dc6b2ef90db5c1ad499 P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.053s	user 0.010s	sys 0.000s
I20260709 13:11:49.284793 30997 tablet_bootstrap.cc:492] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
I20260709 13:11:49.287057 30994 raft_consensus.cc:359] T 054fc96e257a4dc6b2ef90db5c1ad499 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:49.289052 30994 raft_consensus.cc:385] T 054fc96e257a4dc6b2ef90db5c1ad499 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:49.289397 30994 raft_consensus.cc:740] T 054fc96e257a4dc6b2ef90db5c1ad499 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
I20260709 13:11:49.290155 30994 consensus_queue.cc:260] T 054fc96e257a4dc6b2ef90db5c1ad499 P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:49.292277 30994 ts_tablet_manager.cc:1434] T 054fc96e257a4dc6b2ef90db5c1ad499 P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 13:11:49.293231 30994 tablet_bootstrap.cc:492] T 0d8c1da1f7254fde94f7deaad5f41709 P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
I20260709 13:11:49.294041 30997 tablet_bootstrap.cc:654] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:49.278546 30995 ts_tablet_manager.cc:1434] T 26172e2c14cd4d178764ed13012c21f7 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.000s
I20260709 13:11:49.297518 30995 tablet_bootstrap.cc:492] T d7edce5d08f24852b7afca9c57ba9c19 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:49.303249 30995 tablet_bootstrap.cc:654] T d7edce5d08f24852b7afca9c57ba9c19 P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:49.304399 31168 raft_consensus.cc:493] T 26172e2c14cd4d178764ed13012c21f7 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:49.305014 31168 raft_consensus.cc:515] T 26172e2c14cd4d178764ed13012c21f7 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:49.308154 30994 tablet_bootstrap.cc:654] T 0d8c1da1f7254fde94f7deaad5f41709 P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:49.308990 31168 leader_election.cc:290] T 26172e2c14cd4d178764ed13012c21f7 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:49.309019 30949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26172e2c14cd4d178764ed13012c21f7" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:49.310799 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "26172e2c14cd4d178764ed13012c21f7" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:49.311846 30799 raft_consensus.cc:2393] T 26172e2c14cd4d178764ed13012c21f7 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d22c9d3d15904327a78a0e548a8ff6cd in current term 1: Already voted for candidate 262942f67dfa4b8fa42234b18ac31ffd in this term.
I20260709 13:11:49.313333 30689 leader_election.cc:304] T 26172e2c14cd4d178764ed13012c21f7 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22c9d3d15904327a78a0e548a8ff6cd; no voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd
I20260709 13:11:49.314285 31109 consensus_queue.cc:1048] T 64a95fa1266848048486722c02f0d4cf P 05213ac836744a53b9c089a5c052f4be [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:49.314930 31168 raft_consensus.cc:3060] T 26172e2c14cd4d178764ed13012c21f7 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:49.317471 31070 raft_consensus.cc:493] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:49.318004 31070 raft_consensus.cc:515] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:49.322052 31070 leader_election.cc:290] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:49.322177 30873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "726b808ed64848dd89da4340d4d891fb" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:49.322896 30873 raft_consensus.cc:2468] T 726b808ed64848dd89da4340d4d891fb P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 0.
I20260709 13:11:49.323432 31168 raft_consensus.cc:2749] T 26172e2c14cd4d178764ed13012c21f7 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:49.324523 30949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "726b808ed64848dd89da4340d4d891fb" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:49.325244 30949 raft_consensus.cc:2468] T 726b808ed64848dd89da4340d4d891fb P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 0.
I20260709 13:11:49.326407 30691 leader_election.cc:304] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:49.327836 31070 raft_consensus.cc:2804] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:49.328344 31070 raft_consensus.cc:493] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:49.328843 31070 raft_consensus.cc:3060] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:49.334718 30995 tablet_bootstrap.cc:492] T d7edce5d08f24852b7afca9c57ba9c19 P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:49.335182 30995 ts_tablet_manager.cc:1403] T d7edce5d08f24852b7afca9c57ba9c19 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.038s	user 0.010s	sys 0.009s
I20260709 13:11:49.280205 30996 ts_tablet_manager.cc:1434] T 64a95fa1266848048486722c02f0d4cf P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.057s	user 0.006s	sys 0.001s
I20260709 13:11:49.337563 30995 raft_consensus.cc:359] T d7edce5d08f24852b7afca9c57ba9c19 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:49.338215 30995 raft_consensus.cc:385] T d7edce5d08f24852b7afca9c57ba9c19 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:49.338492 30995 raft_consensus.cc:740] T d7edce5d08f24852b7afca9c57ba9c19 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:49.338873 30994 tablet_bootstrap.cc:492] T 0d8c1da1f7254fde94f7deaad5f41709 P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
I20260709 13:11:49.339397 30994 ts_tablet_manager.cc:1403] T 0d8c1da1f7254fde94f7deaad5f41709 P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.046s	user 0.013s	sys 0.006s
I20260709 13:11:49.339289 30995 consensus_queue.cc:260] T d7edce5d08f24852b7afca9c57ba9c19 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:49.341368 31066 consensus_queue.cc:1048] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:49.342814 30994 raft_consensus.cc:359] T 0d8c1da1f7254fde94f7deaad5f41709 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:49.343549 30994 raft_consensus.cc:385] T 0d8c1da1f7254fde94f7deaad5f41709 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:49.343921 30994 raft_consensus.cc:740] T 0d8c1da1f7254fde94f7deaad5f41709 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
I20260709 13:11:49.344614 30994 consensus_queue.cc:260] T 0d8c1da1f7254fde94f7deaad5f41709 P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:49.347077 30994 ts_tablet_manager.cc:1434] T 0d8c1da1f7254fde94f7deaad5f41709 P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:11:49.348308 30994 tablet_bootstrap.cc:492] T b9fcfade988746a2967e115dbbcfe0f1 P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
I20260709 13:11:49.348791 30996 tablet_bootstrap.cc:492] T 93a75acc8893414180a15797033a2d5a P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
I20260709 13:11:49.349685 31089 raft_consensus.cc:493] T d7edce5d08f24852b7afca9c57ba9c19 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:49.350557 30995 ts_tablet_manager.cc:1434] T d7edce5d08f24852b7afca9c57ba9c19 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 13:11:49.350369 31089 raft_consensus.cc:515] T d7edce5d08f24852b7afca9c57ba9c19 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:49.351558 30995 tablet_bootstrap.cc:492] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:49.353897 30726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7edce5d08f24852b7afca9c57ba9c19" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:49.354558 30726 raft_consensus.cc:2468] T d7edce5d08f24852b7afca9c57ba9c19 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 0.
I20260709 13:11:49.355018 30994 tablet_bootstrap.cc:654] T b9fcfade988746a2967e115dbbcfe0f1 P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:49.355432 30800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7edce5d08f24852b7afca9c57ba9c19" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:49.355934 30915 leader_election.cc:304] T d7edce5d08f24852b7afca9c57ba9c19 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:49.356135 30800 raft_consensus.cc:2468] T d7edce5d08f24852b7afca9c57ba9c19 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 0.
I20260709 13:11:49.356837 31178 raft_consensus.cc:2804] T d7edce5d08f24852b7afca9c57ba9c19 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:49.357277 31178 raft_consensus.cc:493] T d7edce5d08f24852b7afca9c57ba9c19 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:49.357600 31178 raft_consensus.cc:3060] T d7edce5d08f24852b7afca9c57ba9c19 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:49.363302 31002 consensus_queue.cc:1048] T c02f992ce0974544a4112672d8792c5f P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:49.353943 31089 leader_election.cc:290] T d7edce5d08f24852b7afca9c57ba9c19 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:49.370971 31178 raft_consensus.cc:515] T d7edce5d08f24852b7afca9c57ba9c19 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:49.388069 31168 consensus_queue.cc:1048] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:49.397970 30726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7edce5d08f24852b7afca9c57ba9c19" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:49.398934 30726 raft_consensus.cc:3060] T d7edce5d08f24852b7afca9c57ba9c19 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:49.412760 31089 raft_consensus.cc:493] T c1efa319af774b23bd4ea782611f5252 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:49.413403 31089 raft_consensus.cc:515] T c1efa319af774b23bd4ea782611f5252 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:49.416998 31089 leader_election.cc:290] T c1efa319af774b23bd4ea782611f5252 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:49.418282 30874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1efa319af774b23bd4ea782611f5252" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
W20260709 13:11:49.419860 30914 leader_election.cc:343] T c1efa319af774b23bd4ea782611f5252 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:49.427963 31177 consensus_queue.cc:1048] T ccc68dd4c56548cba0624daafb5ae23c P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:49.421236 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1efa319af774b23bd4ea782611f5252" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:49.431347 30799 raft_consensus.cc:2468] T c1efa319af774b23bd4ea782611f5252 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 0.
I20260709 13:11:49.432889 30912 leader_election.cc:304] T c1efa319af774b23bd4ea782611f5252 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a
I20260709 13:11:49.434522 31089 raft_consensus.cc:2804] T c1efa319af774b23bd4ea782611f5252 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:49.435086 31089 raft_consensus.cc:493] T c1efa319af774b23bd4ea782611f5252 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:49.435592 31089 raft_consensus.cc:3060] T c1efa319af774b23bd4ea782611f5252 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:49.438594 31070 raft_consensus.cc:515] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:49.445299 31195 raft_consensus.cc:493] T a38ff14550f74527a62591df59fb2c51 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:49.479668 31195 raft_consensus.cc:515] T a38ff14550f74527a62591df59fb2c51 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:49.482088 31195 leader_election.cc:290] T a38ff14550f74527a62591df59fb2c51 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:49.452214 31063 consensus_queue.cc:1048] T c02f992ce0974544a4112672d8792c5f P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:11:49.482836 30671 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:11:49.483208 31089 raft_consensus.cc:515] T c1efa319af774b23bd4ea782611f5252 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:49.457593 30801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7edce5d08f24852b7afca9c57ba9c19" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:49.467568 31178 leader_election.cc:290] T d7edce5d08f24852b7afca9c57ba9c19 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Requested vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:49.490922 30801 raft_consensus.cc:3060] T d7edce5d08f24852b7afca9c57ba9c19 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:49.496255 30948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38ff14550f74527a62591df59fb2c51" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:49.499408 30873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "726b808ed64848dd89da4340d4d891fb" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:11:49.499850 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1efa319af774b23bd4ea782611f5252" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:49.500625 30799 raft_consensus.cc:3060] T c1efa319af774b23bd4ea782611f5252 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:11:49.500742 30765 leader_election.cc:343] T a38ff14550f74527a62591df59fb2c51 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:49.500142 30873 raft_consensus.cc:3060] T 726b808ed64848dd89da4340d4d891fb P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:49.501005 31174 consensus_queue.cc:1048] T ccc68dd4c56548cba0624daafb5ae23c P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:49.502988 30948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "726b808ed64848dd89da4340d4d891fb" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:11:49.503641 30948 raft_consensus.cc:3060] T 726b808ed64848dd89da4340d4d891fb P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:49.501477 30725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38ff14550f74527a62591df59fb2c51" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:49.490640 30995 tablet_bootstrap.cc:654] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
W20260709 13:11:49.505632 30767 leader_election.cc:343] T a38ff14550f74527a62591df59fb2c51 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:49.506194 30767 leader_election.cc:304] T a38ff14550f74527a62591df59fb2c51 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be; no voters: 262942f67dfa4b8fa42234b18ac31ffd, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:49.507571 31109 raft_consensus.cc:2749] T a38ff14550f74527a62591df59fb2c51 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:49.509191 30726 raft_consensus.cc:2468] T d7edce5d08f24852b7afca9c57ba9c19 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 1.
I20260709 13:11:49.510794 30915 leader_election.cc:304] T d7edce5d08f24852b7afca9c57ba9c19 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:49.512456 31174 raft_consensus.cc:2804] T d7edce5d08f24852b7afca9c57ba9c19 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:49.513013 31174 raft_consensus.cc:697] T d7edce5d08f24852b7afca9c57ba9c19 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Becoming Leader. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Running, Role: LEADER
I20260709 13:11:49.513834 31174 consensus_queue.cc:237] T d7edce5d08f24852b7afca9c57ba9c19 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:49.516877 30801 raft_consensus.cc:2468] T d7edce5d08f24852b7afca9c57ba9c19 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 1.
I20260709 13:11:49.520066 31089 leader_election.cc:290] T c1efa319af774b23bd4ea782611f5252 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Requested vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:49.520224 31109 raft_consensus.cc:493] T 054fc96e257a4dc6b2ef90db5c1ad499 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:49.521103 31109 raft_consensus.cc:515] T 054fc96e257a4dc6b2ef90db5c1ad499 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:49.523181 31109 leader_election.cc:290] T 054fc96e257a4dc6b2ef90db5c1ad499 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:49.524869 30947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054fc96e257a4dc6b2ef90db5c1ad499" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
W20260709 13:11:49.526449 30765 leader_election.cc:343] T 054fc96e257a4dc6b2ef90db5c1ad499 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:49.527109 30874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054fc96e257a4dc6b2ef90db5c1ad499" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
W20260709 13:11:49.528613 30766 leader_election.cc:343] T 054fc96e257a4dc6b2ef90db5c1ad499 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:49.529201 30766 leader_election.cc:304] T 054fc96e257a4dc6b2ef90db5c1ad499 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 262942f67dfa4b8fa42234b18ac31ffd
I20260709 13:11:49.529942 31109 raft_consensus.cc:2749] T 054fc96e257a4dc6b2ef90db5c1ad499 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:49.532523 30873 raft_consensus.cc:2468] T 726b808ed64848dd89da4340d4d891fb P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 1.
I20260709 13:11:49.533800 30691 leader_election.cc:304] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:49.534677 31066 raft_consensus.cc:2804] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:49.534735 30875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1efa319af774b23bd4ea782611f5252" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:11:49.535275 31066 raft_consensus.cc:697] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd [term 1 LEADER]: Becoming Leader. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Running, Role: LEADER
W20260709 13:11:49.536265 30914 leader_election.cc:343] T c1efa319af774b23bd4ea782611f5252 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:49.536165 31066 consensus_queue.cc:237] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:49.534674 31002 raft_consensus.cc:493] T 64a95fa1266848048486722c02f0d4cf P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:49.538982 31002 raft_consensus.cc:515] T 64a95fa1266848048486722c02f0d4cf P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:49.542119 31002 leader_election.cc:290] T 64a95fa1266848048486722c02f0d4cf P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:49.542855 30726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64a95fa1266848048486722c02f0d4cf" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:49.546993 30801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "64a95fa1266848048486722c02f0d4cf" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:49.549023 30838 leader_election.cc:304] T 64a95fa1266848048486722c02f0d4cf P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a; no voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:49.549557 30996 tablet_bootstrap.cc:654] T 93a75acc8893414180a15797033a2d5a P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:49.550180 31002 raft_consensus.cc:2749] T 64a95fa1266848048486722c02f0d4cf P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:49.557461 30994 tablet_bootstrap.cc:492] T b9fcfade988746a2967e115dbbcfe0f1 P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
I20260709 13:11:49.558146 30994 ts_tablet_manager.cc:1403] T b9fcfade988746a2967e115dbbcfe0f1 P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.210s	user 0.065s	sys 0.080s
I20260709 13:11:49.561646 30994 raft_consensus.cc:359] T b9fcfade988746a2967e115dbbcfe0f1 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:49.562602 30994 raft_consensus.cc:385] T b9fcfade988746a2967e115dbbcfe0f1 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:49.562999 30994 raft_consensus.cc:740] T b9fcfade988746a2967e115dbbcfe0f1 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
I20260709 13:11:49.563901 31070 leader_election.cc:290] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 election: Requested vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:49.567684 31089 consensus_queue.cc:1048] T 26172e2c14cd4d178764ed13012c21f7 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:49.563213 30948 raft_consensus.cc:2468] T 726b808ed64848dd89da4340d4d891fb P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 1.
I20260709 13:11:49.571687 30994 consensus_queue.cc:260] T b9fcfade988746a2967e115dbbcfe0f1 P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:49.574708 30620 catalog_manager.cc:5697] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd reported cstate change: term changed from 0 to 2, leader changed from <none> to d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1). New cstate: current_term: 2 leader_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:49.575871 30994 ts_tablet_manager.cc:1434] T b9fcfade988746a2967e115dbbcfe0f1 P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.017s	user 0.008s	sys 0.001s
I20260709 13:11:49.577189 30994 tablet_bootstrap.cc:492] T b3754a670e0b4ed99d37283bf1647bf5 P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
I20260709 13:11:49.599982 30997 tablet_bootstrap.cc:492] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
I20260709 13:11:49.600536 30997 ts_tablet_manager.cc:1403] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.316s	user 0.027s	sys 0.039s
I20260709 13:11:49.605446 31109 raft_consensus.cc:493] T b9fcfade988746a2967e115dbbcfe0f1 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:49.606153 31109 raft_consensus.cc:515] T b9fcfade988746a2967e115dbbcfe0f1 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:49.607342 30799 raft_consensus.cc:2468] T c1efa319af774b23bd4ea782611f5252 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 1.
I20260709 13:11:49.610352 30912 leader_election.cc:304] T c1efa319af774b23bd4ea782611f5252 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a
I20260709 13:11:49.611858 31174 raft_consensus.cc:2804] T c1efa319af774b23bd4ea782611f5252 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:49.612682 31174 raft_consensus.cc:697] T c1efa319af774b23bd4ea782611f5252 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Becoming Leader. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Running, Role: LEADER
I20260709 13:11:49.611603 30875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9fcfade988746a2967e115dbbcfe0f1" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:49.616112 31109 leader_election.cc:290] T b9fcfade988746a2967e115dbbcfe0f1 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:49.611650 30949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9fcfade988746a2967e115dbbcfe0f1" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:49.613932 31174 consensus_queue.cc:237] T c1efa319af774b23bd4ea782611f5252 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:49.621770 30995 tablet_bootstrap.cc:492] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:49.622347 30995 ts_tablet_manager.cc:1403] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.271s	user 0.019s	sys 0.040s
I20260709 13:11:49.626648 30997 raft_consensus.cc:359] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:49.626855 30995 raft_consensus.cc:359] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:49.627359 30997 raft_consensus.cc:385] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:49.627628 30995 raft_consensus.cc:385] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:49.627682 30997 raft_consensus.cc:740] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
I20260709 13:11:49.628175 30995 raft_consensus.cc:740] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
W20260709 13:11:49.623843 30766 leader_election.cc:343] T b9fcfade988746a2967e115dbbcfe0f1 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:49.629026 30995 consensus_queue.cc:260] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
W20260709 13:11:49.630146 30765 leader_election.cc:343] T b9fcfade988746a2967e115dbbcfe0f1 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:49.631448 30994 tablet_bootstrap.cc:654] T b3754a670e0b4ed99d37283bf1647bf5 P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:49.628628 30997 consensus_queue.cc:260] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:49.634006 31070 raft_consensus.cc:493] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:49.632616 30765 leader_election.cc:304] T b9fcfade988746a2967e115dbbcfe0f1 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 262942f67dfa4b8fa42234b18ac31ffd
I20260709 13:11:49.631448 30995 ts_tablet_manager.cc:1434] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.002s
I20260709 13:11:49.644634 30875 raft_consensus.cc:3060] T 64a95fa1266848048486722c02f0d4cf P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:49.648813 31070 raft_consensus.cc:515] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:49.649977 30995 tablet_bootstrap.cc:492] T a38ff14550f74527a62591df59fb2c51 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:49.651890 31070 leader_election.cc:290] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:49.636473 30997 ts_tablet_manager.cc:1434] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.036s	user 0.006s	sys 0.000s
I20260709 13:11:49.652596 30873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01cceb88bca348958b59f3697979dff1" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:49.653863 30997 tablet_bootstrap.cc:492] T 93a75acc8893414180a15797033a2d5a P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
W20260709 13:11:49.654026 30691 leader_election.cc:343] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:49.654196 30949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01cceb88bca348958b59f3697979dff1" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:49.654870 30949 raft_consensus.cc:2468] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 0.
I20260709 13:11:49.635138 30996 tablet_bootstrap.cc:492] T 93a75acc8893414180a15797033a2d5a P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
I20260709 13:11:49.655815 30996 ts_tablet_manager.cc:1403] T 93a75acc8893414180a15797033a2d5a P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.307s	user 0.027s	sys 0.023s
I20260709 13:11:49.659006 31109 raft_consensus.cc:2749] T b9fcfade988746a2967e115dbbcfe0f1 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:49.659092 30996 raft_consensus.cc:359] T 93a75acc8893414180a15797033a2d5a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:49.655967 30690 leader_election.cc:304] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a
I20260709 13:11:49.660027 30996 raft_consensus.cc:385] T 93a75acc8893414180a15797033a2d5a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:49.660403 30996 raft_consensus.cc:740] T 93a75acc8893414180a15797033a2d5a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
I20260709 13:11:49.660655 31070 raft_consensus.cc:2804] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:49.668012 31070 raft_consensus.cc:493] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:49.662796 30995 tablet_bootstrap.cc:654] T a38ff14550f74527a62591df59fb2c51 P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:49.644248 31082 consensus_queue.cc:1048] T 26172e2c14cd4d178764ed13012c21f7 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260709 13:11:49.674180 30997 tablet_bootstrap.cc:654] T 93a75acc8893414180a15797033a2d5a P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:49.668438 31070 raft_consensus.cc:3060] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:49.678519 30996 consensus_queue.cc:260] T 93a75acc8893414180a15797033a2d5a P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:49.691027 30996 ts_tablet_manager.cc:1434] T 93a75acc8893414180a15797033a2d5a P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.000s
I20260709 13:11:49.692051 30996 tablet_bootstrap.cc:492] T 054fc96e257a4dc6b2ef90db5c1ad499 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
I20260709 13:11:49.689224 31070 raft_consensus.cc:515] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:49.636596 30620 catalog_manager.cc:5697] T d7edce5d08f24852b7afca9c57ba9c19 P 262942f67dfa4b8fa42234b18ac31ffd reported cstate change: term changed from 0 to 1, leader changed from <none> to 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4). New cstate: current_term: 1 leader_uuid: "262942f67dfa4b8fa42234b18ac31ffd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:49.698302 31070 leader_election.cc:290] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 election: Requested vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:49.699021 30873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01cceb88bca348958b59f3697979dff1" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:11:49.699332 30949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01cceb88bca348958b59f3697979dff1" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:11:49.700007 30949 raft_consensus.cc:3060] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:49.702390 30996 tablet_bootstrap.cc:654] T 054fc96e257a4dc6b2ef90db5c1ad499 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
W20260709 13:11:49.702464 30691 leader_election.cc:343] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:49.784390 30799 raft_consensus.cc:1275] T 1474b6fc151c4a3994a3aef89ea936a2 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Refusing update from remote peer d22c9d3d15904327a78a0e548a8ff6cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:11:49.786895 31070 consensus_queue.cc:1048] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:11:49.793363 31216 log.cc:927] Time spent T 64a95fa1266848048486722c02f0d4cf P 02c5705b45c04706bbb2c6ff3d7b0b8a: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 13:11:49.794628 30873 raft_consensus.cc:1275] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Refusing update from remote peer d22c9d3d15904327a78a0e548a8ff6cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:11:49.766795 30949 raft_consensus.cc:2468] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 1.
I20260709 13:11:49.799957 31069 consensus_queue.cc:1048] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:49.802381 30690 leader_election.cc:304] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a
I20260709 13:11:49.805757 31069 raft_consensus.cc:2804] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:49.806414 31069 raft_consensus.cc:697] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 LEADER]: Becoming Leader. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Running, Role: LEADER
I20260709 13:11:49.807482 31069 consensus_queue.cc:237] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:49.834609 31178 raft_consensus.cc:493] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:49.838313 31178 raft_consensus.cc:515] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:49.844190 31109 raft_consensus.cc:493] T 0d8c1da1f7254fde94f7deaad5f41709 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:49.844944 31109 raft_consensus.cc:515] T 0d8c1da1f7254fde94f7deaad5f41709 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:49.845542 30726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38ff14550f74527a62591df59fb2c51" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
W20260709 13:11:49.847764 30915 leader_election.cc:343] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:49.848203 31109 leader_election.cc:290] T 0d8c1da1f7254fde94f7deaad5f41709 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:49.848884 30726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d8c1da1f7254fde94f7deaad5f41709" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
W20260709 13:11:49.850476 30767 leader_election.cc:343] T 0d8c1da1f7254fde94f7deaad5f41709 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:49.848940 30873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d8c1da1f7254fde94f7deaad5f41709" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:49.851068 30801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38ff14550f74527a62591df59fb2c51" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:49.851784 30801 raft_consensus.cc:2468] T a38ff14550f74527a62591df59fb2c51 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 0.
W20260709 13:11:49.852085 30766 leader_election.cc:343] T 0d8c1da1f7254fde94f7deaad5f41709 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:49.852506 30766 leader_election.cc:304] T 0d8c1da1f7254fde94f7deaad5f41709 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:49.853052 30912 leader_election.cc:304] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:49.853331 31109 raft_consensus.cc:2749] T 0d8c1da1f7254fde94f7deaad5f41709 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:49.853994 31046 raft_consensus.cc:2804] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:49.854584 31046 raft_consensus.cc:493] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:49.855027 31046 raft_consensus.cc:3060] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:49.864816 31178 leader_election.cc:290] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
W20260709 13:11:49.876288 31217 log.cc:927] Time spent T 26172e2c14cd4d178764ed13012c21f7 P 05213ac836744a53b9c089a5c052f4be: Append to log took a long time: real 0.171s	user 0.060s	sys 0.065s
I20260709 13:11:49.876803 30994 tablet_bootstrap.cc:492] T b3754a670e0b4ed99d37283bf1647bf5 P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
I20260709 13:11:49.877368 30994 ts_tablet_manager.cc:1403] T b3754a670e0b4ed99d37283bf1647bf5 P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.300s	user 0.063s	sys 0.043s
I20260709 13:11:49.876803 30997 tablet_bootstrap.cc:492] T 93a75acc8893414180a15797033a2d5a P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
I20260709 13:11:49.878211 30997 ts_tablet_manager.cc:1403] T 93a75acc8893414180a15797033a2d5a P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.225s	user 0.046s	sys 0.031s
I20260709 13:11:49.880004 30994 raft_consensus.cc:359] T b3754a670e0b4ed99d37283bf1647bf5 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:49.880729 30997 raft_consensus.cc:359] T 93a75acc8893414180a15797033a2d5a P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:49.881493 30995 tablet_bootstrap.cc:492] T a38ff14550f74527a62591df59fb2c51 P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
W20260709 13:11:49.881553 31222 log.cc:927] Time spent T 1474b6fc151c4a3994a3aef89ea936a2 P 05213ac836744a53b9c089a5c052f4be: Append to log took a long time: real 0.060s	user 0.000s	sys 0.013s
W20260709 13:11:49.882042 31221 log.cc:927] Time spent T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Append to log took a long time: real 0.054s	user 0.000s	sys 0.013s
I20260709 13:11:49.882655 30996 tablet_bootstrap.cc:492] T 054fc96e257a4dc6b2ef90db5c1ad499 P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
W20260709 13:11:49.876928 31041 log.cc:927] Time spent T 00000000000000000000000000000000 P 35038aac9dd04872a98f269c76b1b5d3: Append to log took a long time: real 0.160s	user 0.006s	sys 0.006s
I20260709 13:11:49.883484 30996 ts_tablet_manager.cc:1403] T 054fc96e257a4dc6b2ef90db5c1ad499 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.192s	user 0.038s	sys 0.035s
I20260709 13:11:49.882795 30994 raft_consensus.cc:385] T b3754a670e0b4ed99d37283bf1647bf5 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:49.885717 30994 raft_consensus.cc:740] T b3754a670e0b4ed99d37283bf1647bf5 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
I20260709 13:11:49.886411 30996 raft_consensus.cc:359] T 054fc96e257a4dc6b2ef90db5c1ad499 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:49.886528 30994 consensus_queue.cc:260] T b3754a670e0b4ed99d37283bf1647bf5 P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:49.887156 30996 raft_consensus.cc:385] T 054fc96e257a4dc6b2ef90db5c1ad499 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:49.887475 30996 raft_consensus.cc:740] T 054fc96e257a4dc6b2ef90db5c1ad499 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
I20260709 13:11:49.888120 30996 consensus_queue.cc:260] T 054fc96e257a4dc6b2ef90db5c1ad499 P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:49.882885 30997 raft_consensus.cc:385] T 93a75acc8893414180a15797033a2d5a P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:49.889376 30997 raft_consensus.cc:740] T 93a75acc8893414180a15797033a2d5a P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
I20260709 13:11:49.890187 30996 ts_tablet_manager.cc:1434] T 054fc96e257a4dc6b2ef90db5c1ad499 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:11:49.890407 30997 consensus_queue.cc:260] T 93a75acc8893414180a15797033a2d5a P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:49.891162 30996 tablet_bootstrap.cc:492] T c1efa319af774b23bd4ea782611f5252 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
I20260709 13:11:49.893390 30997 ts_tablet_manager.cc:1434] T 93a75acc8893414180a15797033a2d5a P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.001s
I20260709 13:11:49.894356 30997 tablet_bootstrap.cc:492] T 054fc96e257a4dc6b2ef90db5c1ad499 P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
I20260709 13:11:49.898674 30994 ts_tablet_manager.cc:1434] T b3754a670e0b4ed99d37283bf1647bf5 P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.001s
I20260709 13:11:49.898893 30996 tablet_bootstrap.cc:654] T c1efa319af774b23bd4ea782611f5252 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:49.899746 30994 tablet_bootstrap.cc:492] T 892368b6886842febffc849d73f76c10 P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
I20260709 13:11:49.902168 30997 tablet_bootstrap.cc:654] T 054fc96e257a4dc6b2ef90db5c1ad499 P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:49.906136 30994 tablet_bootstrap.cc:654] T 892368b6886842febffc849d73f76c10 P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:49.911455 31002 raft_consensus.cc:493] T 054fc96e257a4dc6b2ef90db5c1ad499 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:49.912138 31002 raft_consensus.cc:515] T 054fc96e257a4dc6b2ef90db5c1ad499 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:49.915052 31002 leader_election.cc:290] T 054fc96e257a4dc6b2ef90db5c1ad499 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:49.916054 30801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054fc96e257a4dc6b2ef90db5c1ad499" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:49.916978 30801 raft_consensus.cc:2468] T 054fc96e257a4dc6b2ef90db5c1ad499 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 0.
I20260709 13:11:49.918761 30838 leader_election.cc:304] T 054fc96e257a4dc6b2ef90db5c1ad499 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 05213ac836744a53b9c089a5c052f4be; no voters: 
I20260709 13:11:49.919035 30994 tablet_bootstrap.cc:492] T 892368b6886842febffc849d73f76c10 P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
I20260709 13:11:49.919867 30994 ts_tablet_manager.cc:1403] T 892368b6886842febffc849d73f76c10 P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.020s	user 0.017s	sys 0.001s
I20260709 13:11:49.920027 31002 raft_consensus.cc:2804] T 054fc96e257a4dc6b2ef90db5c1ad499 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:49.920614 31002 raft_consensus.cc:493] T 054fc96e257a4dc6b2ef90db5c1ad499 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:49.921178 31002 raft_consensus.cc:3060] T 054fc96e257a4dc6b2ef90db5c1ad499 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:49.922960 30994 raft_consensus.cc:359] T 892368b6886842febffc849d73f76c10 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:49.923635 30994 raft_consensus.cc:385] T 892368b6886842febffc849d73f76c10 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:49.923873 30994 raft_consensus.cc:740] T 892368b6886842febffc849d73f76c10 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
I20260709 13:11:49.924536 30994 consensus_queue.cc:260] T 892368b6886842febffc849d73f76c10 P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:49.925737 30997 tablet_bootstrap.cc:492] T 054fc96e257a4dc6b2ef90db5c1ad499 P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
I20260709 13:11:49.926434 30997 ts_tablet_manager.cc:1403] T 054fc96e257a4dc6b2ef90db5c1ad499 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.032s	user 0.012s	sys 0.005s
I20260709 13:11:49.926808 30994 ts_tablet_manager.cc:1434] T 892368b6886842febffc849d73f76c10 P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 13:11:49.927762 30994 tablet_bootstrap.cc:492] T 5532e197e7de48f28c034ba1fc490e19 P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
I20260709 13:11:49.927587 30949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054fc96e257a4dc6b2ef90db5c1ad499" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:49.929479 30997 raft_consensus.cc:359] T 054fc96e257a4dc6b2ef90db5c1ad499 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:49.930101 30997 raft_consensus.cc:385] T 054fc96e257a4dc6b2ef90db5c1ad499 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:49.930367 30997 raft_consensus.cc:740] T 054fc96e257a4dc6b2ef90db5c1ad499 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
W20260709 13:11:49.930749 30839 leader_election.cc:343] T 054fc96e257a4dc6b2ef90db5c1ad499 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:49.931092 30997 consensus_queue.cc:260] T 054fc96e257a4dc6b2ef90db5c1ad499 P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:49.935494 30994 tablet_bootstrap.cc:654] T 5532e197e7de48f28c034ba1fc490e19 P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:49.938634 30997 ts_tablet_manager.cc:1434] T 054fc96e257a4dc6b2ef90db5c1ad499 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 13:11:49.939666 30997 tablet_bootstrap.cc:492] T b3754a670e0b4ed99d37283bf1647bf5 P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
I20260709 13:11:49.911159 30619 catalog_manager.cc:5697] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd reported cstate change: term changed from 0 to 1, leader changed from <none> to d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1). New cstate: current_term: 1 leader_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:49.947064 30997 tablet_bootstrap.cc:654] T b3754a670e0b4ed99d37283bf1647bf5 P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:49.948441 30994 tablet_bootstrap.cc:492] T 5532e197e7de48f28c034ba1fc490e19 P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
I20260709 13:11:49.948969 30994 ts_tablet_manager.cc:1403] T 5532e197e7de48f28c034ba1fc490e19 P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.021s	user 0.003s	sys 0.014s
I20260709 13:11:49.882643 31046 raft_consensus.cc:515] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:49.960914 30994 raft_consensus.cc:359] T 5532e197e7de48f28c034ba1fc490e19 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:49.962875 30994 raft_consensus.cc:385] T 5532e197e7de48f28c034ba1fc490e19 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:49.963241 30994 raft_consensus.cc:740] T 5532e197e7de48f28c034ba1fc490e19 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
I20260709 13:11:49.964130 30994 consensus_queue.cc:260] T 5532e197e7de48f28c034ba1fc490e19 P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:49.961634 30995 ts_tablet_manager.cc:1403] T a38ff14550f74527a62591df59fb2c51 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.312s	user 0.010s	sys 0.004s
I20260709 13:11:49.967877 30997 tablet_bootstrap.cc:492] T b3754a670e0b4ed99d37283bf1647bf5 P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
I20260709 13:11:49.967662 31115 raft_consensus.cc:493] T 93a75acc8893414180a15797033a2d5a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:49.968448 30997 ts_tablet_manager.cc:1403] T b3754a670e0b4ed99d37283bf1647bf5 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.029s	user 0.017s	sys 0.001s
I20260709 13:11:49.968564 31115 raft_consensus.cc:515] T 93a75acc8893414180a15797033a2d5a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:49.974450 30997 raft_consensus.cc:359] T b3754a670e0b4ed99d37283bf1647bf5 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:49.975138 30997 raft_consensus.cc:385] T b3754a670e0b4ed99d37283bf1647bf5 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:49.975387 30997 raft_consensus.cc:740] T b3754a670e0b4ed99d37283bf1647bf5 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
I20260709 13:11:49.975721 30994 ts_tablet_manager.cc:1434] T 5532e197e7de48f28c034ba1fc490e19 P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.001s
I20260709 13:11:49.976020 30997 consensus_queue.cc:260] T b3754a670e0b4ed99d37283bf1647bf5 P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:49.976698 30994 tablet_bootstrap.cc:492] T 8fb24a20af104a7486d0f1b2ae0e3843 P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
I20260709 13:11:49.977995 30997 ts_tablet_manager.cc:1434] T b3754a670e0b4ed99d37283bf1647bf5 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:11:49.978844 30997 tablet_bootstrap.cc:492] T 92a4b4a5a07442f49f518b479db8c94b P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
I20260709 13:11:49.976558 30947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93a75acc8893414180a15797033a2d5a" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:49.982945 30994 tablet_bootstrap.cc:654] T 8fb24a20af104a7486d0f1b2ae0e3843 P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:49.983392 30947 raft_consensus.cc:2468] T 93a75acc8893414180a15797033a2d5a P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 0.
I20260709 13:11:49.984354 30997 tablet_bootstrap.cc:654] T 92a4b4a5a07442f49f518b479db8c94b P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:49.968024 30995 raft_consensus.cc:359] T a38ff14550f74527a62591df59fb2c51 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:49.985340 30995 raft_consensus.cc:385] T a38ff14550f74527a62591df59fb2c51 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:49.985635 30995 raft_consensus.cc:740] T a38ff14550f74527a62591df59fb2c51 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:49.986444 30995 consensus_queue.cc:260] T a38ff14550f74527a62591df59fb2c51 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:49.988879 30726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93a75acc8893414180a15797033a2d5a" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:49.989454 30995 ts_tablet_manager.cc:1434] T a38ff14550f74527a62591df59fb2c51 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.024s	user 0.004s	sys 0.002s
I20260709 13:11:49.988885 30839 leader_election.cc:304] T 93a75acc8893414180a15797033a2d5a P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
I20260709 13:11:49.990679 30995 tablet_bootstrap.cc:492] T 892368b6886842febffc849d73f76c10 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
W20260709 13:11:49.991431 30841 leader_election.cc:343] T 93a75acc8893414180a15797033a2d5a P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:49.990924 31063 raft_consensus.cc:2804] T 93a75acc8893414180a15797033a2d5a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:49.997889 31063 raft_consensus.cc:493] T 93a75acc8893414180a15797033a2d5a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:49.998095 30726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38ff14550f74527a62591df59fb2c51" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:49.998406 31063 raft_consensus.cc:3060] T 93a75acc8893414180a15797033a2d5a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:49.998765 30726 raft_consensus.cc:3060] T a38ff14550f74527a62591df59fb2c51 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:49.990767 31046 leader_election.cc:290] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Requested vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:50.006435 31002 raft_consensus.cc:515] T 054fc96e257a4dc6b2ef90db5c1ad499 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:50.000739 30801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38ff14550f74527a62591df59fb2c51" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:50.008203 30801 raft_consensus.cc:3060] T a38ff14550f74527a62591df59fb2c51 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:49.989145 31115 leader_election.cc:290] T 93a75acc8893414180a15797033a2d5a P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:50.018663 31002 leader_election.cc:290] T 054fc96e257a4dc6b2ef90db5c1ad499 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Requested vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:50.025856 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054fc96e257a4dc6b2ef90db5c1ad499" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:50.028020 30799 raft_consensus.cc:3060] T 054fc96e257a4dc6b2ef90db5c1ad499 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:50.039203 31109 raft_consensus.cc:493] T 5532e197e7de48f28c034ba1fc490e19 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:50.041615 30947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "054fc96e257a4dc6b2ef90db5c1ad499" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:11:50.042279 30947 raft_consensus.cc:3060] T 054fc96e257a4dc6b2ef90db5c1ad499 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:50.040133 31109 raft_consensus.cc:515] T 5532e197e7de48f28c034ba1fc490e19 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:50.050417 31066 consensus_queue.cc:1048] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:50.047784 30995 tablet_bootstrap.cc:654] T 892368b6886842febffc849d73f76c10 P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:50.064477 30997 tablet_bootstrap.cc:492] T 92a4b4a5a07442f49f518b479db8c94b P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
I20260709 13:11:50.065025 30997 ts_tablet_manager.cc:1403] T 92a4b4a5a07442f49f518b479db8c94b P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.086s	user 0.028s	sys 0.035s
I20260709 13:11:50.045908 30726 raft_consensus.cc:2468] T a38ff14550f74527a62591df59fb2c51 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 1.
I20260709 13:11:50.068483 31178 raft_consensus.cc:493] T b3754a670e0b4ed99d37283bf1647bf5 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:50.068432 31063 raft_consensus.cc:515] T 93a75acc8893414180a15797033a2d5a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:50.069187 31178 raft_consensus.cc:515] T b3754a670e0b4ed99d37283bf1647bf5 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:50.066432 31082 consensus_queue.cc:1048] T d7edce5d08f24852b7afca9c57ba9c19 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:11:50.071177 31178 leader_election.cc:290] T b3754a670e0b4ed99d37283bf1647bf5 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:50.071403 30949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93a75acc8893414180a15797033a2d5a" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:11:50.072181 30949 raft_consensus.cc:3060] T 93a75acc8893414180a15797033a2d5a P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:50.072685 30800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3754a670e0b4ed99d37283bf1647bf5" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:50.073299 30800 raft_consensus.cc:2468] T b3754a670e0b4ed99d37283bf1647bf5 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 0.
I20260709 13:11:50.075121 30912 leader_election.cc:304] T b3754a670e0b4ed99d37283bf1647bf5 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
I20260709 13:11:50.076287 31178 raft_consensus.cc:2804] T b3754a670e0b4ed99d37283bf1647bf5 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:50.076689 31178 raft_consensus.cc:493] T b3754a670e0b4ed99d37283bf1647bf5 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:50.077028 31178 raft_consensus.cc:3060] T b3754a670e0b4ed99d37283bf1647bf5 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:50.078047 30725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3754a670e0b4ed99d37283bf1647bf5" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
W20260709 13:11:50.079803 30915 leader_election.cc:343] T b3754a670e0b4ed99d37283bf1647bf5 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:50.081720 31066 consensus_queue.cc:1048] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:50.082789 31109 leader_election.cc:290] T 5532e197e7de48f28c034ba1fc490e19 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:50.071498 30947 raft_consensus.cc:2468] T 054fc96e257a4dc6b2ef90db5c1ad499 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 1.
I20260709 13:11:50.089419 30839 leader_election.cc:304] T 054fc96e257a4dc6b2ef90db5c1ad499 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
I20260709 13:11:50.090503 31002 raft_consensus.cc:2804] T 054fc96e257a4dc6b2ef90db5c1ad499 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:50.091383 31002 raft_consensus.cc:697] T 054fc96e257a4dc6b2ef90db5c1ad499 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Becoming Leader. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Running, Role: LEADER
I20260709 13:11:50.092634 30801 raft_consensus.cc:2468] T a38ff14550f74527a62591df59fb2c51 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 1.
I20260709 13:11:50.093425 31002 consensus_queue.cc:237] T 054fc96e257a4dc6b2ef90db5c1ad499 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:50.088091 30799 raft_consensus.cc:2468] T 054fc96e257a4dc6b2ef90db5c1ad499 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 1.
I20260709 13:11:50.088213 30949 raft_consensus.cc:2468] T 93a75acc8893414180a15797033a2d5a P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 1.
I20260709 13:11:50.094635 30915 leader_election.cc:304] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:50.094856 31063 leader_election.cc:290] T 93a75acc8893414180a15797033a2d5a P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Requested vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:50.096577 30996 tablet_bootstrap.cc:492] T c1efa319af774b23bd4ea782611f5252 P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
I20260709 13:11:50.083962 31178 raft_consensus.cc:515] T b3754a670e0b4ed99d37283bf1647bf5 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:50.100049 30996 ts_tablet_manager.cc:1403] T c1efa319af774b23bd4ea782611f5252 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.209s	user 0.064s	sys 0.073s
I20260709 13:11:50.106663 30726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93a75acc8893414180a15797033a2d5a" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:50.099802 31046 consensus_queue.cc:1048] T d7edce5d08f24852b7afca9c57ba9c19 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 13:11:50.107724 30839 leader_election.cc:304] T 93a75acc8893414180a15797033a2d5a P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
I20260709 13:11:50.107764 31082 raft_consensus.cc:2804] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:50.111618 31082 raft_consensus.cc:697] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Becoming Leader. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Running, Role: LEADER
W20260709 13:11:50.111029 30841 leader_election.cc:343] T 93a75acc8893414180a15797033a2d5a P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:50.112731 31082 consensus_queue.cc:237] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:50.114351 30621 catalog_manager.cc:5697] T c1efa319af774b23bd4ea782611f5252 P 262942f67dfa4b8fa42234b18ac31ffd reported cstate change: term changed from 0 to 1, leader changed from <none> to 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4). New cstate: current_term: 1 leader_uuid: "262942f67dfa4b8fa42234b18ac31ffd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:50.116176 31063 raft_consensus.cc:2804] T 93a75acc8893414180a15797033a2d5a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:50.143204 30995 tablet_bootstrap.cc:492] T 892368b6886842febffc849d73f76c10 P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:50.143940 30995 ts_tablet_manager.cc:1403] T 892368b6886842febffc849d73f76c10 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.154s	user 0.015s	sys 0.001s
I20260709 13:11:50.147177 31178 leader_election.cc:290] T b3754a670e0b4ed99d37283bf1647bf5 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Requested vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:50.148460 30725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5532e197e7de48f28c034ba1fc490e19" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:50.156533 30725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3754a670e0b4ed99d37283bf1647bf5" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:50.157856 30801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3754a670e0b4ed99d37283bf1647bf5" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:50.158492 30801 raft_consensus.cc:3060] T b3754a670e0b4ed99d37283bf1647bf5 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:11:50.159097 30915 leader_election.cc:343] T b3754a670e0b4ed99d37283bf1647bf5 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:11:50.160969 30841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:37325, user_credentials={real_user=slave}} blocked reactor thread for 51332.1us
I20260709 13:11:50.163295 30997 raft_consensus.cc:359] T 92a4b4a5a07442f49f518b479db8c94b P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:50.164009 30997 raft_consensus.cc:385] T 92a4b4a5a07442f49f518b479db8c94b P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:50.164283 30997 raft_consensus.cc:740] T 92a4b4a5a07442f49f518b479db8c94b P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
I20260709 13:11:50.164997 30997 consensus_queue.cc:260] T 92a4b4a5a07442f49f518b479db8c94b P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:50.129693 31063 raft_consensus.cc:697] T 93a75acc8893414180a15797033a2d5a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Becoming Leader. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Running, Role: LEADER
I20260709 13:11:50.144976 30996 raft_consensus.cc:359] T c1efa319af774b23bd4ea782611f5252 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:50.168088 30996 raft_consensus.cc:385] T c1efa319af774b23bd4ea782611f5252 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:50.168426 30996 raft_consensus.cc:740] T c1efa319af774b23bd4ea782611f5252 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
I20260709 13:11:50.170928 30997 ts_tablet_manager.cc:1434] T 92a4b4a5a07442f49f518b479db8c94b P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.106s	user 0.007s	sys 0.000s
I20260709 13:11:50.171684 30997 tablet_bootstrap.cc:492] T b9fcfade988746a2967e115dbbcfe0f1 P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
I20260709 13:11:50.169490 30996 consensus_queue.cc:260] T c1efa319af774b23bd4ea782611f5252 P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:50.174188 30994 tablet_bootstrap.cc:492] T 8fb24a20af104a7486d0f1b2ae0e3843 P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
I20260709 13:11:50.174682 30994 ts_tablet_manager.cc:1403] T 8fb24a20af104a7486d0f1b2ae0e3843 P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.198s	user 0.014s	sys 0.002s
W20260709 13:11:50.168234 30838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:43771, user_credentials={real_user=slave}} blocked reactor thread for 76690.3us
I20260709 13:11:50.180438 31177 consensus_queue.cc:1048] T c1efa319af774b23bd4ea782611f5252 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 13:11:50.218098 30996 ts_tablet_manager.cc:1434] T c1efa319af774b23bd4ea782611f5252 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.118s	user 0.019s	sys 0.002s
I20260709 13:11:50.219316 30996 tablet_bootstrap.cc:492] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
W20260709 13:11:50.213014 31232 log.cc:927] Time spent T 726b808ed64848dd89da4340d4d891fb P 02c5705b45c04706bbb2c6ff3d7b0b8a: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 13:11:50.207904 31063 consensus_queue.cc:237] T 93a75acc8893414180a15797033a2d5a P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:50.228497 30994 raft_consensus.cc:359] T 8fb24a20af104a7486d0f1b2ae0e3843 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:50.241348 30994 raft_consensus.cc:385] T 8fb24a20af104a7486d0f1b2ae0e3843 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:50.241704 30994 raft_consensus.cc:740] T 8fb24a20af104a7486d0f1b2ae0e3843 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
I20260709 13:11:50.232520 30995 raft_consensus.cc:359] T 892368b6886842febffc849d73f76c10 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:50.243292 30995 raft_consensus.cc:385] T 892368b6886842febffc849d73f76c10 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:50.243683 30995 raft_consensus.cc:740] T 892368b6886842febffc849d73f76c10 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:50.244333 30994 consensus_queue.cc:260] T 8fb24a20af104a7486d0f1b2ae0e3843 P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:50.248144 30995 consensus_queue.cc:260] T 892368b6886842febffc849d73f76c10 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:50.251407 30801 raft_consensus.cc:2468] T b3754a670e0b4ed99d37283bf1647bf5 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 1.
I20260709 13:11:50.253683 30995 ts_tablet_manager.cc:1434] T 892368b6886842febffc849d73f76c10 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.109s	user 0.004s	sys 0.004s
I20260709 13:11:50.254913 30995 tablet_bootstrap.cc:492] T 93a75acc8893414180a15797033a2d5a P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
W20260709 13:11:50.210747 30767 leader_election.cc:343] T 5532e197e7de48f28c034ba1fc490e19 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:50.258322 30994 ts_tablet_manager.cc:1434] T 8fb24a20af104a7486d0f1b2ae0e3843 P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.083s	user 0.010s	sys 0.000s
I20260709 13:11:50.267556 30994 tablet_bootstrap.cc:492] T 8155b3fc02b348e1adb290ecfe4592c5 P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
I20260709 13:11:50.217391 30619 catalog_manager.cc:5697] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd reported cstate change: term changed from 0 to 1, leader changed from <none> to d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1). New cstate: current_term: 1 leader_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:50.265580 30912 leader_election.cc:304] T b3754a670e0b4ed99d37283bf1647bf5 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:50.273648 30875 raft_consensus.cc:3060] T c1efa319af774b23bd4ea782611f5252 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:50.273978 31046 raft_consensus.cc:2804] T b3754a670e0b4ed99d37283bf1647bf5 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:50.274595 31046 raft_consensus.cc:697] T b3754a670e0b4ed99d37283bf1647bf5 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Becoming Leader. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Running, Role: LEADER
I20260709 13:11:50.275414 31046 consensus_queue.cc:237] T b3754a670e0b4ed99d37283bf1647bf5 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
W20260709 13:11:50.261708 30767 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:37325, user_credentials={real_user=slave}} blocked reactor thread for 55151.7us
I20260709 13:11:50.303032 30996 tablet_bootstrap.cc:654] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:50.312373 30997 tablet_bootstrap.cc:654] T b9fcfade988746a2967e115dbbcfe0f1 P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:50.311553 30621 catalog_manager.cc:5697] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd reported cstate change: term changed from 0 to 1, leader changed from <none> to 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4). New cstate: current_term: 1 leader_uuid: "262942f67dfa4b8fa42234b18ac31ffd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:50.310995 30874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5532e197e7de48f28c034ba1fc490e19" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:50.308517 31230 raft_consensus.cc:493] T 892368b6886842febffc849d73f76c10 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:50.325486 31230 raft_consensus.cc:515] T 892368b6886842febffc849d73f76c10 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
W20260709 13:11:50.327333 30766 leader_election.cc:343] T 5532e197e7de48f28c034ba1fc490e19 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:50.327570 31230 leader_election.cc:290] T 892368b6886842febffc849d73f76c10 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:50.327845 30766 leader_election.cc:304] T 5532e197e7de48f28c034ba1fc490e19 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:50.328735 31177 consensus_queue.cc:1048] T c1efa319af774b23bd4ea782611f5252 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:50.329870 31230 raft_consensus.cc:2749] T 5532e197e7de48f28c034ba1fc490e19 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:50.333575 30874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "892368b6886842febffc849d73f76c10" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:50.337863 31066 consensus_queue.cc:1048] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:11:50.341198 30766 leader_election.cc:343] T 892368b6886842febffc849d73f76c10 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:50.342726 30995 tablet_bootstrap.cc:654] T 93a75acc8893414180a15797033a2d5a P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:50.361219 30725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "892368b6886842febffc849d73f76c10" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:50.361979 30725 raft_consensus.cc:2468] T 892368b6886842febffc849d73f76c10 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05213ac836744a53b9c089a5c052f4be in term 0.
I20260709 13:11:50.366969 30767 leader_election.cc:304] T 892368b6886842febffc849d73f76c10 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a
I20260709 13:11:50.368124 31109 raft_consensus.cc:2804] T 892368b6886842febffc849d73f76c10 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:50.366695 30619 catalog_manager.cc:5697] T 054fc96e257a4dc6b2ef90db5c1ad499 P 02c5705b45c04706bbb2c6ff3d7b0b8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3). New cstate: current_term: 1 leader_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:50.368738 31109 raft_consensus.cc:493] T 892368b6886842febffc849d73f76c10 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:50.370198 31109 raft_consensus.cc:3060] T 892368b6886842febffc849d73f76c10 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:50.382421 30994 tablet_bootstrap.cc:654] T 8155b3fc02b348e1adb290ecfe4592c5 P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:50.389060 30997 tablet_bootstrap.cc:492] T b9fcfade988746a2967e115dbbcfe0f1 P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
I20260709 13:11:50.389750 30997 ts_tablet_manager.cc:1403] T b9fcfade988746a2967e115dbbcfe0f1 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.218s	user 0.023s	sys 0.006s
I20260709 13:11:50.393729 31109 raft_consensus.cc:515] T 892368b6886842febffc849d73f76c10 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:50.396340 31109 leader_election.cc:290] T 892368b6886842febffc849d73f76c10 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 election: Requested vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:50.397037 30875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "892368b6886842febffc849d73f76c10" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:11:50.397084 30725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "892368b6886842febffc849d73f76c10" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:50.397925 30725 raft_consensus.cc:3060] T 892368b6886842febffc849d73f76c10 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:50.398640 30996 tablet_bootstrap.cc:492] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
I20260709 13:11:50.399101 30996 ts_tablet_manager.cc:1403] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.180s	user 0.012s	sys 0.003s
I20260709 13:11:50.400573 30997 raft_consensus.cc:359] T b9fcfade988746a2967e115dbbcfe0f1 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:50.401275 30997 raft_consensus.cc:385] T b9fcfade988746a2967e115dbbcfe0f1 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:50.401607 30996 raft_consensus.cc:359] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:50.401742 30997 raft_consensus.cc:740] T b9fcfade988746a2967e115dbbcfe0f1 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
I20260709 13:11:50.400720 30621 catalog_manager.cc:5697] T b3754a670e0b4ed99d37283bf1647bf5 P 262942f67dfa4b8fa42234b18ac31ffd reported cstate change: term changed from 0 to 1, leader changed from <none> to 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4). New cstate: current_term: 1 leader_uuid: "262942f67dfa4b8fa42234b18ac31ffd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:50.402289 30996 raft_consensus.cc:385] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:50.402612 30996 raft_consensus.cc:740] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
I20260709 13:11:50.403290 30996 consensus_queue.cc:260] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:50.402791 30997 consensus_queue.cc:260] T b9fcfade988746a2967e115dbbcfe0f1 P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:50.405552 30996 ts_tablet_manager.cc:1434] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:11:50.406566 30997 ts_tablet_manager.cc:1434] T b9fcfade988746a2967e115dbbcfe0f1 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 13:11:50.406713 30996 tablet_bootstrap.cc:492] T 892368b6886842febffc849d73f76c10 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
I20260709 13:11:50.407442 30997 tablet_bootstrap.cc:492] T 784deee1ea9543df840c58ecaa6f911a P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
W20260709 13:11:50.418193 30691 consensus_peers.cc:597] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd -> Peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Couldn't send request to peer 02c5705b45c04706bbb2c6ff3d7b0b8a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:50.423393 30725 raft_consensus.cc:2468] T 892368b6886842febffc849d73f76c10 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05213ac836744a53b9c089a5c052f4be in term 1.
W20260709 13:11:50.429544 30766 leader_election.cc:343] T 892368b6886842febffc849d73f76c10 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:50.429757 30997 tablet_bootstrap.cc:654] T 784deee1ea9543df840c58ecaa6f911a P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:50.431243 30767 leader_election.cc:304] T 892368b6886842febffc849d73f76c10 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a
I20260709 13:11:50.432014 31109 raft_consensus.cc:2804] T 892368b6886842febffc849d73f76c10 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:50.432701 31109 raft_consensus.cc:697] T 892368b6886842febffc849d73f76c10 P 05213ac836744a53b9c089a5c052f4be [term 1 LEADER]: Becoming Leader. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Running, Role: LEADER
I20260709 13:11:50.433518 31109 consensus_queue.cc:237] T 892368b6886842febffc849d73f76c10 P 05213ac836744a53b9c089a5c052f4be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:50.436374 30995 tablet_bootstrap.cc:492] T 93a75acc8893414180a15797033a2d5a P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:50.437036 30995 ts_tablet_manager.cc:1403] T 93a75acc8893414180a15797033a2d5a P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.182s	user 0.006s	sys 0.006s
I20260709 13:11:50.441151 30995 raft_consensus.cc:359] T 93a75acc8893414180a15797033a2d5a P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:50.442828 30995 raft_consensus.cc:385] T 93a75acc8893414180a15797033a2d5a P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:50.443482 30995 raft_consensus.cc:740] T 93a75acc8893414180a15797033a2d5a P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:50.443066 30623 catalog_manager.cc:5697] T 93a75acc8893414180a15797033a2d5a P 02c5705b45c04706bbb2c6ff3d7b0b8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3). New cstate: current_term: 1 leader_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:50.451366 30997 tablet_bootstrap.cc:492] T 784deee1ea9543df840c58ecaa6f911a P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
I20260709 13:11:50.451191 30995 consensus_queue.cc:260] T 93a75acc8893414180a15797033a2d5a P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:50.451840 30997 ts_tablet_manager.cc:1403] T 784deee1ea9543df840c58ecaa6f911a P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.045s	user 0.016s	sys 0.004s
I20260709 13:11:50.454571 30997 raft_consensus.cc:359] T 784deee1ea9543df840c58ecaa6f911a P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:50.455201 30997 raft_consensus.cc:385] T 784deee1ea9543df840c58ecaa6f911a P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:50.455461 30997 raft_consensus.cc:740] T 784deee1ea9543df840c58ecaa6f911a P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
I20260709 13:11:50.456305 30994 tablet_bootstrap.cc:492] T 8155b3fc02b348e1adb290ecfe4592c5 P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
I20260709 13:11:50.456192 30997 consensus_queue.cc:260] T 784deee1ea9543df840c58ecaa6f911a P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:50.456820 30994 ts_tablet_manager.cc:1403] T 8155b3fc02b348e1adb290ecfe4592c5 P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.190s	user 0.014s	sys 0.000s
I20260709 13:11:50.458504 30997 ts_tablet_manager.cc:1434] T 784deee1ea9543df840c58ecaa6f911a P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:11:50.459450 30997 tablet_bootstrap.cc:492] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
I20260709 13:11:50.450203 30996 tablet_bootstrap.cc:654] T 892368b6886842febffc849d73f76c10 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:50.459640 30994 raft_consensus.cc:359] T 8155b3fc02b348e1adb290ecfe4592c5 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:50.460594 30995 ts_tablet_manager.cc:1434] T 93a75acc8893414180a15797033a2d5a P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 13:11:50.463809 30994 raft_consensus.cc:385] T 8155b3fc02b348e1adb290ecfe4592c5 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:50.464128 30994 raft_consensus.cc:740] T 8155b3fc02b348e1adb290ecfe4592c5 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
I20260709 13:11:50.464716 30995 tablet_bootstrap.cc:492] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:50.464923 30994 consensus_queue.cc:260] T 8155b3fc02b348e1adb290ecfe4592c5 P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:50.467216 30997 tablet_bootstrap.cc:654] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:50.467352 31109 raft_consensus.cc:493] T 8fb24a20af104a7486d0f1b2ae0e3843 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:50.468091 30994 ts_tablet_manager.cc:1434] T 8155b3fc02b348e1adb290ecfe4592c5 P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.002s
I20260709 13:11:50.467806 31109 raft_consensus.cc:515] T 8fb24a20af104a7486d0f1b2ae0e3843 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:50.469122 30994 tablet_bootstrap.cc:492] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
I20260709 13:11:50.470294 31109 leader_election.cc:290] T 8fb24a20af104a7486d0f1b2ae0e3843 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:50.471697 30725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb24a20af104a7486d0f1b2ae0e3843" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:50.472702 30948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb24a20af104a7486d0f1b2ae0e3843" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
W20260709 13:11:50.473254 30767 leader_election.cc:343] T 8fb24a20af104a7486d0f1b2ae0e3843 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:11:50.482120 30765 leader_election.cc:343] T 8fb24a20af104a7486d0f1b2ae0e3843 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:50.482820 30765 leader_election.cc:304] T 8fb24a20af104a7486d0f1b2ae0e3843 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be; no voters: 262942f67dfa4b8fa42234b18ac31ffd, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:50.483615 30995 tablet_bootstrap.cc:654] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:50.484051 31109 raft_consensus.cc:2749] T 8fb24a20af104a7486d0f1b2ae0e3843 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:50.491832 30994 tablet_bootstrap.cc:654] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:50.505218 30994 tablet_bootstrap.cc:492] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
I20260709 13:11:50.505733 30994 ts_tablet_manager.cc:1403] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.037s	user 0.005s	sys 0.010s
I20260709 13:11:50.508091 30994 raft_consensus.cc:359] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:50.509495 30997 tablet_bootstrap.cc:492] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
I20260709 13:11:50.510070 30997 ts_tablet_manager.cc:1403] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.051s	user 0.013s	sys 0.004s
I20260709 13:11:50.512914 30995 tablet_bootstrap.cc:492] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:50.513417 30995 ts_tablet_manager.cc:1403] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.049s	user 0.011s	sys 0.011s
I20260709 13:11:50.513594 30997 raft_consensus.cc:359] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:50.514654 30997 raft_consensus.cc:385] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:50.515027 30997 raft_consensus.cc:740] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
I20260709 13:11:50.516016 30995 raft_consensus.cc:359] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:50.516698 30995 raft_consensus.cc:385] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:50.516973 30995 raft_consensus.cc:740] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:50.508683 30994 raft_consensus.cc:385] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:50.517758 30995 consensus_queue.cc:260] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:50.518321 30994 raft_consensus.cc:740] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
I20260709 13:11:50.517673 30997 consensus_queue.cc:260] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:50.519861 30994 consensus_queue.cc:260] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:50.521984 30994 ts_tablet_manager.cc:1434] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:11:50.521376 30623 catalog_manager.cc:5697] T 892368b6886842febffc849d73f76c10 P 05213ac836744a53b9c089a5c052f4be reported cstate change: term changed from 0 to 1, leader changed from <none> to 05213ac836744a53b9c089a5c052f4be (127.21.63.2). New cstate: current_term: 1 leader_uuid: "05213ac836744a53b9c089a5c052f4be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:50.525938 30995 ts_tablet_manager.cc:1434] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:11:50.524245 30997 ts_tablet_manager.cc:1434] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 13:11:50.527070 30997 tablet_bootstrap.cc:492] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
I20260709 13:11:50.532002 30996 tablet_bootstrap.cc:492] T 892368b6886842febffc849d73f76c10 P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
I20260709 13:11:50.532510 30996 ts_tablet_manager.cc:1403] T 892368b6886842febffc849d73f76c10 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.126s	user 0.007s	sys 0.013s
I20260709 13:11:50.532706 30997 tablet_bootstrap.cc:654] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:50.522897 30994 tablet_bootstrap.cc:492] T af695b428b974773ba476dce0d31c5a1 P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
I20260709 13:11:50.536571 30996 raft_consensus.cc:359] T 892368b6886842febffc849d73f76c10 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:50.537415 30996 raft_consensus.cc:385] T 892368b6886842febffc849d73f76c10 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:50.537757 30996 raft_consensus.cc:740] T 892368b6886842febffc849d73f76c10 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
I20260709 13:11:50.538913 30996 consensus_queue.cc:260] T 892368b6886842febffc849d73f76c10 P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:50.541031 30994 tablet_bootstrap.cc:654] T af695b428b974773ba476dce0d31c5a1 P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:50.545292 30995 tablet_bootstrap.cc:492] T b3754a670e0b4ed99d37283bf1647bf5 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:50.552414 30995 tablet_bootstrap.cc:654] T b3754a670e0b4ed99d37283bf1647bf5 P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:50.555675 30994 tablet_bootstrap.cc:492] T af695b428b974773ba476dce0d31c5a1 P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
I20260709 13:11:50.556195 30994 ts_tablet_manager.cc:1403] T af695b428b974773ba476dce0d31c5a1 P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.033s	user 0.010s	sys 0.007s
I20260709 13:11:50.559614 30994 raft_consensus.cc:359] T af695b428b974773ba476dce0d31c5a1 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:50.560343 30994 raft_consensus.cc:385] T af695b428b974773ba476dce0d31c5a1 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:50.560645 30994 raft_consensus.cc:740] T af695b428b974773ba476dce0d31c5a1 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
I20260709 13:11:50.561671 30994 consensus_queue.cc:260] T af695b428b974773ba476dce0d31c5a1 P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:50.564538 30996 ts_tablet_manager.cc:1434] T 892368b6886842febffc849d73f76c10 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.032s	user 0.005s	sys 0.001s
I20260709 13:11:50.564936 30994 ts_tablet_manager.cc:1434] T af695b428b974773ba476dce0d31c5a1 P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.005s
I20260709 13:11:50.565712 30996 tablet_bootstrap.cc:492] T 01cceb88bca348958b59f3697979dff1 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
I20260709 13:11:50.565860 30994 tablet_bootstrap.cc:492] T c0100ec8bd1e4a45b3c0ea52281f9204 P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
I20260709 13:11:50.572741 30994 tablet_bootstrap.cc:654] T c0100ec8bd1e4a45b3c0ea52281f9204 P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:50.597523 30996 tablet_bootstrap.cc:654] T 01cceb88bca348958b59f3697979dff1 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:50.604476 30996 tablet_bootstrap.cc:492] T 01cceb88bca348958b59f3697979dff1 P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
I20260709 13:11:50.605225 30996 ts_tablet_manager.cc:1403] T 01cceb88bca348958b59f3697979dff1 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.040s	user 0.014s	sys 0.000s
I20260709 13:11:50.608541 30996 raft_consensus.cc:359] T 01cceb88bca348958b59f3697979dff1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:50.609469 30996 raft_consensus.cc:385] T 01cceb88bca348958b59f3697979dff1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:50.609825 30996 raft_consensus.cc:740] T 01cceb88bca348958b59f3697979dff1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
I20260709 13:11:50.610952 30996 consensus_queue.cc:260] T 01cceb88bca348958b59f3697979dff1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:50.611691 30997 tablet_bootstrap.cc:492] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
I20260709 13:11:50.613549 30997 ts_tablet_manager.cc:1403] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.087s	user 0.010s	sys 0.020s
I20260709 13:11:50.614532 30994 tablet_bootstrap.cc:492] T c0100ec8bd1e4a45b3c0ea52281f9204 P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
I20260709 13:11:50.615010 30994 ts_tablet_manager.cc:1403] T c0100ec8bd1e4a45b3c0ea52281f9204 P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.049s	user 0.019s	sys 0.019s
I20260709 13:11:50.615960 30997 raft_consensus.cc:359] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:50.617153 30996 ts_tablet_manager.cc:1434] T 01cceb88bca348958b59f3697979dff1 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:11:50.617383 30997 raft_consensus.cc:385] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:50.617722 30997 raft_consensus.cc:740] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
I20260709 13:11:50.617651 30994 raft_consensus.cc:359] T c0100ec8bd1e4a45b3c0ea52281f9204 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:50.618172 30996 tablet_bootstrap.cc:492] T 92a4b4a5a07442f49f518b479db8c94b P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
I20260709 13:11:50.618530 30994 raft_consensus.cc:385] T c0100ec8bd1e4a45b3c0ea52281f9204 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:50.618945 30994 raft_consensus.cc:740] T c0100ec8bd1e4a45b3c0ea52281f9204 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
I20260709 13:11:50.618569 30997 consensus_queue.cc:260] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:50.619808 30994 consensus_queue.cc:260] T c0100ec8bd1e4a45b3c0ea52281f9204 P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:50.620846 30997 ts_tablet_manager.cc:1434] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:11:50.617074 30995 tablet_bootstrap.cc:492] T b3754a670e0b4ed99d37283bf1647bf5 P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:50.621443 30995 ts_tablet_manager.cc:1403] T b3754a670e0b4ed99d37283bf1647bf5 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.076s	user 0.018s	sys 0.012s
I20260709 13:11:50.624024 30997 tablet_bootstrap.cc:492] T 8fb24a20af104a7486d0f1b2ae0e3843 P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
I20260709 13:11:50.624404 30996 tablet_bootstrap.cc:654] T 92a4b4a5a07442f49f518b479db8c94b P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:50.623963 30995 raft_consensus.cc:359] T b3754a670e0b4ed99d37283bf1647bf5 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:50.624920 30995 raft_consensus.cc:385] T b3754a670e0b4ed99d37283bf1647bf5 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:50.625233 30995 raft_consensus.cc:740] T b3754a670e0b4ed99d37283bf1647bf5 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:50.625977 30995 consensus_queue.cc:260] T b3754a670e0b4ed99d37283bf1647bf5 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:50.626070 30994 ts_tablet_manager.cc:1434] T c0100ec8bd1e4a45b3c0ea52281f9204 P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 13:11:50.630796 30994 tablet_bootstrap.cc:492] T 65776989d6674454816411c251aff5c1 P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
I20260709 13:11:50.631034 30995 ts_tablet_manager.cc:1434] T b3754a670e0b4ed99d37283bf1647bf5 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:11:50.631402 30997 tablet_bootstrap.cc:654] T 8fb24a20af104a7486d0f1b2ae0e3843 P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:50.632050 30995 tablet_bootstrap.cc:492] T 92a4b4a5a07442f49f518b479db8c94b P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:50.680231 31063 raft_consensus.cc:493] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:50.682152 31046 raft_consensus.cc:493] T 92a4b4a5a07442f49f518b479db8c94b P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:50.681877 31063 raft_consensus.cc:515] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:50.689069 31233 raft_consensus.cc:493] T 01cceb88bca348958b59f3697979dff1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:50.690135 31233 raft_consensus.cc:515] T 01cceb88bca348958b59f3697979dff1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:50.690660 31069 raft_consensus.cc:493] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:50.691009 31082 raft_consensus.cc:493] T 784deee1ea9543df840c58ecaa6f911a P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:50.691362 31069 raft_consensus.cc:515] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:50.697786 30725 raft_consensus.cc:3060] T b3754a670e0b4ed99d37283bf1647bf5 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:50.691888 31082 raft_consensus.cc:515] T 784deee1ea9543df840c58ecaa6f911a P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:50.683401 31046 raft_consensus.cc:515] T 92a4b4a5a07442f49f518b479db8c94b P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:50.698789 31063 leader_election.cc:290] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:50.708220 31082 leader_election.cc:290] T 784deee1ea9543df840c58ecaa6f911a P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:50.710521 30723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784deee1ea9543df840c58ecaa6f911a" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:50.713266 31002 raft_consensus.cc:493] T 892368b6886842febffc849d73f76c10 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:50.714286 31002 raft_consensus.cc:515] T 892368b6886842febffc849d73f76c10 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:50.718220 31069 leader_election.cc:290] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:50.699506 30724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d8c1da1f7254fde94f7deaad5f41709" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:50.716279 31046 leader_election.cc:290] T 92a4b4a5a07442f49f518b479db8c94b P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:50.720454 30724 raft_consensus.cc:2468] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 0.
I20260709 13:11:50.699486 30948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01cceb88bca348958b59f3697979dff1" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:50.729434 31002 leader_election.cc:290] T 892368b6886842febffc849d73f76c10 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
W20260709 13:11:50.730670 30915 leader_election.cc:343] T 784deee1ea9543df840c58ecaa6f911a P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:50.736049 30726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a4b4a5a07442f49f518b479db8c94b" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:50.738073 31177 consensus_queue.cc:1048] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:50.739552 30873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a4b4a5a07442f49f518b479db8c94b" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:50.740854 30723 raft_consensus.cc:3060] T 93a75acc8893414180a15797033a2d5a P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:50.741046 31046 consensus_queue.cc:1048] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:50.747922 30995 tablet_bootstrap.cc:654] T 92a4b4a5a07442f49f518b479db8c94b P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
W20260709 13:11:50.749536 30915 leader_election.cc:343] T 92a4b4a5a07442f49f518b479db8c94b P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:50.745654 30798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d8c1da1f7254fde94f7deaad5f41709" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:50.760907 30798 raft_consensus.cc:2468] T 0d8c1da1f7254fde94f7deaad5f41709 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 0.
I20260709 13:11:50.762012 30873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d8c1da1f7254fde94f7deaad5f41709" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:50.762725 30873 raft_consensus.cc:2468] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 0.
I20260709 13:11:50.766129 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "892368b6886842febffc849d73f76c10" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:50.771025 30838 leader_election.cc:304] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 05213ac836744a53b9c089a5c052f4be; no voters: 
I20260709 13:11:50.771160 30875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784deee1ea9543df840c58ecaa6f911a" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
W20260709 13:11:50.773129 30914 leader_election.cc:343] T 92a4b4a5a07442f49f518b479db8c94b P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:50.773496 31002 raft_consensus.cc:2804] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:50.773694 30914 leader_election.cc:304] T 92a4b4a5a07442f49f518b479db8c94b P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:50.774210 31002 raft_consensus.cc:493] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:50.774873 31002 raft_consensus.cc:3060] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:50.776024 30724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "892368b6886842febffc849d73f76c10" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:50.776531 31046 raft_consensus.cc:2749] T 92a4b4a5a07442f49f518b479db8c94b P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:50.777071 30724 raft_consensus.cc:2393] T 892368b6886842febffc849d73f76c10 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in current term 1: Already voted for candidate 05213ac836744a53b9c089a5c052f4be in this term.
I20260709 13:11:50.781098 30841 leader_election.cc:304] T 892368b6886842febffc849d73f76c10 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a; no voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:50.782074 31115 raft_consensus.cc:3060] T 892368b6886842febffc849d73f76c10 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:50.785972 31233 leader_election.cc:290] T 01cceb88bca348958b59f3697979dff1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:50.787410 30801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d8c1da1f7254fde94f7deaad5f41709" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:50.788098 30801 raft_consensus.cc:2468] T 0d8c1da1f7254fde94f7deaad5f41709 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 0.
I20260709 13:11:50.788185 30725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01cceb88bca348958b59f3697979dff1" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
W20260709 13:11:50.790848 30914 leader_election.cc:343] T 784deee1ea9543df840c58ecaa6f911a P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:50.791105 30841 leader_election.cc:304] T 01cceb88bca348958b59f3697979dff1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a; no voters: 262942f67dfa4b8fa42234b18ac31ffd, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:50.791528 30914 leader_election.cc:304] T 784deee1ea9543df840c58ecaa6f911a P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:50.793345 30691 leader_election.cc:304] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:50.794353 30997 tablet_bootstrap.cc:492] T 8fb24a20af104a7486d0f1b2ae0e3843 P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
I20260709 13:11:50.794714 31070 raft_consensus.cc:2804] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:50.795163 30997 ts_tablet_manager.cc:1403] T 8fb24a20af104a7486d0f1b2ae0e3843 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.171s	user 0.016s	sys 0.000s
I20260709 13:11:50.795266 31070 raft_consensus.cc:493] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:50.795948 31070 raft_consensus.cc:3060] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:50.797633 31046 raft_consensus.cc:2749] T 784deee1ea9543df840c58ecaa6f911a P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:50.806344 30997 raft_consensus.cc:359] T 8fb24a20af104a7486d0f1b2ae0e3843 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:50.807294 30997 raft_consensus.cc:385] T 8fb24a20af104a7486d0f1b2ae0e3843 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:50.807610 30997 raft_consensus.cc:740] T 8fb24a20af104a7486d0f1b2ae0e3843 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
I20260709 13:11:50.808415 30996 tablet_bootstrap.cc:492] T 92a4b4a5a07442f49f518b479db8c94b P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
I20260709 13:11:50.809614 30996 ts_tablet_manager.cc:1403] T 92a4b4a5a07442f49f518b479db8c94b P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.192s	user 0.013s	sys 0.000s
I20260709 13:11:50.818861 31089 consensus_queue.cc:1048] T b3754a670e0b4ed99d37283bf1647bf5 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:50.823280 31089 consensus_queue.cc:1048] T b3754a670e0b4ed99d37283bf1647bf5 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:50.825491 31239 consensus_queue.cc:1048] T 93a75acc8893414180a15797033a2d5a P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:50.827219 31239 raft_consensus.cc:2749] T 01cceb88bca348958b59f3697979dff1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:50.829079 31063 consensus_queue.cc:1048] T 054fc96e257a4dc6b2ef90db5c1ad499 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:50.830776 31082 raft_consensus.cc:493] T b9fcfade988746a2967e115dbbcfe0f1 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:50.831499 31082 raft_consensus.cc:515] T b9fcfade988746a2967e115dbbcfe0f1 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:50.833178 31233 consensus_queue.cc:1048] T 93a75acc8893414180a15797033a2d5a P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:50.834355 31082 leader_election.cc:290] T b9fcfade988746a2967e115dbbcfe0f1 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
W20260709 13:11:50.835669 30841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:37325, user_credentials={real_user=slave}} blocked reactor thread for 44650.2us
I20260709 13:11:50.829589 30994 tablet_bootstrap.cc:654] T 65776989d6674454816411c251aff5c1 P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:50.846398 30800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9fcfade988746a2967e115dbbcfe0f1" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:50.847440 30800 raft_consensus.cc:2468] T b9fcfade988746a2967e115dbbcfe0f1 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 0.
W20260709 13:11:50.850211 30671 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:11:50.838212 31070 raft_consensus.cc:515] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:50.808351 30997 consensus_queue.cc:260] T 8fb24a20af104a7486d0f1b2ae0e3843 P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:50.882982 30912 leader_election.cc:304] T b9fcfade988746a2967e115dbbcfe0f1 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
I20260709 13:11:50.884099 31177 raft_consensus.cc:2804] T b9fcfade988746a2967e115dbbcfe0f1 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:50.884589 31177 raft_consensus.cc:493] T b9fcfade988746a2967e115dbbcfe0f1 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:50.885005 31177 raft_consensus.cc:3060] T b9fcfade988746a2967e115dbbcfe0f1 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:50.891336 30873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9fcfade988746a2967e115dbbcfe0f1" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:50.897414 30797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d8c1da1f7254fde94f7deaad5f41709" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:50.898187 30797 raft_consensus.cc:3060] T 0d8c1da1f7254fde94f7deaad5f41709 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:50.900960 30997 ts_tablet_manager.cc:1434] T 8fb24a20af104a7486d0f1b2ae0e3843 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.105s	user 0.012s	sys 0.002s
I20260709 13:11:50.911324 31115 raft_consensus.cc:2749] T 892368b6886842febffc849d73f76c10 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:50.913590 30872 raft_consensus.cc:3060] T 01cceb88bca348958b59f3697979dff1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:50.907433 30996 raft_consensus.cc:359] T 92a4b4a5a07442f49f518b479db8c94b P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:50.916546 31233 consensus_queue.cc:1048] T 054fc96e257a4dc6b2ef90db5c1ad499 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:11:50.920485 30914 leader_election.cc:343] T b9fcfade988746a2967e115dbbcfe0f1 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:50.919040 30996 raft_consensus.cc:385] T 92a4b4a5a07442f49f518b479db8c94b P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:50.921669 30797 raft_consensus.cc:2468] T 0d8c1da1f7254fde94f7deaad5f41709 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 1.
I20260709 13:11:50.923905 30996 raft_consensus.cc:740] T 92a4b4a5a07442f49f518b479db8c94b P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
I20260709 13:11:50.927603 31109 raft_consensus.cc:493] T 8155b3fc02b348e1adb290ecfe4592c5 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:50.931258 30689 leader_election.cc:304] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:50.928305 31109 raft_consensus.cc:515] T 8155b3fc02b348e1adb290ecfe4592c5 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:50.935344 31069 raft_consensus.cc:2804] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:50.948869 30996 consensus_queue.cc:260] T 92a4b4a5a07442f49f518b479db8c94b P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:50.963546 30995 tablet_bootstrap.cc:492] T 92a4b4a5a07442f49f518b479db8c94b P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:50.964401 30995 ts_tablet_manager.cc:1403] T 92a4b4a5a07442f49f518b479db8c94b P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.333s	user 0.017s	sys 0.005s
I20260709 13:11:50.965969 31002 raft_consensus.cc:515] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:50.968320 31002 leader_election.cc:290] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Requested vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:50.974011 30875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d8c1da1f7254fde94f7deaad5f41709" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:11:50.974946 30875 raft_consensus.cc:2393] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d22c9d3d15904327a78a0e548a8ff6cd in current term 1: Already voted for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in this term.
I20260709 13:11:50.977221 30726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8155b3fc02b348e1adb290ecfe4592c5" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
W20260709 13:11:50.979069 30767 leader_election.cc:343] T 8155b3fc02b348e1adb290ecfe4592c5 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:50.979584 30996 ts_tablet_manager.cc:1434] T 92a4b4a5a07442f49f518b479db8c94b P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.170s	user 0.008s	sys 0.004s
I20260709 13:11:50.970954 30796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d8c1da1f7254fde94f7deaad5f41709" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:50.971628 31070 leader_election.cc:290] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 election: Requested vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:50.981150 30796 raft_consensus.cc:2393] T 0d8c1da1f7254fde94f7deaad5f41709 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in current term 1: Already voted for candidate d22c9d3d15904327a78a0e548a8ff6cd in this term.
I20260709 13:11:50.982829 31046 raft_consensus.cc:493] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:50.983393 31046 raft_consensus.cc:515] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:50.977221 31109 leader_election.cc:290] T 8155b3fc02b348e1adb290ecfe4592c5 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:50.996193 30874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8155b3fc02b348e1adb290ecfe4592c5" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:50.997838 30723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d8c1da1f7254fde94f7deaad5f41709" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:51.018579 31195 raft_consensus.cc:493] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:51.033514 30995 raft_consensus.cc:359] T 92a4b4a5a07442f49f518b479db8c94b P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:51.034358 30995 raft_consensus.cc:385] T 92a4b4a5a07442f49f518b479db8c94b P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:51.035209 30995 raft_consensus.cc:740] T 92a4b4a5a07442f49f518b479db8c94b P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:51.039242 31230 raft_consensus.cc:493] T af695b428b974773ba476dce0d31c5a1 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:51.039877 31230 raft_consensus.cc:515] T af695b428b974773ba476dce0d31c5a1 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:51.042575 30997 tablet_bootstrap.cc:492] T ac85f05e63ed463681b3ec790b39c4d8 P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
W20260709 13:11:51.051950 30766 leader_election.cc:343] T 8155b3fc02b348e1adb290ecfe4592c5 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:51.053828 30724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af695b428b974773ba476dce0d31c5a1" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:51.052461 30766 leader_election.cc:304] T 8155b3fc02b348e1adb290ecfe4592c5 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:51.042150 31230 leader_election.cc:290] T af695b428b974773ba476dce0d31c5a1 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:51.058557 30724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0100ec8bd1e4a45b3c0ea52281f9204" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
W20260709 13:11:51.060045 30915 leader_election.cc:343] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:51.061204 30996 tablet_bootstrap.cc:492] T 5532e197e7de48f28c034ba1fc490e19 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
I20260709 13:11:51.064405 31046 leader_election.cc:290] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:51.067207 30995 consensus_queue.cc:260] T 92a4b4a5a07442f49f518b479db8c94b P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:51.070024 30995 ts_tablet_manager.cc:1434] T 92a4b4a5a07442f49f518b479db8c94b P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.081s	user 0.006s	sys 0.000s
I20260709 13:11:51.071426 30995 tablet_bootstrap.cc:492] T 5532e197e7de48f28c034ba1fc490e19 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:51.075611 31178 raft_consensus.cc:493] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:51.076324 31178 raft_consensus.cc:515] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:51.079262 31178 leader_election.cc:290] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:51.082427 30873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8c0b2b0b11472ca76e15fd3ce2e6f1" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
W20260709 13:11:51.084333 30914 leader_election.cc:343] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:51.049932 31195 raft_consensus.cc:515] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:51.087337 31069 raft_consensus.cc:697] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 LEADER]: Becoming Leader. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Running, Role: LEADER
I20260709 13:11:51.104437 31069 consensus_queue.cc:237] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:51.105367 31178 raft_consensus.cc:493] T 8fb24a20af104a7486d0f1b2ae0e3843 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:51.118011 31178 raft_consensus.cc:515] T 8fb24a20af104a7486d0f1b2ae0e3843 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:51.159775 30994 tablet_bootstrap.cc:492] T 65776989d6674454816411c251aff5c1 P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
I20260709 13:11:51.142288 31177 raft_consensus.cc:515] T b9fcfade988746a2967e115dbbcfe0f1 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:51.175860 31178 leader_election.cc:290] T 8fb24a20af104a7486d0f1b2ae0e3843 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:51.176908 31283 raft_consensus.cc:493] T c0100ec8bd1e4a45b3c0ea52281f9204 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:51.177512 31230 raft_consensus.cc:2749] T 8155b3fc02b348e1adb290ecfe4592c5 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:51.177680 31283 raft_consensus.cc:515] T c0100ec8bd1e4a45b3c0ea52281f9204 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
W20260709 13:11:51.178831 30767 leader_election.cc:343] T af695b428b974773ba476dce0d31c5a1 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:51.182185 31195 leader_election.cc:290] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:51.182540 31230 consensus_queue.cc:1048] T 892368b6886842febffc849d73f76c10 P 05213ac836744a53b9c089a5c052f4be [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:51.186754 31283 leader_election.cc:290] T c0100ec8bd1e4a45b3c0ea52281f9204 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:51.188279 30996 tablet_bootstrap.cc:654] T 5532e197e7de48f28c034ba1fc490e19 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:51.189711 31177 leader_election.cc:290] T b9fcfade988746a2967e115dbbcfe0f1 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Requested vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:51.192498 30724 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0100ec8bd1e4a45b3c0ea52281f9204" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:51.192762 30994 ts_tablet_manager.cc:1403] T 65776989d6674454816411c251aff5c1 P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.562s	user 0.011s	sys 0.023s
I20260709 13:11:51.193312 30721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb24a20af104a7486d0f1b2ae0e3843" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
W20260709 13:11:51.194787 30767 leader_election.cc:343] T c0100ec8bd1e4a45b3c0ea52281f9204 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:11:51.196020 30915 leader_election.cc:343] T 8fb24a20af104a7486d0f1b2ae0e3843 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:51.196434 30994 raft_consensus.cc:359] T 65776989d6674454816411c251aff5c1 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:51.197510 30994 raft_consensus.cc:385] T 65776989d6674454816411c251aff5c1 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:51.197870 30994 raft_consensus.cc:740] T 65776989d6674454816411c251aff5c1 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
I20260709 13:11:51.198727 30994 consensus_queue.cc:260] T 65776989d6674454816411c251aff5c1 P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:51.206665 30995 tablet_bootstrap.cc:654] T 5532e197e7de48f28c034ba1fc490e19 P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:51.210994 30997 tablet_bootstrap.cc:654] T ac85f05e63ed463681b3ec790b39c4d8 P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:51.223008 30949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8c0b2b0b11472ca76e15fd3ce2e6f1" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:51.223210 30947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0100ec8bd1e4a45b3c0ea52281f9204" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:51.223723 30949 raft_consensus.cc:2468] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05213ac836744a53b9c089a5c052f4be in term 0.
I20260709 13:11:51.223987 30947 raft_consensus.cc:2468] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05213ac836744a53b9c089a5c052f4be in term 0.
I20260709 13:11:51.240247 31066 raft_consensus.cc:493] T 92a4b4a5a07442f49f518b479db8c94b P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:51.240449 30873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9fcfade988746a2967e115dbbcfe0f1" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:11:51.240943 31066 raft_consensus.cc:515] T 92a4b4a5a07442f49f518b479db8c94b P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
W20260709 13:11:51.244833 30914 leader_election.cc:343] T b9fcfade988746a2967e115dbbcfe0f1 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:51.246255 30765 leader_election.cc:304] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
I20260709 13:11:51.247618 30765 leader_election.cc:304] T c0100ec8bd1e4a45b3c0ea52281f9204 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:51.249374 31283 raft_consensus.cc:2804] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:51.249958 31283 raft_consensus.cc:493] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:51.250385 31283 raft_consensus.cc:3060] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:51.251955 30996 tablet_bootstrap.cc:492] T 5532e197e7de48f28c034ba1fc490e19 P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
I20260709 13:11:51.252542 30996 ts_tablet_manager.cc:1403] T 5532e197e7de48f28c034ba1fc490e19 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.216s	user 0.009s	sys 0.005s
I20260709 13:11:51.255913 30996 raft_consensus.cc:359] T 5532e197e7de48f28c034ba1fc490e19 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:51.256633 30996 raft_consensus.cc:385] T 5532e197e7de48f28c034ba1fc490e19 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:51.256953 30996 raft_consensus.cc:740] T 5532e197e7de48f28c034ba1fc490e19 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
I20260709 13:11:51.256901 30797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8c0b2b0b11472ca76e15fd3ce2e6f1" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:51.258090 30996 consensus_queue.cc:260] T 5532e197e7de48f28c034ba1fc490e19 P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:51.260320 30996 ts_tablet_manager.cc:1434] T 5532e197e7de48f28c034ba1fc490e19 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.004s
I20260709 13:11:51.261531 30996 tablet_bootstrap.cc:492] T 8155b3fc02b348e1adb290ecfe4592c5 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
I20260709 13:11:51.262660 30801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0100ec8bd1e4a45b3c0ea52281f9204" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:51.263516 30801 raft_consensus.cc:2468] T c0100ec8bd1e4a45b3c0ea52281f9204 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 0.
I20260709 13:11:51.268626 30996 tablet_bootstrap.cc:654] T 8155b3fc02b348e1adb290ecfe4592c5 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:51.275573 30841 leader_election.cc:304] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a; no voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:51.276887 31002 raft_consensus.cc:2749] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:11:51.243389 31066 leader_election.cc:290] T 92a4b4a5a07442f49f518b479db8c94b P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:51.294781 31239 raft_consensus.cc:493] T 92a4b4a5a07442f49f518b479db8c94b P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:51.269217 31286 raft_consensus.cc:2804] T c0100ec8bd1e4a45b3c0ea52281f9204 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:51.307088 31286 raft_consensus.cc:493] T c0100ec8bd1e4a45b3c0ea52281f9204 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:51.311002 31286 raft_consensus.cc:3060] T c0100ec8bd1e4a45b3c0ea52281f9204 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:51.314067 31283 raft_consensus.cc:515] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:51.315637 30797 raft_consensus.cc:2393] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 262942f67dfa4b8fa42234b18ac31ffd in current term 1: Already voted for candidate 05213ac836744a53b9c089a5c052f4be in this term.
I20260709 13:11:51.319612 30949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8c0b2b0b11472ca76e15fd3ce2e6f1" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:11:51.320417 30949 raft_consensus.cc:3060] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:51.322402 31283 leader_election.cc:290] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 election: Requested vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:51.330353 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb24a20af104a7486d0f1b2ae0e3843" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:51.331153 30799 raft_consensus.cc:2468] T 8fb24a20af104a7486d0f1b2ae0e3843 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 0.
I20260709 13:11:51.309360 31239 raft_consensus.cc:515] T 92a4b4a5a07442f49f518b479db8c94b P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:51.331547 30995 tablet_bootstrap.cc:492] T 5532e197e7de48f28c034ba1fc490e19 P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:51.332901 30995 ts_tablet_manager.cc:1403] T 5532e197e7de48f28c034ba1fc490e19 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.262s	user 0.014s	sys 0.000s
I20260709 13:11:51.334797 30994 ts_tablet_manager.cc:1434] T 65776989d6674454816411c251aff5c1 P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.142s	user 0.007s	sys 0.002s
I20260709 13:11:51.336506 30997 tablet_bootstrap.cc:492] T ac85f05e63ed463681b3ec790b39c4d8 P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
I20260709 13:11:51.337179 30997 ts_tablet_manager.cc:1403] T ac85f05e63ed463681b3ec790b39c4d8 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.326s	user 0.012s	sys 0.022s
I20260709 13:11:51.338266 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9fcfade988746a2967e115dbbcfe0f1" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:51.338894 30799 raft_consensus.cc:3060] T b9fcfade988746a2967e115dbbcfe0f1 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:51.341143 30997 raft_consensus.cc:359] T ac85f05e63ed463681b3ec790b39c4d8 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:51.343930 30997 raft_consensus.cc:385] T ac85f05e63ed463681b3ec790b39c4d8 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:51.344448 30997 raft_consensus.cc:740] T ac85f05e63ed463681b3ec790b39c4d8 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
I20260709 13:11:51.345476 30997 consensus_queue.cc:260] T ac85f05e63ed463681b3ec790b39c4d8 P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:51.344048 31286 raft_consensus.cc:515] T c0100ec8bd1e4a45b3c0ea52281f9204 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:51.348755 30912 leader_election.cc:304] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:51.351270 31177 raft_consensus.cc:2804] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 13:11:51.351583 30766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:43993, user_credentials={real_user=slave}} blocked reactor thread for 326274us
I20260709 13:11:51.352741 30912 leader_election.cc:304] T 8fb24a20af104a7486d0f1b2ae0e3843 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:51.353506 30947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0100ec8bd1e4a45b3c0ea52281f9204" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:11:51.351693 31177 raft_consensus.cc:493] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:51.355907 31177 raft_consensus.cc:3060] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:51.357358 30945 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a4b4a5a07442f49f518b479db8c94b" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:51.358121 30945 raft_consensus.cc:2468] T 92a4b4a5a07442f49f518b479db8c94b P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 0.
I20260709 13:11:51.360745 31286 leader_election.cc:290] T c0100ec8bd1e4a45b3c0ea52281f9204 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 election: Requested vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:51.362066 30722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0100ec8bd1e4a45b3c0ea52281f9204" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:51.362968 31288 raft_consensus.cc:2804] T 8fb24a20af104a7486d0f1b2ae0e3843 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:51.363412 31288 raft_consensus.cc:493] T 8fb24a20af104a7486d0f1b2ae0e3843 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:51.363328 30690 leader_election.cc:304] T 92a4b4a5a07442f49f518b479db8c94b P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:51.363976 31288 raft_consensus.cc:3060] T 8fb24a20af104a7486d0f1b2ae0e3843 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:51.364627 31070 raft_consensus.cc:2804] T 92a4b4a5a07442f49f518b479db8c94b P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:51.364202 31177 raft_consensus.cc:515] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:51.350414 30997 ts_tablet_manager.cc:1434] T ac85f05e63ed463681b3ec790b39c4d8 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:11:51.365115 31070 raft_consensus.cc:493] T 92a4b4a5a07442f49f518b479db8c94b P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:51.365538 31070 raft_consensus.cc:3060] T 92a4b4a5a07442f49f518b479db8c94b P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:51.366091 30997 tablet_bootstrap.cc:492] T 65776989d6674454816411c251aff5c1 P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
W20260709 13:11:51.363911 30767 leader_election.cc:343] T c0100ec8bd1e4a45b3c0ea52281f9204 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:51.341984 30875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a4b4a5a07442f49f518b479db8c94b" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:51.370363 30949 raft_consensus.cc:2468] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05213ac836744a53b9c089a5c052f4be in term 1.
I20260709 13:11:51.372872 30996 tablet_bootstrap.cc:492] T 8155b3fc02b348e1adb290ecfe4592c5 P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
I20260709 13:11:51.373430 30996 ts_tablet_manager.cc:1403] T 8155b3fc02b348e1adb290ecfe4592c5 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.112s	user 0.020s	sys 0.013s
I20260709 13:11:51.373764 30799 raft_consensus.cc:2468] T b9fcfade988746a2967e115dbbcfe0f1 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 1.
I20260709 13:11:51.374172 30874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af695b428b974773ba476dce0d31c5a1" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:51.375800 30875 raft_consensus.cc:2468] T 92a4b4a5a07442f49f518b479db8c94b P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 0.
I20260709 13:11:51.376284 30996 raft_consensus.cc:359] T 8155b3fc02b348e1adb290ecfe4592c5 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:51.376976 30996 raft_consensus.cc:385] T 8155b3fc02b348e1adb290ecfe4592c5 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:51.377267 30996 raft_consensus.cc:740] T 8155b3fc02b348e1adb290ecfe4592c5 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
I20260709 13:11:51.378003 30996 consensus_queue.cc:260] T 8155b3fc02b348e1adb290ecfe4592c5 P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:51.380563 30996 ts_tablet_manager.cc:1434] T 8155b3fc02b348e1adb290ecfe4592c5 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:11:51.381965 30996 tablet_bootstrap.cc:492] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
I20260709 13:11:51.385659 30765 leader_election.cc:304] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
I20260709 13:11:51.386911 30947 raft_consensus.cc:2393] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 05213ac836744a53b9c089a5c052f4be in current term 1: Already voted for candidate 262942f67dfa4b8fa42234b18ac31ffd in this term.
I20260709 13:11:51.392233 31286 raft_consensus.cc:2804] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:51.392863 31286 raft_consensus.cc:697] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 05213ac836744a53b9c089a5c052f4be [term 1 LEADER]: Becoming Leader. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Running, Role: LEADER
I20260709 13:11:51.393823 31286 consensus_queue.cc:237] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 05213ac836744a53b9c089a5c052f4be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:51.395848 30875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8c0b2b0b11472ca76e15fd3ce2e6f1" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:51.396795 31285 consensus_queue.cc:1048] T 892368b6886842febffc849d73f76c10 P 05213ac836744a53b9c089a5c052f4be [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:51.397107 30765 leader_election.cc:304] T c0100ec8bd1e4a45b3c0ea52281f9204 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be; no voters: 262942f67dfa4b8fa42234b18ac31ffd, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:51.399323 30871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8c0b2b0b11472ca76e15fd3ce2e6f1" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:11:51.399981 31230 raft_consensus.cc:2749] T c0100ec8bd1e4a45b3c0ea52281f9204 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 13:11:51.400846 30766 leader_election.cc:343] T af695b428b974773ba476dce0d31c5a1 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:51.401584 30766 leader_election.cc:304] T af695b428b974773ba476dce0d31c5a1 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:51.404348 31288 raft_consensus.cc:515] T 8fb24a20af104a7486d0f1b2ae0e3843 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:51.404924 31109 raft_consensus.cc:2749] T af695b428b974773ba476dce0d31c5a1 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:51.407200 30997 tablet_bootstrap.cc:654] T 65776989d6674454816411c251aff5c1 P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:51.410214 31177 leader_election.cc:290] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Requested vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:51.411423 30994 tablet_bootstrap.cc:492] T f8f19b152a0f46ac896478f3f05c8822 P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
I20260709 13:11:51.411247 30722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0100ec8bd1e4a45b3c0ea52281f9204" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:51.408092 31070 raft_consensus.cc:515] T 92a4b4a5a07442f49f518b479db8c94b P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
W20260709 13:11:51.412611 30915 leader_election.cc:343] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:51.413913 31288 leader_election.cc:290] T 8fb24a20af104a7486d0f1b2ae0e3843 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Requested vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:51.416397 30722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb24a20af104a7486d0f1b2ae0e3843" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
W20260709 13:11:51.422230 30766 leader_election.cc:343] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:11:51.423230 30915 leader_election.cc:343] T 8fb24a20af104a7486d0f1b2ae0e3843 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:11:51.423540 30766 leader_election.cc:343] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:51.426875 31239 leader_election.cc:290] T 92a4b4a5a07442f49f518b479db8c94b P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:51.428014 30722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a4b4a5a07442f49f518b479db8c94b" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:51.437652 30722 raft_consensus.cc:2393] T 92a4b4a5a07442f49f518b479db8c94b P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in current term 1: Already voted for candidate d22c9d3d15904327a78a0e548a8ff6cd in this term.
I20260709 13:11:51.440457 30947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a4b4a5a07442f49f518b479db8c94b" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:51.441516 30947 raft_consensus.cc:2468] T 92a4b4a5a07442f49f518b479db8c94b P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 0.
I20260709 13:11:51.445729 30839 leader_election.cc:304] T 92a4b4a5a07442f49f518b479db8c94b P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 262942f67dfa4b8fa42234b18ac31ffd; no voters: d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:51.447955 31002 raft_consensus.cc:2804] T 92a4b4a5a07442f49f518b479db8c94b P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:51.448576 31002 raft_consensus.cc:493] T 92a4b4a5a07442f49f518b479db8c94b P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:51.449112 31002 raft_consensus.cc:3060] T 92a4b4a5a07442f49f518b479db8c94b P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:11:51.437616 31295 log.cc:927] Time spent T 892368b6886842febffc849d73f76c10 P d22c9d3d15904327a78a0e548a8ff6cd: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 13:11:51.467303 30996 tablet_bootstrap.cc:654] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:51.472252 30801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8fb24a20af104a7486d0f1b2ae0e3843" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:51.472899 30801 raft_consensus.cc:3060] T 8fb24a20af104a7486d0f1b2ae0e3843 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:51.477455 30997 tablet_bootstrap.cc:492] T 65776989d6674454816411c251aff5c1 P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
I20260709 13:11:51.477967 30997 ts_tablet_manager.cc:1403] T 65776989d6674454816411c251aff5c1 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.112s	user 0.016s	sys 0.008s
I20260709 13:11:51.478291 30994 tablet_bootstrap.cc:654] T f8f19b152a0f46ac896478f3f05c8822 P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:51.481019 30797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0100ec8bd1e4a45b3c0ea52281f9204" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:51.481823 30797 raft_consensus.cc:2393] T c0100ec8bd1e4a45b3c0ea52281f9204 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 262942f67dfa4b8fa42234b18ac31ffd in current term 1: Already voted for candidate 05213ac836744a53b9c089a5c052f4be in this term.
I20260709 13:11:51.484942 30997 raft_consensus.cc:359] T 65776989d6674454816411c251aff5c1 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:51.485663 30997 raft_consensus.cc:385] T 65776989d6674454816411c251aff5c1 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:51.485198 30622 catalog_manager.cc:5697] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 05213ac836744a53b9c089a5c052f4be reported cstate change: term changed from 0 to 1, leader changed from <none> to 05213ac836744a53b9c089a5c052f4be (127.21.63.2). New cstate: current_term: 1 leader_uuid: "05213ac836744a53b9c089a5c052f4be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:51.487787 30996 tablet_bootstrap.cc:492] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
I20260709 13:11:51.488319 30996 ts_tablet_manager.cc:1403] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.107s	user 0.011s	sys 0.001s
I20260709 13:11:51.485963 30997 raft_consensus.cc:740] T 65776989d6674454816411c251aff5c1 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
I20260709 13:11:51.489342 30997 consensus_queue.cc:260] T 65776989d6674454816411c251aff5c1 P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:51.489666 30912 leader_election.cc:304] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 05213ac836744a53b9c089a5c052f4be
I20260709 13:11:51.490667 31288 raft_consensus.cc:2749] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:51.490985 30997 ts_tablet_manager.cc:1434] T 65776989d6674454816411c251aff5c1 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.001s
I20260709 13:11:51.492080 30997 tablet_bootstrap.cc:492] T 8b6052cddae64c5c9d16f60272735788 P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
I20260709 13:11:51.493631 30912 leader_election.cc:304] T b9fcfade988746a2967e115dbbcfe0f1 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a
I20260709 13:11:51.494431 31288 raft_consensus.cc:2804] T b9fcfade988746a2967e115dbbcfe0f1 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:51.494900 31288 raft_consensus.cc:697] T b9fcfade988746a2967e115dbbcfe0f1 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Becoming Leader. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Running, Role: LEADER
I20260709 13:11:51.495850 31288 consensus_queue.cc:237] T b9fcfade988746a2967e115dbbcfe0f1 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:51.501358 30801 raft_consensus.cc:2468] T 8fb24a20af104a7486d0f1b2ae0e3843 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 1.
I20260709 13:11:51.504886 30997 tablet_bootstrap.cc:654] T 8b6052cddae64c5c9d16f60272735788 P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:51.517409 31002 raft_consensus.cc:515] T 92a4b4a5a07442f49f518b479db8c94b P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:51.519349 30995 raft_consensus.cc:359] T 5532e197e7de48f28c034ba1fc490e19 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:51.520128 30995 raft_consensus.cc:385] T 5532e197e7de48f28c034ba1fc490e19 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:51.520479 30995 raft_consensus.cc:740] T 5532e197e7de48f28c034ba1fc490e19 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:51.524995 30871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a4b4a5a07442f49f518b479db8c94b" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:11:51.525226 30722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a4b4a5a07442f49f518b479db8c94b" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:51.525800 30871 raft_consensus.cc:2393] T 92a4b4a5a07442f49f518b479db8c94b P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d22c9d3d15904327a78a0e548a8ff6cd in current term 1: Already voted for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in this term.
I20260709 13:11:51.526006 30722 raft_consensus.cc:2393] T 92a4b4a5a07442f49f518b479db8c94b P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in current term 1: Already voted for candidate d22c9d3d15904327a78a0e548a8ff6cd in this term.
I20260709 13:11:51.526624 31070 leader_election.cc:290] T 92a4b4a5a07442f49f518b479db8c94b P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 election: Requested vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:51.521464 31002 leader_election.cc:290] T 92a4b4a5a07442f49f518b479db8c94b P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Requested vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:51.530558 30912 leader_election.cc:304] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd; no voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:51.531782 31177 raft_consensus.cc:2749] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:11:51.535379 30949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a4b4a5a07442f49f518b479db8c94b" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:11:51.535969 30949 raft_consensus.cc:3060] T 92a4b4a5a07442f49f518b479db8c94b P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:51.523104 30996 raft_consensus.cc:359] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:51.536649 30996 raft_consensus.cc:385] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:51.536983 30996 raft_consensus.cc:740] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
I20260709 13:11:51.538333 30996 consensus_queue.cc:260] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:51.530543 30623 catalog_manager.cc:5697] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd reported cstate change: term changed from 0 to 1, leader changed from <none> to d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1). New cstate: current_term: 1 leader_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:51.538228 30995 consensus_queue.cc:260] T 5532e197e7de48f28c034ba1fc490e19 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:51.546967 31283 raft_consensus.cc:493] T 65776989d6674454816411c251aff5c1 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:51.548069 31283 raft_consensus.cc:515] T 65776989d6674454816411c251aff5c1 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:51.550132 30912 leader_election.cc:304] T 8fb24a20af104a7486d0f1b2ae0e3843 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:51.550948 31177 raft_consensus.cc:2804] T 8fb24a20af104a7486d0f1b2ae0e3843 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:51.551395 31177 raft_consensus.cc:697] T 8fb24a20af104a7486d0f1b2ae0e3843 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Becoming Leader. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Running, Role: LEADER
I20260709 13:11:51.552132 31177 consensus_queue.cc:237] T 8fb24a20af104a7486d0f1b2ae0e3843 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:51.553149 31283 leader_election.cc:290] T 65776989d6674454816411c251aff5c1 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:51.553382 30947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65776989d6674454816411c251aff5c1" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:51.554119 30947 raft_consensus.cc:2468] T 65776989d6674454816411c251aff5c1 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05213ac836744a53b9c089a5c052f4be in term 0.
I20260709 13:11:51.555909 30949 raft_consensus.cc:2468] T 92a4b4a5a07442f49f518b479db8c94b P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 1.
I20260709 13:11:51.558296 30765 leader_election.cc:304] T 65776989d6674454816411c251aff5c1 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
I20260709 13:11:51.559701 30839 leader_election.cc:304] T 92a4b4a5a07442f49f518b479db8c94b P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 262942f67dfa4b8fa42234b18ac31ffd; no voters: d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:51.560868 31239 raft_consensus.cc:2804] T 92a4b4a5a07442f49f518b479db8c94b P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:51.561540 31239 raft_consensus.cc:697] T 92a4b4a5a07442f49f518b479db8c94b P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Becoming Leader. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Running, Role: LEADER
I20260709 13:11:51.562256 30949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "92a4b4a5a07442f49f518b479db8c94b" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:11:51.562834 30949 raft_consensus.cc:2393] T 92a4b4a5a07442f49f518b479db8c94b P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d22c9d3d15904327a78a0e548a8ff6cd in current term 1: Already voted for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in this term.
I20260709 13:11:51.566259 30996 ts_tablet_manager.cc:1434] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.078s	user 0.010s	sys 0.000s
I20260709 13:11:51.560880 31283 raft_consensus.cc:2804] T 65776989d6674454816411c251aff5c1 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:51.570395 30871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65776989d6674454816411c251aff5c1" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:51.565984 30690 leader_election.cc:304] T 92a4b4a5a07442f49f518b479db8c94b P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22c9d3d15904327a78a0e548a8ff6cd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 262942f67dfa4b8fa42234b18ac31ffd
I20260709 13:11:51.572448 31240 raft_consensus.cc:493] T 8155b3fc02b348e1adb290ecfe4592c5 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:51.573307 31279 raft_consensus.cc:2749] T 92a4b4a5a07442f49f518b479db8c94b P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:11:51.572978 31240 raft_consensus.cc:515] T 8155b3fc02b348e1adb290ecfe4592c5 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:51.574149 30994 tablet_bootstrap.cc:492] T f8f19b152a0f46ac896478f3f05c8822 P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
I20260709 13:11:51.574929 30994 ts_tablet_manager.cc:1403] T f8f19b152a0f46ac896478f3f05c8822 P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.164s	user 0.009s	sys 0.005s
I20260709 13:11:51.574250 31239 consensus_queue.cc:237] T 92a4b4a5a07442f49f518b479db8c94b P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:51.569245 31283 raft_consensus.cc:493] T 65776989d6674454816411c251aff5c1 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:51.575971 31283 raft_consensus.cc:3060] T 65776989d6674454816411c251aff5c1 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:51.577497 31240 leader_election.cc:290] T 8155b3fc02b348e1adb290ecfe4592c5 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
W20260709 13:11:51.576411 30766 leader_election.cc:343] T 65776989d6674454816411c251aff5c1 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:51.580344 30722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8155b3fc02b348e1adb290ecfe4592c5" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
W20260709 13:11:51.582558 30841 leader_election.cc:343] T 8155b3fc02b348e1adb290ecfe4592c5 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:51.585464 30801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8155b3fc02b348e1adb290ecfe4592c5" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:51.586179 30801 raft_consensus.cc:2468] T 8155b3fc02b348e1adb290ecfe4592c5 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 0.
I20260709 13:11:51.587451 30838 leader_election.cc:304] T 8155b3fc02b348e1adb290ecfe4592c5 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 05213ac836744a53b9c089a5c052f4be; no voters: d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:51.586198 30994 raft_consensus.cc:359] T f8f19b152a0f46ac896478f3f05c8822 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:51.589861 31240 raft_consensus.cc:2804] T 8155b3fc02b348e1adb290ecfe4592c5 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:51.590176 30994 raft_consensus.cc:385] T f8f19b152a0f46ac896478f3f05c8822 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:51.590332 31240 raft_consensus.cc:493] T 8155b3fc02b348e1adb290ecfe4592c5 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:51.590579 30994 raft_consensus.cc:740] T f8f19b152a0f46ac896478f3f05c8822 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
I20260709 13:11:51.590795 31240 raft_consensus.cc:3060] T 8155b3fc02b348e1adb290ecfe4592c5 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:51.590965 30997 tablet_bootstrap.cc:492] T 8b6052cddae64c5c9d16f60272735788 P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
I20260709 13:11:51.591693 30997 ts_tablet_manager.cc:1403] T 8b6052cddae64c5c9d16f60272735788 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.100s	user 0.014s	sys 0.015s
I20260709 13:11:51.591536 30994 consensus_queue.cc:260] T f8f19b152a0f46ac896478f3f05c8822 P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:51.599054 30995 ts_tablet_manager.cc:1434] T 5532e197e7de48f28c034ba1fc490e19 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.266s	user 0.008s	sys 0.000s
I20260709 13:11:51.599227 30997 raft_consensus.cc:359] T 8b6052cddae64c5c9d16f60272735788 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:51.600198 30997 raft_consensus.cc:385] T 8b6052cddae64c5c9d16f60272735788 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:51.600359 30995 tablet_bootstrap.cc:492] T 8155b3fc02b348e1adb290ecfe4592c5 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:51.600607 30997 raft_consensus.cc:740] T 8b6052cddae64c5c9d16f60272735788 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
I20260709 13:11:51.601737 30996 tablet_bootstrap.cc:492] T b9fcfade988746a2967e115dbbcfe0f1 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
I20260709 13:11:51.601605 30997 consensus_queue.cc:260] T 8b6052cddae64c5c9d16f60272735788 P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:51.604274 30997 ts_tablet_manager.cc:1434] T 8b6052cddae64c5c9d16f60272735788 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.012s	user 0.003s	sys 0.004s
I20260709 13:11:51.605641 30997 tablet_bootstrap.cc:492] T ea563bd67a9849d18c9c3dfb21d58a93 P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
I20260709 13:11:51.617671 31283 raft_consensus.cc:515] T 65776989d6674454816411c251aff5c1 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:51.621220 30874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65776989d6674454816411c251aff5c1" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
W20260709 13:11:51.622779 30766 leader_election.cc:343] T 65776989d6674454816411c251aff5c1 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:51.623942 30994 ts_tablet_manager.cc:1434] T f8f19b152a0f46ac896478f3f05c8822 P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.040s	user 0.006s	sys 0.000s
I20260709 13:11:51.623445 30624 catalog_manager.cc:5697] T b9fcfade988746a2967e115dbbcfe0f1 P 262942f67dfa4b8fa42234b18ac31ffd reported cstate change: term changed from 0 to 1, leader changed from <none> to 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4). New cstate: current_term: 1 leader_uuid: "262942f67dfa4b8fa42234b18ac31ffd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:51.625448 30994 tablet_bootstrap.cc:492] T 8b6052cddae64c5c9d16f60272735788 P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
I20260709 13:11:51.631210 30996 tablet_bootstrap.cc:654] T b9fcfade988746a2967e115dbbcfe0f1 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:51.635262 31240 raft_consensus.cc:515] T 8155b3fc02b348e1adb290ecfe4592c5 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:51.639375 30949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65776989d6674454816411c251aff5c1" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:11:51.640069 30949 raft_consensus.cc:3060] T 65776989d6674454816411c251aff5c1 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:51.642088 30722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8155b3fc02b348e1adb290ecfe4592c5" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
W20260709 13:11:51.643581 30841 leader_election.cc:343] T 8155b3fc02b348e1adb290ecfe4592c5 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:51.644193 30997 tablet_bootstrap.cc:654] T ea563bd67a9849d18c9c3dfb21d58a93 P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:51.644491 30801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8155b3fc02b348e1adb290ecfe4592c5" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:51.645135 30801 raft_consensus.cc:3060] T 8155b3fc02b348e1adb290ecfe4592c5 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:51.648309 31240 leader_election.cc:290] T 8155b3fc02b348e1adb290ecfe4592c5 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Requested vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:51.649245 30995 tablet_bootstrap.cc:654] T 8155b3fc02b348e1adb290ecfe4592c5 P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:51.649037 31279 consensus_queue.cc:1048] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:51.650808 30949 raft_consensus.cc:2468] T 65776989d6674454816411c251aff5c1 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05213ac836744a53b9c089a5c052f4be in term 1.
I20260709 13:11:51.651880 30765 leader_election.cc:304] T 65776989d6674454816411c251aff5c1 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a
I20260709 13:11:51.652746 31286 raft_consensus.cc:2804] T 65776989d6674454816411c251aff5c1 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:51.653271 31286 raft_consensus.cc:697] T 65776989d6674454816411c251aff5c1 P 05213ac836744a53b9c089a5c052f4be [term 1 LEADER]: Becoming Leader. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Running, Role: LEADER
I20260709 13:11:51.652741 30801 raft_consensus.cc:2468] T 8155b3fc02b348e1adb290ecfe4592c5 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 1.
I20260709 13:11:51.654136 31286 consensus_queue.cc:237] T 65776989d6674454816411c251aff5c1 P 05213ac836744a53b9c089a5c052f4be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:51.655077 30838 leader_election.cc:304] T 8155b3fc02b348e1adb290ecfe4592c5 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 05213ac836744a53b9c089a5c052f4be; no voters: d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:51.657152 31002 raft_consensus.cc:2804] T 8155b3fc02b348e1adb290ecfe4592c5 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:51.660787 31283 leader_election.cc:290] T 65776989d6674454816411c251aff5c1 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 election: Requested vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:51.661479 31002 raft_consensus.cc:697] T 8155b3fc02b348e1adb290ecfe4592c5 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Becoming Leader. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Running, Role: LEADER
I20260709 13:11:51.662492 31002 consensus_queue.cc:237] T 8155b3fc02b348e1adb290ecfe4592c5 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:51.673411 31070 raft_consensus.cc:493] T 5532e197e7de48f28c034ba1fc490e19 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:51.674057 31070 raft_consensus.cc:515] T 5532e197e7de48f28c034ba1fc490e19 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:51.677143 30796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5532e197e7de48f28c034ba1fc490e19" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:51.678006 30796 raft_consensus.cc:2468] T 5532e197e7de48f28c034ba1fc490e19 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 0.
I20260709 13:11:51.680402 30689 leader_election.cc:304] T 5532e197e7de48f28c034ba1fc490e19 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:51.681599 31066 consensus_queue.cc:1048] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:51.682840 30994 tablet_bootstrap.cc:654] T 8b6052cddae64c5c9d16f60272735788 P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:51.683470 31066 raft_consensus.cc:2804] T 5532e197e7de48f28c034ba1fc490e19 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:51.684001 31066 raft_consensus.cc:493] T 5532e197e7de48f28c034ba1fc490e19 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:51.684566 31066 raft_consensus.cc:3060] T 5532e197e7de48f28c034ba1fc490e19 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:51.692682 31070 leader_election.cc:290] T 5532e197e7de48f28c034ba1fc490e19 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:51.694891 30874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5532e197e7de48f28c034ba1fc490e19" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:51.695758 30874 raft_consensus.cc:2468] T 5532e197e7de48f28c034ba1fc490e19 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 0.
I20260709 13:11:51.704447 31288 raft_consensus.cc:493] T ac85f05e63ed463681b3ec790b39c4d8 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:51.709254 31288 raft_consensus.cc:515] T ac85f05e63ed463681b3ec790b39c4d8 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:51.714252 31288 leader_election.cc:290] T ac85f05e63ed463681b3ec790b39c4d8 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:51.715040 30620 catalog_manager.cc:5697] T 92a4b4a5a07442f49f518b479db8c94b P 02c5705b45c04706bbb2c6ff3d7b0b8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3). New cstate: current_term: 1 leader_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:51.707764 31002 raft_consensus.cc:493] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:51.717000 31002 raft_consensus.cc:515] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:51.718047 30874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac85f05e63ed463681b3ec790b39c4d8" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:51.719183 31002 leader_election.cc:290] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:51.720404 30801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8c0b2b0b11472ca76e15fd3ce2e6f1" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:51.721336 30949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f8c0b2b0b11472ca76e15fd3ce2e6f1" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:51.722230 30949 raft_consensus.cc:2393] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in current term 1: Already voted for candidate 05213ac836744a53b9c089a5c052f4be in this term.
I20260709 13:11:51.706923 31066 raft_consensus.cc:515] T 5532e197e7de48f28c034ba1fc490e19 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:51.725055 30839 leader_election.cc:304] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a; no voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd
W20260709 13:11:51.723966 30914 leader_election.cc:343] T ac85f05e63ed463681b3ec790b39c4d8 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:51.726503 30997 tablet_bootstrap.cc:492] T ea563bd67a9849d18c9c3dfb21d58a93 P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
I20260709 13:11:51.727036 30997 ts_tablet_manager.cc:1403] T ea563bd67a9849d18c9c3dfb21d58a93 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.122s	user 0.013s	sys 0.004s
I20260709 13:11:51.729400 30722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac85f05e63ed463681b3ec790b39c4d8" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:51.730319 31066 leader_election.cc:290] T 5532e197e7de48f28c034ba1fc490e19 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 election: Requested vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:51.732167 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5532e197e7de48f28c034ba1fc490e19" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:51.732834 30799 raft_consensus.cc:3060] T 5532e197e7de48f28c034ba1fc490e19 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:51.734018 30874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5532e197e7de48f28c034ba1fc490e19" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:11:51.734583 30874 raft_consensus.cc:3060] T 5532e197e7de48f28c034ba1fc490e19 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:51.734243 30997 raft_consensus.cc:359] T ea563bd67a9849d18c9c3dfb21d58a93 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:51.735051 30997 raft_consensus.cc:385] T ea563bd67a9849d18c9c3dfb21d58a93 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:51.735378 30997 raft_consensus.cc:740] T ea563bd67a9849d18c9c3dfb21d58a93 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
I20260709 13:11:51.737962 30996 tablet_bootstrap.cc:492] T b9fcfade988746a2967e115dbbcfe0f1 P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
I20260709 13:11:51.738497 30996 ts_tablet_manager.cc:1403] T b9fcfade988746a2967e115dbbcfe0f1 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.137s	user 0.020s	sys 0.007s
W20260709 13:11:51.738591 30915 leader_election.cc:343] T ac85f05e63ed463681b3ec790b39c4d8 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:51.739104 30915 leader_election.cc:304] T ac85f05e63ed463681b3ec790b39c4d8 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:51.739967 31288 raft_consensus.cc:2749] T ac85f05e63ed463681b3ec790b39c4d8 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:51.736105 30997 consensus_queue.cc:260] T ea563bd67a9849d18c9c3dfb21d58a93 P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:51.743369 30997 ts_tablet_manager.cc:1434] T ea563bd67a9849d18c9c3dfb21d58a93 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:11:51.744091 31002 raft_consensus.cc:3060] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:51.744418 30997 tablet_bootstrap.cc:492] T 6fe93136933f4d2880ec355e1bbbf90d P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
I20260709 13:11:51.753273 30622 catalog_manager.cc:5697] T 65776989d6674454816411c251aff5c1 P 05213ac836744a53b9c089a5c052f4be reported cstate change: term changed from 0 to 1, leader changed from <none> to 05213ac836744a53b9c089a5c052f4be (127.21.63.2). New cstate: current_term: 1 leader_uuid: "05213ac836744a53b9c089a5c052f4be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:51.754151 31288 raft_consensus.cc:493] T 8b6052cddae64c5c9d16f60272735788 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:51.754693 31288 raft_consensus.cc:515] T 8b6052cddae64c5c9d16f60272735788 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:51.757879 30801 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6052cddae64c5c9d16f60272735788" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:51.758186 30997 tablet_bootstrap.cc:654] T 6fe93136933f4d2880ec355e1bbbf90d P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
W20260709 13:11:51.760648 30912 leader_election.cc:343] T 8b6052cddae64c5c9d16f60272735788 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:51.763006 31288 leader_election.cc:290] T 8b6052cddae64c5c9d16f60272735788 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:51.765295 30996 raft_consensus.cc:359] T b9fcfade988746a2967e115dbbcfe0f1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:51.766062 30996 raft_consensus.cc:385] T b9fcfade988746a2967e115dbbcfe0f1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:51.766355 30996 raft_consensus.cc:740] T b9fcfade988746a2967e115dbbcfe0f1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
I20260709 13:11:51.767066 30996 consensus_queue.cc:260] T b9fcfade988746a2967e115dbbcfe0f1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:51.768607 30994 tablet_bootstrap.cc:492] T 8b6052cddae64c5c9d16f60272735788 P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
I20260709 13:11:51.763957 30722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6052cddae64c5c9d16f60272735788" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:51.770519 30996 ts_tablet_manager.cc:1434] T b9fcfade988746a2967e115dbbcfe0f1 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.002s
I20260709 13:11:51.770694 30874 raft_consensus.cc:2468] T 5532e197e7de48f28c034ba1fc490e19 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 1.
I20260709 13:11:51.770053 30799 raft_consensus.cc:2468] T 5532e197e7de48f28c034ba1fc490e19 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 1.
I20260709 13:11:51.770162 30624 catalog_manager.cc:5697] T 8fb24a20af104a7486d0f1b2ae0e3843 P 262942f67dfa4b8fa42234b18ac31ffd reported cstate change: term changed from 0 to 1, leader changed from <none> to 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4). New cstate: current_term: 1 leader_uuid: "262942f67dfa4b8fa42234b18ac31ffd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } }
W20260709 13:11:51.772069 30915 leader_election.cc:343] T 8b6052cddae64c5c9d16f60272735788 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:51.772192 30691 leader_election.cc:304] T 5532e197e7de48f28c034ba1fc490e19 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:51.793551 30623 catalog_manager.cc:5697] T 8155b3fc02b348e1adb290ecfe4592c5 P 02c5705b45c04706bbb2c6ff3d7b0b8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3). New cstate: current_term: 1 leader_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:51.796561 31070 raft_consensus.cc:2804] T 5532e197e7de48f28c034ba1fc490e19 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:51.797169 31070 raft_consensus.cc:697] T 5532e197e7de48f28c034ba1fc490e19 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 LEADER]: Becoming Leader. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Running, Role: LEADER
I20260709 13:11:51.797653 31286 raft_consensus.cc:493] T f8f19b152a0f46ac896478f3f05c8822 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:51.798055 31070 consensus_queue.cc:237] T 5532e197e7de48f28c034ba1fc490e19 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:51.772790 30994 ts_tablet_manager.cc:1403] T 8b6052cddae64c5c9d16f60272735788 P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.148s	user 0.012s	sys 0.007s
I20260709 13:11:51.804608 30915 leader_election.cc:304] T 8b6052cddae64c5c9d16f60272735788 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd; no voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:51.805943 31177 raft_consensus.cc:2749] T 8b6052cddae64c5c9d16f60272735788 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:51.769361 30995 tablet_bootstrap.cc:492] T 8155b3fc02b348e1adb290ecfe4592c5 P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:51.773237 30996 tablet_bootstrap.cc:492] T 784deee1ea9543df840c58ecaa6f911a P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
I20260709 13:11:51.798219 31286 raft_consensus.cc:515] T f8f19b152a0f46ac896478f3f05c8822 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:51.814728 30996 tablet_bootstrap.cc:654] T 784deee1ea9543df840c58ecaa6f911a P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:51.816170 30949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8f19b152a0f46ac896478f3f05c8822" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:51.814663 31286 leader_election.cc:290] T f8f19b152a0f46ac896478f3f05c8822 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:51.816586 30875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8f19b152a0f46ac896478f3f05c8822" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:51.817332 30995 ts_tablet_manager.cc:1403] T 8155b3fc02b348e1adb290ecfe4592c5 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.217s	user 0.019s	sys 0.005s
W20260709 13:11:51.817754 30765 leader_election.cc:343] T f8f19b152a0f46ac896478f3f05c8822 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:11:51.818526 30766 leader_election.cc:343] T f8f19b152a0f46ac896478f3f05c8822 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:51.763557 31002 raft_consensus.cc:2749] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:51.819839 31177 raft_consensus.cc:493] T ea563bd67a9849d18c9c3dfb21d58a93 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:51.820849 31177 raft_consensus.cc:515] T ea563bd67a9849d18c9c3dfb21d58a93 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:51.821460 30995 raft_consensus.cc:359] T 8155b3fc02b348e1adb290ecfe4592c5 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:51.823169 31177 leader_election.cc:290] T ea563bd67a9849d18c9c3dfb21d58a93 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:51.824476 30995 raft_consensus.cc:385] T 8155b3fc02b348e1adb290ecfe4592c5 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:51.825353 30995 raft_consensus.cc:740] T 8155b3fc02b348e1adb290ecfe4592c5 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:51.827678 30994 raft_consensus.cc:359] T 8b6052cddae64c5c9d16f60272735788 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:51.829205 30994 raft_consensus.cc:385] T 8b6052cddae64c5c9d16f60272735788 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:51.828954 30995 consensus_queue.cc:260] T 8155b3fc02b348e1adb290ecfe4592c5 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:51.830631 30994 raft_consensus.cc:740] T 8b6052cddae64c5c9d16f60272735788 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
I20260709 13:11:51.831442 30994 consensus_queue.cc:260] T 8b6052cddae64c5c9d16f60272735788 P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:51.833448 30875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea563bd67a9849d18c9c3dfb21d58a93" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:51.834128 30994 ts_tablet_manager.cc:1434] T 8b6052cddae64c5c9d16f60272735788 P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.030s	user 0.004s	sys 0.002s
I20260709 13:11:51.835261 30994 tablet_bootstrap.cc:492] T d5175e0d898c453b8a7641d94dc1ac58 P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
W20260709 13:11:51.837985 30914 leader_election.cc:343] T ea563bd67a9849d18c9c3dfb21d58a93 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:51.839877 30995 ts_tablet_manager.cc:1434] T 8155b3fc02b348e1adb290ecfe4592c5 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 13:11:51.845345 30995 tablet_bootstrap.cc:492] T 8fb24a20af104a7486d0f1b2ae0e3843 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:51.850124 30995 tablet_bootstrap.cc:654] T 8fb24a20af104a7486d0f1b2ae0e3843 P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:51.852481 30994 tablet_bootstrap.cc:654] T d5175e0d898c453b8a7641d94dc1ac58 P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:51.820299 30766 leader_election.cc:304] T f8f19b152a0f46ac896478f3f05c8822 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 262942f67dfa4b8fa42234b18ac31ffd
I20260709 13:11:51.858258 31286 raft_consensus.cc:2749] T f8f19b152a0f46ac896478f3f05c8822 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:51.867008 30725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea563bd67a9849d18c9c3dfb21d58a93" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
W20260709 13:11:51.874419 30915 leader_election.cc:343] T ea563bd67a9849d18c9c3dfb21d58a93 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:51.874955 30915 leader_election.cc:304] T ea563bd67a9849d18c9c3dfb21d58a93 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:51.876266 31177 raft_consensus.cc:2749] T ea563bd67a9849d18c9c3dfb21d58a93 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:51.877367 31286 consensus_queue.cc:1048] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 05213ac836744a53b9c089a5c052f4be [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:51.883390 31109 consensus_queue.cc:1048] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 05213ac836744a53b9c089a5c052f4be [LEADER]: Connected to new peer: Peer: permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:51.903712 31177 consensus_queue.cc:1048] T b9fcfade988746a2967e115dbbcfe0f1 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:51.910808 30995 tablet_bootstrap.cc:492] T 8fb24a20af104a7486d0f1b2ae0e3843 P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:51.911379 30995 ts_tablet_manager.cc:1403] T 8fb24a20af104a7486d0f1b2ae0e3843 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.066s	user 0.009s	sys 0.000s
I20260709 13:11:51.915742 30997 tablet_bootstrap.cc:492] T 6fe93136933f4d2880ec355e1bbbf90d P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
I20260709 13:11:51.916307 30997 ts_tablet_manager.cc:1403] T 6fe93136933f4d2880ec355e1bbbf90d P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.172s	user 0.017s	sys 0.012s
I20260709 13:11:51.935446 30875 raft_consensus.cc:3060] T b9fcfade988746a2967e115dbbcfe0f1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:51.948604 30997 raft_consensus.cc:359] T 6fe93136933f4d2880ec355e1bbbf90d P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:51.949432 30997 raft_consensus.cc:385] T 6fe93136933f4d2880ec355e1bbbf90d P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:51.949685 30997 raft_consensus.cc:740] T 6fe93136933f4d2880ec355e1bbbf90d P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
I20260709 13:11:51.950344 30997 consensus_queue.cc:260] T 6fe93136933f4d2880ec355e1bbbf90d P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:51.953156 31240 raft_consensus.cc:493] T b9fcfade988746a2967e115dbbcfe0f1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:51.969110 31240 raft_consensus.cc:515] T b9fcfade988746a2967e115dbbcfe0f1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:51.970253 30997 ts_tablet_manager.cc:1434] T 6fe93136933f4d2880ec355e1bbbf90d P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.054s	user 0.006s	sys 0.000s
I20260709 13:11:51.954864 31046 consensus_queue.cc:1048] T b9fcfade988746a2967e115dbbcfe0f1 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:51.974048 30944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9fcfade988746a2967e115dbbcfe0f1" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:51.977535 30796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9fcfade988746a2967e115dbbcfe0f1" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:51.978266 31240 leader_election.cc:290] T b9fcfade988746a2967e115dbbcfe0f1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:51.979161 30838 leader_election.cc:304] T b9fcfade988746a2967e115dbbcfe0f1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a; no voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd
I20260709 13:11:51.955560 30995 raft_consensus.cc:359] T 8fb24a20af104a7486d0f1b2ae0e3843 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:51.962888 30994 tablet_bootstrap.cc:492] T d5175e0d898c453b8a7641d94dc1ac58 P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
I20260709 13:11:51.963219 30623 catalog_manager.cc:5697] T 5532e197e7de48f28c034ba1fc490e19 P d22c9d3d15904327a78a0e548a8ff6cd reported cstate change: term changed from 0 to 1, leader changed from <none> to d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1). New cstate: current_term: 1 leader_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:51.980558 30994 ts_tablet_manager.cc:1403] T d5175e0d898c453b8a7641d94dc1ac58 P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.146s	user 0.018s	sys 0.000s
I20260709 13:11:51.983780 30997 tablet_bootstrap.cc:492] T 94220be3f55a4acbbd6bd5723b9f32e9 P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
I20260709 13:11:51.983438 30994 raft_consensus.cc:359] T d5175e0d898c453b8a7641d94dc1ac58 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:51.967051 30996 tablet_bootstrap.cc:492] T 784deee1ea9543df840c58ecaa6f911a P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
I20260709 13:11:51.985176 30996 ts_tablet_manager.cc:1403] T 784deee1ea9543df840c58ecaa6f911a P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.212s	user 0.009s	sys 0.023s
I20260709 13:11:51.988034 30996 raft_consensus.cc:359] T 784deee1ea9543df840c58ecaa6f911a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:51.988766 30996 raft_consensus.cc:385] T 784deee1ea9543df840c58ecaa6f911a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:51.989048 30996 raft_consensus.cc:740] T 784deee1ea9543df840c58ecaa6f911a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
I20260709 13:11:51.989673 30996 consensus_queue.cc:260] T 784deee1ea9543df840c58ecaa6f911a P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:51.994374 31002 consensus_queue.cc:1048] T 92a4b4a5a07442f49f518b479db8c94b P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:51.992359 30994 raft_consensus.cc:385] T d5175e0d898c453b8a7641d94dc1ac58 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:51.996565 30994 raft_consensus.cc:740] T d5175e0d898c453b8a7641d94dc1ac58 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
I20260709 13:11:51.997473 30994 consensus_queue.cc:260] T d5175e0d898c453b8a7641d94dc1ac58 P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:51.999315 30994 ts_tablet_manager.cc:1434] T d5175e0d898c453b8a7641d94dc1ac58 P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 13:11:52.000864 30995 raft_consensus.cc:385] T 8fb24a20af104a7486d0f1b2ae0e3843 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:52.001201 30995 raft_consensus.cc:740] T 8fb24a20af104a7486d0f1b2ae0e3843 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:52.004267 30994 tablet_bootstrap.cc:492] T 80a61882ef804a85a31e90d67930c5e9 P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
I20260709 13:11:52.007802 31240 raft_consensus.cc:2749] T b9fcfade988746a2967e115dbbcfe0f1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:52.014108 30996 ts_tablet_manager.cc:1434] T 784deee1ea9543df840c58ecaa6f911a P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.029s	user 0.005s	sys 0.002s
I20260709 13:11:52.015599 30996 tablet_bootstrap.cc:492] T ac85f05e63ed463681b3ec790b39c4d8 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
I20260709 13:11:52.016985 30995 consensus_queue.cc:260] T 8fb24a20af104a7486d0f1b2ae0e3843 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:52.019236 30995 ts_tablet_manager.cc:1434] T 8fb24a20af104a7486d0f1b2ae0e3843 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.108s	user 0.006s	sys 0.001s
I20260709 13:11:52.034363 30725 raft_consensus.cc:3060] T 8fb24a20af104a7486d0f1b2ae0e3843 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:52.037076 30997 tablet_bootstrap.cc:654] T 94220be3f55a4acbbd6bd5723b9f32e9 P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:52.043613 31089 consensus_queue.cc:1048] T 8fb24a20af104a7486d0f1b2ae0e3843 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:52.056171 30995 tablet_bootstrap.cc:492] T 784deee1ea9543df840c58ecaa6f911a P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:52.065191 31283 raft_consensus.cc:493] T d5175e0d898c453b8a7641d94dc1ac58 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:52.065796 31283 raft_consensus.cc:515] T d5175e0d898c453b8a7641d94dc1ac58 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:52.067981 31283 leader_election.cc:290] T d5175e0d898c453b8a7641d94dc1ac58 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:52.068136 31288 consensus_queue.cc:1048] T 8fb24a20af104a7486d0f1b2ae0e3843 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:52.075415 30722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5175e0d898c453b8a7641d94dc1ac58" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
W20260709 13:11:52.077214 30767 leader_election.cc:343] T d5175e0d898c453b8a7641d94dc1ac58 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:52.095559 31233 consensus_queue.cc:1048] T 92a4b4a5a07442f49f518b479db8c94b P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:52.100723 30871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5175e0d898c453b8a7641d94dc1ac58" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
W20260709 13:11:52.102237 30766 leader_election.cc:343] T d5175e0d898c453b8a7641d94dc1ac58 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:52.102713 30766 leader_election.cc:304] T d5175e0d898c453b8a7641d94dc1ac58 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:52.103483 31283 raft_consensus.cc:2749] T d5175e0d898c453b8a7641d94dc1ac58 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:52.126549 31233 consensus_queue.cc:1048] T 8155b3fc02b348e1adb290ecfe4592c5 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:52.127254 30994 tablet_bootstrap.cc:700] T 80a61882ef804a85a31e90d67930c5e9 P 05213ac836744a53b9c089a5c052f4be: Time spent opening tablet: real 0.103s	user 0.004s	sys 0.000s
I20260709 13:11:52.128055 30994 tablet_bootstrap.cc:654] T 80a61882ef804a85a31e90d67930c5e9 P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:52.141445 30996 tablet_bootstrap.cc:654] T ac85f05e63ed463681b3ec790b39c4d8 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:52.143631 30995 tablet_bootstrap.cc:654] T 784deee1ea9543df840c58ecaa6f911a P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:52.155256 30997 tablet_bootstrap.cc:492] T 94220be3f55a4acbbd6bd5723b9f32e9 P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
I20260709 13:11:52.155748 30997 ts_tablet_manager.cc:1403] T 94220be3f55a4acbbd6bd5723b9f32e9 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.172s	user 0.026s	sys 0.022s
W20260709 13:11:52.169284 30671 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:11:52.166846 30997 raft_consensus.cc:359] T 94220be3f55a4acbbd6bd5723b9f32e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:52.184132 30997 raft_consensus.cc:385] T 94220be3f55a4acbbd6bd5723b9f32e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:52.185024 30722 raft_consensus.cc:3060] T 8155b3fc02b348e1adb290ecfe4592c5 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:52.192829 31283 consensus_queue.cc:1048] T 65776989d6674454816411c251aff5c1 P 05213ac836744a53b9c089a5c052f4be [LEADER]: Connected to new peer: Peer: permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:52.203635 30996 tablet_bootstrap.cc:492] T ac85f05e63ed463681b3ec790b39c4d8 P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
I20260709 13:11:52.204370 30996 ts_tablet_manager.cc:1403] T ac85f05e63ed463681b3ec790b39c4d8 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.189s	user 0.016s	sys 0.002s
I20260709 13:11:52.213871 30995 tablet_bootstrap.cc:492] T 784deee1ea9543df840c58ecaa6f911a P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:52.214401 30995 ts_tablet_manager.cc:1403] T 784deee1ea9543df840c58ecaa6f911a P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.158s	user 0.009s	sys 0.010s
I20260709 13:11:52.184478 30997 raft_consensus.cc:740] T 94220be3f55a4acbbd6bd5723b9f32e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
I20260709 13:11:52.221823 30994 tablet_bootstrap.cc:492] T 80a61882ef804a85a31e90d67930c5e9 P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
W20260709 13:11:52.221933 30766 consensus_peers.cc:597] T 65776989d6674454816411c251aff5c1 P 05213ac836744a53b9c089a5c052f4be -> Peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Couldn't send request to peer 02c5705b45c04706bbb2c6ff3d7b0b8a. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:52.222550 30994 ts_tablet_manager.cc:1403] T 80a61882ef804a85a31e90d67930c5e9 P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.219s	user 0.019s	sys 0.022s
I20260709 13:11:52.222929 30995 raft_consensus.cc:359] T 784deee1ea9543df840c58ecaa6f911a P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:52.223690 30995 raft_consensus.cc:385] T 784deee1ea9543df840c58ecaa6f911a P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:52.223968 30995 raft_consensus.cc:740] T 784deee1ea9543df840c58ecaa6f911a P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:52.225330 30994 raft_consensus.cc:359] T 80a61882ef804a85a31e90d67930c5e9 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:52.226024 30994 raft_consensus.cc:385] T 80a61882ef804a85a31e90d67930c5e9 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:52.226274 30994 raft_consensus.cc:740] T 80a61882ef804a85a31e90d67930c5e9 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
I20260709 13:11:52.226869 30994 consensus_queue.cc:260] T 80a61882ef804a85a31e90d67930c5e9 P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:52.229326 30994 ts_tablet_manager.cc:1434] T 80a61882ef804a85a31e90d67930c5e9 P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.001s
I20260709 13:11:52.224855 30995 consensus_queue.cc:260] T 784deee1ea9543df840c58ecaa6f911a P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:52.233400 30997 consensus_queue.cc:260] T 94220be3f55a4acbbd6bd5723b9f32e9 P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:52.242820 31089 raft_consensus.cc:493] T 6fe93136933f4d2880ec355e1bbbf90d P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:52.243534 30995 ts_tablet_manager.cc:1434] T 784deee1ea9543df840c58ecaa6f911a P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.029s	user 0.006s	sys 0.003s
I20260709 13:11:52.243369 31089 raft_consensus.cc:515] T 6fe93136933f4d2880ec355e1bbbf90d P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:52.246474 31089 leader_election.cc:290] T 6fe93136933f4d2880ec355e1bbbf90d P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:52.244213 30996 raft_consensus.cc:359] T ac85f05e63ed463681b3ec790b39c4d8 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:52.249209 30996 raft_consensus.cc:385] T ac85f05e63ed463681b3ec790b39c4d8 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:52.249595 30996 raft_consensus.cc:740] T ac85f05e63ed463681b3ec790b39c4d8 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
I20260709 13:11:52.252876 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fe93136933f4d2880ec355e1bbbf90d" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:52.248823 30723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fe93136933f4d2880ec355e1bbbf90d" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
W20260709 13:11:52.255172 30915 leader_election.cc:343] T 6fe93136933f4d2880ec355e1bbbf90d P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:52.258507 30995 tablet_bootstrap.cc:492] T c0100ec8bd1e4a45b3c0ea52281f9204 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:52.259196 30994 tablet_bootstrap.cc:492] T 6fe93136933f4d2880ec355e1bbbf90d P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
I20260709 13:11:52.258217 30996 consensus_queue.cc:260] T ac85f05e63ed463681b3ec790b39c4d8 P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:52.262152 30996 ts_tablet_manager.cc:1434] T ac85f05e63ed463681b3ec790b39c4d8 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.054s	user 0.006s	sys 0.000s
I20260709 13:11:52.263731 30996 tablet_bootstrap.cc:492] T 65776989d6674454816411c251aff5c1 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
W20260709 13:11:52.268042 30912 leader_election.cc:343] T 6fe93136933f4d2880ec355e1bbbf90d P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:52.268435 31279 consensus_queue.cc:1048] T 5532e197e7de48f28c034ba1fc490e19 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:52.268590 30912 leader_election.cc:304] T 6fe93136933f4d2880ec355e1bbbf90d P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd; no voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:52.269584 31288 raft_consensus.cc:2749] T 6fe93136933f4d2880ec355e1bbbf90d P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:52.270514 30996 tablet_bootstrap.cc:654] T 65776989d6674454816411c251aff5c1 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:52.277784 30997 ts_tablet_manager.cc:1434] T 94220be3f55a4acbbd6bd5723b9f32e9 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.113s	user 0.006s	sys 0.001s
I20260709 13:11:52.280546 30997 tablet_bootstrap.cc:492] T d16c44f56c42466497ba4f7f1b2b5344 P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
I20260709 13:11:52.287526 30997 tablet_bootstrap.cc:654] T d16c44f56c42466497ba4f7f1b2b5344 P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:52.288157 31283 raft_consensus.cc:493] T 8b6052cddae64c5c9d16f60272735788 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:52.288753 31283 raft_consensus.cc:515] T 8b6052cddae64c5c9d16f60272735788 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:52.291163 31283 leader_election.cc:290] T 8b6052cddae64c5c9d16f60272735788 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:52.293637 30944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6052cddae64c5c9d16f60272735788" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:52.294519 30944 raft_consensus.cc:2468] T 8b6052cddae64c5c9d16f60272735788 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05213ac836744a53b9c089a5c052f4be in term 0.
I20260709 13:11:52.296350 30765 leader_election.cc:304] T 8b6052cddae64c5c9d16f60272735788 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
I20260709 13:11:52.308734 31109 raft_consensus.cc:2804] T 8b6052cddae64c5c9d16f60272735788 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:52.309158 31109 raft_consensus.cc:493] T 8b6052cddae64c5c9d16f60272735788 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:52.309363 31279 raft_consensus.cc:493] T 8155b3fc02b348e1adb290ecfe4592c5 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:52.309568 31109 raft_consensus.cc:3060] T 8b6052cddae64c5c9d16f60272735788 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:52.310014 31279 raft_consensus.cc:515] T 8155b3fc02b348e1adb290ecfe4592c5 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:52.304471 30723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6052cddae64c5c9d16f60272735788" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:52.315209 31279 leader_election.cc:290] T 8155b3fc02b348e1adb290ecfe4592c5 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
W20260709 13:11:52.320022 30767 leader_election.cc:343] T 8b6052cddae64c5c9d16f60272735788 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:52.322499 31002 consensus_queue.cc:1048] T 8155b3fc02b348e1adb290ecfe4592c5 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:52.323957 30994 tablet_bootstrap.cc:654] T 6fe93136933f4d2880ec355e1bbbf90d P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:52.326838 30796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8155b3fc02b348e1adb290ecfe4592c5" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:52.333194 30995 tablet_bootstrap.cc:654] T c0100ec8bd1e4a45b3c0ea52281f9204 P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:52.342964 30875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8155b3fc02b348e1adb290ecfe4592c5" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:52.345952 31002 raft_consensus.cc:493] T ac85f05e63ed463681b3ec790b39c4d8 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:52.346581 31002 raft_consensus.cc:515] T ac85f05e63ed463681b3ec790b39c4d8 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:52.351508 30947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac85f05e63ed463681b3ec790b39c4d8" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:52.352056 30689 leader_election.cc:304] T 8155b3fc02b348e1adb290ecfe4592c5 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22c9d3d15904327a78a0e548a8ff6cd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 05213ac836744a53b9c089a5c052f4be
I20260709 13:11:52.352357 30947 raft_consensus.cc:2468] T ac85f05e63ed463681b3ec790b39c4d8 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 0.
I20260709 13:11:52.374498 30839 leader_election.cc:304] T ac85f05e63ed463681b3ec790b39c4d8 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
I20260709 13:11:52.381811 31239 raft_consensus.cc:2804] T ac85f05e63ed463681b3ec790b39c4d8 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:52.382371 31239 raft_consensus.cc:493] T ac85f05e63ed463681b3ec790b39c4d8 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:52.382771 31239 raft_consensus.cc:3060] T ac85f05e63ed463681b3ec790b39c4d8 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:52.393813 31002 leader_election.cc:290] T ac85f05e63ed463681b3ec790b39c4d8 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:52.398218 30723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac85f05e63ed463681b3ec790b39c4d8" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:52.398577 31070 consensus_queue.cc:1048] T 5532e197e7de48f28c034ba1fc490e19 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:11:52.401238 30841 leader_election.cc:343] T ac85f05e63ed463681b3ec790b39c4d8 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:52.411504 30997 tablet_bootstrap.cc:492] T d16c44f56c42466497ba4f7f1b2b5344 P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
I20260709 13:11:52.412034 30997 ts_tablet_manager.cc:1403] T d16c44f56c42466497ba4f7f1b2b5344 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.132s	user 0.051s	sys 0.060s
I20260709 13:11:52.423074 30996 tablet_bootstrap.cc:492] T 65776989d6674454816411c251aff5c1 P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
I20260709 13:11:52.423642 30996 ts_tablet_manager.cc:1403] T 65776989d6674454816411c251aff5c1 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.160s	user 0.004s	sys 0.026s
I20260709 13:11:52.426474 30996 raft_consensus.cc:359] T 65776989d6674454816411c251aff5c1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:52.427104 30996 raft_consensus.cc:385] T 65776989d6674454816411c251aff5c1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:52.427357 30996 raft_consensus.cc:740] T 65776989d6674454816411c251aff5c1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
I20260709 13:11:52.440851 30996 consensus_queue.cc:260] T 65776989d6674454816411c251aff5c1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:52.443344 30996 ts_tablet_manager.cc:1434] T 65776989d6674454816411c251aff5c1 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 13:11:52.414911 30997 raft_consensus.cc:359] T d16c44f56c42466497ba4f7f1b2b5344 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:52.428120 31239 raft_consensus.cc:515] T ac85f05e63ed463681b3ec790b39c4d8 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:52.444880 30997 raft_consensus.cc:385] T d16c44f56c42466497ba4f7f1b2b5344 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:52.445222 30997 raft_consensus.cc:740] T d16c44f56c42466497ba4f7f1b2b5344 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
I20260709 13:11:52.447724 31239 leader_election.cc:290] T ac85f05e63ed463681b3ec790b39c4d8 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Requested vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:52.449862 30725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac85f05e63ed463681b3ec790b39c4d8" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:52.451149 30947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac85f05e63ed463681b3ec790b39c4d8" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:11:52.452167 30947 raft_consensus.cc:3060] T ac85f05e63ed463681b3ec790b39c4d8 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:11:52.452350 30841 leader_election.cc:343] T ac85f05e63ed463681b3ec790b39c4d8 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:52.444279 31109 raft_consensus.cc:515] T 8b6052cddae64c5c9d16f60272735788 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:52.445928 30997 consensus_queue.cc:260] T d16c44f56c42466497ba4f7f1b2b5344 P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:52.462570 30723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6052cddae64c5c9d16f60272735788" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
W20260709 13:11:52.464349 30767 leader_election.cc:343] T 8b6052cddae64c5c9d16f60272735788 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:52.466696 31240 raft_consensus.cc:493] T 784deee1ea9543df840c58ecaa6f911a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:52.467353 31109 leader_election.cc:290] T 8b6052cddae64c5c9d16f60272735788 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 election: Requested vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:52.474339 30994 tablet_bootstrap.cc:492] T 6fe93136933f4d2880ec355e1bbbf90d P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
I20260709 13:11:52.475051 30994 ts_tablet_manager.cc:1403] T 6fe93136933f4d2880ec355e1bbbf90d P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.225s	user 0.019s	sys 0.009s
I20260709 13:11:52.467196 31240 raft_consensus.cc:515] T 784deee1ea9543df840c58ecaa6f911a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:52.477892 30997 ts_tablet_manager.cc:1434] T d16c44f56c42466497ba4f7f1b2b5344 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.066s	user 0.004s	sys 0.004s
I20260709 13:11:52.478485 31240 leader_election.cc:290] T 784deee1ea9543df840c58ecaa6f911a P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:52.479370 30997 tablet_bootstrap.cc:492] T 80a61882ef804a85a31e90d67930c5e9 P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
I20260709 13:11:52.480858 30949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784deee1ea9543df840c58ecaa6f911a" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:52.481509 30949 raft_consensus.cc:2468] T 784deee1ea9543df840c58ecaa6f911a P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 0.
I20260709 13:11:52.481523 30723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784deee1ea9543df840c58ecaa6f911a" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:52.483074 30723 raft_consensus.cc:2468] T 784deee1ea9543df840c58ecaa6f911a P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 0.
I20260709 13:11:52.484062 30839 leader_election.cc:304] T 784deee1ea9543df840c58ecaa6f911a P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
I20260709 13:11:52.485869 31069 raft_consensus.cc:2749] T 8155b3fc02b348e1adb290ecfe4592c5 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:52.444516 30996 tablet_bootstrap.cc:492] T af695b428b974773ba476dce0d31c5a1 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
I20260709 13:11:52.490401 31002 raft_consensus.cc:2804] T 784deee1ea9543df840c58ecaa6f911a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:52.493662 31109 raft_consensus.cc:493] T 80a61882ef804a85a31e90d67930c5e9 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:52.494228 31109 raft_consensus.cc:515] T 80a61882ef804a85a31e90d67930c5e9 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:52.497442 30949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80a61882ef804a85a31e90d67930c5e9" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:52.490798 31002 raft_consensus.cc:493] T 784deee1ea9543df840c58ecaa6f911a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:52.498006 31002 raft_consensus.cc:3060] T 784deee1ea9543df840c58ecaa6f911a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:11:52.498970 30765 leader_election.cc:343] T 80a61882ef804a85a31e90d67930c5e9 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:52.500046 31109 leader_election.cc:290] T 80a61882ef804a85a31e90d67930c5e9 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:52.490738 30994 raft_consensus.cc:359] T 6fe93136933f4d2880ec355e1bbbf90d P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:52.501128 30994 raft_consensus.cc:385] T 6fe93136933f4d2880ec355e1bbbf90d P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:52.501490 30994 raft_consensus.cc:740] T 6fe93136933f4d2880ec355e1bbbf90d P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
I20260709 13:11:52.502323 30871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80a61882ef804a85a31e90d67930c5e9" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:52.502264 30994 consensus_queue.cc:260] T 6fe93136933f4d2880ec355e1bbbf90d P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
W20260709 13:11:52.504138 30766 leader_election.cc:343] T 80a61882ef804a85a31e90d67930c5e9 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:52.505580 30994 ts_tablet_manager.cc:1434] T 6fe93136933f4d2880ec355e1bbbf90d P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.030s	user 0.008s	sys 0.001s
I20260709 13:11:52.506812 30994 tablet_bootstrap.cc:492] T d16c44f56c42466497ba4f7f1b2b5344 P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
I20260709 13:11:52.510392 30996 tablet_bootstrap.cc:654] T af695b428b974773ba476dce0d31c5a1 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:52.504613 30766 leader_election.cc:304] T 80a61882ef804a85a31e90d67930c5e9 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 262942f67dfa4b8fa42234b18ac31ffd
I20260709 13:11:52.513175 31109 raft_consensus.cc:2749] T 80a61882ef804a85a31e90d67930c5e9 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:52.516363 30997 tablet_bootstrap.cc:654] T 80a61882ef804a85a31e90d67930c5e9 P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:52.521592 30995 tablet_bootstrap.cc:492] T c0100ec8bd1e4a45b3c0ea52281f9204 P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:52.522149 30995 ts_tablet_manager.cc:1403] T c0100ec8bd1e4a45b3c0ea52281f9204 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.273s	user 0.019s	sys 0.010s
I20260709 13:11:52.523819 30994 tablet_bootstrap.cc:654] T d16c44f56c42466497ba4f7f1b2b5344 P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:52.525177 30995 raft_consensus.cc:359] T c0100ec8bd1e4a45b3c0ea52281f9204 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:52.526158 30995 raft_consensus.cc:385] T c0100ec8bd1e4a45b3c0ea52281f9204 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:52.526510 30995 raft_consensus.cc:740] T c0100ec8bd1e4a45b3c0ea52281f9204 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:52.527420 30995 consensus_queue.cc:260] T c0100ec8bd1e4a45b3c0ea52281f9204 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:52.470116 30947 raft_consensus.cc:2468] T ac85f05e63ed463681b3ec790b39c4d8 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 1.
I20260709 13:11:52.470629 30944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6052cddae64c5c9d16f60272735788" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:11:52.529091 30944 raft_consensus.cc:3060] T 8b6052cddae64c5c9d16f60272735788 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:52.536322 30839 leader_election.cc:304] T ac85f05e63ed463681b3ec790b39c4d8 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 262942f67dfa4b8fa42234b18ac31ffd; no voters: d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:52.537461 31239 raft_consensus.cc:2804] T ac85f05e63ed463681b3ec790b39c4d8 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:52.551033 31002 raft_consensus.cc:515] T 784deee1ea9543df840c58ecaa6f911a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:52.555016 31002 leader_election.cc:290] T 784deee1ea9543df840c58ecaa6f911a P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Requested vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:52.538152 31239 raft_consensus.cc:697] T ac85f05e63ed463681b3ec790b39c4d8 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Becoming Leader. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Running, Role: LEADER
I20260709 13:11:52.558554 31239 consensus_queue.cc:237] T ac85f05e63ed463681b3ec790b39c4d8 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:52.562011 30722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784deee1ea9543df840c58ecaa6f911a" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:52.562906 30722 raft_consensus.cc:3060] T 784deee1ea9543df840c58ecaa6f911a P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:52.567639 30995 ts_tablet_manager.cc:1434] T c0100ec8bd1e4a45b3c0ea52281f9204 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.045s	user 0.009s	sys 0.001s
I20260709 13:11:52.578382 30947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "784deee1ea9543df840c58ecaa6f911a" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:11:52.584318 30947 raft_consensus.cc:3060] T 784deee1ea9543df840c58ecaa6f911a P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:52.585654 30996 tablet_bootstrap.cc:492] T af695b428b974773ba476dce0d31c5a1 P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
I20260709 13:11:52.586256 30996 ts_tablet_manager.cc:1403] T af695b428b974773ba476dce0d31c5a1 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.142s	user 0.018s	sys 0.007s
I20260709 13:11:52.594110 30944 raft_consensus.cc:2468] T 8b6052cddae64c5c9d16f60272735788 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05213ac836744a53b9c089a5c052f4be in term 1.
I20260709 13:11:52.597404 31002 raft_consensus.cc:493] T 65776989d6674454816411c251aff5c1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:52.597981 30995 tablet_bootstrap.cc:492] T ac85f05e63ed463681b3ec790b39c4d8 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:52.598069 31002 raft_consensus.cc:515] T 65776989d6674454816411c251aff5c1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:52.601902 30997 tablet_bootstrap.cc:492] T 80a61882ef804a85a31e90d67930c5e9 P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
I20260709 13:11:52.602392 30997 ts_tablet_manager.cc:1403] T 80a61882ef804a85a31e90d67930c5e9 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.123s	user 0.020s	sys 0.011s
I20260709 13:11:52.602763 30765 leader_election.cc:304] T 8b6052cddae64c5c9d16f60272735788 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:52.604315 30994 tablet_bootstrap.cc:492] T d16c44f56c42466497ba4f7f1b2b5344 P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
I20260709 13:11:52.604915 30994 ts_tablet_manager.cc:1403] T d16c44f56c42466497ba4f7f1b2b5344 P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.098s	user 0.007s	sys 0.021s
I20260709 13:11:52.599749 30996 raft_consensus.cc:359] T af695b428b974773ba476dce0d31c5a1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:52.605538 30996 raft_consensus.cc:385] T af695b428b974773ba476dce0d31c5a1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:52.605818 30996 raft_consensus.cc:740] T af695b428b974773ba476dce0d31c5a1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
I20260709 13:11:52.606236 31109 raft_consensus.cc:2804] T 8b6052cddae64c5c9d16f60272735788 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:52.606771 31109 raft_consensus.cc:697] T 8b6052cddae64c5c9d16f60272735788 P 05213ac836744a53b9c089a5c052f4be [term 1 LEADER]: Becoming Leader. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Running, Role: LEADER
I20260709 13:11:52.606503 30996 consensus_queue.cc:260] T af695b428b974773ba476dce0d31c5a1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:52.607589 31109 consensus_queue.cc:237] T 8b6052cddae64c5c9d16f60272735788 P 05213ac836744a53b9c089a5c052f4be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:52.608444 30996 ts_tablet_manager.cc:1434] T af695b428b974773ba476dce0d31c5a1 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.000s
I20260709 13:11:52.608500 30994 raft_consensus.cc:359] T d16c44f56c42466497ba4f7f1b2b5344 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:52.609467 30994 raft_consensus.cc:385] T d16c44f56c42466497ba4f7f1b2b5344 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:52.609808 30994 raft_consensus.cc:740] T d16c44f56c42466497ba4f7f1b2b5344 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
I20260709 13:11:52.610527 30994 consensus_queue.cc:260] T d16c44f56c42466497ba4f7f1b2b5344 P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:52.613933 30722 raft_consensus.cc:2468] T 784deee1ea9543df840c58ecaa6f911a P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 1.
I20260709 13:11:52.622110 30996 tablet_bootstrap.cc:492] T ea563bd67a9849d18c9c3dfb21d58a93 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
I20260709 13:11:52.620445 30997 raft_consensus.cc:359] T 80a61882ef804a85a31e90d67930c5e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:52.624033 30841 leader_election.cc:304] T 784deee1ea9543df840c58ecaa6f911a P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:52.624222 31066 raft_consensus.cc:493] T c0100ec8bd1e4a45b3c0ea52281f9204 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:52.624325 30997 raft_consensus.cc:385] T 80a61882ef804a85a31e90d67930c5e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:52.624855 30997 raft_consensus.cc:740] T 80a61882ef804a85a31e90d67930c5e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
I20260709 13:11:52.624863 31066 raft_consensus.cc:515] T c0100ec8bd1e4a45b3c0ea52281f9204 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:52.625440 31240 raft_consensus.cc:2804] T 784deee1ea9543df840c58ecaa6f911a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:52.625958 31240 raft_consensus.cc:697] T 784deee1ea9543df840c58ecaa6f911a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Becoming Leader. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Running, Role: LEADER
I20260709 13:11:52.625828 30997 consensus_queue.cc:260] T 80a61882ef804a85a31e90d67930c5e9 P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:52.627908 30949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0100ec8bd1e4a45b3c0ea52281f9204" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:52.628161 30997 ts_tablet_manager.cc:1434] T 80a61882ef804a85a31e90d67930c5e9 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.025s	user 0.003s	sys 0.003s
I20260709 13:11:52.628582 30949 raft_consensus.cc:2393] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d22c9d3d15904327a78a0e548a8ff6cd in current term 1: Already voted for candidate 262942f67dfa4b8fa42234b18ac31ffd in this term.
I20260709 13:11:52.629189 31066 leader_election.cc:290] T c0100ec8bd1e4a45b3c0ea52281f9204 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:52.629916 31240 consensus_queue.cc:237] T 784deee1ea9543df840c58ecaa6f911a P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:52.633023 30996 tablet_bootstrap.cc:654] T ea563bd67a9849d18c9c3dfb21d58a93 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:52.629377 30997 tablet_bootstrap.cc:492] T f8f19b152a0f46ac896478f3f05c8822 P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
I20260709 13:11:52.634693 30947 raft_consensus.cc:2468] T 784deee1ea9543df840c58ecaa6f911a P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 1.
I20260709 13:11:52.637068 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0100ec8bd1e4a45b3c0ea52281f9204" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:52.639508 30799 raft_consensus.cc:2393] T c0100ec8bd1e4a45b3c0ea52281f9204 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d22c9d3d15904327a78a0e548a8ff6cd in current term 1: Already voted for candidate 05213ac836744a53b9c089a5c052f4be in this term.
I20260709 13:11:52.645058 30995 tablet_bootstrap.cc:654] T ac85f05e63ed463681b3ec790b39c4d8 P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:52.644603 30620 catalog_manager.cc:5697] T ac85f05e63ed463681b3ec790b39c4d8 P 02c5705b45c04706bbb2c6ff3d7b0b8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3). New cstate: current_term: 1 leader_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:52.651604 30994 ts_tablet_manager.cc:1434] T d16c44f56c42466497ba4f7f1b2b5344 P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.046s	user 0.006s	sys 0.000s
I20260709 13:11:52.655406 31002 leader_election.cc:290] T 65776989d6674454816411c251aff5c1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:52.656406 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65776989d6674454816411c251aff5c1" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:52.663663 30994 tablet_bootstrap.cc:492] T e2376eaea6bc4f209a21a110370da7b2 P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
I20260709 13:11:52.658357 31288 raft_consensus.cc:493] T 94220be3f55a4acbbd6bd5723b9f32e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:52.672572 30947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65776989d6674454816411c251aff5c1" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:52.662782 30689 leader_election.cc:304] T c0100ec8bd1e4a45b3c0ea52281f9204 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22c9d3d15904327a78a0e548a8ff6cd; no voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd
I20260709 13:11:52.680609 31279 raft_consensus.cc:3060] T c0100ec8bd1e4a45b3c0ea52281f9204 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:52.685052 30839 leader_election.cc:304] T 65776989d6674454816411c251aff5c1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a; no voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd
I20260709 13:11:52.668176 31288 raft_consensus.cc:515] T 94220be3f55a4acbbd6bd5723b9f32e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:52.686851 31233 raft_consensus.cc:2749] T 65776989d6674454816411c251aff5c1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:52.690498 31089 raft_consensus.cc:493] T d16c44f56c42466497ba4f7f1b2b5344 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:52.696724 31089 raft_consensus.cc:515] T d16c44f56c42466497ba4f7f1b2b5344 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:52.699012 31089 leader_election.cc:290] T d16c44f56c42466497ba4f7f1b2b5344 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:52.699434 30997 tablet_bootstrap.cc:654] T f8f19b152a0f46ac896478f3f05c8822 P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:52.703586 31288 leader_election.cc:290] T 94220be3f55a4acbbd6bd5723b9f32e9 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:52.709208 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16c44f56c42466497ba4f7f1b2b5344" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:52.709813 30799 raft_consensus.cc:2468] T d16c44f56c42466497ba4f7f1b2b5344 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 0.
I20260709 13:11:52.710978 30994 tablet_bootstrap.cc:654] T e2376eaea6bc4f209a21a110370da7b2 P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:52.712313 30912 leader_election.cc:304] T d16c44f56c42466497ba4f7f1b2b5344 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
I20260709 13:11:52.715593 30871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94220be3f55a4acbbd6bd5723b9f32e9" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:52.719975 31046 raft_consensus.cc:493] T 80a61882ef804a85a31e90d67930c5e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:52.720660 31082 raft_consensus.cc:2804] T d16c44f56c42466497ba4f7f1b2b5344 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:52.722548 31082 raft_consensus.cc:493] T d16c44f56c42466497ba4f7f1b2b5344 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:11:52.723165 30914 leader_election.cc:343] T 94220be3f55a4acbbd6bd5723b9f32e9 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:52.726723 31082 raft_consensus.cc:3060] T d16c44f56c42466497ba4f7f1b2b5344 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:52.726068 30725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16c44f56c42466497ba4f7f1b2b5344" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:52.728488 30722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94220be3f55a4acbbd6bd5723b9f32e9" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:52.721295 31046 raft_consensus.cc:515] T 80a61882ef804a85a31e90d67930c5e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:52.731532 31046 leader_election.cc:290] T 80a61882ef804a85a31e90d67930c5e9 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
W20260709 13:11:52.732853 30915 leader_election.cc:343] T d16c44f56c42466497ba4f7f1b2b5344 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:52.741707 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80a61882ef804a85a31e90d67930c5e9" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:52.743594 30799 raft_consensus.cc:2468] T 80a61882ef804a85a31e90d67930c5e9 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 0.
I20260709 13:11:52.746065 30996 tablet_bootstrap.cc:492] T ea563bd67a9849d18c9c3dfb21d58a93 P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
I20260709 13:11:52.745738 30871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80a61882ef804a85a31e90d67930c5e9" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
W20260709 13:11:52.746193 30915 leader_election.cc:343] T 94220be3f55a4acbbd6bd5723b9f32e9 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:52.746594 30996 ts_tablet_manager.cc:1403] T ea563bd67a9849d18c9c3dfb21d58a93 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.125s	user 0.014s	sys 0.003s
W20260709 13:11:52.747524 30914 leader_election.cc:343] T 80a61882ef804a85a31e90d67930c5e9 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:52.747455 30915 leader_election.cc:304] T 94220be3f55a4acbbd6bd5723b9f32e9 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:52.749408 30912 leader_election.cc:304] T 80a61882ef804a85a31e90d67930c5e9 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a
I20260709 13:11:52.745491 30624 catalog_manager.cc:5697] T 8b6052cddae64c5c9d16f60272735788 P 05213ac836744a53b9c089a5c052f4be reported cstate change: term changed from 0 to 1, leader changed from <none> to 05213ac836744a53b9c089a5c052f4be (127.21.63.2). New cstate: current_term: 1 leader_uuid: "05213ac836744a53b9c089a5c052f4be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:52.753458 31089 raft_consensus.cc:2749] T 94220be3f55a4acbbd6bd5723b9f32e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:52.754582 31177 raft_consensus.cc:2804] T 80a61882ef804a85a31e90d67930c5e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:52.755194 30996 raft_consensus.cc:359] T ea563bd67a9849d18c9c3dfb21d58a93 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:52.756237 30996 raft_consensus.cc:385] T ea563bd67a9849d18c9c3dfb21d58a93 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:52.756019 31177 raft_consensus.cc:493] T 80a61882ef804a85a31e90d67930c5e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:52.753684 31279 raft_consensus.cc:2749] T c0100ec8bd1e4a45b3c0ea52281f9204 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:52.757076 31177 raft_consensus.cc:3060] T 80a61882ef804a85a31e90d67930c5e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:52.756495 30996 raft_consensus.cc:740] T ea563bd67a9849d18c9c3dfb21d58a93 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
I20260709 13:11:52.760512 30996 consensus_queue.cc:260] T ea563bd67a9849d18c9c3dfb21d58a93 P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:52.764284 31082 raft_consensus.cc:515] T d16c44f56c42466497ba4f7f1b2b5344 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:52.766772 30997 tablet_bootstrap.cc:492] T f8f19b152a0f46ac896478f3f05c8822 P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
I20260709 13:11:52.771132 30997 ts_tablet_manager.cc:1403] T f8f19b152a0f46ac896478f3f05c8822 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.142s	user 0.013s	sys 0.003s
I20260709 13:11:52.762905 30996 ts_tablet_manager.cc:1434] T ea563bd67a9849d18c9c3dfb21d58a93 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 13:11:52.772756 30996 tablet_bootstrap.cc:492] T f8f19b152a0f46ac896478f3f05c8822 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
I20260709 13:11:52.776875 30997 raft_consensus.cc:359] T f8f19b152a0f46ac896478f3f05c8822 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:52.777566 30997 raft_consensus.cc:385] T f8f19b152a0f46ac896478f3f05c8822 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:52.777837 30997 raft_consensus.cc:740] T f8f19b152a0f46ac896478f3f05c8822 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
I20260709 13:11:52.778620 30997 consensus_queue.cc:260] T f8f19b152a0f46ac896478f3f05c8822 P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:52.780974 30997 ts_tablet_manager.cc:1434] T f8f19b152a0f46ac896478f3f05c8822 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:11:52.785579 30997 tablet_bootstrap.cc:492] T 8b3e41a728f1457ebf97274b95c7b23a P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
I20260709 13:11:52.786830 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16c44f56c42466497ba4f7f1b2b5344" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:52.787781 30799 raft_consensus.cc:3060] T d16c44f56c42466497ba4f7f1b2b5344 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:52.793296 31177 raft_consensus.cc:515] T 80a61882ef804a85a31e90d67930c5e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:52.795747 31082 leader_election.cc:290] T d16c44f56c42466497ba4f7f1b2b5344 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Requested vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:52.795789 30797 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80a61882ef804a85a31e90d67930c5e9" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:52.796222 30871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80a61882ef804a85a31e90d67930c5e9" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:11:52.796694 30797 raft_consensus.cc:3060] T 80a61882ef804a85a31e90d67930c5e9 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:11:52.797879 30914 leader_election.cc:343] T 80a61882ef804a85a31e90d67930c5e9 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:52.798290 30994 tablet_bootstrap.cc:492] T e2376eaea6bc4f209a21a110370da7b2 P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
I20260709 13:11:52.798784 30994 ts_tablet_manager.cc:1403] T e2376eaea6bc4f209a21a110370da7b2 P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.138s	user 0.012s	sys 0.002s
I20260709 13:11:52.799113 30995 tablet_bootstrap.cc:492] T ac85f05e63ed463681b3ec790b39c4d8 P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:52.799592 30995 ts_tablet_manager.cc:1403] T ac85f05e63ed463681b3ec790b39c4d8 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.218s	user 0.020s	sys 0.012s
I20260709 13:11:52.801683 30994 raft_consensus.cc:359] T e2376eaea6bc4f209a21a110370da7b2 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:52.802441 30994 raft_consensus.cc:385] T e2376eaea6bc4f209a21a110370da7b2 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:52.802892 30994 raft_consensus.cc:740] T e2376eaea6bc4f209a21a110370da7b2 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
I20260709 13:11:52.803638 30994 consensus_queue.cc:260] T e2376eaea6bc4f209a21a110370da7b2 P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:52.804482 30797 raft_consensus.cc:2468] T 80a61882ef804a85a31e90d67930c5e9 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 1.
I20260709 13:11:52.805785 30912 leader_election.cc:304] T 80a61882ef804a85a31e90d67930c5e9 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a
I20260709 13:11:52.809453 31288 raft_consensus.cc:2804] T 80a61882ef804a85a31e90d67930c5e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:52.810191 31288 raft_consensus.cc:697] T 80a61882ef804a85a31e90d67930c5e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Becoming Leader. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Running, Role: LEADER
I20260709 13:11:52.813298 31288 consensus_queue.cc:237] T 80a61882ef804a85a31e90d67930c5e9 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:52.819826 30722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16c44f56c42466497ba4f7f1b2b5344" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:52.814709 30871 raft_consensus.cc:3060] T 65776989d6674454816411c251aff5c1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:11:52.821563 30915 leader_election.cc:343] T d16c44f56c42466497ba4f7f1b2b5344 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:52.826218 31177 leader_election.cc:290] T 80a61882ef804a85a31e90d67930c5e9 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Requested vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:52.828022 30799 raft_consensus.cc:2468] T d16c44f56c42466497ba4f7f1b2b5344 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 1.
I20260709 13:11:52.802311 30995 raft_consensus.cc:359] T ac85f05e63ed463681b3ec790b39c4d8 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:52.829864 30995 raft_consensus.cc:385] T ac85f05e63ed463681b3ec790b39c4d8 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:52.830233 30995 raft_consensus.cc:740] T ac85f05e63ed463681b3ec790b39c4d8 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:52.830999 30995 consensus_queue.cc:260] T ac85f05e63ed463681b3ec790b39c4d8 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:52.831745 30994 ts_tablet_manager.cc:1434] T e2376eaea6bc4f209a21a110370da7b2 P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.000s
I20260709 13:11:52.832894 30994 tablet_bootstrap.cc:492] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
I20260709 13:11:52.833256 30995 ts_tablet_manager.cc:1434] T ac85f05e63ed463681b3ec790b39c4d8 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.033s	user 0.007s	sys 0.000s
I20260709 13:11:52.835108 30996 tablet_bootstrap.cc:654] T f8f19b152a0f46ac896478f3f05c8822 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:52.835933 30997 tablet_bootstrap.cc:654] T 8b3e41a728f1457ebf97274b95c7b23a P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:52.840739 30994 tablet_bootstrap.cc:654] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:52.845983 30995 tablet_bootstrap.cc:492] T af695b428b974773ba476dce0d31c5a1 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:52.878244 30912 leader_election.cc:304] T d16c44f56c42466497ba4f7f1b2b5344 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:52.890166 31089 raft_consensus.cc:2804] T d16c44f56c42466497ba4f7f1b2b5344 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:52.908739 31089 raft_consensus.cc:697] T d16c44f56c42466497ba4f7f1b2b5344 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Becoming Leader. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Running, Role: LEADER
I20260709 13:11:52.909601 31089 consensus_queue.cc:237] T d16c44f56c42466497ba4f7f1b2b5344 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:52.933279 30622 catalog_manager.cc:5697] T 784deee1ea9543df840c58ecaa6f911a P 02c5705b45c04706bbb2c6ff3d7b0b8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3). New cstate: current_term: 1 leader_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:52.955147 30994 tablet_bootstrap.cc:492] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
I20260709 13:11:52.955646 30994 ts_tablet_manager.cc:1403] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.123s	user 0.017s	sys 0.008s
I20260709 13:11:52.969448 30996 tablet_bootstrap.cc:492] T f8f19b152a0f46ac896478f3f05c8822 P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
I20260709 13:11:52.970088 30996 ts_tablet_manager.cc:1403] T f8f19b152a0f46ac896478f3f05c8822 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.198s	user 0.019s	sys 0.007s
I20260709 13:11:52.967185 30624 catalog_manager.cc:5697] T 80a61882ef804a85a31e90d67930c5e9 P 262942f67dfa4b8fa42234b18ac31ffd reported cstate change: term changed from 0 to 1, leader changed from <none> to 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4). New cstate: current_term: 1 leader_uuid: "262942f67dfa4b8fa42234b18ac31ffd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:52.977137 30994 raft_consensus.cc:359] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:52.977988 30994 raft_consensus.cc:385] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:52.978277 30994 raft_consensus.cc:740] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
I20260709 13:11:52.981943 31283 raft_consensus.cc:493] T af695b428b974773ba476dce0d31c5a1 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:52.982503 31283 raft_consensus.cc:515] T af695b428b974773ba476dce0d31c5a1 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:52.983762 30994 consensus_queue.cc:260] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:52.984603 31283 leader_election.cc:290] T af695b428b974773ba476dce0d31c5a1 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:52.989567 30722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af695b428b974773ba476dce0d31c5a1" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:52.990891 30874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af695b428b974773ba476dce0d31c5a1" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
W20260709 13:11:52.991099 30767 leader_election.cc:343] T af695b428b974773ba476dce0d31c5a1 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:52.991494 30874 raft_consensus.cc:2468] T af695b428b974773ba476dce0d31c5a1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05213ac836744a53b9c089a5c052f4be in term 0.
I20260709 13:11:52.992712 30766 leader_election.cc:304] T af695b428b974773ba476dce0d31c5a1 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 05213ac836744a53b9c089a5c052f4be; no voters: d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:52.993763 31109 raft_consensus.cc:2804] T af695b428b974773ba476dce0d31c5a1 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:52.994098 31109 raft_consensus.cc:493] T af695b428b974773ba476dce0d31c5a1 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:52.994397 31109 raft_consensus.cc:3060] T af695b428b974773ba476dce0d31c5a1 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:52.997352 30995 tablet_bootstrap.cc:654] T af695b428b974773ba476dce0d31c5a1 P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:52.999639 31283 raft_consensus.cc:493] T 6fe93136933f4d2880ec355e1bbbf90d P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.000188 31283 raft_consensus.cc:515] T 6fe93136933f4d2880ec355e1bbbf90d P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:52.989552 30994 ts_tablet_manager.cc:1434] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.034s	user 0.004s	sys 0.004s
I20260709 13:11:52.990474 30996 raft_consensus.cc:359] T f8f19b152a0f46ac896478f3f05c8822 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:53.005441 30996 raft_consensus.cc:385] T f8f19b152a0f46ac896478f3f05c8822 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:53.005746 30996 raft_consensus.cc:740] T f8f19b152a0f46ac896478f3f05c8822 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
I20260709 13:11:53.006951 30944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fe93136933f4d2880ec355e1bbbf90d" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:53.007788 30994 tablet_bootstrap.cc:492] T cae2b1cb8db54cbb812489cc1b4b1213 P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
I20260709 13:11:53.007688 30944 raft_consensus.cc:2468] T 6fe93136933f4d2880ec355e1bbbf90d P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05213ac836744a53b9c089a5c052f4be in term 0.
I20260709 13:11:53.012548 30765 leader_election.cc:304] T 6fe93136933f4d2880ec355e1bbbf90d P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
I20260709 13:11:53.013916 30996 consensus_queue.cc:260] T f8f19b152a0f46ac896478f3f05c8822 P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:53.014216 30722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fe93136933f4d2880ec355e1bbbf90d" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:53.016274 30996 ts_tablet_manager.cc:1434] T f8f19b152a0f46ac896478f3f05c8822 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.046s	user 0.005s	sys 0.001s
I20260709 13:11:53.009485 31109 raft_consensus.cc:515] T af695b428b974773ba476dce0d31c5a1 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
W20260709 13:11:53.016453 30767 leader_election.cc:343] T 6fe93136933f4d2880ec355e1bbbf90d P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:53.015201 31283 leader_election.cc:290] T 6fe93136933f4d2880ec355e1bbbf90d P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:53.017581 31283 raft_consensus.cc:2804] T 6fe93136933f4d2880ec355e1bbbf90d P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:53.018663 30725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af695b428b974773ba476dce0d31c5a1" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:53.022035 31283 raft_consensus.cc:493] T 6fe93136933f4d2880ec355e1bbbf90d P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:53.022812 31283 raft_consensus.cc:3060] T 6fe93136933f4d2880ec355e1bbbf90d P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:53.023785 30997 tablet_bootstrap.cc:492] T 8b3e41a728f1457ebf97274b95c7b23a P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
I20260709 13:11:53.024367 30997 ts_tablet_manager.cc:1403] T 8b3e41a728f1457ebf97274b95c7b23a P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.239s	user 0.029s	sys 0.017s
W20260709 13:11:53.026678 30767 leader_election.cc:343] T af695b428b974773ba476dce0d31c5a1 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:53.027240 30997 raft_consensus.cc:359] T 8b3e41a728f1457ebf97274b95c7b23a P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:53.028152 30997 raft_consensus.cc:385] T 8b3e41a728f1457ebf97274b95c7b23a P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:53.028514 30997 raft_consensus.cc:740] T 8b3e41a728f1457ebf97274b95c7b23a P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
I20260709 13:11:53.029445 30996 tablet_bootstrap.cc:492] T d5175e0d898c453b8a7641d94dc1ac58 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
I20260709 13:11:53.033143 31109 leader_election.cc:290] T af695b428b974773ba476dce0d31c5a1 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 election: Requested vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:53.033313 30997 consensus_queue.cc:260] T 8b3e41a728f1457ebf97274b95c7b23a P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:53.038266 30997 ts_tablet_manager.cc:1434] T 8b3e41a728f1457ebf97274b95c7b23a P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 13:11:53.040252 30997 tablet_bootstrap.cc:492] T 99e98c1324084d85b8a97aaf0c248df7 P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
I20260709 13:11:53.044219 30996 tablet_bootstrap.cc:654] T d5175e0d898c453b8a7641d94dc1ac58 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:53.045729 30874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af695b428b974773ba476dce0d31c5a1" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:11:53.046326 30874 raft_consensus.cc:3060] T af695b428b974773ba476dce0d31c5a1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:53.048503 30994 tablet_bootstrap.cc:654] T cae2b1cb8db54cbb812489cc1b4b1213 P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:53.049178 30624 catalog_manager.cc:5697] T d16c44f56c42466497ba4f7f1b2b5344 P 262942f67dfa4b8fa42234b18ac31ffd reported cstate change: term changed from 0 to 1, leader changed from <none> to 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4). New cstate: current_term: 1 leader_uuid: "262942f67dfa4b8fa42234b18ac31ffd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:53.051225 30995 tablet_bootstrap.cc:492] T af695b428b974773ba476dce0d31c5a1 P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:53.051719 30995 ts_tablet_manager.cc:1403] T af695b428b974773ba476dce0d31c5a1 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.206s	user 0.018s	sys 0.001s
I20260709 13:11:53.054571 30995 raft_consensus.cc:359] T af695b428b974773ba476dce0d31c5a1 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:53.055207 30995 raft_consensus.cc:385] T af695b428b974773ba476dce0d31c5a1 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:53.055485 30995 raft_consensus.cc:740] T af695b428b974773ba476dce0d31c5a1 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:53.056077 30995 consensus_queue.cc:260] T af695b428b974773ba476dce0d31c5a1 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:53.051744 30997 tablet_bootstrap.cc:654] T 99e98c1324084d85b8a97aaf0c248df7 P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:53.058156 30995 ts_tablet_manager.cc:1434] T af695b428b974773ba476dce0d31c5a1 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:11:53.059128 30995 tablet_bootstrap.cc:492] T 8b6052cddae64c5c9d16f60272735788 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
W20260709 13:11:53.061841 30765 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.4:41313, user_credentials={real_user=slave}} blocked reactor thread for 49387.8us
I20260709 13:11:53.072238 30995 tablet_bootstrap.cc:654] T 8b6052cddae64c5c9d16f60272735788 P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:53.073020 31283 raft_consensus.cc:515] T 6fe93136933f4d2880ec355e1bbbf90d P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:53.073511 30874 raft_consensus.cc:2468] T af695b428b974773ba476dce0d31c5a1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05213ac836744a53b9c089a5c052f4be in term 1.
I20260709 13:11:53.075245 31066 raft_consensus.cc:493] T ac85f05e63ed463681b3ec790b39c4d8 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.085340 31066 raft_consensus.cc:515] T ac85f05e63ed463681b3ec790b39c4d8 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:53.087261 31066 leader_election.cc:290] T ac85f05e63ed463681b3ec790b39c4d8 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:53.087749 30947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fe93136933f4d2880ec355e1bbbf90d" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:11:53.088371 30947 raft_consensus.cc:3060] T 6fe93136933f4d2880ec355e1bbbf90d P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:53.089205 30944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac85f05e63ed463681b3ec790b39c4d8" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:53.090003 30944 raft_consensus.cc:2393] T ac85f05e63ed463681b3ec790b39c4d8 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d22c9d3d15904327a78a0e548a8ff6cd in current term 1: Already voted for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in this term.
I20260709 13:11:53.074115 30996 tablet_bootstrap.cc:492] T d5175e0d898c453b8a7641d94dc1ac58 P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
I20260709 13:11:53.078351 31357 consensus_queue.cc:1048] T ac85f05e63ed463681b3ec790b39c4d8 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:53.079295 31109 raft_consensus.cc:493] T e2376eaea6bc4f209a21a110370da7b2 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.093289 31109 raft_consensus.cc:515] T e2376eaea6bc4f209a21a110370da7b2 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:53.094913 30875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac85f05e63ed463681b3ec790b39c4d8" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:53.097643 31109 leader_election.cc:290] T e2376eaea6bc4f209a21a110370da7b2 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:53.092413 30996 ts_tablet_manager.cc:1403] T d5175e0d898c453b8a7641d94dc1ac58 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.063s	user 0.010s	sys 0.004s
I20260709 13:11:53.098333 30691 leader_election.cc:304] T ac85f05e63ed463681b3ec790b39c4d8 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22c9d3d15904327a78a0e548a8ff6cd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 262942f67dfa4b8fa42234b18ac31ffd
I20260709 13:11:53.101387 30994 tablet_bootstrap.cc:492] T cae2b1cb8db54cbb812489cc1b4b1213 P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
I20260709 13:11:53.102049 30994 ts_tablet_manager.cc:1403] T cae2b1cb8db54cbb812489cc1b4b1213 P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.098s	user 0.007s	sys 0.009s
I20260709 13:11:53.096922 30725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2376eaea6bc4f209a21a110370da7b2" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:53.102551 31066 raft_consensus.cc:3060] T ac85f05e63ed463681b3ec790b39c4d8 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:53.104888 31283 leader_election.cc:290] T 6fe93136933f4d2880ec355e1bbbf90d P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 election: Requested vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:53.104491 30766 leader_election.cc:304] T af695b428b974773ba476dce0d31c5a1 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 05213ac836744a53b9c089a5c052f4be; no voters: d22c9d3d15904327a78a0e548a8ff6cd
W20260709 13:11:53.105271 30767 leader_election.cc:343] T e2376eaea6bc4f209a21a110370da7b2 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:53.104560 30722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fe93136933f4d2880ec355e1bbbf90d" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:53.105886 31283 raft_consensus.cc:2804] T af695b428b974773ba476dce0d31c5a1 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:53.106346 31283 raft_consensus.cc:697] T af695b428b974773ba476dce0d31c5a1 P 05213ac836744a53b9c089a5c052f4be [term 1 LEADER]: Becoming Leader. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Running, Role: LEADER
W20260709 13:11:53.106873 30767 leader_election.cc:343] T 6fe93136933f4d2880ec355e1bbbf90d P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:53.107151 31283 consensus_queue.cc:237] T af695b428b974773ba476dce0d31c5a1 P 05213ac836744a53b9c089a5c052f4be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:53.107704 30874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2376eaea6bc4f209a21a110370da7b2" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
W20260709 13:11:53.110316 30766 leader_election.cc:343] T e2376eaea6bc4f209a21a110370da7b2 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:53.110747 30766 leader_election.cc:304] T e2376eaea6bc4f209a21a110370da7b2 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:53.115967 31373 raft_consensus.cc:2749] T e2376eaea6bc4f209a21a110370da7b2 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:53.117012 30996 raft_consensus.cc:359] T d5175e0d898c453b8a7641d94dc1ac58 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:53.117693 30996 raft_consensus.cc:385] T d5175e0d898c453b8a7641d94dc1ac58 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:53.118170 30996 raft_consensus.cc:740] T d5175e0d898c453b8a7641d94dc1ac58 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
I20260709 13:11:53.119127 30994 raft_consensus.cc:359] T cae2b1cb8db54cbb812489cc1b4b1213 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:53.119817 30994 raft_consensus.cc:385] T cae2b1cb8db54cbb812489cc1b4b1213 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:53.120088 30994 raft_consensus.cc:740] T cae2b1cb8db54cbb812489cc1b4b1213 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
I20260709 13:11:53.111335 31109 raft_consensus.cc:493] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.120883 31109 raft_consensus.cc:515] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:53.122270 30994 consensus_queue.cc:260] T cae2b1cb8db54cbb812489cc1b4b1213 P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:53.125840 31066 raft_consensus.cc:2749] T ac85f05e63ed463681b3ec790b39c4d8 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:53.130862 30997 tablet_bootstrap.cc:492] T 99e98c1324084d85b8a97aaf0c248df7 P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
I20260709 13:11:53.131564 30997 ts_tablet_manager.cc:1403] T 99e98c1324084d85b8a97aaf0c248df7 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.091s	user 0.002s	sys 0.013s
I20260709 13:11:53.120862 30996 consensus_queue.cc:260] T d5175e0d898c453b8a7641d94dc1ac58 P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:53.115600 30947 raft_consensus.cc:2468] T 6fe93136933f4d2880ec355e1bbbf90d P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05213ac836744a53b9c089a5c052f4be in term 1.
I20260709 13:11:53.134246 30765 leader_election.cc:304] T 6fe93136933f4d2880ec355e1bbbf90d P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:53.135903 31373 raft_consensus.cc:2804] T 6fe93136933f4d2880ec355e1bbbf90d P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:53.140883 31109 leader_election.cc:290] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:53.146355 30947 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4dcd4ecc9840d2b8785d14dd4537b8" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:53.146867 30874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4dcd4ecc9840d2b8785d14dd4537b8" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:53.154085 30995 tablet_bootstrap.cc:492] T 8b6052cddae64c5c9d16f60272735788 P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:53.154575 30995 ts_tablet_manager.cc:1403] T 8b6052cddae64c5c9d16f60272735788 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.096s	user 0.018s	sys 0.011s
I20260709 13:11:53.156308 30997 raft_consensus.cc:359] T 99e98c1324084d85b8a97aaf0c248df7 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:53.157192 30997 raft_consensus.cc:385] T 99e98c1324084d85b8a97aaf0c248df7 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:53.157552 30997 raft_consensus.cc:740] T 99e98c1324084d85b8a97aaf0c248df7 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
I20260709 13:11:53.161466 30996 ts_tablet_manager.cc:1434] T d5175e0d898c453b8a7641d94dc1ac58 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.063s	user 0.007s	sys 0.000s
I20260709 13:11:53.164395 31089 raft_consensus.cc:493] T f8f19b152a0f46ac896478f3f05c8822 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.164716 31002 consensus_queue.cc:1048] T ac85f05e63ed463681b3ec790b39c4d8 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:53.164939 31089 raft_consensus.cc:515] T f8f19b152a0f46ac896478f3f05c8822 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:53.150309 31373 raft_consensus.cc:697] T 6fe93136933f4d2880ec355e1bbbf90d P 05213ac836744a53b9c089a5c052f4be [term 1 LEADER]: Becoming Leader. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Running, Role: LEADER
I20260709 13:11:53.166208 31373 consensus_queue.cc:237] T 6fe93136933f4d2880ec355e1bbbf90d P 05213ac836744a53b9c089a5c052f4be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:53.167039 31089 leader_election.cc:290] T f8f19b152a0f46ac896478f3f05c8822 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
W20260709 13:11:53.154358 30765 leader_election.cc:343] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:53.167773 30997 consensus_queue.cc:260] T 99e98c1324084d85b8a97aaf0c248df7 P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:53.169418 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8f19b152a0f46ac896478f3f05c8822" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:53.170140 30799 raft_consensus.cc:2468] T f8f19b152a0f46ac896478f3f05c8822 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 0.
I20260709 13:11:53.174120 30912 leader_election.cc:304] T f8f19b152a0f46ac896478f3f05c8822 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
I20260709 13:11:53.175227 31363 raft_consensus.cc:2804] T f8f19b152a0f46ac896478f3f05c8822 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:53.175572 31363 raft_consensus.cc:493] T f8f19b152a0f46ac896478f3f05c8822 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:53.175702 30997 ts_tablet_manager.cc:1434] T 99e98c1324084d85b8a97aaf0c248df7 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.044s	user 0.006s	sys 0.002s
I20260709 13:11:53.175941 31363 raft_consensus.cc:3060] T f8f19b152a0f46ac896478f3f05c8822 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:53.176782 30997 tablet_bootstrap.cc:492] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
I20260709 13:11:53.178656 30874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8f19b152a0f46ac896478f3f05c8822" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:53.179343 30874 raft_consensus.cc:2468] T f8f19b152a0f46ac896478f3f05c8822 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 0.
I20260709 13:11:53.181250 31357 raft_consensus.cc:493] T ea563bd67a9849d18c9c3dfb21d58a93 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.181751 31357 raft_consensus.cc:515] T ea563bd67a9849d18c9c3dfb21d58a93 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:53.183794 31357 leader_election.cc:290] T ea563bd67a9849d18c9c3dfb21d58a93 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:53.186409 30997 tablet_bootstrap.cc:654] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
W20260709 13:11:53.170301 30766 leader_election.cc:343] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:53.188256 30944 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea563bd67a9849d18c9c3dfb21d58a93" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:53.189348 30994 ts_tablet_manager.cc:1434] T cae2b1cb8db54cbb812489cc1b4b1213 P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.087s	user 0.005s	sys 0.003s
I20260709 13:11:53.195199 30996 tablet_bootstrap.cc:492] T 80a61882ef804a85a31e90d67930c5e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
I20260709 13:11:53.193585 30766 leader_election.cc:304] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 262942f67dfa4b8fa42234b18ac31ffd
I20260709 13:11:53.199910 30726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea563bd67a9849d18c9c3dfb21d58a93" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:53.201206 30623 catalog_manager.cc:5697] T af695b428b974773ba476dce0d31c5a1 P 05213ac836744a53b9c089a5c052f4be reported cstate change: term changed from 0 to 1, leader changed from <none> to 05213ac836744a53b9c089a5c052f4be (127.21.63.2). New cstate: current_term: 1 leader_uuid: "05213ac836744a53b9c089a5c052f4be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:53.193050 30944 raft_consensus.cc:2468] T ea563bd67a9849d18c9c3dfb21d58a93 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 0.
I20260709 13:11:53.204504 30839 leader_election.cc:304] T ea563bd67a9849d18c9c3dfb21d58a93 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
I20260709 13:11:53.206487 31381 raft_consensus.cc:2749] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:53.209367 30995 raft_consensus.cc:359] T 8b6052cddae64c5c9d16f60272735788 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:53.210060 30995 raft_consensus.cc:385] T 8b6052cddae64c5c9d16f60272735788 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:53.210312 31240 raft_consensus.cc:2804] T ea563bd67a9849d18c9c3dfb21d58a93 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:53.210769 31240 raft_consensus.cc:493] T ea563bd67a9849d18c9c3dfb21d58a93 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:53.211753 30994 tablet_bootstrap.cc:492] T f63e902c7c7a4d0fa5d2107620ce851e P 05213ac836744a53b9c089a5c052f4be: Bootstrap starting.
I20260709 13:11:53.212507 31239 consensus_queue.cc:1048] T 784deee1ea9543df840c58ecaa6f911a P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:53.211140 31240 raft_consensus.cc:3060] T ea563bd67a9849d18c9c3dfb21d58a93 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:53.210314 30995 raft_consensus.cc:740] T 8b6052cddae64c5c9d16f60272735788 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:53.222031 31279 raft_consensus.cc:493] T af695b428b974773ba476dce0d31c5a1 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.222532 31279 raft_consensus.cc:515] T af695b428b974773ba476dce0d31c5a1 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:53.226533 31363 raft_consensus.cc:515] T f8f19b152a0f46ac896478f3f05c8822 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:53.235816 30995 consensus_queue.cc:260] T 8b6052cddae64c5c9d16f60272735788 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:53.243297 30995 ts_tablet_manager.cc:1434] T 8b6052cddae64c5c9d16f60272735788 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.088s	user 0.008s	sys 0.000s
I20260709 13:11:53.262504 30726 raft_consensus.cc:3060] T 8b6052cddae64c5c9d16f60272735788 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:53.265686 30996 tablet_bootstrap.cc:654] T 80a61882ef804a85a31e90d67930c5e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:53.280748 30994 tablet_bootstrap.cc:654] T f63e902c7c7a4d0fa5d2107620ce851e P 05213ac836744a53b9c089a5c052f4be: Neither blocks nor log segments found. Creating new log.
W20260709 13:11:53.282442 30671 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:11:53.252990 31288 raft_consensus.cc:493] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 13:11:53.256865 30841 leader_election.cc:343] T ea563bd67a9849d18c9c3dfb21d58a93 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:53.291786 30995 tablet_bootstrap.cc:492] T ea563bd67a9849d18c9c3dfb21d58a93 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:53.385407 31363 leader_election.cc:290] T f8f19b152a0f46ac896478f3f05c8822 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Requested vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:53.386021 31283 raft_consensus.cc:493] T c0100ec8bd1e4a45b3c0ea52281f9204 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.386581 31283 raft_consensus.cc:515] T c0100ec8bd1e4a45b3c0ea52281f9204 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:53.365689 31279 leader_election.cc:290] T af695b428b974773ba476dce0d31c5a1 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:53.295935 31288 raft_consensus.cc:515] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:53.388594 31283 leader_election.cc:290] T c0100ec8bd1e4a45b3c0ea52281f9204 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
W20260709 13:11:53.372504 30841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:37325, user_credentials={real_user=slave}} blocked reactor thread for 118356us
I20260709 13:11:53.422907 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af695b428b974773ba476dce0d31c5a1" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:53.440699 30875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af695b428b974773ba476dce0d31c5a1" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:53.441681 30875 raft_consensus.cc:2393] T af695b428b974773ba476dce0d31c5a1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d22c9d3d15904327a78a0e548a8ff6cd in current term 1: Already voted for candidate 05213ac836744a53b9c089a5c052f4be in this term.
I20260709 13:11:53.444828 31283 raft_consensus.cc:493] T cae2b1cb8db54cbb812489cc1b4b1213 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.445415 31283 raft_consensus.cc:515] T cae2b1cb8db54cbb812489cc1b4b1213 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:53.453059 30691 leader_election.cc:304] T af695b428b974773ba476dce0d31c5a1 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22c9d3d15904327a78a0e548a8ff6cd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 05213ac836744a53b9c089a5c052f4be
I20260709 13:11:53.454197 31069 raft_consensus.cc:3060] T af695b428b974773ba476dce0d31c5a1 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:53.449052 30622 catalog_manager.cc:5697] T 6fe93136933f4d2880ec355e1bbbf90d P 05213ac836744a53b9c089a5c052f4be reported cstate change: term changed from 0 to 1, leader changed from <none> to 05213ac836744a53b9c089a5c052f4be (127.21.63.2). New cstate: current_term: 1 leader_uuid: "05213ac836744a53b9c089a5c052f4be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:53.446326 30800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8f19b152a0f46ac896478f3f05c8822" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:53.461766 30997 tablet_bootstrap.cc:492] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
I20260709 13:11:53.462360 30997 ts_tablet_manager.cc:1403] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.286s	user 0.045s	sys 0.028s
W20260709 13:11:53.451148 31380 log.cc:927] Time spent T af695b428b974773ba476dce0d31c5a1 P 05213ac836744a53b9c089a5c052f4be: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 13:11:53.451969 31373 consensus_queue.cc:1048] T 8b6052cddae64c5c9d16f60272735788 P 05213ac836744a53b9c089a5c052f4be [LEADER]: Connected to new peer: Peer: permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260709 13:11:53.471408 31066 raft_consensus.cc:493] T 8b6052cddae64c5c9d16f60272735788 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.471877 31066 raft_consensus.cc:515] T 8b6052cddae64c5c9d16f60272735788 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:53.474431 31240 raft_consensus.cc:515] T ea563bd67a9849d18c9c3dfb21d58a93 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:53.475484 31066 leader_election.cc:290] T 8b6052cddae64c5c9d16f60272735788 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
W20260709 13:11:53.476230 31387 log.cc:927] Time spent T 6fe93136933f4d2880ec355e1bbbf90d P 05213ac836744a53b9c089a5c052f4be: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 13:11:53.468529 31288 leader_election.cc:290] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:53.482470 31363 raft_consensus.cc:493] T 8b3e41a728f1457ebf97274b95c7b23a P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.462469 30800 raft_consensus.cc:3060] T f8f19b152a0f46ac896478f3f05c8822 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:53.497335 30994 tablet_bootstrap.cc:492] T f63e902c7c7a4d0fa5d2107620ce851e P 05213ac836744a53b9c089a5c052f4be: No bootstrap required, opened a new log
I20260709 13:11:53.498056 30994 ts_tablet_manager.cc:1403] T f63e902c7c7a4d0fa5d2107620ce851e P 05213ac836744a53b9c089a5c052f4be: Time spent bootstrapping tablet: real 0.287s	user 0.017s	sys 0.004s
I20260709 13:11:53.499752 30723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0100ec8bd1e4a45b3c0ea52281f9204" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:53.500563 30723 raft_consensus.cc:2468] T c0100ec8bd1e4a45b3c0ea52281f9204 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 1.
I20260709 13:11:53.505573 30949 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0100ec8bd1e4a45b3c0ea52281f9204" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:53.508014 30949 raft_consensus.cc:2468] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05213ac836744a53b9c089a5c052f4be in term 1.
I20260709 13:11:53.483184 31363 raft_consensus.cc:515] T 8b3e41a728f1457ebf97274b95c7b23a P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:53.513655 30994 raft_consensus.cc:359] T f63e902c7c7a4d0fa5d2107620ce851e P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:53.517086 30994 raft_consensus.cc:385] T f63e902c7c7a4d0fa5d2107620ce851e P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 13:11:53.514655 30691 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:43993, user_credentials={real_user=slave}} blocked reactor thread for 52706.3us
I20260709 13:11:53.488628 31240 leader_election.cc:290] T ea563bd67a9849d18c9c3dfb21d58a93 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Requested vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:53.513221 30995 tablet_bootstrap.cc:654] T ea563bd67a9849d18c9c3dfb21d58a93 P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:53.517489 30994 raft_consensus.cc:740] T f63e902c7c7a4d0fa5d2107620ce851e P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Initialized, Role: FOLLOWER
I20260709 13:11:53.520993 30997 raft_consensus.cc:359] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:53.521705 30997 raft_consensus.cc:385] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:53.522081 30997 raft_consensus.cc:740] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
I20260709 13:11:53.522670 30997 consensus_queue.cc:260] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:53.525890 30723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0100ec8bd1e4a45b3c0ea52281f9204" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:53.524910 30994 consensus_queue.cc:260] T f63e902c7c7a4d0fa5d2107620ce851e P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:53.526607 30723 raft_consensus.cc:2468] T c0100ec8bd1e4a45b3c0ea52281f9204 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05213ac836744a53b9c089a5c052f4be in term 1.
I20260709 13:11:53.527195 31283 leader_election.cc:290] T cae2b1cb8db54cbb812489cc1b4b1213 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:53.529230 30943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea563bd67a9849d18c9c3dfb21d58a93" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:11:53.529961 30943 raft_consensus.cc:3060] T ea563bd67a9849d18c9c3dfb21d58a93 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:53.530436 31389 raft_consensus.cc:493] T 99e98c1324084d85b8a97aaf0c248df7 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.530923 31389 raft_consensus.cc:515] T 99e98c1324084d85b8a97aaf0c248df7 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:53.533460 30996 tablet_bootstrap.cc:492] T 80a61882ef804a85a31e90d67930c5e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
I20260709 13:11:53.533993 30996 ts_tablet_manager.cc:1403] T 80a61882ef804a85a31e90d67930c5e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.345s	user 0.020s	sys 0.004s
I20260709 13:11:53.540169 31363 leader_election.cc:290] T 8b3e41a728f1457ebf97274b95c7b23a P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:53.514271 30915 leader_election.cc:304] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:53.545091 30795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6052cddae64c5c9d16f60272735788" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:53.547138 31389 leader_election.cc:290] T 99e98c1324084d85b8a97aaf0c248df7 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:53.550163 30994 ts_tablet_manager.cc:1434] T f63e902c7c7a4d0fa5d2107620ce851e P 05213ac836744a53b9c089a5c052f4be: Time spent starting tablet: real 0.052s	user 0.008s	sys 0.000s
I20260709 13:11:53.551846 30765 leader_election.cc:304] T c0100ec8bd1e4a45b3c0ea52281f9204 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
I20260709 13:11:53.544919 30996 raft_consensus.cc:359] T 80a61882ef804a85a31e90d67930c5e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:53.556844 30996 raft_consensus.cc:385] T 80a61882ef804a85a31e90d67930c5e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:53.557215 30996 raft_consensus.cc:740] T 80a61882ef804a85a31e90d67930c5e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
I20260709 13:11:53.561168 31381 raft_consensus.cc:2804] T c0100ec8bd1e4a45b3c0ea52281f9204 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:53.561595 31381 raft_consensus.cc:493] T c0100ec8bd1e4a45b3c0ea52281f9204 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:53.561904 31381 raft_consensus.cc:3060] T c0100ec8bd1e4a45b3c0ea52281f9204 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:53.564917 30915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:37325, user_credentials={real_user=slave}} blocked reactor thread for 50725us
I20260709 13:11:53.565601 31288 raft_consensus.cc:2804] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:53.566051 31288 raft_consensus.cc:493] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:53.566414 31288 raft_consensus.cc:3060] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:53.568239 30800 raft_consensus.cc:2468] T f8f19b152a0f46ac896478f3f05c8822 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 1.
I20260709 13:11:53.569073 30874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae2b1cb8db54cbb812489cc1b4b1213" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:53.574116 30796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0100ec8bd1e4a45b3c0ea52281f9204" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:53.564641 30997 ts_tablet_manager.cc:1434] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.102s	user 0.004s	sys 0.003s
I20260709 13:11:53.578310 30912 leader_election.cc:304] T f8f19b152a0f46ac896478f3f05c8822 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
I20260709 13:11:53.580861 31069 raft_consensus.cc:2749] T af695b428b974773ba476dce0d31c5a1 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:53.579946 30996 consensus_queue.cc:260] T 80a61882ef804a85a31e90d67930c5e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:53.584069 31178 raft_consensus.cc:2804] T f8f19b152a0f46ac896478f3f05c8822 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:53.585513 30997 tablet_bootstrap.cc:492] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd: Bootstrap starting.
I20260709 13:11:53.587611 31373 consensus_queue.cc:1048] T 8b6052cddae64c5c9d16f60272735788 P 05213ac836744a53b9c089a5c052f4be [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:53.589692 31178 raft_consensus.cc:697] T f8f19b152a0f46ac896478f3f05c8822 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Becoming Leader. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Running, Role: LEADER
I20260709 13:11:53.590610 31178 consensus_queue.cc:237] T f8f19b152a0f46ac896478f3f05c8822 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:53.596454 31288 raft_consensus.cc:515] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:53.598930 31288 leader_election.cc:290] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 2 election: Requested vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:53.579427 30721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b3e41a728f1457ebf97274b95c7b23a" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:53.603819 30997 tablet_bootstrap.cc:654] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:53.609709 30725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae2b1cb8db54cbb812489cc1b4b1213" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:53.610805 31239 raft_consensus.cc:493] T d5175e0d898c453b8a7641d94dc1ac58 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.610983 31363 consensus_queue.cc:1048] T 80a61882ef804a85a31e90d67930c5e9 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:53.611313 31239 raft_consensus.cc:515] T d5175e0d898c453b8a7641d94dc1ac58 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:53.616127 30996 ts_tablet_manager.cc:1434] T 80a61882ef804a85a31e90d67930c5e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.082s	user 0.008s	sys 0.000s
I20260709 13:11:53.620831 31239 leader_election.cc:290] T d5175e0d898c453b8a7641d94dc1ac58 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:53.622087 30722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e98c1324084d85b8a97aaf0c248df7" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
W20260709 13:11:53.583535 30766 leader_election.cc:343] T cae2b1cb8db54cbb812489cc1b4b1213 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:53.583374 30948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6052cddae64c5c9d16f60272735788" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:53.630524 30690 leader_election.cc:304] T 8b6052cddae64c5c9d16f60272735788 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22c9d3d15904327a78a0e548a8ff6cd; no voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd
I20260709 13:11:53.632169 31390 raft_consensus.cc:2749] T 8b6052cddae64c5c9d16f60272735788 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:53.633508 30721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea563bd67a9849d18c9c3dfb21d58a93" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
W20260709 13:11:53.634236 30767 leader_election.cc:343] T cae2b1cb8db54cbb812489cc1b4b1213 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:53.634891 30767 leader_election.cc:304] T cae2b1cb8db54cbb812489cc1b4b1213 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd
W20260709 13:11:53.606702 30915 consensus_peers.cc:597] T d16c44f56c42466497ba4f7f1b2b5344 P 262942f67dfa4b8fa42234b18ac31ffd -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:53.636103 30915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:37325, user_credentials={real_user=slave}} blocked reactor thread for 51929.4us
I20260709 13:11:53.642060 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5175e0d898c453b8a7641d94dc1ac58" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:53.642688 30799 raft_consensus.cc:2468] T d5175e0d898c453b8a7641d94dc1ac58 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 0.
I20260709 13:11:53.644634 31373 raft_consensus.cc:2749] T cae2b1cb8db54cbb812489cc1b4b1213 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:53.652395 30943 raft_consensus.cc:2468] T ea563bd67a9849d18c9c3dfb21d58a93 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 1.
I20260709 13:11:53.654980 31233 consensus_queue.cc:1048] T 784deee1ea9543df840c58ecaa6f911a P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:53.656422 30996 tablet_bootstrap.cc:492] T 94220be3f55a4acbbd6bd5723b9f32e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
I20260709 13:11:53.658255 30871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f8f19b152a0f46ac896478f3f05c8822" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:11:53.658866 30871 raft_consensus.cc:3060] T f8f19b152a0f46ac896478f3f05c8822 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:53.660211 30874 raft_consensus.cc:3060] T 80a61882ef804a85a31e90d67930c5e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:53.661855 30872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e98c1324084d85b8a97aaf0c248df7" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
W20260709 13:11:53.665706 30914 leader_election.cc:343] T 99e98c1324084d85b8a97aaf0c248df7 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:53.674115 30839 leader_election.cc:304] T ea563bd67a9849d18c9c3dfb21d58a93 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
I20260709 13:11:53.675078 31240 raft_consensus.cc:2804] T ea563bd67a9849d18c9c3dfb21d58a93 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:53.675932 30996 tablet_bootstrap.cc:654] T 94220be3f55a4acbbd6bd5723b9f32e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:53.677249 31240 raft_consensus.cc:697] T ea563bd67a9849d18c9c3dfb21d58a93 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Becoming Leader. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Running, Role: LEADER
I20260709 13:11:53.678261 31240 consensus_queue.cc:237] T ea563bd67a9849d18c9c3dfb21d58a93 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:53.679527 30872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b3e41a728f1457ebf97274b95c7b23a" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:53.681620 30871 raft_consensus.cc:2468] T f8f19b152a0f46ac896478f3f05c8822 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 1.
W20260709 13:11:53.682595 30767 consensus_peers.cc:597] T 6fe93136933f4d2880ec355e1bbbf90d P 05213ac836744a53b9c089a5c052f4be -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:53.685111 30841 leader_election.cc:343] T ea563bd67a9849d18c9c3dfb21d58a93 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:53.686755 30725 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0100ec8bd1e4a45b3c0ea52281f9204" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:53.687300 31373 consensus_queue.cc:1048] T af695b428b974773ba476dce0d31c5a1 P 05213ac836744a53b9c089a5c052f4be [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:53.687486 30725 raft_consensus.cc:3060] T c0100ec8bd1e4a45b3c0ea52281f9204 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:53.695521 30995 tablet_bootstrap.cc:492] T ea563bd67a9849d18c9c3dfb21d58a93 P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:53.696405 30995 ts_tablet_manager.cc:1403] T ea563bd67a9849d18c9c3dfb21d58a93 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.425s	user 0.017s	sys 0.024s
I20260709 13:11:53.705775 30997 tablet_bootstrap.cc:492] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd: No bootstrap required, opened a new log
I20260709 13:11:53.708923 30997 ts_tablet_manager.cc:1403] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd: Time spent bootstrapping tablet: real 0.124s	user 0.018s	sys 0.002s
I20260709 13:11:53.717094 30722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5175e0d898c453b8a7641d94dc1ac58" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
W20260709 13:11:53.720230 30914 leader_election.cc:343] T 8b3e41a728f1457ebf97274b95c7b23a P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:53.722925 31288 consensus_queue.cc:1048] T 80a61882ef804a85a31e90d67930c5e9 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:11:53.739615 30838 leader_election.cc:304] T d5175e0d898c453b8a7641d94dc1ac58 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 05213ac836744a53b9c089a5c052f4be; no voters: 
I20260709 13:11:53.740852 31233 raft_consensus.cc:2804] T d5175e0d898c453b8a7641d94dc1ac58 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:53.741256 31233 raft_consensus.cc:493] T d5175e0d898c453b8a7641d94dc1ac58 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:53.741621 31233 raft_consensus.cc:3060] T d5175e0d898c453b8a7641d94dc1ac58 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:53.742075 30995 raft_consensus.cc:359] T ea563bd67a9849d18c9c3dfb21d58a93 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:53.744246 30995 raft_consensus.cc:385] T ea563bd67a9849d18c9c3dfb21d58a93 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:53.745154 30995 raft_consensus.cc:740] T ea563bd67a9849d18c9c3dfb21d58a93 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:53.746932 30995 consensus_queue.cc:260] T ea563bd67a9849d18c9c3dfb21d58a93 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
W20260709 13:11:53.757036 30841 leader_election.cc:343] T d5175e0d898c453b8a7641d94dc1ac58 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:53.757972 30996 tablet_bootstrap.cc:492] T 94220be3f55a4acbbd6bd5723b9f32e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
I20260709 13:11:53.758446 30996 ts_tablet_manager.cc:1403] T 94220be3f55a4acbbd6bd5723b9f32e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.112s	user 0.021s	sys 0.021s
I20260709 13:11:53.754530 30624 catalog_manager.cc:5697] T f8f19b152a0f46ac896478f3f05c8822 P 262942f67dfa4b8fa42234b18ac31ffd reported cstate change: term changed from 0 to 1, leader changed from <none> to 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4). New cstate: current_term: 1 leader_uuid: "262942f67dfa4b8fa42234b18ac31ffd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:53.762501 30997 raft_consensus.cc:359] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:53.762593 30996 raft_consensus.cc:359] T 94220be3f55a4acbbd6bd5723b9f32e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:53.763250 30997 raft_consensus.cc:385] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:53.763298 30996 raft_consensus.cc:385] T 94220be3f55a4acbbd6bd5723b9f32e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:53.763623 30997 raft_consensus.cc:740] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Initialized, Role: FOLLOWER
I20260709 13:11:53.763671 30996 raft_consensus.cc:740] T 94220be3f55a4acbbd6bd5723b9f32e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
W20260709 13:11:53.746279 30915 leader_election.cc:343] T 8b3e41a728f1457ebf97274b95c7b23a P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:53.764905 31389 raft_consensus.cc:493] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.781213 30915 leader_election.cc:304] T 8b3e41a728f1457ebf97274b95c7b23a P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:53.781705 31389 raft_consensus.cc:515] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
W20260709 13:11:53.782332 30915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:37325, user_credentials={real_user=slave}} blocked reactor thread for 44142.3us
W20260709 13:11:53.783002 30915 leader_election.cc:343] T 99e98c1324084d85b8a97aaf0c248df7 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:53.783393 30915 leader_election.cc:304] T 99e98c1324084d85b8a97aaf0c248df7 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:53.783954 31389 leader_election.cc:290] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:53.784143 31178 raft_consensus.cc:2749] T 8b3e41a728f1457ebf97274b95c7b23a P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:53.784483 31389 raft_consensus.cc:2749] T 99e98c1324084d85b8a97aaf0c248df7 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:53.770757 31373 consensus_queue.cc:1048] T af695b428b974773ba476dce0d31c5a1 P 05213ac836744a53b9c089a5c052f4be [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:53.775462 30725 raft_consensus.cc:2468] T c0100ec8bd1e4a45b3c0ea52281f9204 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 2.
I20260709 13:11:53.730737 31381 raft_consensus.cc:515] T c0100ec8bd1e4a45b3c0ea52281f9204 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:53.790511 30796 raft_consensus.cc:2393] T c0100ec8bd1e4a45b3c0ea52281f9204 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 262942f67dfa4b8fa42234b18ac31ffd in current term 2: Already voted for candidate 05213ac836744a53b9c089a5c052f4be in this term.
I20260709 13:11:53.791477 31381 leader_election.cc:290] T c0100ec8bd1e4a45b3c0ea52281f9204 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 2 election: Requested vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:53.793201 30796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0100ec8bd1e4a45b3c0ea52281f9204" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:53.793970 30796 raft_consensus.cc:2393] T c0100ec8bd1e4a45b3c0ea52281f9204 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 262942f67dfa4b8fa42234b18ac31ffd in current term 2: Already voted for candidate 05213ac836744a53b9c089a5c052f4be in this term.
I20260709 13:11:53.764362 30997 consensus_queue.cc:260] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:53.795083 30722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0100ec8bd1e4a45b3c0ea52281f9204" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:53.764371 30996 consensus_queue.cc:260] T 94220be3f55a4acbbd6bd5723b9f32e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:53.804262 30623 catalog_manager.cc:5697] T ea563bd67a9849d18c9c3dfb21d58a93 P 02c5705b45c04706bbb2c6ff3d7b0b8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3). New cstate: current_term: 1 leader_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:53.805212 30997 ts_tablet_manager.cc:1434] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd: Time spent starting tablet: real 0.096s	user 0.008s	sys 0.000s
I20260709 13:11:53.829234 30996 ts_tablet_manager.cc:1434] T 94220be3f55a4acbbd6bd5723b9f32e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.071s	user 0.006s	sys 0.000s
I20260709 13:11:53.831751 30995 ts_tablet_manager.cc:1434] T ea563bd67a9849d18c9c3dfb21d58a93 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.135s	user 0.005s	sys 0.004s
I20260709 13:11:53.838460 30872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4dcd4ecc9840d2b8785d14dd4537b8" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:53.812287 30943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c0100ec8bd1e4a45b3c0ea52281f9204" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
W20260709 13:11:53.848400 30914 leader_election.cc:343] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:53.852324 30995 tablet_bootstrap.cc:492] T d5175e0d898c453b8a7641d94dc1ac58 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:53.845780 30722 raft_consensus.cc:2393] T c0100ec8bd1e4a45b3c0ea52281f9204 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 05213ac836744a53b9c089a5c052f4be in current term 2: Already voted for candidate 262942f67dfa4b8fa42234b18ac31ffd in this term.
I20260709 13:11:53.861981 30943 raft_consensus.cc:2393] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 05213ac836744a53b9c089a5c052f4be in current term 2: Already voted for candidate 262942f67dfa4b8fa42234b18ac31ffd in this term.
I20260709 13:11:53.863538 30996 tablet_bootstrap.cc:492] T 8b3e41a728f1457ebf97274b95c7b23a P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
W20260709 13:11:53.866618 31404 log.cc:927] Time spent T ea563bd67a9849d18c9c3dfb21d58a93 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 13:11:53.868956 31389 raft_consensus.cc:493] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.869527 31389 raft_consensus.cc:515] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:53.870373 30915 leader_election.cc:304] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:53.871678 31389 leader_election.cc:290] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:53.872246 31389 raft_consensus.cc:2804] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:53.877959 31240 raft_consensus.cc:493] T 94220be3f55a4acbbd6bd5723b9f32e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:53.878397 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4dcd4ecc9840d2b8785d14dd4537b8" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:53.878507 31240 raft_consensus.cc:515] T 94220be3f55a4acbbd6bd5723b9f32e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:53.879129 30799 raft_consensus.cc:2468] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 0.
I20260709 13:11:53.880645 30767 leader_election.cc:304] T c0100ec8bd1e4a45b3c0ea52281f9204 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be; no voters: 262942f67dfa4b8fa42234b18ac31ffd, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:53.881997 31109 raft_consensus.cc:2749] T c0100ec8bd1e4a45b3c0ea52281f9204 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:53.883447 30722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f63e902c7c7a4d0fa5d2107620ce851e" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:53.883873 31233 raft_consensus.cc:515] T d5175e0d898c453b8a7641d94dc1ac58 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:53.878026 30800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f63e902c7c7a4d0fa5d2107620ce851e" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:53.885335 30800 raft_consensus.cc:2468] T f63e902c7c7a4d0fa5d2107620ce851e P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 0.
I20260709 13:11:53.886291 31233 leader_election.cc:290] T d5175e0d898c453b8a7641d94dc1ac58 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Requested vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:53.822001 31381 consensus_queue.cc:1048] T 6fe93136933f4d2880ec355e1bbbf90d P 05213ac836744a53b9c089a5c052f4be [LEADER]: Connected to new peer: Peer: permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.040s
I20260709 13:11:53.895740 30995 tablet_bootstrap.cc:654] T d5175e0d898c453b8a7641d94dc1ac58 P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
W20260709 13:11:53.893673 30915 leader_election.cc:343] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:53.884486 31089 consensus_queue.cc:1048] T d16c44f56c42466497ba4f7f1b2b5344 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.020s
I20260709 13:11:53.906004 31240 leader_election.cc:290] T 94220be3f55a4acbbd6bd5723b9f32e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:53.909652 30721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94220be3f55a4acbbd6bd5723b9f32e9" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:53.910391 31389 raft_consensus.cc:697] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 LEADER]: Becoming Leader. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Running, Role: LEADER
I20260709 13:11:53.943965 31389 consensus_queue.cc:237] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:53.887176 30912 leader_election.cc:304] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a
I20260709 13:11:53.888175 30943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94220be3f55a4acbbd6bd5723b9f32e9" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:53.889570 30800 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5175e0d898c453b8a7641d94dc1ac58" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:53.888386 30726 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5175e0d898c453b8a7641d94dc1ac58" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
W20260709 13:11:53.941143 30915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:37325, user_credentials={real_user=slave}} blocked reactor thread for 47512.6us
W20260709 13:11:53.949034 30912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:43771, user_credentials={real_user=slave}} blocked reactor thread for 61911.7us
I20260709 13:11:53.969663 31089 raft_consensus.cc:2804] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:53.986348 30912 leader_election.cc:304] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:53.972177 30800 raft_consensus.cc:3060] T d5175e0d898c453b8a7641d94dc1ac58 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:53.994297 30996 tablet_bootstrap.cc:654] T 8b3e41a728f1457ebf97274b95c7b23a P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:53.991493 31089 raft_consensus.cc:493] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:53.997555 31089 raft_consensus.cc:3060] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:53.970921 30943 raft_consensus.cc:2468] T 94220be3f55a4acbbd6bd5723b9f32e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 0.
W20260709 13:11:53.979355 30841 leader_election.cc:343] T 94220be3f55a4acbbd6bd5723b9f32e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:54.011822 30800 raft_consensus.cc:2468] T d5175e0d898c453b8a7641d94dc1ac58 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 1.
I20260709 13:11:54.003355 31178 raft_consensus.cc:2804] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:54.013815 31178 raft_consensus.cc:493] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:54.014213 31178 raft_consensus.cc:3060] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:54.019197 30838 leader_election.cc:304] T d5175e0d898c453b8a7641d94dc1ac58 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 05213ac836744a53b9c089a5c052f4be; no voters: 
I20260709 13:11:54.021862 31240 raft_consensus.cc:2804] T d5175e0d898c453b8a7641d94dc1ac58 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:54.022344 31240 raft_consensus.cc:697] T d5175e0d898c453b8a7641d94dc1ac58 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Becoming Leader. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Running, Role: LEADER
I20260709 13:11:54.023140 31240 consensus_queue.cc:237] T d5175e0d898c453b8a7641d94dc1ac58 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:54.024703 31395 raft_consensus.cc:493] T ea563bd67a9849d18c9c3dfb21d58a93 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:54.038193 30839 leader_election.cc:304] T 94220be3f55a4acbbd6bd5723b9f32e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 262942f67dfa4b8fa42234b18ac31ffd; no voters: d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:54.039326 31002 raft_consensus.cc:2804] T 94220be3f55a4acbbd6bd5723b9f32e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:54.039816 31002 raft_consensus.cc:493] T 94220be3f55a4acbbd6bd5723b9f32e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:54.040205 31002 raft_consensus.cc:3060] T 94220be3f55a4acbbd6bd5723b9f32e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:54.049563 31002 raft_consensus.cc:515] T 94220be3f55a4acbbd6bd5723b9f32e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:54.025421 31395 raft_consensus.cc:515] T ea563bd67a9849d18c9c3dfb21d58a93 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:54.065670 31395 leader_election.cc:290] T ea563bd67a9849d18c9c3dfb21d58a93 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:54.067730 30875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea563bd67a9849d18c9c3dfb21d58a93" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
W20260709 13:11:54.029474 30841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:37325, user_credentials={real_user=slave}} blocked reactor thread for 56661.4us
I20260709 13:11:54.111833 31002 leader_election.cc:290] T 94220be3f55a4acbbd6bd5723b9f32e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Requested vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:54.122597 31089 raft_consensus.cc:515] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:54.128997 31415 consensus_queue.cc:1048] T f8f19b152a0f46ac896478f3f05c8822 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:54.122800 30946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea563bd67a9849d18c9c3dfb21d58a93" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:54.130383 30946 raft_consensus.cc:2393] T ea563bd67a9849d18c9c3dfb21d58a93 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d22c9d3d15904327a78a0e548a8ff6cd in current term 1: Already voted for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in this term.
I20260709 13:11:54.126304 31178 raft_consensus.cc:515] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:54.136333 30690 leader_election.cc:304] T ea563bd67a9849d18c9c3dfb21d58a93 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22c9d3d15904327a78a0e548a8ff6cd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 262942f67dfa4b8fa42234b18ac31ffd
I20260709 13:11:54.138509 31395 raft_consensus.cc:3060] T ea563bd67a9849d18c9c3dfb21d58a93 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:54.146698 30948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94220be3f55a4acbbd6bd5723b9f32e9" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:11:54.147433 30948 raft_consensus.cc:3060] T 94220be3f55a4acbbd6bd5723b9f32e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:11:54.150410 30841 leader_election.cc:343] T d5175e0d898c453b8a7641d94dc1ac58 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:54.152185 30996 tablet_bootstrap.cc:492] T 8b3e41a728f1457ebf97274b95c7b23a P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
I20260709 13:11:54.152741 30996 ts_tablet_manager.cc:1403] T 8b3e41a728f1457ebf97274b95c7b23a P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.294s	user 0.010s	sys 0.004s
I20260709 13:11:54.155293 31089 leader_election.cc:290] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Requested vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:54.159577 31395 raft_consensus.cc:2749] T ea563bd67a9849d18c9c3dfb21d58a93 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:54.160215 30796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4dcd4ecc9840d2b8785d14dd4537b8" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:54.160917 30796 raft_consensus.cc:3060] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:54.161382 30722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94220be3f55a4acbbd6bd5723b9f32e9" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:54.163393 31178 leader_election.cc:290] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Requested vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:54.168286 30996 raft_consensus.cc:359] T 8b3e41a728f1457ebf97274b95c7b23a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:54.169320 30996 raft_consensus.cc:385] T 8b3e41a728f1457ebf97274b95c7b23a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:54.169579 30996 raft_consensus.cc:740] T 8b3e41a728f1457ebf97274b95c7b23a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
I20260709 13:11:54.170290 30996 consensus_queue.cc:260] T 8b3e41a728f1457ebf97274b95c7b23a P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:54.175539 30996 ts_tablet_manager.cc:1434] T 8b3e41a728f1457ebf97274b95c7b23a P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.001s
I20260709 13:11:54.176589 30996 tablet_bootstrap.cc:492] T e2376eaea6bc4f209a21a110370da7b2 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
I20260709 13:11:54.175416 30798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f63e902c7c7a4d0fa5d2107620ce851e" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:54.180627 30798 raft_consensus.cc:3060] T f63e902c7c7a4d0fa5d2107620ce851e P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:54.186954 30796 raft_consensus.cc:2468] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 1.
I20260709 13:11:54.199214 30996 tablet_bootstrap.cc:654] T e2376eaea6bc4f209a21a110370da7b2 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:54.202822 30912 leader_election.cc:304] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
I20260709 13:11:54.203707 31178 raft_consensus.cc:2804] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:54.204275 31178 raft_consensus.cc:697] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Becoming Leader. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Running, Role: LEADER
I20260709 13:11:54.205310 31178 consensus_queue.cc:237] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
W20260709 13:11:54.206768 30841 leader_election.cc:343] T 94220be3f55a4acbbd6bd5723b9f32e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:54.212136 30798 raft_consensus.cc:2468] T f63e902c7c7a4d0fa5d2107620ce851e P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 1.
I20260709 13:11:54.199254 30948 raft_consensus.cc:2468] T 94220be3f55a4acbbd6bd5723b9f32e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 1.
W20260709 13:11:54.191298 30915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:37325, user_credentials={real_user=slave}} blocked reactor thread for 45934.2us
I20260709 13:11:54.184849 31357 consensus_queue.cc:1048] T ea563bd67a9849d18c9c3dfb21d58a93 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 13:11:54.213874 30873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4dcd4ecc9840d2b8785d14dd4537b8" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:11:54.214720 31418 consensus_queue.cc:1048] T ea563bd67a9849d18c9c3dfb21d58a93 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:11:54.223939 30914 leader_election.cc:343] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:54.225914 30995 tablet_bootstrap.cc:492] T d5175e0d898c453b8a7641d94dc1ac58 P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:54.226409 30995 ts_tablet_manager.cc:1403] T d5175e0d898c453b8a7641d94dc1ac58 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.376s	user 0.016s	sys 0.016s
I20260709 13:11:54.226888 30723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f63e902c7c7a4d0fa5d2107620ce851e" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:54.228654 30624 catalog_manager.cc:5697] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd reported cstate change: term changed from 0 to 2, leader changed from <none> to 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4). New cstate: current_term: 2 leader_uuid: "262942f67dfa4b8fa42234b18ac31ffd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:54.230157 30839 leader_election.cc:304] T 94220be3f55a4acbbd6bd5723b9f32e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 262942f67dfa4b8fa42234b18ac31ffd; no voters: d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:54.231211 31418 raft_consensus.cc:2804] T 94220be3f55a4acbbd6bd5723b9f32e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:54.231710 31418 raft_consensus.cc:697] T 94220be3f55a4acbbd6bd5723b9f32e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Becoming Leader. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Running, Role: LEADER
W20260709 13:11:54.230880 30915 leader_election.cc:343] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:54.233034 31418 consensus_queue.cc:237] T 94220be3f55a4acbbd6bd5723b9f32e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:54.235456 30995 raft_consensus.cc:359] T d5175e0d898c453b8a7641d94dc1ac58 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:54.236153 30995 raft_consensus.cc:385] T d5175e0d898c453b8a7641d94dc1ac58 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:54.236544 30995 raft_consensus.cc:740] T d5175e0d898c453b8a7641d94dc1ac58 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:54.246524 30912 leader_election.cc:304] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:54.273713 30995 consensus_queue.cc:260] T d5175e0d898c453b8a7641d94dc1ac58 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:54.270889 31363 raft_consensus.cc:2804] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:54.284533 30995 ts_tablet_manager.cc:1434] T d5175e0d898c453b8a7641d94dc1ac58 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.058s	user 0.008s	sys 0.000s
I20260709 13:11:54.284636 31363 raft_consensus.cc:697] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Becoming Leader. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Running, Role: LEADER
I20260709 13:11:54.285521 30995 tablet_bootstrap.cc:492] T 6fe93136933f4d2880ec355e1bbbf90d P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:54.285638 31363 consensus_queue.cc:237] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:54.284610 30996 tablet_bootstrap.cc:492] T e2376eaea6bc4f209a21a110370da7b2 P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
I20260709 13:11:54.289951 30996 ts_tablet_manager.cc:1403] T e2376eaea6bc4f209a21a110370da7b2 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.114s	user 0.013s	sys 0.014s
I20260709 13:11:54.290067 31178 consensus_queue.cc:1048] T f8f19b152a0f46ac896478f3f05c8822 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:54.292709 30996 raft_consensus.cc:359] T e2376eaea6bc4f209a21a110370da7b2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:54.293370 30996 raft_consensus.cc:385] T e2376eaea6bc4f209a21a110370da7b2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:54.293617 30996 raft_consensus.cc:740] T e2376eaea6bc4f209a21a110370da7b2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
I20260709 13:11:54.294194 30996 consensus_queue.cc:260] T e2376eaea6bc4f209a21a110370da7b2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:54.304961 30996 ts_tablet_manager.cc:1434] T e2376eaea6bc4f209a21a110370da7b2 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.000s
I20260709 13:11:54.305959 30996 tablet_bootstrap.cc:492] T cae2b1cb8db54cbb812489cc1b4b1213 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
I20260709 13:11:54.312235 30996 tablet_bootstrap.cc:654] T cae2b1cb8db54cbb812489cc1b4b1213 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:54.329713 30624 catalog_manager.cc:5697] T 94220be3f55a4acbbd6bd5723b9f32e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3). New cstate: current_term: 1 leader_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:54.332258 30996 tablet_bootstrap.cc:492] T cae2b1cb8db54cbb812489cc1b4b1213 P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
I20260709 13:11:54.332819 30996 ts_tablet_manager.cc:1403] T cae2b1cb8db54cbb812489cc1b4b1213 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.027s	user 0.009s	sys 0.012s
I20260709 13:11:54.332124 30624 catalog_manager.cc:5697] T d5175e0d898c453b8a7641d94dc1ac58 P 02c5705b45c04706bbb2c6ff3d7b0b8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3). New cstate: current_term: 1 leader_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:54.334347 30995 tablet_bootstrap.cc:654] T 6fe93136933f4d2880ec355e1bbbf90d P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:54.335566 30996 raft_consensus.cc:359] T cae2b1cb8db54cbb812489cc1b4b1213 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:54.336232 30996 raft_consensus.cc:385] T cae2b1cb8db54cbb812489cc1b4b1213 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:54.336539 30996 raft_consensus.cc:740] T cae2b1cb8db54cbb812489cc1b4b1213 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
I20260709 13:11:54.337321 30996 consensus_queue.cc:260] T cae2b1cb8db54cbb812489cc1b4b1213 P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:54.340345 30996 ts_tablet_manager.cc:1434] T cae2b1cb8db54cbb812489cc1b4b1213 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:11:54.341357 30996 tablet_bootstrap.cc:492] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
I20260709 13:11:54.347110 30996 tablet_bootstrap.cc:654] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
W20260709 13:11:54.353806 30671 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:11:54.364909 31279 raft_consensus.cc:493] T d5175e0d898c453b8a7641d94dc1ac58 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:54.365664 31279 raft_consensus.cc:515] T d5175e0d898c453b8a7641d94dc1ac58 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:54.370218 30873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5175e0d898c453b8a7641d94dc1ac58" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:54.372048 30995 tablet_bootstrap.cc:492] T 6fe93136933f4d2880ec355e1bbbf90d P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:54.372771 30995 ts_tablet_manager.cc:1403] T 6fe93136933f4d2880ec355e1bbbf90d P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.087s	user 0.007s	sys 0.008s
I20260709 13:11:54.375331 30620 catalog_manager.cc:5697] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd reported cstate change: term changed from 0 to 1, leader changed from <none> to 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4). New cstate: current_term: 1 leader_uuid: "262942f67dfa4b8fa42234b18ac31ffd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:54.376578 31279 leader_election.cc:290] T d5175e0d898c453b8a7641d94dc1ac58 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:54.375524 30995 raft_consensus.cc:359] T 6fe93136933f4d2880ec355e1bbbf90d P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:54.376789 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5175e0d898c453b8a7641d94dc1ac58" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:54.377866 30799 raft_consensus.cc:2393] T d5175e0d898c453b8a7641d94dc1ac58 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d22c9d3d15904327a78a0e548a8ff6cd in current term 1: Already voted for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in this term.
I20260709 13:11:54.380862 30689 leader_election.cc:304] T d5175e0d898c453b8a7641d94dc1ac58 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22c9d3d15904327a78a0e548a8ff6cd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 05213ac836744a53b9c089a5c052f4be
I20260709 13:11:54.382299 31390 raft_consensus.cc:3060] T d5175e0d898c453b8a7641d94dc1ac58 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:54.377279 30995 raft_consensus.cc:385] T 6fe93136933f4d2880ec355e1bbbf90d P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:54.384415 30995 raft_consensus.cc:740] T 6fe93136933f4d2880ec355e1bbbf90d P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:54.379336 30996 tablet_bootstrap.cc:492] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
I20260709 13:11:54.385336 30996 ts_tablet_manager.cc:1403] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.044s	user 0.023s	sys 0.012s
I20260709 13:11:54.388012 30995 consensus_queue.cc:260] T 6fe93136933f4d2880ec355e1bbbf90d P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:54.377424 30620 catalog_manager.cc:5697] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 262942f67dfa4b8fa42234b18ac31ffd reported cstate change: term changed from 0 to 1, leader changed from <none> to 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4). New cstate: current_term: 1 leader_uuid: "262942f67dfa4b8fa42234b18ac31ffd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:54.389945 31390 raft_consensus.cc:2749] T d5175e0d898c453b8a7641d94dc1ac58 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:54.390094 30996 raft_consensus.cc:359] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:54.392378 30996 raft_consensus.cc:385] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:54.392652 30996 raft_consensus.cc:740] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
I20260709 13:11:54.393872 30996 consensus_queue.cc:260] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:54.395269 30995 ts_tablet_manager.cc:1434] T 6fe93136933f4d2880ec355e1bbbf90d P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.000s
I20260709 13:11:54.395956 30996 ts_tablet_manager.cc:1434] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.000s
I20260709 13:11:54.396162 30995 tablet_bootstrap.cc:492] T cae2b1cb8db54cbb812489cc1b4b1213 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:54.396970 30996 tablet_bootstrap.cc:492] T 99e98c1324084d85b8a97aaf0c248df7 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Bootstrap starting.
I20260709 13:11:54.402396 30995 tablet_bootstrap.cc:654] T cae2b1cb8db54cbb812489cc1b4b1213 P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:54.405063 30996 tablet_bootstrap.cc:654] T 99e98c1324084d85b8a97aaf0c248df7 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:54.418260 30996 tablet_bootstrap.cc:492] T 99e98c1324084d85b8a97aaf0c248df7 P 02c5705b45c04706bbb2c6ff3d7b0b8a: No bootstrap required, opened a new log
I20260709 13:11:54.418773 30996 ts_tablet_manager.cc:1403] T 99e98c1324084d85b8a97aaf0c248df7 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent bootstrapping tablet: real 0.022s	user 0.005s	sys 0.010s
I20260709 13:11:54.420914 31419 raft_consensus.cc:493] T 8b3e41a728f1457ebf97274b95c7b23a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:54.421450 31419 raft_consensus.cc:515] T 8b3e41a728f1457ebf97274b95c7b23a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:54.422430 30996 raft_consensus.cc:359] T 99e98c1324084d85b8a97aaf0c248df7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:54.422907 30996 raft_consensus.cc:385] T 99e98c1324084d85b8a97aaf0c248df7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:54.423064 30996 raft_consensus.cc:740] T 99e98c1324084d85b8a97aaf0c248df7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Initialized, Role: FOLLOWER
I20260709 13:11:54.423625 30996 consensus_queue.cc:260] T 99e98c1324084d85b8a97aaf0c248df7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:54.426388 30996 ts_tablet_manager.cc:1434] T 99e98c1324084d85b8a97aaf0c248df7 P 02c5705b45c04706bbb2c6ff3d7b0b8a: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:11:54.430275 30942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b3e41a728f1457ebf97274b95c7b23a" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:54.430920 30942 raft_consensus.cc:2468] T 8b3e41a728f1457ebf97274b95c7b23a P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 0.
I20260709 13:11:54.431084 31419 leader_election.cc:290] T 8b3e41a728f1457ebf97274b95c7b23a P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:54.432178 30839 leader_election.cc:304] T 8b3e41a728f1457ebf97274b95c7b23a P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
I20260709 13:11:54.433003 31419 raft_consensus.cc:2804] T 8b3e41a728f1457ebf97274b95c7b23a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:54.433390 31419 raft_consensus.cc:493] T 8b3e41a728f1457ebf97274b95c7b23a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:54.434231 30995 tablet_bootstrap.cc:492] T cae2b1cb8db54cbb812489cc1b4b1213 P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:54.434696 30995 ts_tablet_manager.cc:1403] T cae2b1cb8db54cbb812489cc1b4b1213 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.039s	user 0.022s	sys 0.010s
I20260709 13:11:54.433777 31419 raft_consensus.cc:3060] T 8b3e41a728f1457ebf97274b95c7b23a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:54.437538 30995 raft_consensus.cc:359] T cae2b1cb8db54cbb812489cc1b4b1213 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:54.438203 30995 raft_consensus.cc:385] T cae2b1cb8db54cbb812489cc1b4b1213 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:54.438475 30995 raft_consensus.cc:740] T cae2b1cb8db54cbb812489cc1b4b1213 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:54.439172 30995 consensus_queue.cc:260] T cae2b1cb8db54cbb812489cc1b4b1213 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:54.441705 30995 ts_tablet_manager.cc:1434] T cae2b1cb8db54cbb812489cc1b4b1213 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:11:54.442696 30995 tablet_bootstrap.cc:492] T 94220be3f55a4acbbd6bd5723b9f32e9 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:54.445978 30721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b3e41a728f1457ebf97274b95c7b23a" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:54.448510 30995 tablet_bootstrap.cc:654] T 94220be3f55a4acbbd6bd5723b9f32e9 P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
W20260709 13:11:54.450248 30841 leader_election.cc:343] T 8b3e41a728f1457ebf97274b95c7b23a P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:54.452473 31419 raft_consensus.cc:515] T 8b3e41a728f1457ebf97274b95c7b23a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:54.454988 30995 tablet_bootstrap.cc:492] T 94220be3f55a4acbbd6bd5723b9f32e9 P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:54.455479 30995 ts_tablet_manager.cc:1403] T 94220be3f55a4acbbd6bd5723b9f32e9 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.013s	user 0.011s	sys 0.000s
I20260709 13:11:54.455230 30942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b3e41a728f1457ebf97274b95c7b23a" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:11:54.455758 30721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b3e41a728f1457ebf97274b95c7b23a" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:54.455993 30942 raft_consensus.cc:3060] T 8b3e41a728f1457ebf97274b95c7b23a P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:11:54.457296 30841 leader_election.cc:343] T 8b3e41a728f1457ebf97274b95c7b23a P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:54.458983 31419 leader_election.cc:290] T 8b3e41a728f1457ebf97274b95c7b23a P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Requested vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:54.461417 30995 raft_consensus.cc:359] T 94220be3f55a4acbbd6bd5723b9f32e9 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:54.461907 30995 raft_consensus.cc:385] T 94220be3f55a4acbbd6bd5723b9f32e9 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:54.462129 30995 raft_consensus.cc:740] T 94220be3f55a4acbbd6bd5723b9f32e9 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:54.462514 30942 raft_consensus.cc:2468] T 8b3e41a728f1457ebf97274b95c7b23a P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 1.
I20260709 13:11:54.462625 30995 consensus_queue.cc:260] T 94220be3f55a4acbbd6bd5723b9f32e9 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:54.463616 30839 leader_election.cc:304] T 8b3e41a728f1457ebf97274b95c7b23a P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 262942f67dfa4b8fa42234b18ac31ffd; no voters: d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:54.464367 31419 raft_consensus.cc:2804] T 8b3e41a728f1457ebf97274b95c7b23a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:54.464844 30995 ts_tablet_manager.cc:1434] T 94220be3f55a4acbbd6bd5723b9f32e9 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:11:54.464958 31419 raft_consensus.cc:697] T 8b3e41a728f1457ebf97274b95c7b23a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Becoming Leader. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Running, Role: LEADER
I20260709 13:11:54.465890 31419 consensus_queue.cc:237] T 8b3e41a728f1457ebf97274b95c7b23a P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:54.466631 30995 tablet_bootstrap.cc:492] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:54.473116 30995 tablet_bootstrap.cc:654] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:54.487155 30995 tablet_bootstrap.cc:492] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:54.487586 30995 ts_tablet_manager.cc:1403] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.021s	user 0.010s	sys 0.010s
I20260709 13:11:54.490060 30995 raft_consensus.cc:359] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:54.490712 30995 raft_consensus.cc:385] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:54.490958 30995 raft_consensus.cc:740] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:54.491477 30624 catalog_manager.cc:5697] T 8b3e41a728f1457ebf97274b95c7b23a P 02c5705b45c04706bbb2c6ff3d7b0b8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3). New cstate: current_term: 1 leader_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:54.491540 30995 consensus_queue.cc:260] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:54.493817 30995 ts_tablet_manager.cc:1434] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:11:54.497205 30995 tablet_bootstrap.cc:492] T d16c44f56c42466497ba4f7f1b2b5344 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:54.502707 30995 tablet_bootstrap.cc:654] T d16c44f56c42466497ba4f7f1b2b5344 P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:54.502796 31279 raft_consensus.cc:493] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:54.503335 31279 raft_consensus.cc:515] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:54.506971 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2376eaea6bc4f209a21a110370da7b2" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:54.507587 30799 raft_consensus.cc:2468] T e2376eaea6bc4f209a21a110370da7b2 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 0.
I20260709 13:11:54.508769 30872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2376eaea6bc4f209a21a110370da7b2" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:54.510082 30872 raft_consensus.cc:2468] T e2376eaea6bc4f209a21a110370da7b2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 0.
I20260709 13:11:54.509352 31279 leader_election.cc:290] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:54.509173 30689 leader_election.cc:304] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:54.512104 31279 raft_consensus.cc:2804] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:54.512538 31279 raft_consensus.cc:493] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:54.512945 31279 raft_consensus.cc:3060] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:54.518980 31279 raft_consensus.cc:515] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:54.521552 30995 tablet_bootstrap.cc:492] T d16c44f56c42466497ba4f7f1b2b5344 P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:54.521566 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2376eaea6bc4f209a21a110370da7b2" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:54.521910 30872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2376eaea6bc4f209a21a110370da7b2" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:11:54.522109 30995 ts_tablet_manager.cc:1403] T d16c44f56c42466497ba4f7f1b2b5344 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.026s	user 0.022s	sys 0.001s
I20260709 13:11:54.522414 30872 raft_consensus.cc:3060] T e2376eaea6bc4f209a21a110370da7b2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:54.522589 30799 raft_consensus.cc:3060] T e2376eaea6bc4f209a21a110370da7b2 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:54.524869 30995 raft_consensus.cc:359] T d16c44f56c42466497ba4f7f1b2b5344 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:54.525525 30995 raft_consensus.cc:385] T d16c44f56c42466497ba4f7f1b2b5344 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:54.525756 30995 raft_consensus.cc:740] T d16c44f56c42466497ba4f7f1b2b5344 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:54.526374 30995 consensus_queue.cc:260] T d16c44f56c42466497ba4f7f1b2b5344 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:54.528765 30799 raft_consensus.cc:2468] T e2376eaea6bc4f209a21a110370da7b2 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 1.
I20260709 13:11:54.532262 30995 ts_tablet_manager.cc:1434] T d16c44f56c42466497ba4f7f1b2b5344 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:11:54.532517 30689 leader_election.cc:304] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:54.533257 31390 raft_consensus.cc:2804] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:54.533716 31390 raft_consensus.cc:697] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 LEADER]: Becoming Leader. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Running, Role: LEADER
I20260709 13:11:54.534489 31390 consensus_queue.cc:237] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:54.539006 30723 raft_consensus.cc:3060] T d16c44f56c42466497ba4f7f1b2b5344 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:54.540015 30995 tablet_bootstrap.cc:492] T f63e902c7c7a4d0fa5d2107620ce851e P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:54.544986 30872 raft_consensus.cc:2468] T e2376eaea6bc4f209a21a110370da7b2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 1.
I20260709 13:11:54.548182 31279 leader_election.cc:290] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 election: Requested vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:54.548921 30995 tablet_bootstrap.cc:654] T f63e902c7c7a4d0fa5d2107620ce851e P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:54.552817 31418 consensus_queue.cc:1048] T d5175e0d898c453b8a7641d94dc1ac58 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:54.553704 30796 raft_consensus.cc:1275] T c0100ec8bd1e4a45b3c0ea52281f9204 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Refusing update from remote peer 262942f67dfa4b8fa42234b18ac31ffd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:11:54.554862 30721 raft_consensus.cc:1275] T c0100ec8bd1e4a45b3c0ea52281f9204 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Refusing update from remote peer 262942f67dfa4b8fa42234b18ac31ffd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:11:54.555505 31178 consensus_queue.cc:1048] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:54.557792 31089 consensus_queue.cc:1048] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:54.561882 30995 tablet_bootstrap.cc:492] T f63e902c7c7a4d0fa5d2107620ce851e P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:54.562525 30995 ts_tablet_manager.cc:1403] T f63e902c7c7a4d0fa5d2107620ce851e P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.029s	user 0.007s	sys 0.003s
I20260709 13:11:54.578811 30725 raft_consensus.cc:3060] T 6fe93136933f4d2880ec355e1bbbf90d P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:54.577955 30995 raft_consensus.cc:359] T f63e902c7c7a4d0fa5d2107620ce851e P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:54.582180 30995 raft_consensus.cc:385] T f63e902c7c7a4d0fa5d2107620ce851e P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:54.582531 30995 raft_consensus.cc:740] T f63e902c7c7a4d0fa5d2107620ce851e P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:54.583222 30995 consensus_queue.cc:260] T f63e902c7c7a4d0fa5d2107620ce851e P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:54.591668 30995 ts_tablet_manager.cc:1434] T f63e902c7c7a4d0fa5d2107620ce851e P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.000s
I20260709 13:11:54.592625 30995 tablet_bootstrap.cc:492] T 8b3e41a728f1457ebf97274b95c7b23a P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:54.595327 31419 consensus_queue.cc:1048] T d5175e0d898c453b8a7641d94dc1ac58 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:54.598389 30995 tablet_bootstrap.cc:654] T 8b3e41a728f1457ebf97274b95c7b23a P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:54.641745 30873 raft_consensus.cc:3060] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:54.643124 31069 raft_consensus.cc:493] T d16c44f56c42466497ba4f7f1b2b5344 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 262942f67dfa4b8fa42234b18ac31ffd)
I20260709 13:11:54.643662 31069 raft_consensus.cc:515] T d16c44f56c42466497ba4f7f1b2b5344 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:54.652459 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16c44f56c42466497ba4f7f1b2b5344" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:54.652992 31069 leader_election.cc:290] T d16c44f56c42466497ba4f7f1b2b5344 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:54.669452 31178 consensus_queue.cc:1048] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:54.669551 30620 catalog_manager.cc:5697] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd reported cstate change: term changed from 0 to 1, leader changed from <none> to d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1). New cstate: current_term: 1 leader_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:54.693039 30948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d16c44f56c42466497ba4f7f1b2b5344" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:54.697149 30690 leader_election.cc:304] T d16c44f56c42466497ba4f7f1b2b5344 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22c9d3d15904327a78a0e548a8ff6cd; no voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd
I20260709 13:11:54.698524 31395 raft_consensus.cc:2749] T d16c44f56c42466497ba4f7f1b2b5344 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:54.745115 31419 raft_consensus.cc:493] T cae2b1cb8db54cbb812489cc1b4b1213 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:54.746203 31419 raft_consensus.cc:515] T cae2b1cb8db54cbb812489cc1b4b1213 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:54.751690 30995 tablet_bootstrap.cc:492] T 8b3e41a728f1457ebf97274b95c7b23a P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:54.752246 30995 ts_tablet_manager.cc:1403] T 8b3e41a728f1457ebf97274b95c7b23a P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.160s	user 0.065s	sys 0.085s
I20260709 13:11:54.754762 30995 raft_consensus.cc:359] T 8b3e41a728f1457ebf97274b95c7b23a P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:54.755429 30995 raft_consensus.cc:385] T 8b3e41a728f1457ebf97274b95c7b23a P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:54.755689 30995 raft_consensus.cc:740] T 8b3e41a728f1457ebf97274b95c7b23a P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:54.756361 30995 consensus_queue.cc:260] T 8b3e41a728f1457ebf97274b95c7b23a P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:54.758733 30995 ts_tablet_manager.cc:1434] T 8b3e41a728f1457ebf97274b95c7b23a P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.004s
I20260709 13:11:54.759624 30995 tablet_bootstrap.cc:492] T 99e98c1324084d85b8a97aaf0c248df7 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:54.761557 30726 raft_consensus.cc:3060] T 94220be3f55a4acbbd6bd5723b9f32e9 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:54.763489 30723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae2b1cb8db54cbb812489cc1b4b1213" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:54.764201 30723 raft_consensus.cc:2468] T cae2b1cb8db54cbb812489cc1b4b1213 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 0.
I20260709 13:11:54.765163 30995 tablet_bootstrap.cc:654] T 99e98c1324084d85b8a97aaf0c248df7 P d22c9d3d15904327a78a0e548a8ff6cd: Neither blocks nor log segments found. Creating new log.
I20260709 13:11:54.765337 30841 leader_election.cc:304] T cae2b1cb8db54cbb812489cc1b4b1213 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:54.766194 31418 raft_consensus.cc:2804] T cae2b1cb8db54cbb812489cc1b4b1213 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:54.766562 31418 raft_consensus.cc:493] T cae2b1cb8db54cbb812489cc1b4b1213 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:54.766925 31418 raft_consensus.cc:3060] T cae2b1cb8db54cbb812489cc1b4b1213 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:54.775030 31239 consensus_queue.cc:1048] T 94220be3f55a4acbbd6bd5723b9f32e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:54.775290 31418 raft_consensus.cc:515] T cae2b1cb8db54cbb812489cc1b4b1213 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:54.778520 31239 consensus_queue.cc:1048] T 94220be3f55a4acbbd6bd5723b9f32e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:54.781230 30723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae2b1cb8db54cbb812489cc1b4b1213" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:54.782132 30723 raft_consensus.cc:3060] T cae2b1cb8db54cbb812489cc1b4b1213 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:54.789083 31363 consensus_queue.cc:1048] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:54.790553 30796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae2b1cb8db54cbb812489cc1b4b1213" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:54.791332 30796 raft_consensus.cc:2468] T cae2b1cb8db54cbb812489cc1b4b1213 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 0.
I20260709 13:11:54.794322 31419 leader_election.cc:290] T cae2b1cb8db54cbb812489cc1b4b1213 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:54.797744 30796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cae2b1cb8db54cbb812489cc1b4b1213" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:54.798430 30796 raft_consensus.cc:3060] T cae2b1cb8db54cbb812489cc1b4b1213 P 05213ac836744a53b9c089a5c052f4be [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:54.806700 31418 leader_election.cc:290] T cae2b1cb8db54cbb812489cc1b4b1213 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Requested vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:54.808739 30723 raft_consensus.cc:2468] T cae2b1cb8db54cbb812489cc1b4b1213 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 1.
I20260709 13:11:54.813692 30841 leader_election.cc:304] T cae2b1cb8db54cbb812489cc1b4b1213 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:54.814512 31418 raft_consensus.cc:2804] T cae2b1cb8db54cbb812489cc1b4b1213 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:54.814953 30995 tablet_bootstrap.cc:492] T 99e98c1324084d85b8a97aaf0c248df7 P d22c9d3d15904327a78a0e548a8ff6cd: No bootstrap required, opened a new log
I20260709 13:11:54.815730 31418 raft_consensus.cc:697] T cae2b1cb8db54cbb812489cc1b4b1213 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Becoming Leader. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Running, Role: LEADER
I20260709 13:11:54.816727 31418 consensus_queue.cc:237] T cae2b1cb8db54cbb812489cc1b4b1213 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:54.815480 30995 ts_tablet_manager.cc:1403] T 99e98c1324084d85b8a97aaf0c248df7 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.056s	user 0.015s	sys 0.012s
I20260709 13:11:54.819182 30796 raft_consensus.cc:2468] T cae2b1cb8db54cbb812489cc1b4b1213 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 1.
I20260709 13:11:54.821133 30995 raft_consensus.cc:359] T 99e98c1324084d85b8a97aaf0c248df7 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:54.821848 30995 raft_consensus.cc:385] T 99e98c1324084d85b8a97aaf0c248df7 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:11:54.822108 30995 raft_consensus.cc:740] T 99e98c1324084d85b8a97aaf0c248df7 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:54.822772 30995 consensus_queue.cc:260] T 99e98c1324084d85b8a97aaf0c248df7 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:54.824980 30995 ts_tablet_manager.cc:1434] T 99e98c1324084d85b8a97aaf0c248df7 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:11:54.828840 31419 raft_consensus.cc:493] T 99e98c1324084d85b8a97aaf0c248df7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:54.829453 31419 raft_consensus.cc:515] T 99e98c1324084d85b8a97aaf0c248df7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:54.832398 30942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e98c1324084d85b8a97aaf0c248df7" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:54.833205 30942 raft_consensus.cc:2468] T 99e98c1324084d85b8a97aaf0c248df7 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 0.
I20260709 13:11:54.834538 30839 leader_election.cc:304] T 99e98c1324084d85b8a97aaf0c248df7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
I20260709 13:11:54.835352 31233 raft_consensus.cc:2804] T 99e98c1324084d85b8a97aaf0c248df7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:11:54.835719 31233 raft_consensus.cc:493] T 99e98c1324084d85b8a97aaf0c248df7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:11:54.836052 31233 raft_consensus.cc:3060] T 99e98c1324084d85b8a97aaf0c248df7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:54.841269 31419 leader_election.cc:290] T 99e98c1324084d85b8a97aaf0c248df7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:54.842842 31233 raft_consensus.cc:515] T 99e98c1324084d85b8a97aaf0c248df7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:54.843371 30723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e98c1324084d85b8a97aaf0c248df7" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:54.844045 30723 raft_consensus.cc:2468] T 99e98c1324084d85b8a97aaf0c248df7 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 0.
I20260709 13:11:54.845511 30942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e98c1324084d85b8a97aaf0c248df7" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:11:54.846087 30942 raft_consensus.cc:3060] T 99e98c1324084d85b8a97aaf0c248df7 P 262942f67dfa4b8fa42234b18ac31ffd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:54.848628 30620 catalog_manager.cc:5697] T cae2b1cb8db54cbb812489cc1b4b1213 P 02c5705b45c04706bbb2c6ff3d7b0b8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3). New cstate: current_term: 1 leader_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:54.852481 31069 raft_consensus.cc:493] T 8b3e41a728f1457ebf97274b95c7b23a P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:11:54.852962 31069 raft_consensus.cc:515] T 8b3e41a728f1457ebf97274b95c7b23a P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:54.854789 31069 leader_election.cc:290] T 8b3e41a728f1457ebf97274b95c7b23a P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:54.856187 30873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b3e41a728f1457ebf97274b95c7b23a" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:54.858544 30723 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e98c1324084d85b8a97aaf0c248df7" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:54.859063 30723 raft_consensus.cc:3060] T 99e98c1324084d85b8a97aaf0c248df7 P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:54.867184 30723 raft_consensus.cc:2468] T 99e98c1324084d85b8a97aaf0c248df7 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 1.
I20260709 13:11:54.867908 31233 leader_election.cc:290] T 99e98c1324084d85b8a97aaf0c248df7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Requested vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:54.868275 30841 leader_election.cc:304] T 99e98c1324084d85b8a97aaf0c248df7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:54.856554 30942 raft_consensus.cc:2468] T 99e98c1324084d85b8a97aaf0c248df7 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 1.
I20260709 13:11:54.870368 31233 raft_consensus.cc:2804] T 99e98c1324084d85b8a97aaf0c248df7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:11:54.871061 31233 raft_consensus.cc:697] T 99e98c1324084d85b8a97aaf0c248df7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Becoming Leader. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Running, Role: LEADER
I20260709 13:11:54.872651 30943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b3e41a728f1457ebf97274b95c7b23a" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:54.873558 30943 raft_consensus.cc:2393] T 8b3e41a728f1457ebf97274b95c7b23a P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d22c9d3d15904327a78a0e548a8ff6cd in current term 1: Already voted for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in this term.
I20260709 13:11:54.874071 30721 raft_consensus.cc:3060] T f63e902c7c7a4d0fa5d2107620ce851e P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:54.877894 30690 leader_election.cc:304] T 8b3e41a728f1457ebf97274b95c7b23a P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d22c9d3d15904327a78a0e548a8ff6cd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 262942f67dfa4b8fa42234b18ac31ffd
I20260709 13:11:54.878791 31395 raft_consensus.cc:3060] T 8b3e41a728f1457ebf97274b95c7b23a P d22c9d3d15904327a78a0e548a8ff6cd [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:11:54.888989 31046 consensus_queue.cc:1048] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:54.890383 31046 consensus_queue.cc:1048] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:54.899102 31395 raft_consensus.cc:2749] T 8b3e41a728f1457ebf97274b95c7b23a P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:11:54.874485 31233 consensus_queue.cc:237] T 99e98c1324084d85b8a97aaf0c248df7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:54.920917 30622 catalog_manager.cc:5697] T 99e98c1324084d85b8a97aaf0c248df7 P 02c5705b45c04706bbb2c6ff3d7b0b8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3). New cstate: current_term: 1 leader_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:54.954196 31395 consensus_queue.cc:1048] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:54.970461 31390 consensus_queue.cc:1048] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:55.018388 31233 consensus_queue.cc:1048] T 8b3e41a728f1457ebf97274b95c7b23a P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:55.029665 31420 consensus_queue.cc:1048] T 8b3e41a728f1457ebf97274b95c7b23a P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:11:55.086931 21756 tablet_server.cc:179] TabletServer@127.21.63.1:0 shutting down...
I20260709 13:11:55.126719 21756 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:11:55.130744 21756 tablet_replica.cc:333] T 8b3e41a728f1457ebf97274b95c7b23a P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
I20260709 13:11:55.131688 21756 raft_consensus.cc:2243] T 8b3e41a728f1457ebf97274b95c7b23a P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:55.132234 21756 raft_consensus.cc:2272] T 8b3e41a728f1457ebf97274b95c7b23a P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:55.134943 21756 tablet_replica.cc:333] T f63e902c7c7a4d0fa5d2107620ce851e P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
I20260709 13:11:55.135733 21756 raft_consensus.cc:2243] T f63e902c7c7a4d0fa5d2107620ce851e P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:55.136225 21756 raft_consensus.cc:2272] T f63e902c7c7a4d0fa5d2107620ce851e P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:55.138875 30915 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 1 similar messages]
I20260709 13:11:55.139233 21756 tablet_replica.cc:333] T d16c44f56c42466497ba4f7f1b2b5344 P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
I20260709 13:11:55.140398 21756 raft_consensus.cc:2243] T d16c44f56c42466497ba4f7f1b2b5344 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:55.140964 21756 raft_consensus.cc:2272] T d16c44f56c42466497ba4f7f1b2b5344 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:55.142330 30915 consensus_peers.cc:597] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:55.143574 21756 tablet_replica.cc:333] T 94220be3f55a4acbbd6bd5723b9f32e9 P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
I20260709 13:11:55.144515 21756 raft_consensus.cc:2243] T 94220be3f55a4acbbd6bd5723b9f32e9 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:55.145076 21756 raft_consensus.cc:2272] T 94220be3f55a4acbbd6bd5723b9f32e9 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:55.147217 21756 tablet_replica.cc:333] T cae2b1cb8db54cbb812489cc1b4b1213 P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
I20260709 13:11:55.147840 21756 raft_consensus.cc:2243] T cae2b1cb8db54cbb812489cc1b4b1213 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:55.148231 21756 raft_consensus.cc:2272] T cae2b1cb8db54cbb812489cc1b4b1213 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:55.149899 21756 tablet_replica.cc:333] T 6fe93136933f4d2880ec355e1bbbf90d P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
I20260709 13:11:55.150471 21756 raft_consensus.cc:2243] T 6fe93136933f4d2880ec355e1bbbf90d P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:55.150938 21756 raft_consensus.cc:2272] T 6fe93136933f4d2880ec355e1bbbf90d P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:55.152511 30841 consensus_peers.cc:597] T d05da516ed9040c988b1fdfe2fdde9ed P 02c5705b45c04706bbb2c6ff3d7b0b8a -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:55.153196 21756 tablet_replica.cc:333] T d5175e0d898c453b8a7641d94dc1ac58 P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
I20260709 13:11:55.153856 21756 raft_consensus.cc:2243] T d5175e0d898c453b8a7641d94dc1ac58 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:55.154273 21756 raft_consensus.cc:2272] T d5175e0d898c453b8a7641d94dc1ac58 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:55.156388 21756 tablet_replica.cc:333] T af695b428b974773ba476dce0d31c5a1 P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
I20260709 13:11:55.157049 21756 raft_consensus.cc:2243] T af695b428b974773ba476dce0d31c5a1 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:55.157444 21756 raft_consensus.cc:2272] T af695b428b974773ba476dce0d31c5a1 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:55.157801 30915 consensus_peers.cc:597] T b3754a670e0b4ed99d37283bf1647bf5 P 262942f67dfa4b8fa42234b18ac31ffd -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:55.159533 21756 tablet_replica.cc:333] T c0100ec8bd1e4a45b3c0ea52281f9204 P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
I20260709 13:11:55.160151 21756 raft_consensus.cc:2243] T c0100ec8bd1e4a45b3c0ea52281f9204 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:55.160640 21756 raft_consensus.cc:2272] T c0100ec8bd1e4a45b3c0ea52281f9204 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:55.163157 21756 tablet_replica.cc:333] T 8155b3fc02b348e1adb290ecfe4592c5 P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
I20260709 13:11:55.163782 21756 raft_consensus.cc:2243] T 8155b3fc02b348e1adb290ecfe4592c5 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:55.164171 21756 raft_consensus.cc:2272] T 8155b3fc02b348e1adb290ecfe4592c5 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:55.165817 21756 tablet_replica.cc:333] T 8fb24a20af104a7486d0f1b2ae0e3843 P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
I20260709 13:11:55.166395 21756 raft_consensus.cc:2243] T 8fb24a20af104a7486d0f1b2ae0e3843 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:11:55.166560 30767 consensus_peers.cc:597] T 8b6052cddae64c5c9d16f60272735788 P 05213ac836744a53b9c089a5c052f4be -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:55.166880 21756 raft_consensus.cc:2272] T 8fb24a20af104a7486d0f1b2ae0e3843 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:55.168891 21756 tablet_replica.cc:333] T 5532e197e7de48f28c034ba1fc490e19 P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
I20260709 13:11:55.169545 21756 raft_consensus.cc:2243] T 5532e197e7de48f28c034ba1fc490e19 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:55.170292 21756 raft_consensus.cc:2272] T 5532e197e7de48f28c034ba1fc490e19 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:55.172396 21756 tablet_replica.cc:333] T 92a4b4a5a07442f49f518b479db8c94b P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
I20260709 13:11:55.173199 21756 raft_consensus.cc:2243] T 92a4b4a5a07442f49f518b479db8c94b P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:55.173657 21756 raft_consensus.cc:2272] T 92a4b4a5a07442f49f518b479db8c94b P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:55.175717 21756 tablet_replica.cc:333] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
I20260709 13:11:55.176386 21756 raft_consensus.cc:2243] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:55.177349 21756 raft_consensus.cc:2272] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:55.180086 21756 tablet_replica.cc:333] T d7edce5d08f24852b7afca9c57ba9c19 P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
I20260709 13:11:55.180555 21756 raft_consensus.cc:2243] T d7edce5d08f24852b7afca9c57ba9c19 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:55.181067 21756 raft_consensus.cc:2272] T d7edce5d08f24852b7afca9c57ba9c19 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:55.183147 21756 tablet_replica.cc:333] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
I20260709 13:11:55.183688 21756 raft_consensus.cc:2243] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:55.184475 21756 raft_consensus.cc:2272] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:55.186785 21756 tablet_replica.cc:333] T 99e98c1324084d85b8a97aaf0c248df7 P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
I20260709 13:11:55.187501 21756 raft_consensus.cc:2243] T 99e98c1324084d85b8a97aaf0c248df7 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:55.187963 21756 raft_consensus.cc:2272] T 99e98c1324084d85b8a97aaf0c248df7 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:55.190498 21756 tablet_replica.cc:333] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
I20260709 13:11:55.191396 21756 raft_consensus.cc:2243] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:55.192430 21756 raft_consensus.cc:2272] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:55.195429 21756 tablet_replica.cc:333] T bbb2acd63baa48a3a2ace94016522e92 P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
W20260709 13:11:55.195587 30841 consensus_peers.cc:597] T ea563bd67a9849d18c9c3dfb21d58a93 P 02c5705b45c04706bbb2c6ff3d7b0b8a -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:55.196367 21756 raft_consensus.cc:2243] T bbb2acd63baa48a3a2ace94016522e92 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:55.196900 21756 raft_consensus.cc:2272] T bbb2acd63baa48a3a2ace94016522e92 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:55.199244 21756 tablet_replica.cc:333] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
I20260709 13:11:55.199757 21756 raft_consensus.cc:2243] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:55.200490 21756 raft_consensus.cc:2272] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:55.202641 21756 tablet_replica.cc:333] T 93a75acc8893414180a15797033a2d5a P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
I20260709 13:11:55.203240 21756 raft_consensus.cc:2243] T 93a75acc8893414180a15797033a2d5a P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:55.203711 21756 raft_consensus.cc:2272] T 93a75acc8893414180a15797033a2d5a P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:55.206033 21756 tablet_replica.cc:333] T 892368b6886842febffc849d73f76c10 P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
I20260709 13:11:55.206552 21756 raft_consensus.cc:2243] T 892368b6886842febffc849d73f76c10 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:55.206916 21756 raft_consensus.cc:2272] T 892368b6886842febffc849d73f76c10 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:55.208806 21756 tablet_replica.cc:333] T d05da516ed9040c988b1fdfe2fdde9ed P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
I20260709 13:11:55.209357 21756 raft_consensus.cc:2243] T d05da516ed9040c988b1fdfe2fdde9ed P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:55.209775 21756 raft_consensus.cc:2272] T d05da516ed9040c988b1fdfe2fdde9ed P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:55.211743 21756 tablet_replica.cc:333] T 7ed3eac558b34457aa26b1c40652f887 P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
I20260709 13:11:55.212258 21756 raft_consensus.cc:2243] T 7ed3eac558b34457aa26b1c40652f887 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:55.212646 21756 raft_consensus.cc:2272] T 7ed3eac558b34457aa26b1c40652f887 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:55.214672 21756 tablet_replica.cc:333] T b3754a670e0b4ed99d37283bf1647bf5 P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
I20260709 13:11:55.215421 21756 raft_consensus.cc:2243] T b3754a670e0b4ed99d37283bf1647bf5 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:55.215881 21756 raft_consensus.cc:2272] T b3754a670e0b4ed99d37283bf1647bf5 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:55.218497 21756 tablet_replica.cc:333] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
I20260709 13:11:55.219179 21756 raft_consensus.cc:2243] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:55.220093 21756 raft_consensus.cc:2272] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:55.222735 21756 tablet_replica.cc:333] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
W20260709 13:11:55.223101 30767 consensus_peers.cc:597] T 892368b6886842febffc849d73f76c10 P 05213ac836744a53b9c089a5c052f4be -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:55.223454 21756 raft_consensus.cc:2243] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:11:55.224498 30915 consensus_peers.cc:597] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:55.224507 21756 raft_consensus.cc:2272] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:55.227299 21756 tablet_replica.cc:333] T 8b6052cddae64c5c9d16f60272735788 P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
I20260709 13:11:55.228048 21756 raft_consensus.cc:2243] T 8b6052cddae64c5c9d16f60272735788 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:55.228526 21756 raft_consensus.cc:2272] T 8b6052cddae64c5c9d16f60272735788 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:55.230562 21756 tablet_replica.cc:333] T 26172e2c14cd4d178764ed13012c21f7 P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
I20260709 13:11:55.231150 21756 raft_consensus.cc:2243] T 26172e2c14cd4d178764ed13012c21f7 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:55.231542 21756 raft_consensus.cc:2272] T 26172e2c14cd4d178764ed13012c21f7 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:55.233314 21756 tablet_replica.cc:333] T 27f26e218737414893d5c4bfa4b2cd94 P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
I20260709 13:11:55.233915 21756 raft_consensus.cc:2243] T 27f26e218737414893d5c4bfa4b2cd94 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:55.234845 21756 raft_consensus.cc:2272] T 27f26e218737414893d5c4bfa4b2cd94 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:55.237645 21756 tablet_replica.cc:333] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
I20260709 13:11:55.238390 21756 raft_consensus.cc:2243] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:55.239213 21756 raft_consensus.cc:2272] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:55.241262 21756 tablet_replica.cc:333] T 784deee1ea9543df840c58ecaa6f911a P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
I20260709 13:11:55.241798 21756 raft_consensus.cc:2243] T 784deee1ea9543df840c58ecaa6f911a P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:55.242194 21756 raft_consensus.cc:2272] T 784deee1ea9543df840c58ecaa6f911a P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:55.244005 21756 tablet_replica.cc:333] T 437c20f3d53f43048eca57834248cc0d P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
I20260709 13:11:55.244531 21756 raft_consensus.cc:2243] T 437c20f3d53f43048eca57834248cc0d P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:55.244990 21756 raft_consensus.cc:2272] T 437c20f3d53f43048eca57834248cc0d P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:55.246872 21756 tablet_replica.cc:333] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
I20260709 13:11:55.247431 21756 raft_consensus.cc:2243] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:55.248198 21756 raft_consensus.cc:2272] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:55.250000 21756 tablet_replica.cc:333] T bc2a86f27b0f4fb59d7cfa586a23fef1 P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
I20260709 13:11:55.250540 21756 raft_consensus.cc:2243] T bc2a86f27b0f4fb59d7cfa586a23fef1 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:55.250948 21756 raft_consensus.cc:2272] T bc2a86f27b0f4fb59d7cfa586a23fef1 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:55.252959 21756 tablet_replica.cc:333] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
I20260709 13:11:55.253494 21756 raft_consensus.cc:2243] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:55.254293 21756 raft_consensus.cc:2272] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:55.256192 21756 tablet_replica.cc:333] T ea563bd67a9849d18c9c3dfb21d58a93 P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
I20260709 13:11:55.256744 21756 raft_consensus.cc:2243] T ea563bd67a9849d18c9c3dfb21d58a93 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:55.257156 21756 raft_consensus.cc:2272] T ea563bd67a9849d18c9c3dfb21d58a93 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:55.259661 21756 tablet_replica.cc:333] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
I20260709 13:11:55.260211 21756 raft_consensus.cc:2243] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:55.261212 21756 raft_consensus.cc:2272] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:55.263134 21756 tablet_replica.cc:333] T 64a95fa1266848048486722c02f0d4cf P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
I20260709 13:11:55.263617 21756 raft_consensus.cc:2243] T 64a95fa1266848048486722c02f0d4cf P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:55.264000 21756 raft_consensus.cc:2272] T 64a95fa1266848048486722c02f0d4cf P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:55.265800 21756 tablet_replica.cc:333] T ac85f05e63ed463681b3ec790b39c4d8 P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
I20260709 13:11:55.266227 21756 raft_consensus.cc:2243] T ac85f05e63ed463681b3ec790b39c4d8 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:55.266654 21756 raft_consensus.cc:2272] T ac85f05e63ed463681b3ec790b39c4d8 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:55.268725 21756 tablet_replica.cc:333] T 0ac93952b93645b1ad1d7d96cb60c3e9 P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
I20260709 13:11:55.269207 21756 raft_consensus.cc:2243] T 0ac93952b93645b1ad1d7d96cb60c3e9 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:55.270146 21756 raft_consensus.cc:2272] T 0ac93952b93645b1ad1d7d96cb60c3e9 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:55.271894 21756 tablet_replica.cc:333] T 39980eee51514ba493958931f406a145 P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
I20260709 13:11:55.272321 21756 raft_consensus.cc:2243] T 39980eee51514ba493958931f406a145 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:55.272725 21756 raft_consensus.cc:2272] T 39980eee51514ba493958931f406a145 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:55.274420 21756 tablet_replica.cc:333] T a38ff14550f74527a62591df59fb2c51 P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
I20260709 13:11:55.274812 21756 raft_consensus.cc:2243] T a38ff14550f74527a62591df59fb2c51 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:11:55.275122 21756 raft_consensus.cc:2272] T a38ff14550f74527a62591df59fb2c51 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:55.277218 21756 tablet_replica.cc:333] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
I20260709 13:11:55.277668 21756 raft_consensus.cc:2243] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:11:55.278383 21756 raft_consensus.cc:2272] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:11:55.280503 21756 tablet_replica.cc:333] T 7b988f0495c34b7591ad2fb0d5f65693 P d22c9d3d15904327a78a0e548a8ff6cd: stopping tablet replica
I20260709 13:11:55.280982 21756 raft_consensus.cc:2243] T 7b988f0495c34b7591ad2fb0d5f65693 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:11:55.281679 21756 raft_consensus.cc:2272] T 7b988f0495c34b7591ad2fb0d5f65693 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:11:55.293259 30841 consensus_peers.cc:597] T 94220be3f55a4acbbd6bd5723b9f32e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:55.304806 30841 consensus_peers.cc:597] T cae2b1cb8db54cbb812489cc1b4b1213 P 02c5705b45c04706bbb2c6ff3d7b0b8a -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:55.307747 30841 consensus_peers.cc:597] T 99e98c1324084d85b8a97aaf0c248df7 P 02c5705b45c04706bbb2c6ff3d7b0b8a -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:55.316349 30915 consensus_peers.cc:597] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:55.330331 30915 consensus_peers.cc:597] T 39980eee51514ba493958931f406a145 P 262942f67dfa4b8fa42234b18ac31ffd -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:55.330333 30841 consensus_peers.cc:597] T 93a75acc8893414180a15797033a2d5a P 02c5705b45c04706bbb2c6ff3d7b0b8a -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:55.335448 30915 consensus_peers.cc:597] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:55.353148 21756 tablet_server.cc:196] TabletServer@127.21.63.1:0 shutdown complete.
I20260709 13:11:55.365006 31233 consensus_queue.cc:1048] T 99e98c1324084d85b8a97aaf0c248df7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 13:11:55.387744 30841 consensus_peers.cc:597] T 92a4b4a5a07442f49f518b479db8c94b P 02c5705b45c04706bbb2c6ff3d7b0b8a -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:37325: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:55.389959 30767 consensus_peers.cc:597] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 05213ac836744a53b9c089a5c052f4be -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:37325: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:55.418071 30841 consensus_peers.cc:597] T ac85f05e63ed463681b3ec790b39c4d8 P 02c5705b45c04706bbb2c6ff3d7b0b8a -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:37325: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:55.437278 30841 consensus_peers.cc:597] T d5175e0d898c453b8a7641d94dc1ac58 P 02c5705b45c04706bbb2c6ff3d7b0b8a -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:37325: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:55.445596 31233 consensus_queue.cc:1048] T cae2b1cb8db54cbb812489cc1b4b1213 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:11:55.454303 21756 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:11:55.471151 31472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:55.474115 30915 consensus_peers.cc:597] T 8fb24a20af104a7486d0f1b2ae0e3843 P 262942f67dfa4b8fa42234b18ac31ffd -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:37325: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:55.474370 31473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:11:55.474704 30915 consensus_peers.cc:597] T 26172e2c14cd4d178764ed13012c21f7 P 262942f67dfa4b8fa42234b18ac31ffd -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:37325: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:55.484303 30767 consensus_peers.cc:597] T bbb2acd63baa48a3a2ace94016522e92 P 05213ac836744a53b9c089a5c052f4be -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:37325: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:55.488019 30915 consensus_peers.cc:597] T d16c44f56c42466497ba4f7f1b2b5344 P 262942f67dfa4b8fa42234b18ac31ffd -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:37325: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:55.488621 30915 consensus_peers.cc:597] T d7edce5d08f24852b7afca9c57ba9c19 P 262942f67dfa4b8fa42234b18ac31ffd -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:37325: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:55.489143 21756 server_base.cc:1061] running on GCE node
W20260709 13:11:55.489799 31476 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:11:55.493098 21756 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:11:55.493368 21756 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:11:55.493575 21756 hybrid_clock.cc:648] HybridClock initialized: now 1783602715493560 us; error 0 us; skew 500 ppm
I20260709 13:11:55.494117 21756 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:11:55.497324 21756 webserver.cc:533] Webserver started at http://127.21.63.1:33247/ using document root <none> and password file <none>
I20260709 13:11:55.497988 21756 fs_manager.cc:362] Metadata directory not provided
I20260709 13:11:55.498227 21756 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:11:55.505040 21756 fs_manager.cc:714] Time spent opening directory manager: real 0.006s	user 0.003s	sys 0.002s
W20260709 13:11:55.505913 30841 consensus_peers.cc:597] T 437c20f3d53f43048eca57834248cc0d P 02c5705b45c04706bbb2c6ff3d7b0b8a -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:37325: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:55.515795 31480 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:11:55.516953 21756 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 13:11:55.517280 21756 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602429420501-21756-0/minicluster-data/ts-0-root
uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
format_stamp: "Formatted at 2026-07-09 13:11:44 on dist-test-slave-c6mt"
I20260709 13:11:55.517833 21756 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602429420501-21756-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602429420501-21756-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783602429420501-21756-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260709 13:11:55.520818 30841 consensus_peers.cc:597] T 8155b3fc02b348e1adb290ecfe4592c5 P 02c5705b45c04706bbb2c6ff3d7b0b8a -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:37325: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:55.555586 21756 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:11:55.556941 21756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:11:55.558423 21756 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:11:55.564344 31487 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
W20260709 13:11:55.570488 30767 consensus_peers.cc:597] T af695b428b974773ba476dce0d31c5a1 P 05213ac836744a53b9c089a5c052f4be -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:37325: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:55.592355 30767 consensus_peers.cc:597] T 6fe93136933f4d2880ec355e1bbbf90d P 05213ac836744a53b9c089a5c052f4be -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:37325: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:55.604435 30767 consensus_peers.cc:597] T 64a95fa1266848048486722c02f0d4cf P 05213ac836744a53b9c089a5c052f4be -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:37325: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:55.624405 30841 consensus_peers.cc:597] T 784deee1ea9543df840c58ecaa6f911a P 02c5705b45c04706bbb2c6ff3d7b0b8a -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:37325: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:55.643630 30841 consensus_peers.cc:597] T 8b3e41a728f1457ebf97274b95c7b23a P 02c5705b45c04706bbb2c6ff3d7b0b8a -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:37325: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:55.819487 21756 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 13:11:55.819912 21756 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.257s	user 0.003s	sys 0.007s
I20260709 13:11:55.820302 21756 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 13:11:55.825192 31487 tablet_bootstrap.cc:492] T d7edce5d08f24852b7afca9c57ba9c19 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:55.848697 31487 tablet_bootstrap.cc:492] T d7edce5d08f24852b7afca9c57ba9c19 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:55.849560 31487 tablet_bootstrap.cc:492] T d7edce5d08f24852b7afca9c57ba9c19 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:55.850116 31487 ts_tablet_manager.cc:1403] T d7edce5d08f24852b7afca9c57ba9c19 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.025s	user 0.011s	sys 0.012s
I20260709 13:11:55.852037 31487 raft_consensus.cc:359] T d7edce5d08f24852b7afca9c57ba9c19 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:55.852552 31487 raft_consensus.cc:740] T d7edce5d08f24852b7afca9c57ba9c19 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:55.853302 31487 consensus_queue.cc:260] T d7edce5d08f24852b7afca9c57ba9c19 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:55.855367 31487 ts_tablet_manager.cc:1434] T d7edce5d08f24852b7afca9c57ba9c19 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:11:55.856124 31487 tablet_bootstrap.cc:492] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:55.876379 31487 tablet_bootstrap.cc:492] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:55.877210 31487 tablet_bootstrap.cc:492] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:55.877731 31487 ts_tablet_manager.cc:1403] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.008s
I20260709 13:11:55.879891 31487 raft_consensus.cc:359] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:55.880568 31487 raft_consensus.cc:740] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:55.881230 31487 consensus_queue.cc:260] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:55.882843 31487 ts_tablet_manager.cc:1434] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:11:55.883464 31487 tablet_bootstrap.cc:492] T ac85f05e63ed463681b3ec790b39c4d8 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:55.897805 31487 tablet_bootstrap.cc:492] T ac85f05e63ed463681b3ec790b39c4d8 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:55.899312 31487 tablet_bootstrap.cc:492] T ac85f05e63ed463681b3ec790b39c4d8 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:55.899958 31487 ts_tablet_manager.cc:1403] T ac85f05e63ed463681b3ec790b39c4d8 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.017s	user 0.015s	sys 0.000s
I20260709 13:11:55.902309 31487 raft_consensus.cc:359] T ac85f05e63ed463681b3ec790b39c4d8 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:55.903177 31487 raft_consensus.cc:740] T ac85f05e63ed463681b3ec790b39c4d8 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:55.903864 31487 consensus_queue.cc:260] T ac85f05e63ed463681b3ec790b39c4d8 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:55.906226 31487 ts_tablet_manager.cc:1434] T ac85f05e63ed463681b3ec790b39c4d8 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:11:55.907225 31487 tablet_bootstrap.cc:492] T ea563bd67a9849d18c9c3dfb21d58a93 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:55.923990 31487 tablet_bootstrap.cc:492] T ea563bd67a9849d18c9c3dfb21d58a93 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:55.924983 31487 tablet_bootstrap.cc:492] T ea563bd67a9849d18c9c3dfb21d58a93 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:55.925609 31487 ts_tablet_manager.cc:1403] T ea563bd67a9849d18c9c3dfb21d58a93 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.019s	user 0.011s	sys 0.004s
I20260709 13:11:55.928153 31487 raft_consensus.cc:359] T ea563bd67a9849d18c9c3dfb21d58a93 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:55.928869 31487 raft_consensus.cc:740] T ea563bd67a9849d18c9c3dfb21d58a93 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:55.929672 31487 consensus_queue.cc:260] T ea563bd67a9849d18c9c3dfb21d58a93 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:55.931622 31487 ts_tablet_manager.cc:1434] T ea563bd67a9849d18c9c3dfb21d58a93 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:11:55.932396 31487 tablet_bootstrap.cc:492] T 5532e197e7de48f28c034ba1fc490e19 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:55.954335 31487 tablet_bootstrap.cc:492] T 5532e197e7de48f28c034ba1fc490e19 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:55.955251 31487 tablet_bootstrap.cc:492] T 5532e197e7de48f28c034ba1fc490e19 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:55.955771 31487 ts_tablet_manager.cc:1403] T 5532e197e7de48f28c034ba1fc490e19 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.024s	user 0.009s	sys 0.008s
I20260709 13:11:55.957883 31487 raft_consensus.cc:359] T 5532e197e7de48f28c034ba1fc490e19 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:55.958863 31487 raft_consensus.cc:740] T 5532e197e7de48f28c034ba1fc490e19 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:55.959569 31487 consensus_queue.cc:260] T 5532e197e7de48f28c034ba1fc490e19 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:55.961593 31487 ts_tablet_manager.cc:1434] T 5532e197e7de48f28c034ba1fc490e19 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:11:55.962503 31487 tablet_bootstrap.cc:492] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:55.980971 31487 tablet_bootstrap.cc:492] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:55.981943 31487 tablet_bootstrap.cc:492] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:55.982574 31487 ts_tablet_manager.cc:1403] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.004s
I20260709 13:11:55.984977 31487 raft_consensus.cc:359] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:55.985635 31487 raft_consensus.cc:740] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:55.986277 31487 consensus_queue.cc:260] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:55.988155 31487 ts_tablet_manager.cc:1434] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:11:55.988976 31487 tablet_bootstrap.cc:492] T 437c20f3d53f43048eca57834248cc0d P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:56.012022 31487 tablet_bootstrap.cc:492] T 437c20f3d53f43048eca57834248cc0d P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:56.013334 31487 tablet_bootstrap.cc:492] T 437c20f3d53f43048eca57834248cc0d P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:56.014952 31487 ts_tablet_manager.cc:1403] T 437c20f3d53f43048eca57834248cc0d P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.026s	user 0.015s	sys 0.008s
I20260709 13:11:56.017906 31487 raft_consensus.cc:359] T 437c20f3d53f43048eca57834248cc0d P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:56.018626 31487 raft_consensus.cc:740] T 437c20f3d53f43048eca57834248cc0d P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:56.019241 31487 consensus_queue.cc:260] T 437c20f3d53f43048eca57834248cc0d P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:56.021108 31487 ts_tablet_manager.cc:1434] T 437c20f3d53f43048eca57834248cc0d P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:11:56.021865 31487 tablet_bootstrap.cc:492] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:56.036639 31487 tablet_bootstrap.cc:492] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:56.037400 31487 tablet_bootstrap.cc:492] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:56.037995 31487 ts_tablet_manager.cc:1403] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.016s	user 0.006s	sys 0.006s
I20260709 13:11:56.039785 31487 raft_consensus.cc:359] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:56.040482 31487 raft_consensus.cc:740] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:56.041191 31487 consensus_queue.cc:260] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:56.043185 31487 ts_tablet_manager.cc:1434] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.005s	user 0.000s	sys 0.004s
I20260709 13:11:56.043879 31487 tablet_bootstrap.cc:492] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:56.055819 31487 tablet_bootstrap.cc:492] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:56.056773 31487 tablet_bootstrap.cc:492] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:56.057369 31487 ts_tablet_manager.cc:1403] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.014s	user 0.008s	sys 0.004s
I20260709 13:11:56.059677 31487 raft_consensus.cc:359] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:56.060349 31487 raft_consensus.cc:740] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:56.061010 31487 consensus_queue.cc:260] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:56.062439 31487 ts_tablet_manager.cc:1434] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:11:56.063031 31487 tablet_bootstrap.cc:492] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:56.069205 21756 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 13:11:56.069525 21756 ts_tablet_manager.cc:595] Time spent register tablets: real 0.249s	user 0.236s	sys 0.000s
I20260709 13:11:56.105209 31487 tablet_bootstrap.cc:492] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:56.106362 31487 tablet_bootstrap.cc:492] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:56.107082 31487 ts_tablet_manager.cc:1403] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.044s	user 0.026s	sys 0.016s
I20260709 13:11:56.109555 31487 raft_consensus.cc:359] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:56.110245 31487 raft_consensus.cc:740] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:56.110934 31487 consensus_queue.cc:260] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:56.112746 31487 ts_tablet_manager.cc:1434] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.005s	user 0.005s	sys 0.000s
I20260709 13:11:56.113399 31487 tablet_bootstrap.cc:492] T 99e98c1324084d85b8a97aaf0c248df7 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:56.130246 31487 tablet_bootstrap.cc:492] T 99e98c1324084d85b8a97aaf0c248df7 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:56.130954 31487 tablet_bootstrap.cc:492] T 99e98c1324084d85b8a97aaf0c248df7 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:56.131605 31487 ts_tablet_manager.cc:1403] T 99e98c1324084d85b8a97aaf0c248df7 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.018s	user 0.016s	sys 0.000s
I20260709 13:11:56.134142 31487 raft_consensus.cc:359] T 99e98c1324084d85b8a97aaf0c248df7 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:56.134907 31487 raft_consensus.cc:740] T 99e98c1324084d85b8a97aaf0c248df7 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:56.135819 31487 consensus_queue.cc:260] T 99e98c1324084d85b8a97aaf0c248df7 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:56.137863 31487 ts_tablet_manager.cc:1434] T 99e98c1324084d85b8a97aaf0c248df7 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:11:56.138654 31487 tablet_bootstrap.cc:492] T af695b428b974773ba476dce0d31c5a1 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:56.167610 31487 tablet_bootstrap.cc:492] T af695b428b974773ba476dce0d31c5a1 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:56.168723 31487 tablet_bootstrap.cc:492] T af695b428b974773ba476dce0d31c5a1 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:56.169340 31487 ts_tablet_manager.cc:1403] T af695b428b974773ba476dce0d31c5a1 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.031s	user 0.014s	sys 0.000s
I20260709 13:11:56.171725 31487 raft_consensus.cc:359] T af695b428b974773ba476dce0d31c5a1 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:56.172323 31487 raft_consensus.cc:740] T af695b428b974773ba476dce0d31c5a1 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:56.172976 31487 consensus_queue.cc:260] T af695b428b974773ba476dce0d31c5a1 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:56.174841 31487 ts_tablet_manager.cc:1434] T af695b428b974773ba476dce0d31c5a1 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:11:56.175648 31487 tablet_bootstrap.cc:492] T 26172e2c14cd4d178764ed13012c21f7 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:56.199023 31487 tablet_bootstrap.cc:492] T 26172e2c14cd4d178764ed13012c21f7 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:56.199950 31487 tablet_bootstrap.cc:492] T 26172e2c14cd4d178764ed13012c21f7 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:56.200577 31487 ts_tablet_manager.cc:1403] T 26172e2c14cd4d178764ed13012c21f7 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.025s	user 0.014s	sys 0.000s
I20260709 13:11:56.202972 31487 raft_consensus.cc:359] T 26172e2c14cd4d178764ed13012c21f7 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:56.203610 31487 raft_consensus.cc:740] T 26172e2c14cd4d178764ed13012c21f7 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:56.204255 31487 consensus_queue.cc:260] T 26172e2c14cd4d178764ed13012c21f7 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:56.206176 31487 ts_tablet_manager.cc:1434] T 26172e2c14cd4d178764ed13012c21f7 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:11:56.206914 31487 tablet_bootstrap.cc:492] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:56.231953 31487 tablet_bootstrap.cc:492] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:56.233189 31487 tablet_bootstrap.cc:492] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:56.236155 31487 ts_tablet_manager.cc:1403] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.029s	user 0.015s	sys 0.007s
I20260709 13:11:56.238862 31487 raft_consensus.cc:359] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:56.239782 31487 raft_consensus.cc:740] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:56.240675 31487 consensus_queue.cc:260] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:56.242441 31487 ts_tablet_manager.cc:1434] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:11:56.243175 31487 tablet_bootstrap.cc:492] T 93a75acc8893414180a15797033a2d5a P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:56.259588 31487 tablet_bootstrap.cc:492] T 93a75acc8893414180a15797033a2d5a P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:56.260685 31487 tablet_bootstrap.cc:492] T 93a75acc8893414180a15797033a2d5a P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:56.261425 31487 ts_tablet_manager.cc:1403] T 93a75acc8893414180a15797033a2d5a P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.018s	user 0.012s	sys 0.003s
I20260709 13:11:56.264194 31487 raft_consensus.cc:359] T 93a75acc8893414180a15797033a2d5a P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:56.264977 31487 raft_consensus.cc:740] T 93a75acc8893414180a15797033a2d5a P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:56.266021 21756 rpc_server.cc:307] RPC server started. Bound to: 127.21.63.1:37325
I20260709 13:11:56.265805 31487 consensus_queue.cc:260] T 93a75acc8893414180a15797033a2d5a P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:56.266137 31561 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.63.1:37325 every 8 connection(s)
I20260709 13:11:56.268078 31487 ts_tablet_manager.cc:1434] T 93a75acc8893414180a15797033a2d5a P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:11:56.268903 31487 tablet_bootstrap.cc:492] T 92a4b4a5a07442f49f518b479db8c94b P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:56.301172 31487 tablet_bootstrap.cc:492] T 92a4b4a5a07442f49f518b479db8c94b P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:56.302223 31487 tablet_bootstrap.cc:492] T 92a4b4a5a07442f49f518b479db8c94b P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:56.302904 31487 ts_tablet_manager.cc:1403] T 92a4b4a5a07442f49f518b479db8c94b P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.034s	user 0.019s	sys 0.004s
I20260709 13:11:56.305335 31563 heartbeater.cc:344] Connected to a master server at 127.21.63.62:40095
I20260709 13:11:56.305801 31563 heartbeater.cc:461] Registering TS with master...
I20260709 13:11:56.306577 31563 heartbeater.cc:507] Master 127.21.63.62:40095 requested a full tablet report, sending...
I20260709 13:11:56.313500 31233 raft_consensus.cc:493] T 726b808ed64848dd89da4340d4d891fb P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.314072 31233 raft_consensus.cc:515] T 726b808ed64848dd89da4340d4d891fb P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:56.316306 31233 leader_election.cc:290] T 726b808ed64848dd89da4340d4d891fb P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:56.317270 30948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "726b808ed64848dd89da4340d4d891fb" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:56.318073 31534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "726b808ed64848dd89da4340d4d891fb" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:56.318746 31534 raft_consensus.cc:2468] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 1.
I20260709 13:11:56.320004 30841 leader_election.cc:304] T 726b808ed64848dd89da4340d4d891fb P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 262942f67dfa4b8fa42234b18ac31ffd
I20260709 13:11:56.320241 31487 raft_consensus.cc:359] T 92a4b4a5a07442f49f518b479db8c94b P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:56.321005 31487 raft_consensus.cc:740] T 92a4b4a5a07442f49f518b479db8c94b P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:56.321481 31233 raft_consensus.cc:2804] T 726b808ed64848dd89da4340d4d891fb P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:56.321964 31233 raft_consensus.cc:493] T 726b808ed64848dd89da4340d4d891fb P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting leader election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.321658 31487 consensus_queue.cc:260] T 92a4b4a5a07442f49f518b479db8c94b P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:56.322418 31233 raft_consensus.cc:3060] T 726b808ed64848dd89da4340d4d891fb P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:56.323935 31487 ts_tablet_manager.cc:1434] T 92a4b4a5a07442f49f518b479db8c94b P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 13:11:56.324857 31487 tablet_bootstrap.cc:492] T c0100ec8bd1e4a45b3c0ea52281f9204 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:56.325028 30624 ts_manager.cc:194] Re-registered known tserver with Master: d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:56.328084 31233 raft_consensus.cc:515] T 726b808ed64848dd89da4340d4d891fb P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:56.330760 31534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "726b808ed64848dd89da4340d4d891fb" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:56.331324 31534 raft_consensus.cc:3060] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:56.334057 30948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "726b808ed64848dd89da4340d4d891fb" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:11:56.338621 31534 raft_consensus.cc:2468] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 2.
I20260709 13:11:56.339385 31233 leader_election.cc:290] T 726b808ed64848dd89da4340d4d891fb P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 election: Requested vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:56.341274 30841 leader_election.cc:304] T 726b808ed64848dd89da4340d4d891fb P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:56.342407 31419 raft_consensus.cc:2804] T 726b808ed64848dd89da4340d4d891fb P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:56.342958 31419 raft_consensus.cc:697] T 726b808ed64848dd89da4340d4d891fb P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 LEADER]: Becoming Leader. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Running, Role: LEADER
I20260709 13:11:56.343757 31419 consensus_queue.cc:237] T 726b808ed64848dd89da4340d4d891fb P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:56.364894 30622 catalog_manager.cc:5697] T 726b808ed64848dd89da4340d4d891fb P 02c5705b45c04706bbb2c6ff3d7b0b8a reported cstate change: term changed from 1 to 2, leader changed from d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1) to 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3). New cstate: current_term: 2 leader_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:56.369913 31487 tablet_bootstrap.cc:492] T c0100ec8bd1e4a45b3c0ea52281f9204 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:56.370970 31487 tablet_bootstrap.cc:492] T c0100ec8bd1e4a45b3c0ea52281f9204 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:56.371642 31487 ts_tablet_manager.cc:1403] T c0100ec8bd1e4a45b3c0ea52281f9204 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.047s	user 0.028s	sys 0.016s
I20260709 13:11:56.376812 31089 raft_consensus.cc:493] T 27f26e218737414893d5c4bfa4b2cd94 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.377349 31089 raft_consensus.cc:515] T 27f26e218737414893d5c4bfa4b2cd94 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:56.379172 31089 leader_election.cc:290] T 27f26e218737414893d5c4bfa4b2cd94 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:56.379886 30872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27f26e218737414893d5c4bfa4b2cd94" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:56.381220 31532 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27f26e218737414893d5c4bfa4b2cd94" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
W20260709 13:11:56.382601 30915 leader_election.cc:343] T 27f26e218737414893d5c4bfa4b2cd94 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:56.382993 30915 leader_election.cc:304] T 27f26e218737414893d5c4bfa4b2cd94 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:56.383272 31487 raft_consensus.cc:359] T c0100ec8bd1e4a45b3c0ea52281f9204 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:56.383968 31178 raft_consensus.cc:2749] T 27f26e218737414893d5c4bfa4b2cd94 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:56.384050 31487 raft_consensus.cc:740] T c0100ec8bd1e4a45b3c0ea52281f9204 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:56.384974 31487 consensus_queue.cc:260] T c0100ec8bd1e4a45b3c0ea52281f9204 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:56.387003 31487 ts_tablet_manager.cc:1434] T c0100ec8bd1e4a45b3c0ea52281f9204 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.000s
I20260709 13:11:56.387796 31487 tablet_bootstrap.cc:492] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:56.389158 31495 raft_consensus.cc:493] T 8770fcfd98154b2095c2c9f74135593b P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.389690 31495 raft_consensus.cc:515] T 8770fcfd98154b2095c2c9f74135593b P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:56.391180 31488 raft_consensus.cc:493] T 0d8c1da1f7254fde94f7deaad5f41709 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.391762 31488 raft_consensus.cc:515] T 0d8c1da1f7254fde94f7deaad5f41709 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:56.393239 31419 raft_consensus.cc:493] T bcac31d5b7ab48ec9c071a8f77750a88 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.393831 31419 raft_consensus.cc:515] T bcac31d5b7ab48ec9c071a8f77750a88 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:56.391778 31495 leader_election.cc:290] T 8770fcfd98154b2095c2c9f74135593b P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:56.395787 30872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8770fcfd98154b2095c2c9f74135593b" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:56.396134 31419 leader_election.cc:290] T bcac31d5b7ab48ec9c071a8f77750a88 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:56.397241 30948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcac31d5b7ab48ec9c071a8f77750a88" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:56.398442 31531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcac31d5b7ab48ec9c071a8f77750a88" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:56.398762 30871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d8c1da1f7254fde94f7deaad5f41709" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:56.399181 31531 raft_consensus.cc:2468] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 1.
I20260709 13:11:56.400100 31532 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d8c1da1f7254fde94f7deaad5f41709" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:56.400550 31534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8770fcfd98154b2095c2c9f74135593b" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:56.401247 31534 raft_consensus.cc:2468] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 1.
I20260709 13:11:56.402380 30915 leader_election.cc:304] T 8770fcfd98154b2095c2c9f74135593b P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a
W20260709 13:11:56.402560 30767 leader_election.cc:343] T 0d8c1da1f7254fde94f7deaad5f41709 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:56.403043 30767 leader_election.cc:304] T 0d8c1da1f7254fde94f7deaad5f41709 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:56.404757 31460 raft_consensus.cc:2749] T 0d8c1da1f7254fde94f7deaad5f41709 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:56.405911 31178 raft_consensus.cc:2804] T 8770fcfd98154b2095c2c9f74135593b P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:56.406237 30841 leader_election.cc:304] T bcac31d5b7ab48ec9c071a8f77750a88 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 262942f67dfa4b8fa42234b18ac31ffd
I20260709 13:11:56.406494 31178 raft_consensus.cc:493] T 8770fcfd98154b2095c2c9f74135593b P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Starting leader election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.407073 31178 raft_consensus.cc:3060] T 8770fcfd98154b2095c2c9f74135593b P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:56.407142 31419 raft_consensus.cc:2804] T bcac31d5b7ab48ec9c071a8f77750a88 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:56.407692 31488 leader_election.cc:290] T 0d8c1da1f7254fde94f7deaad5f41709 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:56.407747 31419 raft_consensus.cc:493] T bcac31d5b7ab48ec9c071a8f77750a88 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting leader election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.408344 31419 raft_consensus.cc:3060] T bcac31d5b7ab48ec9c071a8f77750a88 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:56.408762 31233 raft_consensus.cc:493] T efddea050a344563877c3e01ee2a4c81 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.409341 31233 raft_consensus.cc:515] T efddea050a344563877c3e01ee2a4c81 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:56.411609 31233 leader_election.cc:290] T efddea050a344563877c3e01ee2a4c81 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:56.418071 31532 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efddea050a344563877c3e01ee2a4c81" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:56.418460 30798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efddea050a344563877c3e01ee2a4c81" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:56.418817 31532 raft_consensus.cc:2468] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 1.
I20260709 13:11:56.418860 31388 raft_consensus.cc:493] T bcac31d5b7ab48ec9c071a8f77750a88 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.420598 31460 raft_consensus.cc:493] T 5028253a91e74a73afca005c77e8b565 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.420300 31419 raft_consensus.cc:515] T bcac31d5b7ab48ec9c071a8f77750a88 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:56.421161 31388 raft_consensus.cc:515] T bcac31d5b7ab48ec9c071a8f77750a88 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:56.421236 31460 raft_consensus.cc:515] T 5028253a91e74a73afca005c77e8b565 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:56.423524 31388 leader_election.cc:290] T bcac31d5b7ab48ec9c071a8f77750a88 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:56.423688 31419 leader_election.cc:290] T bcac31d5b7ab48ec9c071a8f77750a88 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 election: Requested vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:56.420253 30841 leader_election.cc:304] T efddea050a344563877c3e01ee2a4c81 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:56.439947 31233 raft_consensus.cc:2804] T efddea050a344563877c3e01ee2a4c81 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:56.440157 30871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5028253a91e74a73afca005c77e8b565" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:56.443537 31531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5028253a91e74a73afca005c77e8b565" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:56.423883 30948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcac31d5b7ab48ec9c071a8f77750a88" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:11:56.444650 30948 raft_consensus.cc:3060] T bcac31d5b7ab48ec9c071a8f77750a88 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:56.447551 31495 raft_consensus.cc:493] T 726b808ed64848dd89da4340d4d891fb P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.448132 31495 raft_consensus.cc:515] T 726b808ed64848dd89da4340d4d891fb P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:56.450294 31495 leader_election.cc:290] T 726b808ed64848dd89da4340d4d891fb P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
W20260709 13:11:56.450493 30767 leader_election.cc:343] T 5028253a91e74a73afca005c77e8b565 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:56.443697 31532 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcac31d5b7ab48ec9c071a8f77750a88" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:56.451637 31532 raft_consensus.cc:3060] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:56.451943 30766 leader_election.cc:304] T 5028253a91e74a73afca005c77e8b565 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:56.440456 31233 raft_consensus.cc:493] T efddea050a344563877c3e01ee2a4c81 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting leader election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.454663 31233 raft_consensus.cc:3060] T efddea050a344563877c3e01ee2a4c81 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:56.457805 31379 raft_consensus.cc:493] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.458384 31379 raft_consensus.cc:515] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:56.427871 31460 leader_election.cc:290] T 5028253a91e74a73afca005c77e8b565 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:56.466972 31460 raft_consensus.cc:2749] T 5028253a91e74a73afca005c77e8b565 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:56.467550 31534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fb3f8d627b04c8a8971f7e4d3a5c010" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:56.461199 31488 raft_consensus.cc:493] T 5532e197e7de48f28c034ba1fc490e19 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
W20260709 13:11:56.469964 30767 leader_election.cc:343] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:56.468713 31488 raft_consensus.cc:515] T 5532e197e7de48f28c034ba1fc490e19 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:56.475135 31420 raft_consensus.cc:493] T e2376eaea6bc4f209a21a110370da7b2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.475241 31534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5532e197e7de48f28c034ba1fc490e19" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:56.476082 31534 raft_consensus.cc:2468] T 5532e197e7de48f28c034ba1fc490e19 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05213ac836744a53b9c089a5c052f4be in term 1.
I20260709 13:11:56.475945 31420 raft_consensus.cc:515] T e2376eaea6bc4f209a21a110370da7b2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:56.477807 30767 leader_election.cc:304] T 5532e197e7de48f28c034ba1fc490e19 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:56.478164 31420 leader_election.cc:290] T e2376eaea6bc4f209a21a110370da7b2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:56.460618 31379 leader_election.cc:290] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:56.481244 31534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2376eaea6bc4f209a21a110370da7b2" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:56.481556 31379 raft_consensus.cc:2804] T 5532e197e7de48f28c034ba1fc490e19 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:56.482180 31534 raft_consensus.cc:2468] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 1.
I20260709 13:11:56.483913 30841 leader_election.cc:304] T e2376eaea6bc4f209a21a110370da7b2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:56.484841 31420 raft_consensus.cc:2804] T e2376eaea6bc4f209a21a110370da7b2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:56.484746 31178 raft_consensus.cc:515] T 8770fcfd98154b2095c2c9f74135593b P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:56.486549 31420 raft_consensus.cc:493] T e2376eaea6bc4f209a21a110370da7b2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting leader election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.487094 31420 raft_consensus.cc:3060] T e2376eaea6bc4f209a21a110370da7b2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:56.488164 31178 leader_election.cc:290] T 8770fcfd98154b2095c2c9f74135593b P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 2 election: Requested vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:56.490162 31419 raft_consensus.cc:493] T 8770fcfd98154b2095c2c9f74135593b P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.490904 31419 raft_consensus.cc:515] T 8770fcfd98154b2095c2c9f74135593b P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:56.482174 31379 raft_consensus.cc:493] T 5532e197e7de48f28c034ba1fc490e19 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Starting leader election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.491897 31379 raft_consensus.cc:3060] T 5532e197e7de48f28c034ba1fc490e19 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:56.495501 31233 raft_consensus.cc:515] T efddea050a344563877c3e01ee2a4c81 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:56.497084 31487 tablet_bootstrap.cc:492] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:56.498011 31487 tablet_bootstrap.cc:492] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:56.498642 31487 ts_tablet_manager.cc:1403] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.111s	user 0.032s	sys 0.039s
I20260709 13:11:56.501130 31487 raft_consensus.cc:359] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:56.501912 31487 raft_consensus.cc:740] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:56.502566 31487 consensus_queue.cc:260] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:56.492914 31419 leader_election.cc:290] T 8770fcfd98154b2095c2c9f74135593b P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:56.475072 31488 leader_election.cc:290] T 5532e197e7de48f28c034ba1fc490e19 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:56.493887 30948 raft_consensus.cc:2468] T bcac31d5b7ab48ec9c071a8f77750a88 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 2.
I20260709 13:11:56.493908 30946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8770fcfd98154b2095c2c9f74135593b" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:56.505333 31419 raft_consensus.cc:493] T e28f28c3f05f40e68531e20dd7c1a1a0 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.505692 30946 raft_consensus.cc:2393] T 8770fcfd98154b2095c2c9f74135593b P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in current term 2: Already voted for candidate 262942f67dfa4b8fa42234b18ac31ffd in this term.
I20260709 13:11:56.507055 31178 raft_consensus.cc:493] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.507580 31178 raft_consensus.cc:515] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:56.507928 30839 leader_election.cc:304] T bcac31d5b7ab48ec9c071a8f77750a88 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
I20260709 13:11:56.509922 31178 leader_election.cc:290] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:56.509670 31420 raft_consensus.cc:515] T e2376eaea6bc4f209a21a110370da7b2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:56.511508 31233 leader_election.cc:290] T efddea050a344563877c3e01ee2a4c81 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 election: Requested vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:56.512283 31233 raft_consensus.cc:2804] T bcac31d5b7ab48ec9c071a8f77750a88 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Leader election won for term 2
W20260709 13:11:56.516949 30766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:43993, user_credentials={real_user=slave}} blocked reactor thread for 65097.6us
I20260709 13:11:56.518627 30871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5532e197e7de48f28c034ba1fc490e19" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:56.520623 31488 raft_consensus.cc:493] T e2376eaea6bc4f209a21a110370da7b2 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.520938 31460 raft_consensus.cc:493] T 1474b6fc151c4a3994a3aef89ea936a2 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.521200 31488 raft_consensus.cc:515] T e2376eaea6bc4f209a21a110370da7b2 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:56.521482 31460 raft_consensus.cc:515] T 1474b6fc151c4a3994a3aef89ea936a2 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
W20260709 13:11:56.522838 30915 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:37325, user_credentials={real_user=slave}} blocked reactor thread for 102745us
I20260709 13:11:56.523590 31460 leader_election.cc:290] T 1474b6fc151c4a3994a3aef89ea936a2 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:56.524400 31488 leader_election.cc:290] T e2376eaea6bc4f209a21a110370da7b2 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:56.525025 31534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2376eaea6bc4f209a21a110370da7b2" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:56.525189 30948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fb3f8d627b04c8a8971f7e4d3a5c010" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:56.525697 31534 raft_consensus.cc:2468] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05213ac836744a53b9c089a5c052f4be in term 1.
I20260709 13:11:56.526646 31531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1474b6fc151c4a3994a3aef89ea936a2" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:56.526780 31532 raft_consensus.cc:2468] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 2.
I20260709 13:11:56.527457 31089 raft_consensus.cc:493] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.527908 31089 raft_consensus.cc:515] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:56.528132 31530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcac31d5b7ab48ec9c071a8f77750a88" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:56.528302 30765 leader_election.cc:304] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be; no voters: 262942f67dfa4b8fa42234b18ac31ffd, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:56.529584 31089 leader_election.cc:290] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:56.529667 31460 raft_consensus.cc:2749] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:56.530109 31233 raft_consensus.cc:697] T bcac31d5b7ab48ec9c071a8f77750a88 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 LEADER]: Becoming Leader. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Running, Role: LEADER
I20260709 13:11:56.531117 31528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01cceb88bca348958b59f3697979dff1" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:56.531769 31528 raft_consensus.cc:2468] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 1.
I20260709 13:11:56.532174 31530 raft_consensus.cc:2393] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 262942f67dfa4b8fa42234b18ac31ffd in current term 2: Already voted for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in this term.
I20260709 13:11:56.532078 30871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1474b6fc151c4a3994a3aef89ea936a2" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:56.533149 30767 leader_election.cc:304] T e2376eaea6bc4f209a21a110370da7b2 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:56.533530 31487 ts_tablet_manager.cc:1434] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.000s
W20260709 13:11:56.534054 30767 leader_election.cc:343] T 1474b6fc151c4a3994a3aef89ea936a2 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:56.534567 31487 tablet_bootstrap.cc:492] T 784deee1ea9543df840c58ecaa6f911a P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:56.534332 30871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2376eaea6bc4f209a21a110370da7b2" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:56.534652 31460 raft_consensus.cc:2804] T e2376eaea6bc4f209a21a110370da7b2 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:56.535122 31460 raft_consensus.cc:493] T e2376eaea6bc4f209a21a110370da7b2 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Starting leader election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.535317 30871 raft_consensus.cc:2393] T e2376eaea6bc4f209a21a110370da7b2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05213ac836744a53b9c089a5c052f4be in current term 2: Already voted for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in this term.
I20260709 13:11:56.535548 31534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8770fcfd98154b2095c2c9f74135593b" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:56.536108 31460 raft_consensus.cc:3060] T e2376eaea6bc4f209a21a110370da7b2 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:56.537220 30915 leader_election.cc:304] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:56.538069 31531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ac93952b93645b1ad1d7d96cb60c3e9" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:56.536142 31534 raft_consensus.cc:3060] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:56.505842 31419 raft_consensus.cc:515] T e28f28c3f05f40e68531e20dd7c1a1a0 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:56.539546 31089 raft_consensus.cc:2804] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:56.540117 31089 raft_consensus.cc:493] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Starting leader election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.540536 31089 raft_consensus.cc:3060] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:56.541195 31419 leader_election.cc:290] T e28f28c3f05f40e68531e20dd7c1a1a0 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:56.543980 31488 raft_consensus.cc:493] T efddea050a344563877c3e01ee2a4c81 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.544480 31488 raft_consensus.cc:515] T efddea050a344563877c3e01ee2a4c81 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:56.548543 31420 leader_election.cc:290] T e2376eaea6bc4f209a21a110370da7b2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 election: Requested vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:56.549614 31420 raft_consensus.cc:493] T 5532e197e7de48f28c034ba1fc490e19 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.550107 31420 raft_consensus.cc:515] T 5532e197e7de48f28c034ba1fc490e19 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:56.550405 31488 leader_election.cc:290] T efddea050a344563877c3e01ee2a4c81 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:56.550635 31531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efddea050a344563877c3e01ee2a4c81" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:56.551299 31531 raft_consensus.cc:2468] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05213ac836744a53b9c089a5c052f4be in term 1.
I20260709 13:11:56.552142 31420 leader_election.cc:290] T 5532e197e7de48f28c034ba1fc490e19 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:56.552579 30767 leader_election.cc:304] T efddea050a344563877c3e01ee2a4c81 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:56.553584 31089 raft_consensus.cc:515] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
W20260709 13:11:56.554648 30915 leader_election.cc:343] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:56.556885 31578 raft_consensus.cc:2804] T efddea050a344563877c3e01ee2a4c81 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:56.557281 31578 raft_consensus.cc:493] T efddea050a344563877c3e01ee2a4c81 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Starting leader election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.557657 31578 raft_consensus.cc:3060] T efddea050a344563877c3e01ee2a4c81 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:56.558471 30798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2376eaea6bc4f209a21a110370da7b2" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:56.559787 30796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efddea050a344563877c3e01ee2a4c81" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:56.560065 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2376eaea6bc4f209a21a110370da7b2" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:56.560804 30795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5532e197e7de48f28c034ba1fc490e19" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:56.561012 30794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e28f28c3f05f40e68531e20dd7c1a1a0" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:56.562438 31419 raft_consensus.cc:493] T 5028253a91e74a73afca005c77e8b565 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.563027 31419 raft_consensus.cc:515] T 5028253a91e74a73afca005c77e8b565 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:56.520382 31379 raft_consensus.cc:515] T 5532e197e7de48f28c034ba1fc490e19 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:56.565567 31089 leader_election.cc:290] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 2 election: Requested vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:56.565821 30795 raft_consensus.cc:2393] T 5532e197e7de48f28c034ba1fc490e19 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in current term 2: Already voted for candidate 05213ac836744a53b9c089a5c052f4be in this term.
I20260709 13:11:56.567200 31531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01cceb88bca348958b59f3697979dff1" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:56.567836 31531 raft_consensus.cc:3060] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:56.568634 31534 raft_consensus.cc:2468] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 2.
I20260709 13:11:56.570593 31532 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "726b808ed64848dd89da4340d4d891fb" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:56.571408 31532 raft_consensus.cc:2393] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 262942f67dfa4b8fa42234b18ac31ffd in current term 2: Already voted for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in this term.
I20260709 13:11:56.571842 31419 leader_election.cc:290] T 5028253a91e74a73afca005c77e8b565 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:56.573798 31419 raft_consensus.cc:493] T 01cceb88bca348958b59f3697979dff1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.576505 30915 leader_election.cc:304] T 8770fcfd98154b2095c2c9f74135593b P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:56.576946 31419 raft_consensus.cc:515] T 01cceb88bca348958b59f3697979dff1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:56.577849 31574 raft_consensus.cc:2804] T 8770fcfd98154b2095c2c9f74135593b P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:56.578423 31574 raft_consensus.cc:697] T 8770fcfd98154b2095c2c9f74135593b P 262942f67dfa4b8fa42234b18ac31ffd [term 2 LEADER]: Becoming Leader. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Running, Role: LEADER
I20260709 13:11:56.576948 31460 raft_consensus.cc:515] T e2376eaea6bc4f209a21a110370da7b2 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:56.579646 30798 raft_consensus.cc:2393] T e2376eaea6bc4f209a21a110370da7b2 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in current term 2: Already voted for candidate 05213ac836744a53b9c089a5c052f4be in this term.
I20260709 13:11:56.579309 31574 consensus_queue.cc:237] T 8770fcfd98154b2095c2c9f74135593b P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:56.574223 30624 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37818
I20260709 13:11:56.582283 31460 leader_election.cc:290] T e2376eaea6bc4f209a21a110370da7b2 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 2 election: Requested vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:56.583060 30948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01cceb88bca348958b59f3697979dff1" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:56.584028 30948 raft_consensus.cc:2393] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in current term 2: Already voted for candidate 262942f67dfa4b8fa42234b18ac31ffd in this term.
I20260709 13:11:56.584695 31531 raft_consensus.cc:2468] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 2.
I20260709 13:11:56.586544 30915 leader_election.cc:304] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:56.586710 31578 raft_consensus.cc:515] T efddea050a344563877c3e01ee2a4c81 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:56.589193 31419 leader_election.cc:290] T 01cceb88bca348958b59f3697979dff1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:56.590227 30796 raft_consensus.cc:2393] T efddea050a344563877c3e01ee2a4c81 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in current term 2: Already voted for candidate 05213ac836744a53b9c089a5c052f4be in this term.
I20260709 13:11:56.591652 31532 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2376eaea6bc4f209a21a110370da7b2" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:56.573199 31379 leader_election.cc:290] T 5532e197e7de48f28c034ba1fc490e19 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 2 election: Requested vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:56.581537 31534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5532e197e7de48f28c034ba1fc490e19" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:56.596166 31532 raft_consensus.cc:3060] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:56.596333 31534 raft_consensus.cc:3060] T 5532e197e7de48f28c034ba1fc490e19 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:56.592053 31531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efddea050a344563877c3e01ee2a4c81" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:56.593019 31563 heartbeater.cc:499] Master 127.21.63.62:40095 was elected leader, sending a full tablet report...
I20260709 13:11:56.597326 31531 raft_consensus.cc:3060] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:56.604151 31532 raft_consensus.cc:2468] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05213ac836744a53b9c089a5c052f4be in term 2.
I20260709 13:11:56.605036 31531 raft_consensus.cc:2468] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05213ac836744a53b9c089a5c052f4be in term 2.
I20260709 13:11:56.605635 30767 leader_election.cc:304] T e2376eaea6bc4f209a21a110370da7b2 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:56.606683 30767 leader_election.cc:304] T efddea050a344563877c3e01ee2a4c81 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:56.607342 31460 raft_consensus.cc:2804] T e2376eaea6bc4f209a21a110370da7b2 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:56.608286 31460 raft_consensus.cc:697] T e2376eaea6bc4f209a21a110370da7b2 P 05213ac836744a53b9c089a5c052f4be [term 2 LEADER]: Becoming Leader. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Running, Role: LEADER
I20260709 13:11:56.609239 31460 consensus_queue.cc:237] T e2376eaea6bc4f209a21a110370da7b2 P 05213ac836744a53b9c089a5c052f4be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:56.609903 31575 raft_consensus.cc:2804] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:56.607403 31379 raft_consensus.cc:2804] T efddea050a344563877c3e01ee2a4c81 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:56.611162 31575 raft_consensus.cc:697] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 LEADER]: Becoming Leader. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Running, Role: LEADER
I20260709 13:11:56.611231 31379 raft_consensus.cc:697] T efddea050a344563877c3e01ee2a4c81 P 05213ac836744a53b9c089a5c052f4be [term 2 LEADER]: Becoming Leader. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Running, Role: LEADER
I20260709 13:11:56.612262 31379 consensus_queue.cc:237] T efddea050a344563877c3e01ee2a4c81 P 05213ac836744a53b9c089a5c052f4be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:56.612316 31575 consensus_queue.cc:237] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:56.530978 31233 consensus_queue.cc:237] T bcac31d5b7ab48ec9c071a8f77750a88 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:11:56.610018 31534 raft_consensus.cc:2468] T 5532e197e7de48f28c034ba1fc490e19 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05213ac836744a53b9c089a5c052f4be in term 2.
I20260709 13:11:56.617661 30767 leader_election.cc:304] T 5532e197e7de48f28c034ba1fc490e19 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:56.618659 31488 raft_consensus.cc:2804] T 5532e197e7de48f28c034ba1fc490e19 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:56.619236 31488 raft_consensus.cc:697] T 5532e197e7de48f28c034ba1fc490e19 P 05213ac836744a53b9c089a5c052f4be [term 2 LEADER]: Becoming Leader. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Running, Role: LEADER
I20260709 13:11:56.576254 30766 leader_election.cc:304] T 1474b6fc151c4a3994a3aef89ea936a2 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:56.593252 31578 leader_election.cc:290] T efddea050a344563877c3e01ee2a4c81 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 2 election: Requested vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:56.620352 31488 consensus_queue.cc:237] T 5532e197e7de48f28c034ba1fc490e19 P 05213ac836744a53b9c089a5c052f4be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:56.621558 31578 raft_consensus.cc:2749] T 1474b6fc151c4a3994a3aef89ea936a2 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:11:56.622638 30796 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5028253a91e74a73afca005c77e8b565" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:56.623298 30796 raft_consensus.cc:2468] T 5028253a91e74a73afca005c77e8b565 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 1.
I20260709 13:11:56.624226 31577 raft_consensus.cc:493] T 7b988f0495c34b7591ad2fb0d5f65693 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.624787 31577 raft_consensus.cc:515] T 7b988f0495c34b7591ad2fb0d5f65693 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:56.627188 31577 leader_election.cc:290] T 7b988f0495c34b7591ad2fb0d5f65693 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:11:56.629284 31578 raft_consensus.cc:493] T e28f28c3f05f40e68531e20dd7c1a1a0 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.630466 30948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b988f0495c34b7591ad2fb0d5f65693" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:56.633047 31388 raft_consensus.cc:493] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.633750 31531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b988f0495c34b7591ad2fb0d5f65693" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:56.633654 31388 raft_consensus.cc:515] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
W20260709 13:11:56.635238 30767 leader_election.cc:343] T 7b988f0495c34b7591ad2fb0d5f65693 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:56.635773 30767 leader_election.cc:304] T 7b988f0495c34b7591ad2fb0d5f65693 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be; no voters: 262942f67dfa4b8fa42234b18ac31ffd, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:56.636065 31388 leader_election.cc:290] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:56.636718 31577 raft_consensus.cc:2749] T 7b988f0495c34b7591ad2fb0d5f65693 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:56.639489 30871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcac31d5b7ab48ec9c071a8f77750a88" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:56.640986 31530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2376eaea6bc4f209a21a110370da7b2" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:56.641865 31530 raft_consensus.cc:2393] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in current term 2: Already voted for candidate 05213ac836744a53b9c089a5c052f4be in this term.
I20260709 13:11:56.641750 30873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "726b808ed64848dd89da4340d4d891fb" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:56.642915 31529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5532e197e7de48f28c034ba1fc490e19" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:56.629963 31578 raft_consensus.cc:515] T e28f28c3f05f40e68531e20dd7c1a1a0 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:56.643316 31528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e28f28c3f05f40e68531e20dd7c1a1a0" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:56.643741 31529 raft_consensus.cc:2393] T 5532e197e7de48f28c034ba1fc490e19 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in current term 2: Already voted for candidate 05213ac836744a53b9c089a5c052f4be in this term.
I20260709 13:11:56.644076 31528 raft_consensus.cc:2468] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 1.
I20260709 13:11:56.645145 30872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8770fcfd98154b2095c2c9f74135593b" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:11:56.645823 30872 raft_consensus.cc:3060] T 8770fcfd98154b2095c2c9f74135593b P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:56.647531 31578 leader_election.cc:290] T e28f28c3f05f40e68531e20dd7c1a1a0 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:11:56.645848 31532 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efddea050a344563877c3e01ee2a4c81" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:56.649088 30875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01cceb88bca348958b59f3697979dff1" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:56.649655 31532 raft_consensus.cc:2393] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in current term 2: Already voted for candidate 05213ac836744a53b9c089a5c052f4be in this term.
I20260709 13:11:56.649751 30875 raft_consensus.cc:2468] T 01cceb88bca348958b59f3697979dff1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 1.
I20260709 13:11:56.650378 30871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ac93952b93645b1ad1d7d96cb60c3e9" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:56.650866 31527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5028253a91e74a73afca005c77e8b565" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:56.651968 30871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01cceb88bca348958b59f3697979dff1" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:11:56.652529 30871 raft_consensus.cc:3060] T 01cceb88bca348958b59f3697979dff1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:56.655027 31527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01cceb88bca348958b59f3697979dff1" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:56.661425 30871 raft_consensus.cc:2468] T 01cceb88bca348958b59f3697979dff1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 2.
I20260709 13:11:56.661580 31534 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e28f28c3f05f40e68531e20dd7c1a1a0" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:56.662336 31534 raft_consensus.cc:2468] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 05213ac836744a53b9c089a5c052f4be in term 1.
I20260709 13:11:56.663314 30767 leader_election.cc:304] T e28f28c3f05f40e68531e20dd7c1a1a0 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
W20260709 13:11:56.643755 30766 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:43993, user_credentials={real_user=slave}} blocked reactor thread for 67590.3us
I20260709 13:11:56.664088 31379 raft_consensus.cc:2804] T e28f28c3f05f40e68531e20dd7c1a1a0 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:56.664933 31379 raft_consensus.cc:493] T e28f28c3f05f40e68531e20dd7c1a1a0 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Starting leader election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.665354 31379 raft_consensus.cc:3060] T e28f28c3f05f40e68531e20dd7c1a1a0 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:56.666949 30838 leader_election.cc:304] T 5028253a91e74a73afca005c77e8b565 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 05213ac836744a53b9c089a5c052f4be; no voters: 
I20260709 13:11:56.667251 30798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fb3f8d627b04c8a8971f7e4d3a5c010" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:56.667974 30798 raft_consensus.cc:2468] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 1.
I20260709 13:11:56.670921 31582 raft_consensus.cc:493] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.671643 31582 raft_consensus.cc:515] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:56.677876 30948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ac93952b93645b1ad1d7d96cb60c3e9" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:56.678529 30948 raft_consensus.cc:2468] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 1.
I20260709 13:11:56.679762 30839 leader_election.cc:304] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
I20260709 13:11:56.677882 31527 raft_consensus.cc:2393] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in current term 2: Already voted for candidate 262942f67dfa4b8fa42234b18ac31ffd in this term.
I20260709 13:11:56.681311 31419 raft_consensus.cc:2804] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:56.681766 31419 raft_consensus.cc:493] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting leader election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.682124 31419 raft_consensus.cc:3060] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:56.683916 31582 leader_election.cc:290] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:56.689050 31531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8770fcfd98154b2095c2c9f74135593b" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:56.692201 31527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ac93952b93645b1ad1d7d96cb60c3e9" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:56.702889 31531 raft_consensus.cc:2393] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in current term 2: Already voted for candidate 262942f67dfa4b8fa42234b18ac31ffd in this term.
I20260709 13:11:56.688608 30841 leader_election.cc:304] T e2376eaea6bc4f209a21a110370da7b2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a; no voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:56.697227 31596 raft_consensus.cc:2804] T 5028253a91e74a73afca005c77e8b565 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:56.708022 30912 leader_election.cc:304] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
I20260709 13:11:56.698941 30914 leader_election.cc:304] T bcac31d5b7ab48ec9c071a8f77750a88 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:56.709755 31590 raft_consensus.cc:2804] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:56.710223 30914 leader_election.cc:304] T 726b808ed64848dd89da4340d4d891fb P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:56.690243 31420 raft_consensus.cc:493] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.711124 31420 raft_consensus.cc:515] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:56.708387 31596 raft_consensus.cc:493] T 5028253a91e74a73afca005c77e8b565 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting leader election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.712320 31178 raft_consensus.cc:3060] T 726b808ed64848dd89da4340d4d891fb P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:56.712585 31596 raft_consensus.cc:3060] T 5028253a91e74a73afca005c77e8b565 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:56.713156 31420 leader_election.cc:290] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:56.713677 31420 raft_consensus.cc:2749] T e2376eaea6bc4f209a21a110370da7b2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:56.712355 30914 leader_election.cc:304] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:56.701719 31379 raft_consensus.cc:515] T e28f28c3f05f40e68531e20dd7c1a1a0 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:56.710355 31495 raft_consensus.cc:2749] T bcac31d5b7ab48ec9c071a8f77750a88 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:56.717948 30624 catalog_manager.cc:5697] T e2376eaea6bc4f209a21a110370da7b2 P 05213ac836744a53b9c089a5c052f4be reported cstate change: term changed from 1 to 2, leader changed from d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1) to 05213ac836744a53b9c089a5c052f4be (127.21.63.2). New cstate: current_term: 2 leader_uuid: "05213ac836744a53b9c089a5c052f4be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:56.719069 31495 raft_consensus.cc:2749] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:56.710255 31590 raft_consensus.cc:493] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Starting leader election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.721215 31590 raft_consensus.cc:3060] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:56.695271 31532 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fb3f8d627b04c8a8971f7e4d3a5c010" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:56.695628 30872 raft_consensus.cc:2468] T 8770fcfd98154b2095c2c9f74135593b P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 2.
I20260709 13:11:56.722939 31531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e28f28c3f05f40e68531e20dd7c1a1a0" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:56.732697 31531 raft_consensus.cc:3060] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:56.733227 30798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d8c1da1f7254fde94f7deaad5f41709" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:56.733894 30798 raft_consensus.cc:2468] T 0d8c1da1f7254fde94f7deaad5f41709 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 1.
I20260709 13:11:56.739773 31531 raft_consensus.cc:2468] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05213ac836744a53b9c089a5c052f4be in term 2.
I20260709 13:11:56.741127 30767 leader_election.cc:304] T e28f28c3f05f40e68531e20dd7c1a1a0 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:11:56.725909 31379 leader_election.cc:290] T e28f28c3f05f40e68531e20dd7c1a1a0 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 2 election: Requested vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
W20260709 13:11:56.726262 30915 leader_election.cc:343] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:56.742738 31379 raft_consensus.cc:2804] T e28f28c3f05f40e68531e20dd7c1a1a0 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:56.743304 31379 raft_consensus.cc:697] T e28f28c3f05f40e68531e20dd7c1a1a0 P 05213ac836744a53b9c089a5c052f4be [term 2 LEADER]: Becoming Leader. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Running, Role: LEADER
I20260709 13:11:56.721904 31178 raft_consensus.cc:2749] T 726b808ed64848dd89da4340d4d891fb P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:56.743317 30622 catalog_manager.cc:5697] T 8770fcfd98154b2095c2c9f74135593b P 262942f67dfa4b8fa42234b18ac31ffd reported cstate change: term changed from 1 to 2, leader changed from d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1) to 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4). New cstate: current_term: 2 leader_uuid: "262942f67dfa4b8fa42234b18ac31ffd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:56.747931 30871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efddea050a344563877c3e01ee2a4c81" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:56.748903 30871 raft_consensus.cc:2393] T efddea050a344563877c3e01ee2a4c81 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05213ac836744a53b9c089a5c052f4be in current term 2: Already voted for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in this term.
I20260709 13:11:56.750799 30871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5532e197e7de48f28c034ba1fc490e19" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:11:56.751621 30871 raft_consensus.cc:3060] T 5532e197e7de48f28c034ba1fc490e19 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:56.753225 31590 raft_consensus.cc:515] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
W20260709 13:11:56.757123 30841 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:37325, user_credentials={real_user=slave}} blocked reactor thread for 68595.7us
I20260709 13:11:56.757963 30841 leader_election.cc:304] T 5532e197e7de48f28c034ba1fc490e19 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a; no voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:56.758935 31420 raft_consensus.cc:493] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.759107 30841 leader_election.cc:304] T e28f28c3f05f40e68531e20dd7c1a1a0 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 05213ac836744a53b9c089a5c052f4be
I20260709 13:11:56.759522 31420 raft_consensus.cc:515] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:56.760306 30841 leader_election.cc:304] T efddea050a344563877c3e01ee2a4c81 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a; no voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:56.761688 31590 leader_election.cc:290] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 2 election: Requested vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:56.761929 31420 leader_election.cc:290] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:56.763902 30798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fb3f8d627b04c8a8971f7e4d3a5c010" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:56.764766 30798 raft_consensus.cc:3060] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:56.766193 30841 leader_election.cc:343] T 5028253a91e74a73afca005c77e8b565 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:56.767527 30841 leader_election.cc:304] T 01cceb88bca348958b59f3697979dff1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a; no voters: 262942f67dfa4b8fa42234b18ac31ffd, d22c9d3d15904327a78a0e548a8ff6cd
W20260709 13:11:56.769232 30841 leader_election.cc:343] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:56.770180 30841 leader_election.cc:304] T 8770fcfd98154b2095c2c9f74135593b P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a; no voters: 262942f67dfa4b8fa42234b18ac31ffd, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:56.773895 31582 raft_consensus.cc:2804] T e28f28c3f05f40e68531e20dd7c1a1a0 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:56.774400 31582 raft_consensus.cc:493] T e28f28c3f05f40e68531e20dd7c1a1a0 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting leader election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.774890 31582 raft_consensus.cc:3060] T e28f28c3f05f40e68531e20dd7c1a1a0 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:56.744216 31379 consensus_queue.cc:237] T e28f28c3f05f40e68531e20dd7c1a1a0 P 05213ac836744a53b9c089a5c052f4be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:56.700276 31591 raft_consensus.cc:493] T 27f26e218737414893d5c4bfa4b2cd94 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.795385 31591 raft_consensus.cc:515] T 27f26e218737414893d5c4bfa4b2cd94 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:56.775485 31599 raft_consensus.cc:2749] T efddea050a344563877c3e01ee2a4c81 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:56.804796 31574 raft_consensus.cc:493] T 7b988f0495c34b7591ad2fb0d5f65693 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.805480 31574 raft_consensus.cc:515] T 7b988f0495c34b7591ad2fb0d5f65693 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:56.776818 31595 raft_consensus.cc:2749] T 01cceb88bca348958b59f3697979dff1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:56.809221 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b988f0495c34b7591ad2fb0d5f65693" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:56.810328 30799 raft_consensus.cc:2468] T 7b988f0495c34b7591ad2fb0d5f65693 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 1.
I20260709 13:11:56.811951 30912 leader_election.cc:304] T 7b988f0495c34b7591ad2fb0d5f65693 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
I20260709 13:11:56.812942 31574 leader_election.cc:290] T 7b988f0495c34b7591ad2fb0d5f65693 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:56.813905 31574 raft_consensus.cc:2804] T 7b988f0495c34b7591ad2fb0d5f65693 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:56.814800 31574 raft_consensus.cc:493] T 7b988f0495c34b7591ad2fb0d5f65693 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Starting leader election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.777604 31579 raft_consensus.cc:2749] T 8770fcfd98154b2095c2c9f74135593b P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:56.816069 31574 raft_consensus.cc:3060] T 7b988f0495c34b7591ad2fb0d5f65693 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:11:56.784622 30766 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.21.63.62:40095, user_credentials={real_user=slave}} blocked reactor thread for 46959.9us
I20260709 13:11:56.731031 31596 raft_consensus.cc:515] T 5028253a91e74a73afca005c77e8b565 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:56.834522 30798 raft_consensus.cc:2468] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 2.
I20260709 13:11:56.779253 31419 raft_consensus.cc:515] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:56.826674 30622 catalog_manager.cc:5697] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd reported cstate change: term changed from 1 to 2, leader changed from d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1) to 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4). New cstate: current_term: 2 leader_uuid: "262942f67dfa4b8fa42234b18ac31ffd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:56.841940 31487 tablet_bootstrap.cc:492] T 784deee1ea9543df840c58ecaa6f911a P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:56.836627 30912 leader_election.cc:304] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
I20260709 13:11:56.843654 31178 raft_consensus.cc:2804] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:56.844100 31178 raft_consensus.cc:697] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 LEADER]: Becoming Leader. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Running, Role: LEADER
I20260709 13:11:56.838119 30623 catalog_manager.cc:5697] T efddea050a344563877c3e01ee2a4c81 P 05213ac836744a53b9c089a5c052f4be reported cstate change: term changed from 1 to 2, leader changed from d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1) to 05213ac836744a53b9c089a5c052f4be (127.21.63.2). New cstate: current_term: 2 leader_uuid: "05213ac836744a53b9c089a5c052f4be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:56.845212 31178 consensus_queue.cc:237] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:56.847015 30623 catalog_manager.cc:5697] T 5532e197e7de48f28c034ba1fc490e19 P 05213ac836744a53b9c089a5c052f4be reported cstate change: term changed from 1 to 2, leader changed from d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1) to 05213ac836744a53b9c089a5c052f4be (127.21.63.2). New cstate: current_term: 2 leader_uuid: "05213ac836744a53b9c089a5c052f4be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:56.867455 30875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "efddea050a344563877c3e01ee2a4c81" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:11:56.869403 30873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e28f28c3f05f40e68531e20dd7c1a1a0" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" is_pre_election: true
I20260709 13:11:56.872747 30875 raft_consensus.cc:2393] T efddea050a344563877c3e01ee2a4c81 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 05213ac836744a53b9c089a5c052f4be in current term 2: Already voted for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in this term.
I20260709 13:11:56.882977 30798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1474b6fc151c4a3994a3aef89ea936a2" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be" is_pre_election: true
I20260709 13:11:56.883687 30798 raft_consensus.cc:2468] T 1474b6fc151c4a3994a3aef89ea936a2 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 2.
I20260709 13:11:56.894467 30871 raft_consensus.cc:2468] T 5532e197e7de48f28c034ba1fc490e19 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05213ac836744a53b9c089a5c052f4be in term 2.
I20260709 13:11:56.898739 31487 tablet_bootstrap.cc:492] T 784deee1ea9543df840c58ecaa6f911a P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:56.870447 30872 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2376eaea6bc4f209a21a110370da7b2" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:11:56.900784 31487 ts_tablet_manager.cc:1403] T 784deee1ea9543df840c58ecaa6f911a P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.367s	user 0.018s	sys 0.017s
I20260709 13:11:56.902191 30838 leader_election.cc:304] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 05213ac836744a53b9c089a5c052f4be; no voters: 
I20260709 13:11:56.899559 30874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e28f28c3f05f40e68531e20dd7c1a1a0" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:11:56.897399 31420 raft_consensus.cc:2749] T 5532e197e7de48f28c034ba1fc490e19 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:11:56.911489 31599 raft_consensus.cc:2804] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:56.912014 31599 raft_consensus.cc:493] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting leader election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.912462 31599 raft_consensus.cc:3060] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:56.917423 31527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d8c1da1f7254fde94f7deaad5f41709" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:56.932345 30872 raft_consensus.cc:2393] T e2376eaea6bc4f209a21a110370da7b2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 05213ac836744a53b9c089a5c052f4be in current term 2: Already voted for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in this term.
I20260709 13:11:56.946807 30620 catalog_manager.cc:5697] T bcac31d5b7ab48ec9c071a8f77750a88 P 02c5705b45c04706bbb2c6ff3d7b0b8a reported cstate change: term changed from 1 to 2, leader changed from d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1) to 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3). New cstate: current_term: 2 leader_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:56.929482 31591 leader_election.cc:290] T 27f26e218737414893d5c4bfa4b2cd94 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
W20260709 13:11:56.961752 30838 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:43771, user_credentials={real_user=slave}} blocked reactor thread for 59648.5us
I20260709 13:11:56.963485 30946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27f26e218737414893d5c4bfa4b2cd94" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd" is_pre_election: true
I20260709 13:11:56.964243 30946 raft_consensus.cc:2468] T 27f26e218737414893d5c4bfa4b2cd94 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 1.
I20260709 13:11:56.985495 30838 leader_election.cc:304] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 05213ac836744a53b9c089a5c052f4be; no voters: 
I20260709 13:11:56.986353 31419 leader_election.cc:290] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 election: Requested vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:56.986824 31591 raft_consensus.cc:2804] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 13:11:56.987453 31591 raft_consensus.cc:493] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Starting leader election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:56.987912 31591 raft_consensus.cc:3060] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:57.001024 31529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27f26e218737414893d5c4bfa4b2cd94" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:57.002260 31530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5028253a91e74a73afca005c77e8b565" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:57.004809 31530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ac93952b93645b1ad1d7d96cb60c3e9" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:57.011191 31596 leader_election.cc:290] T 5028253a91e74a73afca005c77e8b565 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 election: Requested vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:56.957880 31527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1474b6fc151c4a3994a3aef89ea936a2" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:57.019683 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5028253a91e74a73afca005c77e8b565" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:57.020578 30799 raft_consensus.cc:3060] T 5028253a91e74a73afca005c77e8b565 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:57.058090 31574 raft_consensus.cc:515] T 7b988f0495c34b7591ad2fb0d5f65693 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:57.063423 31487 raft_consensus.cc:359] T 784deee1ea9543df840c58ecaa6f911a P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:57.066601 31582 raft_consensus.cc:515] T e28f28c3f05f40e68531e20dd7c1a1a0 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:57.091411 31487 raft_consensus.cc:740] T 784deee1ea9543df840c58ecaa6f911a P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:57.092286 30873 raft_consensus.cc:2393] T e28f28c3f05f40e68531e20dd7c1a1a0 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 05213ac836744a53b9c089a5c052f4be in current term 2: Already voted for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in this term.
I20260709 13:11:57.092536 31487 consensus_queue.cc:260] T 784deee1ea9543df840c58ecaa6f911a P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:57.075577 31532 raft_consensus.cc:1275] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Refusing update from remote peer 02c5705b45c04706bbb2c6ff3d7b0b8a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:57.076828 31527 raft_consensus.cc:1275] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Refusing update from remote peer 05213ac836744a53b9c089a5c052f4be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:57.098301 31578 consensus_queue.cc:1048] T e2376eaea6bc4f209a21a110370da7b2 P 05213ac836744a53b9c089a5c052f4be [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:57.102444 31582 leader_election.cc:290] T e28f28c3f05f40e68531e20dd7c1a1a0 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 election: Requested vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:57.104813 30799 raft_consensus.cc:2468] T 5028253a91e74a73afca005c77e8b565 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 2.
I20260709 13:11:57.090222 31531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fb3f8d627b04c8a8971f7e4d3a5c010" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:57.067381 30623 catalog_manager.cc:5697] T e28f28c3f05f40e68531e20dd7c1a1a0 P 05213ac836744a53b9c089a5c052f4be reported cstate change: term changed from 1 to 2, leader changed from d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1) to 05213ac836744a53b9c089a5c052f4be (127.21.63.2). New cstate: current_term: 2 leader_uuid: "05213ac836744a53b9c089a5c052f4be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:57.107535 31526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b988f0495c34b7591ad2fb0d5f65693" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" is_pre_election: true
I20260709 13:11:57.120888 31574 leader_election.cc:290] T 7b988f0495c34b7591ad2fb0d5f65693 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 2 election: Requested vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:57.107486 31487 ts_tablet_manager.cc:1434] T 784deee1ea9543df840c58ecaa6f911a P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.206s	user 0.007s	sys 0.001s
I20260709 13:11:57.123092 31487 tablet_bootstrap.cc:492] T 8155b3fc02b348e1adb290ecfe4592c5 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:57.108637 30798 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e28f28c3f05f40e68531e20dd7c1a1a0" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:57.131465 30873 raft_consensus.cc:1275] T 5532e197e7de48f28c034ba1fc490e19 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Refusing update from remote peer 05213ac836744a53b9c089a5c052f4be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:57.133344 31460 consensus_queue.cc:1048] T 5532e197e7de48f28c034ba1fc490e19 P 05213ac836744a53b9c089a5c052f4be [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:57.134860 31526 raft_consensus.cc:1275] T 5532e197e7de48f28c034ba1fc490e19 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Refusing update from remote peer 05213ac836744a53b9c089a5c052f4be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:57.139444 31460 consensus_queue.cc:1048] T 5532e197e7de48f28c034ba1fc490e19 P 05213ac836744a53b9c089a5c052f4be [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:57.144737 30839 leader_election.cc:304] T 27f26e218737414893d5c4bfa4b2cd94 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
I20260709 13:11:57.146119 31606 raft_consensus.cc:2804] T 27f26e218737414893d5c4bfa4b2cd94 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:11:57.146520 31606 raft_consensus.cc:493] T 27f26e218737414893d5c4bfa4b2cd94 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting leader election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:11:57.146965 31606 raft_consensus.cc:3060] T 27f26e218737414893d5c4bfa4b2cd94 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:57.158305 30946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0ac93952b93645b1ad1d7d96cb60c3e9" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:11:57.159000 30946 raft_consensus.cc:3060] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:57.164330 30943 raft_consensus.cc:1275] T 726b808ed64848dd89da4340d4d891fb P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Refusing update from remote peer 02c5705b45c04706bbb2c6ff3d7b0b8a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:57.164986 31606 raft_consensus.cc:515] T 27f26e218737414893d5c4bfa4b2cd94 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:57.166736 30875 raft_consensus.cc:1275] T efddea050a344563877c3e01ee2a4c81 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Refusing update from remote peer 05213ac836744a53b9c089a5c052f4be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:57.168128 30946 raft_consensus.cc:2468] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 2.
I20260709 13:11:57.170766 31530 raft_consensus.cc:1275] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Refusing update from remote peer 262942f67dfa4b8fa42234b18ac31ffd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:57.172647 31591 raft_consensus.cc:515] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:57.187175 31606 leader_election.cc:290] T 27f26e218737414893d5c4bfa4b2cd94 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 election: Requested vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:11:57.084606 31599 raft_consensus.cc:515] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
W20260709 13:11:57.191640 30841 leader_election.cc:343] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:57.192516 31599 leader_election.cc:290] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 election: Requested vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:57.193321 31587 consensus_queue.cc:1048] T efddea050a344563877c3e01ee2a4c81 P 05213ac836744a53b9c089a5c052f4be [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:11:57.194732 30841 leader_election.cc:343] T 27f26e218737414893d5c4bfa4b2cd94 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:57.181164 30872 raft_consensus.cc:1275] T e2376eaea6bc4f209a21a110370da7b2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Refusing update from remote peer 05213ac836744a53b9c089a5c052f4be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:57.181772 31531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b988f0495c34b7591ad2fb0d5f65693" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:57.195940 31605 consensus_queue.cc:1048] T 726b808ed64848dd89da4340d4d891fb P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:57.196475 31587 consensus_queue.cc:1048] T e2376eaea6bc4f209a21a110370da7b2 P 05213ac836744a53b9c089a5c052f4be [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:11:57.195989 30841 leader_election.cc:343] T 5028253a91e74a73afca005c77e8b565 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:11:57.197852 30841 leader_election.cc:343] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:57.183511 30948 raft_consensus.cc:1275] T bcac31d5b7ab48ec9c071a8f77750a88 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Refusing update from remote peer 02c5705b45c04706bbb2c6ff3d7b0b8a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:11:57.202087 30841 leader_election.cc:343] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:57.205161 30839 leader_election.cc:304] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 262942f67dfa4b8fa42234b18ac31ffd; no voters: d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:57.207177 31531 raft_consensus.cc:1275] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Refusing update from remote peer 05213ac836744a53b9c089a5c052f4be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:57.209156 31578 consensus_queue.cc:1048] T efddea050a344563877c3e01ee2a4c81 P 05213ac836744a53b9c089a5c052f4be [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:57.209203 30948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27f26e218737414893d5c4bfa4b2cd94" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:11:57.210387 30948 raft_consensus.cc:3060] T 27f26e218737414893d5c4bfa4b2cd94 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:57.209524 31605 raft_consensus.cc:2804] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:57.214201 31605 raft_consensus.cc:697] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 LEADER]: Becoming Leader. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Running, Role: LEADER
I20260709 13:11:57.216276 31605 consensus_queue.cc:237] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:57.214737 31579 consensus_queue.cc:1048] T bcac31d5b7ab48ec9c071a8f77750a88 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:57.232081 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b988f0495c34b7591ad2fb0d5f65693" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:57.233843 30838 leader_election.cc:304] T 5028253a91e74a73afca005c77e8b565 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 05213ac836744a53b9c089a5c052f4be; no voters: d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:57.234803 30799 raft_consensus.cc:3060] T 7b988f0495c34b7591ad2fb0d5f65693 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:57.235805 31599 raft_consensus.cc:2804] T 5028253a91e74a73afca005c77e8b565 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:57.238597 31599 raft_consensus.cc:697] T 5028253a91e74a73afca005c77e8b565 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 LEADER]: Becoming Leader. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Running, Role: LEADER
I20260709 13:11:57.239490 31599 consensus_queue.cc:237] T 5028253a91e74a73afca005c77e8b565 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:57.245215 30875 raft_consensus.cc:1275] T 8770fcfd98154b2095c2c9f74135593b P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Refusing update from remote peer 262942f67dfa4b8fa42234b18ac31ffd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:57.245615 30948 raft_consensus.cc:2468] T 27f26e218737414893d5c4bfa4b2cd94 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 2.
I20260709 13:11:57.261162 30839 leader_election.cc:304] T 27f26e218737414893d5c4bfa4b2cd94 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
I20260709 13:11:57.257383 31579 consensus_queue.cc:1048] T 726b808ed64848dd89da4340d4d891fb P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:57.248939 31591 leader_election.cc:290] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 3 election: Requested vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:11:57.267019 31606 raft_consensus.cc:2804] T 27f26e218737414893d5c4bfa4b2cd94 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:57.268743 31606 raft_consensus.cc:697] T 27f26e218737414893d5c4bfa4b2cd94 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 LEADER]: Becoming Leader. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Running, Role: LEADER
W20260709 13:11:57.269865 30915 leader_election.cc:343] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:57.269706 31606 consensus_queue.cc:237] T 27f26e218737414893d5c4bfa4b2cd94 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
W20260709 13:11:57.271076 30915 leader_election.cc:343] T 7b988f0495c34b7591ad2fb0d5f65693 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:57.275018 30799 raft_consensus.cc:2468] T 7b988f0495c34b7591ad2fb0d5f65693 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 2.
I20260709 13:11:57.279891 31574 consensus_queue.cc:1048] T 8770fcfd98154b2095c2c9f74135593b P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:57.281410 31527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e28f28c3f05f40e68531e20dd7c1a1a0" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:57.282596 31530 raft_consensus.cc:1275] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Refusing update from remote peer 02c5705b45c04706bbb2c6ff3d7b0b8a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:57.282768 31527 raft_consensus.cc:2393] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in current term 2: Already voted for candidate 05213ac836744a53b9c089a5c052f4be in this term.
I20260709 13:11:57.284704 31527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "27f26e218737414893d5c4bfa4b2cd94" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:57.285450 31530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1474b6fc151c4a3994a3aef89ea936a2" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:57.288110 31531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d8c1da1f7254fde94f7deaad5f41709" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
I20260709 13:11:57.289422 30873 raft_consensus.cc:1275] T 01cceb88bca348958b59f3697979dff1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Refusing update from remote peer 262942f67dfa4b8fa42234b18ac31ffd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:57.309604 30795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1474b6fc151c4a3994a3aef89ea936a2" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:57.310663 30795 raft_consensus.cc:3060] T 1474b6fc151c4a3994a3aef89ea936a2 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:57.314141 31495 consensus_queue.cc:1048] T 8770fcfd98154b2095c2c9f74135593b P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:11:57.316300 30915 leader_election.cc:343] T 7b988f0495c34b7591ad2fb0d5f65693 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:57.316970 30841 leader_election.cc:304] T e28f28c3f05f40e68531e20dd7c1a1a0 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a; no voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:57.318298 31599 raft_consensus.cc:2749] T e28f28c3f05f40e68531e20dd7c1a1a0 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 13:11:57.318145 31579 consensus_queue.cc:1048] T bcac31d5b7ab48ec9c071a8f77750a88 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:11:57.320168 30841 leader_election.cc:343] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 3 election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:11:57.323657 30841 leader_election.cc:343] T 27f26e218737414893d5c4bfa4b2cd94 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:11:57.325101 30841 leader_election.cc:343] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:11:57.325807 31526 raft_consensus.cc:1275] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Refusing update from remote peer 262942f67dfa4b8fa42234b18ac31ffd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:57.332515 31589 consensus_queue.cc:1048] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:57.334824 31178 consensus_queue.cc:1048] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:57.342321 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d8c1da1f7254fde94f7deaad5f41709" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:11:57.342458 30912 leader_election.cc:304] T 7b988f0495c34b7591ad2fb0d5f65693 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:57.343441 30799 raft_consensus.cc:3060] T 0d8c1da1f7254fde94f7deaad5f41709 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:57.344702 31574 raft_consensus.cc:2804] T 7b988f0495c34b7591ad2fb0d5f65693 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:57.345690 31574 raft_consensus.cc:697] T 7b988f0495c34b7591ad2fb0d5f65693 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 LEADER]: Becoming Leader. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Running, Role: LEADER
I20260709 13:11:57.346846 31574 consensus_queue.cc:237] T 7b988f0495c34b7591ad2fb0d5f65693 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:57.351095 31530 raft_consensus.cc:1275] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Refusing update from remote peer 05213ac836744a53b9c089a5c052f4be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:57.353355 31587 consensus_queue.cc:1048] T e28f28c3f05f40e68531e20dd7c1a1a0 P 05213ac836744a53b9c089a5c052f4be [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:57.367901 30795 raft_consensus.cc:2468] T 1474b6fc151c4a3994a3aef89ea936a2 P 05213ac836744a53b9c089a5c052f4be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 3.
I20260709 13:11:57.374851 30799 raft_consensus.cc:2468] T 0d8c1da1f7254fde94f7deaad5f41709 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 2.
I20260709 13:11:57.377373 30838 leader_election.cc:304] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 05213ac836744a53b9c089a5c052f4be; no voters: d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:57.379056 31599 raft_consensus.cc:2804] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:11:57.379918 31599 raft_consensus.cc:697] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 3 LEADER]: Becoming Leader. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Running, Role: LEADER
I20260709 13:11:57.381100 31599 consensus_queue.cc:237] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:57.407940 31487 tablet_bootstrap.cc:492] T 8155b3fc02b348e1adb290ecfe4592c5 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:57.409137 31487 tablet_bootstrap.cc:492] T 8155b3fc02b348e1adb290ecfe4592c5 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:57.409824 31487 ts_tablet_manager.cc:1403] T 8155b3fc02b348e1adb290ecfe4592c5 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.287s	user 0.060s	sys 0.102s
W20260709 13:11:57.414623 30915 consensus_peers.cc:597] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:57.416916 30838 leader_election.cc:304] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 05213ac836744a53b9c089a5c052f4be; no voters: d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:11:57.412478 31487 raft_consensus.cc:359] T 8155b3fc02b348e1adb290ecfe4592c5 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:57.418843 31487 raft_consensus.cc:740] T 8155b3fc02b348e1adb290ecfe4592c5 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:57.418144 31579 raft_consensus.cc:2804] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:11:57.419631 31487 consensus_queue.cc:260] T 8155b3fc02b348e1adb290ecfe4592c5 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:57.425194 30620 catalog_manager.cc:5697] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd reported cstate change: term changed from 1 to 2, leader changed from d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1) to 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4). New cstate: current_term: 2 leader_uuid: "262942f67dfa4b8fa42234b18ac31ffd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:57.426234 30872 raft_consensus.cc:1275] T e28f28c3f05f40e68531e20dd7c1a1a0 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Refusing update from remote peer 05213ac836744a53b9c089a5c052f4be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:57.427220 31579 raft_consensus.cc:697] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 LEADER]: Becoming Leader. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Running, Role: LEADER
I20260709 13:11:57.428188 31579 consensus_queue.cc:237] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:57.436216 31578 consensus_queue.cc:1048] T e28f28c3f05f40e68531e20dd7c1a1a0 P 05213ac836744a53b9c089a5c052f4be [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:57.441995 31487 ts_tablet_manager.cc:1434] T 8155b3fc02b348e1adb290ecfe4592c5 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.001s
I20260709 13:11:57.446038 31487 tablet_bootstrap.cc:492] T 64a95fa1266848048486722c02f0d4cf P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:57.478363 30795 raft_consensus.cc:1275] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Refusing update from remote peer 262942f67dfa4b8fa42234b18ac31ffd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:57.481096 31388 consensus_queue.cc:1048] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:57.485080 31487 tablet_bootstrap.cc:492] T 64a95fa1266848048486722c02f0d4cf P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:57.486150 31487 tablet_bootstrap.cc:492] T 64a95fa1266848048486722c02f0d4cf P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:57.486815 31487 ts_tablet_manager.cc:1403] T 64a95fa1266848048486722c02f0d4cf P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.041s	user 0.024s	sys 0.006s
I20260709 13:11:57.490110 31487 raft_consensus.cc:359] T 64a95fa1266848048486722c02f0d4cf P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:57.490789 31487 raft_consensus.cc:740] T 64a95fa1266848048486722c02f0d4cf P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:57.491719 31487 consensus_queue.cc:260] T 64a95fa1266848048486722c02f0d4cf P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:57.493582 30622 catalog_manager.cc:5697] T 7b988f0495c34b7591ad2fb0d5f65693 P 262942f67dfa4b8fa42234b18ac31ffd reported cstate change: term changed from 1 to 2, leader changed from d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1) to 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4). New cstate: current_term: 2 leader_uuid: "262942f67dfa4b8fa42234b18ac31ffd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:57.494494 31487 ts_tablet_manager.cc:1434] T 64a95fa1266848048486722c02f0d4cf P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:11:57.495767 31487 tablet_bootstrap.cc:492] T 6fe93136933f4d2880ec355e1bbbf90d P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:57.517020 30620 catalog_manager.cc:5697] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a reported cstate change: term changed from 1 to 2, leader changed from d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1) to 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3). New cstate: current_term: 2 leader_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:57.524973 31487 tablet_bootstrap.cc:492] T 6fe93136933f4d2880ec355e1bbbf90d P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:57.518849 30620 catalog_manager.cc:5697] T 5028253a91e74a73afca005c77e8b565 P 02c5705b45c04706bbb2c6ff3d7b0b8a reported cstate change: term changed from 1 to 2, leader changed from d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1) to 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3). New cstate: current_term: 2 leader_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:57.525950 31487 tablet_bootstrap.cc:492] T 6fe93136933f4d2880ec355e1bbbf90d P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:57.526821 30620 catalog_manager.cc:5697] T 27f26e218737414893d5c4bfa4b2cd94 P 02c5705b45c04706bbb2c6ff3d7b0b8a reported cstate change: term changed from 1 to 2, leader changed from d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1) to 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3). New cstate: current_term: 2 leader_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } }
I20260709 13:11:57.526623 31487 ts_tablet_manager.cc:1403] T 6fe93136933f4d2880ec355e1bbbf90d P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.031s	user 0.014s	sys 0.000s
I20260709 13:11:57.530593 31487 raft_consensus.cc:359] T 6fe93136933f4d2880ec355e1bbbf90d P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:57.531368 31487 raft_consensus.cc:740] T 6fe93136933f4d2880ec355e1bbbf90d P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:57.531988 31487 consensus_queue.cc:260] T 6fe93136933f4d2880ec355e1bbbf90d P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:57.533843 31487 ts_tablet_manager.cc:1434] T 6fe93136933f4d2880ec355e1bbbf90d P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.002s
I20260709 13:11:57.534467 31487 tablet_bootstrap.cc:492] T f63e902c7c7a4d0fa5d2107620ce851e P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:57.538056 30620 catalog_manager.cc:5697] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a reported cstate change: term changed from 1 to 2, leader changed from d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1) to 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3). New cstate: current_term: 2 leader_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:57.539934 30620 catalog_manager.cc:5697] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a reported cstate change: term changed from 2 to 3, leader changed from d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1) to 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3). New cstate: current_term: 3 leader_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: HEALTHY } } }
I20260709 13:11:57.558446 31487 tablet_bootstrap.cc:492] T f63e902c7c7a4d0fa5d2107620ce851e P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:57.559512 31487 tablet_bootstrap.cc:492] T f63e902c7c7a4d0fa5d2107620ce851e P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:57.560204 31487 ts_tablet_manager.cc:1403] T f63e902c7c7a4d0fa5d2107620ce851e P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.026s	user 0.013s	sys 0.008s
I20260709 13:11:57.563015 31487 raft_consensus.cc:359] T f63e902c7c7a4d0fa5d2107620ce851e P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:57.564242 31487 raft_consensus.cc:740] T f63e902c7c7a4d0fa5d2107620ce851e P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:57.564993 31487 consensus_queue.cc:260] T f63e902c7c7a4d0fa5d2107620ce851e P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
W20260709 13:11:57.565950 30915 consensus_peers.cc:597] T 39980eee51514ba493958931f406a145 P 262942f67dfa4b8fa42234b18ac31ffd -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:57.567003 31487 ts_tablet_manager.cc:1434] T f63e902c7c7a4d0fa5d2107620ce851e P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:11:57.567818 31487 tablet_bootstrap.cc:492] T cae2b1cb8db54cbb812489cc1b4b1213 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
W20260709 13:11:57.588977 30767 consensus_peers.cc:597] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 05213ac836744a53b9c089a5c052f4be -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:57.596302 31487 tablet_bootstrap.cc:492] T cae2b1cb8db54cbb812489cc1b4b1213 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:57.597218 31487 tablet_bootstrap.cc:492] T cae2b1cb8db54cbb812489cc1b4b1213 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:57.597784 31487 ts_tablet_manager.cc:1403] T cae2b1cb8db54cbb812489cc1b4b1213 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.030s	user 0.017s	sys 0.011s
I20260709 13:11:57.599771 31487 raft_consensus.cc:359] T cae2b1cb8db54cbb812489cc1b4b1213 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:57.600344 31487 raft_consensus.cc:740] T cae2b1cb8db54cbb812489cc1b4b1213 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:57.601110 31487 consensus_queue.cc:260] T cae2b1cb8db54cbb812489cc1b4b1213 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:57.602843 31487 ts_tablet_manager.cc:1434] T cae2b1cb8db54cbb812489cc1b4b1213 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:11:57.603585 31487 tablet_bootstrap.cc:492] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
W20260709 13:11:57.612057 30841 consensus_peers.cc:597] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:57.619992 31487 tablet_bootstrap.cc:492] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:57.621126 31487 tablet_bootstrap.cc:492] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:57.621798 31487 ts_tablet_manager.cc:1403] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.018s	user 0.016s	sys 0.000s
I20260709 13:11:57.624022 31487 raft_consensus.cc:359] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:57.624598 31487 raft_consensus.cc:740] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:57.625236 31487 consensus_queue.cc:260] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:57.626850 31487 ts_tablet_manager.cc:1434] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.005s	user 0.003s	sys 0.000s
I20260709 13:11:57.627972 31487 tablet_bootstrap.cc:492] T 7b988f0495c34b7591ad2fb0d5f65693 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:57.645213 31487 tablet_bootstrap.cc:492] T 7b988f0495c34b7591ad2fb0d5f65693 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:57.646193 31487 tablet_bootstrap.cc:492] T 7b988f0495c34b7591ad2fb0d5f65693 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:57.646833 31487 ts_tablet_manager.cc:1403] T 7b988f0495c34b7591ad2fb0d5f65693 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.019s	user 0.016s	sys 0.000s
I20260709 13:11:57.648017 30946 raft_consensus.cc:1275] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Refusing update from remote peer 02c5705b45c04706bbb2c6ff3d7b0b8a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:57.649065 31487 raft_consensus.cc:359] T 7b988f0495c34b7591ad2fb0d5f65693 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:57.649739 31487 raft_consensus.cc:740] T 7b988f0495c34b7591ad2fb0d5f65693 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:57.650410 31487 consensus_queue.cc:260] T 7b988f0495c34b7591ad2fb0d5f65693 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:57.653026 30943 raft_consensus.cc:1275] T 27f26e218737414893d5c4bfa4b2cd94 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Refusing update from remote peer 02c5705b45c04706bbb2c6ff3d7b0b8a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:57.653824 31487 ts_tablet_manager.cc:1434] T 7b988f0495c34b7591ad2fb0d5f65693 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:11:57.654665 31487 tablet_bootstrap.cc:492] T d05da516ed9040c988b1fdfe2fdde9ed P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:57.655846 31606 consensus_queue.cc:1048] T 27f26e218737414893d5c4bfa4b2cd94 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:57.657276 31606 consensus_queue.cc:1048] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:11:57.662519 30915 consensus_peers.cc:597] T b3754a670e0b4ed99d37283bf1647bf5 P 262942f67dfa4b8fa42234b18ac31ffd -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:57.676476 31487 tablet_bootstrap.cc:492] T d05da516ed9040c988b1fdfe2fdde9ed P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:57.677593 31487 tablet_bootstrap.cc:492] T d05da516ed9040c988b1fdfe2fdde9ed P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:57.678283 31487 ts_tablet_manager.cc:1403] T d05da516ed9040c988b1fdfe2fdde9ed P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.024s	user 0.022s	sys 0.000s
W20260709 13:11:57.679126 30841 consensus_peers.cc:597] T 27f26e218737414893d5c4bfa4b2cd94 P 02c5705b45c04706bbb2c6ff3d7b0b8a -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:11:57.679818 30841 consensus_peers.cc:597] T d05da516ed9040c988b1fdfe2fdde9ed P 02c5705b45c04706bbb2c6ff3d7b0b8a -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:57.680902 31487 raft_consensus.cc:359] T d05da516ed9040c988b1fdfe2fdde9ed P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:57.681632 31487 raft_consensus.cc:740] T d05da516ed9040c988b1fdfe2fdde9ed P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:57.682301 31487 consensus_queue.cc:260] T d05da516ed9040c988b1fdfe2fdde9ed P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:57.684682 31487 ts_tablet_manager.cc:1434] T d05da516ed9040c988b1fdfe2fdde9ed P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:11:57.685410 31487 tablet_bootstrap.cc:492] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:57.687947 30795 raft_consensus.cc:1275] T 5028253a91e74a73afca005c77e8b565 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Refusing update from remote peer 02c5705b45c04706bbb2c6ff3d7b0b8a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:57.690253 31603 consensus_queue.cc:1048] T 5028253a91e74a73afca005c77e8b565 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:11:57.708408 30767 consensus_peers.cc:597] T 8b6052cddae64c5c9d16f60272735788 P 05213ac836744a53b9c089a5c052f4be -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:57.720710 31487 tablet_bootstrap.cc:492] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:57.721782 31487 tablet_bootstrap.cc:492] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:57.722862 31487 ts_tablet_manager.cc:1403] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.038s	user 0.015s	sys 0.021s
I20260709 13:11:57.725821 31487 raft_consensus.cc:359] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:57.726512 31487 raft_consensus.cc:740] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:57.727174 31487 consensus_queue.cc:260] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:57.729277 31487 ts_tablet_manager.cc:1434] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.001s
I20260709 13:11:57.730095 31487 tablet_bootstrap.cc:492] T bc2a86f27b0f4fb59d7cfa586a23fef1 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:57.730597 31531 raft_consensus.cc:3060] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:57.748121 31531 raft_consensus.cc:1275] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Refusing update from remote peer 02c5705b45c04706bbb2c6ff3d7b0b8a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 13:11:57.748626 30915 consensus_peers.cc:597] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:57.753711 31605 consensus_queue.cc:1048] T 5028253a91e74a73afca005c77e8b565 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:57.756392 31487 tablet_bootstrap.cc:492] T bc2a86f27b0f4fb59d7cfa586a23fef1 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:57.757387 31487 tablet_bootstrap.cc:492] T bc2a86f27b0f4fb59d7cfa586a23fef1 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:57.758042 31487 ts_tablet_manager.cc:1403] T bc2a86f27b0f4fb59d7cfa586a23fef1 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.028s	user 0.012s	sys 0.009s
I20260709 13:11:57.761749 31487 raft_consensus.cc:359] T bc2a86f27b0f4fb59d7cfa586a23fef1 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:57.762415 31487 raft_consensus.cc:740] T bc2a86f27b0f4fb59d7cfa586a23fef1 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:57.763058 31487 consensus_queue.cc:260] T bc2a86f27b0f4fb59d7cfa586a23fef1 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:57.765162 31487 ts_tablet_manager.cc:1434] T bc2a86f27b0f4fb59d7cfa586a23fef1 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:11:57.766588 31487 tablet_bootstrap.cc:492] T 892368b6886842febffc849d73f76c10 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:57.783097 30799 raft_consensus.cc:1275] T 7b988f0495c34b7591ad2fb0d5f65693 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Refusing update from remote peer 262942f67dfa4b8fa42234b18ac31ffd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:57.784519 31495 consensus_queue.cc:1048] T 7b988f0495c34b7591ad2fb0d5f65693 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:57.784801 31487 tablet_bootstrap.cc:492] T 892368b6886842febffc849d73f76c10 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:57.785811 31487 tablet_bootstrap.cc:492] T 892368b6886842febffc849d73f76c10 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:57.786422 31487 ts_tablet_manager.cc:1403] T 892368b6886842febffc849d73f76c10 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.020s	user 0.015s	sys 0.002s
I20260709 13:11:57.788872 31487 raft_consensus.cc:359] T 892368b6886842febffc849d73f76c10 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:57.789525 31487 raft_consensus.cc:740] T 892368b6886842febffc849d73f76c10 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:57.790055 31487 consensus_queue.cc:260] T 892368b6886842febffc849d73f76c10 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:57.792341 31487 ts_tablet_manager.cc:1434] T 892368b6886842febffc849d73f76c10 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:11:57.793550 31487 tablet_bootstrap.cc:492] T 39980eee51514ba493958931f406a145 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:57.799918 31532 raft_consensus.cc:3060] T 7b988f0495c34b7591ad2fb0d5f65693 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:57.807881 31532 raft_consensus.cc:1275] T 7b988f0495c34b7591ad2fb0d5f65693 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Refusing update from remote peer 262942f67dfa4b8fa42234b18ac31ffd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:57.809494 31575 consensus_queue.cc:1048] T 7b988f0495c34b7591ad2fb0d5f65693 P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:57.831274 31487 tablet_bootstrap.cc:492] T 39980eee51514ba493958931f406a145 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:57.832247 31487 tablet_bootstrap.cc:492] T 39980eee51514ba493958931f406a145 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:57.834542 31487 ts_tablet_manager.cc:1403] T 39980eee51514ba493958931f406a145 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.041s	user 0.015s	sys 0.009s
I20260709 13:11:57.838181 31532 raft_consensus.cc:3060] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:11:57.838195 31487 raft_consensus.cc:359] T 39980eee51514ba493958931f406a145 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:57.839067 31487 raft_consensus.cc:740] T 39980eee51514ba493958931f406a145 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:57.839766 31487 consensus_queue.cc:260] T 39980eee51514ba493958931f406a145 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:57.842008 31487 ts_tablet_manager.cc:1434] T 39980eee51514ba493958931f406a145 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 13:11:57.843556 31487 tablet_bootstrap.cc:492] T 7ed3eac558b34457aa26b1c40652f887 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:57.850711 31532 raft_consensus.cc:1275] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [term 3 FOLLOWER]: Refusing update from remote peer 02c5705b45c04706bbb2c6ff3d7b0b8a: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:57.854717 31582 consensus_queue.cc:1048] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:11:57.870276 30841 consensus_peers.cc:597] T 94220be3f55a4acbbd6bd5723b9f32e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:57.875139 31487 tablet_bootstrap.cc:492] T 7ed3eac558b34457aa26b1c40652f887 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:57.876137 31487 tablet_bootstrap.cc:492] T 7ed3eac558b34457aa26b1c40652f887 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:57.876828 31487 ts_tablet_manager.cc:1403] T 7ed3eac558b34457aa26b1c40652f887 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.034s	user 0.021s	sys 0.008s
I20260709 13:11:57.878434 30799 raft_consensus.cc:1275] T 1474b6fc151c4a3994a3aef89ea936a2 P 05213ac836744a53b9c089a5c052f4be [term 3 FOLLOWER]: Refusing update from remote peer 02c5705b45c04706bbb2c6ff3d7b0b8a: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:11:57.879573 31487 raft_consensus.cc:359] T 7ed3eac558b34457aa26b1c40652f887 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:57.880301 31487 raft_consensus.cc:740] T 7ed3eac558b34457aa26b1c40652f887 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:57.881023 31487 consensus_queue.cc:260] T 7ed3eac558b34457aa26b1c40652f887 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:57.881721 31582 consensus_queue.cc:1048] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:11:57.883533 31487 ts_tablet_manager.cc:1434] T 7ed3eac558b34457aa26b1c40652f887 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 13:11:57.892206 31487 tablet_bootstrap.cc:492] T 8b6052cddae64c5c9d16f60272735788 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
W20260709 13:11:57.908962 30915 consensus_peers.cc:597] T 8fb24a20af104a7486d0f1b2ae0e3843 P 262942f67dfa4b8fa42234b18ac31ffd -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:57.967691 30841 consensus_peers.cc:597] T d5175e0d898c453b8a7641d94dc1ac58 P 02c5705b45c04706bbb2c6ff3d7b0b8a -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
W20260709 13:11:57.969177 30767 consensus_peers.cc:597] T bbb2acd63baa48a3a2ace94016522e92 P 05213ac836744a53b9c089a5c052f4be -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:57.980794 31487 tablet_bootstrap.cc:492] T 8b6052cddae64c5c9d16f60272735788 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:57.981745 31487 tablet_bootstrap.cc:492] T 8b6052cddae64c5c9d16f60272735788 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:57.982399 31487 ts_tablet_manager.cc:1403] T 8b6052cddae64c5c9d16f60272735788 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.090s	user 0.040s	sys 0.024s
W20260709 13:11:57.984987 30841 consensus_peers.cc:597] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:11:57.984838 31487 raft_consensus.cc:359] T 8b6052cddae64c5c9d16f60272735788 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:57.985492 31487 raft_consensus.cc:740] T 8b6052cddae64c5c9d16f60272735788 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:57.986233 31487 consensus_queue.cc:260] T 8b6052cddae64c5c9d16f60272735788 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:57.988720 31487 ts_tablet_manager.cc:1434] T 8b6052cddae64c5c9d16f60272735788 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.004s
I20260709 13:11:57.989487 31487 tablet_bootstrap.cc:492] T bbb2acd63baa48a3a2ace94016522e92 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:58.003109 31487 tablet_bootstrap.cc:492] T bbb2acd63baa48a3a2ace94016522e92 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:58.004076 31487 tablet_bootstrap.cc:492] T bbb2acd63baa48a3a2ace94016522e92 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:58.004748 31487 ts_tablet_manager.cc:1403] T bbb2acd63baa48a3a2ace94016522e92 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.015s	user 0.015s	sys 0.000s
I20260709 13:11:58.007100 31487 raft_consensus.cc:359] T bbb2acd63baa48a3a2ace94016522e92 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:58.007752 31487 raft_consensus.cc:740] T bbb2acd63baa48a3a2ace94016522e92 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:58.008351 31487 consensus_queue.cc:260] T bbb2acd63baa48a3a2ace94016522e92 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:58.010382 31487 ts_tablet_manager.cc:1434] T bbb2acd63baa48a3a2ace94016522e92 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:11:58.011840 31487 tablet_bootstrap.cc:492] T 8b3e41a728f1457ebf97274b95c7b23a P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:58.032716 31487 tablet_bootstrap.cc:492] T 8b3e41a728f1457ebf97274b95c7b23a P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
W20260709 13:11:58.032878 30915 consensus_peers.cc:597] T d16c44f56c42466497ba4f7f1b2b5344 P 262942f67dfa4b8fa42234b18ac31ffd -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 13:11:58.033632 31487 tablet_bootstrap.cc:492] T 8b3e41a728f1457ebf97274b95c7b23a P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:58.034353 31487 ts_tablet_manager.cc:1403] T 8b3e41a728f1457ebf97274b95c7b23a P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.004s
I20260709 13:11:58.035652 30799 raft_consensus.cc:1275] T 0d8c1da1f7254fde94f7deaad5f41709 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Refusing update from remote peer 02c5705b45c04706bbb2c6ff3d7b0b8a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:11:58.036733 31487 raft_consensus.cc:359] T 8b3e41a728f1457ebf97274b95c7b23a P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:58.037418 31487 raft_consensus.cc:740] T 8b3e41a728f1457ebf97274b95c7b23a P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:58.037402 31582 consensus_queue.cc:1048] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:11:58.038331 31487 consensus_queue.cc:260] T 8b3e41a728f1457ebf97274b95c7b23a P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:58.040729 31487 ts_tablet_manager.cc:1434] T 8b3e41a728f1457ebf97274b95c7b23a P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 13:11:58.041535 31487 tablet_bootstrap.cc:492] T 8fb24a20af104a7486d0f1b2ae0e3843 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:58.060458 31487 tablet_bootstrap.cc:492] T 8fb24a20af104a7486d0f1b2ae0e3843 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:58.061457 31487 tablet_bootstrap.cc:492] T 8fb24a20af104a7486d0f1b2ae0e3843 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:58.062325 31487 ts_tablet_manager.cc:1403] T 8fb24a20af104a7486d0f1b2ae0e3843 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.004s
I20260709 13:11:58.065477 31487 raft_consensus.cc:359] T 8fb24a20af104a7486d0f1b2ae0e3843 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:58.066363 31487 raft_consensus.cc:740] T 8fb24a20af104a7486d0f1b2ae0e3843 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:58.067256 31487 consensus_queue.cc:260] T 8fb24a20af104a7486d0f1b2ae0e3843 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:58.070119 31487 ts_tablet_manager.cc:1434] T 8fb24a20af104a7486d0f1b2ae0e3843 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:11:58.071837 31487 tablet_bootstrap.cc:492] T d16c44f56c42466497ba4f7f1b2b5344 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:58.124310 31487 tablet_bootstrap.cc:492] T d16c44f56c42466497ba4f7f1b2b5344 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:58.125597 31487 tablet_bootstrap.cc:492] T d16c44f56c42466497ba4f7f1b2b5344 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:58.126605 31487 ts_tablet_manager.cc:1403] T d16c44f56c42466497ba4f7f1b2b5344 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.055s	user 0.032s	sys 0.020s
I20260709 13:11:58.138480 31487 raft_consensus.cc:359] T d16c44f56c42466497ba4f7f1b2b5344 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:58.139009 31487 raft_consensus.cc:740] T d16c44f56c42466497ba4f7f1b2b5344 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:58.139559 31487 consensus_queue.cc:260] T d16c44f56c42466497ba4f7f1b2b5344 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:58.141610 31487 ts_tablet_manager.cc:1434] T d16c44f56c42466497ba4f7f1b2b5344 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.015s	user 0.004s	sys 0.000s
I20260709 13:11:58.142522 31487 tablet_bootstrap.cc:492] T a38ff14550f74527a62591df59fb2c51 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:58.158586 31487 tablet_bootstrap.cc:492] T a38ff14550f74527a62591df59fb2c51 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:58.159333 31487 tablet_bootstrap.cc:492] T a38ff14550f74527a62591df59fb2c51 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:58.159865 31487 ts_tablet_manager.cc:1403] T a38ff14550f74527a62591df59fb2c51 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.018s	user 0.010s	sys 0.000s
I20260709 13:11:58.161739 31487 raft_consensus.cc:359] T a38ff14550f74527a62591df59fb2c51 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:58.162223 31487 raft_consensus.cc:740] T a38ff14550f74527a62591df59fb2c51 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:58.162842 31487 consensus_queue.cc:260] T a38ff14550f74527a62591df59fb2c51 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:58.164371 31487 ts_tablet_manager.cc:1434] T a38ff14550f74527a62591df59fb2c51 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:11:58.165691 31487 tablet_bootstrap.cc:492] T d5175e0d898c453b8a7641d94dc1ac58 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:58.180222 31487 tablet_bootstrap.cc:492] T d5175e0d898c453b8a7641d94dc1ac58 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:58.181092 31487 tablet_bootstrap.cc:492] T d5175e0d898c453b8a7641d94dc1ac58 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:58.181640 31487 ts_tablet_manager.cc:1403] T d5175e0d898c453b8a7641d94dc1ac58 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.016s	user 0.013s	sys 0.001s
I20260709 13:11:58.183485 31487 raft_consensus.cc:359] T d5175e0d898c453b8a7641d94dc1ac58 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:58.183991 31487 raft_consensus.cc:740] T d5175e0d898c453b8a7641d94dc1ac58 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:58.184516 31487 consensus_queue.cc:260] T d5175e0d898c453b8a7641d94dc1ac58 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:58.186314 31487 ts_tablet_manager.cc:1434] T d5175e0d898c453b8a7641d94dc1ac58 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:11:58.187461 31487 tablet_bootstrap.cc:492] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:58.201085 31487 tablet_bootstrap.cc:492] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:58.201979 31487 tablet_bootstrap.cc:492] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:58.202658 31487 ts_tablet_manager.cc:1403] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.015s	user 0.013s	sys 0.000s
I20260709 13:11:58.204819 31487 raft_consensus.cc:359] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:58.205358 31487 raft_consensus.cc:740] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:58.205968 31487 consensus_queue.cc:260] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:11:58.208802 31487 ts_tablet_manager.cc:1434] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:11:58.210050 31487 tablet_bootstrap.cc:492] T b3754a670e0b4ed99d37283bf1647bf5 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:58.231014 31487 tablet_bootstrap.cc:492] T b3754a670e0b4ed99d37283bf1647bf5 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:58.231936 31487 tablet_bootstrap.cc:492] T b3754a670e0b4ed99d37283bf1647bf5 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:58.232623 31487 ts_tablet_manager.cc:1403] T b3754a670e0b4ed99d37283bf1647bf5 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.023s	user 0.010s	sys 0.007s
I20260709 13:11:58.234906 31487 raft_consensus.cc:359] T b3754a670e0b4ed99d37283bf1647bf5 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:58.235581 31487 raft_consensus.cc:740] T b3754a670e0b4ed99d37283bf1647bf5 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:58.236315 31487 consensus_queue.cc:260] T b3754a670e0b4ed99d37283bf1647bf5 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:58.237809 31487 ts_tablet_manager.cc:1434] T b3754a670e0b4ed99d37283bf1647bf5 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:11:58.239041 31487 tablet_bootstrap.cc:492] T 27f26e218737414893d5c4bfa4b2cd94 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:58.253046 31487 tablet_bootstrap.cc:492] T 27f26e218737414893d5c4bfa4b2cd94 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:58.253957 31487 tablet_bootstrap.cc:492] T 27f26e218737414893d5c4bfa4b2cd94 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:58.254619 31487 ts_tablet_manager.cc:1403] T 27f26e218737414893d5c4bfa4b2cd94 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.016s	user 0.006s	sys 0.007s
I20260709 13:11:58.256912 31487 raft_consensus.cc:359] T 27f26e218737414893d5c4bfa4b2cd94 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:58.257401 31487 raft_consensus.cc:740] T 27f26e218737414893d5c4bfa4b2cd94 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:58.258001 31487 consensus_queue.cc:260] T 27f26e218737414893d5c4bfa4b2cd94 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:58.259914 31487 ts_tablet_manager.cc:1434] T 27f26e218737414893d5c4bfa4b2cd94 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:11:58.261772 31487 tablet_bootstrap.cc:492] T 0ac93952b93645b1ad1d7d96cb60c3e9 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:58.280401 31487 tablet_bootstrap.cc:492] T 0ac93952b93645b1ad1d7d96cb60c3e9 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:58.281379 31487 tablet_bootstrap.cc:492] T 0ac93952b93645b1ad1d7d96cb60c3e9 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:58.282275 31487 ts_tablet_manager.cc:1403] T 0ac93952b93645b1ad1d7d96cb60c3e9 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.021s	user 0.017s	sys 0.001s
I20260709 13:11:58.284176 31487 raft_consensus.cc:359] T 0ac93952b93645b1ad1d7d96cb60c3e9 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:58.284700 31487 raft_consensus.cc:740] T 0ac93952b93645b1ad1d7d96cb60c3e9 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:58.285224 31487 consensus_queue.cc:260] T 0ac93952b93645b1ad1d7d96cb60c3e9 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:58.286602 31487 ts_tablet_manager.cc:1434] T 0ac93952b93645b1ad1d7d96cb60c3e9 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.004s	user 0.003s	sys 0.000s
I20260709 13:11:58.289048 31487 tablet_bootstrap.cc:492] T 94220be3f55a4acbbd6bd5723b9f32e9 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:58.306730 31487 tablet_bootstrap.cc:492] T 94220be3f55a4acbbd6bd5723b9f32e9 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:58.307698 31487 tablet_bootstrap.cc:492] T 94220be3f55a4acbbd6bd5723b9f32e9 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:58.308359 31487 ts_tablet_manager.cc:1403] T 94220be3f55a4acbbd6bd5723b9f32e9 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.020s	user 0.016s	sys 0.001s
I20260709 13:11:58.310874 31487 raft_consensus.cc:359] T 94220be3f55a4acbbd6bd5723b9f32e9 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:58.311537 31487 raft_consensus.cc:740] T 94220be3f55a4acbbd6bd5723b9f32e9 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:58.312175 31487 consensus_queue.cc:260] T 94220be3f55a4acbbd6bd5723b9f32e9 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:11:58.313712 31487 ts_tablet_manager.cc:1434] T 94220be3f55a4acbbd6bd5723b9f32e9 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:11:58.314778 31487 tablet_bootstrap.cc:492] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap starting.
I20260709 13:11:58.329160 31487 tablet_bootstrap.cc:492] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:11:58.331728 31487 tablet_bootstrap.cc:492] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd: Bootstrap complete.
I20260709 13:11:58.332376 31487 ts_tablet_manager.cc:1403] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent bootstrapping tablet: real 0.018s	user 0.012s	sys 0.000s
I20260709 13:11:58.334337 31487 raft_consensus.cc:359] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:58.334849 31487 raft_consensus.cc:740] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Initialized, Role: FOLLOWER
I20260709 13:11:58.335467 31487 consensus_queue.cc:260] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:11:58.337713 31487 ts_tablet_manager.cc:1434] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:11:58.447926 31527 raft_consensus.cc:3060] T 0ac93952b93645b1ad1d7d96cb60c3e9 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:58.462659 31527 raft_consensus.cc:3060] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:58.517740 31527 raft_consensus.cc:3060] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:11:58.607884 31530 raft_consensus.cc:3060] T 27f26e218737414893d5c4bfa4b2cd94 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:06.279074 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:06.341881 30873 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "726b808ed64848dd89da4340d4d891fb"
dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
mode: GRACEFUL
new_leader_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
 from {username='slave'} at 127.0.0.1:46350
I20260709 13:12:06.342586 30873 raft_consensus.cc:606] T 726b808ed64848dd89da4340d4d891fb P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 LEADER]: Received request to transfer leadership to TS d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:12:06.345405 30873 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8155b3fc02b348e1adb290ecfe4592c5"
dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
mode: GRACEFUL
new_leader_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
 from {username='slave'} at 127.0.0.1:46350
I20260709 13:12:06.345971 30873 raft_consensus.cc:606] T 8155b3fc02b348e1adb290ecfe4592c5 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Received request to transfer leadership to TS d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:12:06.348461 30873 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b226c44412c74106a0feb2ca411b1661"
dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
mode: GRACEFUL
new_leader_uuid: "05213ac836744a53b9c089a5c052f4be"
 from {username='slave'} at 127.0.0.1:46350
I20260709 13:12:06.349033 30873 raft_consensus.cc:606] T b226c44412c74106a0feb2ca411b1661 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Received request to transfer leadership to TS 05213ac836744a53b9c089a5c052f4be
I20260709 13:12:06.362159 30943 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c1efa319af774b23bd4ea782611f5252"
dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
mode: GRACEFUL
new_leader_uuid: "05213ac836744a53b9c089a5c052f4be"
 from {username='slave'} at 127.0.0.1:33910
I20260709 13:12:06.362700 30943 raft_consensus.cc:606] T c1efa319af774b23bd4ea782611f5252 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Received request to transfer leadership to TS 05213ac836744a53b9c089a5c052f4be
I20260709 13:12:06.365478 30873 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d05da516ed9040c988b1fdfe2fdde9ed"
dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
mode: GRACEFUL
new_leader_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
 from {username='slave'} at 127.0.0.1:46350
I20260709 13:12:06.366036 30873 raft_consensus.cc:606] T d05da516ed9040c988b1fdfe2fdde9ed P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Received request to transfer leadership to TS d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:12:06.368626 30873 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ea563bd67a9849d18c9c3dfb21d58a93"
dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
mode: GRACEFUL
new_leader_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
 from {username='slave'} at 127.0.0.1:46350
I20260709 13:12:06.369268 30873 raft_consensus.cc:606] T ea563bd67a9849d18c9c3dfb21d58a93 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Received request to transfer leadership to TS d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:12:06.370406 21756 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 13:12:06.371796 21756 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:12:06.396890 31696 raft_consensus.cc:993] T ea563bd67a9849d18c9c3dfb21d58a93 P 02c5705b45c04706bbb2c6ff3d7b0b8a: : Instructing follower d22c9d3d15904327a78a0e548a8ff6cd to start an election
I20260709 13:12:06.397560 31599 raft_consensus.cc:1081] T ea563bd67a9849d18c9c3dfb21d58a93 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Signalling peer d22c9d3d15904327a78a0e548a8ff6cd to start an election
I20260709 13:12:06.398914 31532 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ea563bd67a9849d18c9c3dfb21d58a93"
dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
 from {username='slave'} at 127.0.0.1:34014
I20260709 13:12:06.399380 31532 raft_consensus.cc:493] T ea563bd67a9849d18c9c3dfb21d58a93 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:06.399688 31532 raft_consensus.cc:3060] T ea563bd67a9849d18c9c3dfb21d58a93 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:06.405131 31532 raft_consensus.cc:515] T ea563bd67a9849d18c9c3dfb21d58a93 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:12:06.409205 31532 leader_election.cc:290] T ea563bd67a9849d18c9c3dfb21d58a93 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 2 election: Requested vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:12:06.419592 30873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea563bd67a9849d18c9c3dfb21d58a93" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:12:06.420399 30873 raft_consensus.cc:3055] T ea563bd67a9849d18c9c3dfb21d58a93 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:06.420725 30873 raft_consensus.cc:740] T ea563bd67a9849d18c9c3dfb21d58a93 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Running, Role: LEADER
I20260709 13:12:06.421025 30943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea563bd67a9849d18c9c3dfb21d58a93" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:12:06.421613 30943 raft_consensus.cc:3060] T ea563bd67a9849d18c9c3dfb21d58a93 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:06.421559 30873 consensus_queue.cc:260] T ea563bd67a9849d18c9c3dfb21d58a93 P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:12:06.423318 30873 raft_consensus.cc:3060] T ea563bd67a9849d18c9c3dfb21d58a93 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:06.428442 30943 raft_consensus.cc:2468] T ea563bd67a9849d18c9c3dfb21d58a93 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 2.
I20260709 13:12:06.430018 31482 leader_election.cc:304] T ea563bd67a9849d18c9c3dfb21d58a93 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:12:06.430737 30873 raft_consensus.cc:2468] T ea563bd67a9849d18c9c3dfb21d58a93 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 2.
I20260709 13:12:06.431947 31705 raft_consensus.cc:2804] T ea563bd67a9849d18c9c3dfb21d58a93 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:06.435575 31705 raft_consensus.cc:697] T ea563bd67a9849d18c9c3dfb21d58a93 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 LEADER]: Becoming Leader. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Running, Role: LEADER
I20260709 13:12:06.437098 31705 consensus_queue.cc:237] T ea563bd67a9849d18c9c3dfb21d58a93 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:12:06.438402 31696 raft_consensus.cc:993] T d05da516ed9040c988b1fdfe2fdde9ed P 02c5705b45c04706bbb2c6ff3d7b0b8a: : Instructing follower d22c9d3d15904327a78a0e548a8ff6cd to start an election
I20260709 13:12:06.438882 31696 raft_consensus.cc:1081] T d05da516ed9040c988b1fdfe2fdde9ed P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Signalling peer d22c9d3d15904327a78a0e548a8ff6cd to start an election
I20260709 13:12:06.440404 31532 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d05da516ed9040c988b1fdfe2fdde9ed"
dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
 from {username='slave'} at 127.0.0.1:34014
I20260709 13:12:06.441056 31532 raft_consensus.cc:493] T d05da516ed9040c988b1fdfe2fdde9ed P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:06.446053 31532 raft_consensus.cc:3060] T d05da516ed9040c988b1fdfe2fdde9ed P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:06.451257 30624 catalog_manager.cc:5697] T ea563bd67a9849d18c9c3dfb21d58a93 P d22c9d3d15904327a78a0e548a8ff6cd reported cstate change: term changed from 1 to 2, leader changed from 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3) to d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1). New cstate: current_term: 2 leader_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:06.452775 31532 raft_consensus.cc:515] T d05da516ed9040c988b1fdfe2fdde9ed P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:12:06.457587 31532 leader_election.cc:290] T d05da516ed9040c988b1fdfe2fdde9ed P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 2 election: Requested vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:12:06.459918 30873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d05da516ed9040c988b1fdfe2fdde9ed" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:12:06.460516 30873 raft_consensus.cc:3055] T d05da516ed9040c988b1fdfe2fdde9ed P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:06.460821 30873 raft_consensus.cc:740] T d05da516ed9040c988b1fdfe2fdde9ed P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Running, Role: LEADER
I20260709 13:12:06.463393 30873 consensus_queue.cc:260] T d05da516ed9040c988b1fdfe2fdde9ed P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:12:06.464845 30873 raft_consensus.cc:3060] T d05da516ed9040c988b1fdfe2fdde9ed P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:06.465358 31682 raft_consensus.cc:993] T c1efa319af774b23bd4ea782611f5252 P 262942f67dfa4b8fa42234b18ac31ffd: : Instructing follower 05213ac836744a53b9c089a5c052f4be to start an election
I20260709 13:12:06.465891 31682 raft_consensus.cc:1081] T c1efa319af774b23bd4ea782611f5252 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Signalling peer 05213ac836744a53b9c089a5c052f4be to start an election
I20260709 13:12:06.469451 30799 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c1efa319af774b23bd4ea782611f5252"
dest_uuid: "05213ac836744a53b9c089a5c052f4be"
 from {username='slave'} at 127.0.0.1:39606
I20260709 13:12:06.469977 30799 raft_consensus.cc:493] T c1efa319af774b23bd4ea782611f5252 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:06.470301 30799 raft_consensus.cc:3060] T c1efa319af774b23bd4ea782611f5252 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:06.477541 30799 raft_consensus.cc:515] T c1efa319af774b23bd4ea782611f5252 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:12:06.478199 30795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d05da516ed9040c988b1fdfe2fdde9ed" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:12:06.478911 30795 raft_consensus.cc:3060] T d05da516ed9040c988b1fdfe2fdde9ed P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:06.480716 30875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1efa319af774b23bd4ea782611f5252" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:12:06.481272 30875 raft_consensus.cc:3060] T c1efa319af774b23bd4ea782611f5252 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:06.485783 30799 leader_election.cc:290] T c1efa319af774b23bd4ea782611f5252 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 2 election: Requested vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:12:06.486320 30795 raft_consensus.cc:2468] T d05da516ed9040c988b1fdfe2fdde9ed P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 2.
I20260709 13:12:06.488382 30873 raft_consensus.cc:2468] T d05da516ed9040c988b1fdfe2fdde9ed P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 2.
I20260709 13:12:06.489583 31483 leader_election.cc:304] T d05da516ed9040c988b1fdfe2fdde9ed P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:12:06.490069 30943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c1efa319af774b23bd4ea782611f5252" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:12:06.490638 30943 raft_consensus.cc:3055] T c1efa319af774b23bd4ea782611f5252 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:06.490929 30943 raft_consensus.cc:740] T c1efa319af774b23bd4ea782611f5252 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Running, Role: LEADER
I20260709 13:12:06.491662 31705 raft_consensus.cc:2804] T d05da516ed9040c988b1fdfe2fdde9ed P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:06.491832 30943 consensus_queue.cc:260] T c1efa319af774b23bd4ea782611f5252 P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:12:06.492492 31705 raft_consensus.cc:697] T d05da516ed9040c988b1fdfe2fdde9ed P d22c9d3d15904327a78a0e548a8ff6cd [term 2 LEADER]: Becoming Leader. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Running, Role: LEADER
I20260709 13:12:06.493160 30943 raft_consensus.cc:3060] T c1efa319af774b23bd4ea782611f5252 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:06.493602 31705 consensus_queue.cc:237] T d05da516ed9040c988b1fdfe2fdde9ed P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:12:06.500761 30943 raft_consensus.cc:2468] T c1efa319af774b23bd4ea782611f5252 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05213ac836744a53b9c089a5c052f4be in term 2.
I20260709 13:12:06.501278 30875 raft_consensus.cc:2468] T c1efa319af774b23bd4ea782611f5252 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05213ac836744a53b9c089a5c052f4be in term 2.
I20260709 13:12:06.502259 30765 leader_election.cc:304] T c1efa319af774b23bd4ea782611f5252 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
I20260709 13:12:06.503221 31698 raft_consensus.cc:2804] T c1efa319af774b23bd4ea782611f5252 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:06.503729 31698 raft_consensus.cc:697] T c1efa319af774b23bd4ea782611f5252 P 05213ac836744a53b9c089a5c052f4be [term 2 LEADER]: Becoming Leader. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Running, Role: LEADER
I20260709 13:12:06.504554 31698 consensus_queue.cc:237] T c1efa319af774b23bd4ea782611f5252 P 05213ac836744a53b9c089a5c052f4be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:12:06.538411 30620 catalog_manager.cc:5697] T c1efa319af774b23bd4ea782611f5252 P 05213ac836744a53b9c089a5c052f4be reported cstate change: term changed from 1 to 2, leader changed from 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4) to 05213ac836744a53b9c089a5c052f4be (127.21.63.2). New cstate: current_term: 2 leader_uuid: "05213ac836744a53b9c089a5c052f4be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:06.548764 30624 catalog_manager.cc:5697] T d05da516ed9040c988b1fdfe2fdde9ed P d22c9d3d15904327a78a0e548a8ff6cd reported cstate change: term changed from 1 to 2, leader changed from 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3) to d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1). New cstate: current_term: 2 leader_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:06.597985 31706 raft_consensus.cc:993] T b226c44412c74106a0feb2ca411b1661 P 02c5705b45c04706bbb2c6ff3d7b0b8a: : Instructing follower 05213ac836744a53b9c089a5c052f4be to start an election
I20260709 13:12:06.598505 31599 raft_consensus.cc:1081] T b226c44412c74106a0feb2ca411b1661 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Signalling peer 05213ac836744a53b9c089a5c052f4be to start an election
I20260709 13:12:06.599934 30795 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b226c44412c74106a0feb2ca411b1661"
dest_uuid: "05213ac836744a53b9c089a5c052f4be"
 from {username='slave'} at 127.0.0.1:39594
I20260709 13:12:06.600569 30795 raft_consensus.cc:493] T b226c44412c74106a0feb2ca411b1661 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:06.600926 30795 raft_consensus.cc:3060] T b226c44412c74106a0feb2ca411b1661 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:06.607471 30795 raft_consensus.cc:515] T b226c44412c74106a0feb2ca411b1661 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:12:06.609532 30795 leader_election.cc:290] T b226c44412c74106a0feb2ca411b1661 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 2 election: Requested vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:12:06.610426 30943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b226c44412c74106a0feb2ca411b1661" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:12:06.610610 30875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b226c44412c74106a0feb2ca411b1661" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:12:06.611093 30943 raft_consensus.cc:3060] T b226c44412c74106a0feb2ca411b1661 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:06.611244 30875 raft_consensus.cc:3055] T b226c44412c74106a0feb2ca411b1661 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:06.611667 30875 raft_consensus.cc:740] T b226c44412c74106a0feb2ca411b1661 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Running, Role: LEADER
I20260709 13:12:06.612375 30875 consensus_queue.cc:260] T b226c44412c74106a0feb2ca411b1661 P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:12:06.613397 30875 raft_consensus.cc:3060] T b226c44412c74106a0feb2ca411b1661 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:06.615816 30943 raft_consensus.cc:2468] T b226c44412c74106a0feb2ca411b1661 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05213ac836744a53b9c089a5c052f4be in term 2.
I20260709 13:12:06.617224 30765 leader_election.cc:304] T b226c44412c74106a0feb2ca411b1661 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
I20260709 13:12:06.617982 31698 raft_consensus.cc:2804] T b226c44412c74106a0feb2ca411b1661 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:06.618419 31698 raft_consensus.cc:697] T b226c44412c74106a0feb2ca411b1661 P 05213ac836744a53b9c089a5c052f4be [term 2 LEADER]: Becoming Leader. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Running, Role: LEADER
I20260709 13:12:06.619099 30875 raft_consensus.cc:2468] T b226c44412c74106a0feb2ca411b1661 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05213ac836744a53b9c089a5c052f4be in term 2.
I20260709 13:12:06.619233 31698 consensus_queue.cc:237] T b226c44412c74106a0feb2ca411b1661 P 05213ac836744a53b9c089a5c052f4be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:12:06.630239 30624 catalog_manager.cc:5697] T b226c44412c74106a0feb2ca411b1661 P 05213ac836744a53b9c089a5c052f4be reported cstate change: term changed from 1 to 2, leader changed from 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3) to 05213ac836744a53b9c089a5c052f4be (127.21.63.2). New cstate: current_term: 2 leader_uuid: "05213ac836744a53b9c089a5c052f4be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:06.684162 31599 raft_consensus.cc:993] T 726b808ed64848dd89da4340d4d891fb P 02c5705b45c04706bbb2c6ff3d7b0b8a: : Instructing follower d22c9d3d15904327a78a0e548a8ff6cd to start an election
I20260709 13:12:06.684756 31706 raft_consensus.cc:1081] T 726b808ed64848dd89da4340d4d891fb P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 LEADER]: Signalling peer d22c9d3d15904327a78a0e548a8ff6cd to start an election
I20260709 13:12:06.686528 31532 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "726b808ed64848dd89da4340d4d891fb"
dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
 from {username='slave'} at 127.0.0.1:34014
I20260709 13:12:06.687065 31532 raft_consensus.cc:493] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:06.687439 31532 raft_consensus.cc:3060] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:06.694386 31532 raft_consensus.cc:515] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:12:06.696552 31532 leader_election.cc:290] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 3 election: Requested vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:12:06.696938 30875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "726b808ed64848dd89da4340d4d891fb" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:12:06.697167 30943 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "726b808ed64848dd89da4340d4d891fb" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:12:06.697613 30875 raft_consensus.cc:3055] T 726b808ed64848dd89da4340d4d891fb P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:06.698153 30943 raft_consensus.cc:3060] T 726b808ed64848dd89da4340d4d891fb P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:06.698300 30875 raft_consensus.cc:740] T 726b808ed64848dd89da4340d4d891fb P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Running, Role: LEADER
I20260709 13:12:06.699190 30875 consensus_queue.cc:260] T 726b808ed64848dd89da4340d4d891fb P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:12:06.700506 30875 raft_consensus.cc:3060] T 726b808ed64848dd89da4340d4d891fb P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:06.705281 30943 raft_consensus.cc:2468] T 726b808ed64848dd89da4340d4d891fb P 262942f67dfa4b8fa42234b18ac31ffd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 3.
I20260709 13:12:06.706470 31482 leader_election.cc:304] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:12:06.707332 31705 raft_consensus.cc:2804] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:06.707489 30875 raft_consensus.cc:2468] T 726b808ed64848dd89da4340d4d891fb P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 3.
I20260709 13:12:06.707913 31705 raft_consensus.cc:697] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd [term 3 LEADER]: Becoming Leader. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Running, Role: LEADER
I20260709 13:12:06.708969 31705 consensus_queue.cc:237] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:12:06.720281 30624 catalog_manager.cc:5697] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd reported cstate change: term changed from 2 to 3, leader changed from 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3) to d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1). New cstate: current_term: 3 leader_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:06.755026 31706 raft_consensus.cc:993] T 8155b3fc02b348e1adb290ecfe4592c5 P 02c5705b45c04706bbb2c6ff3d7b0b8a: : Instructing follower d22c9d3d15904327a78a0e548a8ff6cd to start an election
I20260709 13:12:06.755491 31706 raft_consensus.cc:1081] T 8155b3fc02b348e1adb290ecfe4592c5 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Signalling peer d22c9d3d15904327a78a0e548a8ff6cd to start an election
I20260709 13:12:06.756986 31532 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8155b3fc02b348e1adb290ecfe4592c5"
dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
 from {username='slave'} at 127.0.0.1:34014
I20260709 13:12:06.757547 31532 raft_consensus.cc:493] T 8155b3fc02b348e1adb290ecfe4592c5 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:06.757896 31532 raft_consensus.cc:3060] T 8155b3fc02b348e1adb290ecfe4592c5 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:06.764856 31532 raft_consensus.cc:515] T 8155b3fc02b348e1adb290ecfe4592c5 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:12:06.766836 31532 leader_election.cc:290] T 8155b3fc02b348e1adb290ecfe4592c5 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 2 election: Requested vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:12:06.767545 30795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8155b3fc02b348e1adb290ecfe4592c5" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:12:06.768111 30795 raft_consensus.cc:3060] T 8155b3fc02b348e1adb290ecfe4592c5 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:06.768034 30875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8155b3fc02b348e1adb290ecfe4592c5" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:12:06.768853 30875 raft_consensus.cc:3055] T 8155b3fc02b348e1adb290ecfe4592c5 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:06.769187 30875 raft_consensus.cc:740] T 8155b3fc02b348e1adb290ecfe4592c5 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Running, Role: LEADER
I20260709 13:12:06.769982 30875 consensus_queue.cc:260] T 8155b3fc02b348e1adb290ecfe4592c5 P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:12:06.771196 30875 raft_consensus.cc:3060] T 8155b3fc02b348e1adb290ecfe4592c5 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:06.774142 30795 raft_consensus.cc:2468] T 8155b3fc02b348e1adb290ecfe4592c5 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 2.
I20260709 13:12:06.775813 31481 leader_election.cc:304] T 8155b3fc02b348e1adb290ecfe4592c5 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:12:06.776449 31705 raft_consensus.cc:2804] T 8155b3fc02b348e1adb290ecfe4592c5 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:06.777014 31705 raft_consensus.cc:697] T 8155b3fc02b348e1adb290ecfe4592c5 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 LEADER]: Becoming Leader. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Running, Role: LEADER
I20260709 13:12:06.778072 30875 raft_consensus.cc:2468] T 8155b3fc02b348e1adb290ecfe4592c5 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 2.
I20260709 13:12:06.777953 31705 consensus_queue.cc:237] T 8155b3fc02b348e1adb290ecfe4592c5 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:12:06.792753 30624 catalog_manager.cc:5697] T 8155b3fc02b348e1adb290ecfe4592c5 P d22c9d3d15904327a78a0e548a8ff6cd reported cstate change: term changed from 1 to 2, leader changed from 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3) to d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1). New cstate: current_term: 2 leader_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:06.851248 30946 raft_consensus.cc:1275] T ea563bd67a9849d18c9c3dfb21d58a93 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Refusing update from remote peer d22c9d3d15904327a78a0e548a8ff6cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:06.852600 31705 consensus_queue.cc:1048] T ea563bd67a9849d18c9c3dfb21d58a93 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:06.868279 30875 raft_consensus.cc:1275] T ea563bd67a9849d18c9c3dfb21d58a93 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Refusing update from remote peer d22c9d3d15904327a78a0e548a8ff6cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:06.870299 31705 consensus_queue.cc:1048] T ea563bd67a9849d18c9c3dfb21d58a93 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:07.023856 30875 raft_consensus.cc:1275] T d05da516ed9040c988b1fdfe2fdde9ed P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Refusing update from remote peer d22c9d3d15904327a78a0e548a8ff6cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:07.025095 31705 consensus_queue.cc:1048] T d05da516ed9040c988b1fdfe2fdde9ed P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:07.034727 30799 raft_consensus.cc:1275] T d05da516ed9040c988b1fdfe2fdde9ed P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Refusing update from remote peer d22c9d3d15904327a78a0e548a8ff6cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:07.036810 31707 consensus_queue.cc:1048] T d05da516ed9040c988b1fdfe2fdde9ed P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:07.048488 30946 raft_consensus.cc:1275] T c1efa319af774b23bd4ea782611f5252 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Refusing update from remote peer 05213ac836744a53b9c089a5c052f4be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:07.050025 31698 consensus_queue.cc:1048] T c1efa319af774b23bd4ea782611f5252 P 05213ac836744a53b9c089a5c052f4be [LEADER]: Connected to new peer: Peer: permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:07.062989 30875 raft_consensus.cc:1275] T b226c44412c74106a0feb2ca411b1661 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Refusing update from remote peer 05213ac836744a53b9c089a5c052f4be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:07.066962 31730 consensus_queue.cc:1048] T b226c44412c74106a0feb2ca411b1661 P 05213ac836744a53b9c089a5c052f4be [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:07.077003 30873 raft_consensus.cc:1275] T c1efa319af774b23bd4ea782611f5252 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Refusing update from remote peer 05213ac836744a53b9c089a5c052f4be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:07.080520 31731 consensus_queue.cc:1048] T c1efa319af774b23bd4ea782611f5252 P 05213ac836744a53b9c089a5c052f4be [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:07.087460 30946 raft_consensus.cc:1275] T b226c44412c74106a0feb2ca411b1661 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Refusing update from remote peer 05213ac836744a53b9c089a5c052f4be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:07.105093 31730 consensus_queue.cc:1048] T b226c44412c74106a0feb2ca411b1661 P 05213ac836744a53b9c089a5c052f4be [LEADER]: Connected to new peer: Peer: permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:07.150440 30875 raft_consensus.cc:1275] T 726b808ed64848dd89da4340d4d891fb P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 3 FOLLOWER]: Refusing update from remote peer d22c9d3d15904327a78a0e548a8ff6cd: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:07.151690 31726 consensus_queue.cc:1048] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:07.161029 30948 raft_consensus.cc:1275] T 726b808ed64848dd89da4340d4d891fb P 262942f67dfa4b8fa42234b18ac31ffd [term 3 FOLLOWER]: Refusing update from remote peer d22c9d3d15904327a78a0e548a8ff6cd: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:07.162549 31707 consensus_queue.cc:1048] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:07.175156 30799 raft_consensus.cc:1275] T 8155b3fc02b348e1adb290ecfe4592c5 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Refusing update from remote peer d22c9d3d15904327a78a0e548a8ff6cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:07.177371 31707 consensus_queue.cc:1048] T 8155b3fc02b348e1adb290ecfe4592c5 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:07.191957 30875 raft_consensus.cc:1275] T 8155b3fc02b348e1adb290ecfe4592c5 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Refusing update from remote peer d22c9d3d15904327a78a0e548a8ff6cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:07.198912 31705 consensus_queue.cc:1048] T 8155b3fc02b348e1adb290ecfe4592c5 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:07.373232 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:07.412835 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:07.448464 30948 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "01cceb88bca348958b59f3697979dff1"
dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
mode: GRACEFUL
new_leader_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
 from {username='slave'} at 127.0.0.1:33910
I20260709 13:12:07.449110 30948 raft_consensus.cc:606] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 LEADER]: Received request to transfer leadership to TS d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:12:07.451452 30875 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0d8c1da1f7254fde94f7deaad5f41709"
dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
mode: GRACEFUL
new_leader_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
 from {username='slave'} at 127.0.0.1:46350
I20260709 13:12:07.451961 30875 raft_consensus.cc:606] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 LEADER]: Received request to transfer leadership to TS d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:12:07.454567 30875 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1474b6fc151c4a3994a3aef89ea936a2"
dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
mode: GRACEFUL
new_leader_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
 from {username='slave'} at 127.0.0.1:46350
I20260709 13:12:07.455009 30875 raft_consensus.cc:606] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 3 LEADER]: Received request to transfer leadership to TS d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:12:07.457767 30875 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5028253a91e74a73afca005c77e8b565"
dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
mode: GRACEFUL
new_leader_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
 from {username='slave'} at 127.0.0.1:46350
I20260709 13:12:07.458325 30875 raft_consensus.cc:606] T 5028253a91e74a73afca005c77e8b565 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 LEADER]: Received request to transfer leadership to TS d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:12:07.460760 30875 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "99e98c1324084d85b8a97aaf0c248df7"
dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
mode: GRACEFUL
new_leader_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
 from {username='slave'} at 127.0.0.1:46350
I20260709 13:12:07.461189 30875 raft_consensus.cc:606] T 99e98c1324084d85b8a97aaf0c248df7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Received request to transfer leadership to TS d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:12:07.462121 21756 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:12:07.463331 21756 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:12:07.568254 31697 raft_consensus.cc:993] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd: : Instructing follower d22c9d3d15904327a78a0e548a8ff6cd to start an election
I20260709 13:12:07.568823 31697 raft_consensus.cc:1081] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 LEADER]: Signalling peer d22c9d3d15904327a78a0e548a8ff6cd to start an election
I20260709 13:12:07.570148 31530 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "01cceb88bca348958b59f3697979dff1"
dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
 from {username='slave'} at 127.0.0.1:34036
I20260709 13:12:07.570533 31530 raft_consensus.cc:493] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:07.570749 31530 raft_consensus.cc:3060] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:07.576942 31530 raft_consensus.cc:515] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:12:07.578540 31530 leader_election.cc:290] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 3 election: Requested vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:12:07.579169 30873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01cceb88bca348958b59f3697979dff1" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:12:07.579447 30948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "01cceb88bca348958b59f3697979dff1" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:12:07.580051 30948 raft_consensus.cc:3055] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:07.579807 30873 raft_consensus.cc:3060] T 01cceb88bca348958b59f3697979dff1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:07.580307 30948 raft_consensus.cc:740] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Running, Role: LEADER
I20260709 13:12:07.581068 30948 consensus_queue.cc:260] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:12:07.581933 30948 raft_consensus.cc:3060] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:07.586786 30873 raft_consensus.cc:2468] T 01cceb88bca348958b59f3697979dff1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 3.
I20260709 13:12:07.588490 30948 raft_consensus.cc:2468] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 3.
I20260709 13:12:07.588940 31483 leader_election.cc:304] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:12:07.590142 31726 raft_consensus.cc:2804] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:07.590622 31726 raft_consensus.cc:697] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd [term 3 LEADER]: Becoming Leader. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Running, Role: LEADER
I20260709 13:12:07.591640 31726 consensus_queue.cc:237] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:12:07.618243 30624 catalog_manager.cc:5697] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd reported cstate change: term changed from 2 to 3, leader changed from 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4) to d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1). New cstate: current_term: 3 leader_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:07.615355 31706 raft_consensus.cc:993] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a: : Instructing follower d22c9d3d15904327a78a0e548a8ff6cd to start an election
I20260709 13:12:07.622257 31706 raft_consensus.cc:1081] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 3 LEADER]: Signalling peer d22c9d3d15904327a78a0e548a8ff6cd to start an election
I20260709 13:12:07.624557 31530 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1474b6fc151c4a3994a3aef89ea936a2"
dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
 from {username='slave'} at 127.0.0.1:34014
I20260709 13:12:07.625265 31530 raft_consensus.cc:493] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:07.625643 31530 raft_consensus.cc:3060] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:07.632735 31530 raft_consensus.cc:515] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:12:07.635578 30795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1474b6fc151c4a3994a3aef89ea936a2" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:12:07.636267 30795 raft_consensus.cc:3060] T 1474b6fc151c4a3994a3aef89ea936a2 P 05213ac836744a53b9c089a5c052f4be [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:07.637024 31530 leader_election.cc:290] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 4 election: Requested vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:12:07.638825 30873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1474b6fc151c4a3994a3aef89ea936a2" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:12:07.639573 30873 raft_consensus.cc:3055] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:07.640188 30873 raft_consensus.cc:740] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Running, Role: LEADER
I20260709 13:12:07.642119 30873 consensus_queue.cc:260] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:12:07.643386 30873 raft_consensus.cc:3060] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:07.644845 30795 raft_consensus.cc:2468] T 1474b6fc151c4a3994a3aef89ea936a2 P 05213ac836744a53b9c089a5c052f4be [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 4.
I20260709 13:12:07.650367 31481 leader_election.cc:304] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:12:07.651336 31726 raft_consensus.cc:2804] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:07.651809 31726 raft_consensus.cc:697] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [term 4 LEADER]: Becoming Leader. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Running, Role: LEADER
I20260709 13:12:07.652704 31726 consensus_queue.cc:237] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:12:07.660812 30873 raft_consensus.cc:2468] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 4.
I20260709 13:12:07.674031 30624 catalog_manager.cc:5697] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd reported cstate change: term changed from 3 to 4, leader changed from 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3) to d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1). New cstate: current_term: 4 leader_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:07.810216 31599 raft_consensus.cc:993] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a: : Instructing follower d22c9d3d15904327a78a0e548a8ff6cd to start an election
I20260709 13:12:07.810720 31741 raft_consensus.cc:1081] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 LEADER]: Signalling peer d22c9d3d15904327a78a0e548a8ff6cd to start an election
I20260709 13:12:07.812333 31530 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0d8c1da1f7254fde94f7deaad5f41709"
dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
 from {username='slave'} at 127.0.0.1:34014
I20260709 13:12:07.812884 31530 raft_consensus.cc:493] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:07.813217 31530 raft_consensus.cc:3060] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:07.820772 31530 raft_consensus.cc:515] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:12:07.822978 30795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d8c1da1f7254fde94f7deaad5f41709" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:12:07.823371 30873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d8c1da1f7254fde94f7deaad5f41709" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:12:07.823532 30795 raft_consensus.cc:3060] T 0d8c1da1f7254fde94f7deaad5f41709 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:07.823935 30873 raft_consensus.cc:3055] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:07.824256 30873 raft_consensus.cc:740] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Running, Role: LEADER
I20260709 13:12:07.825124 31530 leader_election.cc:290] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 3 election: Requested vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:12:07.824975 30873 consensus_queue.cc:260] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:12:07.825973 30873 raft_consensus.cc:3060] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:07.828860 30795 raft_consensus.cc:2468] T 0d8c1da1f7254fde94f7deaad5f41709 P 05213ac836744a53b9c089a5c052f4be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 3.
I20260709 13:12:07.829804 31481 leader_election.cc:304] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:12:07.830430 31726 raft_consensus.cc:2804] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:07.830890 31726 raft_consensus.cc:697] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd [term 3 LEADER]: Becoming Leader. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Running, Role: LEADER
I20260709 13:12:07.830946 30873 raft_consensus.cc:2468] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 3.
I20260709 13:12:07.831884 31726 consensus_queue.cc:237] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:12:07.842422 30624 catalog_manager.cc:5697] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd reported cstate change: term changed from 2 to 3, leader changed from 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3) to d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1). New cstate: current_term: 3 leader_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:07.922927 31599 raft_consensus.cc:993] T 99e98c1324084d85b8a97aaf0c248df7 P 02c5705b45c04706bbb2c6ff3d7b0b8a: : Instructing follower d22c9d3d15904327a78a0e548a8ff6cd to start an election
I20260709 13:12:07.923434 31741 raft_consensus.cc:1081] T 99e98c1324084d85b8a97aaf0c248df7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Signalling peer d22c9d3d15904327a78a0e548a8ff6cd to start an election
I20260709 13:12:07.924991 31530 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "99e98c1324084d85b8a97aaf0c248df7"
dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
 from {username='slave'} at 127.0.0.1:34014
I20260709 13:12:07.925549 31530 raft_consensus.cc:493] T 99e98c1324084d85b8a97aaf0c248df7 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:07.925836 31530 raft_consensus.cc:3060] T 99e98c1324084d85b8a97aaf0c248df7 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:07.930198 31530 raft_consensus.cc:515] T 99e98c1324084d85b8a97aaf0c248df7 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:12:07.931600 31530 leader_election.cc:290] T 99e98c1324084d85b8a97aaf0c248df7 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 2 election: Requested vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:12:07.932489 30875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e98c1324084d85b8a97aaf0c248df7" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:12:07.933132 30946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "99e98c1324084d85b8a97aaf0c248df7" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:12:07.933135 30875 raft_consensus.cc:3055] T 99e98c1324084d85b8a97aaf0c248df7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:07.933753 30946 raft_consensus.cc:3060] T 99e98c1324084d85b8a97aaf0c248df7 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:07.933986 30875 raft_consensus.cc:740] T 99e98c1324084d85b8a97aaf0c248df7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Running, Role: LEADER
I20260709 13:12:07.934741 30875 consensus_queue.cc:260] T 99e98c1324084d85b8a97aaf0c248df7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:12:07.936053 30875 raft_consensus.cc:3060] T 99e98c1324084d85b8a97aaf0c248df7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:07.940917 30946 raft_consensus.cc:2468] T 99e98c1324084d85b8a97aaf0c248df7 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 2.
I20260709 13:12:07.942211 31482 leader_election.cc:304] T 99e98c1324084d85b8a97aaf0c248df7 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:12:07.943017 31726 raft_consensus.cc:2804] T 99e98c1324084d85b8a97aaf0c248df7 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:07.943410 31726 raft_consensus.cc:697] T 99e98c1324084d85b8a97aaf0c248df7 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 LEADER]: Becoming Leader. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Running, Role: LEADER
I20260709 13:12:07.944190 30875 raft_consensus.cc:2468] T 99e98c1324084d85b8a97aaf0c248df7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 2.
I20260709 13:12:07.944239 31726 consensus_queue.cc:237] T 99e98c1324084d85b8a97aaf0c248df7 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:12:07.954092 30624 catalog_manager.cc:5697] T 99e98c1324084d85b8a97aaf0c248df7 P d22c9d3d15904327a78a0e548a8ff6cd reported cstate change: term changed from 1 to 2, leader changed from 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3) to d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1). New cstate: current_term: 2 leader_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:07.997956 31741 raft_consensus.cc:993] T 5028253a91e74a73afca005c77e8b565 P 02c5705b45c04706bbb2c6ff3d7b0b8a: : Instructing follower d22c9d3d15904327a78a0e548a8ff6cd to start an election
I20260709 13:12:07.998481 31599 raft_consensus.cc:1081] T 5028253a91e74a73afca005c77e8b565 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 LEADER]: Signalling peer d22c9d3d15904327a78a0e548a8ff6cd to start an election
I20260709 13:12:07.999966 31532 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5028253a91e74a73afca005c77e8b565"
dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
 from {username='slave'} at 127.0.0.1:34014
I20260709 13:12:08.000579 31532 raft_consensus.cc:493] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:08.001220 31532 raft_consensus.cc:3060] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:08.009104 31532 raft_consensus.cc:515] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:12:08.011379 31532 leader_election.cc:290] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 3 election: Requested vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:12:08.012647 30873 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5028253a91e74a73afca005c77e8b565" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:12:08.013365 30873 raft_consensus.cc:3055] T 5028253a91e74a73afca005c77e8b565 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:08.013921 30873 raft_consensus.cc:740] T 5028253a91e74a73afca005c77e8b565 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Running, Role: LEADER
I20260709 13:12:08.015192 30795 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5028253a91e74a73afca005c77e8b565" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:12:08.015813 30795 raft_consensus.cc:3060] T 5028253a91e74a73afca005c77e8b565 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:08.016122 30873 consensus_queue.cc:260] T 5028253a91e74a73afca005c77e8b565 P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:12:08.017373 30873 raft_consensus.cc:3060] T 5028253a91e74a73afca005c77e8b565 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:08.013842 30875 raft_consensus.cc:1275] T 01cceb88bca348958b59f3697979dff1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 3 FOLLOWER]: Refusing update from remote peer d22c9d3d15904327a78a0e548a8ff6cd: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:08.019980 31726 consensus_queue.cc:1048] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:08.023324 30873 raft_consensus.cc:2468] T 5028253a91e74a73afca005c77e8b565 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 3.
I20260709 13:12:08.026342 31483 leader_election.cc:304] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:12:08.027343 31726 raft_consensus.cc:2804] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:08.027801 31726 raft_consensus.cc:697] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd [term 3 LEADER]: Becoming Leader. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Running, Role: LEADER
I20260709 13:12:08.028564 31726 consensus_queue.cc:237] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:12:08.028055 30795 raft_consensus.cc:2468] T 5028253a91e74a73afca005c77e8b565 P 05213ac836744a53b9c089a5c052f4be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 3.
I20260709 13:12:08.039134 30624 catalog_manager.cc:5697] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd reported cstate change: term changed from 2 to 3, leader changed from 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3) to d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1). New cstate: current_term: 3 leader_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:08.042591 30946 raft_consensus.cc:1275] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd [term 3 FOLLOWER]: Refusing update from remote peer d22c9d3d15904327a78a0e548a8ff6cd: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:08.044072 31746 consensus_queue.cc:1048] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:08.112603 30875 raft_consensus.cc:1275] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 4 FOLLOWER]: Refusing update from remote peer d22c9d3d15904327a78a0e548a8ff6cd: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:12:08.113948 31726 consensus_queue.cc:1048] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:08.125648 30795 raft_consensus.cc:1275] T 1474b6fc151c4a3994a3aef89ea936a2 P 05213ac836744a53b9c089a5c052f4be [term 4 FOLLOWER]: Refusing update from remote peer d22c9d3d15904327a78a0e548a8ff6cd: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:12:08.127807 31726 consensus_queue.cc:1048] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:08.334524 30948 raft_consensus.cc:1275] T 99e98c1324084d85b8a97aaf0c248df7 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Refusing update from remote peer d22c9d3d15904327a78a0e548a8ff6cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:08.335940 31707 consensus_queue.cc:1048] T 99e98c1324084d85b8a97aaf0c248df7 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:08.345450 30799 raft_consensus.cc:1275] T 0d8c1da1f7254fde94f7deaad5f41709 P 05213ac836744a53b9c089a5c052f4be [term 3 FOLLOWER]: Refusing update from remote peer d22c9d3d15904327a78a0e548a8ff6cd: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:08.346843 31726 consensus_queue.cc:1048] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:08.349596 30875 raft_consensus.cc:1275] T 99e98c1324084d85b8a97aaf0c248df7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Refusing update from remote peer d22c9d3d15904327a78a0e548a8ff6cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:08.350896 31726 consensus_queue.cc:1048] T 99e98c1324084d85b8a97aaf0c248df7 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:08.370029 30873 raft_consensus.cc:1275] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 3 FOLLOWER]: Refusing update from remote peer d22c9d3d15904327a78a0e548a8ff6cd: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:08.371749 31705 consensus_queue.cc:1048] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:08.418567 30799 raft_consensus.cc:1275] T 5028253a91e74a73afca005c77e8b565 P 05213ac836744a53b9c089a5c052f4be [term 3 FOLLOWER]: Refusing update from remote peer d22c9d3d15904327a78a0e548a8ff6cd: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:08.419739 31707 consensus_queue.cc:1048] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:08.429767 30875 raft_consensus.cc:1275] T 5028253a91e74a73afca005c77e8b565 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 3 FOLLOWER]: Refusing update from remote peer d22c9d3d15904327a78a0e548a8ff6cd: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:08.431295 31726 consensus_queue.cc:1048] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:08.464174 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:08.505084 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:08.543597 30948 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4fb3f8d627b04c8a8971f7e4d3a5c010"
dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
mode: GRACEFUL
new_leader_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
 from {username='slave'} at 127.0.0.1:33910
I20260709 13:12:08.544212 30948 raft_consensus.cc:606] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 LEADER]: Received request to transfer leadership to TS d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:12:08.548928 30948 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "80a61882ef804a85a31e90d67930c5e9"
dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
mode: GRACEFUL
new_leader_uuid: "05213ac836744a53b9c089a5c052f4be"
 from {username='slave'} at 127.0.0.1:33910
I20260709 13:12:08.549549 30948 raft_consensus.cc:606] T 80a61882ef804a85a31e90d67930c5e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Received request to transfer leadership to TS 05213ac836744a53b9c089a5c052f4be
I20260709 13:12:08.552268 30948 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a38ff14550f74527a62591df59fb2c51"
dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
mode: GRACEFUL
new_leader_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
 from {username='slave'} at 127.0.0.1:33910
I20260709 13:12:08.552839 30948 raft_consensus.cc:606] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Received request to transfer leadership to TS d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:12:08.555326 30948 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ea4dcd4ecc9840d2b8785d14dd4537b8"
dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
mode: GRACEFUL
new_leader_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
 from {username='slave'} at 127.0.0.1:33910
I20260709 13:12:08.555805 30948 raft_consensus.cc:606] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Received request to transfer leadership to TS 02c5705b45c04706bbb2c6ff3d7b0b8a
I20260709 13:12:08.558287 30948 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f63e902c7c7a4d0fa5d2107620ce851e"
dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
mode: GRACEFUL
new_leader_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
 from {username='slave'} at 127.0.0.1:33910
I20260709 13:12:08.558830 30948 raft_consensus.cc:606] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Received request to transfer leadership to TS d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:12:08.561183 21756 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 13:12:08.562701 21756 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:12:08.567027 31697 raft_consensus.cc:993] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 262942f67dfa4b8fa42234b18ac31ffd: : Instructing follower 02c5705b45c04706bbb2c6ff3d7b0b8a to start an election
I20260709 13:12:08.567523 31697 raft_consensus.cc:1081] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Signalling peer 02c5705b45c04706bbb2c6ff3d7b0b8a to start an election
I20260709 13:12:08.568835 30875 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ea4dcd4ecc9840d2b8785d14dd4537b8"
dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
 from {username='slave'} at 127.0.0.1:48630
I20260709 13:12:08.569494 30875 raft_consensus.cc:493] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:08.569790 30875 raft_consensus.cc:3060] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:08.574465 30875 raft_consensus.cc:515] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:12:08.576635 30875 leader_election.cc:290] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 election: Requested vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:12:08.577232 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4dcd4ecc9840d2b8785d14dd4537b8" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:12:08.577316 30946 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ea4dcd4ecc9840d2b8785d14dd4537b8" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:12:08.577977 30799 raft_consensus.cc:3060] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:08.578018 30946 raft_consensus.cc:3055] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:08.578483 30946 raft_consensus.cc:740] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Running, Role: LEADER
I20260709 13:12:08.579269 30946 consensus_queue.cc:260] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:12:08.580379 30946 raft_consensus.cc:3060] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:08.583617 30799 raft_consensus.cc:2468] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 2.
I20260709 13:12:08.584877 30838 leader_election.cc:304] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 05213ac836744a53b9c089a5c052f4be; no voters: 
I20260709 13:12:08.585529 31599 raft_consensus.cc:2804] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:08.585747 30946 raft_consensus.cc:2468] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 2.
I20260709 13:12:08.585956 31599 raft_consensus.cc:697] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 LEADER]: Becoming Leader. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Running, Role: LEADER
I20260709 13:12:08.587401 31599 consensus_queue.cc:237] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:12:08.599845 30624 catalog_manager.cc:5697] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 02c5705b45c04706bbb2c6ff3d7b0b8a reported cstate change: term changed from 1 to 2, leader changed from 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4) to 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3). New cstate: current_term: 2 leader_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:08.695591 31697 raft_consensus.cc:993] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd: : Instructing follower d22c9d3d15904327a78a0e548a8ff6cd to start an election
I20260709 13:12:08.696075 31682 raft_consensus.cc:1081] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 LEADER]: Signalling peer d22c9d3d15904327a78a0e548a8ff6cd to start an election
I20260709 13:12:08.697650 31532 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4fb3f8d627b04c8a8971f7e4d3a5c010"
dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
 from {username='slave'} at 127.0.0.1:34036
I20260709 13:12:08.698109 31532 raft_consensus.cc:493] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:08.698417 31532 raft_consensus.cc:3060] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:08.702730 31532 raft_consensus.cc:515] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:12:08.704093 31532 leader_election.cc:290] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 3 election: Requested vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:12:08.704859 30948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fb3f8d627b04c8a8971f7e4d3a5c010" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:12:08.704998 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4fb3f8d627b04c8a8971f7e4d3a5c010" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:12:08.705452 30948 raft_consensus.cc:3055] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:08.705503 30799 raft_consensus.cc:3060] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:08.705744 30948 raft_consensus.cc:740] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Running, Role: LEADER
I20260709 13:12:08.706540 30948 consensus_queue.cc:260] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:12:08.707432 30948 raft_consensus.cc:3060] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:08.711189 30799 raft_consensus.cc:2468] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 05213ac836744a53b9c089a5c052f4be [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 3.
I20260709 13:12:08.712214 31481 leader_election.cc:304] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:12:08.712991 31746 raft_consensus.cc:2804] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:08.713467 31746 raft_consensus.cc:697] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd [term 3 LEADER]: Becoming Leader. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Running, Role: LEADER
I20260709 13:12:08.714346 31746 consensus_queue.cc:237] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:12:08.715188 30948 raft_consensus.cc:2468] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 3.
I20260709 13:12:08.729563 30624 catalog_manager.cc:5697] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd reported cstate change: term changed from 2 to 3, leader changed from 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4) to d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1). New cstate: current_term: 3 leader_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:08.746578 31682 raft_consensus.cc:993] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd: : Instructing follower d22c9d3d15904327a78a0e548a8ff6cd to start an election
I20260709 13:12:08.747058 31682 raft_consensus.cc:1081] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Signalling peer d22c9d3d15904327a78a0e548a8ff6cd to start an election
I20260709 13:12:08.748497 31530 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a38ff14550f74527a62591df59fb2c51"
dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
 from {username='slave'} at 127.0.0.1:34036
I20260709 13:12:08.749125 31530 raft_consensus.cc:493] T a38ff14550f74527a62591df59fb2c51 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:08.749724 31530 raft_consensus.cc:3060] T a38ff14550f74527a62591df59fb2c51 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:08.756031 31530 raft_consensus.cc:515] T a38ff14550f74527a62591df59fb2c51 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:12:08.757579 31530 leader_election.cc:290] T a38ff14550f74527a62591df59fb2c51 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 2 election: Requested vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:12:08.758396 30948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38ff14550f74527a62591df59fb2c51" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:12:08.758452 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a38ff14550f74527a62591df59fb2c51" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:12:08.759032 30948 raft_consensus.cc:3055] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:08.759155 30799 raft_consensus.cc:3060] T a38ff14550f74527a62591df59fb2c51 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:08.759469 30948 raft_consensus.cc:740] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Running, Role: LEADER
I20260709 13:12:08.760352 30948 consensus_queue.cc:260] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:12:08.761632 30948 raft_consensus.cc:3060] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:08.764928 30799 raft_consensus.cc:2468] T a38ff14550f74527a62591df59fb2c51 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 2.
I20260709 13:12:08.766050 31481 leader_election.cc:304] T a38ff14550f74527a62591df59fb2c51 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:12:08.766894 31726 raft_consensus.cc:2804] T a38ff14550f74527a62591df59fb2c51 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:08.767679 31726 raft_consensus.cc:697] T a38ff14550f74527a62591df59fb2c51 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 LEADER]: Becoming Leader. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Running, Role: LEADER
I20260709 13:12:08.769167 30948 raft_consensus.cc:2468] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 2.
I20260709 13:12:08.768733 31726 consensus_queue.cc:237] T a38ff14550f74527a62591df59fb2c51 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:12:08.778775 30624 catalog_manager.cc:5697] T a38ff14550f74527a62591df59fb2c51 P d22c9d3d15904327a78a0e548a8ff6cd reported cstate change: term changed from 1 to 2, leader changed from 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4) to d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1). New cstate: current_term: 2 leader_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:08.785141 31697 raft_consensus.cc:993] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd: : Instructing follower d22c9d3d15904327a78a0e548a8ff6cd to start an election
I20260709 13:12:08.785640 31682 raft_consensus.cc:1081] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Signalling peer d22c9d3d15904327a78a0e548a8ff6cd to start an election
I20260709 13:12:08.787209 31530 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f63e902c7c7a4d0fa5d2107620ce851e"
dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
 from {username='slave'} at 127.0.0.1:34036
I20260709 13:12:08.787720 31530 raft_consensus.cc:493] T f63e902c7c7a4d0fa5d2107620ce851e P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:08.788048 31530 raft_consensus.cc:3060] T f63e902c7c7a4d0fa5d2107620ce851e P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:08.794754 31530 raft_consensus.cc:515] T f63e902c7c7a4d0fa5d2107620ce851e P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:12:08.796550 31530 leader_election.cc:290] T f63e902c7c7a4d0fa5d2107620ce851e P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 2 election: Requested vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:12:08.797433 30948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f63e902c7c7a4d0fa5d2107620ce851e" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:12:08.798061 30948 raft_consensus.cc:3055] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:08.798076 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f63e902c7c7a4d0fa5d2107620ce851e" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:12:08.798424 30948 raft_consensus.cc:740] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Running, Role: LEADER
I20260709 13:12:08.799019 30799 raft_consensus.cc:3060] T f63e902c7c7a4d0fa5d2107620ce851e P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:08.799436 30948 consensus_queue.cc:260] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:12:08.800727 30948 raft_consensus.cc:3060] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:08.807895 30799 raft_consensus.cc:2468] T f63e902c7c7a4d0fa5d2107620ce851e P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 2.
I20260709 13:12:08.809173 31481 leader_election.cc:304] T f63e902c7c7a4d0fa5d2107620ce851e P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:12:08.809916 31707 raft_consensus.cc:2804] T f63e902c7c7a4d0fa5d2107620ce851e P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:08.810668 31707 raft_consensus.cc:697] T f63e902c7c7a4d0fa5d2107620ce851e P d22c9d3d15904327a78a0e548a8ff6cd [term 2 LEADER]: Becoming Leader. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Running, Role: LEADER
I20260709 13:12:08.811539 31707 consensus_queue.cc:237] T f63e902c7c7a4d0fa5d2107620ce851e P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:12:08.814075 30948 raft_consensus.cc:2468] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 2.
I20260709 13:12:08.825839 30624 catalog_manager.cc:5697] T f63e902c7c7a4d0fa5d2107620ce851e P d22c9d3d15904327a78a0e548a8ff6cd reported cstate change: term changed from 1 to 2, leader changed from 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4) to d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1). New cstate: current_term: 2 leader_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:08.851315 31697 raft_consensus.cc:993] T 80a61882ef804a85a31e90d67930c5e9 P 262942f67dfa4b8fa42234b18ac31ffd: : Instructing follower 05213ac836744a53b9c089a5c052f4be to start an election
I20260709 13:12:08.851769 31682 raft_consensus.cc:1081] T 80a61882ef804a85a31e90d67930c5e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Signalling peer 05213ac836744a53b9c089a5c052f4be to start an election
I20260709 13:12:08.853246 30795 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "80a61882ef804a85a31e90d67930c5e9"
dest_uuid: "05213ac836744a53b9c089a5c052f4be"
 from {username='slave'} at 127.0.0.1:39606
I20260709 13:12:08.853775 30795 raft_consensus.cc:493] T 80a61882ef804a85a31e90d67930c5e9 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:08.854184 30795 raft_consensus.cc:3060] T 80a61882ef804a85a31e90d67930c5e9 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:08.859382 30795 raft_consensus.cc:515] T 80a61882ef804a85a31e90d67930c5e9 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:12:08.861907 30795 leader_election.cc:290] T 80a61882ef804a85a31e90d67930c5e9 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 2 election: Requested vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:12:08.863013 30948 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80a61882ef804a85a31e90d67930c5e9" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:12:08.863605 30948 raft_consensus.cc:3055] T 80a61882ef804a85a31e90d67930c5e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:08.863857 30948 raft_consensus.cc:740] T 80a61882ef804a85a31e90d67930c5e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Running, Role: LEADER
I20260709 13:12:08.864571 30948 consensus_queue.cc:260] T 80a61882ef804a85a31e90d67930c5e9 P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:12:08.866530 30948 raft_consensus.cc:3060] T 80a61882ef804a85a31e90d67930c5e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:08.865010 30875 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "80a61882ef804a85a31e90d67930c5e9" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:12:08.872901 30875 raft_consensus.cc:3060] T 80a61882ef804a85a31e90d67930c5e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:08.878891 30875 raft_consensus.cc:2468] T 80a61882ef804a85a31e90d67930c5e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05213ac836744a53b9c089a5c052f4be in term 2.
I20260709 13:12:08.880720 30948 raft_consensus.cc:2468] T 80a61882ef804a85a31e90d67930c5e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05213ac836744a53b9c089a5c052f4be in term 2.
I20260709 13:12:08.881639 30766 leader_election.cc:304] T 80a61882ef804a85a31e90d67930c5e9 P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 05213ac836744a53b9c089a5c052f4be; no voters: 
I20260709 13:12:08.882692 31730 raft_consensus.cc:2804] T 80a61882ef804a85a31e90d67930c5e9 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:08.883131 31730 raft_consensus.cc:697] T 80a61882ef804a85a31e90d67930c5e9 P 05213ac836744a53b9c089a5c052f4be [term 2 LEADER]: Becoming Leader. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Running, Role: LEADER
I20260709 13:12:08.884229 31730 consensus_queue.cc:237] T 80a61882ef804a85a31e90d67930c5e9 P 05213ac836744a53b9c089a5c052f4be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:12:08.896158 30624 catalog_manager.cc:5697] T 80a61882ef804a85a31e90d67930c5e9 P 05213ac836744a53b9c089a5c052f4be reported cstate change: term changed from 1 to 2, leader changed from 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4) to 05213ac836744a53b9c089a5c052f4be (127.21.63.2). New cstate: current_term: 2 leader_uuid: "05213ac836744a53b9c089a5c052f4be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:08.986642 30948 raft_consensus.cc:1275] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Refusing update from remote peer 02c5705b45c04706bbb2c6ff3d7b0b8a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:08.987810 31706 consensus_queue.cc:1048] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:09.000905 30799 raft_consensus.cc:1275] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Refusing update from remote peer 02c5705b45c04706bbb2c6ff3d7b0b8a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:09.003144 31706 consensus_queue.cc:1048] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:09.162568 30795 raft_consensus.cc:1275] T a38ff14550f74527a62591df59fb2c51 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Refusing update from remote peer d22c9d3d15904327a78a0e548a8ff6cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:09.163944 31726 consensus_queue.cc:1048] T a38ff14550f74527a62591df59fb2c51 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:09.172474 30946 raft_consensus.cc:1275] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [term 3 FOLLOWER]: Refusing update from remote peer d22c9d3d15904327a78a0e548a8ff6cd: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:09.181075 30946 raft_consensus.cc:1275] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Refusing update from remote peer d22c9d3d15904327a78a0e548a8ff6cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:09.182471 31705 consensus_queue.cc:1048] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:09.183768 31705 consensus_queue.cc:1048] T a38ff14550f74527a62591df59fb2c51 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:09.205876 30795 raft_consensus.cc:1275] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 05213ac836744a53b9c089a5c052f4be [term 3 FOLLOWER]: Refusing update from remote peer d22c9d3d15904327a78a0e548a8ff6cd: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:09.207885 31707 consensus_queue.cc:1048] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:09.358546 30948 raft_consensus.cc:1275] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Refusing update from remote peer d22c9d3d15904327a78a0e548a8ff6cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:09.359606 31705 consensus_queue.cc:1048] T f63e902c7c7a4d0fa5d2107620ce851e P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:09.370980 30795 raft_consensus.cc:1275] T f63e902c7c7a4d0fa5d2107620ce851e P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Refusing update from remote peer d22c9d3d15904327a78a0e548a8ff6cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:09.372253 31707 consensus_queue.cc:1048] T f63e902c7c7a4d0fa5d2107620ce851e P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:09.473290 30946 raft_consensus.cc:1275] T 80a61882ef804a85a31e90d67930c5e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Refusing update from remote peer 05213ac836744a53b9c089a5c052f4be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:09.474556 31734 consensus_queue.cc:1048] T 80a61882ef804a85a31e90d67930c5e9 P 05213ac836744a53b9c089a5c052f4be [LEADER]: Connected to new peer: Peer: permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:09.485374 30875 raft_consensus.cc:1275] T 80a61882ef804a85a31e90d67930c5e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Refusing update from remote peer 05213ac836744a53b9c089a5c052f4be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:09.486968 31772 consensus_queue.cc:1048] T 80a61882ef804a85a31e90d67930c5e9 P 05213ac836744a53b9c089a5c052f4be [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:09.563714 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:09.607671 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:09.666175 30795 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "65776989d6674454816411c251aff5c1"
dest_uuid: "05213ac836744a53b9c089a5c052f4be"
mode: GRACEFUL
new_leader_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
 from {username='slave'} at 127.0.0.1:55170
I20260709 13:12:09.666791 30795 raft_consensus.cc:606] T 65776989d6674454816411c251aff5c1 P 05213ac836744a53b9c089a5c052f4be [term 1 LEADER]: Received request to transfer leadership to TS 02c5705b45c04706bbb2c6ff3d7b0b8a
I20260709 13:12:09.670861 30873 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c02f992ce0974544a4112672d8792c5f"
dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
mode: GRACEFUL
new_leader_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
 from {username='slave'} at 127.0.0.1:46350
I20260709 13:12:09.671397 30873 raft_consensus.cc:606] T c02f992ce0974544a4112672d8792c5f P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Received request to transfer leadership to TS 262942f67dfa4b8fa42234b18ac31ffd
I20260709 13:12:09.672916 21756 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:12:09.674574 21756 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:12:10.251019 31794 raft_consensus.cc:993] T 65776989d6674454816411c251aff5c1 P 05213ac836744a53b9c089a5c052f4be: : Instructing follower 02c5705b45c04706bbb2c6ff3d7b0b8a to start an election
I20260709 13:12:10.251652 31794 raft_consensus.cc:1081] T 65776989d6674454816411c251aff5c1 P 05213ac836744a53b9c089a5c052f4be [term 1 LEADER]: Signalling peer 02c5705b45c04706bbb2c6ff3d7b0b8a to start an election
I20260709 13:12:10.283238 30871 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "65776989d6674454816411c251aff5c1"
dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
 from {username='slave'} at 127.0.0.1:48618
I20260709 13:12:10.283918 30871 raft_consensus.cc:493] T 65776989d6674454816411c251aff5c1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:10.284297 30871 raft_consensus.cc:3060] T 65776989d6674454816411c251aff5c1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:12:10.328397 30912 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:43771, user_credentials={real_user=slave}} blocked reactor thread for 47746.7us
I20260709 13:12:10.381186 30871 raft_consensus.cc:515] T 65776989d6674454816411c251aff5c1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:12:10.383771 30871 leader_election.cc:290] T 65776989d6674454816411c251aff5c1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 election: Requested vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:12:10.384507 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65776989d6674454816411c251aff5c1" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:12:10.385130 30799 raft_consensus.cc:3055] T 65776989d6674454816411c251aff5c1 P 05213ac836744a53b9c089a5c052f4be [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:10.385437 30799 raft_consensus.cc:740] T 65776989d6674454816411c251aff5c1 P 05213ac836744a53b9c089a5c052f4be [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Running, Role: LEADER
I20260709 13:12:10.386282 30799 consensus_queue.cc:260] T 65776989d6674454816411c251aff5c1 P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:12:10.387662 30799 raft_consensus.cc:3060] T 65776989d6674454816411c251aff5c1 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:12:10.396725 30839 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.4:41313, user_credentials={real_user=slave}} blocked reactor thread for 100830us
I20260709 13:12:10.403214 30799 raft_consensus.cc:2468] T 65776989d6674454816411c251aff5c1 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 2.
I20260709 13:12:10.406234 30838 leader_election.cc:304] T 65776989d6674454816411c251aff5c1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 05213ac836744a53b9c089a5c052f4be; no voters: 
I20260709 13:12:10.407155 31803 raft_consensus.cc:2804] T 65776989d6674454816411c251aff5c1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:10.410074 31803 raft_consensus.cc:697] T 65776989d6674454816411c251aff5c1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 LEADER]: Becoming Leader. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Running, Role: LEADER
I20260709 13:12:10.410547 30941 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "65776989d6674454816411c251aff5c1" candidate_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:12:10.411864 30941 raft_consensus.cc:3060] T 65776989d6674454816411c251aff5c1 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:10.412295 31803 consensus_queue.cc:237] T 65776989d6674454816411c251aff5c1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:12:10.411079 31706 raft_consensus.cc:993] T c02f992ce0974544a4112672d8792c5f P 02c5705b45c04706bbb2c6ff3d7b0b8a: : Instructing follower 262942f67dfa4b8fa42234b18ac31ffd to start an election
I20260709 13:12:10.415881 31798 raft_consensus.cc:1081] T c02f992ce0974544a4112672d8792c5f P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Signalling peer 262942f67dfa4b8fa42234b18ac31ffd to start an election
I20260709 13:12:10.418002 30942 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c02f992ce0974544a4112672d8792c5f"
dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
 from {username='slave'} at 127.0.0.1:40958
I20260709 13:12:10.418601 30942 raft_consensus.cc:493] T c02f992ce0974544a4112672d8792c5f P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:10.418970 30942 raft_consensus.cc:3060] T c02f992ce0974544a4112672d8792c5f P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:10.427619 30941 raft_consensus.cc:2468] T 65776989d6674454816411c251aff5c1 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02c5705b45c04706bbb2c6ff3d7b0b8a in term 2.
I20260709 13:12:10.429152 30942 raft_consensus.cc:515] T c02f992ce0974544a4112672d8792c5f P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:12:10.432468 30942 leader_election.cc:290] T c02f992ce0974544a4112672d8792c5f P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 2 election: Requested vote from peers 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771), 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993)
I20260709 13:12:10.433588 30871 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c02f992ce0974544a4112672d8792c5f" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:12:10.434226 30871 raft_consensus.cc:3055] T c02f992ce0974544a4112672d8792c5f P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:10.434581 30871 raft_consensus.cc:740] T c02f992ce0974544a4112672d8792c5f P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Running, Role: LEADER
I20260709 13:12:10.435405 30871 consensus_queue.cc:260] T c02f992ce0974544a4112672d8792c5f P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:12:10.436760 30871 raft_consensus.cc:3060] T c02f992ce0974544a4112672d8792c5f P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:10.444598 30871 raft_consensus.cc:2468] T c02f992ce0974544a4112672d8792c5f P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 2.
I20260709 13:12:10.445997 30914 leader_election.cc:304] T c02f992ce0974544a4112672d8792c5f P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
I20260709 13:12:10.447451 31802 raft_consensus.cc:2804] T c02f992ce0974544a4112672d8792c5f P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:10.448431 31802 raft_consensus.cc:697] T c02f992ce0974544a4112672d8792c5f P 262942f67dfa4b8fa42234b18ac31ffd [term 2 LEADER]: Becoming Leader. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Running, Role: LEADER
I20260709 13:12:10.448586 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c02f992ce0974544a4112672d8792c5f" candidate_uuid: "262942f67dfa4b8fa42234b18ac31ffd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:12:10.449460 31802 consensus_queue.cc:237] T c02f992ce0974544a4112672d8792c5f P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:12:10.455484 30799 raft_consensus.cc:3060] T c02f992ce0974544a4112672d8792c5f P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:10.482093 30799 raft_consensus.cc:2468] T c02f992ce0974544a4112672d8792c5f P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 262942f67dfa4b8fa42234b18ac31ffd in term 2.
I20260709 13:12:10.491640 30623 catalog_manager.cc:5697] T c02f992ce0974544a4112672d8792c5f P 262942f67dfa4b8fa42234b18ac31ffd reported cstate change: term changed from 1 to 2, leader changed from 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3) to 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4). New cstate: current_term: 2 leader_uuid: "262942f67dfa4b8fa42234b18ac31ffd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:10.492900 30624 catalog_manager.cc:5697] T 65776989d6674454816411c251aff5c1 P 02c5705b45c04706bbb2c6ff3d7b0b8a reported cstate change: term changed from 1 to 2, leader changed from 05213ac836744a53b9c089a5c052f4be (127.21.63.2) to 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3). New cstate: current_term: 2 leader_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:10.869024 30799 raft_consensus.cc:1275] T 65776989d6674454816411c251aff5c1 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Refusing update from remote peer 02c5705b45c04706bbb2c6ff3d7b0b8a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:10.871533 31803 consensus_queue.cc:1048] T 65776989d6674454816411c251aff5c1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:10.876694 30794 raft_consensus.cc:1275] T c02f992ce0974544a4112672d8792c5f P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Refusing update from remote peer 262942f67dfa4b8fa42234b18ac31ffd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:10.878976 31802 consensus_queue.cc:1048] T c02f992ce0974544a4112672d8792c5f P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:10.910550 30941 raft_consensus.cc:1275] T 65776989d6674454816411c251aff5c1 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Refusing update from remote peer 02c5705b45c04706bbb2c6ff3d7b0b8a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:10.917294 31798 consensus_queue.cc:1048] T 65776989d6674454816411c251aff5c1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:10.973219 30871 raft_consensus.cc:1275] T c02f992ce0974544a4112672d8792c5f P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Refusing update from remote peer 262942f67dfa4b8fa42234b18ac31ffd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:10.975716 31682 consensus_queue.cc:1048] T c02f992ce0974544a4112672d8792c5f P 262942f67dfa4b8fa42234b18ac31ffd [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:10.979820 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:11.167956 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:11.208484 30874 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "11c7485c290740bba00c8b9eaf9b917f"
dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
mode: GRACEFUL
new_leader_uuid: "05213ac836744a53b9c089a5c052f4be"
 from {username='slave'} at 127.0.0.1:46350
I20260709 13:12:11.209410 30874 raft_consensus.cc:606] T 11c7485c290740bba00c8b9eaf9b917f P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Received request to transfer leadership to TS 05213ac836744a53b9c089a5c052f4be
I20260709 13:12:11.213791 30941 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7ed3eac558b34457aa26b1c40652f887"
dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
mode: GRACEFUL
new_leader_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
 from {username='slave'} at 127.0.0.1:33910
I20260709 13:12:11.214421 30941 raft_consensus.cc:606] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Received request to transfer leadership to TS d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:12:11.216005 21756 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:12:11.217202 21756 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
W20260709 13:12:11.353200 30970 debug-util.cc:398] Leaking SignalData structure 0x7b080015ad40 after lost signal to thread 21759
W20260709 13:12:11.354460 30970 debug-util.cc:398] Leaking SignalData structure 0x7b0800131580 after lost signal to thread 30658
W20260709 13:12:11.356089 30970 debug-util.cc:398] Leaking SignalData structure 0x7b08002a6260 after lost signal to thread 30825
W20260709 13:12:11.357952 30970 debug-util.cc:398] Leaking SignalData structure 0x7b08002b08a0 after lost signal to thread 30899
W20260709 13:12:11.361968 30970 debug-util.cc:398] Leaking SignalData structure 0x7b08000a69a0 after lost signal to thread 30973
W20260709 13:12:11.363863 30970 debug-util.cc:398] Leaking SignalData structure 0x7b08000dcfa0 after lost signal to thread 31561
I20260709 13:12:11.543032 31802 raft_consensus.cc:993] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd: : Instructing follower d22c9d3d15904327a78a0e548a8ff6cd to start an election
I20260709 13:12:11.543782 31802 raft_consensus.cc:1081] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Signalling peer d22c9d3d15904327a78a0e548a8ff6cd to start an election
I20260709 13:12:11.545713 31531 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7ed3eac558b34457aa26b1c40652f887"
dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
 from {username='slave'} at 127.0.0.1:34036
I20260709 13:12:11.546223 31531 raft_consensus.cc:493] T 7ed3eac558b34457aa26b1c40652f887 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:11.546536 31531 raft_consensus.cc:3060] T 7ed3eac558b34457aa26b1c40652f887 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:11.554767 31531 raft_consensus.cc:515] T 7ed3eac558b34457aa26b1c40652f887 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:12:11.556854 31531 leader_election.cc:290] T 7ed3eac558b34457aa26b1c40652f887 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 2 election: Requested vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:12:11.557873 30942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ed3eac558b34457aa26b1c40652f887" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:12:11.560531 30942 raft_consensus.cc:3055] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:11.561288 30942 raft_consensus.cc:740] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 262942f67dfa4b8fa42234b18ac31ffd, State: Running, Role: LEADER
I20260709 13:12:11.565150 30799 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ed3eac558b34457aa26b1c40652f887" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:12:11.564512 30942 consensus_queue.cc:260] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:12:11.565797 30799 raft_consensus.cc:3060] T 7ed3eac558b34457aa26b1c40652f887 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:11.566361 30942 raft_consensus.cc:3060] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:11.572264 30799 raft_consensus.cc:2468] T 7ed3eac558b34457aa26b1c40652f887 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 2.
I20260709 13:12:11.576192 31481 leader_election.cc:304] T 7ed3eac558b34457aa26b1c40652f887 P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:12:11.577108 31726 raft_consensus.cc:2804] T 7ed3eac558b34457aa26b1c40652f887 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:11.594708 30942 raft_consensus.cc:2468] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 2.
I20260709 13:12:11.597875 31726 raft_consensus.cc:697] T 7ed3eac558b34457aa26b1c40652f887 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 LEADER]: Becoming Leader. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Running, Role: LEADER
I20260709 13:12:11.599113 31726 consensus_queue.cc:237] T 7ed3eac558b34457aa26b1c40652f887 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:12:11.613761 30624 catalog_manager.cc:5697] T 7ed3eac558b34457aa26b1c40652f887 P d22c9d3d15904327a78a0e548a8ff6cd reported cstate change: term changed from 1 to 2, leader changed from 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4) to d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1). New cstate: current_term: 2 leader_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:11.716413 31821 raft_consensus.cc:993] T 11c7485c290740bba00c8b9eaf9b917f P 02c5705b45c04706bbb2c6ff3d7b0b8a: : Instructing follower 05213ac836744a53b9c089a5c052f4be to start an election
I20260709 13:12:11.716964 31803 raft_consensus.cc:1081] T 11c7485c290740bba00c8b9eaf9b917f P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Signalling peer 05213ac836744a53b9c089a5c052f4be to start an election
I20260709 13:12:11.718813 30794 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "11c7485c290740bba00c8b9eaf9b917f"
dest_uuid: "05213ac836744a53b9c089a5c052f4be"
 from {username='slave'} at 127.0.0.1:39594
I20260709 13:12:11.719442 30794 raft_consensus.cc:493] T 11c7485c290740bba00c8b9eaf9b917f P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:11.719807 30794 raft_consensus.cc:3060] T 11c7485c290740bba00c8b9eaf9b917f P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:11.726481 30794 raft_consensus.cc:515] T 11c7485c290740bba00c8b9eaf9b917f P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:12:11.729256 30874 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c7485c290740bba00c8b9eaf9b917f" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a"
I20260709 13:12:11.729820 30874 raft_consensus.cc:3055] T 11c7485c290740bba00c8b9eaf9b917f P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:11.730114 30874 raft_consensus.cc:740] T 11c7485c290740bba00c8b9eaf9b917f P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 02c5705b45c04706bbb2c6ff3d7b0b8a, State: Running, Role: LEADER
I20260709 13:12:11.730787 30794 leader_election.cc:290] T 11c7485c290740bba00c8b9eaf9b917f P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 2 election: Requested vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313)
I20260709 13:12:11.731769 30942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11c7485c290740bba00c8b9eaf9b917f" candidate_uuid: "05213ac836744a53b9c089a5c052f4be" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:12:11.732362 30942 raft_consensus.cc:3060] T 11c7485c290740bba00c8b9eaf9b917f P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:11.730998 30874 consensus_queue.cc:260] T 11c7485c290740bba00c8b9eaf9b917f P 02c5705b45c04706bbb2c6ff3d7b0b8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:12:11.733260 30874 raft_consensus.cc:3060] T 11c7485c290740bba00c8b9eaf9b917f P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:11.741159 30942 raft_consensus.cc:2468] T 11c7485c290740bba00c8b9eaf9b917f P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05213ac836744a53b9c089a5c052f4be in term 2.
I20260709 13:12:11.742600 30765 leader_election.cc:304] T 11c7485c290740bba00c8b9eaf9b917f P 05213ac836744a53b9c089a5c052f4be [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05213ac836744a53b9c089a5c052f4be, 262942f67dfa4b8fa42234b18ac31ffd; no voters: 
I20260709 13:12:11.743517 31734 raft_consensus.cc:2804] T 11c7485c290740bba00c8b9eaf9b917f P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:11.743959 31734 raft_consensus.cc:697] T 11c7485c290740bba00c8b9eaf9b917f P 05213ac836744a53b9c089a5c052f4be [term 2 LEADER]: Becoming Leader. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Running, Role: LEADER
I20260709 13:12:11.744804 31734 consensus_queue.cc:237] T 11c7485c290740bba00c8b9eaf9b917f P 05213ac836744a53b9c089a5c052f4be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } }
I20260709 13:12:11.745446 30874 raft_consensus.cc:2468] T 11c7485c290740bba00c8b9eaf9b917f P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 05213ac836744a53b9c089a5c052f4be in term 2.
I20260709 13:12:11.756109 30624 catalog_manager.cc:5697] T 11c7485c290740bba00c8b9eaf9b917f P 05213ac836744a53b9c089a5c052f4be reported cstate change: term changed from 1 to 2, leader changed from 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3) to 05213ac836744a53b9c089a5c052f4be (127.21.63.2). New cstate: current_term: 2 leader_uuid: "05213ac836744a53b9c089a5c052f4be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:12.081502 30941 raft_consensus.cc:1275] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Refusing update from remote peer d22c9d3d15904327a78a0e548a8ff6cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:12.084278 31726 consensus_queue.cc:1048] T 7ed3eac558b34457aa26b1c40652f887 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:12.117321 30794 raft_consensus.cc:1275] T 7ed3eac558b34457aa26b1c40652f887 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Refusing update from remote peer d22c9d3d15904327a78a0e548a8ff6cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:12.145489 31822 consensus_queue.cc:1048] T 7ed3eac558b34457aa26b1c40652f887 P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:12.184394 30942 raft_consensus.cc:1275] T 11c7485c290740bba00c8b9eaf9b917f P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Refusing update from remote peer 05213ac836744a53b9c089a5c052f4be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:12.186506 31831 consensus_queue.cc:1048] T 11c7485c290740bba00c8b9eaf9b917f P 05213ac836744a53b9c089a5c052f4be [LEADER]: Connected to new peer: Peer: permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:12.218381 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:12.241983 30871 raft_consensus.cc:1275] T 11c7485c290740bba00c8b9eaf9b917f P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Refusing update from remote peer 05213ac836744a53b9c089a5c052f4be: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:12.244455 31831 consensus_queue.cc:1048] T 11c7485c290740bba00c8b9eaf9b917f P 05213ac836744a53b9c089a5c052f4be [LEADER]: Connected to new peer: Peer: permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:12.372017 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:12.426108 30794 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6fe93136933f4d2880ec355e1bbbf90d"
dest_uuid: "05213ac836744a53b9c089a5c052f4be"
mode: GRACEFUL
new_leader_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
 from {username='slave'} at 127.0.0.1:55170
I20260709 13:12:12.426717 30794 raft_consensus.cc:606] T 6fe93136933f4d2880ec355e1bbbf90d P 05213ac836744a53b9c089a5c052f4be [term 1 LEADER]: Received request to transfer leadership to TS d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:12:12.428392 21756 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 13:12:12.430594 21756 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:12:12.772550 31734 raft_consensus.cc:993] T 6fe93136933f4d2880ec355e1bbbf90d P 05213ac836744a53b9c089a5c052f4be: : Instructing follower d22c9d3d15904327a78a0e548a8ff6cd to start an election
I20260709 13:12:12.773155 31831 raft_consensus.cc:1081] T 6fe93136933f4d2880ec355e1bbbf90d P 05213ac836744a53b9c089a5c052f4be [term 1 LEADER]: Signalling peer d22c9d3d15904327a78a0e548a8ff6cd to start an election
I20260709 13:12:12.775064 31531 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6fe93136933f4d2880ec355e1bbbf90d"
dest_uuid: "d22c9d3d15904327a78a0e548a8ff6cd"
 from {username='slave'} at 127.0.0.1:34026
I20260709 13:12:12.779006 31531 raft_consensus.cc:493] T 6fe93136933f4d2880ec355e1bbbf90d P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:12.779495 31531 raft_consensus.cc:3060] T 6fe93136933f4d2880ec355e1bbbf90d P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:12.791128 31531 raft_consensus.cc:515] T 6fe93136933f4d2880ec355e1bbbf90d P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:12:12.798681 30794 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fe93136933f4d2880ec355e1bbbf90d" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "05213ac836744a53b9c089a5c052f4be"
I20260709 13:12:12.798029 31531 leader_election.cc:290] T 6fe93136933f4d2880ec355e1bbbf90d P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 2 election: Requested vote from peers 262942f67dfa4b8fa42234b18ac31ffd (127.21.63.4:41313), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:12:12.799412 30794 raft_consensus.cc:3055] T 6fe93136933f4d2880ec355e1bbbf90d P 05213ac836744a53b9c089a5c052f4be [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:12.799808 30794 raft_consensus.cc:740] T 6fe93136933f4d2880ec355e1bbbf90d P 05213ac836744a53b9c089a5c052f4be [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 05213ac836744a53b9c089a5c052f4be, State: Running, Role: LEADER
I20260709 13:12:12.800650 30942 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fe93136933f4d2880ec355e1bbbf90d" candidate_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "262942f67dfa4b8fa42234b18ac31ffd"
I20260709 13:12:12.800755 30794 consensus_queue.cc:260] T 6fe93136933f4d2880ec355e1bbbf90d P 05213ac836744a53b9c089a5c052f4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:12:12.801630 30942 raft_consensus.cc:3060] T 6fe93136933f4d2880ec355e1bbbf90d P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:12.802769 30794 raft_consensus.cc:3060] T 6fe93136933f4d2880ec355e1bbbf90d P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:12.817875 30942 raft_consensus.cc:2468] T 6fe93136933f4d2880ec355e1bbbf90d P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 2.
I20260709 13:12:12.819150 31482 leader_election.cc:304] T 6fe93136933f4d2880ec355e1bbbf90d P d22c9d3d15904327a78a0e548a8ff6cd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd, d22c9d3d15904327a78a0e548a8ff6cd; no voters: 
I20260709 13:12:12.819847 31841 raft_consensus.cc:2804] T 6fe93136933f4d2880ec355e1bbbf90d P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:12.820005 30794 raft_consensus.cc:2468] T 6fe93136933f4d2880ec355e1bbbf90d P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d22c9d3d15904327a78a0e548a8ff6cd in term 2.
I20260709 13:12:12.820391 31841 raft_consensus.cc:697] T 6fe93136933f4d2880ec355e1bbbf90d P d22c9d3d15904327a78a0e548a8ff6cd [term 2 LEADER]: Becoming Leader. State: Replica: d22c9d3d15904327a78a0e548a8ff6cd, State: Running, Role: LEADER
I20260709 13:12:12.821529 31841 consensus_queue.cc:237] T 6fe93136933f4d2880ec355e1bbbf90d P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:12:12.831277 30623 catalog_manager.cc:5697] T 6fe93136933f4d2880ec355e1bbbf90d P d22c9d3d15904327a78a0e548a8ff6cd reported cstate change: term changed from 1 to 2, leader changed from 05213ac836744a53b9c089a5c052f4be (127.21.63.2) to d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1). New cstate: current_term: 2 leader_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:13.242305 30941 raft_consensus.cc:1275] T 6fe93136933f4d2880ec355e1bbbf90d P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Refusing update from remote peer d22c9d3d15904327a78a0e548a8ff6cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:13.245667 31841 consensus_queue.cc:1048] T 6fe93136933f4d2880ec355e1bbbf90d P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:13.246958 30794 raft_consensus.cc:1275] T 6fe93136933f4d2880ec355e1bbbf90d P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Refusing update from remote peer d22c9d3d15904327a78a0e548a8ff6cd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:13.248245 31841 consensus_queue.cc:1048] T 6fe93136933f4d2880ec355e1bbbf90d P d22c9d3d15904327a78a0e548a8ff6cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:13.431625 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:13.466157 21756 tablet_server.cc:179] TabletServer@127.21.63.1:37325 shutting down...
W20260709 13:12:13.488812 30841 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 69 similar messages]
W20260709 13:12:13.492451 30841 consensus_peers.cc:597] T ac85f05e63ed463681b3ec790b39c4d8 P 02c5705b45c04706bbb2c6ff3d7b0b8a -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:13.523801 30915 consensus_peers.cc:597] T 39980eee51514ba493958931f406a145 P 262942f67dfa4b8fa42234b18ac31ffd -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:13.525404 21756 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 13:12:13.527501 30841 consensus_peers.cc:597] T cae2b1cb8db54cbb812489cc1b4b1213 P 02c5705b45c04706bbb2c6ff3d7b0b8a -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:13.526460 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:13.531865 21756 raft_consensus.cc:2243] T 94220be3f55a4acbbd6bd5723b9f32e9 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:13.532400 21756 raft_consensus.cc:2272] T 94220be3f55a4acbbd6bd5723b9f32e9 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:13.533905 30841 consensus_peers.cc:597] T 437c20f3d53f43048eca57834248cc0d P 02c5705b45c04706bbb2c6ff3d7b0b8a -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:13.535583 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:13.536375 21756 raft_consensus.cc:2243] T 27f26e218737414893d5c4bfa4b2cd94 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:13.537115 21756 raft_consensus.cc:2272] T 27f26e218737414893d5c4bfa4b2cd94 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:13.537746 30841 consensus_peers.cc:597] T 93a75acc8893414180a15797033a2d5a P 02c5705b45c04706bbb2c6ff3d7b0b8a -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:13.545378 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:13.546341 21756 raft_consensus.cc:2243] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:12:13.547443 21756 raft_consensus.cc:2272] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P d22c9d3d15904327a78a0e548a8ff6cd [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:13.548869 30841 consensus_peers.cc:597] T d5175e0d898c453b8a7641d94dc1ac58 P 02c5705b45c04706bbb2c6ff3d7b0b8a -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:13.549016 30767 consensus_peers.cc:597] T 892368b6886842febffc849d73f76c10 P 05213ac836744a53b9c089a5c052f4be -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:13.550623 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:13.551683 21756 raft_consensus.cc:2243] T b3754a670e0b4ed99d37283bf1647bf5 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:13.552263 21756 raft_consensus.cc:2272] T b3754a670e0b4ed99d37283bf1647bf5 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:13.555124 30767 consensus_peers.cc:597] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 05213ac836744a53b9c089a5c052f4be -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:13.555753 30767 consensus_peers.cc:597] T 64a95fa1266848048486722c02f0d4cf P 05213ac836744a53b9c089a5c052f4be -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:13.557008 30767 consensus_peers.cc:597] T efddea050a344563877c3e01ee2a4c81 P 05213ac836744a53b9c089a5c052f4be -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:13.557914 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:13.558903 21756 raft_consensus.cc:2243] T d5175e0d898c453b8a7641d94dc1ac58 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:13.559401 21756 raft_consensus.cc:2272] T d5175e0d898c453b8a7641d94dc1ac58 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:13.563684 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:13.564570 21756 raft_consensus.cc:2243] T a38ff14550f74527a62591df59fb2c51 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:13.565680 21756 raft_consensus.cc:2272] T a38ff14550f74527a62591df59fb2c51 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:13.569244 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:13.570603 21756 raft_consensus.cc:2243] T d16c44f56c42466497ba4f7f1b2b5344 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:13.571159 21756 raft_consensus.cc:2272] T d16c44f56c42466497ba4f7f1b2b5344 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:13.577307 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:13.578174 21756 raft_consensus.cc:2243] T 8b3e41a728f1457ebf97274b95c7b23a P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:13.578841 21756 raft_consensus.cc:2272] T 8b3e41a728f1457ebf97274b95c7b23a P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:13.580940 30915 consensus_peers.cc:597] T d7edce5d08f24852b7afca9c57ba9c19 P 262942f67dfa4b8fa42234b18ac31ffd -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:13.586817 30841 consensus_peers.cc:597] T 8b3e41a728f1457ebf97274b95c7b23a P 02c5705b45c04706bbb2c6ff3d7b0b8a -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:13.588001 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:13.589257 21756 raft_consensus.cc:2243] T bbb2acd63baa48a3a2ace94016522e92 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:13.589805 21756 raft_consensus.cc:2272] T bbb2acd63baa48a3a2ace94016522e92 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:13.595170 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:13.596125 21756 raft_consensus.cc:2243] T 7ed3eac558b34457aa26b1c40652f887 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:13.597229 21756 raft_consensus.cc:2272] T 7ed3eac558b34457aa26b1c40652f887 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:13.600220 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:13.601049 21756 raft_consensus.cc:2243] T 39980eee51514ba493958931f406a145 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:13.601585 21756 raft_consensus.cc:2272] T 39980eee51514ba493958931f406a145 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:13.604449 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:13.605479 21756 raft_consensus.cc:2243] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:12:13.608484 21756 raft_consensus.cc:2272] T 0d8c1da1f7254fde94f7deaad5f41709 P d22c9d3d15904327a78a0e548a8ff6cd [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:13.611781 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:13.612682 21756 raft_consensus.cc:2243] T bc2a86f27b0f4fb59d7cfa586a23fef1 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:13.614374 21756 raft_consensus.cc:2272] T bc2a86f27b0f4fb59d7cfa586a23fef1 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:13.626339 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:13.627291 21756 raft_consensus.cc:2243] T 7b988f0495c34b7591ad2fb0d5f65693 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:13.627990 21756 raft_consensus.cc:2272] T 7b988f0495c34b7591ad2fb0d5f65693 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:13.633489 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:13.634338 21756 raft_consensus.cc:2243] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [term 4 LEADER]: Raft consensus shutting down.
I20260709 13:12:13.635468 21756 raft_consensus.cc:2272] T 1474b6fc151c4a3994a3aef89ea936a2 P d22c9d3d15904327a78a0e548a8ff6cd [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:13.641351 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:13.642277 21756 raft_consensus.cc:2243] T cae2b1cb8db54cbb812489cc1b4b1213 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:13.642875 21756 raft_consensus.cc:2272] T cae2b1cb8db54cbb812489cc1b4b1213 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:13.644193 30767 consensus_peers.cc:597] T 8b6052cddae64c5c9d16f60272735788 P 05213ac836744a53b9c089a5c052f4be -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:13.648912 30767 consensus_peers.cc:597] T bbb2acd63baa48a3a2ace94016522e92 P 05213ac836744a53b9c089a5c052f4be -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:13.649839 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:13.650861 21756 raft_consensus.cc:2243] T f63e902c7c7a4d0fa5d2107620ce851e P d22c9d3d15904327a78a0e548a8ff6cd [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:13.653585 21756 raft_consensus.cc:2272] T f63e902c7c7a4d0fa5d2107620ce851e P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:13.656908 21756 tablet_replica.cc:333] stopping tablet replica
W20260709 13:12:13.660225 30841 consensus_peers.cc:597] T bcac31d5b7ab48ec9c071a8f77750a88 P 02c5705b45c04706bbb2c6ff3d7b0b8a -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:13.672417 30841 consensus_peers.cc:597] T 784deee1ea9543df840c58ecaa6f911a P 02c5705b45c04706bbb2c6ff3d7b0b8a -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:13.677032 21756 raft_consensus.cc:2243] T 6fe93136933f4d2880ec355e1bbbf90d P d22c9d3d15904327a78a0e548a8ff6cd [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:13.678288 21756 raft_consensus.cc:2272] T 6fe93136933f4d2880ec355e1bbbf90d P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:13.679908 30841 consensus_peers.cc:597] T 94220be3f55a4acbbd6bd5723b9f32e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:13.683162 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:13.684082 21756 raft_consensus.cc:2243] T 64a95fa1266848048486722c02f0d4cf P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:13.684588 21756 raft_consensus.cc:2272] T 64a95fa1266848048486722c02f0d4cf P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:13.687923 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:13.688799 21756 raft_consensus.cc:2243] T 8155b3fc02b348e1adb290ecfe4592c5 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:13.689908 21756 raft_consensus.cc:2272] T 8155b3fc02b348e1adb290ecfe4592c5 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:13.692955 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:13.693625 21756 raft_consensus.cc:2243] T 437c20f3d53f43048eca57834248cc0d P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:13.694115 21756 raft_consensus.cc:2272] T 437c20f3d53f43048eca57834248cc0d P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:13.697234 21756 tablet_replica.cc:333] stopping tablet replica
W20260709 13:12:13.697816 30767 consensus_peers.cc:597] T e28f28c3f05f40e68531e20dd7c1a1a0 P 05213ac836744a53b9c089a5c052f4be -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:13.698071 21756 raft_consensus.cc:2243] T 784deee1ea9543df840c58ecaa6f911a P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:13.698650 21756 raft_consensus.cc:2272] T 784deee1ea9543df840c58ecaa6f911a P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:13.701350 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:13.702003 21756 raft_consensus.cc:2243] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:13.702591 21756 raft_consensus.cc:2272] T 8770fcfd98154b2095c2c9f74135593b P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:13.705902 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:13.706830 21756 raft_consensus.cc:2243] T 92a4b4a5a07442f49f518b479db8c94b P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:12:13.706974 30915 consensus_peers.cc:597] T 8770fcfd98154b2095c2c9f74135593b P 262942f67dfa4b8fa42234b18ac31ffd -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:13.707376 21756 raft_consensus.cc:2272] T 92a4b4a5a07442f49f518b479db8c94b P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:13.710341 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:13.710881 21756 raft_consensus.cc:2243] T 8b6052cddae64c5c9d16f60272735788 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:13.711200 21756 raft_consensus.cc:2272] T 8b6052cddae64c5c9d16f60272735788 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:13.714035 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:13.714784 21756 raft_consensus.cc:2243] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:13.715283 21756 raft_consensus.cc:2272] T e2376eaea6bc4f209a21a110370da7b2 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:13.716842 30915 consensus_peers.cc:597] T 7b988f0495c34b7591ad2fb0d5f65693 P 262942f67dfa4b8fa42234b18ac31ffd -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:13.718093 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:13.718766 21756 raft_consensus.cc:2243] T 26172e2c14cd4d178764ed13012c21f7 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:13.719143 21756 raft_consensus.cc:2272] T 26172e2c14cd4d178764ed13012c21f7 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:13.721776 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:13.722422 21756 raft_consensus.cc:2243] T c0100ec8bd1e4a45b3c0ea52281f9204 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:13.722934 21756 raft_consensus.cc:2272] T c0100ec8bd1e4a45b3c0ea52281f9204 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:13.725656 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:13.726418 21756 raft_consensus.cc:2243] T af695b428b974773ba476dce0d31c5a1 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:13.727511 21756 raft_consensus.cc:2272] T af695b428b974773ba476dce0d31c5a1 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:13.730402 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:13.730966 21756 raft_consensus.cc:2243] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:13.731447 21756 raft_consensus.cc:2272] T bcac31d5b7ab48ec9c071a8f77750a88 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:13.734826 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:13.735563 21756 raft_consensus.cc:2243] T ea563bd67a9849d18c9c3dfb21d58a93 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:13.736737 21756 raft_consensus.cc:2272] T ea563bd67a9849d18c9c3dfb21d58a93 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:13.739619 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:13.740854 21756 raft_consensus.cc:2243] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:12:13.741941 21756 raft_consensus.cc:2272] T 5028253a91e74a73afca005c77e8b565 P d22c9d3d15904327a78a0e548a8ff6cd [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:13.745178 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:13.746016 21756 raft_consensus.cc:2243] T 99e98c1324084d85b8a97aaf0c248df7 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:13.747236 21756 raft_consensus.cc:2272] T 99e98c1324084d85b8a97aaf0c248df7 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:13.749673 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:13.750232 21756 raft_consensus.cc:2243] T 892368b6886842febffc849d73f76c10 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:13.750627 21756 raft_consensus.cc:2272] T 892368b6886842febffc849d73f76c10 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:13.753125 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:13.753654 21756 raft_consensus.cc:2243] T d05da516ed9040c988b1fdfe2fdde9ed P d22c9d3d15904327a78a0e548a8ff6cd [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:13.754876 21756 raft_consensus.cc:2272] T d05da516ed9040c988b1fdfe2fdde9ed P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:13.764884 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:13.765764 21756 raft_consensus.cc:2243] T 93a75acc8893414180a15797033a2d5a P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:13.766254 21756 raft_consensus.cc:2272] T 93a75acc8893414180a15797033a2d5a P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:13.769361 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:13.770068 21756 raft_consensus.cc:2243] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:12:13.771078 21756 raft_consensus.cc:2272] T 01cceb88bca348958b59f3697979dff1 P d22c9d3d15904327a78a0e548a8ff6cd [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:13.773993 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:13.774580 21756 raft_consensus.cc:2243] T d7edce5d08f24852b7afca9c57ba9c19 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:13.775126 21756 raft_consensus.cc:2272] T d7edce5d08f24852b7afca9c57ba9c19 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:13.777889 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:13.778643 21756 raft_consensus.cc:2243] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:12:13.779703 21756 raft_consensus.cc:2272] T 726b808ed64848dd89da4340d4d891fb P d22c9d3d15904327a78a0e548a8ff6cd [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:13.782851 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:13.783402 21756 raft_consensus.cc:2243] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:13.783878 21756 raft_consensus.cc:2272] T efddea050a344563877c3e01ee2a4c81 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:13.788179 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:13.788936 21756 raft_consensus.cc:2243] T 8fb24a20af104a7486d0f1b2ae0e3843 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 13:12:13.789084 30915 consensus_peers.cc:597] T b3754a670e0b4ed99d37283bf1647bf5 P 262942f67dfa4b8fa42234b18ac31ffd -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:13.789471 21756 raft_consensus.cc:2272] T 8fb24a20af104a7486d0f1b2ae0e3843 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:13.792387 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:13.792999 21756 raft_consensus.cc:2243] T 5532e197e7de48f28c034ba1fc490e19 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:13.793470 21756 raft_consensus.cc:2272] T 5532e197e7de48f28c034ba1fc490e19 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:13.795572 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:13.796114 21756 raft_consensus.cc:2243] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:13.796654 21756 raft_consensus.cc:2272] T e28f28c3f05f40e68531e20dd7c1a1a0 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:13.799206 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:13.799831 21756 raft_consensus.cc:2243] T 0ac93952b93645b1ad1d7d96cb60c3e9 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:13.800420 21756 raft_consensus.cc:2272] T 0ac93952b93645b1ad1d7d96cb60c3e9 P d22c9d3d15904327a78a0e548a8ff6cd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:13.802663 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:13.803144 21756 raft_consensus.cc:2243] T ac85f05e63ed463681b3ec790b39c4d8 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:13.803567 21756 raft_consensus.cc:2272] T ac85f05e63ed463681b3ec790b39c4d8 P d22c9d3d15904327a78a0e548a8ff6cd [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:13.821677 30767 consensus_peers.cc:597] T 5532e197e7de48f28c034ba1fc490e19 P 05213ac836744a53b9c089a5c052f4be -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:13.841156 30915 consensus_peers.cc:597] T d16c44f56c42466497ba4f7f1b2b5344 P 262942f67dfa4b8fa42234b18ac31ffd -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:13.842469 30841 consensus_peers.cc:597] T 27f26e218737414893d5c4bfa4b2cd94 P 02c5705b45c04706bbb2c6ff3d7b0b8a -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:13.842780 30767 consensus_peers.cc:597] T af695b428b974773ba476dce0d31c5a1 P 05213ac836744a53b9c089a5c052f4be -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:13.847267 30767 consensus_peers.cc:597] T e2376eaea6bc4f209a21a110370da7b2 P 05213ac836744a53b9c089a5c052f4be -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:13.869243 30841 consensus_peers.cc:597] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:13.871421 30915 connection.cc:573] client connection to 127.21.63.1:37325 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260709 13:12:13.876870 30915 consensus_peers.cc:597] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:37325: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:13.895066 21756 tablet_server.cc:196] TabletServer@127.21.63.1:37325 shutdown complete.
W20260709 13:12:13.906922 30915 consensus_peers.cc:597] T 26172e2c14cd4d178764ed13012c21f7 P 262942f67dfa4b8fa42234b18ac31ffd -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:37325: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:13.912807 30915 consensus_peers.cc:597] T 8fb24a20af104a7486d0f1b2ae0e3843 P 262942f67dfa4b8fa42234b18ac31ffd -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:37325: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:14.018456 21756 tablet_server.cc:179] TabletServer@127.21.63.2:0 shutting down...
W20260709 13:12:14.046452 30912 consensus_peers.cc:597] T 8fb24a20af104a7486d0f1b2ae0e3843 P 262942f67dfa4b8fa42234b18ac31ffd -> Peer 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771): Couldn't send request to peer 05213ac836744a53b9c089a5c052f4be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:14.053056 30841 consensus_peers.cc:597] T 92a4b4a5a07442f49f518b479db8c94b P 02c5705b45c04706bbb2c6ff3d7b0b8a -> Peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Couldn't send request to peer d22c9d3d15904327a78a0e548a8ff6cd. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:37325: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:14.171798 30912 consensus_peers.cc:597] T b3754a670e0b4ed99d37283bf1647bf5 P 262942f67dfa4b8fa42234b18ac31ffd -> Peer 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771): Couldn't send request to peer 05213ac836744a53b9c089a5c052f4be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:14.172397 30912 consensus_peers.cc:597] T ccc68dd4c56548cba0624daafb5ae23c P 262942f67dfa4b8fa42234b18ac31ffd -> Peer 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771): Couldn't send request to peer 05213ac836744a53b9c089a5c052f4be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:14.175061 30912 consensus_peers.cc:597] T 283eb8311aeb4b5599dd844da98ee0b7 P 262942f67dfa4b8fa42234b18ac31ffd -> Peer 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771): Couldn't send request to peer 05213ac836744a53b9c089a5c052f4be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:14.183703 21756 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:12:14.185542 21756 tablet_replica.cc:333] T cae2b1cb8db54cbb812489cc1b4b1213 P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.186547 21756 raft_consensus.cc:2243] T cae2b1cb8db54cbb812489cc1b4b1213 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.189033 21756 raft_consensus.cc:2272] T cae2b1cb8db54cbb812489cc1b4b1213 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.192607 21756 tablet_replica.cc:333] T d16c44f56c42466497ba4f7f1b2b5344 P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.194820 21756 raft_consensus.cc:2243] T d16c44f56c42466497ba4f7f1b2b5344 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.195470 21756 raft_consensus.cc:2272] T d16c44f56c42466497ba4f7f1b2b5344 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.200001 21756 tablet_replica.cc:333] T 6fe93136933f4d2880ec355e1bbbf90d P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.201094 21756 raft_consensus.cc:2243] T 6fe93136933f4d2880ec355e1bbbf90d P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.201732 21756 raft_consensus.cc:2272] T 6fe93136933f4d2880ec355e1bbbf90d P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:14.201229 30838 consensus_peers.cc:597] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 02c5705b45c04706bbb2c6ff3d7b0b8a -> Peer 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771): Couldn't send request to peer 05213ac836744a53b9c089a5c052f4be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:14.206411 21756 tablet_replica.cc:333] T 80a61882ef804a85a31e90d67930c5e9 P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.207402 21756 raft_consensus.cc:2243] T 80a61882ef804a85a31e90d67930c5e9 P 05213ac836744a53b9c089a5c052f4be [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:12:14.208552 30912 consensus_peers.cc:597] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd -> Peer 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771): Couldn't send request to peer 05213ac836744a53b9c089a5c052f4be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:14.208561 21756 raft_consensus.cc:2272] T 80a61882ef804a85a31e90d67930c5e9 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.211848 21756 tablet_replica.cc:333] T 65776989d6674454816411c251aff5c1 P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.212805 21756 raft_consensus.cc:2243] T 65776989d6674454816411c251aff5c1 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.213367 21756 raft_consensus.cc:2272] T 65776989d6674454816411c251aff5c1 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.216248 21756 tablet_replica.cc:333] T c0100ec8bd1e4a45b3c0ea52281f9204 P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.217128 21756 raft_consensus.cc:2243] T c0100ec8bd1e4a45b3c0ea52281f9204 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.217643 21756 raft_consensus.cc:2272] T c0100ec8bd1e4a45b3c0ea52281f9204 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.220391 21756 tablet_replica.cc:333] T af695b428b974773ba476dce0d31c5a1 P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.221246 21756 raft_consensus.cc:2243] T af695b428b974773ba476dce0d31c5a1 P 05213ac836744a53b9c089a5c052f4be [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:14.222354 21756 raft_consensus.cc:2272] T af695b428b974773ba476dce0d31c5a1 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.225318 21756 tablet_replica.cc:333] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.226176 21756 raft_consensus.cc:2243] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 05213ac836744a53b9c089a5c052f4be [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:12:14.227150 30912 consensus_peers.cc:597] T d16c44f56c42466497ba4f7f1b2b5344 P 262942f67dfa4b8fa42234b18ac31ffd -> Peer 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771): Couldn't send request to peer 05213ac836744a53b9c089a5c052f4be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:14.227234 21756 raft_consensus.cc:2272] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.230788 21756 tablet_replica.cc:333] T 8155b3fc02b348e1adb290ecfe4592c5 P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.231643 21756 raft_consensus.cc:2243] T 8155b3fc02b348e1adb290ecfe4592c5 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:12:14.231901 30838 consensus_peers.cc:597] T cae2b1cb8db54cbb812489cc1b4b1213 P 02c5705b45c04706bbb2c6ff3d7b0b8a -> Peer 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771): Couldn't send request to peer 05213ac836744a53b9c089a5c052f4be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:14.232249 21756 raft_consensus.cc:2272] T 8155b3fc02b348e1adb290ecfe4592c5 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.235739 21756 tablet_replica.cc:333] T 8fb24a20af104a7486d0f1b2ae0e3843 P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.236606 21756 raft_consensus.cc:2243] T 8fb24a20af104a7486d0f1b2ae0e3843 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.237156 21756 raft_consensus.cc:2272] T 8fb24a20af104a7486d0f1b2ae0e3843 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.240226 21756 tablet_replica.cc:333] T 5532e197e7de48f28c034ba1fc490e19 P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.241230 21756 raft_consensus.cc:2243] T 5532e197e7de48f28c034ba1fc490e19 P 05213ac836744a53b9c089a5c052f4be [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:14.242409 21756 raft_consensus.cc:2272] T 5532e197e7de48f28c034ba1fc490e19 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.245776 21756 tablet_replica.cc:333] T c1efa319af774b23bd4ea782611f5252 P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.246718 21756 raft_consensus.cc:2243] T c1efa319af774b23bd4ea782611f5252 P 05213ac836744a53b9c089a5c052f4be [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:14.247923 21756 raft_consensus.cc:2272] T c1efa319af774b23bd4ea782611f5252 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.251140 21756 tablet_replica.cc:333] T f8f19b152a0f46ac896478f3f05c8822 P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.252080 21756 raft_consensus.cc:2243] T f8f19b152a0f46ac896478f3f05c8822 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.252737 21756 raft_consensus.cc:2272] T f8f19b152a0f46ac896478f3f05c8822 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.255872 21756 tablet_replica.cc:333] T 5028253a91e74a73afca005c77e8b565 P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.256791 21756 raft_consensus.cc:2243] T 5028253a91e74a73afca005c77e8b565 P 05213ac836744a53b9c089a5c052f4be [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.257472 21756 raft_consensus.cc:2272] T 5028253a91e74a73afca005c77e8b565 P 05213ac836744a53b9c089a5c052f4be [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.260562 21756 tablet_replica.cc:333] T ccc68dd4c56548cba0624daafb5ae23c P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.261343 21756 raft_consensus.cc:2243] T ccc68dd4c56548cba0624daafb5ae23c P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.261905 21756 raft_consensus.cc:2272] T ccc68dd4c56548cba0624daafb5ae23c P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.265103 21756 tablet_replica.cc:333] T 64a95fa1266848048486722c02f0d4cf P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.265879 21756 raft_consensus.cc:2243] T 64a95fa1266848048486722c02f0d4cf P 05213ac836744a53b9c089a5c052f4be [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:14.266911 21756 raft_consensus.cc:2272] T 64a95fa1266848048486722c02f0d4cf P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.269578 21756 tablet_replica.cc:333] T bbb2acd63baa48a3a2ace94016522e92 P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.270236 21756 raft_consensus.cc:2243] T bbb2acd63baa48a3a2ace94016522e92 P 05213ac836744a53b9c089a5c052f4be [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:14.271417 21756 raft_consensus.cc:2272] T bbb2acd63baa48a3a2ace94016522e92 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.274410 21756 tablet_replica.cc:333] T e2376eaea6bc4f209a21a110370da7b2 P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.275012 21756 raft_consensus.cc:2243] T e2376eaea6bc4f209a21a110370da7b2 P 05213ac836744a53b9c089a5c052f4be [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:14.276095 21756 raft_consensus.cc:2272] T e2376eaea6bc4f209a21a110370da7b2 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.278206 21756 tablet_replica.cc:333] T 8b6052cddae64c5c9d16f60272735788 P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.278748 21756 raft_consensus.cc:2243] T 8b6052cddae64c5c9d16f60272735788 P 05213ac836744a53b9c089a5c052f4be [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:14.279537 21756 raft_consensus.cc:2272] T 8b6052cddae64c5c9d16f60272735788 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.282066 21756 tablet_replica.cc:333] T 26172e2c14cd4d178764ed13012c21f7 P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.282605 21756 raft_consensus.cc:2243] T 26172e2c14cd4d178764ed13012c21f7 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.283097 21756 raft_consensus.cc:2272] T 26172e2c14cd4d178764ed13012c21f7 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.285144 21756 tablet_replica.cc:333] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.285651 21756 raft_consensus.cc:2243] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.286096 21756 raft_consensus.cc:2272] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.288396 21756 tablet_replica.cc:333] T 7b988f0495c34b7591ad2fb0d5f65693 P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.289151 21756 raft_consensus.cc:2243] T 7b988f0495c34b7591ad2fb0d5f65693 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.289691 21756 raft_consensus.cc:2272] T 7b988f0495c34b7591ad2fb0d5f65693 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.292410 21756 tablet_replica.cc:333] T 1474b6fc151c4a3994a3aef89ea936a2 P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.293246 21756 raft_consensus.cc:2243] T 1474b6fc151c4a3994a3aef89ea936a2 P 05213ac836744a53b9c089a5c052f4be [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.293761 21756 raft_consensus.cc:2272] T 1474b6fc151c4a3994a3aef89ea936a2 P 05213ac836744a53b9c089a5c052f4be [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.296505 21756 tablet_replica.cc:333] T d4dbe5b2d724412a909fd0641d1a213a P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.297307 21756 raft_consensus.cc:2243] T d4dbe5b2d724412a909fd0641d1a213a P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.297797 21756 raft_consensus.cc:2272] T d4dbe5b2d724412a909fd0641d1a213a P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:14.298295 30912 consensus_peers.cc:597] T c02f992ce0974544a4112672d8792c5f P 262942f67dfa4b8fa42234b18ac31ffd -> Peer 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771): Couldn't send request to peer 05213ac836744a53b9c089a5c052f4be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:14.300556 21756 tablet_replica.cc:333] T d7edce5d08f24852b7afca9c57ba9c19 P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.301393 21756 raft_consensus.cc:2243] T d7edce5d08f24852b7afca9c57ba9c19 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.301890 21756 raft_consensus.cc:2272] T d7edce5d08f24852b7afca9c57ba9c19 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.304543 21756 tablet_replica.cc:333] T b3754a670e0b4ed99d37283bf1647bf5 P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.305266 21756 raft_consensus.cc:2243] T b3754a670e0b4ed99d37283bf1647bf5 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.305756 21756 raft_consensus.cc:2272] T b3754a670e0b4ed99d37283bf1647bf5 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.308619 21756 tablet_replica.cc:333] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.309417 21756 raft_consensus.cc:2243] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 05213ac836744a53b9c089a5c052f4be [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.309954 21756 raft_consensus.cc:2272] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 05213ac836744a53b9c089a5c052f4be [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.312565 21756 tablet_replica.cc:333] T d5175e0d898c453b8a7641d94dc1ac58 P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.313318 21756 raft_consensus.cc:2243] T d5175e0d898c453b8a7641d94dc1ac58 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.313814 21756 raft_consensus.cc:2272] T d5175e0d898c453b8a7641d94dc1ac58 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.316103 21756 tablet_replica.cc:333] T 054fc96e257a4dc6b2ef90db5c1ad499 P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.316772 21756 raft_consensus.cc:2243] T 054fc96e257a4dc6b2ef90db5c1ad499 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.317296 21756 raft_consensus.cc:2272] T 054fc96e257a4dc6b2ef90db5c1ad499 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.319342 21756 tablet_replica.cc:333] T 892368b6886842febffc849d73f76c10 P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.319852 21756 raft_consensus.cc:2243] T 892368b6886842febffc849d73f76c10 P 05213ac836744a53b9c089a5c052f4be [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:14.320622 21756 raft_consensus.cc:2272] T 892368b6886842febffc849d73f76c10 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.322659 21756 tablet_replica.cc:333] T d05da516ed9040c988b1fdfe2fdde9ed P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.323207 21756 raft_consensus.cc:2243] T d05da516ed9040c988b1fdfe2fdde9ed P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.323603 21756 raft_consensus.cc:2272] T d05da516ed9040c988b1fdfe2fdde9ed P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.326073 21756 tablet_replica.cc:333] T 11c7485c290740bba00c8b9eaf9b917f P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.326617 21756 raft_consensus.cc:2243] T 11c7485c290740bba00c8b9eaf9b917f P 05213ac836744a53b9c089a5c052f4be [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:14.327333 21756 raft_consensus.cc:2272] T 11c7485c290740bba00c8b9eaf9b917f P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.329391 21756 tablet_replica.cc:333] T f63e902c7c7a4d0fa5d2107620ce851e P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.329931 21756 raft_consensus.cc:2243] T f63e902c7c7a4d0fa5d2107620ce851e P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.330391 21756 raft_consensus.cc:2272] T f63e902c7c7a4d0fa5d2107620ce851e P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.332309 21756 tablet_replica.cc:333] T c02f992ce0974544a4112672d8792c5f P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.332942 21756 raft_consensus.cc:2243] T c02f992ce0974544a4112672d8792c5f P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.333351 21756 raft_consensus.cc:2272] T c02f992ce0974544a4112672d8792c5f P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.335242 21756 tablet_replica.cc:333] T 283eb8311aeb4b5599dd844da98ee0b7 P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.335775 21756 raft_consensus.cc:2243] T 283eb8311aeb4b5599dd844da98ee0b7 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.336194 21756 raft_consensus.cc:2272] T 283eb8311aeb4b5599dd844da98ee0b7 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.338294 21756 tablet_replica.cc:333] T efddea050a344563877c3e01ee2a4c81 P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.338925 21756 raft_consensus.cc:2243] T efddea050a344563877c3e01ee2a4c81 P 05213ac836744a53b9c089a5c052f4be [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:14.339699 21756 raft_consensus.cc:2272] T efddea050a344563877c3e01ee2a4c81 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.341805 21756 tablet_replica.cc:333] T 0d8c1da1f7254fde94f7deaad5f41709 P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.342337 21756 raft_consensus.cc:2243] T 0d8c1da1f7254fde94f7deaad5f41709 P 05213ac836744a53b9c089a5c052f4be [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.342818 21756 raft_consensus.cc:2272] T 0d8c1da1f7254fde94f7deaad5f41709 P 05213ac836744a53b9c089a5c052f4be [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.345005 21756 tablet_replica.cc:333] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.345521 21756 raft_consensus.cc:2243] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 05213ac836744a53b9c089a5c052f4be [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:14.346309 21756 raft_consensus.cc:2272] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.348575 21756 tablet_replica.cc:333] T e28f28c3f05f40e68531e20dd7c1a1a0 P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.349191 21756 raft_consensus.cc:2243] T e28f28c3f05f40e68531e20dd7c1a1a0 P 05213ac836744a53b9c089a5c052f4be [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:14.350113 21756 raft_consensus.cc:2272] T e28f28c3f05f40e68531e20dd7c1a1a0 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.352572 21756 tablet_replica.cc:333] T 39980eee51514ba493958931f406a145 P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.353163 21756 raft_consensus.cc:2243] T 39980eee51514ba493958931f406a145 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.353724 21756 raft_consensus.cc:2272] T 39980eee51514ba493958931f406a145 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.356083 21756 tablet_replica.cc:333] T b226c44412c74106a0feb2ca411b1661 P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.356850 21756 raft_consensus.cc:2243] T b226c44412c74106a0feb2ca411b1661 P 05213ac836744a53b9c089a5c052f4be [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:14.357930 21756 raft_consensus.cc:2272] T b226c44412c74106a0feb2ca411b1661 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.361150 21756 tablet_replica.cc:333] T 7ed3eac558b34457aa26b1c40652f887 P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.361855 21756 raft_consensus.cc:2243] T 7ed3eac558b34457aa26b1c40652f887 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.362383 21756 raft_consensus.cc:2272] T 7ed3eac558b34457aa26b1c40652f887 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.365237 21756 tablet_replica.cc:333] T b9fcfade988746a2967e115dbbcfe0f1 P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.365933 21756 raft_consensus.cc:2243] T b9fcfade988746a2967e115dbbcfe0f1 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.366489 21756 raft_consensus.cc:2272] T b9fcfade988746a2967e115dbbcfe0f1 P 05213ac836744a53b9c089a5c052f4be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.369699 21756 tablet_replica.cc:333] T a38ff14550f74527a62591df59fb2c51 P 05213ac836744a53b9c089a5c052f4be: stopping tablet replica
I20260709 13:12:14.370352 21756 raft_consensus.cc:2243] T a38ff14550f74527a62591df59fb2c51 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.370918 21756 raft_consensus.cc:2272] T a38ff14550f74527a62591df59fb2c51 P 05213ac836744a53b9c089a5c052f4be [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:14.389295 30838 consensus_peers.cc:597] T 65776989d6674454816411c251aff5c1 P 02c5705b45c04706bbb2c6ff3d7b0b8a -> Peer 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771): Couldn't send request to peer 05213ac836744a53b9c089a5c052f4be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:14.397372 30838 consensus_peers.cc:597] T d5175e0d898c453b8a7641d94dc1ac58 P 02c5705b45c04706bbb2c6ff3d7b0b8a -> Peer 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771): Couldn't send request to peer 05213ac836744a53b9c089a5c052f4be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:14.415257 30912 consensus_peers.cc:597] T b9fcfade988746a2967e115dbbcfe0f1 P 262942f67dfa4b8fa42234b18ac31ffd -> Peer 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771): Couldn't send request to peer 05213ac836744a53b9c089a5c052f4be. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:14.441552 30912 consensus_peers.cc:597] T d4dbe5b2d724412a909fd0641d1a213a P 262942f67dfa4b8fa42234b18ac31ffd -> Peer 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771): Couldn't send request to peer 05213ac836744a53b9c089a5c052f4be. Status: Network error: recv got EOF from unknown peer (error 108). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:14.462636 21756 tablet_server.cc:196] TabletServer@127.21.63.2:0 shutdown complete.
W20260709 13:12:14.523527 30912 consensus_peers.cc:597] T f8f19b152a0f46ac896478f3f05c8822 P 262942f67dfa4b8fa42234b18ac31ffd -> Peer 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771): Couldn't send request to peer 05213ac836744a53b9c089a5c052f4be. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:43771: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:14.530776 30838 consensus_peers.cc:597] T 054fc96e257a4dc6b2ef90db5c1ad499 P 02c5705b45c04706bbb2c6ff3d7b0b8a -> Peer 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771): Couldn't send request to peer 05213ac836744a53b9c089a5c052f4be. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:43771: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:14.551841 30912 consensus_peers.cc:597] T 26172e2c14cd4d178764ed13012c21f7 P 262942f67dfa4b8fa42234b18ac31ffd -> Peer 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771): Couldn't send request to peer 05213ac836744a53b9c089a5c052f4be. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:43771: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:14.560101 30912 consensus_peers.cc:597] T 39980eee51514ba493958931f406a145 P 262942f67dfa4b8fa42234b18ac31ffd -> Peer 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771): Couldn't send request to peer 05213ac836744a53b9c089a5c052f4be. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:43771: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:14.585877 21756 tablet_server.cc:179] TabletServer@127.21.63.3:0 shutting down...
W20260709 13:12:14.594662 30912 consensus_peers.cc:597] T d7edce5d08f24852b7afca9c57ba9c19 P 262942f67dfa4b8fa42234b18ac31ffd -> Peer 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771): Couldn't send request to peer 05213ac836744a53b9c089a5c052f4be. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:43771: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:14.600137 30912 consensus_peers.cc:597] T 7b988f0495c34b7591ad2fb0d5f65693 P 262942f67dfa4b8fa42234b18ac31ffd -> Peer 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771): Couldn't send request to peer 05213ac836744a53b9c089a5c052f4be. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:43771: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:14.631910 30914 consensus_peers.cc:597] T 8770fcfd98154b2095c2c9f74135593b P 262942f67dfa4b8fa42234b18ac31ffd -> Peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Couldn't send request to peer 02c5705b45c04706bbb2c6ff3d7b0b8a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:14.633744 30914 consensus_peers.cc:597] T b9fcfade988746a2967e115dbbcfe0f1 P 262942f67dfa4b8fa42234b18ac31ffd -> Peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Couldn't send request to peer 02c5705b45c04706bbb2c6ff3d7b0b8a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:14.643814 30914 consensus_peers.cc:597] T 283eb8311aeb4b5599dd844da98ee0b7 P 262942f67dfa4b8fa42234b18ac31ffd -> Peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Couldn't send request to peer 02c5705b45c04706bbb2c6ff3d7b0b8a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:14.651491 31856 raft_consensus.cc:493] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:12:14.652056 31856 raft_consensus.cc:515] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } } peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } }
I20260709 13:12:14.654925 21756 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:12:14.655212 31856 leader_election.cc:290] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:12:14.659024 21756 tablet_replica.cc:333] T cae2b1cb8db54cbb812489cc1b4b1213 P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.660156 21756 raft_consensus.cc:2243] T cae2b1cb8db54cbb812489cc1b4b1213 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:14.661347 21756 raft_consensus.cc:2272] T cae2b1cb8db54cbb812489cc1b4b1213 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.664765 21756 tablet_replica.cc:333] T e2376eaea6bc4f209a21a110370da7b2 P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.665710 21756 raft_consensus.cc:2243] T e2376eaea6bc4f209a21a110370da7b2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.666322 21756 raft_consensus.cc:2272] T e2376eaea6bc4f209a21a110370da7b2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:14.666566 30838 leader_election.cc:336] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771): Network error: Client connection negotiation failed: client connection to 127.21.63.2:43771: connect: Connection refused (error 111)
I20260709 13:12:14.669960 21756 tablet_replica.cc:333] T 8b3e41a728f1457ebf97274b95c7b23a P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.671010 21756 raft_consensus.cc:2243] T 8b3e41a728f1457ebf97274b95c7b23a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:14.672205 21756 raft_consensus.cc:2272] T 8b3e41a728f1457ebf97274b95c7b23a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.675334 21756 tablet_replica.cc:333] T 94220be3f55a4acbbd6bd5723b9f32e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.676282 21756 raft_consensus.cc:2243] T 94220be3f55a4acbbd6bd5723b9f32e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:12:14.680575 30841 leader_election.cc:336] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Network error: Client connection negotiation failed: client connection to 127.21.63.1:37325: connect: Connection refused (error 111)
I20260709 13:12:14.681160 30841 leader_election.cc:304] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 02c5705b45c04706bbb2c6ff3d7b0b8a; no voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:12:14.681941 31856 raft_consensus.cc:2749] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:14.677405 21756 raft_consensus.cc:2272] T 94220be3f55a4acbbd6bd5723b9f32e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.685582 21756 tablet_replica.cc:333] T af695b428b974773ba476dce0d31c5a1 P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.686688 21756 raft_consensus.cc:2243] T af695b428b974773ba476dce0d31c5a1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.687323 21756 raft_consensus.cc:2272] T af695b428b974773ba476dce0d31c5a1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.690313 21756 tablet_replica.cc:333] T 65776989d6674454816411c251aff5c1 P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.691345 21756 raft_consensus.cc:2243] T 65776989d6674454816411c251aff5c1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:14.692430 21756 raft_consensus.cc:2272] T 65776989d6674454816411c251aff5c1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.695412 21756 tablet_replica.cc:333] T b9fcfade988746a2967e115dbbcfe0f1 P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.696291 21756 raft_consensus.cc:2243] T b9fcfade988746a2967e115dbbcfe0f1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.696980 21756 raft_consensus.cc:2272] T b9fcfade988746a2967e115dbbcfe0f1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.700727 21756 tablet_replica.cc:333] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.701596 21756 raft_consensus.cc:2243] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.702173 21756 raft_consensus.cc:2272] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.705828 21756 tablet_replica.cc:333] T 8155b3fc02b348e1adb290ecfe4592c5 P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.706634 21756 raft_consensus.cc:2243] T 8155b3fc02b348e1adb290ecfe4592c5 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.707259 21756 raft_consensus.cc:2272] T 8155b3fc02b348e1adb290ecfe4592c5 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.710278 21756 tablet_replica.cc:333] T 5532e197e7de48f28c034ba1fc490e19 P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.711201 21756 raft_consensus.cc:2243] T 5532e197e7de48f28c034ba1fc490e19 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.711879 21756 raft_consensus.cc:2272] T 5532e197e7de48f28c034ba1fc490e19 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.714579 21756 tablet_replica.cc:333] T 92a4b4a5a07442f49f518b479db8c94b P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.715351 21756 raft_consensus.cc:2243] T 92a4b4a5a07442f49f518b479db8c94b P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:14.716372 21756 raft_consensus.cc:2272] T 92a4b4a5a07442f49f518b479db8c94b P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.719202 21756 tablet_replica.cc:333] T 892368b6886842febffc849d73f76c10 P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.719938 21756 raft_consensus.cc:2243] T 892368b6886842febffc849d73f76c10 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.720472 21756 raft_consensus.cc:2272] T 892368b6886842febffc849d73f76c10 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.723163 21756 tablet_replica.cc:333] T d05da516ed9040c988b1fdfe2fdde9ed P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.723868 21756 raft_consensus.cc:2243] T d05da516ed9040c988b1fdfe2fdde9ed P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.724443 21756 raft_consensus.cc:2272] T d05da516ed9040c988b1fdfe2fdde9ed P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.727157 21756 tablet_replica.cc:333] T 93a75acc8893414180a15797033a2d5a P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.727864 21756 raft_consensus.cc:2243] T 93a75acc8893414180a15797033a2d5a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:14.728922 21756 raft_consensus.cc:2272] T 93a75acc8893414180a15797033a2d5a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.731783 21756 tablet_replica.cc:333] T 01cceb88bca348958b59f3697979dff1 P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.732622 21756 raft_consensus.cc:2243] T 01cceb88bca348958b59f3697979dff1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.733210 21756 raft_consensus.cc:2272] T 01cceb88bca348958b59f3697979dff1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.735831 21756 tablet_replica.cc:333] T 726b808ed64848dd89da4340d4d891fb P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.736550 21756 raft_consensus.cc:2243] T 726b808ed64848dd89da4340d4d891fb P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.737116 21756 raft_consensus.cc:2272] T 726b808ed64848dd89da4340d4d891fb P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.739830 21756 tablet_replica.cc:333] T 99e98c1324084d85b8a97aaf0c248df7 P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.740553 21756 raft_consensus.cc:2243] T 99e98c1324084d85b8a97aaf0c248df7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.741218 21756 raft_consensus.cc:2272] T 99e98c1324084d85b8a97aaf0c248df7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.743997 21756 tablet_replica.cc:333] T f8f19b152a0f46ac896478f3f05c8822 P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.744733 21756 raft_consensus.cc:2243] T f8f19b152a0f46ac896478f3f05c8822 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.745231 21756 raft_consensus.cc:2272] T f8f19b152a0f46ac896478f3f05c8822 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.747833 21756 tablet_replica.cc:333] T 5028253a91e74a73afca005c77e8b565 P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.748546 21756 raft_consensus.cc:2243] T 5028253a91e74a73afca005c77e8b565 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.749114 21756 raft_consensus.cc:2272] T 5028253a91e74a73afca005c77e8b565 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.751839 21756 tablet_replica.cc:333] T c1efa319af774b23bd4ea782611f5252 P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.752602 21756 raft_consensus.cc:2243] T c1efa319af774b23bd4ea782611f5252 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.753144 21756 raft_consensus.cc:2272] T c1efa319af774b23bd4ea782611f5252 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.755686 21756 tablet_replica.cc:333] T 11c7485c290740bba00c8b9eaf9b917f P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.756440 21756 raft_consensus.cc:2243] T 11c7485c290740bba00c8b9eaf9b917f P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.756925 21756 raft_consensus.cc:2272] T 11c7485c290740bba00c8b9eaf9b917f P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.759369 21756 tablet_replica.cc:333] T 8770fcfd98154b2095c2c9f74135593b P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.760129 21756 raft_consensus.cc:2243] T 8770fcfd98154b2095c2c9f74135593b P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.760627 21756 raft_consensus.cc:2272] T 8770fcfd98154b2095c2c9f74135593b P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:14.761624 30914 consensus_peers.cc:597] T ccc68dd4c56548cba0624daafb5ae23c P 262942f67dfa4b8fa42234b18ac31ffd -> Peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Couldn't send request to peer 02c5705b45c04706bbb2c6ff3d7b0b8a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:14.763123 21756 tablet_replica.cc:333] T d5175e0d898c453b8a7641d94dc1ac58 P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.763893 21756 raft_consensus.cc:2243] T d5175e0d898c453b8a7641d94dc1ac58 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:14.764804 21756 raft_consensus.cc:2272] T d5175e0d898c453b8a7641d94dc1ac58 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.767361 21756 tablet_replica.cc:333] T 054fc96e257a4dc6b2ef90db5c1ad499 P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.768088 21756 raft_consensus.cc:2243] T 054fc96e257a4dc6b2ef90db5c1ad499 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:14.769122 21756 raft_consensus.cc:2272] T 054fc96e257a4dc6b2ef90db5c1ad499 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.771854 21756 tablet_replica.cc:333] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.771929 31865 raft_consensus.cc:493] T 6fe93136933f4d2880ec355e1bbbf90d P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:12:14.772718 21756 raft_consensus.cc:2243] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:14.772423 31865 raft_consensus.cc:515] T 6fe93136933f4d2880ec355e1bbbf90d P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:12:14.773751 21756 raft_consensus.cc:2272] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.774533 31865 leader_election.cc:290] T 6fe93136933f4d2880ec355e1bbbf90d P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:12:14.776439 21756 tablet_replica.cc:333] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.777268 21756 raft_consensus.cc:2243] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.777833 21756 raft_consensus.cc:2272] T 1474b6fc151c4a3994a3aef89ea936a2 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.780331 21756 tablet_replica.cc:333] T d4dbe5b2d724412a909fd0641d1a213a P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.781067 21756 raft_consensus.cc:2243] T d4dbe5b2d724412a909fd0641d1a213a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.781523 21756 raft_consensus.cc:2272] T d4dbe5b2d724412a909fd0641d1a213a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.784224 21756 tablet_replica.cc:333] T 80a61882ef804a85a31e90d67930c5e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.784981 21756 raft_consensus.cc:2243] T 80a61882ef804a85a31e90d67930c5e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.785566 21756 raft_consensus.cc:2272] T 80a61882ef804a85a31e90d67930c5e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.788448 21756 tablet_replica.cc:333] T 27f26e218737414893d5c4bfa4b2cd94 P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.789284 21756 raft_consensus.cc:2243] T 27f26e218737414893d5c4bfa4b2cd94 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 LEADER]: Raft consensus shutting down.
W20260709 13:12:14.790113 30912 leader_election.cc:336] T 6fe93136933f4d2880ec355e1bbbf90d P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771): Network error: Client connection negotiation failed: client connection to 127.21.63.2:43771: connect: Connection refused (error 111)
W20260709 13:12:14.790958 30915 leader_election.cc:336] T 6fe93136933f4d2880ec355e1bbbf90d P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Network error: Client connection negotiation failed: client connection to 127.21.63.1:37325: connect: Connection refused (error 111)
I20260709 13:12:14.791361 30915 leader_election.cc:304] T 6fe93136933f4d2880ec355e1bbbf90d P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd; no voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:12:14.792032 31682 raft_consensus.cc:2749] T 6fe93136933f4d2880ec355e1bbbf90d P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:12:14.790253 21756 raft_consensus.cc:2272] T 27f26e218737414893d5c4bfa4b2cd94 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.795112 21756 tablet_replica.cc:333] T 64a95fa1266848048486722c02f0d4cf P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.795796 21756 raft_consensus.cc:2243] T 64a95fa1266848048486722c02f0d4cf P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.796281 21756 raft_consensus.cc:2272] T 64a95fa1266848048486722c02f0d4cf P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.798373 21756 tablet_replica.cc:333] T ac85f05e63ed463681b3ec790b39c4d8 P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.798884 21756 raft_consensus.cc:2243] T ac85f05e63ed463681b3ec790b39c4d8 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:14.799697 21756 raft_consensus.cc:2272] T ac85f05e63ed463681b3ec790b39c4d8 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.802469 21756 tablet_replica.cc:333] T ccc68dd4c56548cba0624daafb5ae23c P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.803298 21756 raft_consensus.cc:2243] T ccc68dd4c56548cba0624daafb5ae23c P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.803815 21756 raft_consensus.cc:2272] T ccc68dd4c56548cba0624daafb5ae23c P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.806543 21756 tablet_replica.cc:333] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.807307 21756 raft_consensus.cc:2243] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:14.808254 21756 raft_consensus.cc:2272] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.810945 21756 tablet_replica.cc:333] T c02f992ce0974544a4112672d8792c5f P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.811692 21756 raft_consensus.cc:2243] T c02f992ce0974544a4112672d8792c5f P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.812217 21756 raft_consensus.cc:2272] T c02f992ce0974544a4112672d8792c5f P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.814296 21756 tablet_replica.cc:333] T 283eb8311aeb4b5599dd844da98ee0b7 P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.814987 21756 raft_consensus.cc:2243] T 283eb8311aeb4b5599dd844da98ee0b7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.815389 21756 raft_consensus.cc:2272] T 283eb8311aeb4b5599dd844da98ee0b7 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.817433 21756 tablet_replica.cc:333] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.817960 21756 raft_consensus.cc:2243] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.818373 21756 raft_consensus.cc:2272] T bc2a86f27b0f4fb59d7cfa586a23fef1 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.820331 21756 tablet_replica.cc:333] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.820881 21756 raft_consensus.cc:2243] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.821318 21756 raft_consensus.cc:2272] T 0d8c1da1f7254fde94f7deaad5f41709 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.823133 21756 tablet_replica.cc:333] T 784deee1ea9543df840c58ecaa6f911a P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.823637 21756 raft_consensus.cc:2243] T 784deee1ea9543df840c58ecaa6f911a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:14.824338 21756 raft_consensus.cc:2272] T 784deee1ea9543df840c58ecaa6f911a P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.826151 21756 tablet_replica.cc:333] T 437c20f3d53f43048eca57834248cc0d P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.826630 21756 raft_consensus.cc:2243] T 437c20f3d53f43048eca57834248cc0d P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:14.827396 21756 raft_consensus.cc:2272] T 437c20f3d53f43048eca57834248cc0d P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.829486 21756 tablet_replica.cc:333] T efddea050a344563877c3e01ee2a4c81 P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.829979 21756 raft_consensus.cc:2243] T efddea050a344563877c3e01ee2a4c81 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.830384 21756 raft_consensus.cc:2272] T efddea050a344563877c3e01ee2a4c81 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.832239 21756 tablet_replica.cc:333] T ea563bd67a9849d18c9c3dfb21d58a93 P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.832769 21756 raft_consensus.cc:2243] T ea563bd67a9849d18c9c3dfb21d58a93 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.833187 21756 raft_consensus.cc:2272] T ea563bd67a9849d18c9c3dfb21d58a93 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.835214 21756 tablet_replica.cc:333] T bcac31d5b7ab48ec9c071a8f77750a88 P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.835690 21756 raft_consensus.cc:2243] T bcac31d5b7ab48ec9c071a8f77750a88 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:14.836413 21756 raft_consensus.cc:2272] T bcac31d5b7ab48ec9c071a8f77750a88 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.838260 21756 tablet_replica.cc:333] T e28f28c3f05f40e68531e20dd7c1a1a0 P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.838711 21756 raft_consensus.cc:2243] T e28f28c3f05f40e68531e20dd7c1a1a0 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.839119 21756 raft_consensus.cc:2272] T e28f28c3f05f40e68531e20dd7c1a1a0 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:14.841068 21756 tablet_replica.cc:333] T b226c44412c74106a0feb2ca411b1661 P 02c5705b45c04706bbb2c6ff3d7b0b8a: stopping tablet replica
I20260709 13:12:14.841495 21756 raft_consensus.cc:2243] T b226c44412c74106a0feb2ca411b1661 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:14.841925 21756 raft_consensus.cc:2272] T b226c44412c74106a0feb2ca411b1661 P 02c5705b45c04706bbb2c6ff3d7b0b8a [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:14.858292 30914 consensus_peers.cc:597] T d4dbe5b2d724412a909fd0641d1a213a P 262942f67dfa4b8fa42234b18ac31ffd -> Peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Couldn't send request to peer 02c5705b45c04706bbb2c6ff3d7b0b8a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:14.861580 31861 raft_consensus.cc:493] T 726b808ed64848dd89da4340d4d891fb P 262942f67dfa4b8fa42234b18ac31ffd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:12:14.862231 31861 raft_consensus.cc:515] T 726b808ed64848dd89da4340d4d891fb P 262942f67dfa4b8fa42234b18ac31ffd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:12:14.864491 31861 leader_election.cc:290] T 726b808ed64848dd89da4340d4d891fb P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
W20260709 13:12:14.871883 30915 leader_election.cc:336] T 726b808ed64848dd89da4340d4d891fb P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Network error: Client connection negotiation failed: client connection to 127.21.63.1:37325: connect: Connection refused (error 111)
W20260709 13:12:14.876535 30914 leader_election.cc:336] T 726b808ed64848dd89da4340d4d891fb P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:14.878865 30914 leader_election.cc:304] T 726b808ed64848dd89da4340d4d891fb P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:12:14.879655 31861 raft_consensus.cc:2749] T 726b808ed64848dd89da4340d4d891fb P 262942f67dfa4b8fa42234b18ac31ffd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:14.901064 31861 raft_consensus.cc:493] T 99e98c1324084d85b8a97aaf0c248df7 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:12:14.901607 31861 raft_consensus.cc:515] T 99e98c1324084d85b8a97aaf0c248df7 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:12:14.903838 31861 leader_election.cc:290] T 99e98c1324084d85b8a97aaf0c248df7 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
W20260709 13:12:14.907080 30914 leader_election.cc:336] T 99e98c1324084d85b8a97aaf0c248df7 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 13:12:14.909356 30915 leader_election.cc:336] T 99e98c1324084d85b8a97aaf0c248df7 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Network error: Client connection negotiation failed: client connection to 127.21.63.1:37325: connect: Connection refused (error 111)
I20260709 13:12:14.909852 30915 leader_election.cc:304] T 99e98c1324084d85b8a97aaf0c248df7 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:12:14.910502 31861 raft_consensus.cc:2749] T 99e98c1324084d85b8a97aaf0c248df7 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:12:14.932602 21756 tablet_server.cc:196] TabletServer@127.21.63.3:0 shutdown complete.
W20260709 13:12:14.975557 30914 consensus_peers.cc:597] T c02f992ce0974544a4112672d8792c5f P 262942f67dfa4b8fa42234b18ac31ffd -> Peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Couldn't send request to peer 02c5705b45c04706bbb2c6ff3d7b0b8a. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.3:43993: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:15.017448 31849 raft_consensus.cc:493] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:12:15.018047 31849 raft_consensus.cc:515] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:12:15.020820 31849 leader_election.cc:290] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:12:15.021606 21756 tablet_server.cc:179] TabletServer@127.21.63.4:0 shutting down...
W20260709 13:12:15.028393 30914 consensus_peers.cc:597] T f8f19b152a0f46ac896478f3f05c8822 P 262942f67dfa4b8fa42234b18ac31ffd -> Peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Couldn't send request to peer 02c5705b45c04706bbb2c6ff3d7b0b8a. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.3:43993: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:15.031361 30912 leader_election.cc:336] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771): Network error: Client connection negotiation failed: client connection to 127.21.63.2:43771: connect: Connection refused (error 111)
W20260709 13:12:15.031981 30915 leader_election.cc:336] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Network error: Client connection negotiation failed: client connection to 127.21.63.1:37325: connect: Connection refused (error 111)
I20260709 13:12:15.032394 30915 leader_election.cc:304] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd; no voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:12:15.033161 31861 raft_consensus.cc:2749] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 13:12:15.041524 31682 raft_consensus.cc:493] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:12:15.042019 31682 raft_consensus.cc:515] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:12:15.043898 31682 leader_election.cc:290] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
W20260709 13:12:15.057070 30915 leader_election.cc:336] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Network error: Client connection negotiation failed: client connection to 127.21.63.1:37325: connect: Connection refused (error 111)
W20260709 13:12:15.058840 30912 leader_election.cc:336] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771): Network error: Client connection negotiation failed: client connection to 127.21.63.2:43771: connect: Connection refused (error 111)
I20260709 13:12:15.062439 30912 leader_election.cc:304] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd; no voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:12:15.064096 31682 raft_consensus.cc:2749] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:12:15.068203 31861 raft_consensus.cc:493] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:12:15.068780 31861 raft_consensus.cc:515] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:12:15.071343 31861 leader_election.cc:290] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
W20260709 13:12:15.077607 30915 leader_election.cc:336] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Network error: Client connection negotiation failed: client connection to 127.21.63.1:37325: connect: Connection refused (error 111)
W20260709 13:12:15.080761 30912 leader_election.cc:336] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771): Network error: Client connection negotiation failed: client connection to 127.21.63.2:43771: connect: Connection refused (error 111)
I20260709 13:12:15.081173 30912 leader_election.cc:304] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd; no voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:12:15.081830 31849 raft_consensus.cc:2749] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:12:15.102586 21756 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:12:15.103704 21756 tablet_replica.cc:333] T 8b3e41a728f1457ebf97274b95c7b23a P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
I20260709 13:12:15.104820 21756 raft_consensus.cc:2243] T 8b3e41a728f1457ebf97274b95c7b23a P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:15.105504 21756 raft_consensus.cc:2272] T 8b3e41a728f1457ebf97274b95c7b23a P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:15.108743 31849 raft_consensus.cc:493] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:12:15.108984 21756 tablet_replica.cc:333] T 99e98c1324084d85b8a97aaf0c248df7 P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
I20260709 13:12:15.109167 31849 raft_consensus.cc:515] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } } peers { permanent_uuid: "05213ac836744a53b9c089a5c052f4be" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 43771 } }
I20260709 13:12:15.110188 21756 raft_consensus.cc:2243] T 99e98c1324084d85b8a97aaf0c248df7 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:15.110874 21756 raft_consensus.cc:2272] T 99e98c1324084d85b8a97aaf0c248df7 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:15.111166 31849 leader_election.cc:290] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325), 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771)
I20260709 13:12:15.114137 21756 tablet_replica.cc:333] T f8f19b152a0f46ac896478f3f05c8822 P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
I20260709 13:12:15.115337 21756 raft_consensus.cc:2243] T f8f19b152a0f46ac896478f3f05c8822 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Raft consensus shutting down.
W20260709 13:12:15.116034 30915 leader_election.cc:336] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Network error: Client connection negotiation failed: client connection to 127.21.63.1:37325: connect: Connection refused (error 111)
I20260709 13:12:15.116494 21756 raft_consensus.cc:2272] T f8f19b152a0f46ac896478f3f05c8822 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:15.116585 30912 leader_election.cc:336] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 05213ac836744a53b9c089a5c052f4be (127.21.63.2:43771): Network error: Client connection negotiation failed: client connection to 127.21.63.2:43771: connect: Connection refused (error 111)
I20260709 13:12:15.117182 30912 leader_election.cc:304] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd; no voters: 05213ac836744a53b9c089a5c052f4be, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:12:15.117774 31849 raft_consensus.cc:2749] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:12:15.120059 21756 tablet_replica.cc:333] T d16c44f56c42466497ba4f7f1b2b5344 P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
I20260709 13:12:15.121088 21756 raft_consensus.cc:2243] T d16c44f56c42466497ba4f7f1b2b5344 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:15.122213 21756 raft_consensus.cc:2272] T d16c44f56c42466497ba4f7f1b2b5344 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:15.125442 21756 tablet_replica.cc:333] T 94220be3f55a4acbbd6bd5723b9f32e9 P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
I20260709 13:12:15.126333 21756 raft_consensus.cc:2243] T 94220be3f55a4acbbd6bd5723b9f32e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:15.126885 21756 raft_consensus.cc:2272] T 94220be3f55a4acbbd6bd5723b9f32e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:15.129897 21756 tablet_replica.cc:333] T 6fe93136933f4d2880ec355e1bbbf90d P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
I20260709 13:12:15.130795 21756 raft_consensus.cc:2243] T 6fe93136933f4d2880ec355e1bbbf90d P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:15.131376 21756 raft_consensus.cc:2272] T 6fe93136933f4d2880ec355e1bbbf90d P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:15.134130 21756 tablet_replica.cc:333] T 65776989d6674454816411c251aff5c1 P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
I20260709 13:12:15.135030 21756 raft_consensus.cc:2243] T 65776989d6674454816411c251aff5c1 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:15.135601 21756 raft_consensus.cc:2272] T 65776989d6674454816411c251aff5c1 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:15.138648 21756 tablet_replica.cc:333] T 8fb24a20af104a7486d0f1b2ae0e3843 P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
I20260709 13:12:15.139590 21756 raft_consensus.cc:2243] T 8fb24a20af104a7486d0f1b2ae0e3843 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:15.140748 21756 raft_consensus.cc:2272] T 8fb24a20af104a7486d0f1b2ae0e3843 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:15.143996 21756 tablet_replica.cc:333] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
I20260709 13:12:15.144917 21756 raft_consensus.cc:2243] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:15.145516 21756 raft_consensus.cc:2272] T 6f8c0b2b0b11472ca76e15fd3ce2e6f1 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:15.148583 21756 tablet_replica.cc:333] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
I20260709 13:12:15.149551 21756 raft_consensus.cc:2243] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:15.150652 21756 raft_consensus.cc:2272] T c0100ec8bd1e4a45b3c0ea52281f9204 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:15.153573 21756 tablet_replica.cc:333] T 92a4b4a5a07442f49f518b479db8c94b P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
I20260709 13:12:15.154479 21756 raft_consensus.cc:2243] T 92a4b4a5a07442f49f518b479db8c94b P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:15.155087 21756 raft_consensus.cc:2272] T 92a4b4a5a07442f49f518b479db8c94b P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:15.158027 21756 tablet_replica.cc:333] T 054fc96e257a4dc6b2ef90db5c1ad499 P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
I20260709 13:12:15.158855 21756 raft_consensus.cc:2243] T 054fc96e257a4dc6b2ef90db5c1ad499 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:15.159547 21756 raft_consensus.cc:2272] T 054fc96e257a4dc6b2ef90db5c1ad499 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:15.162447 21756 tablet_replica.cc:333] T b9fcfade988746a2967e115dbbcfe0f1 P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
I20260709 13:12:15.163318 21756 raft_consensus.cc:2243] T b9fcfade988746a2967e115dbbcfe0f1 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:15.164453 21756 raft_consensus.cc:2272] T b9fcfade988746a2967e115dbbcfe0f1 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:15.167775 21756 tablet_replica.cc:333] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
I20260709 13:12:15.168715 21756 raft_consensus.cc:2243] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:15.169333 21756 raft_consensus.cc:2272] T a38ff14550f74527a62591df59fb2c51 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:15.172379 21756 tablet_replica.cc:333] T d7edce5d08f24852b7afca9c57ba9c19 P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
I20260709 13:12:15.173316 21756 raft_consensus.cc:2243] T d7edce5d08f24852b7afca9c57ba9c19 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:15.174419 21756 raft_consensus.cc:2272] T d7edce5d08f24852b7afca9c57ba9c19 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:15.177631 21756 tablet_replica.cc:333] T 726b808ed64848dd89da4340d4d891fb P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
I20260709 13:12:15.178509 21756 raft_consensus.cc:2243] T 726b808ed64848dd89da4340d4d891fb P 262942f67dfa4b8fa42234b18ac31ffd [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:15.179196 21756 raft_consensus.cc:2272] T 726b808ed64848dd89da4340d4d891fb P 262942f67dfa4b8fa42234b18ac31ffd [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:15.182085 21756 tablet_replica.cc:333] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
I20260709 13:12:15.182955 21756 raft_consensus.cc:2243] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:15.183626 21756 raft_consensus.cc:2272] T 01cceb88bca348958b59f3697979dff1 P 262942f67dfa4b8fa42234b18ac31ffd [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:15.186349 21756 tablet_replica.cc:333] T 93a75acc8893414180a15797033a2d5a P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
I20260709 13:12:15.187126 21756 raft_consensus.cc:2243] T 93a75acc8893414180a15797033a2d5a P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:15.187628 21756 raft_consensus.cc:2272] T 93a75acc8893414180a15797033a2d5a P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:15.190266 21756 tablet_replica.cc:333] T c1efa319af774b23bd4ea782611f5252 P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
I20260709 13:12:15.191033 21756 raft_consensus.cc:2243] T c1efa319af774b23bd4ea782611f5252 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:15.191608 21756 raft_consensus.cc:2272] T c1efa319af774b23bd4ea782611f5252 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:15.194367 21756 tablet_replica.cc:333] T 11c7485c290740bba00c8b9eaf9b917f P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
I20260709 13:12:15.195189 21756 raft_consensus.cc:2243] T 11c7485c290740bba00c8b9eaf9b917f P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:15.195758 21756 raft_consensus.cc:2272] T 11c7485c290740bba00c8b9eaf9b917f P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:15.198396 21756 tablet_replica.cc:333] T 8770fcfd98154b2095c2c9f74135593b P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
I20260709 13:12:15.199195 21756 raft_consensus.cc:2243] T 8770fcfd98154b2095c2c9f74135593b P 262942f67dfa4b8fa42234b18ac31ffd [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:15.200227 21756 raft_consensus.cc:2272] T 8770fcfd98154b2095c2c9f74135593b P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:15.203224 21756 tablet_replica.cc:333] T bbb2acd63baa48a3a2ace94016522e92 P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
I20260709 13:12:15.204034 21756 raft_consensus.cc:2243] T bbb2acd63baa48a3a2ace94016522e92 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:15.204591 21756 raft_consensus.cc:2272] T bbb2acd63baa48a3a2ace94016522e92 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:15.207520 21756 tablet_replica.cc:333] T 8b6052cddae64c5c9d16f60272735788 P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
I20260709 13:12:15.208307 21756 raft_consensus.cc:2243] T 8b6052cddae64c5c9d16f60272735788 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:15.208874 21756 raft_consensus.cc:2272] T 8b6052cddae64c5c9d16f60272735788 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:15.211727 21756 tablet_replica.cc:333] T 26172e2c14cd4d178764ed13012c21f7 P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
I20260709 13:12:15.212495 21756 raft_consensus.cc:2243] T 26172e2c14cd4d178764ed13012c21f7 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:15.213541 21756 raft_consensus.cc:2272] T 26172e2c14cd4d178764ed13012c21f7 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:15.216032 21756 tablet_replica.cc:333] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
I20260709 13:12:15.216827 21756 raft_consensus.cc:2243] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:15.217373 21756 raft_consensus.cc:2272] T ea4dcd4ecc9840d2b8785d14dd4537b8 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:15.219986 21756 tablet_replica.cc:333] T 7b988f0495c34b7591ad2fb0d5f65693 P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
I20260709 13:12:15.220785 21756 raft_consensus.cc:2243] T 7b988f0495c34b7591ad2fb0d5f65693 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:15.221863 21756 raft_consensus.cc:2272] T 7b988f0495c34b7591ad2fb0d5f65693 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:15.222856 31849 raft_consensus.cc:493] T ea563bd67a9849d18c9c3dfb21d58a93 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d22c9d3d15904327a78a0e548a8ff6cd)
I20260709 13:12:15.223322 31849 raft_consensus.cc:515] T ea563bd67a9849d18c9c3dfb21d58a93 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "02c5705b45c04706bbb2c6ff3d7b0b8a" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 43993 } } peers { permanent_uuid: "262942f67dfa4b8fa42234b18ac31ffd" member_type: VOTER last_known_addr { host: "127.21.63.4" port: 41313 } } peers { permanent_uuid: "d22c9d3d15904327a78a0e548a8ff6cd" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 37325 } }
I20260709 13:12:15.224988 21756 tablet_replica.cc:333] T d4dbe5b2d724412a909fd0641d1a213a P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
I20260709 13:12:15.225895 21756 raft_consensus.cc:2243] T d4dbe5b2d724412a909fd0641d1a213a P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:15.226918 21756 raft_consensus.cc:2272] T d4dbe5b2d724412a909fd0641d1a213a P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:15.227662 31849 leader_election.cc:290] T ea563bd67a9849d18c9c3dfb21d58a93 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993), d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325)
I20260709 13:12:15.229642 21756 tablet_replica.cc:333] T 80a61882ef804a85a31e90d67930c5e9 P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
W20260709 13:12:15.230063 30915 leader_election.cc:336] T ea563bd67a9849d18c9c3dfb21d58a93 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d22c9d3d15904327a78a0e548a8ff6cd (127.21.63.1:37325): Network error: Client connection negotiation failed: client connection to 127.21.63.1:37325: connect: Connection refused (error 111)
I20260709 13:12:15.230507 21756 raft_consensus.cc:2243] T 80a61882ef804a85a31e90d67930c5e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 13:12:15.230686 30914 leader_election.cc:336] T ea563bd67a9849d18c9c3dfb21d58a93 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 02c5705b45c04706bbb2c6ff3d7b0b8a (127.21.63.3:43993): Network error: Client connection negotiation failed: client connection to 127.21.63.3:43993: connect: Connection refused (error 111)
I20260709 13:12:15.231150 21756 raft_consensus.cc:2272] T 80a61882ef804a85a31e90d67930c5e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:15.231153 30914 leader_election.cc:304] T ea563bd67a9849d18c9c3dfb21d58a93 P 262942f67dfa4b8fa42234b18ac31ffd [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 262942f67dfa4b8fa42234b18ac31ffd; no voters: 02c5705b45c04706bbb2c6ff3d7b0b8a, d22c9d3d15904327a78a0e548a8ff6cd
I20260709 13:12:15.232074 31849 raft_consensus.cc:2749] T ea563bd67a9849d18c9c3dfb21d58a93 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 13:12:15.234025 21756 tablet_replica.cc:333] T 27f26e218737414893d5c4bfa4b2cd94 P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
I20260709 13:12:15.234880 21756 raft_consensus.cc:2243] T 27f26e218737414893d5c4bfa4b2cd94 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:15.235421 21756 raft_consensus.cc:2272] T 27f26e218737414893d5c4bfa4b2cd94 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:15.238034 21756 tablet_replica.cc:333] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
I20260709 13:12:15.238746 21756 raft_consensus.cc:2243] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:15.239231 21756 raft_consensus.cc:2272] T 7ed3eac558b34457aa26b1c40652f887 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:15.241927 21756 tablet_replica.cc:333] T b226c44412c74106a0feb2ca411b1661 P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
I20260709 13:12:15.242650 21756 raft_consensus.cc:2243] T b226c44412c74106a0feb2ca411b1661 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:15.243201 21756 raft_consensus.cc:2272] T b226c44412c74106a0feb2ca411b1661 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:15.245689 21756 tablet_replica.cc:333] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
I20260709 13:12:15.246296 21756 raft_consensus.cc:2243] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:15.246723 21756 raft_consensus.cc:2272] T f63e902c7c7a4d0fa5d2107620ce851e P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:15.248479 21756 tablet_replica.cc:333] T c02f992ce0974544a4112672d8792c5f P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
I20260709 13:12:15.249030 21756 raft_consensus.cc:2243] T c02f992ce0974544a4112672d8792c5f P 262942f67dfa4b8fa42234b18ac31ffd [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:15.249763 21756 raft_consensus.cc:2272] T c02f992ce0974544a4112672d8792c5f P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:15.251608 21756 tablet_replica.cc:333] T 283eb8311aeb4b5599dd844da98ee0b7 P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
I20260709 13:12:15.252125 21756 raft_consensus.cc:2243] T 283eb8311aeb4b5599dd844da98ee0b7 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:15.252923 21756 raft_consensus.cc:2272] T 283eb8311aeb4b5599dd844da98ee0b7 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:15.254940 21756 tablet_replica.cc:333] T ac85f05e63ed463681b3ec790b39c4d8 P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
I20260709 13:12:15.255419 21756 raft_consensus.cc:2243] T ac85f05e63ed463681b3ec790b39c4d8 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:15.255823 21756 raft_consensus.cc:2272] T ac85f05e63ed463681b3ec790b39c4d8 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:15.257754 21756 tablet_replica.cc:333] T ccc68dd4c56548cba0624daafb5ae23c P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
I20260709 13:12:15.258304 21756 raft_consensus.cc:2243] T ccc68dd4c56548cba0624daafb5ae23c P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:15.259003 21756 raft_consensus.cc:2272] T ccc68dd4c56548cba0624daafb5ae23c P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:15.260860 21756 tablet_replica.cc:333] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
I20260709 13:12:15.261353 21756 raft_consensus.cc:2243] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:15.261763 21756 raft_consensus.cc:2272] T 0ac93952b93645b1ad1d7d96cb60c3e9 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:15.263576 21756 tablet_replica.cc:333] T 39980eee51514ba493958931f406a145 P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
I20260709 13:12:15.264125 21756 raft_consensus.cc:2243] T 39980eee51514ba493958931f406a145 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:15.264905 21756 raft_consensus.cc:2272] T 39980eee51514ba493958931f406a145 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:15.266793 21756 tablet_replica.cc:333] T b3754a670e0b4ed99d37283bf1647bf5 P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
I20260709 13:12:15.267315 21756 raft_consensus.cc:2243] T b3754a670e0b4ed99d37283bf1647bf5 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:15.268075 21756 raft_consensus.cc:2272] T b3754a670e0b4ed99d37283bf1647bf5 P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:15.269974 21756 tablet_replica.cc:333] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
I20260709 13:12:15.270469 21756 raft_consensus.cc:2243] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:15.270874 21756 raft_consensus.cc:2272] T 4fb3f8d627b04c8a8971f7e4d3a5c010 P 262942f67dfa4b8fa42234b18ac31ffd [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:15.272809 21756 tablet_replica.cc:333] T 784deee1ea9543df840c58ecaa6f911a P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
I20260709 13:12:15.273296 21756 raft_consensus.cc:2243] T 784deee1ea9543df840c58ecaa6f911a P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:15.273723 21756 raft_consensus.cc:2272] T 784deee1ea9543df840c58ecaa6f911a P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:15.275697 21756 tablet_replica.cc:333] T 437c20f3d53f43048eca57834248cc0d P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
I20260709 13:12:15.276160 21756 raft_consensus.cc:2243] T 437c20f3d53f43048eca57834248cc0d P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:15.276548 21756 raft_consensus.cc:2272] T 437c20f3d53f43048eca57834248cc0d P 262942f67dfa4b8fa42234b18ac31ffd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:15.278403 21756 tablet_replica.cc:333] T ea563bd67a9849d18c9c3dfb21d58a93 P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
I20260709 13:12:15.278872 21756 raft_consensus.cc:2243] T ea563bd67a9849d18c9c3dfb21d58a93 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:15.279304 21756 raft_consensus.cc:2272] T ea563bd67a9849d18c9c3dfb21d58a93 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:15.281157 21756 tablet_replica.cc:333] T bcac31d5b7ab48ec9c071a8f77750a88 P 262942f67dfa4b8fa42234b18ac31ffd: stopping tablet replica
I20260709 13:12:15.281595 21756 raft_consensus.cc:2243] T bcac31d5b7ab48ec9c071a8f77750a88 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:15.281988 21756 raft_consensus.cc:2272] T bcac31d5b7ab48ec9c071a8f77750a88 P 262942f67dfa4b8fa42234b18ac31ffd [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:16.346753 21756 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 30970)
I20260709 13:12:16.494572 21756 tablet_server.cc:196] TabletServer@127.21.63.4:0 shutdown complete.
I20260709 13:12:16.557036 21756 master.cc:562] Master@127.21.63.62:40095 shutting down...
I20260709 13:12:16.588124 21756 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 35038aac9dd04872a98f269c76b1b5d3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:16.588820 21756 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 35038aac9dd04872a98f269c76b1b5d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:16.589140 21756 tablet_replica.cc:333] T 00000000000000000000000000000000 P 35038aac9dd04872a98f269c76b1b5d3: stopping tablet replica
I20260709 13:12:16.613124 21756 master.cc:584] Master@127.21.63.62:40095 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (32592 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 13:12:16.701311 21756 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.63.62:36925
I20260709 13:12:16.702414 21756 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:12:16.708194 31870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:16.708714 31871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:16.709281 31873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:16.710563 21756 server_base.cc:1061] running on GCE node
I20260709 13:12:16.711715 21756 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:12:16.711934 21756 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:12:16.712098 21756 hybrid_clock.cc:648] HybridClock initialized: now 1783602736712081 us; error 0 us; skew 500 ppm
I20260709 13:12:16.712625 21756 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:12:16.715234 21756 webserver.cc:533] Webserver started at http://127.21.63.62:35031/ using document root <none> and password file <none>
I20260709 13:12:16.715734 21756 fs_manager.cc:362] Metadata directory not provided
I20260709 13:12:16.715937 21756 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:12:16.716219 21756 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:12:16.717752 21756 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602429420501-21756-0/minicluster-data/master-0-root/instance:
uuid: "33e978ec2bde4a4a89c7f981d4853921"
format_stamp: "Formatted at 2026-07-09 13:12:16 on dist-test-slave-c6mt"
I20260709 13:12:16.722117 21756 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.001s
I20260709 13:12:16.725652 31878 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:16.726521 21756 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:12:16.726835 21756 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602429420501-21756-0/minicluster-data/master-0-root
uuid: "33e978ec2bde4a4a89c7f981d4853921"
format_stamp: "Formatted at 2026-07-09 13:12:16 on dist-test-slave-c6mt"
I20260709 13:12:16.727156 21756 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602429420501-21756-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602429420501-21756-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602429420501-21756-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:12:16.767611 21756 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:12:16.768918 21756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:12:16.808029 21756 rpc_server.cc:307] RPC server started. Bound to: 127.21.63.62:36925
I20260709 13:12:16.808159 31929 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.63.62:36925 every 8 connection(s)
I20260709 13:12:16.811875 31930 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:16.822674 31930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33e978ec2bde4a4a89c7f981d4853921: Bootstrap starting.
I20260709 13:12:16.827557 31930 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33e978ec2bde4a4a89c7f981d4853921: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:16.831945 31930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33e978ec2bde4a4a89c7f981d4853921: No bootstrap required, opened a new log
I20260709 13:12:16.834106 31930 raft_consensus.cc:359] T 00000000000000000000000000000000 P 33e978ec2bde4a4a89c7f981d4853921 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33e978ec2bde4a4a89c7f981d4853921" member_type: VOTER }
I20260709 13:12:16.834474 31930 raft_consensus.cc:385] T 00000000000000000000000000000000 P 33e978ec2bde4a4a89c7f981d4853921 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:16.834653 31930 raft_consensus.cc:740] T 00000000000000000000000000000000 P 33e978ec2bde4a4a89c7f981d4853921 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33e978ec2bde4a4a89c7f981d4853921, State: Initialized, Role: FOLLOWER
I20260709 13:12:16.835136 31930 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33e978ec2bde4a4a89c7f981d4853921 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33e978ec2bde4a4a89c7f981d4853921" member_type: VOTER }
I20260709 13:12:16.835528 31930 raft_consensus.cc:399] T 00000000000000000000000000000000 P 33e978ec2bde4a4a89c7f981d4853921 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:12:16.835723 31930 raft_consensus.cc:493] T 00000000000000000000000000000000 P 33e978ec2bde4a4a89c7f981d4853921 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:12:16.835938 31930 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 33e978ec2bde4a4a89c7f981d4853921 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:16.841024 31930 raft_consensus.cc:515] T 00000000000000000000000000000000 P 33e978ec2bde4a4a89c7f981d4853921 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33e978ec2bde4a4a89c7f981d4853921" member_type: VOTER }
I20260709 13:12:16.841521 31930 leader_election.cc:304] T 00000000000000000000000000000000 P 33e978ec2bde4a4a89c7f981d4853921 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 33e978ec2bde4a4a89c7f981d4853921; no voters: 
I20260709 13:12:16.842784 31930 leader_election.cc:290] T 00000000000000000000000000000000 P 33e978ec2bde4a4a89c7f981d4853921 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:12:16.843264 31933 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 33e978ec2bde4a4a89c7f981d4853921 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:16.844825 31933 raft_consensus.cc:697] T 00000000000000000000000000000000 P 33e978ec2bde4a4a89c7f981d4853921 [term 1 LEADER]: Becoming Leader. State: Replica: 33e978ec2bde4a4a89c7f981d4853921, State: Running, Role: LEADER
I20260709 13:12:16.845603 31933 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33e978ec2bde4a4a89c7f981d4853921 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33e978ec2bde4a4a89c7f981d4853921" member_type: VOTER }
I20260709 13:12:16.846077 31930 sys_catalog.cc:565] T 00000000000000000000000000000000 P 33e978ec2bde4a4a89c7f981d4853921 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:12:16.850499 31934 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33e978ec2bde4a4a89c7f981d4853921 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "33e978ec2bde4a4a89c7f981d4853921" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33e978ec2bde4a4a89c7f981d4853921" member_type: VOTER } }
I20260709 13:12:16.851280 31935 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33e978ec2bde4a4a89c7f981d4853921 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33e978ec2bde4a4a89c7f981d4853921. Latest consensus state: current_term: 1 leader_uuid: "33e978ec2bde4a4a89c7f981d4853921" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33e978ec2bde4a4a89c7f981d4853921" member_type: VOTER } }
I20260709 13:12:16.851989 31935 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33e978ec2bde4a4a89c7f981d4853921 [sys.catalog]: This master's current role is: LEADER
I20260709 13:12:16.851357 31934 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33e978ec2bde4a4a89c7f981d4853921 [sys.catalog]: This master's current role is: LEADER
I20260709 13:12:16.854208 31941 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:12:16.862542 31941 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:12:16.863767 21756 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:12:16.872753 31941 catalog_manager.cc:1383] Generated new cluster ID: f786c88f747648eebfa3f8693dedfe1a
I20260709 13:12:16.873075 31941 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:12:16.895977 31941 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:12:16.897464 31941 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:12:16.914036 31941 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 33e978ec2bde4a4a89c7f981d4853921: Generated new TSK 0
I20260709 13:12:16.914721 31941 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:12:16.931041 21756 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:12:16.937494 31951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:16.938309 31952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:16.939558 31954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:16.940485 21756 server_base.cc:1061] running on GCE node
I20260709 13:12:16.941476 21756 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:12:16.941692 21756 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:12:16.941861 21756 hybrid_clock.cc:648] HybridClock initialized: now 1783602736941845 us; error 0 us; skew 500 ppm
I20260709 13:12:16.942375 21756 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:12:16.944796 21756 webserver.cc:533] Webserver started at http://127.21.63.1:34303/ using document root <none> and password file <none>
I20260709 13:12:16.945262 21756 fs_manager.cc:362] Metadata directory not provided
I20260709 13:12:16.945465 21756 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:12:16.945729 21756 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:12:16.946847 21756 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602429420501-21756-0/minicluster-data/ts-0-root/instance:
uuid: "91778255eb264fe6a6dc208998e849ba"
format_stamp: "Formatted at 2026-07-09 13:12:16 on dist-test-slave-c6mt"
I20260709 13:12:16.951344 21756 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 13:12:16.955152 31959 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:16.956131 21756 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:12:16.956489 21756 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602429420501-21756-0/minicluster-data/ts-0-root
uuid: "91778255eb264fe6a6dc208998e849ba"
format_stamp: "Formatted at 2026-07-09 13:12:16 on dist-test-slave-c6mt"
I20260709 13:12:16.956844 21756 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602429420501-21756-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602429420501-21756-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602429420501-21756-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:12:16.971897 21756 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:12:16.973246 21756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:12:16.974722 21756 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:12:16.977074 21756 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:12:16.977305 21756 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:16.977562 21756 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:12:16.977749 21756 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:17.025020 21756 rpc_server.cc:307] RPC server started. Bound to: 127.21.63.1:32979
I20260709 13:12:17.025138 32021 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.63.1:32979 every 8 connection(s)
I20260709 13:12:17.030866 21756 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:12:17.038988 32026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:17.040899 32027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:17.044425 32029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:17.045324 21756 server_base.cc:1061] running on GCE node
I20260709 13:12:17.045471 32022 heartbeater.cc:344] Connected to a master server at 127.21.63.62:36925
I20260709 13:12:17.045962 32022 heartbeater.cc:461] Registering TS with master...
I20260709 13:12:17.046715 21756 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:12:17.046718 32022 heartbeater.cc:507] Master 127.21.63.62:36925 requested a full tablet report, sending...
W20260709 13:12:17.047083 21756 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:12:17.047356 21756 hybrid_clock.cc:648] HybridClock initialized: now 1783602737047341 us; error 0 us; skew 500 ppm
I20260709 13:12:17.048344 21756 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:12:17.049577 31895 ts_manager.cc:194] Registered new tserver with Master: 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979)
I20260709 13:12:17.051010 21756 webserver.cc:533] Webserver started at http://127.21.63.2:44301/ using document root <none> and password file <none>
I20260709 13:12:17.051591 21756 fs_manager.cc:362] Metadata directory not provided
I20260709 13:12:17.051766 31895 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49436
I20260709 13:12:17.052271 21756 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:12:17.052767 21756 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:12:17.054342 21756 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602429420501-21756-0/minicluster-data/ts-1-root/instance:
uuid: "31690560715140e6847173104a2b0f2e"
format_stamp: "Formatted at 2026-07-09 13:12:17 on dist-test-slave-c6mt"
I20260709 13:12:17.059825 21756 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:12:17.063583 32034 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:17.064488 21756 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:12:17.064940 21756 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602429420501-21756-0/minicluster-data/ts-1-root
uuid: "31690560715140e6847173104a2b0f2e"
format_stamp: "Formatted at 2026-07-09 13:12:17 on dist-test-slave-c6mt"
I20260709 13:12:17.065292 21756 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602429420501-21756-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602429420501-21756-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602429420501-21756-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:12:17.079530 21756 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:12:17.080724 21756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:12:17.082221 21756 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:12:17.084395 21756 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:12:17.084614 21756 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:17.084879 21756 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:12:17.085048 21756 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:17.129739 21756 rpc_server.cc:307] RPC server started. Bound to: 127.21.63.2:44689
I20260709 13:12:17.129844 32096 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.63.2:44689 every 8 connection(s)
I20260709 13:12:17.135076 21756 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:12:17.143841 32101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:17.143999 32100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:17.147918 32103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:17.148010 21756 server_base.cc:1061] running on GCE node
I20260709 13:12:17.148375 32097 heartbeater.cc:344] Connected to a master server at 127.21.63.62:36925
I20260709 13:12:17.148918 32097 heartbeater.cc:461] Registering TS with master...
I20260709 13:12:17.150211 32097 heartbeater.cc:507] Master 127.21.63.62:36925 requested a full tablet report, sending...
I20260709 13:12:17.150688 21756 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:12:17.151000 21756 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:12:17.151248 21756 hybrid_clock.cc:648] HybridClock initialized: now 1783602737151229 us; error 0 us; skew 500 ppm
I20260709 13:12:17.151851 21756 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:12:17.152571 31895 ts_manager.cc:194] Registered new tserver with Master: 31690560715140e6847173104a2b0f2e (127.21.63.2:44689)
I20260709 13:12:17.154548 21756 webserver.cc:533] Webserver started at http://127.21.63.3:33627/ using document root <none> and password file <none>
I20260709 13:12:17.154733 31895 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49450
I20260709 13:12:17.155202 21756 fs_manager.cc:362] Metadata directory not provided
I20260709 13:12:17.155537 21756 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:12:17.155848 21756 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:12:17.157392 21756 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602429420501-21756-0/minicluster-data/ts-2-root/instance:
uuid: "e69f3153d090418380c201dddfef4c60"
format_stamp: "Formatted at 2026-07-09 13:12:17 on dist-test-slave-c6mt"
I20260709 13:12:17.162173 21756 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.003s
I20260709 13:12:17.165545 32108 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:17.166301 21756 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:12:17.166606 21756 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602429420501-21756-0/minicluster-data/ts-2-root
uuid: "e69f3153d090418380c201dddfef4c60"
format_stamp: "Formatted at 2026-07-09 13:12:17 on dist-test-slave-c6mt"
I20260709 13:12:17.166893 21756 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602429420501-21756-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602429420501-21756-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602429420501-21756-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:12:17.199029 21756 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:12:17.200294 21756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:12:17.201810 21756 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:12:17.204044 21756 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:12:17.204283 21756 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:17.204528 21756 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:12:17.204717 21756 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 13:12:17.251950 21756 rpc_server.cc:307] RPC server started. Bound to: 127.21.63.3:40639
I20260709 13:12:17.252117 32170 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.63.3:40639 every 8 connection(s)
I20260709 13:12:17.268289 32171 heartbeater.cc:344] Connected to a master server at 127.21.63.62:36925
I20260709 13:12:17.268929 32171 heartbeater.cc:461] Registering TS with master...
I20260709 13:12:17.269735 32171 heartbeater.cc:507] Master 127.21.63.62:36925 requested a full tablet report, sending...
I20260709 13:12:17.271742 31895 ts_manager.cc:194] Registered new tserver with Master: e69f3153d090418380c201dddfef4c60 (127.21.63.3:40639)
I20260709 13:12:17.271930 21756 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.016412124s
I20260709 13:12:17.273419 31895 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49454
I20260709 13:12:17.299438 31895 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49456:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:12:17.302599 31895 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:12:17.398706 32136 tablet_service.cc:1511] Processing CreateTablet for tablet f9a7e52d74da4cd79abac3858391ae77 (DEFAULT_TABLE table=test-workload [id=238d798efc0d441585dcbe3a69230c0a]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:12:17.399506 32134 tablet_service.cc:1511] Processing CreateTablet for tablet 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 (DEFAULT_TABLE table=test-workload [id=238d798efc0d441585dcbe3a69230c0a]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:12:17.399590 32135 tablet_service.cc:1511] Processing CreateTablet for tablet d0a7cae5f3084d0ba50763cc87888571 (DEFAULT_TABLE table=test-workload [id=238d798efc0d441585dcbe3a69230c0a]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:12:17.400219 32136 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9a7e52d74da4cd79abac3858391ae77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:17.401067 32134 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ea7beb6ca5f4de4b64c0e8a9e9d99a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:17.401957 32133 tablet_service.cc:1511] Processing CreateTablet for tablet 3a76c5023ae14f8ba25de09e3a9b60de (DEFAULT_TABLE table=test-workload [id=238d798efc0d441585dcbe3a69230c0a]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:12:17.404186 32132 tablet_service.cc:1511] Processing CreateTablet for tablet 7b2f10befb2e474fbe821af8d5e92fd5 (DEFAULT_TABLE table=test-workload [id=238d798efc0d441585dcbe3a69230c0a]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:12:17.406852 32131 tablet_service.cc:1511] Processing CreateTablet for tablet b40f21aa6dd54f64bed8e716a77df4d6 (DEFAULT_TABLE table=test-workload [id=238d798efc0d441585dcbe3a69230c0a]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:12:17.408257 32130 tablet_service.cc:1511] Processing CreateTablet for tablet d1fc79d86bee43c5b062ced8a090aab1 (DEFAULT_TABLE table=test-workload [id=238d798efc0d441585dcbe3a69230c0a]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:12:17.410445 32133 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a76c5023ae14f8ba25de09e3a9b60de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:17.416000 32129 tablet_service.cc:1511] Processing CreateTablet for tablet 709f871588d74579bab67c29aca59135 (DEFAULT_TABLE table=test-workload [id=238d798efc0d441585dcbe3a69230c0a]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:12:17.417275 32132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b2f10befb2e474fbe821af8d5e92fd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:17.417918 32130 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1fc79d86bee43c5b062ced8a090aab1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:17.418601 32131 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b40f21aa6dd54f64bed8e716a77df4d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:17.419389 32129 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 709f871588d74579bab67c29aca59135. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:17.420531 32135 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0a7cae5f3084d0ba50763cc87888571. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:17.425318 32062 tablet_service.cc:1511] Processing CreateTablet for tablet f9a7e52d74da4cd79abac3858391ae77 (DEFAULT_TABLE table=test-workload [id=238d798efc0d441585dcbe3a69230c0a]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:12:17.425899 31987 tablet_service.cc:1511] Processing CreateTablet for tablet f9a7e52d74da4cd79abac3858391ae77 (DEFAULT_TABLE table=test-workload [id=238d798efc0d441585dcbe3a69230c0a]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 13:12:17.427577 32062 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9a7e52d74da4cd79abac3858391ae77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:17.427687 31987 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f9a7e52d74da4cd79abac3858391ae77. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:17.430353 32061 tablet_service.cc:1511] Processing CreateTablet for tablet d0a7cae5f3084d0ba50763cc87888571 (DEFAULT_TABLE table=test-workload [id=238d798efc0d441585dcbe3a69230c0a]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:12:17.432225 32061 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0a7cae5f3084d0ba50763cc87888571. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:17.435365 32060 tablet_service.cc:1511] Processing CreateTablet for tablet 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 (DEFAULT_TABLE table=test-workload [id=238d798efc0d441585dcbe3a69230c0a]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:12:17.436906 32060 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ea7beb6ca5f4de4b64c0e8a9e9d99a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:17.440655 32059 tablet_service.cc:1511] Processing CreateTablet for tablet 3a76c5023ae14f8ba25de09e3a9b60de (DEFAULT_TABLE table=test-workload [id=238d798efc0d441585dcbe3a69230c0a]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:12:17.442067 32059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a76c5023ae14f8ba25de09e3a9b60de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:17.449901 31986 tablet_service.cc:1511] Processing CreateTablet for tablet d0a7cae5f3084d0ba50763cc87888571 (DEFAULT_TABLE table=test-workload [id=238d798efc0d441585dcbe3a69230c0a]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 13:12:17.451396 31986 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d0a7cae5f3084d0ba50763cc87888571. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:17.456280 31982 tablet_service.cc:1511] Processing CreateTablet for tablet b40f21aa6dd54f64bed8e716a77df4d6 (DEFAULT_TABLE table=test-workload [id=238d798efc0d441585dcbe3a69230c0a]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:12:17.457669 31982 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b40f21aa6dd54f64bed8e716a77df4d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:17.461856 31981 tablet_service.cc:1511] Processing CreateTablet for tablet d1fc79d86bee43c5b062ced8a090aab1 (DEFAULT_TABLE table=test-workload [id=238d798efc0d441585dcbe3a69230c0a]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:12:17.463439 31981 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1fc79d86bee43c5b062ced8a090aab1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:17.465062 32058 tablet_service.cc:1511] Processing CreateTablet for tablet 7b2f10befb2e474fbe821af8d5e92fd5 (DEFAULT_TABLE table=test-workload [id=238d798efc0d441585dcbe3a69230c0a]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:12:17.466533 32058 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b2f10befb2e474fbe821af8d5e92fd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:17.469895 31980 tablet_service.cc:1511] Processing CreateTablet for tablet 709f871588d74579bab67c29aca59135 (DEFAULT_TABLE table=test-workload [id=238d798efc0d441585dcbe3a69230c0a]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:12:17.470754 32128 tablet_service.cc:1511] Processing CreateTablet for tablet 0dfa6288e543429c99109ebbd5a223b9 (DEFAULT_TABLE table=test-workload [id=238d798efc0d441585dcbe3a69230c0a]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:12:17.471489 31980 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 709f871588d74579bab67c29aca59135. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:17.473032 32128 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0dfa6288e543429c99109ebbd5a223b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:17.473915 31983 tablet_service.cc:1511] Processing CreateTablet for tablet 7b2f10befb2e474fbe821af8d5e92fd5 (DEFAULT_TABLE table=test-workload [id=238d798efc0d441585dcbe3a69230c0a]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 13:12:17.453841 31979 tablet_service.cc:1511] Processing CreateTablet for tablet 0dfa6288e543429c99109ebbd5a223b9 (DEFAULT_TABLE table=test-workload [id=238d798efc0d441585dcbe3a69230c0a]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:12:17.453841 31985 tablet_service.cc:1511] Processing CreateTablet for tablet 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 (DEFAULT_TABLE table=test-workload [id=238d798efc0d441585dcbe3a69230c0a]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 13:12:17.477981 31978 tablet_service.cc:1511] Processing CreateTablet for tablet 5e568fd2b2a149aeadd6c64bc70af9b7 (DEFAULT_TABLE table=test-workload [id=238d798efc0d441585dcbe3a69230c0a]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:12:17.481078 31984 tablet_service.cc:1511] Processing CreateTablet for tablet 3a76c5023ae14f8ba25de09e3a9b60de (DEFAULT_TABLE table=test-workload [id=238d798efc0d441585dcbe3a69230c0a]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 13:12:17.483088 31984 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a76c5023ae14f8ba25de09e3a9b60de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:17.485019 31985 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5ea7beb6ca5f4de4b64c0e8a9e9d99a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:17.488989 31979 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0dfa6288e543429c99109ebbd5a223b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:17.491272 31983 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b2f10befb2e474fbe821af8d5e92fd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:17.497167 31978 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e568fd2b2a149aeadd6c64bc70af9b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:17.514344 32057 tablet_service.cc:1511] Processing CreateTablet for tablet b40f21aa6dd54f64bed8e716a77df4d6 (DEFAULT_TABLE table=test-workload [id=238d798efc0d441585dcbe3a69230c0a]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 13:12:17.515805 32057 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b40f21aa6dd54f64bed8e716a77df4d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:17.531126 32056 tablet_service.cc:1511] Processing CreateTablet for tablet d1fc79d86bee43c5b062ced8a090aab1 (DEFAULT_TABLE table=test-workload [id=238d798efc0d441585dcbe3a69230c0a]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 13:12:17.533911 32055 tablet_service.cc:1511] Processing CreateTablet for tablet 709f871588d74579bab67c29aca59135 (DEFAULT_TABLE table=test-workload [id=238d798efc0d441585dcbe3a69230c0a]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 13:12:17.536288 32054 tablet_service.cc:1511] Processing CreateTablet for tablet 0dfa6288e543429c99109ebbd5a223b9 (DEFAULT_TABLE table=test-workload [id=238d798efc0d441585dcbe3a69230c0a]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 13:12:17.532521 32056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d1fc79d86bee43c5b062ced8a090aab1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:17.554067 32132 tablet_service.cc:1511] Processing CreateTablet for tablet 5e568fd2b2a149aeadd6c64bc70af9b7 (DEFAULT_TABLE table=test-workload [id=238d798efc0d441585dcbe3a69230c0a]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:12:17.556137 32059 tablet_service.cc:1511] Processing CreateTablet for tablet 5e568fd2b2a149aeadd6c64bc70af9b7 (DEFAULT_TABLE table=test-workload [id=238d798efc0d441585dcbe3a69230c0a]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 13:12:17.558372 32132 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e568fd2b2a149aeadd6c64bc70af9b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:17.558979 32054 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0dfa6288e543429c99109ebbd5a223b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:17.561206 32055 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 709f871588d74579bab67c29aca59135. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:17.564747 32188 tablet_bootstrap.cc:492] T f9a7e52d74da4cd79abac3858391ae77 P 91778255eb264fe6a6dc208998e849ba: Bootstrap starting.
I20260709 13:12:17.573644 32186 tablet_bootstrap.cc:492] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60: Bootstrap starting.
I20260709 13:12:17.581542 32188 tablet_bootstrap.cc:654] T f9a7e52d74da4cd79abac3858391ae77 P 91778255eb264fe6a6dc208998e849ba: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:17.604357 32186 tablet_bootstrap.cc:654] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:17.610957 32187 tablet_bootstrap.cc:492] T d0a7cae5f3084d0ba50763cc87888571 P 31690560715140e6847173104a2b0f2e: Bootstrap starting.
I20260709 13:12:17.611815 32059 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e568fd2b2a149aeadd6c64bc70af9b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:17.632994 32187 tablet_bootstrap.cc:654] T d0a7cae5f3084d0ba50763cc87888571 P 31690560715140e6847173104a2b0f2e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:17.643536 32187 tablet_bootstrap.cc:492] T d0a7cae5f3084d0ba50763cc87888571 P 31690560715140e6847173104a2b0f2e: No bootstrap required, opened a new log
I20260709 13:12:17.644029 32188 tablet_bootstrap.cc:492] T f9a7e52d74da4cd79abac3858391ae77 P 91778255eb264fe6a6dc208998e849ba: No bootstrap required, opened a new log
I20260709 13:12:17.644588 32188 ts_tablet_manager.cc:1403] T f9a7e52d74da4cd79abac3858391ae77 P 91778255eb264fe6a6dc208998e849ba: Time spent bootstrapping tablet: real 0.080s	user 0.036s	sys 0.019s
I20260709 13:12:17.644104 32187 ts_tablet_manager.cc:1403] T d0a7cae5f3084d0ba50763cc87888571 P 31690560715140e6847173104a2b0f2e: Time spent bootstrapping tablet: real 0.034s	user 0.006s	sys 0.011s
I20260709 13:12:17.645623 32186 tablet_bootstrap.cc:492] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60: No bootstrap required, opened a new log
I20260709 13:12:17.646121 32186 ts_tablet_manager.cc:1403] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60: Time spent bootstrapping tablet: real 0.073s	user 0.001s	sys 0.013s
I20260709 13:12:17.648263 32188 raft_consensus.cc:359] T f9a7e52d74da4cd79abac3858391ae77 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } }
I20260709 13:12:17.649048 32188 raft_consensus.cc:385] T f9a7e52d74da4cd79abac3858391ae77 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:17.649389 32188 raft_consensus.cc:740] T f9a7e52d74da4cd79abac3858391ae77 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91778255eb264fe6a6dc208998e849ba, State: Initialized, Role: FOLLOWER
I20260709 13:12:17.650144 32188 consensus_queue.cc:260] T f9a7e52d74da4cd79abac3858391ae77 P 91778255eb264fe6a6dc208998e849ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } }
I20260709 13:12:17.649152 32187 raft_consensus.cc:359] T d0a7cae5f3084d0ba50763cc87888571 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:17.651852 32187 raft_consensus.cc:385] T d0a7cae5f3084d0ba50763cc87888571 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:17.652262 32187 raft_consensus.cc:740] T d0a7cae5f3084d0ba50763cc87888571 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31690560715140e6847173104a2b0f2e, State: Initialized, Role: FOLLOWER
I20260709 13:12:17.653414 32187 consensus_queue.cc:260] T d0a7cae5f3084d0ba50763cc87888571 P 31690560715140e6847173104a2b0f2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:17.656885 32097 heartbeater.cc:499] Master 127.21.63.62:36925 was elected leader, sending a full tablet report...
I20260709 13:12:17.661628 32187 ts_tablet_manager.cc:1434] T d0a7cae5f3084d0ba50763cc87888571 P 31690560715140e6847173104a2b0f2e: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.001s
I20260709 13:12:17.661446 32186 raft_consensus.cc:359] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:17.662657 32186 raft_consensus.cc:385] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:17.663053 32186 raft_consensus.cc:740] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e69f3153d090418380c201dddfef4c60, State: Initialized, Role: FOLLOWER
I20260709 13:12:17.664183 32186 consensus_queue.cc:260] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:17.663021 32187 tablet_bootstrap.cc:492] T 3a76c5023ae14f8ba25de09e3a9b60de P 31690560715140e6847173104a2b0f2e: Bootstrap starting.
I20260709 13:12:17.666872 32171 heartbeater.cc:499] Master 127.21.63.62:36925 was elected leader, sending a full tablet report...
I20260709 13:12:17.669726 32188 ts_tablet_manager.cc:1434] T f9a7e52d74da4cd79abac3858391ae77 P 91778255eb264fe6a6dc208998e849ba: Time spent starting tablet: real 0.025s	user 0.019s	sys 0.004s
I20260709 13:12:17.670215 32186 ts_tablet_manager.cc:1434] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60: Time spent starting tablet: real 0.024s	user 0.009s	sys 0.000s
I20260709 13:12:17.670549 32022 heartbeater.cc:499] Master 127.21.63.62:36925 was elected leader, sending a full tablet report...
I20260709 13:12:17.670826 32188 tablet_bootstrap.cc:492] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba: Bootstrap starting.
I20260709 13:12:17.671347 32186 tablet_bootstrap.cc:492] T 3a76c5023ae14f8ba25de09e3a9b60de P e69f3153d090418380c201dddfef4c60: Bootstrap starting.
I20260709 13:12:17.678117 32188 tablet_bootstrap.cc:654] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:17.683893 32188 tablet_bootstrap.cc:492] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba: No bootstrap required, opened a new log
I20260709 13:12:17.684425 32188 ts_tablet_manager.cc:1403] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba: Time spent bootstrapping tablet: real 0.014s	user 0.008s	sys 0.003s
I20260709 13:12:17.685684 32187 tablet_bootstrap.cc:654] T 3a76c5023ae14f8ba25de09e3a9b60de P 31690560715140e6847173104a2b0f2e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:17.687070 32186 tablet_bootstrap.cc:654] T 3a76c5023ae14f8ba25de09e3a9b60de P e69f3153d090418380c201dddfef4c60: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:17.687383 32188 raft_consensus.cc:359] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:17.688205 32188 raft_consensus.cc:385] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:17.688499 32188 raft_consensus.cc:740] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91778255eb264fe6a6dc208998e849ba, State: Initialized, Role: FOLLOWER
I20260709 13:12:17.689332 32188 consensus_queue.cc:260] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:17.691493 32188 ts_tablet_manager.cc:1434] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba: Time spent starting tablet: real 0.007s	user 0.000s	sys 0.004s
I20260709 13:12:17.692545 32188 tablet_bootstrap.cc:492] T 3a76c5023ae14f8ba25de09e3a9b60de P 91778255eb264fe6a6dc208998e849ba: Bootstrap starting.
I20260709 13:12:17.699339 32188 tablet_bootstrap.cc:654] T 3a76c5023ae14f8ba25de09e3a9b60de P 91778255eb264fe6a6dc208998e849ba: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:17.711463 32188 tablet_bootstrap.cc:492] T 3a76c5023ae14f8ba25de09e3a9b60de P 91778255eb264fe6a6dc208998e849ba: No bootstrap required, opened a new log
I20260709 13:12:17.711971 32188 ts_tablet_manager.cc:1403] T 3a76c5023ae14f8ba25de09e3a9b60de P 91778255eb264fe6a6dc208998e849ba: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.004s
I20260709 13:12:17.712975 32187 tablet_bootstrap.cc:492] T 3a76c5023ae14f8ba25de09e3a9b60de P 31690560715140e6847173104a2b0f2e: No bootstrap required, opened a new log
I20260709 13:12:17.713696 32186 tablet_bootstrap.cc:492] T 3a76c5023ae14f8ba25de09e3a9b60de P e69f3153d090418380c201dddfef4c60: No bootstrap required, opened a new log
I20260709 13:12:17.713721 32187 ts_tablet_manager.cc:1403] T 3a76c5023ae14f8ba25de09e3a9b60de P 31690560715140e6847173104a2b0f2e: Time spent bootstrapping tablet: real 0.051s	user 0.015s	sys 0.018s
I20260709 13:12:17.714659 32186 ts_tablet_manager.cc:1403] T 3a76c5023ae14f8ba25de09e3a9b60de P e69f3153d090418380c201dddfef4c60: Time spent bootstrapping tablet: real 0.044s	user 0.002s	sys 0.016s
I20260709 13:12:17.715161 32188 raft_consensus.cc:359] T 3a76c5023ae14f8ba25de09e3a9b60de P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:17.716063 32188 raft_consensus.cc:385] T 3a76c5023ae14f8ba25de09e3a9b60de P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:17.716454 32188 raft_consensus.cc:740] T 3a76c5023ae14f8ba25de09e3a9b60de P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91778255eb264fe6a6dc208998e849ba, State: Initialized, Role: FOLLOWER
I20260709 13:12:17.717589 32187 raft_consensus.cc:359] T 3a76c5023ae14f8ba25de09e3a9b60de P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:17.717623 32188 consensus_queue.cc:260] T 3a76c5023ae14f8ba25de09e3a9b60de P 91778255eb264fe6a6dc208998e849ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:17.718775 32187 raft_consensus.cc:385] T 3a76c5023ae14f8ba25de09e3a9b60de P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:17.718624 32186 raft_consensus.cc:359] T 3a76c5023ae14f8ba25de09e3a9b60de P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:17.719581 32187 raft_consensus.cc:740] T 3a76c5023ae14f8ba25de09e3a9b60de P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31690560715140e6847173104a2b0f2e, State: Initialized, Role: FOLLOWER
I20260709 13:12:17.719698 32186 raft_consensus.cc:385] T 3a76c5023ae14f8ba25de09e3a9b60de P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:17.720148 32186 raft_consensus.cc:740] T 3a76c5023ae14f8ba25de09e3a9b60de P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e69f3153d090418380c201dddfef4c60, State: Initialized, Role: FOLLOWER
I20260709 13:12:17.720377 32188 ts_tablet_manager.cc:1434] T 3a76c5023ae14f8ba25de09e3a9b60de P 91778255eb264fe6a6dc208998e849ba: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 13:12:17.720544 32187 consensus_queue.cc:260] T 3a76c5023ae14f8ba25de09e3a9b60de P 31690560715140e6847173104a2b0f2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:17.721185 32186 consensus_queue.cc:260] T 3a76c5023ae14f8ba25de09e3a9b60de P e69f3153d090418380c201dddfef4c60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:17.721801 32188 tablet_bootstrap.cc:492] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba: Bootstrap starting.
I20260709 13:12:17.723912 32186 ts_tablet_manager.cc:1434] T 3a76c5023ae14f8ba25de09e3a9b60de P e69f3153d090418380c201dddfef4c60: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 13:12:17.725121 32186 tablet_bootstrap.cc:492] T 7b2f10befb2e474fbe821af8d5e92fd5 P e69f3153d090418380c201dddfef4c60: Bootstrap starting.
I20260709 13:12:17.730000 32188 tablet_bootstrap.cc:654] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:17.731112 32187 ts_tablet_manager.cc:1434] T 3a76c5023ae14f8ba25de09e3a9b60de P 31690560715140e6847173104a2b0f2e: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:12:17.732782 32186 tablet_bootstrap.cc:654] T 7b2f10befb2e474fbe821af8d5e92fd5 P e69f3153d090418380c201dddfef4c60: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:17.741724 32188 tablet_bootstrap.cc:492] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba: No bootstrap required, opened a new log
I20260709 13:12:17.742180 32188 ts_tablet_manager.cc:1403] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.004s
I20260709 13:12:17.743947 32187 tablet_bootstrap.cc:492] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 31690560715140e6847173104a2b0f2e: Bootstrap starting.
I20260709 13:12:17.745309 32186 tablet_bootstrap.cc:492] T 7b2f10befb2e474fbe821af8d5e92fd5 P e69f3153d090418380c201dddfef4c60: No bootstrap required, opened a new log
I20260709 13:12:17.745093 32188 raft_consensus.cc:359] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:17.746011 32188 raft_consensus.cc:385] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:17.746138 32186 ts_tablet_manager.cc:1403] T 7b2f10befb2e474fbe821af8d5e92fd5 P e69f3153d090418380c201dddfef4c60: Time spent bootstrapping tablet: real 0.021s	user 0.006s	sys 0.013s
I20260709 13:12:17.746345 32188 raft_consensus.cc:740] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91778255eb264fe6a6dc208998e849ba, State: Initialized, Role: FOLLOWER
I20260709 13:12:17.747385 32188 consensus_queue.cc:260] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:17.749878 32188 ts_tablet_manager.cc:1434] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:12:17.749670 32186 raft_consensus.cc:359] T 7b2f10befb2e474fbe821af8d5e92fd5 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:17.750553 32186 raft_consensus.cc:385] T 7b2f10befb2e474fbe821af8d5e92fd5 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:17.750967 32188 tablet_bootstrap.cc:492] T 7b2f10befb2e474fbe821af8d5e92fd5 P 91778255eb264fe6a6dc208998e849ba: Bootstrap starting.
I20260709 13:12:17.750913 32186 raft_consensus.cc:740] T 7b2f10befb2e474fbe821af8d5e92fd5 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e69f3153d090418380c201dddfef4c60, State: Initialized, Role: FOLLOWER
I20260709 13:12:17.751838 32186 consensus_queue.cc:260] T 7b2f10befb2e474fbe821af8d5e92fd5 P e69f3153d090418380c201dddfef4c60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:17.752388 32187 tablet_bootstrap.cc:654] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 31690560715140e6847173104a2b0f2e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:17.754554 32186 ts_tablet_manager.cc:1434] T 7b2f10befb2e474fbe821af8d5e92fd5 P e69f3153d090418380c201dddfef4c60: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 13:12:17.755718 32186 tablet_bootstrap.cc:492] T d1fc79d86bee43c5b062ced8a090aab1 P e69f3153d090418380c201dddfef4c60: Bootstrap starting.
W20260709 13:12:17.756547 32172 tablet.cc:2406] T 3a76c5023ae14f8ba25de09e3a9b60de P e69f3153d090418380c201dddfef4c60: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:12:17.757550 32188 tablet_bootstrap.cc:654] T 7b2f10befb2e474fbe821af8d5e92fd5 P 91778255eb264fe6a6dc208998e849ba: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:17.765633 32186 tablet_bootstrap.cc:654] T d1fc79d86bee43c5b062ced8a090aab1 P e69f3153d090418380c201dddfef4c60: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:17.770362 32188 tablet_bootstrap.cc:492] T 7b2f10befb2e474fbe821af8d5e92fd5 P 91778255eb264fe6a6dc208998e849ba: No bootstrap required, opened a new log
I20260709 13:12:17.771255 32188 ts_tablet_manager.cc:1403] T 7b2f10befb2e474fbe821af8d5e92fd5 P 91778255eb264fe6a6dc208998e849ba: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.003s
I20260709 13:12:17.776867 32188 raft_consensus.cc:359] T 7b2f10befb2e474fbe821af8d5e92fd5 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:17.778165 32188 raft_consensus.cc:385] T 7b2f10befb2e474fbe821af8d5e92fd5 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:17.778491 32188 raft_consensus.cc:740] T 7b2f10befb2e474fbe821af8d5e92fd5 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91778255eb264fe6a6dc208998e849ba, State: Initialized, Role: FOLLOWER
I20260709 13:12:17.779402 32188 consensus_queue.cc:260] T 7b2f10befb2e474fbe821af8d5e92fd5 P 91778255eb264fe6a6dc208998e849ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:17.781805 32188 ts_tablet_manager.cc:1434] T 7b2f10befb2e474fbe821af8d5e92fd5 P 91778255eb264fe6a6dc208998e849ba: Time spent starting tablet: real 0.010s	user 0.011s	sys 0.000s
I20260709 13:12:17.783079 32188 tablet_bootstrap.cc:492] T d0a7cae5f3084d0ba50763cc87888571 P 91778255eb264fe6a6dc208998e849ba: Bootstrap starting.
I20260709 13:12:17.786224 32186 tablet_bootstrap.cc:492] T d1fc79d86bee43c5b062ced8a090aab1 P e69f3153d090418380c201dddfef4c60: No bootstrap required, opened a new log
I20260709 13:12:17.786913 32186 ts_tablet_manager.cc:1403] T d1fc79d86bee43c5b062ced8a090aab1 P e69f3153d090418380c201dddfef4c60: Time spent bootstrapping tablet: real 0.031s	user 0.016s	sys 0.000s
I20260709 13:12:17.789688 32188 tablet_bootstrap.cc:654] T d0a7cae5f3084d0ba50763cc87888571 P 91778255eb264fe6a6dc208998e849ba: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:17.790184 32186 raft_consensus.cc:359] T d1fc79d86bee43c5b062ced8a090aab1 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:17.791003 32186 raft_consensus.cc:385] T d1fc79d86bee43c5b062ced8a090aab1 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:17.791407 32186 raft_consensus.cc:740] T d1fc79d86bee43c5b062ced8a090aab1 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e69f3153d090418380c201dddfef4c60, State: Initialized, Role: FOLLOWER
I20260709 13:12:17.792356 32186 consensus_queue.cc:260] T d1fc79d86bee43c5b062ced8a090aab1 P e69f3153d090418380c201dddfef4c60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:17.795517 32186 ts_tablet_manager.cc:1434] T d1fc79d86bee43c5b062ced8a090aab1 P e69f3153d090418380c201dddfef4c60: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:12:17.797005 32186 tablet_bootstrap.cc:492] T d0a7cae5f3084d0ba50763cc87888571 P e69f3153d090418380c201dddfef4c60: Bootstrap starting.
I20260709 13:12:17.801736 32188 tablet_bootstrap.cc:492] T d0a7cae5f3084d0ba50763cc87888571 P 91778255eb264fe6a6dc208998e849ba: No bootstrap required, opened a new log
I20260709 13:12:17.802310 32188 ts_tablet_manager.cc:1403] T d0a7cae5f3084d0ba50763cc87888571 P 91778255eb264fe6a6dc208998e849ba: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.004s
I20260709 13:12:17.803872 32186 tablet_bootstrap.cc:654] T d0a7cae5f3084d0ba50763cc87888571 P e69f3153d090418380c201dddfef4c60: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:17.805371 32188 raft_consensus.cc:359] T d0a7cae5f3084d0ba50763cc87888571 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:17.806165 32188 raft_consensus.cc:385] T d0a7cae5f3084d0ba50763cc87888571 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:17.806465 32188 raft_consensus.cc:740] T d0a7cae5f3084d0ba50763cc87888571 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91778255eb264fe6a6dc208998e849ba, State: Initialized, Role: FOLLOWER
I20260709 13:12:17.807611 32188 consensus_queue.cc:260] T d0a7cae5f3084d0ba50763cc87888571 P 91778255eb264fe6a6dc208998e849ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:17.817463 32188 ts_tablet_manager.cc:1434] T d0a7cae5f3084d0ba50763cc87888571 P 91778255eb264fe6a6dc208998e849ba: Time spent starting tablet: real 0.015s	user 0.003s	sys 0.006s
I20260709 13:12:17.818555 32188 tablet_bootstrap.cc:492] T 0dfa6288e543429c99109ebbd5a223b9 P 91778255eb264fe6a6dc208998e849ba: Bootstrap starting.
I20260709 13:12:17.819583 32187 tablet_bootstrap.cc:492] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 31690560715140e6847173104a2b0f2e: No bootstrap required, opened a new log
I20260709 13:12:17.820125 32187 ts_tablet_manager.cc:1403] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 31690560715140e6847173104a2b0f2e: Time spent bootstrapping tablet: real 0.076s	user 0.024s	sys 0.048s
I20260709 13:12:17.821086 32186 tablet_bootstrap.cc:492] T d0a7cae5f3084d0ba50763cc87888571 P e69f3153d090418380c201dddfef4c60: No bootstrap required, opened a new log
I20260709 13:12:17.821662 32186 ts_tablet_manager.cc:1403] T d0a7cae5f3084d0ba50763cc87888571 P e69f3153d090418380c201dddfef4c60: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.007s
I20260709 13:12:17.823262 32187 raft_consensus.cc:359] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:17.824043 32187 raft_consensus.cc:385] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:17.824337 32187 raft_consensus.cc:740] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31690560715140e6847173104a2b0f2e, State: Initialized, Role: FOLLOWER
I20260709 13:12:17.824891 32188 tablet_bootstrap.cc:654] T 0dfa6288e543429c99109ebbd5a223b9 P 91778255eb264fe6a6dc208998e849ba: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:17.824564 32186 raft_consensus.cc:359] T d0a7cae5f3084d0ba50763cc87888571 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:17.825353 32186 raft_consensus.cc:385] T d0a7cae5f3084d0ba50763cc87888571 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:17.825187 32187 consensus_queue.cc:260] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 31690560715140e6847173104a2b0f2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:17.825718 32186 raft_consensus.cc:740] T d0a7cae5f3084d0ba50763cc87888571 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e69f3153d090418380c201dddfef4c60, State: Initialized, Role: FOLLOWER
I20260709 13:12:17.826704 32186 consensus_queue.cc:260] T d0a7cae5f3084d0ba50763cc87888571 P e69f3153d090418380c201dddfef4c60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:17.827576 32187 ts_tablet_manager.cc:1434] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 31690560715140e6847173104a2b0f2e: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 13:12:17.828549 32187 tablet_bootstrap.cc:492] T 7b2f10befb2e474fbe821af8d5e92fd5 P 31690560715140e6847173104a2b0f2e: Bootstrap starting.
I20260709 13:12:17.829902 32186 ts_tablet_manager.cc:1434] T d0a7cae5f3084d0ba50763cc87888571 P e69f3153d090418380c201dddfef4c60: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.003s
I20260709 13:12:17.831235 32186 tablet_bootstrap.cc:492] T b40f21aa6dd54f64bed8e716a77df4d6 P e69f3153d090418380c201dddfef4c60: Bootstrap starting.
I20260709 13:12:17.836337 32188 tablet_bootstrap.cc:492] T 0dfa6288e543429c99109ebbd5a223b9 P 91778255eb264fe6a6dc208998e849ba: No bootstrap required, opened a new log
I20260709 13:12:17.836905 32188 ts_tablet_manager.cc:1403] T 0dfa6288e543429c99109ebbd5a223b9 P 91778255eb264fe6a6dc208998e849ba: Time spent bootstrapping tablet: real 0.019s	user 0.002s	sys 0.015s
I20260709 13:12:17.838210 32186 tablet_bootstrap.cc:654] T b40f21aa6dd54f64bed8e716a77df4d6 P e69f3153d090418380c201dddfef4c60: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:17.839820 32188 raft_consensus.cc:359] T 0dfa6288e543429c99109ebbd5a223b9 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:17.840602 32188 raft_consensus.cc:385] T 0dfa6288e543429c99109ebbd5a223b9 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:17.841073 32188 raft_consensus.cc:740] T 0dfa6288e543429c99109ebbd5a223b9 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91778255eb264fe6a6dc208998e849ba, State: Initialized, Role: FOLLOWER
I20260709 13:12:17.842219 32188 consensus_queue.cc:260] T 0dfa6288e543429c99109ebbd5a223b9 P 91778255eb264fe6a6dc208998e849ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:17.844909 32187 tablet_bootstrap.cc:654] T 7b2f10befb2e474fbe821af8d5e92fd5 P 31690560715140e6847173104a2b0f2e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:17.845500 32188 ts_tablet_manager.cc:1434] T 0dfa6288e543429c99109ebbd5a223b9 P 91778255eb264fe6a6dc208998e849ba: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.005s
I20260709 13:12:17.846966 32188 tablet_bootstrap.cc:492] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba: Bootstrap starting.
I20260709 13:12:17.850173 32186 tablet_bootstrap.cc:492] T b40f21aa6dd54f64bed8e716a77df4d6 P e69f3153d090418380c201dddfef4c60: No bootstrap required, opened a new log
I20260709 13:12:17.850770 32186 ts_tablet_manager.cc:1403] T b40f21aa6dd54f64bed8e716a77df4d6 P e69f3153d090418380c201dddfef4c60: Time spent bootstrapping tablet: real 0.020s	user 0.009s	sys 0.008s
I20260709 13:12:17.853749 32188 tablet_bootstrap.cc:654] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:17.855072 32186 raft_consensus.cc:359] T b40f21aa6dd54f64bed8e716a77df4d6 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:17.855845 32186 raft_consensus.cc:385] T b40f21aa6dd54f64bed8e716a77df4d6 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:17.856132 32186 raft_consensus.cc:740] T b40f21aa6dd54f64bed8e716a77df4d6 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e69f3153d090418380c201dddfef4c60, State: Initialized, Role: FOLLOWER
W20260709 13:12:17.856597 31943 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:12:17.857134 32186 consensus_queue.cc:260] T b40f21aa6dd54f64bed8e716a77df4d6 P e69f3153d090418380c201dddfef4c60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:17.862445 32186 ts_tablet_manager.cc:1434] T b40f21aa6dd54f64bed8e716a77df4d6 P e69f3153d090418380c201dddfef4c60: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:12:17.863538 32186 tablet_bootstrap.cc:492] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60: Bootstrap starting.
I20260709 13:12:17.863583 32188 tablet_bootstrap.cc:492] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba: No bootstrap required, opened a new log
I20260709 13:12:17.864218 32188 ts_tablet_manager.cc:1403] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba: Time spent bootstrapping tablet: real 0.018s	user 0.009s	sys 0.006s
I20260709 13:12:17.866734 32188 raft_consensus.cc:359] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:17.867504 32188 raft_consensus.cc:385] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:17.867782 32188 raft_consensus.cc:740] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91778255eb264fe6a6dc208998e849ba, State: Initialized, Role: FOLLOWER
I20260709 13:12:17.866308 32187 tablet_bootstrap.cc:492] T 7b2f10befb2e474fbe821af8d5e92fd5 P 31690560715140e6847173104a2b0f2e: No bootstrap required, opened a new log
I20260709 13:12:17.868552 32188 consensus_queue.cc:260] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:17.869391 32187 ts_tablet_manager.cc:1403] T 7b2f10befb2e474fbe821af8d5e92fd5 P 31690560715140e6847173104a2b0f2e: Time spent bootstrapping tablet: real 0.041s	user 0.013s	sys 0.006s
I20260709 13:12:17.870556 32186 tablet_bootstrap.cc:654] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:17.870937 32188 ts_tablet_manager.cc:1434] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba: Time spent starting tablet: real 0.006s	user 0.002s	sys 0.003s
I20260709 13:12:17.871960 32188 tablet_bootstrap.cc:492] T 709f871588d74579bab67c29aca59135 P 91778255eb264fe6a6dc208998e849ba: Bootstrap starting.
I20260709 13:12:17.876968 32187 raft_consensus.cc:359] T 7b2f10befb2e474fbe821af8d5e92fd5 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:17.878188 32187 raft_consensus.cc:385] T 7b2f10befb2e474fbe821af8d5e92fd5 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:17.878531 32187 raft_consensus.cc:740] T 7b2f10befb2e474fbe821af8d5e92fd5 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31690560715140e6847173104a2b0f2e, State: Initialized, Role: FOLLOWER
I20260709 13:12:17.879521 32187 consensus_queue.cc:260] T 7b2f10befb2e474fbe821af8d5e92fd5 P 31690560715140e6847173104a2b0f2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:17.880388 32188 tablet_bootstrap.cc:654] T 709f871588d74579bab67c29aca59135 P 91778255eb264fe6a6dc208998e849ba: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:17.882066 32186 tablet_bootstrap.cc:492] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60: No bootstrap required, opened a new log
I20260709 13:12:17.882618 32186 ts_tablet_manager.cc:1403] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60: Time spent bootstrapping tablet: real 0.019s	user 0.008s	sys 0.008s
I20260709 13:12:17.888890 32186 raft_consensus.cc:359] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } }
I20260709 13:12:17.889812 32187 ts_tablet_manager.cc:1434] T 7b2f10befb2e474fbe821af8d5e92fd5 P 31690560715140e6847173104a2b0f2e: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 13:12:17.889829 32186 raft_consensus.cc:385] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:17.890493 32186 raft_consensus.cc:740] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e69f3153d090418380c201dddfef4c60, State: Initialized, Role: FOLLOWER
I20260709 13:12:17.891227 32187 tablet_bootstrap.cc:492] T f9a7e52d74da4cd79abac3858391ae77 P 31690560715140e6847173104a2b0f2e: Bootstrap starting.
I20260709 13:12:17.891664 32188 tablet_bootstrap.cc:492] T 709f871588d74579bab67c29aca59135 P 91778255eb264fe6a6dc208998e849ba: No bootstrap required, opened a new log
I20260709 13:12:17.891364 32186 consensus_queue.cc:260] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } }
I20260709 13:12:17.892251 32188 ts_tablet_manager.cc:1403] T 709f871588d74579bab67c29aca59135 P 91778255eb264fe6a6dc208998e849ba: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.008s
I20260709 13:12:17.893534 32192 raft_consensus.cc:493] T 0dfa6288e543429c99109ebbd5a223b9 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:17.894053 32192 raft_consensus.cc:515] T 0dfa6288e543429c99109ebbd5a223b9 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:17.896653 32186 ts_tablet_manager.cc:1434] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.001s
I20260709 13:12:17.898002 32186 tablet_bootstrap.cc:492] T 709f871588d74579bab67c29aca59135 P e69f3153d090418380c201dddfef4c60: Bootstrap starting.
I20260709 13:12:17.900110 32192 leader_election.cc:290] T 0dfa6288e543429c99109ebbd5a223b9 P 91778255eb264fe6a6dc208998e849ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31690560715140e6847173104a2b0f2e (127.21.63.2:44689), e69f3153d090418380c201dddfef4c60 (127.21.63.3:40639)
I20260709 13:12:17.905390 32186 tablet_bootstrap.cc:654] T 709f871588d74579bab67c29aca59135 P e69f3153d090418380c201dddfef4c60: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:17.907946 32188 raft_consensus.cc:359] T 709f871588d74579bab67c29aca59135 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:17.911845 32188 raft_consensus.cc:385] T 709f871588d74579bab67c29aca59135 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:17.912845 32188 raft_consensus.cc:740] T 709f871588d74579bab67c29aca59135 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91778255eb264fe6a6dc208998e849ba, State: Initialized, Role: FOLLOWER
I20260709 13:12:17.914681 32188 consensus_queue.cc:260] T 709f871588d74579bab67c29aca59135 P 91778255eb264fe6a6dc208998e849ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:17.911916 32187 tablet_bootstrap.cc:654] T f9a7e52d74da4cd79abac3858391ae77 P 31690560715140e6847173104a2b0f2e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:17.922091 32188 ts_tablet_manager.cc:1434] T 709f871588d74579bab67c29aca59135 P 91778255eb264fe6a6dc208998e849ba: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.000s
I20260709 13:12:17.923502 32188 tablet_bootstrap.cc:492] T b40f21aa6dd54f64bed8e716a77df4d6 P 91778255eb264fe6a6dc208998e849ba: Bootstrap starting.
I20260709 13:12:17.931771 32188 tablet_bootstrap.cc:654] T b40f21aa6dd54f64bed8e716a77df4d6 P 91778255eb264fe6a6dc208998e849ba: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:17.933059 32072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dfa6288e543429c99109ebbd5a223b9" candidate_uuid: "91778255eb264fe6a6dc208998e849ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31690560715140e6847173104a2b0f2e" is_pre_election: true
I20260709 13:12:17.936127 32192 raft_consensus.cc:493] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:17.936951 32192 raft_consensus.cc:515] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:17.937670 32187 tablet_bootstrap.cc:492] T f9a7e52d74da4cd79abac3858391ae77 P 31690560715140e6847173104a2b0f2e: No bootstrap required, opened a new log
I20260709 13:12:17.938520 32187 ts_tablet_manager.cc:1403] T f9a7e52d74da4cd79abac3858391ae77 P 31690560715140e6847173104a2b0f2e: Time spent bootstrapping tablet: real 0.048s	user 0.014s	sys 0.003s
I20260709 13:12:17.940080 32192 leader_election.cc:290] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31690560715140e6847173104a2b0f2e (127.21.63.2:44689), e69f3153d090418380c201dddfef4c60 (127.21.63.3:40639)
W20260709 13:12:17.941172 31962 leader_election.cc:343] T 0dfa6288e543429c99109ebbd5a223b9 P 91778255eb264fe6a6dc208998e849ba [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 31690560715140e6847173104a2b0f2e (127.21.63.2:44689): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:17.944033 32072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1fc79d86bee43c5b062ced8a090aab1" candidate_uuid: "91778255eb264fe6a6dc208998e849ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31690560715140e6847173104a2b0f2e" is_pre_election: true
I20260709 13:12:17.944851 32187 raft_consensus.cc:359] T f9a7e52d74da4cd79abac3858391ae77 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } }
I20260709 13:12:17.946112 32193 raft_consensus.cc:493] T 3a76c5023ae14f8ba25de09e3a9b60de P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:17.947616 32193 raft_consensus.cc:515] T 3a76c5023ae14f8ba25de09e3a9b60de P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
W20260709 13:12:17.954597 31962 leader_election.cc:343] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 31690560715140e6847173104a2b0f2e (127.21.63.2:44689): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:17.946182 32187 raft_consensus.cc:385] T f9a7e52d74da4cd79abac3858391ae77 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:17.960794 32187 raft_consensus.cc:740] T f9a7e52d74da4cd79abac3858391ae77 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31690560715140e6847173104a2b0f2e, State: Initialized, Role: FOLLOWER
I20260709 13:12:17.963095 32187 consensus_queue.cc:260] T f9a7e52d74da4cd79abac3858391ae77 P 31690560715140e6847173104a2b0f2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } }
I20260709 13:12:17.967031 32186 tablet_bootstrap.cc:492] T 709f871588d74579bab67c29aca59135 P e69f3153d090418380c201dddfef4c60: No bootstrap required, opened a new log
I20260709 13:12:17.968039 32186 ts_tablet_manager.cc:1403] T 709f871588d74579bab67c29aca59135 P e69f3153d090418380c201dddfef4c60: Time spent bootstrapping tablet: real 0.070s	user 0.024s	sys 0.035s
I20260709 13:12:17.968189 32188 tablet_bootstrap.cc:492] T b40f21aa6dd54f64bed8e716a77df4d6 P 91778255eb264fe6a6dc208998e849ba: No bootstrap required, opened a new log
I20260709 13:12:17.969010 32188 ts_tablet_manager.cc:1403] T b40f21aa6dd54f64bed8e716a77df4d6 P 91778255eb264fe6a6dc208998e849ba: Time spent bootstrapping tablet: real 0.046s	user 0.026s	sys 0.011s
I20260709 13:12:17.964746 32193 leader_election.cc:290] T 3a76c5023ae14f8ba25de09e3a9b60de P 31690560715140e6847173104a2b0f2e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979), e69f3153d090418380c201dddfef4c60 (127.21.63.3:40639)
I20260709 13:12:17.979935 32188 raft_consensus.cc:359] T b40f21aa6dd54f64bed8e716a77df4d6 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:17.980448 32192 raft_consensus.cc:493] T 3a76c5023ae14f8ba25de09e3a9b60de P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:17.981153 32192 raft_consensus.cc:515] T 3a76c5023ae14f8ba25de09e3a9b60de P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:17.984361 32192 leader_election.cc:290] T 3a76c5023ae14f8ba25de09e3a9b60de P 91778255eb264fe6a6dc208998e849ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31690560715140e6847173104a2b0f2e (127.21.63.2:44689), e69f3153d090418380c201dddfef4c60 (127.21.63.3:40639)
I20260709 13:12:17.980696 32188 raft_consensus.cc:385] T b40f21aa6dd54f64bed8e716a77df4d6 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:17.986536 32188 raft_consensus.cc:740] T b40f21aa6dd54f64bed8e716a77df4d6 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91778255eb264fe6a6dc208998e849ba, State: Initialized, Role: FOLLOWER
I20260709 13:12:17.983120 32186 raft_consensus.cc:359] T 709f871588d74579bab67c29aca59135 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:17.987464 32186 raft_consensus.cc:385] T 709f871588d74579bab67c29aca59135 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:17.987743 32186 raft_consensus.cc:740] T 709f871588d74579bab67c29aca59135 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e69f3153d090418380c201dddfef4c60, State: Initialized, Role: FOLLOWER
I20260709 13:12:17.988415 32186 consensus_queue.cc:260] T 709f871588d74579bab67c29aca59135 P e69f3153d090418380c201dddfef4c60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:17.988437 32188 consensus_queue.cc:260] T b40f21aa6dd54f64bed8e716a77df4d6 P 91778255eb264fe6a6dc208998e849ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:17.990698 32186 ts_tablet_manager.cc:1434] T 709f871588d74579bab67c29aca59135 P e69f3153d090418380c201dddfef4c60: Time spent starting tablet: real 0.022s	user 0.004s	sys 0.000s
I20260709 13:12:17.991776 32186 tablet_bootstrap.cc:492] T 0dfa6288e543429c99109ebbd5a223b9 P e69f3153d090418380c201dddfef4c60: Bootstrap starting.
I20260709 13:12:17.993439 32188 ts_tablet_manager.cc:1434] T b40f21aa6dd54f64bed8e716a77df4d6 P 91778255eb264fe6a6dc208998e849ba: Time spent starting tablet: real 0.024s	user 0.006s	sys 0.001s
I20260709 13:12:17.993710 32072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a76c5023ae14f8ba25de09e3a9b60de" candidate_uuid: "91778255eb264fe6a6dc208998e849ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31690560715140e6847173104a2b0f2e" is_pre_election: true
I20260709 13:12:17.994448 32194 raft_consensus.cc:493] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:18.018502 32187 ts_tablet_manager.cc:1434] T f9a7e52d74da4cd79abac3858391ae77 P 31690560715140e6847173104a2b0f2e: Time spent starting tablet: real 0.080s	user 0.013s	sys 0.012s
I20260709 13:12:18.017280 32144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a76c5023ae14f8ba25de09e3a9b60de" candidate_uuid: "91778255eb264fe6a6dc208998e849ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e69f3153d090418380c201dddfef4c60" is_pre_election: true
I20260709 13:12:18.026218 32144 raft_consensus.cc:2468] T 3a76c5023ae14f8ba25de09e3a9b60de P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91778255eb264fe6a6dc208998e849ba in term 0.
I20260709 13:12:18.025679 32072 raft_consensus.cc:2468] T 3a76c5023ae14f8ba25de09e3a9b60de P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91778255eb264fe6a6dc208998e849ba in term 0.
I20260709 13:12:18.026710 32187 tablet_bootstrap.cc:492] T d1fc79d86bee43c5b062ced8a090aab1 P 31690560715140e6847173104a2b0f2e: Bootstrap starting.
I20260709 13:12:18.015904 32146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dfa6288e543429c99109ebbd5a223b9" candidate_uuid: "91778255eb264fe6a6dc208998e849ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e69f3153d090418380c201dddfef4c60" is_pre_election: true
I20260709 13:12:18.027813 32194 raft_consensus.cc:515] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } }
I20260709 13:12:18.016628 32145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1fc79d86bee43c5b062ced8a090aab1" candidate_uuid: "91778255eb264fe6a6dc208998e849ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e69f3153d090418380c201dddfef4c60" is_pre_election: true
I20260709 13:12:18.029281 32145 raft_consensus.cc:2468] T d1fc79d86bee43c5b062ced8a090aab1 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91778255eb264fe6a6dc208998e849ba in term 0.
I20260709 13:12:18.030055 31997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a76c5023ae14f8ba25de09e3a9b60de" candidate_uuid: "31690560715140e6847173104a2b0f2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91778255eb264fe6a6dc208998e849ba" is_pre_election: true
I20260709 13:12:18.030791 31997 raft_consensus.cc:2468] T 3a76c5023ae14f8ba25de09e3a9b60de P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31690560715140e6847173104a2b0f2e in term 0.
I20260709 13:12:18.002573 32186 tablet_bootstrap.cc:654] T 0dfa6288e543429c99109ebbd5a223b9 P e69f3153d090418380c201dddfef4c60: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:18.031430 31961 leader_election.cc:304] T 3a76c5023ae14f8ba25de09e3a9b60de P 91778255eb264fe6a6dc208998e849ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91778255eb264fe6a6dc208998e849ba, e69f3153d090418380c201dddfef4c60; no voters: 
I20260709 13:12:18.037184 32192 raft_consensus.cc:2804] T 3a76c5023ae14f8ba25de09e3a9b60de P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:18.041141 32192 raft_consensus.cc:493] T 3a76c5023ae14f8ba25de09e3a9b60de P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:18.041611 32192 raft_consensus.cc:3060] T 3a76c5023ae14f8ba25de09e3a9b60de P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:18.045612 32037 leader_election.cc:304] T 3a76c5023ae14f8ba25de09e3a9b60de P 31690560715140e6847173104a2b0f2e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31690560715140e6847173104a2b0f2e, 91778255eb264fe6a6dc208998e849ba; no voters: 
I20260709 13:12:18.047122 32222 raft_consensus.cc:2804] T 3a76c5023ae14f8ba25de09e3a9b60de P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:18.047621 32222 raft_consensus.cc:493] T 3a76c5023ae14f8ba25de09e3a9b60de P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:18.048031 32222 raft_consensus.cc:3060] T 3a76c5023ae14f8ba25de09e3a9b60de P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:18.058941 32194 leader_election.cc:290] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979), 31690560715140e6847173104a2b0f2e (127.21.63.2:44689)
I20260709 13:12:18.062150 32194 raft_consensus.cc:493] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:18.063107 32187 tablet_bootstrap.cc:654] T d1fc79d86bee43c5b062ced8a090aab1 P 31690560715140e6847173104a2b0f2e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:18.062732 32194 raft_consensus.cc:515] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.065099 32145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a76c5023ae14f8ba25de09e3a9b60de" candidate_uuid: "31690560715140e6847173104a2b0f2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e69f3153d090418380c201dddfef4c60" is_pre_election: true
I20260709 13:12:18.065424 32194 leader_election.cc:290] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979), 31690560715140e6847173104a2b0f2e (127.21.63.2:44689)
I20260709 13:12:18.065793 32145 raft_consensus.cc:2468] T 3a76c5023ae14f8ba25de09e3a9b60de P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31690560715140e6847173104a2b0f2e in term 0.
I20260709 13:12:18.066358 32225 raft_consensus.cc:493] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:18.067153 32225 raft_consensus.cc:515] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.070703 32072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ea7beb6ca5f4de4b64c0e8a9e9d99a7" candidate_uuid: "91778255eb264fe6a6dc208998e849ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31690560715140e6847173104a2b0f2e" is_pre_election: true
I20260709 13:12:18.071341 32072 raft_consensus.cc:2468] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91778255eb264fe6a6dc208998e849ba in term 0.
W20260709 13:12:18.056442 31961 leader_election.cc:343] T 0dfa6288e543429c99109ebbd5a223b9 P 91778255eb264fe6a6dc208998e849ba [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e69f3153d090418380c201dddfef4c60 (127.21.63.3:40639): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:18.080008 32194 raft_consensus.cc:493] T b40f21aa6dd54f64bed8e716a77df4d6 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:18.080611 32194 raft_consensus.cc:515] T b40f21aa6dd54f64bed8e716a77df4d6 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.082685 32194 leader_election.cc:290] T b40f21aa6dd54f64bed8e716a77df4d6 P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979), 31690560715140e6847173104a2b0f2e (127.21.63.2:44689)
I20260709 13:12:18.077580 31962 leader_election.cc:304] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31690560715140e6847173104a2b0f2e, 91778255eb264fe6a6dc208998e849ba; no voters: 
I20260709 13:12:18.083650 32192 raft_consensus.cc:515] T 3a76c5023ae14f8ba25de09e3a9b60de P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.078541 31961 leader_election.cc:304] T 0dfa6288e543429c99109ebbd5a223b9 P 91778255eb264fe6a6dc208998e849ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91778255eb264fe6a6dc208998e849ba; no voters: 31690560715140e6847173104a2b0f2e, e69f3153d090418380c201dddfef4c60
I20260709 13:12:18.092527 32225 leader_election.cc:290] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31690560715140e6847173104a2b0f2e (127.21.63.2:44689), e69f3153d090418380c201dddfef4c60 (127.21.63.3:40639)
I20260709 13:12:18.089006 32192 leader_election.cc:290] T 3a76c5023ae14f8ba25de09e3a9b60de P 91778255eb264fe6a6dc208998e849ba [CANDIDATE]: Term 1 election: Requested vote from peers 31690560715140e6847173104a2b0f2e (127.21.63.2:44689), e69f3153d090418380c201dddfef4c60 (127.21.63.3:40639)
I20260709 13:12:18.093156 32225 raft_consensus.cc:2804] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:18.097194 32225 raft_consensus.cc:493] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:18.097584 32225 raft_consensus.cc:3060] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:18.099287 32222 raft_consensus.cc:515] T 3a76c5023ae14f8ba25de09e3a9b60de P 31690560715140e6847173104a2b0f2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.099871 31997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9a7e52d74da4cd79abac3858391ae77" candidate_uuid: "e69f3153d090418380c201dddfef4c60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91778255eb264fe6a6dc208998e849ba" is_pre_election: true
I20260709 13:12:18.100613 32193 raft_consensus.cc:493] T 7b2f10befb2e474fbe821af8d5e92fd5 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:18.100728 31997 raft_consensus.cc:2468] T f9a7e52d74da4cd79abac3858391ae77 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e69f3153d090418380c201dddfef4c60 in term 0.
I20260709 13:12:18.102342 31996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ea7beb6ca5f4de4b64c0e8a9e9d99a7" candidate_uuid: "e69f3153d090418380c201dddfef4c60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91778255eb264fe6a6dc208998e849ba" is_pre_election: true
I20260709 13:12:18.102698 32222 leader_election.cc:290] T 3a76c5023ae14f8ba25de09e3a9b60de P 31690560715140e6847173104a2b0f2e [CANDIDATE]: Term 1 election: Requested vote from peers 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979), e69f3153d090418380c201dddfef4c60 (127.21.63.3:40639)
I20260709 13:12:18.104243 32145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a76c5023ae14f8ba25de09e3a9b60de" candidate_uuid: "31690560715140e6847173104a2b0f2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e69f3153d090418380c201dddfef4c60"
I20260709 13:12:18.104882 32145 raft_consensus.cc:3060] T 3a76c5023ae14f8ba25de09e3a9b60de P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:18.105634 32186 tablet_bootstrap.cc:492] T 0dfa6288e543429c99109ebbd5a223b9 P e69f3153d090418380c201dddfef4c60: No bootstrap required, opened a new log
I20260709 13:12:18.106348 32186 ts_tablet_manager.cc:1403] T 0dfa6288e543429c99109ebbd5a223b9 P e69f3153d090418380c201dddfef4c60: Time spent bootstrapping tablet: real 0.115s	user 0.005s	sys 0.024s
I20260709 13:12:18.105494 32193 raft_consensus.cc:515] T 7b2f10befb2e474fbe821af8d5e92fd5 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.107802 31997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40f21aa6dd54f64bed8e716a77df4d6" candidate_uuid: "e69f3153d090418380c201dddfef4c60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91778255eb264fe6a6dc208998e849ba" is_pre_election: true
I20260709 13:12:18.108423 31997 raft_consensus.cc:2468] T b40f21aa6dd54f64bed8e716a77df4d6 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e69f3153d090418380c201dddfef4c60 in term 0.
I20260709 13:12:18.109401 32186 raft_consensus.cc:359] T 0dfa6288e543429c99109ebbd5a223b9 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.110105 32186 raft_consensus.cc:385] T 0dfa6288e543429c99109ebbd5a223b9 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:18.103658 32192 raft_consensus.cc:2749] T 0dfa6288e543429c99109ebbd5a223b9 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:18.110471 32186 raft_consensus.cc:740] T 0dfa6288e543429c99109ebbd5a223b9 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e69f3153d090418380c201dddfef4c60, State: Initialized, Role: FOLLOWER
I20260709 13:12:18.111238 32146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b2f10befb2e474fbe821af8d5e92fd5" candidate_uuid: "31690560715140e6847173104a2b0f2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e69f3153d090418380c201dddfef4c60" is_pre_election: true
I20260709 13:12:18.112098 32146 raft_consensus.cc:2468] T 7b2f10befb2e474fbe821af8d5e92fd5 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31690560715140e6847173104a2b0f2e in term 0.
I20260709 13:12:18.113107 32193 leader_election.cc:290] T 7b2f10befb2e474fbe821af8d5e92fd5 P 31690560715140e6847173104a2b0f2e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979), e69f3153d090418380c201dddfef4c60 (127.21.63.3:40639)
I20260709 13:12:18.114457 32036 leader_election.cc:304] T 7b2f10befb2e474fbe821af8d5e92fd5 P 31690560715140e6847173104a2b0f2e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31690560715140e6847173104a2b0f2e, e69f3153d090418380c201dddfef4c60; no voters: 
I20260709 13:12:18.111279 32186 consensus_queue.cc:260] T 0dfa6288e543429c99109ebbd5a223b9 P e69f3153d090418380c201dddfef4c60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.117107 32186 ts_tablet_manager.cc:1434] T 0dfa6288e543429c99109ebbd5a223b9 P e69f3153d090418380c201dddfef4c60: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:12:18.117285 32187 tablet_bootstrap.cc:492] T d1fc79d86bee43c5b062ced8a090aab1 P 31690560715140e6847173104a2b0f2e: No bootstrap required, opened a new log
I20260709 13:12:18.118016 32187 ts_tablet_manager.cc:1403] T d1fc79d86bee43c5b062ced8a090aab1 P 31690560715140e6847173104a2b0f2e: Time spent bootstrapping tablet: real 0.092s	user 0.015s	sys 0.002s
I20260709 13:12:18.117201 32193 raft_consensus.cc:2804] T 7b2f10befb2e474fbe821af8d5e92fd5 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:18.117988 32186 tablet_bootstrap.cc:492] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60: Bootstrap starting.
I20260709 13:12:18.121865 32187 raft_consensus.cc:359] T d1fc79d86bee43c5b062ced8a090aab1 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.122627 32187 raft_consensus.cc:385] T d1fc79d86bee43c5b062ced8a090aab1 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:18.122960 32187 raft_consensus.cc:740] T d1fc79d86bee43c5b062ced8a090aab1 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31690560715140e6847173104a2b0f2e, State: Initialized, Role: FOLLOWER
I20260709 13:12:18.118659 32193 raft_consensus.cc:493] T 7b2f10befb2e474fbe821af8d5e92fd5 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:18.113852 32111 leader_election.cc:304] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91778255eb264fe6a6dc208998e849ba, e69f3153d090418380c201dddfef4c60; no voters: 
I20260709 13:12:18.124436 32186 tablet_bootstrap.cc:654] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:18.131979 32145 raft_consensus.cc:2468] T 3a76c5023ae14f8ba25de09e3a9b60de P e69f3153d090418380c201dddfef4c60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31690560715140e6847173104a2b0f2e in term 1.
I20260709 13:12:18.132680 32111 leader_election.cc:304] T b40f21aa6dd54f64bed8e716a77df4d6 P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91778255eb264fe6a6dc208998e849ba, e69f3153d090418380c201dddfef4c60; no voters: 
I20260709 13:12:18.133608 32036 leader_election.cc:304] T 3a76c5023ae14f8ba25de09e3a9b60de P 31690560715140e6847173104a2b0f2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31690560715140e6847173104a2b0f2e, e69f3153d090418380c201dddfef4c60; no voters: 
I20260709 13:12:18.134160 32194 raft_consensus.cc:2804] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:18.134436 32222 raft_consensus.cc:2804] T 3a76c5023ae14f8ba25de09e3a9b60de P 31690560715140e6847173104a2b0f2e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:18.134549 32194 raft_consensus.cc:493] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:18.135115 32194 raft_consensus.cc:3060] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:12:18.124208 31962 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.2:44689, user_credentials={real_user=slave}} blocked reactor thread for 51334.9us
I20260709 13:12:18.136744 32072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a76c5023ae14f8ba25de09e3a9b60de" candidate_uuid: "91778255eb264fe6a6dc208998e849ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31690560715140e6847173104a2b0f2e"
I20260709 13:12:18.138221 31997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a76c5023ae14f8ba25de09e3a9b60de" candidate_uuid: "31690560715140e6847173104a2b0f2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91778255eb264fe6a6dc208998e849ba"
I20260709 13:12:18.139111 31997 raft_consensus.cc:2393] T 3a76c5023ae14f8ba25de09e3a9b60de P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 31690560715140e6847173104a2b0f2e in current term 1: Already voted for candidate 91778255eb264fe6a6dc208998e849ba in this term.
I20260709 13:12:18.140328 31997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b2f10befb2e474fbe821af8d5e92fd5" candidate_uuid: "31690560715140e6847173104a2b0f2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91778255eb264fe6a6dc208998e849ba" is_pre_election: true
I20260709 13:12:18.140944 31997 raft_consensus.cc:2468] T 7b2f10befb2e474fbe821af8d5e92fd5 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31690560715140e6847173104a2b0f2e in term 0.
I20260709 13:12:18.140512 32226 raft_consensus.cc:2804] T b40f21aa6dd54f64bed8e716a77df4d6 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:18.142148 32226 raft_consensus.cc:493] T b40f21aa6dd54f64bed8e716a77df4d6 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:18.143157 32226 raft_consensus.cc:3060] T b40f21aa6dd54f64bed8e716a77df4d6 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:12:18.118614 31961 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:40639, user_credentials={real_user=slave}} blocked reactor thread for 63032.6us
I20260709 13:12:18.145159 31961 leader_election.cc:304] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91778255eb264fe6a6dc208998e849ba, e69f3153d090418380c201dddfef4c60; no voters: 31690560715140e6847173104a2b0f2e
I20260709 13:12:18.146276 32228 raft_consensus.cc:2804] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:18.146780 32228 raft_consensus.cc:493] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:18.147154 32228 raft_consensus.cc:3060] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:18.124599 32193 raft_consensus.cc:3060] T 7b2f10befb2e474fbe821af8d5e92fd5 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:18.154795 32194 raft_consensus.cc:515] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } }
I20260709 13:12:18.155264 32228 raft_consensus.cc:515] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.124465 32187 consensus_queue.cc:260] T d1fc79d86bee43c5b062ced8a090aab1 P 31690560715140e6847173104a2b0f2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.159195 32071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1fc79d86bee43c5b062ced8a090aab1" candidate_uuid: "91778255eb264fe6a6dc208998e849ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31690560715140e6847173104a2b0f2e"
I20260709 13:12:18.159968 32071 raft_consensus.cc:3060] T d1fc79d86bee43c5b062ced8a090aab1 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:18.160588 32146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ea7beb6ca5f4de4b64c0e8a9e9d99a7" candidate_uuid: "91778255eb264fe6a6dc208998e849ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e69f3153d090418380c201dddfef4c60" is_pre_election: true
I20260709 13:12:18.161480 32146 raft_consensus.cc:2468] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91778255eb264fe6a6dc208998e849ba in term 0.
I20260709 13:12:18.163774 32229 raft_consensus.cc:493] T d0a7cae5f3084d0ba50763cc87888571 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:18.164242 32229 raft_consensus.cc:515] T d0a7cae5f3084d0ba50763cc87888571 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.164793 32235 raft_consensus.cc:493] T 709f871588d74579bab67c29aca59135 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:18.165300 32145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a76c5023ae14f8ba25de09e3a9b60de" candidate_uuid: "91778255eb264fe6a6dc208998e849ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e69f3153d090418380c201dddfef4c60"
I20260709 13:12:18.166051 32232 raft_consensus.cc:493] T d0a7cae5f3084d0ba50763cc87888571 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:18.166155 32145 raft_consensus.cc:2393] T 3a76c5023ae14f8ba25de09e3a9b60de P e69f3153d090418380c201dddfef4c60 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 91778255eb264fe6a6dc208998e849ba in current term 1: Already voted for candidate 31690560715140e6847173104a2b0f2e in this term.
I20260709 13:12:18.166704 32229 leader_election.cc:290] T d0a7cae5f3084d0ba50763cc87888571 P 91778255eb264fe6a6dc208998e849ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31690560715140e6847173104a2b0f2e (127.21.63.2:44689), e69f3153d090418380c201dddfef4c60 (127.21.63.3:40639)
I20260709 13:12:18.167291 32222 raft_consensus.cc:697] T 3a76c5023ae14f8ba25de09e3a9b60de P 31690560715140e6847173104a2b0f2e [term 1 LEADER]: Becoming Leader. State: Replica: 31690560715140e6847173104a2b0f2e, State: Running, Role: LEADER
I20260709 13:12:18.168946 32068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ea7beb6ca5f4de4b64c0e8a9e9d99a7" candidate_uuid: "e69f3153d090418380c201dddfef4c60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31690560715140e6847173104a2b0f2e" is_pre_election: true
I20260709 13:12:18.168141 32145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0a7cae5f3084d0ba50763cc87888571" candidate_uuid: "91778255eb264fe6a6dc208998e849ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e69f3153d090418380c201dddfef4c60" is_pre_election: true
I20260709 13:12:18.169473 32067 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40f21aa6dd54f64bed8e716a77df4d6" candidate_uuid: "e69f3153d090418380c201dddfef4c60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31690560715140e6847173104a2b0f2e" is_pre_election: true
I20260709 13:12:18.169921 32145 raft_consensus.cc:2468] T d0a7cae5f3084d0ba50763cc87888571 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91778255eb264fe6a6dc208998e849ba in term 0.
I20260709 13:12:18.170163 32069 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9a7e52d74da4cd79abac3858391ae77" candidate_uuid: "e69f3153d090418380c201dddfef4c60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31690560715140e6847173104a2b0f2e" is_pre_election: true
I20260709 13:12:18.170847 32069 raft_consensus.cc:2468] T f9a7e52d74da4cd79abac3858391ae77 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e69f3153d090418380c201dddfef4c60 in term 0.
I20260709 13:12:18.171170 31961 leader_election.cc:304] T d0a7cae5f3084d0ba50763cc87888571 P 91778255eb264fe6a6dc208998e849ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91778255eb264fe6a6dc208998e849ba, e69f3153d090418380c201dddfef4c60; no voters: 
I20260709 13:12:18.171135 32232 raft_consensus.cc:515] T d0a7cae5f3084d0ba50763cc87888571 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.171774 32070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0a7cae5f3084d0ba50763cc87888571" candidate_uuid: "91778255eb264fe6a6dc208998e849ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31690560715140e6847173104a2b0f2e" is_pre_election: true
I20260709 13:12:18.172124 32229 raft_consensus.cc:2804] T d0a7cae5f3084d0ba50763cc87888571 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:18.168602 32222 consensus_queue.cc:237] T 3a76c5023ae14f8ba25de09e3a9b60de P 31690560715140e6847173104a2b0f2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.172581 32229 raft_consensus.cc:493] T d0a7cae5f3084d0ba50763cc87888571 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:18.172379 32225 raft_consensus.cc:515] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.173085 32229 raft_consensus.cc:3060] T d0a7cae5f3084d0ba50763cc87888571 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:18.173625 31996 raft_consensus.cc:2393] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e69f3153d090418380c201dddfef4c60 in current term 1: Already voted for candidate 91778255eb264fe6a6dc208998e849ba in this term.
I20260709 13:12:18.174331 31997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0a7cae5f3084d0ba50763cc87888571" candidate_uuid: "31690560715140e6847173104a2b0f2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91778255eb264fe6a6dc208998e849ba" is_pre_election: true
I20260709 13:12:18.172477 32070 raft_consensus.cc:2468] T d0a7cae5f3084d0ba50763cc87888571 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91778255eb264fe6a6dc208998e849ba in term 0.
I20260709 13:12:18.169638 32068 raft_consensus.cc:2468] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e69f3153d090418380c201dddfef4c60 in term 0.
I20260709 13:12:18.177728 32225 leader_election.cc:290] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba [CANDIDATE]: Term 1 election: Requested vote from peers 31690560715140e6847173104a2b0f2e (127.21.63.2:44689), e69f3153d090418380c201dddfef4c60 (127.21.63.3:40639)
W20260709 13:12:18.179101 32111 leader_election.cc:343] T b40f21aa6dd54f64bed8e716a77df4d6 P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 31690560715140e6847173104a2b0f2e (127.21.63.2:44689): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:18.180511 32068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ea7beb6ca5f4de4b64c0e8a9e9d99a7" candidate_uuid: "91778255eb264fe6a6dc208998e849ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31690560715140e6847173104a2b0f2e"
I20260709 13:12:18.180744 32111 leader_election.cc:304] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31690560715140e6847173104a2b0f2e, e69f3153d090418380c201dddfef4c60; no voters: 91778255eb264fe6a6dc208998e849ba
I20260709 13:12:18.180866 32229 raft_consensus.cc:515] T d0a7cae5f3084d0ba50763cc87888571 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.180794 32232 leader_election.cc:290] T d0a7cae5f3084d0ba50763cc87888571 P 31690560715140e6847173104a2b0f2e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979), e69f3153d090418380c201dddfef4c60 (127.21.63.3:40639)
I20260709 13:12:18.183995 31997 raft_consensus.cc:2393] T d0a7cae5f3084d0ba50763cc87888571 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31690560715140e6847173104a2b0f2e in current term 1: Already voted for candidate 91778255eb264fe6a6dc208998e849ba in this term.
I20260709 13:12:18.184592 32146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ea7beb6ca5f4de4b64c0e8a9e9d99a7" candidate_uuid: "91778255eb264fe6a6dc208998e849ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e69f3153d090418380c201dddfef4c60"
I20260709 13:12:18.185195 32146 raft_consensus.cc:3060] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:18.185730 32225 raft_consensus.cc:493] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:18.181926 32226 raft_consensus.cc:515] T b40f21aa6dd54f64bed8e716a77df4d6 P e69f3153d090418380c201dddfef4c60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.186383 32225 raft_consensus.cc:515] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.188077 32226 leader_election.cc:290] T b40f21aa6dd54f64bed8e716a77df4d6 P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 1 election: Requested vote from peers 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979), 31690560715140e6847173104a2b0f2e (127.21.63.2:44689)
I20260709 13:12:18.193559 32193 raft_consensus.cc:515] T 7b2f10befb2e474fbe821af8d5e92fd5 P 31690560715140e6847173104a2b0f2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.196934 31997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40f21aa6dd54f64bed8e716a77df4d6" candidate_uuid: "e69f3153d090418380c201dddfef4c60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91778255eb264fe6a6dc208998e849ba"
I20260709 13:12:18.197243 31996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b2f10befb2e474fbe821af8d5e92fd5" candidate_uuid: "31690560715140e6847173104a2b0f2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91778255eb264fe6a6dc208998e849ba"
I20260709 13:12:18.197578 31997 raft_consensus.cc:3060] T b40f21aa6dd54f64bed8e716a77df4d6 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:18.197314 32144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b2f10befb2e474fbe821af8d5e92fd5" candidate_uuid: "31690560715140e6847173104a2b0f2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e69f3153d090418380c201dddfef4c60"
I20260709 13:12:18.198050 31996 raft_consensus.cc:3060] T 7b2f10befb2e474fbe821af8d5e92fd5 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:18.198275 32144 raft_consensus.cc:3060] T 7b2f10befb2e474fbe821af8d5e92fd5 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:18.182842 32068 raft_consensus.cc:3060] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:18.203044 31962 leader_election.cc:304] T 3a76c5023ae14f8ba25de09e3a9b60de P 91778255eb264fe6a6dc208998e849ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91778255eb264fe6a6dc208998e849ba; no voters: 31690560715140e6847173104a2b0f2e, e69f3153d090418380c201dddfef4c60
I20260709 13:12:18.203635 32072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e568fd2b2a149aeadd6c64bc70af9b7" candidate_uuid: "91778255eb264fe6a6dc208998e849ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31690560715140e6847173104a2b0f2e" is_pre_election: true
I20260709 13:12:18.205101 32070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40f21aa6dd54f64bed8e716a77df4d6" candidate_uuid: "e69f3153d090418380c201dddfef4c60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31690560715140e6847173104a2b0f2e"
I20260709 13:12:18.205291 31997 raft_consensus.cc:2468] T b40f21aa6dd54f64bed8e716a77df4d6 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e69f3153d090418380c201dddfef4c60 in term 1.
W20260709 13:12:18.206175 31962 leader_election.cc:343] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 31690560715140e6847173104a2b0f2e (127.21.63.2:44689): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:18.206355 32192 raft_consensus.cc:2749] T 3a76c5023ae14f8ba25de09e3a9b60de P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:12:18.189503 32145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0a7cae5f3084d0ba50763cc87888571" candidate_uuid: "31690560715140e6847173104a2b0f2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e69f3153d090418380c201dddfef4c60" is_pre_election: true
I20260709 13:12:18.207350 32145 raft_consensus.cc:2468] T d0a7cae5f3084d0ba50763cc87888571 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31690560715140e6847173104a2b0f2e in term 0.
I20260709 13:12:18.208653 32036 leader_election.cc:304] T d0a7cae5f3084d0ba50763cc87888571 P 31690560715140e6847173104a2b0f2e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31690560715140e6847173104a2b0f2e, e69f3153d090418380c201dddfef4c60; no voters: 91778255eb264fe6a6dc208998e849ba
I20260709 13:12:18.209154 31893 catalog_manager.cc:5697] T 3a76c5023ae14f8ba25de09e3a9b60de P 31690560715140e6847173104a2b0f2e reported cstate change: term changed from 0 to 1, leader changed from <none> to 31690560715140e6847173104a2b0f2e (127.21.63.2). New cstate: current_term: 1 leader_uuid: "31690560715140e6847173104a2b0f2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:18.210330 31996 raft_consensus.cc:2468] T 7b2f10befb2e474fbe821af8d5e92fd5 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31690560715140e6847173104a2b0f2e in term 1.
I20260709 13:12:18.211247 32186 tablet_bootstrap.cc:492] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60: No bootstrap required, opened a new log
I20260709 13:12:18.211519 32037 leader_election.cc:304] T 7b2f10befb2e474fbe821af8d5e92fd5 P 31690560715140e6847173104a2b0f2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31690560715140e6847173104a2b0f2e, 91778255eb264fe6a6dc208998e849ba; no voters: 
I20260709 13:12:18.211746 32186 ts_tablet_manager.cc:1403] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60: Time spent bootstrapping tablet: real 0.094s	user 0.017s	sys 0.028s
I20260709 13:12:18.200769 32228 leader_election.cc:290] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba [CANDIDATE]: Term 1 election: Requested vote from peers 31690560715140e6847173104a2b0f2e (127.21.63.2:44689), e69f3153d090418380c201dddfef4c60 (127.21.63.3:40639)
I20260709 13:12:18.215092 32234 raft_consensus.cc:2804] T d0a7cae5f3084d0ba50763cc87888571 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:18.214529 32146 raft_consensus.cc:2468] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91778255eb264fe6a6dc208998e849ba in term 1.
I20260709 13:12:18.215947 32234 raft_consensus.cc:493] T d0a7cae5f3084d0ba50763cc87888571 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:18.216033 32145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1fc79d86bee43c5b062ced8a090aab1" candidate_uuid: "91778255eb264fe6a6dc208998e849ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e69f3153d090418380c201dddfef4c60"
I20260709 13:12:18.216338 32234 raft_consensus.cc:3060] T d0a7cae5f3084d0ba50763cc87888571 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:18.216710 32145 raft_consensus.cc:3060] T d1fc79d86bee43c5b062ced8a090aab1 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:18.214818 32225 leader_election.cc:290] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 31690560715140e6847173104a2b0f2e (127.21.63.2:44689), e69f3153d090418380c201dddfef4c60 (127.21.63.3:40639)
I20260709 13:12:18.190282 32071 raft_consensus.cc:2468] T d1fc79d86bee43c5b062ced8a090aab1 P 31690560715140e6847173104a2b0f2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91778255eb264fe6a6dc208998e849ba in term 1.
I20260709 13:12:18.213863 32222 raft_consensus.cc:2804] T 7b2f10befb2e474fbe821af8d5e92fd5 P 31690560715140e6847173104a2b0f2e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:18.201364 32193 leader_election.cc:290] T 7b2f10befb2e474fbe821af8d5e92fd5 P 31690560715140e6847173104a2b0f2e [CANDIDATE]: Term 1 election: Requested vote from peers 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979), e69f3153d090418380c201dddfef4c60 (127.21.63.3:40639)
I20260709 13:12:18.165299 32235 raft_consensus.cc:515] T 709f871588d74579bab67c29aca59135 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.218179 32226 raft_consensus.cc:2764] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:12:18.234836 32186 raft_consensus.cc:359] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.258041 32068 raft_consensus.cc:2468] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 31690560715140e6847173104a2b0f2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91778255eb264fe6a6dc208998e849ba in term 1.
I20260709 13:12:18.219755 32111 leader_election.cc:304] T b40f21aa6dd54f64bed8e716a77df4d6 P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91778255eb264fe6a6dc208998e849ba, e69f3153d090418380c201dddfef4c60; no voters: 
W20260709 13:12:18.262740 32111 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.1:32979, user_credentials={real_user=slave}} blocked reactor thread for 45013.8us
I20260709 13:12:18.263165 32234 raft_consensus.cc:515] T d0a7cae5f3084d0ba50763cc87888571 P 31690560715140e6847173104a2b0f2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.264617 32237 raft_consensus.cc:493] T 0dfa6288e543429c99109ebbd5a223b9 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:18.224246 31962 leader_election.cc:304] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31690560715140e6847173104a2b0f2e, 91778255eb264fe6a6dc208998e849ba; no voters: 
I20260709 13:12:18.264801 31996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9a7e52d74da4cd79abac3858391ae77" candidate_uuid: "e69f3153d090418380c201dddfef4c60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91778255eb264fe6a6dc208998e849ba"
I20260709 13:12:18.223973 32229 leader_election.cc:290] T d0a7cae5f3084d0ba50763cc87888571 P 91778255eb264fe6a6dc208998e849ba [CANDIDATE]: Term 1 election: Requested vote from peers 31690560715140e6847173104a2b0f2e (127.21.63.2:44689), e69f3153d090418380c201dddfef4c60 (127.21.63.3:40639)
I20260709 13:12:18.265146 32237 raft_consensus.cc:515] T 0dfa6288e543429c99109ebbd5a223b9 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.265426 32228 raft_consensus.cc:2804] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:18.234308 32146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e568fd2b2a149aeadd6c64bc70af9b7" candidate_uuid: "91778255eb264fe6a6dc208998e849ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e69f3153d090418380c201dddfef4c60" is_pre_election: true
I20260709 13:12:18.266494 32228 raft_consensus.cc:697] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba [term 1 LEADER]: Becoming Leader. State: Replica: 91778255eb264fe6a6dc208998e849ba, State: Running, Role: LEADER
I20260709 13:12:18.267241 32237 leader_election.cc:290] T 0dfa6288e543429c99109ebbd5a223b9 P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979), 31690560715140e6847173104a2b0f2e (127.21.63.2:44689)
I20260709 13:12:18.267433 32228 consensus_queue.cc:237] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.268488 32145 raft_consensus.cc:2468] T d1fc79d86bee43c5b062ced8a090aab1 P e69f3153d090418380c201dddfef4c60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91778255eb264fe6a6dc208998e849ba in term 1.
I20260709 13:12:18.269858 32234 leader_election.cc:290] T d0a7cae5f3084d0ba50763cc87888571 P 31690560715140e6847173104a2b0f2e [CANDIDATE]: Term 1 election: Requested vote from peers 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979), e69f3153d090418380c201dddfef4c60 (127.21.63.3:40639)
I20260709 13:12:18.233777 31961 leader_election.cc:304] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91778255eb264fe6a6dc208998e849ba, e69f3153d090418380c201dddfef4c60; no voters: 
I20260709 13:12:18.270498 32145 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0a7cae5f3084d0ba50763cc87888571" candidate_uuid: "31690560715140e6847173104a2b0f2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e69f3153d090418380c201dddfef4c60"
I20260709 13:12:18.271083 32145 raft_consensus.cc:3060] T d0a7cae5f3084d0ba50763cc87888571 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:18.226774 32194 leader_election.cc:290] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 1 election: Requested vote from peers 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979), 31690560715140e6847173104a2b0f2e (127.21.63.2:44689)
I20260709 13:12:18.271708 32229 raft_consensus.cc:2804] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:18.225131 32070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0a7cae5f3084d0ba50763cc87888571" candidate_uuid: "91778255eb264fe6a6dc208998e849ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31690560715140e6847173104a2b0f2e"
I20260709 13:12:18.261138 32187 ts_tablet_manager.cc:1434] T d1fc79d86bee43c5b062ced8a090aab1 P 31690560715140e6847173104a2b0f2e: Time spent starting tablet: real 0.143s	user 0.008s	sys 0.002s
I20260709 13:12:18.272291 32229 raft_consensus.cc:697] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba [term 1 LEADER]: Becoming Leader. State: Replica: 91778255eb264fe6a6dc208998e849ba, State: Running, Role: LEADER
I20260709 13:12:18.272948 31997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0a7cae5f3084d0ba50763cc87888571" candidate_uuid: "31690560715140e6847173104a2b0f2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91778255eb264fe6a6dc208998e849ba"
I20260709 13:12:18.273272 32229 consensus_queue.cc:237] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.262818 32226 raft_consensus.cc:2804] T b40f21aa6dd54f64bed8e716a77df4d6 P e69f3153d090418380c201dddfef4c60 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:18.274462 32070 raft_consensus.cc:2393] T d0a7cae5f3084d0ba50763cc87888571 P 31690560715140e6847173104a2b0f2e [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 91778255eb264fe6a6dc208998e849ba in current term 1: Already voted for candidate 31690560715140e6847173104a2b0f2e in this term.
I20260709 13:12:18.273681 32187 tablet_bootstrap.cc:492] T 0dfa6288e543429c99109ebbd5a223b9 P 31690560715140e6847173104a2b0f2e: Bootstrap starting.
I20260709 13:12:18.234983 32143 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d0a7cae5f3084d0ba50763cc87888571" candidate_uuid: "91778255eb264fe6a6dc208998e849ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e69f3153d090418380c201dddfef4c60"
I20260709 13:12:18.222100 32222 raft_consensus.cc:697] T 7b2f10befb2e474fbe821af8d5e92fd5 P 31690560715140e6847173104a2b0f2e [term 1 LEADER]: Becoming Leader. State: Replica: 31690560715140e6847173104a2b0f2e, State: Running, Role: LEADER
I20260709 13:12:18.274401 32226 raft_consensus.cc:697] T b40f21aa6dd54f64bed8e716a77df4d6 P e69f3153d090418380c201dddfef4c60 [term 1 LEADER]: Becoming Leader. State: Replica: e69f3153d090418380c201dddfef4c60, State: Running, Role: LEADER
I20260709 13:12:18.265430 31996 raft_consensus.cc:3060] T f9a7e52d74da4cd79abac3858391ae77 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:18.277251 32226 consensus_queue.cc:237] T b40f21aa6dd54f64bed8e716a77df4d6 P e69f3153d090418380c201dddfef4c60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
W20260709 13:12:18.278234 32111 leader_election.cc:343] T b40f21aa6dd54f64bed8e716a77df4d6 P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 31690560715140e6847173104a2b0f2e (127.21.63.2:44689): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:18.279418 32145 raft_consensus.cc:2468] T d0a7cae5f3084d0ba50763cc87888571 P e69f3153d090418380c201dddfef4c60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31690560715140e6847173104a2b0f2e in term 1.
I20260709 13:12:18.262025 32235 leader_election.cc:290] T 709f871588d74579bab67c29aca59135 P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979), 31690560715140e6847173104a2b0f2e (127.21.63.2:44689)
I20260709 13:12:18.280228 32222 consensus_queue.cc:237] T 7b2f10befb2e474fbe821af8d5e92fd5 P 31690560715140e6847173104a2b0f2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.280537 31995 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709f871588d74579bab67c29aca59135" candidate_uuid: "e69f3153d090418380c201dddfef4c60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91778255eb264fe6a6dc208998e849ba" is_pre_election: true
I20260709 13:12:18.281032 32071 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709f871588d74579bab67c29aca59135" candidate_uuid: "e69f3153d090418380c201dddfef4c60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31690560715140e6847173104a2b0f2e" is_pre_election: true
I20260709 13:12:18.281142 32072 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dfa6288e543429c99109ebbd5a223b9" candidate_uuid: "e69f3153d090418380c201dddfef4c60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31690560715140e6847173104a2b0f2e" is_pre_election: true
I20260709 13:12:18.281318 31995 raft_consensus.cc:2468] T 709f871588d74579bab67c29aca59135 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e69f3153d090418380c201dddfef4c60 in term 0.
I20260709 13:12:18.282140 31994 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dfa6288e543429c99109ebbd5a223b9" candidate_uuid: "e69f3153d090418380c201dddfef4c60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91778255eb264fe6a6dc208998e849ba" is_pre_election: true
I20260709 13:12:18.282814 31994 raft_consensus.cc:2468] T 0dfa6288e543429c99109ebbd5a223b9 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e69f3153d090418380c201dddfef4c60 in term 0.
I20260709 13:12:18.283236 32068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9a7e52d74da4cd79abac3858391ae77" candidate_uuid: "e69f3153d090418380c201dddfef4c60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31690560715140e6847173104a2b0f2e"
I20260709 13:12:18.283792 32068 raft_consensus.cc:3060] T f9a7e52d74da4cd79abac3858391ae77 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:18.283753 32036 leader_election.cc:304] T d0a7cae5f3084d0ba50763cc87888571 P 31690560715140e6847173104a2b0f2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31690560715140e6847173104a2b0f2e, e69f3153d090418380c201dddfef4c60; no voters: 
I20260709 13:12:18.273761 31997 raft_consensus.cc:2393] T d0a7cae5f3084d0ba50763cc87888571 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 31690560715140e6847173104a2b0f2e in current term 1: Already voted for candidate 91778255eb264fe6a6dc208998e849ba in this term.
I20260709 13:12:18.239670 32144 raft_consensus.cc:2468] T 7b2f10befb2e474fbe821af8d5e92fd5 P e69f3153d090418380c201dddfef4c60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31690560715140e6847173104a2b0f2e in term 1.
I20260709 13:12:18.258944 32186 raft_consensus.cc:385] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:18.297156 32234 raft_consensus.cc:2804] T d0a7cae5f3084d0ba50763cc87888571 P 31690560715140e6847173104a2b0f2e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:18.297796 32186 raft_consensus.cc:740] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e69f3153d090418380c201dddfef4c60, State: Initialized, Role: FOLLOWER
I20260709 13:12:18.298032 32234 raft_consensus.cc:697] T d0a7cae5f3084d0ba50763cc87888571 P 31690560715140e6847173104a2b0f2e [term 1 LEADER]: Becoming Leader. State: Replica: 31690560715140e6847173104a2b0f2e, State: Running, Role: LEADER
I20260709 13:12:18.298727 32186 consensus_queue.cc:260] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.299680 32146 raft_consensus.cc:2468] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 91778255eb264fe6a6dc208998e849ba in term 0.
I20260709 13:12:18.299293 32234 consensus_queue.cc:237] T d0a7cae5f3084d0ba50763cc87888571 P 31690560715140e6847173104a2b0f2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.302508 32186 ts_tablet_manager.cc:1434] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60: Time spent starting tablet: real 0.090s	user 0.000s	sys 0.010s
I20260709 13:12:18.303436 31961 leader_election.cc:304] T d0a7cae5f3084d0ba50763cc87888571 P 91778255eb264fe6a6dc208998e849ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 91778255eb264fe6a6dc208998e849ba; no voters: 31690560715140e6847173104a2b0f2e, e69f3153d090418380c201dddfef4c60
I20260709 13:12:18.304323 31893 catalog_manager.cc:5697] T b40f21aa6dd54f64bed8e716a77df4d6 P e69f3153d090418380c201dddfef4c60 reported cstate change: term changed from 0 to 1, leader changed from <none> to e69f3153d090418380c201dddfef4c60 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "e69f3153d090418380c201dddfef4c60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:18.310390 32225 raft_consensus.cc:2749] T d0a7cae5f3084d0ba50763cc87888571 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:12:18.310930 31961 leader_election.cc:304] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91778255eb264fe6a6dc208998e849ba, e69f3153d090418380c201dddfef4c60; no voters: 31690560715140e6847173104a2b0f2e
I20260709 13:12:18.311792 32225 raft_consensus.cc:2804] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:18.312213 32225 raft_consensus.cc:493] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:18.312641 32225 raft_consensus.cc:3060] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:18.305426 31996 raft_consensus.cc:2468] T f9a7e52d74da4cd79abac3858391ae77 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e69f3153d090418380c201dddfef4c60 in term 1.
I20260709 13:12:18.306936 32111 leader_election.cc:304] T 709f871588d74579bab67c29aca59135 P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91778255eb264fe6a6dc208998e849ba, e69f3153d090418380c201dddfef4c60; no voters: 
I20260709 13:12:18.315143 32111 leader_election.cc:304] T 0dfa6288e543429c99109ebbd5a223b9 P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91778255eb264fe6a6dc208998e849ba, e69f3153d090418380c201dddfef4c60; no voters: 
I20260709 13:12:18.315696 32226 raft_consensus.cc:2804] T 709f871588d74579bab67c29aca59135 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:18.316123 32226 raft_consensus.cc:493] T 709f871588d74579bab67c29aca59135 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:18.316190 32194 raft_consensus.cc:2804] T 0dfa6288e543429c99109ebbd5a223b9 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:18.316538 32226 raft_consensus.cc:3060] T 709f871588d74579bab67c29aca59135 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:18.316761 32194 raft_consensus.cc:493] T 0dfa6288e543429c99109ebbd5a223b9 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:18.317247 32194 raft_consensus.cc:3060] T 0dfa6288e543429c99109ebbd5a223b9 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:18.320501 31895 catalog_manager.cc:5697] T 7b2f10befb2e474fbe821af8d5e92fd5 P 31690560715140e6847173104a2b0f2e reported cstate change: term changed from 0 to 1, leader changed from <none> to 31690560715140e6847173104a2b0f2e (127.21.63.2). New cstate: current_term: 1 leader_uuid: "31690560715140e6847173104a2b0f2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:18.296684 32068 raft_consensus.cc:2468] T f9a7e52d74da4cd79abac3858391ae77 P 31690560715140e6847173104a2b0f2e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e69f3153d090418380c201dddfef4c60 in term 1.
I20260709 13:12:18.324149 32225 raft_consensus.cc:515] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
W20260709 13:12:18.328819 32111 leader_election.cc:343] T 709f871588d74579bab67c29aca59135 P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 31690560715140e6847173104a2b0f2e (127.21.63.2:44689): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:18.308506 32187 tablet_bootstrap.cc:654] T 0dfa6288e543429c99109ebbd5a223b9 P 31690560715140e6847173104a2b0f2e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:18.331481 32068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e568fd2b2a149aeadd6c64bc70af9b7" candidate_uuid: "91778255eb264fe6a6dc208998e849ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31690560715140e6847173104a2b0f2e"
I20260709 13:12:18.332814 32226 raft_consensus.cc:515] T 709f871588d74579bab67c29aca59135 P e69f3153d090418380c201dddfef4c60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
W20260709 13:12:18.333611 31962 leader_election.cc:343] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 31690560715140e6847173104a2b0f2e (127.21.63.2:44689): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:18.334877 32226 leader_election.cc:290] T 709f871588d74579bab67c29aca59135 P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 1 election: Requested vote from peers 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979), 31690560715140e6847173104a2b0f2e (127.21.63.2:44689)
I20260709 13:12:18.335290 32225 leader_election.cc:290] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba [CANDIDATE]: Term 1 election: Requested vote from peers 31690560715140e6847173104a2b0f2e (127.21.63.2:44689), e69f3153d090418380c201dddfef4c60 (127.21.63.3:40639)
I20260709 13:12:18.337199 32146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e568fd2b2a149aeadd6c64bc70af9b7" candidate_uuid: "91778255eb264fe6a6dc208998e849ba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e69f3153d090418380c201dddfef4c60"
I20260709 13:12:18.337708 32146 raft_consensus.cc:3060] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:12:18.330667 32111 leader_election.cc:343] T 0dfa6288e543429c99109ebbd5a223b9 P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 31690560715140e6847173104a2b0f2e (127.21.63.2:44689): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:18.342971 32111 leader_election.cc:304] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31690560715140e6847173104a2b0f2e, e69f3153d090418380c201dddfef4c60; no voters: 
I20260709 13:12:18.343770 32226 raft_consensus.cc:2804] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:18.343842 32146 raft_consensus.cc:2468] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 91778255eb264fe6a6dc208998e849ba in term 1.
I20260709 13:12:18.344408 32226 raft_consensus.cc:697] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 [term 1 LEADER]: Becoming Leader. State: Replica: e69f3153d090418380c201dddfef4c60, State: Running, Role: LEADER
I20260709 13:12:18.345124 31961 leader_election.cc:304] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91778255eb264fe6a6dc208998e849ba, e69f3153d090418380c201dddfef4c60; no voters: 31690560715140e6847173104a2b0f2e
I20260709 13:12:18.345343 32226 consensus_queue.cc:237] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } }
I20260709 13:12:18.346911 32194 raft_consensus.cc:515] T 0dfa6288e543429c99109ebbd5a223b9 P e69f3153d090418380c201dddfef4c60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.347916 32225 raft_consensus.cc:2804] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:18.348408 32225 raft_consensus.cc:697] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba [term 1 LEADER]: Becoming Leader. State: Replica: 91778255eb264fe6a6dc208998e849ba, State: Running, Role: LEADER
I20260709 13:12:18.349313 32194 leader_election.cc:290] T 0dfa6288e543429c99109ebbd5a223b9 P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 1 election: Requested vote from peers 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979), 31690560715140e6847173104a2b0f2e (127.21.63.2:44689)
I20260709 13:12:18.349390 32225 consensus_queue.cc:237] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.351871 31996 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709f871588d74579bab67c29aca59135" candidate_uuid: "e69f3153d090418380c201dddfef4c60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91778255eb264fe6a6dc208998e849ba"
I20260709 13:12:18.352496 31996 raft_consensus.cc:3060] T 709f871588d74579bab67c29aca59135 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:18.355024 31893 catalog_manager.cc:5697] T d0a7cae5f3084d0ba50763cc87888571 P 31690560715140e6847173104a2b0f2e reported cstate change: term changed from 0 to 1, leader changed from <none> to 31690560715140e6847173104a2b0f2e (127.21.63.2). New cstate: current_term: 1 leader_uuid: "31690560715140e6847173104a2b0f2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:18.362278 32068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709f871588d74579bab67c29aca59135" candidate_uuid: "e69f3153d090418380c201dddfef4c60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31690560715140e6847173104a2b0f2e"
I20260709 13:12:18.365187 31997 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dfa6288e543429c99109ebbd5a223b9" candidate_uuid: "e69f3153d090418380c201dddfef4c60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91778255eb264fe6a6dc208998e849ba"
I20260709 13:12:18.365813 31997 raft_consensus.cc:3060] T 0dfa6288e543429c99109ebbd5a223b9 P 91778255eb264fe6a6dc208998e849ba [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:18.366117 32068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dfa6288e543429c99109ebbd5a223b9" candidate_uuid: "e69f3153d090418380c201dddfef4c60" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31690560715140e6847173104a2b0f2e"
W20260709 13:12:18.368544 32111 leader_election.cc:343] T 709f871588d74579bab67c29aca59135 P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 31690560715140e6847173104a2b0f2e (127.21.63.2:44689): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:12:18.369313 32111 leader_election.cc:343] T 0dfa6288e543429c99109ebbd5a223b9 P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 31690560715140e6847173104a2b0f2e (127.21.63.2:44689): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:18.375706 31996 raft_consensus.cc:2468] T 709f871588d74579bab67c29aca59135 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e69f3153d090418380c201dddfef4c60 in term 1.
I20260709 13:12:18.381021 31997 raft_consensus.cc:2468] T 0dfa6288e543429c99109ebbd5a223b9 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e69f3153d090418380c201dddfef4c60 in term 1.
I20260709 13:12:18.380952 31895 catalog_manager.cc:5697] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba reported cstate change: term changed from 0 to 1, leader changed from <none> to 91778255eb264fe6a6dc208998e849ba (127.21.63.1). New cstate: current_term: 1 leader_uuid: "91778255eb264fe6a6dc208998e849ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:18.384774 32187 tablet_bootstrap.cc:492] T 0dfa6288e543429c99109ebbd5a223b9 P 31690560715140e6847173104a2b0f2e: No bootstrap required, opened a new log
I20260709 13:12:18.385318 32187 ts_tablet_manager.cc:1403] T 0dfa6288e543429c99109ebbd5a223b9 P 31690560715140e6847173104a2b0f2e: Time spent bootstrapping tablet: real 0.112s	user 0.011s	sys 0.002s
I20260709 13:12:18.382823 32111 leader_election.cc:304] T 709f871588d74579bab67c29aca59135 P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91778255eb264fe6a6dc208998e849ba, e69f3153d090418380c201dddfef4c60; no voters: 31690560715140e6847173104a2b0f2e
I20260709 13:12:18.387720 32111 leader_election.cc:304] T 0dfa6288e543429c99109ebbd5a223b9 P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 91778255eb264fe6a6dc208998e849ba, e69f3153d090418380c201dddfef4c60; no voters: 31690560715140e6847173104a2b0f2e
I20260709 13:12:18.389492 32187 raft_consensus.cc:359] T 0dfa6288e543429c99109ebbd5a223b9 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.391160 32194 raft_consensus.cc:2804] T 0dfa6288e543429c99109ebbd5a223b9 P e69f3153d090418380c201dddfef4c60 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:18.391433 32187 raft_consensus.cc:385] T 0dfa6288e543429c99109ebbd5a223b9 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:18.391769 32187 raft_consensus.cc:740] T 0dfa6288e543429c99109ebbd5a223b9 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31690560715140e6847173104a2b0f2e, State: Initialized, Role: FOLLOWER
I20260709 13:12:18.390038 31893 catalog_manager.cc:5697] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 reported cstate change: term changed from 0 to 1, leader changed from <none> to e69f3153d090418380c201dddfef4c60 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "e69f3153d090418380c201dddfef4c60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:18.387197 32226 raft_consensus.cc:2804] T 709f871588d74579bab67c29aca59135 P e69f3153d090418380c201dddfef4c60 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:18.392601 32187 consensus_queue.cc:260] T 0dfa6288e543429c99109ebbd5a223b9 P 31690560715140e6847173104a2b0f2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.394402 32194 raft_consensus.cc:697] T 0dfa6288e543429c99109ebbd5a223b9 P e69f3153d090418380c201dddfef4c60 [term 1 LEADER]: Becoming Leader. State: Replica: e69f3153d090418380c201dddfef4c60, State: Running, Role: LEADER
I20260709 13:12:18.395080 32187 ts_tablet_manager.cc:1434] T 0dfa6288e543429c99109ebbd5a223b9 P 31690560715140e6847173104a2b0f2e: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:12:18.395334 32194 consensus_queue.cc:237] T 0dfa6288e543429c99109ebbd5a223b9 P e69f3153d090418380c201dddfef4c60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.396136 32187 tablet_bootstrap.cc:492] T b40f21aa6dd54f64bed8e716a77df4d6 P 31690560715140e6847173104a2b0f2e: Bootstrap starting.
I20260709 13:12:18.403355 32187 tablet_bootstrap.cc:654] T b40f21aa6dd54f64bed8e716a77df4d6 P 31690560715140e6847173104a2b0f2e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:18.403216 32226 raft_consensus.cc:697] T 709f871588d74579bab67c29aca59135 P e69f3153d090418380c201dddfef4c60 [term 1 LEADER]: Becoming Leader. State: Replica: e69f3153d090418380c201dddfef4c60, State: Running, Role: LEADER
I20260709 13:12:18.406499 32226 consensus_queue.cc:237] T 709f871588d74579bab67c29aca59135 P e69f3153d090418380c201dddfef4c60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.413165 31895 catalog_manager.cc:5697] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba reported cstate change: term changed from 0 to 1, leader changed from <none> to 91778255eb264fe6a6dc208998e849ba (127.21.63.1). New cstate: current_term: 1 leader_uuid: "91778255eb264fe6a6dc208998e849ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:18.415701 31895 catalog_manager.cc:5697] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba reported cstate change: term changed from 0 to 1, leader changed from <none> to 91778255eb264fe6a6dc208998e849ba (127.21.63.1). New cstate: current_term: 1 leader_uuid: "91778255eb264fe6a6dc208998e849ba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:18.417791 32187 tablet_bootstrap.cc:492] T b40f21aa6dd54f64bed8e716a77df4d6 P 31690560715140e6847173104a2b0f2e: No bootstrap required, opened a new log
I20260709 13:12:18.418284 32187 ts_tablet_manager.cc:1403] T b40f21aa6dd54f64bed8e716a77df4d6 P 31690560715140e6847173104a2b0f2e: Time spent bootstrapping tablet: real 0.022s	user 0.017s	sys 0.000s
I20260709 13:12:18.420817 32187 raft_consensus.cc:359] T b40f21aa6dd54f64bed8e716a77df4d6 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.421456 32187 raft_consensus.cc:385] T b40f21aa6dd54f64bed8e716a77df4d6 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:18.421727 32187 raft_consensus.cc:740] T b40f21aa6dd54f64bed8e716a77df4d6 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31690560715140e6847173104a2b0f2e, State: Initialized, Role: FOLLOWER
I20260709 13:12:18.422281 32187 consensus_queue.cc:260] T b40f21aa6dd54f64bed8e716a77df4d6 P 31690560715140e6847173104a2b0f2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.423434 31893 catalog_manager.cc:5697] T 709f871588d74579bab67c29aca59135 P e69f3153d090418380c201dddfef4c60 reported cstate change: term changed from 0 to 1, leader changed from <none> to e69f3153d090418380c201dddfef4c60 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "e69f3153d090418380c201dddfef4c60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:18.426923 32187 ts_tablet_manager.cc:1434] T b40f21aa6dd54f64bed8e716a77df4d6 P 31690560715140e6847173104a2b0f2e: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:12:18.427333 31893 catalog_manager.cc:5697] T 0dfa6288e543429c99109ebbd5a223b9 P e69f3153d090418380c201dddfef4c60 reported cstate change: term changed from 0 to 1, leader changed from <none> to e69f3153d090418380c201dddfef4c60 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "e69f3153d090418380c201dddfef4c60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:18.427949 32187 tablet_bootstrap.cc:492] T 709f871588d74579bab67c29aca59135 P 31690560715140e6847173104a2b0f2e: Bootstrap starting.
I20260709 13:12:18.435597 32187 tablet_bootstrap.cc:654] T 709f871588d74579bab67c29aca59135 P 31690560715140e6847173104a2b0f2e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:18.440419 32187 tablet_bootstrap.cc:492] T 709f871588d74579bab67c29aca59135 P 31690560715140e6847173104a2b0f2e: No bootstrap required, opened a new log
I20260709 13:12:18.440867 32187 ts_tablet_manager.cc:1403] T 709f871588d74579bab67c29aca59135 P 31690560715140e6847173104a2b0f2e: Time spent bootstrapping tablet: real 0.013s	user 0.010s	sys 0.001s
I20260709 13:12:18.442965 32187 raft_consensus.cc:359] T 709f871588d74579bab67c29aca59135 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.443426 32187 raft_consensus.cc:385] T 709f871588d74579bab67c29aca59135 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:18.443607 32187 raft_consensus.cc:740] T 709f871588d74579bab67c29aca59135 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31690560715140e6847173104a2b0f2e, State: Initialized, Role: FOLLOWER
I20260709 13:12:18.444036 32187 consensus_queue.cc:260] T 709f871588d74579bab67c29aca59135 P 31690560715140e6847173104a2b0f2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.445696 32187 ts_tablet_manager.cc:1434] T 709f871588d74579bab67c29aca59135 P 31690560715140e6847173104a2b0f2e: Time spent starting tablet: real 0.005s	user 0.004s	sys 0.000s
I20260709 13:12:18.446863 32187 tablet_bootstrap.cc:492] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e: Bootstrap starting.
I20260709 13:12:18.451418 32187 tablet_bootstrap.cc:654] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:18.455832 32187 tablet_bootstrap.cc:492] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e: No bootstrap required, opened a new log
I20260709 13:12:18.456295 32187 ts_tablet_manager.cc:1403] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e: Time spent bootstrapping tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:12:18.458357 32187 raft_consensus.cc:359] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.458902 32187 raft_consensus.cc:385] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:18.459127 32187 raft_consensus.cc:740] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31690560715140e6847173104a2b0f2e, State: Initialized, Role: FOLLOWER
I20260709 13:12:18.459708 32187 consensus_queue.cc:260] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.461508 32187 ts_tablet_manager.cc:1434] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 13:12:18.511868 21756 ts_manager.cc:295] Set tserver state for 91778255eb264fe6a6dc208998e849ba to MAINTENANCE_MODE
I20260709 13:12:18.512651 21756 tablet_server.cc:179] TabletServer@127.21.63.1:0 shutting down...
I20260709 13:12:18.534272 21756 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:12:18.534934 21756 tablet_replica.cc:333] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba: stopping tablet replica
I20260709 13:12:18.535554 21756 raft_consensus.cc:2243] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:18.536190 21756 pending_rounds.cc:70] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba: Trying to abort 1 pending ops.
I20260709 13:12:18.536405 21756 pending_rounds.cc:77] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305636815954350080 op_type: NO_OP noop_request { }
I20260709 13:12:18.536725 21756 raft_consensus.cc:2889] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:12:18.537055 21756 raft_consensus.cc:2272] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:18.539325 21756 tablet_replica.cc:333] T 709f871588d74579bab67c29aca59135 P 91778255eb264fe6a6dc208998e849ba: stopping tablet replica
I20260709 13:12:18.539955 21756 raft_consensus.cc:2243] T 709f871588d74579bab67c29aca59135 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:18.540343 21756 raft_consensus.cc:2272] T 709f871588d74579bab67c29aca59135 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:18.542050 21756 tablet_replica.cc:333] T 0dfa6288e543429c99109ebbd5a223b9 P 91778255eb264fe6a6dc208998e849ba: stopping tablet replica
I20260709 13:12:18.542567 21756 raft_consensus.cc:2243] T 0dfa6288e543429c99109ebbd5a223b9 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:18.542922 21756 raft_consensus.cc:2272] T 0dfa6288e543429c99109ebbd5a223b9 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:18.543380 32234 raft_consensus.cc:493] T b40f21aa6dd54f64bed8e716a77df4d6 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:18.543921 32234 raft_consensus.cc:515] T b40f21aa6dd54f64bed8e716a77df4d6 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.545012 21756 tablet_replica.cc:333] T d0a7cae5f3084d0ba50763cc87888571 P 91778255eb264fe6a6dc208998e849ba: stopping tablet replica
I20260709 13:12:18.545747 32234 leader_election.cc:290] T b40f21aa6dd54f64bed8e716a77df4d6 P 31690560715140e6847173104a2b0f2e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979), e69f3153d090418380c201dddfef4c60 (127.21.63.3:40639)
I20260709 13:12:18.545816 21756 raft_consensus.cc:2243] T d0a7cae5f3084d0ba50763cc87888571 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:18.546499 21756 raft_consensus.cc:2272] T d0a7cae5f3084d0ba50763cc87888571 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:18.546736 32037 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 145 similar messages]
I20260709 13:12:18.546923 32146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b40f21aa6dd54f64bed8e716a77df4d6" candidate_uuid: "31690560715140e6847173104a2b0f2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e69f3153d090418380c201dddfef4c60" is_pre_election: true
I20260709 13:12:18.548830 21756 tablet_replica.cc:333] T 7b2f10befb2e474fbe821af8d5e92fd5 P 91778255eb264fe6a6dc208998e849ba: stopping tablet replica
I20260709 13:12:18.549422 21756 raft_consensus.cc:2243] T 7b2f10befb2e474fbe821af8d5e92fd5 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:18.549870 21756 raft_consensus.cc:2272] T 7b2f10befb2e474fbe821af8d5e92fd5 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:18.550333 32037 leader_election.cc:336] T b40f21aa6dd54f64bed8e716a77df4d6 P 31690560715140e6847173104a2b0f2e [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 13:12:18.550837 32037 leader_election.cc:304] T b40f21aa6dd54f64bed8e716a77df4d6 P 31690560715140e6847173104a2b0f2e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31690560715140e6847173104a2b0f2e; no voters: 91778255eb264fe6a6dc208998e849ba, e69f3153d090418380c201dddfef4c60
I20260709 13:12:18.551510 32234 raft_consensus.cc:2749] T b40f21aa6dd54f64bed8e716a77df4d6 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:18.552071 21756 tablet_replica.cc:333] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba: stopping tablet replica
I20260709 13:12:18.553018 21756 raft_consensus.cc:2243] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:18.553691 21756 pending_rounds.cc:70] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba: Trying to abort 1 pending ops.
I20260709 13:12:18.553900 21756 pending_rounds.cc:77] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305636816291409920 op_type: NO_OP noop_request { }
I20260709 13:12:18.554198 21756 raft_consensus.cc:2889] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:12:18.554530 21756 raft_consensus.cc:2272] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:18.556646 21756 tablet_replica.cc:333] T 3a76c5023ae14f8ba25de09e3a9b60de P 91778255eb264fe6a6dc208998e849ba: stopping tablet replica
I20260709 13:12:18.557276 21756 raft_consensus.cc:2243] T 3a76c5023ae14f8ba25de09e3a9b60de P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:18.557670 21756 raft_consensus.cc:2272] T 3a76c5023ae14f8ba25de09e3a9b60de P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:18.559412 21756 tablet_replica.cc:333] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba: stopping tablet replica
I20260709 13:12:18.559881 21756 raft_consensus.cc:2243] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:18.560444 21756 pending_rounds.cc:70] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba: Trying to abort 1 pending ops.
I20260709 13:12:18.560645 21756 pending_rounds.cc:77] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305636815976554496 op_type: NO_OP noop_request { }
I20260709 13:12:18.560976 21756 raft_consensus.cc:2889] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:12:18.561245 21756 raft_consensus.cc:2272] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:18.563293 21756 tablet_replica.cc:333] T b40f21aa6dd54f64bed8e716a77df4d6 P 91778255eb264fe6a6dc208998e849ba: stopping tablet replica
I20260709 13:12:18.563800 21756 raft_consensus.cc:2243] T b40f21aa6dd54f64bed8e716a77df4d6 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:18.564195 21756 raft_consensus.cc:2272] T b40f21aa6dd54f64bed8e716a77df4d6 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:18.566082 21756 tablet_replica.cc:333] T f9a7e52d74da4cd79abac3858391ae77 P 91778255eb264fe6a6dc208998e849ba: stopping tablet replica
I20260709 13:12:18.566708 21756 raft_consensus.cc:2243] T f9a7e52d74da4cd79abac3858391ae77 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:18.567109 21756 raft_consensus.cc:2272] T f9a7e52d74da4cd79abac3858391ae77 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:18.580199 32234 consensus_queue.cc:1048] T 3a76c5023ae14f8ba25de09e3a9b60de P 31690560715140e6847173104a2b0f2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:18.600926 21756 tablet_server.cc:196] TabletServer@127.21.63.1:0 shutdown complete.
W20260709 13:12:18.606196 32037 consensus_peers.cc:597] T 3a76c5023ae14f8ba25de09e3a9b60de P 31690560715140e6847173104a2b0f2e -> Peer 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979): Couldn't send request to peer 91778255eb264fe6a6dc208998e849ba. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:32979: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:18.676983 32068 raft_consensus.cc:3060] T b40f21aa6dd54f64bed8e716a77df4d6 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:18.683748 32226 consensus_queue.cc:1048] T b40f21aa6dd54f64bed8e716a77df4d6 P e69f3153d090418380c201dddfef4c60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:18.690289 32234 consensus_queue.cc:1048] T 7b2f10befb2e474fbe821af8d5e92fd5 P 31690560715140e6847173104a2b0f2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:12:18.700377 32111 consensus_peers.cc:597] T b40f21aa6dd54f64bed8e716a77df4d6 P e69f3153d090418380c201dddfef4c60 -> Peer 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979): Couldn't send request to peer 91778255eb264fe6a6dc208998e849ba. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:32979: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:18.712821 32037 consensus_peers.cc:597] T 7b2f10befb2e474fbe821af8d5e92fd5 P 31690560715140e6847173104a2b0f2e -> Peer 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979): Couldn't send request to peer 91778255eb264fe6a6dc208998e849ba. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:32979: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:18.720731 32233 raft_consensus.cc:493] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:18.721181 32233 raft_consensus.cc:515] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.723166 32233 leader_election.cc:290] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979), e69f3153d090418380c201dddfef4c60 (127.21.63.3:40639)
I20260709 13:12:18.724293 32146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e568fd2b2a149aeadd6c64bc70af9b7" candidate_uuid: "31690560715140e6847173104a2b0f2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e69f3153d090418380c201dddfef4c60" is_pre_election: true
I20260709 13:12:18.725289 32146 raft_consensus.cc:2393] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 31690560715140e6847173104a2b0f2e in current term 1: Already voted for candidate 91778255eb264fe6a6dc208998e849ba in this term.
W20260709 13:12:18.727336 32037 leader_election.cc:336] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979): Network error: Client connection negotiation failed: client connection to 127.21.63.1:32979: connect: Connection refused (error 111)
I20260709 13:12:18.727775 32037 leader_election.cc:304] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31690560715140e6847173104a2b0f2e; no voters: 91778255eb264fe6a6dc208998e849ba, e69f3153d090418380c201dddfef4c60
I20260709 13:12:18.728528 32233 raft_consensus.cc:3060] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:18.733539 32233 raft_consensus.cc:2749] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 13:12:18.752887 32111 consensus_peers.cc:597] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 -> Peer 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979): Couldn't send request to peer 91778255eb264fe6a6dc208998e849ba. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:32979: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:18.756524 32233 raft_consensus.cc:493] T 709f871588d74579bab67c29aca59135 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:18.757107 32233 raft_consensus.cc:515] T 709f871588d74579bab67c29aca59135 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.759068 32233 leader_election.cc:290] T 709f871588d74579bab67c29aca59135 P 31690560715140e6847173104a2b0f2e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979), e69f3153d090418380c201dddfef4c60 (127.21.63.3:40639)
I20260709 13:12:18.760237 32146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709f871588d74579bab67c29aca59135" candidate_uuid: "31690560715140e6847173104a2b0f2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e69f3153d090418380c201dddfef4c60" is_pre_election: true
W20260709 13:12:18.762498 32037 leader_election.cc:336] T 709f871588d74579bab67c29aca59135 P 31690560715140e6847173104a2b0f2e [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979): Network error: Client connection negotiation failed: client connection to 127.21.63.1:32979: connect: Connection refused (error 111)
I20260709 13:12:18.762826 32037 leader_election.cc:304] T 709f871588d74579bab67c29aca59135 P 31690560715140e6847173104a2b0f2e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31690560715140e6847173104a2b0f2e; no voters: 91778255eb264fe6a6dc208998e849ba, e69f3153d090418380c201dddfef4c60
I20260709 13:12:18.763464 32233 raft_consensus.cc:2749] T 709f871588d74579bab67c29aca59135 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:18.787213 32233 raft_consensus.cc:493] T 0dfa6288e543429c99109ebbd5a223b9 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:18.787705 32233 raft_consensus.cc:515] T 0dfa6288e543429c99109ebbd5a223b9 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:18.789497 32233 leader_election.cc:290] T 0dfa6288e543429c99109ebbd5a223b9 P 31690560715140e6847173104a2b0f2e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979), e69f3153d090418380c201dddfef4c60 (127.21.63.3:40639)
I20260709 13:12:18.790534 32146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dfa6288e543429c99109ebbd5a223b9" candidate_uuid: "31690560715140e6847173104a2b0f2e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e69f3153d090418380c201dddfef4c60" is_pre_election: true
W20260709 13:12:18.792964 32037 leader_election.cc:336] T 0dfa6288e543429c99109ebbd5a223b9 P 31690560715140e6847173104a2b0f2e [CANDIDATE]: Term 1 pre-election: RPC error from VoteRequest() call to peer 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979): Network error: Client connection negotiation failed: client connection to 127.21.63.1:32979: connect: Connection refused (error 111)
I20260709 13:12:18.793341 32037 leader_election.cc:304] T 0dfa6288e543429c99109ebbd5a223b9 P 31690560715140e6847173104a2b0f2e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31690560715140e6847173104a2b0f2e; no voters: 91778255eb264fe6a6dc208998e849ba, e69f3153d090418380c201dddfef4c60
I20260709 13:12:18.793970 32233 raft_consensus.cc:2749] T 0dfa6288e543429c99109ebbd5a223b9 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:18.804338 32233 consensus_queue.cc:1048] T d0a7cae5f3084d0ba50763cc87888571 P 31690560715140e6847173104a2b0f2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:12:18.818873 32037 consensus_peers.cc:597] T d0a7cae5f3084d0ba50763cc87888571 P 31690560715140e6847173104a2b0f2e -> Peer 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979): Couldn't send request to peer 91778255eb264fe6a6dc208998e849ba. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:32979: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:18.852391 32068 raft_consensus.cc:3060] T 0dfa6288e543429c99109ebbd5a223b9 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:12:18.857550 31943 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:12:18.858853 32235 consensus_queue.cc:1048] T 0dfa6288e543429c99109ebbd5a223b9 P e69f3153d090418380c201dddfef4c60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:18.864599 32070 raft_consensus.cc:3060] T 709f871588d74579bab67c29aca59135 P 31690560715140e6847173104a2b0f2e [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:12:18.874442 32111 consensus_peers.cc:597] T 0dfa6288e543429c99109ebbd5a223b9 P e69f3153d090418380c201dddfef4c60 -> Peer 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979): Couldn't send request to peer 91778255eb264fe6a6dc208998e849ba. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:32979: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:18.875960 32235 consensus_queue.cc:1048] T 709f871588d74579bab67c29aca59135 P e69f3153d090418380c201dddfef4c60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:12:18.877945 32111 consensus_peers.cc:597] T 709f871588d74579bab67c29aca59135 P e69f3153d090418380c201dddfef4c60 -> Peer 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979): Couldn't send request to peer 91778255eb264fe6a6dc208998e849ba. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:32979: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:18.878875 32235 consensus_queue.cc:1048] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:12:19.858590 31943 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:12:19.908557 32194 raft_consensus.cc:493] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:19.909193 32194 raft_consensus.cc:515] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:19.912920 32194 leader_election.cc:290] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979), 31690560715140e6847173104a2b0f2e (127.21.63.2:44689)
I20260709 13:12:19.914863 32070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e568fd2b2a149aeadd6c64bc70af9b7" candidate_uuid: "e69f3153d090418380c201dddfef4c60" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31690560715140e6847173104a2b0f2e" is_pre_election: true
I20260709 13:12:19.915529 32070 raft_consensus.cc:2468] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e69f3153d090418380c201dddfef4c60 in term 1.
W20260709 13:12:19.915864 32111 leader_election.cc:336] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979): Network error: Client connection negotiation failed: client connection to 127.21.63.1:32979: connect: Connection refused (error 111)
I20260709 13:12:19.916770 32111 leader_election.cc:304] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31690560715140e6847173104a2b0f2e, e69f3153d090418380c201dddfef4c60; no voters: 91778255eb264fe6a6dc208998e849ba
I20260709 13:12:19.917402 32194 raft_consensus.cc:2804] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:12:19.917853 32194 raft_consensus.cc:493] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:19.918187 32194 raft_consensus.cc:3060] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:19.923655 32194 raft_consensus.cc:515] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:19.925414 32194 leader_election.cc:290] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 2 election: Requested vote from peers 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979), 31690560715140e6847173104a2b0f2e (127.21.63.2:44689)
I20260709 13:12:19.926934 32070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e568fd2b2a149aeadd6c64bc70af9b7" candidate_uuid: "e69f3153d090418380c201dddfef4c60" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31690560715140e6847173104a2b0f2e"
I20260709 13:12:19.927614 32070 raft_consensus.cc:3060] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:12:19.928913 32111 leader_election.cc:336] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979): Network error: Client connection negotiation failed: client connection to 127.21.63.1:32979: connect: Connection refused (error 111)
I20260709 13:12:19.932848 32070 raft_consensus.cc:2468] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e69f3153d090418380c201dddfef4c60 in term 2.
I20260709 13:12:19.933952 32111 leader_election.cc:304] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31690560715140e6847173104a2b0f2e, e69f3153d090418380c201dddfef4c60; no voters: 91778255eb264fe6a6dc208998e849ba
I20260709 13:12:19.934702 32194 raft_consensus.cc:2804] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:19.935679 32194 raft_consensus.cc:697] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [term 2 LEADER]: Becoming Leader. State: Replica: e69f3153d090418380c201dddfef4c60, State: Running, Role: LEADER
I20260709 13:12:19.936527 32194 consensus_queue.cc:237] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:19.943769 31893 catalog_manager.cc:5697] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 reported cstate change: term changed from 1 to 2, leader changed from 91778255eb264fe6a6dc208998e849ba (127.21.63.1) to e69f3153d090418380c201dddfef4c60 (127.21.63.3). New cstate: current_term: 2 leader_uuid: "e69f3153d090418380c201dddfef4c60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:20.035581 32233 raft_consensus.cc:493] T d1fc79d86bee43c5b062ced8a090aab1 P 31690560715140e6847173104a2b0f2e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:20.036101 32233 raft_consensus.cc:515] T d1fc79d86bee43c5b062ced8a090aab1 P 31690560715140e6847173104a2b0f2e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:20.038043 32233 leader_election.cc:290] T d1fc79d86bee43c5b062ced8a090aab1 P 31690560715140e6847173104a2b0f2e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979), e69f3153d090418380c201dddfef4c60 (127.21.63.3:40639)
I20260709 13:12:20.039247 32146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1fc79d86bee43c5b062ced8a090aab1" candidate_uuid: "31690560715140e6847173104a2b0f2e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e69f3153d090418380c201dddfef4c60" is_pre_election: true
I20260709 13:12:20.039925 32146 raft_consensus.cc:2468] T d1fc79d86bee43c5b062ced8a090aab1 P e69f3153d090418380c201dddfef4c60 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31690560715140e6847173104a2b0f2e in term 1.
I20260709 13:12:20.041306 32036 leader_election.cc:304] T d1fc79d86bee43c5b062ced8a090aab1 P 31690560715140e6847173104a2b0f2e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31690560715140e6847173104a2b0f2e, e69f3153d090418380c201dddfef4c60; no voters: 
I20260709 13:12:20.042116 32233 raft_consensus.cc:2804] T d1fc79d86bee43c5b062ced8a090aab1 P 31690560715140e6847173104a2b0f2e [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 13:12:20.042127 32037 leader_election.cc:336] T d1fc79d86bee43c5b062ced8a090aab1 P 31690560715140e6847173104a2b0f2e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979): Network error: Client connection negotiation failed: client connection to 127.21.63.1:32979: connect: Connection refused (error 111)
I20260709 13:12:20.042616 32233 raft_consensus.cc:493] T d1fc79d86bee43c5b062ced8a090aab1 P 31690560715140e6847173104a2b0f2e [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:20.042987 32233 raft_consensus.cc:3060] T d1fc79d86bee43c5b062ced8a090aab1 P 31690560715140e6847173104a2b0f2e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:20.048786 32233 raft_consensus.cc:515] T d1fc79d86bee43c5b062ced8a090aab1 P 31690560715140e6847173104a2b0f2e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:20.050657 32233 leader_election.cc:290] T d1fc79d86bee43c5b062ced8a090aab1 P 31690560715140e6847173104a2b0f2e [CANDIDATE]: Term 2 election: Requested vote from peers 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979), e69f3153d090418380c201dddfef4c60 (127.21.63.3:40639)
I20260709 13:12:20.051765 32146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d1fc79d86bee43c5b062ced8a090aab1" candidate_uuid: "31690560715140e6847173104a2b0f2e" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e69f3153d090418380c201dddfef4c60"
I20260709 13:12:20.052537 32146 raft_consensus.cc:3060] T d1fc79d86bee43c5b062ced8a090aab1 P e69f3153d090418380c201dddfef4c60 [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:12:20.054546 32037 leader_election.cc:336] T d1fc79d86bee43c5b062ced8a090aab1 P 31690560715140e6847173104a2b0f2e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979): Network error: Client connection negotiation failed: client connection to 127.21.63.1:32979: connect: Connection refused (error 111)
I20260709 13:12:20.058724 32146 raft_consensus.cc:2468] T d1fc79d86bee43c5b062ced8a090aab1 P e69f3153d090418380c201dddfef4c60 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31690560715140e6847173104a2b0f2e in term 2.
I20260709 13:12:20.059777 32036 leader_election.cc:304] T d1fc79d86bee43c5b062ced8a090aab1 P 31690560715140e6847173104a2b0f2e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31690560715140e6847173104a2b0f2e, e69f3153d090418380c201dddfef4c60; no voters: 91778255eb264fe6a6dc208998e849ba
I20260709 13:12:20.060423 32233 raft_consensus.cc:2804] T d1fc79d86bee43c5b062ced8a090aab1 P 31690560715140e6847173104a2b0f2e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:20.061599 32233 raft_consensus.cc:697] T d1fc79d86bee43c5b062ced8a090aab1 P 31690560715140e6847173104a2b0f2e [term 2 LEADER]: Becoming Leader. State: Replica: 31690560715140e6847173104a2b0f2e, State: Running, Role: LEADER
I20260709 13:12:20.062357 32233 consensus_queue.cc:237] T d1fc79d86bee43c5b062ced8a090aab1 P 31690560715140e6847173104a2b0f2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:20.069267 31893 catalog_manager.cc:5697] T d1fc79d86bee43c5b062ced8a090aab1 P 31690560715140e6847173104a2b0f2e reported cstate change: term changed from 1 to 2, leader changed from 91778255eb264fe6a6dc208998e849ba (127.21.63.1) to 31690560715140e6847173104a2b0f2e (127.21.63.2). New cstate: current_term: 2 leader_uuid: "31690560715140e6847173104a2b0f2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:20.202745 32194 raft_consensus.cc:493] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:20.203334 32194 raft_consensus.cc:515] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:20.204986 32194 leader_election.cc:290] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979), 31690560715140e6847173104a2b0f2e (127.21.63.2:44689)
I20260709 13:12:20.206596 32070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ea7beb6ca5f4de4b64c0e8a9e9d99a7" candidate_uuid: "e69f3153d090418380c201dddfef4c60" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31690560715140e6847173104a2b0f2e" is_pre_election: true
I20260709 13:12:20.207270 32070 raft_consensus.cc:2468] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 31690560715140e6847173104a2b0f2e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e69f3153d090418380c201dddfef4c60 in term 1.
I20260709 13:12:20.208261 32111 leader_election.cc:304] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31690560715140e6847173104a2b0f2e, e69f3153d090418380c201dddfef4c60; no voters: 
W20260709 13:12:20.208962 32111 leader_election.cc:336] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979): Network error: Client connection negotiation failed: client connection to 127.21.63.1:32979: connect: Connection refused (error 111)
I20260709 13:12:20.208958 32194 raft_consensus.cc:2804] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 13:12:20.209529 32194 raft_consensus.cc:493] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:20.209802 32194 raft_consensus.cc:3060] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:20.214700 32194 raft_consensus.cc:515] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:20.216238 32194 leader_election.cc:290] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 2 election: Requested vote from peers 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979), 31690560715140e6847173104a2b0f2e (127.21.63.2:44689)
I20260709 13:12:20.218247 32070 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ea7beb6ca5f4de4b64c0e8a9e9d99a7" candidate_uuid: "e69f3153d090418380c201dddfef4c60" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31690560715140e6847173104a2b0f2e"
I20260709 13:12:20.218851 32070 raft_consensus.cc:3060] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 31690560715140e6847173104a2b0f2e [term 1 FOLLOWER]: Advancing to term 2
W20260709 13:12:20.220207 32111 leader_election.cc:336] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979): Network error: Client connection negotiation failed: client connection to 127.21.63.1:32979: connect: Connection refused (error 111)
I20260709 13:12:20.224242 32070 raft_consensus.cc:2468] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 31690560715140e6847173104a2b0f2e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e69f3153d090418380c201dddfef4c60 in term 2.
I20260709 13:12:20.225404 32111 leader_election.cc:304] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31690560715140e6847173104a2b0f2e, e69f3153d090418380c201dddfef4c60; no voters: 91778255eb264fe6a6dc208998e849ba
I20260709 13:12:20.226090 32194 raft_consensus.cc:2804] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:20.226480 32194 raft_consensus.cc:697] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60 [term 2 LEADER]: Becoming Leader. State: Replica: e69f3153d090418380c201dddfef4c60, State: Running, Role: LEADER
I20260709 13:12:20.227290 32194 consensus_queue.cc:237] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:20.234526 31893 catalog_manager.cc:5697] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60 reported cstate change: term changed from 1 to 2, leader changed from 91778255eb264fe6a6dc208998e849ba (127.21.63.1) to e69f3153d090418380c201dddfef4c60 (127.21.63.3). New cstate: current_term: 2 leader_uuid: "e69f3153d090418380c201dddfef4c60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:20.347664 32070 raft_consensus.cc:1275] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [term 2 FOLLOWER]: Refusing update from remote peer e69f3153d090418380c201dddfef4c60: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:12:20.349339 32267 consensus_queue.cc:1048] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:12:20.370703 32111 consensus_peers.cc:597] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 -> Peer 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979): Couldn't send request to peer 91778255eb264fe6a6dc208998e849ba. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:32979: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:20.482662 32037 consensus_peers.cc:597] T d1fc79d86bee43c5b062ced8a090aab1 P 31690560715140e6847173104a2b0f2e -> Peer 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979): Couldn't send request to peer 91778255eb264fe6a6dc208998e849ba. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:32979: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:20.673877 32146 raft_consensus.cc:1275] T d1fc79d86bee43c5b062ced8a090aab1 P e69f3153d090418380c201dddfef4c60 [term 2 FOLLOWER]: Refusing update from remote peer 31690560715140e6847173104a2b0f2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:12:20.675191 32233 consensus_queue.cc:1048] T d1fc79d86bee43c5b062ced8a090aab1 P 31690560715140e6847173104a2b0f2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:20.774067 32068 raft_consensus.cc:1275] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 31690560715140e6847173104a2b0f2e [term 2 FOLLOWER]: Refusing update from remote peer e69f3153d090418380c201dddfef4c60: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 13:12:20.775444 32278 consensus_queue.cc:1048] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:12:20.789835 32111 consensus_peers.cc:597] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60 -> Peer 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979): Couldn't send request to peer 91778255eb264fe6a6dc208998e849ba. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:32979: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:20.859776 31943 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:12:20.920307 32111 consensus_peers.cc:597] T b40f21aa6dd54f64bed8e716a77df4d6 P e69f3153d090418380c201dddfef4c60 -> Peer 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979): Couldn't send request to peer 91778255eb264fe6a6dc208998e849ba. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:32979: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:21.088416 32111 consensus_peers.cc:597] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 -> Peer 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979): Couldn't send request to peer 91778255eb264fe6a6dc208998e849ba. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:32979: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:21.174636 32037 consensus_peers.cc:597] T 7b2f10befb2e474fbe821af8d5e92fd5 P 31690560715140e6847173104a2b0f2e -> Peer 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979): Couldn't send request to peer 91778255eb264fe6a6dc208998e849ba. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:32979: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:21.258172 32037 consensus_peers.cc:597] T 3a76c5023ae14f8ba25de09e3a9b60de P 31690560715140e6847173104a2b0f2e -> Peer 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979): Couldn't send request to peer 91778255eb264fe6a6dc208998e849ba. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:32979: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:21.306710 32111 consensus_peers.cc:597] T 0dfa6288e543429c99109ebbd5a223b9 P e69f3153d090418380c201dddfef4c60 -> Peer 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979): Couldn't send request to peer 91778255eb264fe6a6dc208998e849ba. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:32979: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:21.470569 32111 consensus_peers.cc:597] T 709f871588d74579bab67c29aca59135 P e69f3153d090418380c201dddfef4c60 -> Peer 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979): Couldn't send request to peer 91778255eb264fe6a6dc208998e849ba. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:32979: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:21.526561 32037 consensus_peers.cc:597] T d0a7cae5f3084d0ba50763cc87888571 P 31690560715140e6847173104a2b0f2e -> Peer 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979): Couldn't send request to peer 91778255eb264fe6a6dc208998e849ba. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:32979: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:21.861138 31943 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:12:22.862226 31943 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:12:22.968111 32037 consensus_peers.cc:597] T d1fc79d86bee43c5b062ced8a090aab1 P 31690560715140e6847173104a2b0f2e -> Peer 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979): Couldn't send request to peer 91778255eb264fe6a6dc208998e849ba. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:32979: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:23.065932 32111 consensus_peers.cc:597] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 -> Peer 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979): Couldn't send request to peer 91778255eb264fe6a6dc208998e849ba. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:32979: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:23.193766 32111 consensus_peers.cc:597] T b40f21aa6dd54f64bed8e716a77df4d6 P e69f3153d090418380c201dddfef4c60 -> Peer 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979): Couldn't send request to peer 91778255eb264fe6a6dc208998e849ba. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:32979: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:12:23.256911 32111 consensus_peers.cc:597] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60 -> Peer 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979): Couldn't send request to peer 91778255eb264fe6a6dc208998e849ba. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:32979: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 13:12:23.477543 32111 consensus_peers.cc:597] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 -> Peer 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979): Couldn't send request to peer 91778255eb264fe6a6dc208998e849ba. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:32979: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 13:12:23.585461 32111 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.21.63.1:32979: connect: Connection refused (error 111) [suppressed 100 similar messages]
W20260709 13:12:23.611467 32037 consensus_peers.cc:597] T 3a76c5023ae14f8ba25de09e3a9b60de P 31690560715140e6847173104a2b0f2e -> Peer 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979): Couldn't send request to peer 91778255eb264fe6a6dc208998e849ba. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:32979: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:12:23.621315 21756 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:12:23.629324 32286 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:23.630330 32287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:23.630910 32289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:23.633158 21756 server_base.cc:1061] running on GCE node
W20260709 13:12:23.634680 32037 consensus_peers.cc:597] T 7b2f10befb2e474fbe821af8d5e92fd5 P 31690560715140e6847173104a2b0f2e -> Peer 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979): Couldn't send request to peer 91778255eb264fe6a6dc208998e849ba. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:32979: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:12:23.637200 21756 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:12:23.637485 21756 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:12:23.637660 21756 hybrid_clock.cc:648] HybridClock initialized: now 1783602743637644 us; error 0 us; skew 500 ppm
I20260709 13:12:23.638378 21756 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:12:23.641456 21756 webserver.cc:533] Webserver started at http://127.21.63.1:34303/ using document root <none> and password file <none>
I20260709 13:12:23.642041 21756 fs_manager.cc:362] Metadata directory not provided
I20260709 13:12:23.642222 21756 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 13:12:23.646552 21756 fs_manager.cc:714] Time spent opening directory manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 13:12:23.651151 32294 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:23.651901 21756 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.001s
I20260709 13:12:23.652233 21756 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602429420501-21756-0/minicluster-data/ts-0-root
uuid: "91778255eb264fe6a6dc208998e849ba"
format_stamp: "Formatted at 2026-07-09 13:12:16 on dist-test-slave-c6mt"
I20260709 13:12:23.652631 21756 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602429420501-21756-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602429420501-21756-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783602429420501-21756-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:12:23.688351 21756 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:12:23.689718 21756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:12:23.691313 21756 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:12:23.695057 32301 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
W20260709 13:12:23.735126 32111 consensus_peers.cc:597] T 0dfa6288e543429c99109ebbd5a223b9 P e69f3153d090418380c201dddfef4c60 -> Peer 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979): Couldn't send request to peer 91778255eb264fe6a6dc208998e849ba. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:32979: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:12:23.756484 21756 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 13:12:23.756898 21756 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.063s	user 0.002s	sys 0.001s
I20260709 13:12:23.757212 21756 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 13:12:23.762882 32301 tablet_bootstrap.cc:492] T 0dfa6288e543429c99109ebbd5a223b9 P 91778255eb264fe6a6dc208998e849ba: Bootstrap starting.
I20260709 13:12:23.775008 32301 tablet_bootstrap.cc:492] T 0dfa6288e543429c99109ebbd5a223b9 P 91778255eb264fe6a6dc208998e849ba: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:23.775821 32301 tablet_bootstrap.cc:492] T 0dfa6288e543429c99109ebbd5a223b9 P 91778255eb264fe6a6dc208998e849ba: Bootstrap complete.
I20260709 13:12:23.776520 32301 ts_tablet_manager.cc:1403] T 0dfa6288e543429c99109ebbd5a223b9 P 91778255eb264fe6a6dc208998e849ba: Time spent bootstrapping tablet: real 0.014s	user 0.010s	sys 0.000s
I20260709 13:12:23.779377 32301 raft_consensus.cc:359] T 0dfa6288e543429c99109ebbd5a223b9 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:23.780369 32301 raft_consensus.cc:740] T 0dfa6288e543429c99109ebbd5a223b9 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91778255eb264fe6a6dc208998e849ba, State: Initialized, Role: FOLLOWER
I20260709 13:12:23.781260 32301 consensus_queue.cc:260] T 0dfa6288e543429c99109ebbd5a223b9 P 91778255eb264fe6a6dc208998e849ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:23.784262 32301 ts_tablet_manager.cc:1434] T 0dfa6288e543429c99109ebbd5a223b9 P 91778255eb264fe6a6dc208998e849ba: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:12:23.785161 32301 tablet_bootstrap.cc:492] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba: Bootstrap starting.
I20260709 13:12:23.794857 21756 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 13:12:23.795176 21756 ts_tablet_manager.cc:595] Time spent register tablets: real 0.038s	user 0.036s	sys 0.000s
I20260709 13:12:23.799727 32301 tablet_bootstrap.cc:492] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:12:23.800741 32301 tablet_bootstrap.cc:492] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba: Bootstrap complete.
I20260709 13:12:23.801359 32301 ts_tablet_manager.cc:1403] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba: Time spent bootstrapping tablet: real 0.016s	user 0.015s	sys 0.000s
I20260709 13:12:23.803855 32301 raft_consensus.cc:359] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:23.804860 32301 raft_consensus.cc:740] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91778255eb264fe6a6dc208998e849ba, State: Initialized, Role: FOLLOWER
I20260709 13:12:23.805778 32301 consensus_queue.cc:260] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:23.807991 32301 ts_tablet_manager.cc:1434] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:12:23.809111 32301 tablet_bootstrap.cc:492] T 709f871588d74579bab67c29aca59135 P 91778255eb264fe6a6dc208998e849ba: Bootstrap starting.
W20260709 13:12:23.817894 32111 consensus_peers.cc:597] T 709f871588d74579bab67c29aca59135 P e69f3153d090418380c201dddfef4c60 -> Peer 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979): Couldn't send request to peer 91778255eb264fe6a6dc208998e849ba. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:32979: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:12:23.829376 32301 tablet_bootstrap.cc:492] T 709f871588d74579bab67c29aca59135 P 91778255eb264fe6a6dc208998e849ba: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:23.830679 32301 tablet_bootstrap.cc:492] T 709f871588d74579bab67c29aca59135 P 91778255eb264fe6a6dc208998e849ba: Bootstrap complete.
I20260709 13:12:23.831624 32301 ts_tablet_manager.cc:1403] T 709f871588d74579bab67c29aca59135 P 91778255eb264fe6a6dc208998e849ba: Time spent bootstrapping tablet: real 0.023s	user 0.021s	sys 0.000s
I20260709 13:12:23.846773 32301 raft_consensus.cc:359] T 709f871588d74579bab67c29aca59135 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:23.847851 32301 raft_consensus.cc:740] T 709f871588d74579bab67c29aca59135 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91778255eb264fe6a6dc208998e849ba, State: Initialized, Role: FOLLOWER
I20260709 13:12:23.848928 32301 consensus_queue.cc:260] T 709f871588d74579bab67c29aca59135 P 91778255eb264fe6a6dc208998e849ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:23.852523 32301 ts_tablet_manager.cc:1434] T 709f871588d74579bab67c29aca59135 P 91778255eb264fe6a6dc208998e849ba: Time spent starting tablet: real 0.020s	user 0.009s	sys 0.003s
I20260709 13:12:23.854187 32301 tablet_bootstrap.cc:492] T d0a7cae5f3084d0ba50763cc87888571 P 91778255eb264fe6a6dc208998e849ba: Bootstrap starting.
W20260709 13:12:23.863222 31943 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 13:12:23.864925 32037 consensus_peers.cc:597] T d0a7cae5f3084d0ba50763cc87888571 P 31690560715140e6847173104a2b0f2e -> Peer 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979): Couldn't send request to peer 91778255eb264fe6a6dc208998e849ba. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:32979: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 13:12:23.870000 32301 tablet_bootstrap.cc:492] T d0a7cae5f3084d0ba50763cc87888571 P 91778255eb264fe6a6dc208998e849ba: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:23.870878 32301 tablet_bootstrap.cc:492] T d0a7cae5f3084d0ba50763cc87888571 P 91778255eb264fe6a6dc208998e849ba: Bootstrap complete.
I20260709 13:12:23.871457 32301 ts_tablet_manager.cc:1403] T d0a7cae5f3084d0ba50763cc87888571 P 91778255eb264fe6a6dc208998e849ba: Time spent bootstrapping tablet: real 0.018s	user 0.009s	sys 0.004s
I20260709 13:12:23.873837 32301 raft_consensus.cc:359] T d0a7cae5f3084d0ba50763cc87888571 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:23.874404 32301 raft_consensus.cc:740] T d0a7cae5f3084d0ba50763cc87888571 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91778255eb264fe6a6dc208998e849ba, State: Initialized, Role: FOLLOWER
I20260709 13:12:23.875033 32301 consensus_queue.cc:260] T d0a7cae5f3084d0ba50763cc87888571 P 91778255eb264fe6a6dc208998e849ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:23.876794 32301 ts_tablet_manager.cc:1434] T d0a7cae5f3084d0ba50763cc87888571 P 91778255eb264fe6a6dc208998e849ba: Time spent starting tablet: real 0.005s	user 0.008s	sys 0.000s
I20260709 13:12:23.877494 32301 tablet_bootstrap.cc:492] T b40f21aa6dd54f64bed8e716a77df4d6 P 91778255eb264fe6a6dc208998e849ba: Bootstrap starting.
I20260709 13:12:23.890995 32301 tablet_bootstrap.cc:492] T b40f21aa6dd54f64bed8e716a77df4d6 P 91778255eb264fe6a6dc208998e849ba: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:23.891703 32301 tablet_bootstrap.cc:492] T b40f21aa6dd54f64bed8e716a77df4d6 P 91778255eb264fe6a6dc208998e849ba: Bootstrap complete.
I20260709 13:12:23.892230 21756 rpc_server.cc:307] RPC server started. Bound to: 127.21.63.1:32979
I20260709 13:12:23.892354 32365 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.63.1:32979 every 8 connection(s)
I20260709 13:12:23.892241 32301 ts_tablet_manager.cc:1403] T b40f21aa6dd54f64bed8e716a77df4d6 P 91778255eb264fe6a6dc208998e849ba: Time spent bootstrapping tablet: real 0.015s	user 0.008s	sys 0.004s
I20260709 13:12:23.895218 32301 raft_consensus.cc:359] T b40f21aa6dd54f64bed8e716a77df4d6 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:23.895778 32301 raft_consensus.cc:740] T b40f21aa6dd54f64bed8e716a77df4d6 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91778255eb264fe6a6dc208998e849ba, State: Initialized, Role: FOLLOWER
I20260709 13:12:23.896720 32301 consensus_queue.cc:260] T b40f21aa6dd54f64bed8e716a77df4d6 P 91778255eb264fe6a6dc208998e849ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:23.898588 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:23.898695 32301 ts_tablet_manager.cc:1434] T b40f21aa6dd54f64bed8e716a77df4d6 P 91778255eb264fe6a6dc208998e849ba: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:12:23.899840 32301 tablet_bootstrap.cc:492] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba: Bootstrap starting.
I20260709 13:12:23.929699 32301 tablet_bootstrap.cc:492] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:12:23.930523 32301 tablet_bootstrap.cc:492] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba: Bootstrap complete.
I20260709 13:12:23.931115 32301 ts_tablet_manager.cc:1403] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba: Time spent bootstrapping tablet: real 0.032s	user 0.025s	sys 0.004s
I20260709 13:12:23.933928 32301 raft_consensus.cc:359] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:23.934948 32301 raft_consensus.cc:740] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91778255eb264fe6a6dc208998e849ba, State: Initialized, Role: FOLLOWER
I20260709 13:12:23.935916 32366 heartbeater.cc:344] Connected to a master server at 127.21.63.62:36925
I20260709 13:12:23.935880 32301 consensus_queue.cc:260] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:23.936496 32366 heartbeater.cc:461] Registering TS with master...
I20260709 13:12:23.937711 32366 heartbeater.cc:507] Master 127.21.63.62:36925 requested a full tablet report, sending...
I20260709 13:12:23.938292 32301 ts_tablet_manager.cc:1434] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:12:23.939251 32301 tablet_bootstrap.cc:492] T 7b2f10befb2e474fbe821af8d5e92fd5 P 91778255eb264fe6a6dc208998e849ba: Bootstrap starting.
I20260709 13:12:23.939954 32146 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5e568fd2b2a149aeadd6c64bc70af9b7"
dest_uuid: "e69f3153d090418380c201dddfef4c60"
mode: GRACEFUL
new_leader_uuid: "31690560715140e6847173104a2b0f2e"
 from {username='slave'} at 127.0.0.1:55620
I20260709 13:12:23.940807 32146 raft_consensus.cc:606] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [term 2 LEADER]: Received request to transfer leadership to TS 31690560715140e6847173104a2b0f2e
I20260709 13:12:23.944373 32146 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f9a7e52d74da4cd79abac3858391ae77"
dest_uuid: "e69f3153d090418380c201dddfef4c60"
mode: GRACEFUL
new_leader_uuid: "31690560715140e6847173104a2b0f2e"
 from {username='slave'} at 127.0.0.1:55620
I20260709 13:12:23.945245 32146 raft_consensus.cc:606] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 [term 1 LEADER]: Received request to transfer leadership to TS 31690560715140e6847173104a2b0f2e
I20260709 13:12:23.946385 31893 ts_manager.cc:194] Re-registered known tserver with Master: 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979)
I20260709 13:12:23.947402 21756 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:12:23.948410 21756 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:12:23.949685 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:23.961037 31893 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49460
I20260709 13:12:23.962248 32301 tablet_bootstrap.cc:492] T 7b2f10befb2e474fbe821af8d5e92fd5 P 91778255eb264fe6a6dc208998e849ba: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:23.963414 32301 tablet_bootstrap.cc:492] T 7b2f10befb2e474fbe821af8d5e92fd5 P 91778255eb264fe6a6dc208998e849ba: Bootstrap complete.
I20260709 13:12:23.964166 32301 ts_tablet_manager.cc:1403] T 7b2f10befb2e474fbe821af8d5e92fd5 P 91778255eb264fe6a6dc208998e849ba: Time spent bootstrapping tablet: real 0.025s	user 0.015s	sys 0.007s
I20260709 13:12:23.966312 32366 heartbeater.cc:499] Master 127.21.63.62:36925 was elected leader, sending a full tablet report...
I20260709 13:12:23.966971 32301 raft_consensus.cc:359] T 7b2f10befb2e474fbe821af8d5e92fd5 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:23.967787 32301 raft_consensus.cc:740] T 7b2f10befb2e474fbe821af8d5e92fd5 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91778255eb264fe6a6dc208998e849ba, State: Initialized, Role: FOLLOWER
I20260709 13:12:23.968746 32301 consensus_queue.cc:260] T 7b2f10befb2e474fbe821af8d5e92fd5 P 91778255eb264fe6a6dc208998e849ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:23.974318 32301 ts_tablet_manager.cc:1434] T 7b2f10befb2e474fbe821af8d5e92fd5 P 91778255eb264fe6a6dc208998e849ba: Time spent starting tablet: real 0.010s	user 0.000s	sys 0.005s
I20260709 13:12:23.975250 32301 tablet_bootstrap.cc:492] T 3a76c5023ae14f8ba25de09e3a9b60de P 91778255eb264fe6a6dc208998e849ba: Bootstrap starting.
I20260709 13:12:23.979094 32339 raft_consensus.cc:3060] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:23.988854 32339 pending_rounds.cc:85] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba: Aborting all ops after (but not including) 0
I20260709 13:12:23.989180 32339 pending_rounds.cc:107] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:12:23.989586 32339 raft_consensus.cc:2889] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:23.991647 32301 tablet_bootstrap.cc:492] T 3a76c5023ae14f8ba25de09e3a9b60de P 91778255eb264fe6a6dc208998e849ba: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:23.992444 32301 tablet_bootstrap.cc:492] T 3a76c5023ae14f8ba25de09e3a9b60de P 91778255eb264fe6a6dc208998e849ba: Bootstrap complete.
I20260709 13:12:23.993187 32301 ts_tablet_manager.cc:1403] T 3a76c5023ae14f8ba25de09e3a9b60de P 91778255eb264fe6a6dc208998e849ba: Time spent bootstrapping tablet: real 0.018s	user 0.016s	sys 0.001s
I20260709 13:12:23.995602 32301 raft_consensus.cc:359] T 3a76c5023ae14f8ba25de09e3a9b60de P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:23.996357 32301 raft_consensus.cc:740] T 3a76c5023ae14f8ba25de09e3a9b60de P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91778255eb264fe6a6dc208998e849ba, State: Initialized, Role: FOLLOWER
I20260709 13:12:23.997078 32301 consensus_queue.cc:260] T 3a76c5023ae14f8ba25de09e3a9b60de P 91778255eb264fe6a6dc208998e849ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:23.999502 32301 ts_tablet_manager.cc:1434] T 3a76c5023ae14f8ba25de09e3a9b60de P 91778255eb264fe6a6dc208998e849ba: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:12:24.000370 32301 tablet_bootstrap.cc:492] T f9a7e52d74da4cd79abac3858391ae77 P 91778255eb264fe6a6dc208998e849ba: Bootstrap starting.
I20260709 13:12:24.013208 32301 tablet_bootstrap.cc:492] T f9a7e52d74da4cd79abac3858391ae77 P 91778255eb264fe6a6dc208998e849ba: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 13:12:24.013998 32301 tablet_bootstrap.cc:492] T f9a7e52d74da4cd79abac3858391ae77 P 91778255eb264fe6a6dc208998e849ba: Bootstrap complete.
I20260709 13:12:24.014564 32301 ts_tablet_manager.cc:1403] T f9a7e52d74da4cd79abac3858391ae77 P 91778255eb264fe6a6dc208998e849ba: Time spent bootstrapping tablet: real 0.014s	user 0.008s	sys 0.004s
I20260709 13:12:24.016443 32301 raft_consensus.cc:359] T f9a7e52d74da4cd79abac3858391ae77 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } }
I20260709 13:12:24.016957 32301 raft_consensus.cc:740] T f9a7e52d74da4cd79abac3858391ae77 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91778255eb264fe6a6dc208998e849ba, State: Initialized, Role: FOLLOWER
I20260709 13:12:24.017542 32301 consensus_queue.cc:260] T f9a7e52d74da4cd79abac3858391ae77 P 91778255eb264fe6a6dc208998e849ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } }
I20260709 13:12:24.018874 32301 ts_tablet_manager.cc:1434] T f9a7e52d74da4cd79abac3858391ae77 P 91778255eb264fe6a6dc208998e849ba: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:12:24.019728 32301 tablet_bootstrap.cc:492] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba: Bootstrap starting.
I20260709 13:12:24.035192 32301 tablet_bootstrap.cc:492] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 13:12:24.035929 32301 tablet_bootstrap.cc:492] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba: Bootstrap complete.
I20260709 13:12:24.036466 32301 ts_tablet_manager.cc:1403] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba: Time spent bootstrapping tablet: real 0.017s	user 0.009s	sys 0.004s
I20260709 13:12:24.038455 32301 raft_consensus.cc:359] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:24.039049 32301 raft_consensus.cc:740] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91778255eb264fe6a6dc208998e849ba, State: Initialized, Role: FOLLOWER
I20260709 13:12:24.039594 32301 consensus_queue.cc:260] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:24.041165 32301 ts_tablet_manager.cc:1434] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba: Time spent starting tablet: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:12:24.067127 32339 raft_consensus.cc:3060] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:24.072584 32339 pending_rounds.cc:85] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba: Aborting all ops after (but not including) 0
I20260709 13:12:24.072882 32339 pending_rounds.cc:107] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:12:24.073107 32339 raft_consensus.cc:2889] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:24.125854 32311 raft_consensus.cc:993] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60: : Instructing follower 31690560715140e6847173104a2b0f2e to start an election
I20260709 13:12:24.126451 32283 raft_consensus.cc:1081] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [term 2 LEADER]: Signalling peer 31690560715140e6847173104a2b0f2e to start an election
I20260709 13:12:24.127897 32068 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5e568fd2b2a149aeadd6c64bc70af9b7"
dest_uuid: "31690560715140e6847173104a2b0f2e"
 from {username='slave'} at 127.0.0.1:36594
I20260709 13:12:24.128480 32068 raft_consensus.cc:493] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:24.129011 32068 raft_consensus.cc:3060] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:24.135492 32068 raft_consensus.cc:515] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:24.137691 32068 leader_election.cc:290] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [CANDIDATE]: Term 3 election: Requested vote from peers 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979), e69f3153d090418380c201dddfef4c60 (127.21.63.3:40639)
I20260709 13:12:24.138746 32339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e568fd2b2a149aeadd6c64bc70af9b7" candidate_uuid: "31690560715140e6847173104a2b0f2e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "91778255eb264fe6a6dc208998e849ba"
I20260709 13:12:24.138936 32146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e568fd2b2a149aeadd6c64bc70af9b7" candidate_uuid: "31690560715140e6847173104a2b0f2e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "e69f3153d090418380c201dddfef4c60"
I20260709 13:12:24.139526 32339 raft_consensus.cc:3060] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:24.139607 32146 raft_consensus.cc:3055] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:24.140074 32146 raft_consensus.cc:740] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e69f3153d090418380c201dddfef4c60, State: Running, Role: LEADER
I20260709 13:12:24.141038 32146 consensus_queue.cc:260] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:24.142268 32146 raft_consensus.cc:3060] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:24.145066 32339 raft_consensus.cc:2468] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31690560715140e6847173104a2b0f2e in term 3.
I20260709 13:12:24.146332 32037 leader_election.cc:304] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31690560715140e6847173104a2b0f2e, 91778255eb264fe6a6dc208998e849ba; no voters: 
I20260709 13:12:24.147469 32328 raft_consensus.cc:2804] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:24.147897 32328 raft_consensus.cc:697] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [term 3 LEADER]: Becoming Leader. State: Replica: 31690560715140e6847173104a2b0f2e, State: Running, Role: LEADER
I20260709 13:12:24.148865 32328 consensus_queue.cc:237] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:24.149515 32146 raft_consensus.cc:2468] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31690560715140e6847173104a2b0f2e in term 3.
I20260709 13:12:24.157982 31893 catalog_manager.cc:5697] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e reported cstate change: term changed from 2 to 3, leader changed from e69f3153d090418380c201dddfef4c60 (127.21.63.3) to 31690560715140e6847173104a2b0f2e (127.21.63.2). New cstate: current_term: 3 leader_uuid: "31690560715140e6847173104a2b0f2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:24.231415 32337 raft_consensus.cc:3060] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:24.238794 32337 pending_rounds.cc:85] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba: Aborting all ops after (but not including) 0
I20260709 13:12:24.239048 32337 pending_rounds.cc:107] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 13:12:24.239291 32337 raft_consensus.cc:2889] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 13:12:24.253281 32311 raft_consensus.cc:993] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60: : Instructing follower 31690560715140e6847173104a2b0f2e to start an election
I20260709 13:12:24.253762 32283 raft_consensus.cc:1081] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 [term 1 LEADER]: Signalling peer 31690560715140e6847173104a2b0f2e to start an election
I20260709 13:12:24.255190 32068 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f9a7e52d74da4cd79abac3858391ae77"
dest_uuid: "31690560715140e6847173104a2b0f2e"
 from {username='slave'} at 127.0.0.1:36594
I20260709 13:12:24.255654 32068 raft_consensus.cc:493] T f9a7e52d74da4cd79abac3858391ae77 P 31690560715140e6847173104a2b0f2e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:24.255935 32068 raft_consensus.cc:3060] T f9a7e52d74da4cd79abac3858391ae77 P 31690560715140e6847173104a2b0f2e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:24.260942 32068 raft_consensus.cc:515] T f9a7e52d74da4cd79abac3858391ae77 P 31690560715140e6847173104a2b0f2e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } }
I20260709 13:12:24.262462 32068 leader_election.cc:290] T f9a7e52d74da4cd79abac3858391ae77 P 31690560715140e6847173104a2b0f2e [CANDIDATE]: Term 2 election: Requested vote from peers 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979), e69f3153d090418380c201dddfef4c60 (127.21.63.3:40639)
I20260709 13:12:24.263252 32337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9a7e52d74da4cd79abac3858391ae77" candidate_uuid: "31690560715140e6847173104a2b0f2e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91778255eb264fe6a6dc208998e849ba"
I20260709 13:12:24.263427 32146 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9a7e52d74da4cd79abac3858391ae77" candidate_uuid: "31690560715140e6847173104a2b0f2e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e69f3153d090418380c201dddfef4c60"
I20260709 13:12:24.263926 32337 raft_consensus.cc:3060] T f9a7e52d74da4cd79abac3858391ae77 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:24.264089 32146 raft_consensus.cc:3055] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:24.264488 32146 raft_consensus.cc:740] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e69f3153d090418380c201dddfef4c60, State: Running, Role: LEADER
I20260709 13:12:24.265285 32146 consensus_queue.cc:260] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } }
I20260709 13:12:24.266237 32146 raft_consensus.cc:3060] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:24.269304 32337 raft_consensus.cc:2468] T f9a7e52d74da4cd79abac3858391ae77 P 91778255eb264fe6a6dc208998e849ba [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31690560715140e6847173104a2b0f2e in term 2.
I20260709 13:12:24.270395 32037 leader_election.cc:304] T f9a7e52d74da4cd79abac3858391ae77 P 31690560715140e6847173104a2b0f2e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31690560715140e6847173104a2b0f2e, 91778255eb264fe6a6dc208998e849ba; no voters: 
I20260709 13:12:24.271281 32328 raft_consensus.cc:2804] T f9a7e52d74da4cd79abac3858391ae77 P 31690560715140e6847173104a2b0f2e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:24.271715 32328 raft_consensus.cc:697] T f9a7e52d74da4cd79abac3858391ae77 P 31690560715140e6847173104a2b0f2e [term 2 LEADER]: Becoming Leader. State: Replica: 31690560715140e6847173104a2b0f2e, State: Running, Role: LEADER
I20260709 13:12:24.272397 32146 raft_consensus.cc:2468] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31690560715140e6847173104a2b0f2e in term 2.
I20260709 13:12:24.272740 32328 consensus_queue.cc:237] T f9a7e52d74da4cd79abac3858391ae77 P 31690560715140e6847173104a2b0f2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } }
I20260709 13:12:24.279605 31893 catalog_manager.cc:5697] T f9a7e52d74da4cd79abac3858391ae77 P 31690560715140e6847173104a2b0f2e reported cstate change: term changed from 1 to 2, leader changed from e69f3153d090418380c201dddfef4c60 (127.21.63.3) to 31690560715140e6847173104a2b0f2e (127.21.63.2). New cstate: current_term: 2 leader_uuid: "31690560715140e6847173104a2b0f2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:24.625743 32146 raft_consensus.cc:1275] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [term 3 FOLLOWER]: Refusing update from remote peer 31690560715140e6847173104a2b0f2e: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:12:24.626952 32285 consensus_queue.cc:1048] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:24.638182 32337 raft_consensus.cc:1275] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba [term 3 FOLLOWER]: Refusing update from remote peer 31690560715140e6847173104a2b0f2e: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:12:24.639796 32285 consensus_queue.cc:1048] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:24.696928 32337 raft_consensus.cc:1275] T f9a7e52d74da4cd79abac3858391ae77 P 91778255eb264fe6a6dc208998e849ba [term 2 FOLLOWER]: Refusing update from remote peer 31690560715140e6847173104a2b0f2e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:24.698335 32285 consensus_queue.cc:1048] T f9a7e52d74da4cd79abac3858391ae77 P 31690560715140e6847173104a2b0f2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:24.708407 32146 raft_consensus.cc:1275] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 [term 2 FOLLOWER]: Refusing update from remote peer 31690560715140e6847173104a2b0f2e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:24.710703 32395 consensus_queue.cc:1048] T f9a7e52d74da4cd79abac3858391ae77 P 31690560715140e6847173104a2b0f2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:12:24.864437 31943 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:12:24.963511 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:24.981824 32068 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a76c5023ae14f8ba25de09e3a9b60de"
dest_uuid: "31690560715140e6847173104a2b0f2e"
mode: GRACEFUL
new_leader_uuid: "e69f3153d090418380c201dddfef4c60"
 from {username='slave'} at 127.0.0.1:36610
I20260709 13:12:24.982460 32068 raft_consensus.cc:606] T 3a76c5023ae14f8ba25de09e3a9b60de P 31690560715140e6847173104a2b0f2e [term 1 LEADER]: Received request to transfer leadership to TS e69f3153d090418380c201dddfef4c60
I20260709 13:12:24.985621 32068 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5e568fd2b2a149aeadd6c64bc70af9b7"
dest_uuid: "31690560715140e6847173104a2b0f2e"
mode: GRACEFUL
new_leader_uuid: "e69f3153d090418380c201dddfef4c60"
 from {username='slave'} at 127.0.0.1:36610
I20260709 13:12:24.986127 32068 raft_consensus.cc:606] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [term 3 LEADER]: Received request to transfer leadership to TS e69f3153d090418380c201dddfef4c60
I20260709 13:12:24.987257 21756 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:12:24.987715 21756 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:12:24.988286 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:25.018195 32395 raft_consensus.cc:993] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e: : Instructing follower e69f3153d090418380c201dddfef4c60 to start an election
I20260709 13:12:25.018777 32395 raft_consensus.cc:1081] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [term 3 LEADER]: Signalling peer e69f3153d090418380c201dddfef4c60 to start an election
I20260709 13:12:25.020233 32146 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5e568fd2b2a149aeadd6c64bc70af9b7"
dest_uuid: "e69f3153d090418380c201dddfef4c60"
 from {username='slave'} at 127.0.0.1:55614
I20260709 13:12:25.020884 32146 raft_consensus.cc:493] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:25.021322 32146 raft_consensus.cc:3060] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:25.027247 32146 raft_consensus.cc:515] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:25.029107 32146 leader_election.cc:290] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 4 election: Requested vote from peers 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979), 31690560715140e6847173104a2b0f2e (127.21.63.2:44689)
I20260709 13:12:25.029798 32337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e568fd2b2a149aeadd6c64bc70af9b7" candidate_uuid: "e69f3153d090418380c201dddfef4c60" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "91778255eb264fe6a6dc208998e849ba"
I20260709 13:12:25.030231 32068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e568fd2b2a149aeadd6c64bc70af9b7" candidate_uuid: "e69f3153d090418380c201dddfef4c60" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "31690560715140e6847173104a2b0f2e"
I20260709 13:12:25.030512 32337 raft_consensus.cc:3060] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:25.030881 32068 raft_consensus.cc:3055] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [term 3 LEADER]: Stepping down as leader of term 3
I20260709 13:12:25.031213 32068 raft_consensus.cc:740] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 31690560715140e6847173104a2b0f2e, State: Running, Role: LEADER
I20260709 13:12:25.032205 32068 consensus_queue.cc:260] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:25.033511 32068 raft_consensus.cc:3060] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [term 3 FOLLOWER]: Advancing to term 4
I20260709 13:12:25.037773 32337 raft_consensus.cc:2468] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e69f3153d090418380c201dddfef4c60 in term 4.
I20260709 13:12:25.039291 32111 leader_election.cc:304] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91778255eb264fe6a6dc208998e849ba, e69f3153d090418380c201dddfef4c60; no voters: 
I20260709 13:12:25.039958 32283 raft_consensus.cc:2804] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 13:12:25.040427 32283 raft_consensus.cc:697] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [term 4 LEADER]: Becoming Leader. State: Replica: e69f3153d090418380c201dddfef4c60, State: Running, Role: LEADER
I20260709 13:12:25.041256 32068 raft_consensus.cc:2468] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e69f3153d090418380c201dddfef4c60 in term 4.
I20260709 13:12:25.041311 32283 consensus_queue.cc:237] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:25.047892 31895 catalog_manager.cc:5697] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 reported cstate change: term changed from 3 to 4, leader changed from 31690560715140e6847173104a2b0f2e (127.21.63.2) to e69f3153d090418380c201dddfef4c60 (127.21.63.3). New cstate: current_term: 4 leader_uuid: "e69f3153d090418380c201dddfef4c60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:25.173759 32328 raft_consensus.cc:993] T 3a76c5023ae14f8ba25de09e3a9b60de P 31690560715140e6847173104a2b0f2e: : Instructing follower e69f3153d090418380c201dddfef4c60 to start an election
I20260709 13:12:25.174300 32395 raft_consensus.cc:1081] T 3a76c5023ae14f8ba25de09e3a9b60de P 31690560715140e6847173104a2b0f2e [term 1 LEADER]: Signalling peer e69f3153d090418380c201dddfef4c60 to start an election
I20260709 13:12:25.175803 32146 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3a76c5023ae14f8ba25de09e3a9b60de"
dest_uuid: "e69f3153d090418380c201dddfef4c60"
 from {username='slave'} at 127.0.0.1:55614
I20260709 13:12:25.176281 32146 raft_consensus.cc:493] T 3a76c5023ae14f8ba25de09e3a9b60de P e69f3153d090418380c201dddfef4c60 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:25.176585 32146 raft_consensus.cc:3060] T 3a76c5023ae14f8ba25de09e3a9b60de P e69f3153d090418380c201dddfef4c60 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:25.181496 32146 raft_consensus.cc:515] T 3a76c5023ae14f8ba25de09e3a9b60de P e69f3153d090418380c201dddfef4c60 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:25.183123 32146 leader_election.cc:290] T 3a76c5023ae14f8ba25de09e3a9b60de P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 2 election: Requested vote from peers 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979), 31690560715140e6847173104a2b0f2e (127.21.63.2:44689)
I20260709 13:12:25.184127 32337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a76c5023ae14f8ba25de09e3a9b60de" candidate_uuid: "e69f3153d090418380c201dddfef4c60" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91778255eb264fe6a6dc208998e849ba"
I20260709 13:12:25.184508 32068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a76c5023ae14f8ba25de09e3a9b60de" candidate_uuid: "e69f3153d090418380c201dddfef4c60" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "31690560715140e6847173104a2b0f2e"
I20260709 13:12:25.184808 32337 raft_consensus.cc:3060] T 3a76c5023ae14f8ba25de09e3a9b60de P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:25.185237 32068 raft_consensus.cc:3055] T 3a76c5023ae14f8ba25de09e3a9b60de P 31690560715140e6847173104a2b0f2e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:25.185602 32068 raft_consensus.cc:740] T 3a76c5023ae14f8ba25de09e3a9b60de P 31690560715140e6847173104a2b0f2e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 31690560715140e6847173104a2b0f2e, State: Running, Role: LEADER
I20260709 13:12:25.186497 32068 consensus_queue.cc:260] T 3a76c5023ae14f8ba25de09e3a9b60de P 31690560715140e6847173104a2b0f2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:25.187804 32068 raft_consensus.cc:3060] T 3a76c5023ae14f8ba25de09e3a9b60de P 31690560715140e6847173104a2b0f2e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:25.191613 32337 raft_consensus.cc:2468] T 3a76c5023ae14f8ba25de09e3a9b60de P 91778255eb264fe6a6dc208998e849ba [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e69f3153d090418380c201dddfef4c60 in term 2.
I20260709 13:12:25.192873 32111 leader_election.cc:304] T 3a76c5023ae14f8ba25de09e3a9b60de P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91778255eb264fe6a6dc208998e849ba, e69f3153d090418380c201dddfef4c60; no voters: 
I20260709 13:12:25.193694 32283 raft_consensus.cc:2804] T 3a76c5023ae14f8ba25de09e3a9b60de P e69f3153d090418380c201dddfef4c60 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:25.194104 32283 raft_consensus.cc:697] T 3a76c5023ae14f8ba25de09e3a9b60de P e69f3153d090418380c201dddfef4c60 [term 2 LEADER]: Becoming Leader. State: Replica: e69f3153d090418380c201dddfef4c60, State: Running, Role: LEADER
I20260709 13:12:25.194335 32068 raft_consensus.cc:2468] T 3a76c5023ae14f8ba25de09e3a9b60de P 31690560715140e6847173104a2b0f2e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e69f3153d090418380c201dddfef4c60 in term 2.
I20260709 13:12:25.195015 32283 consensus_queue.cc:237] T 3a76c5023ae14f8ba25de09e3a9b60de P e69f3153d090418380c201dddfef4c60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:25.201941 31895 catalog_manager.cc:5697] T 3a76c5023ae14f8ba25de09e3a9b60de P e69f3153d090418380c201dddfef4c60 reported cstate change: term changed from 1 to 2, leader changed from 31690560715140e6847173104a2b0f2e (127.21.63.2) to e69f3153d090418380c201dddfef4c60 (127.21.63.3). New cstate: current_term: 2 leader_uuid: "e69f3153d090418380c201dddfef4c60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:25.530926 32337 raft_consensus.cc:1275] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba [term 4 FOLLOWER]: Refusing update from remote peer e69f3153d090418380c201dddfef4c60: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:12:25.532282 32396 consensus_queue.cc:1048] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:25.541796 32068 raft_consensus.cc:1275] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [term 4 FOLLOWER]: Refusing update from remote peer e69f3153d090418380c201dddfef4c60: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 13:12:25.543392 32283 consensus_queue.cc:1048] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:25.618386 32337 raft_consensus.cc:1275] T 3a76c5023ae14f8ba25de09e3a9b60de P 91778255eb264fe6a6dc208998e849ba [term 2 FOLLOWER]: Refusing update from remote peer e69f3153d090418380c201dddfef4c60: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:25.619614 32283 consensus_queue.cc:1048] T 3a76c5023ae14f8ba25de09e3a9b60de P e69f3153d090418380c201dddfef4c60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:25.628299 32068 raft_consensus.cc:1275] T 3a76c5023ae14f8ba25de09e3a9b60de P 31690560715140e6847173104a2b0f2e [term 2 FOLLOWER]: Refusing update from remote peer e69f3153d090418380c201dddfef4c60: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:25.630939 32413 consensus_queue.cc:1048] T 3a76c5023ae14f8ba25de09e3a9b60de P e69f3153d090418380c201dddfef4c60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 13:12:25.865501 31943 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:12:25.994472 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:26.004976 32144 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5ea7beb6ca5f4de4b64c0e8a9e9d99a7"
dest_uuid: "e69f3153d090418380c201dddfef4c60"
mode: GRACEFUL
new_leader_uuid: "31690560715140e6847173104a2b0f2e"
 from {username='slave'} at 127.0.0.1:55620
I20260709 13:12:26.006006 32144 raft_consensus.cc:606] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60 [term 2 LEADER]: Received request to transfer leadership to TS 31690560715140e6847173104a2b0f2e
I20260709 13:12:26.009789 32144 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "709f871588d74579bab67c29aca59135"
dest_uuid: "e69f3153d090418380c201dddfef4c60"
mode: GRACEFUL
new_leader_uuid: "31690560715140e6847173104a2b0f2e"
 from {username='slave'} at 127.0.0.1:55620
I20260709 13:12:26.010298 32144 raft_consensus.cc:606] T 709f871588d74579bab67c29aca59135 P e69f3153d090418380c201dddfef4c60 [term 1 LEADER]: Received request to transfer leadership to TS 31690560715140e6847173104a2b0f2e
I20260709 13:12:26.011780 21756 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:12:26.012593 21756 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:12:26.013309 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:26.231287 32283 raft_consensus.cc:993] T 709f871588d74579bab67c29aca59135 P e69f3153d090418380c201dddfef4c60: : Instructing follower 31690560715140e6847173104a2b0f2e to start an election
I20260709 13:12:26.231792 32413 raft_consensus.cc:1081] T 709f871588d74579bab67c29aca59135 P e69f3153d090418380c201dddfef4c60 [term 1 LEADER]: Signalling peer 31690560715140e6847173104a2b0f2e to start an election
I20260709 13:12:26.233318 32068 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "709f871588d74579bab67c29aca59135"
dest_uuid: "31690560715140e6847173104a2b0f2e"
 from {username='slave'} at 127.0.0.1:36594
I20260709 13:12:26.233716 32068 raft_consensus.cc:493] T 709f871588d74579bab67c29aca59135 P 31690560715140e6847173104a2b0f2e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:26.233934 32068 raft_consensus.cc:3060] T 709f871588d74579bab67c29aca59135 P 31690560715140e6847173104a2b0f2e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:26.238576 32068 raft_consensus.cc:515] T 709f871588d74579bab67c29aca59135 P 31690560715140e6847173104a2b0f2e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:26.240200 32068 leader_election.cc:290] T 709f871588d74579bab67c29aca59135 P 31690560715140e6847173104a2b0f2e [CANDIDATE]: Term 2 election: Requested vote from peers 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979), e69f3153d090418380c201dddfef4c60 (127.21.63.3:40639)
I20260709 13:12:26.241060 32339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709f871588d74579bab67c29aca59135" candidate_uuid: "31690560715140e6847173104a2b0f2e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91778255eb264fe6a6dc208998e849ba"
I20260709 13:12:26.241338 32144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "709f871588d74579bab67c29aca59135" candidate_uuid: "31690560715140e6847173104a2b0f2e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e69f3153d090418380c201dddfef4c60"
I20260709 13:12:26.241744 32339 raft_consensus.cc:3060] T 709f871588d74579bab67c29aca59135 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:26.241958 32144 raft_consensus.cc:3055] T 709f871588d74579bab67c29aca59135 P e69f3153d090418380c201dddfef4c60 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:26.242290 32144 raft_consensus.cc:740] T 709f871588d74579bab67c29aca59135 P e69f3153d090418380c201dddfef4c60 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e69f3153d090418380c201dddfef4c60, State: Running, Role: LEADER
I20260709 13:12:26.243232 32144 consensus_queue.cc:260] T 709f871588d74579bab67c29aca59135 P e69f3153d090418380c201dddfef4c60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:26.244580 32144 raft_consensus.cc:3060] T 709f871588d74579bab67c29aca59135 P e69f3153d090418380c201dddfef4c60 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:26.249641 32339 raft_consensus.cc:2468] T 709f871588d74579bab67c29aca59135 P 91778255eb264fe6a6dc208998e849ba [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31690560715140e6847173104a2b0f2e in term 2.
I20260709 13:12:26.251055 32037 leader_election.cc:304] T 709f871588d74579bab67c29aca59135 P 31690560715140e6847173104a2b0f2e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31690560715140e6847173104a2b0f2e, 91778255eb264fe6a6dc208998e849ba; no voters: 
I20260709 13:12:26.251902 32395 raft_consensus.cc:2804] T 709f871588d74579bab67c29aca59135 P 31690560715140e6847173104a2b0f2e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:26.252835 32144 raft_consensus.cc:2468] T 709f871588d74579bab67c29aca59135 P e69f3153d090418380c201dddfef4c60 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31690560715140e6847173104a2b0f2e in term 2.
I20260709 13:12:26.252969 32395 raft_consensus.cc:697] T 709f871588d74579bab67c29aca59135 P 31690560715140e6847173104a2b0f2e [term 2 LEADER]: Becoming Leader. State: Replica: 31690560715140e6847173104a2b0f2e, State: Running, Role: LEADER
I20260709 13:12:26.254269 32395 consensus_queue.cc:237] T 709f871588d74579bab67c29aca59135 P 31690560715140e6847173104a2b0f2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:26.261313 31895 catalog_manager.cc:5697] T 709f871588d74579bab67c29aca59135 P 31690560715140e6847173104a2b0f2e reported cstate change: term changed from 1 to 2, leader changed from e69f3153d090418380c201dddfef4c60 (127.21.63.3) to 31690560715140e6847173104a2b0f2e (127.21.63.2). New cstate: current_term: 2 leader_uuid: "31690560715140e6847173104a2b0f2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:26.426621 32283 raft_consensus.cc:993] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60: : Instructing follower 31690560715140e6847173104a2b0f2e to start an election
I20260709 13:12:26.427081 32413 raft_consensus.cc:1081] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60 [term 2 LEADER]: Signalling peer 31690560715140e6847173104a2b0f2e to start an election
I20260709 13:12:26.428494 32068 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5ea7beb6ca5f4de4b64c0e8a9e9d99a7"
dest_uuid: "31690560715140e6847173104a2b0f2e"
 from {username='slave'} at 127.0.0.1:36594
I20260709 13:12:26.429152 32068 raft_consensus.cc:493] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 31690560715140e6847173104a2b0f2e [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:26.429505 32068 raft_consensus.cc:3060] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 31690560715140e6847173104a2b0f2e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:26.433947 32068 raft_consensus.cc:515] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 31690560715140e6847173104a2b0f2e [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:26.435217 32068 leader_election.cc:290] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 31690560715140e6847173104a2b0f2e [CANDIDATE]: Term 3 election: Requested vote from peers 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979), e69f3153d090418380c201dddfef4c60 (127.21.63.3:40639)
I20260709 13:12:26.436285 32144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ea7beb6ca5f4de4b64c0e8a9e9d99a7" candidate_uuid: "31690560715140e6847173104a2b0f2e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "e69f3153d090418380c201dddfef4c60"
I20260709 13:12:26.436285 32337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5ea7beb6ca5f4de4b64c0e8a9e9d99a7" candidate_uuid: "31690560715140e6847173104a2b0f2e" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "91778255eb264fe6a6dc208998e849ba"
I20260709 13:12:26.436884 32144 raft_consensus.cc:3055] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:26.437259 32144 raft_consensus.cc:740] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e69f3153d090418380c201dddfef4c60, State: Running, Role: LEADER
I20260709 13:12:26.437311 32337 raft_consensus.cc:3060] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:26.438102 32144 consensus_queue.cc:260] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:26.439078 32144 raft_consensus.cc:3060] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:26.444079 32144 raft_consensus.cc:2468] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31690560715140e6847173104a2b0f2e in term 3.
I20260709 13:12:26.444461 32337 raft_consensus.cc:2468] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31690560715140e6847173104a2b0f2e in term 3.
I20260709 13:12:26.445415 32036 leader_election.cc:304] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 31690560715140e6847173104a2b0f2e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31690560715140e6847173104a2b0f2e, e69f3153d090418380c201dddfef4c60; no voters: 
I20260709 13:12:26.446336 32395 raft_consensus.cc:2804] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 31690560715140e6847173104a2b0f2e [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:26.446796 32395 raft_consensus.cc:697] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 31690560715140e6847173104a2b0f2e [term 3 LEADER]: Becoming Leader. State: Replica: 31690560715140e6847173104a2b0f2e, State: Running, Role: LEADER
I20260709 13:12:26.447702 32395 consensus_queue.cc:237] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 31690560715140e6847173104a2b0f2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:26.455087 31895 catalog_manager.cc:5697] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 31690560715140e6847173104a2b0f2e reported cstate change: term changed from 2 to 3, leader changed from e69f3153d090418380c201dddfef4c60 (127.21.63.3) to 31690560715140e6847173104a2b0f2e (127.21.63.2). New cstate: current_term: 3 leader_uuid: "31690560715140e6847173104a2b0f2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:26.799263 32144 raft_consensus.cc:1275] T 709f871588d74579bab67c29aca59135 P e69f3153d090418380c201dddfef4c60 [term 2 FOLLOWER]: Refusing update from remote peer 31690560715140e6847173104a2b0f2e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:26.800865 32415 consensus_queue.cc:1048] T 709f871588d74579bab67c29aca59135 P 31690560715140e6847173104a2b0f2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:26.813138 32337 raft_consensus.cc:1275] T 709f871588d74579bab67c29aca59135 P 91778255eb264fe6a6dc208998e849ba [term 2 FOLLOWER]: Refusing update from remote peer 31690560715140e6847173104a2b0f2e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:26.814815 32395 consensus_queue.cc:1048] T 709f871588d74579bab67c29aca59135 P 31690560715140e6847173104a2b0f2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:26.858695 32144 raft_consensus.cc:1275] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60 [term 3 FOLLOWER]: Refusing update from remote peer 31690560715140e6847173104a2b0f2e: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:12:26.860035 32395 consensus_queue.cc:1048] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 31690560715140e6847173104a2b0f2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:12:26.866509 31943 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:12:26.870893 32337 raft_consensus.cc:1275] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba [term 3 FOLLOWER]: Refusing update from remote peer 31690560715140e6847173104a2b0f2e: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 13:12:26.872872 32395 consensus_queue.cc:1048] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 31690560715140e6847173104a2b0f2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:27.022234 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:27.030584 32068 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7b2f10befb2e474fbe821af8d5e92fd5"
dest_uuid: "31690560715140e6847173104a2b0f2e"
mode: GRACEFUL
new_leader_uuid: "e69f3153d090418380c201dddfef4c60"
 from {username='slave'} at 127.0.0.1:36610
I20260709 13:12:27.031177 32068 raft_consensus.cc:606] T 7b2f10befb2e474fbe821af8d5e92fd5 P 31690560715140e6847173104a2b0f2e [term 1 LEADER]: Received request to transfer leadership to TS e69f3153d090418380c201dddfef4c60
I20260709 13:12:27.034039 32068 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f9a7e52d74da4cd79abac3858391ae77"
dest_uuid: "31690560715140e6847173104a2b0f2e"
mode: GRACEFUL
new_leader_uuid: "e69f3153d090418380c201dddfef4c60"
 from {username='slave'} at 127.0.0.1:36610
I20260709 13:12:27.034546 32068 raft_consensus.cc:606] T f9a7e52d74da4cd79abac3858391ae77 P 31690560715140e6847173104a2b0f2e [term 2 LEADER]: Received request to transfer leadership to TS e69f3153d090418380c201dddfef4c60
I20260709 13:12:27.035576 21756 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:12:27.036197 21756 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:12:27.036780 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:27.190847 32415 raft_consensus.cc:993] T f9a7e52d74da4cd79abac3858391ae77 P 31690560715140e6847173104a2b0f2e: : Instructing follower e69f3153d090418380c201dddfef4c60 to start an election
I20260709 13:12:27.191391 32395 raft_consensus.cc:1081] T f9a7e52d74da4cd79abac3858391ae77 P 31690560715140e6847173104a2b0f2e [term 2 LEADER]: Signalling peer e69f3153d090418380c201dddfef4c60 to start an election
I20260709 13:12:27.193002 32144 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f9a7e52d74da4cd79abac3858391ae77"
dest_uuid: "e69f3153d090418380c201dddfef4c60"
 from {username='slave'} at 127.0.0.1:55614
I20260709 13:12:27.193519 32144 raft_consensus.cc:493] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:27.193799 32144 raft_consensus.cc:3060] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:27.199452 32144 raft_consensus.cc:515] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } }
I20260709 13:12:27.201402 32144 leader_election.cc:290] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 3 election: Requested vote from peers 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979), 31690560715140e6847173104a2b0f2e (127.21.63.2:44689)
I20260709 13:12:27.202263 32068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9a7e52d74da4cd79abac3858391ae77" candidate_uuid: "e69f3153d090418380c201dddfef4c60" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "31690560715140e6847173104a2b0f2e"
I20260709 13:12:27.202158 32337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f9a7e52d74da4cd79abac3858391ae77" candidate_uuid: "e69f3153d090418380c201dddfef4c60" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "91778255eb264fe6a6dc208998e849ba"
I20260709 13:12:27.202940 32068 raft_consensus.cc:3055] T f9a7e52d74da4cd79abac3858391ae77 P 31690560715140e6847173104a2b0f2e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:27.203246 32068 raft_consensus.cc:740] T f9a7e52d74da4cd79abac3858391ae77 P 31690560715140e6847173104a2b0f2e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 31690560715140e6847173104a2b0f2e, State: Running, Role: LEADER
I20260709 13:12:27.204067 32068 consensus_queue.cc:260] T f9a7e52d74da4cd79abac3858391ae77 P 31690560715140e6847173104a2b0f2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } }
I20260709 13:12:27.205013 32068 raft_consensus.cc:3060] T f9a7e52d74da4cd79abac3858391ae77 P 31690560715140e6847173104a2b0f2e [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:27.203084 32337 raft_consensus.cc:3060] T f9a7e52d74da4cd79abac3858391ae77 P 91778255eb264fe6a6dc208998e849ba [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:27.209569 32068 raft_consensus.cc:2468] T f9a7e52d74da4cd79abac3858391ae77 P 31690560715140e6847173104a2b0f2e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e69f3153d090418380c201dddfef4c60 in term 3.
I20260709 13:12:27.210813 32111 leader_election.cc:304] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31690560715140e6847173104a2b0f2e, e69f3153d090418380c201dddfef4c60; no voters: 
I20260709 13:12:27.211542 32413 raft_consensus.cc:2804] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:27.211781 32337 raft_consensus.cc:2468] T f9a7e52d74da4cd79abac3858391ae77 P 91778255eb264fe6a6dc208998e849ba [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e69f3153d090418380c201dddfef4c60 in term 3.
I20260709 13:12:27.212703 32413 raft_consensus.cc:697] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 [term 3 LEADER]: Becoming Leader. State: Replica: e69f3153d090418380c201dddfef4c60, State: Running, Role: LEADER
I20260709 13:12:27.213677 32413 consensus_queue.cc:237] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } }
I20260709 13:12:27.220852 31895 catalog_manager.cc:5697] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 reported cstate change: term changed from 2 to 3, leader changed from 31690560715140e6847173104a2b0f2e (127.21.63.2) to e69f3153d090418380c201dddfef4c60 (127.21.63.3). New cstate: current_term: 3 leader_uuid: "e69f3153d090418380c201dddfef4c60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:27.251828 32415 raft_consensus.cc:993] T 7b2f10befb2e474fbe821af8d5e92fd5 P 31690560715140e6847173104a2b0f2e: : Instructing follower e69f3153d090418380c201dddfef4c60 to start an election
I20260709 13:12:27.252359 32395 raft_consensus.cc:1081] T 7b2f10befb2e474fbe821af8d5e92fd5 P 31690560715140e6847173104a2b0f2e [term 1 LEADER]: Signalling peer e69f3153d090418380c201dddfef4c60 to start an election
I20260709 13:12:27.253911 32144 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7b2f10befb2e474fbe821af8d5e92fd5"
dest_uuid: "e69f3153d090418380c201dddfef4c60"
 from {username='slave'} at 127.0.0.1:55614
I20260709 13:12:27.254508 32144 raft_consensus.cc:493] T 7b2f10befb2e474fbe821af8d5e92fd5 P e69f3153d090418380c201dddfef4c60 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:27.254901 32144 raft_consensus.cc:3060] T 7b2f10befb2e474fbe821af8d5e92fd5 P e69f3153d090418380c201dddfef4c60 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:27.260694 32144 raft_consensus.cc:515] T 7b2f10befb2e474fbe821af8d5e92fd5 P e69f3153d090418380c201dddfef4c60 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:27.262413 32144 leader_election.cc:290] T 7b2f10befb2e474fbe821af8d5e92fd5 P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 2 election: Requested vote from peers 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979), 31690560715140e6847173104a2b0f2e (127.21.63.2:44689)
I20260709 13:12:27.263168 32337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b2f10befb2e474fbe821af8d5e92fd5" candidate_uuid: "e69f3153d090418380c201dddfef4c60" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91778255eb264fe6a6dc208998e849ba"
I20260709 13:12:27.263365 32068 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b2f10befb2e474fbe821af8d5e92fd5" candidate_uuid: "e69f3153d090418380c201dddfef4c60" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "31690560715140e6847173104a2b0f2e"
I20260709 13:12:27.263818 32337 raft_consensus.cc:3060] T 7b2f10befb2e474fbe821af8d5e92fd5 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:27.264039 32068 raft_consensus.cc:3055] T 7b2f10befb2e474fbe821af8d5e92fd5 P 31690560715140e6847173104a2b0f2e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:27.264456 32068 raft_consensus.cc:740] T 7b2f10befb2e474fbe821af8d5e92fd5 P 31690560715140e6847173104a2b0f2e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 31690560715140e6847173104a2b0f2e, State: Running, Role: LEADER
I20260709 13:12:27.265216 32068 consensus_queue.cc:260] T 7b2f10befb2e474fbe821af8d5e92fd5 P 31690560715140e6847173104a2b0f2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:27.266111 32068 raft_consensus.cc:3060] T 7b2f10befb2e474fbe821af8d5e92fd5 P 31690560715140e6847173104a2b0f2e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:27.269991 32337 raft_consensus.cc:2468] T 7b2f10befb2e474fbe821af8d5e92fd5 P 91778255eb264fe6a6dc208998e849ba [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e69f3153d090418380c201dddfef4c60 in term 2.
I20260709 13:12:27.271101 32068 raft_consensus.cc:2468] T 7b2f10befb2e474fbe821af8d5e92fd5 P 31690560715140e6847173104a2b0f2e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e69f3153d090418380c201dddfef4c60 in term 2.
I20260709 13:12:27.271121 32111 leader_election.cc:304] T 7b2f10befb2e474fbe821af8d5e92fd5 P e69f3153d090418380c201dddfef4c60 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91778255eb264fe6a6dc208998e849ba, e69f3153d090418380c201dddfef4c60; no voters: 
I20260709 13:12:27.272023 32413 raft_consensus.cc:2804] T 7b2f10befb2e474fbe821af8d5e92fd5 P e69f3153d090418380c201dddfef4c60 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:27.272501 32413 raft_consensus.cc:697] T 7b2f10befb2e474fbe821af8d5e92fd5 P e69f3153d090418380c201dddfef4c60 [term 2 LEADER]: Becoming Leader. State: Replica: e69f3153d090418380c201dddfef4c60, State: Running, Role: LEADER
I20260709 13:12:27.273499 32413 consensus_queue.cc:237] T 7b2f10befb2e474fbe821af8d5e92fd5 P e69f3153d090418380c201dddfef4c60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:27.280869 31895 catalog_manager.cc:5697] T 7b2f10befb2e474fbe821af8d5e92fd5 P e69f3153d090418380c201dddfef4c60 reported cstate change: term changed from 1 to 2, leader changed from 31690560715140e6847173104a2b0f2e (127.21.63.2) to e69f3153d090418380c201dddfef4c60 (127.21.63.3). New cstate: current_term: 2 leader_uuid: "e69f3153d090418380c201dddfef4c60" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:27.670755 32068 raft_consensus.cc:1275] T f9a7e52d74da4cd79abac3858391ae77 P 31690560715140e6847173104a2b0f2e [term 3 FOLLOWER]: Refusing update from remote peer e69f3153d090418380c201dddfef4c60: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:27.672263 32413 consensus_queue.cc:1048] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:27.681715 32337 raft_consensus.cc:1275] T f9a7e52d74da4cd79abac3858391ae77 P 91778255eb264fe6a6dc208998e849ba [term 3 FOLLOWER]: Refusing update from remote peer e69f3153d090418380c201dddfef4c60: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:27.683329 32431 consensus_queue.cc:1048] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:27.733934 32337 raft_consensus.cc:1275] T 7b2f10befb2e474fbe821af8d5e92fd5 P 91778255eb264fe6a6dc208998e849ba [term 2 FOLLOWER]: Refusing update from remote peer e69f3153d090418380c201dddfef4c60: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:27.735302 32431 consensus_queue.cc:1048] T 7b2f10befb2e474fbe821af8d5e92fd5 P e69f3153d090418380c201dddfef4c60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:27.748202 32068 raft_consensus.cc:1275] T 7b2f10befb2e474fbe821af8d5e92fd5 P 31690560715140e6847173104a2b0f2e [term 2 FOLLOWER]: Refusing update from remote peer e69f3153d090418380c201dddfef4c60: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:27.749871 32431 consensus_queue.cc:1048] T 7b2f10befb2e474fbe821af8d5e92fd5 P e69f3153d090418380c201dddfef4c60 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:12:27.867714 31943 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:12:28.044723 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:28.053248 32144 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0dfa6288e543429c99109ebbd5a223b9"
dest_uuid: "e69f3153d090418380c201dddfef4c60"
mode: GRACEFUL
new_leader_uuid: "31690560715140e6847173104a2b0f2e"
 from {username='slave'} at 127.0.0.1:55620
I20260709 13:12:28.053793 32144 raft_consensus.cc:606] T 0dfa6288e543429c99109ebbd5a223b9 P e69f3153d090418380c201dddfef4c60 [term 1 LEADER]: Received request to transfer leadership to TS 31690560715140e6847173104a2b0f2e
I20260709 13:12:28.056728 32144 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5e568fd2b2a149aeadd6c64bc70af9b7"
dest_uuid: "e69f3153d090418380c201dddfef4c60"
mode: GRACEFUL
new_leader_uuid: "31690560715140e6847173104a2b0f2e"
 from {username='slave'} at 127.0.0.1:55620
I20260709 13:12:28.057165 32144 raft_consensus.cc:606] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [term 4 LEADER]: Received request to transfer leadership to TS 31690560715140e6847173104a2b0f2e
I20260709 13:12:28.058143 21756 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:12:28.058655 21756 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:12:28.059212 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:28.196223 32431 raft_consensus.cc:993] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60: : Instructing follower 31690560715140e6847173104a2b0f2e to start an election
I20260709 13:12:28.196755 32431 raft_consensus.cc:1081] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [term 4 LEADER]: Signalling peer 31690560715140e6847173104a2b0f2e to start an election
I20260709 13:12:28.198263 32068 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5e568fd2b2a149aeadd6c64bc70af9b7"
dest_uuid: "31690560715140e6847173104a2b0f2e"
 from {username='slave'} at 127.0.0.1:36594
I20260709 13:12:28.198805 32068 raft_consensus.cc:493] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:28.199105 32068 raft_consensus.cc:3060] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:28.203882 32068 raft_consensus.cc:515] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:28.205572 32068 leader_election.cc:290] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [CANDIDATE]: Term 5 election: Requested vote from peers 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979), e69f3153d090418380c201dddfef4c60 (127.21.63.3:40639)
I20260709 13:12:28.206430 32337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e568fd2b2a149aeadd6c64bc70af9b7" candidate_uuid: "31690560715140e6847173104a2b0f2e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "91778255eb264fe6a6dc208998e849ba"
I20260709 13:12:28.206902 32144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e568fd2b2a149aeadd6c64bc70af9b7" candidate_uuid: "31690560715140e6847173104a2b0f2e" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: true dest_uuid: "e69f3153d090418380c201dddfef4c60"
I20260709 13:12:28.207170 32337 raft_consensus.cc:3060] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:28.207751 32144 raft_consensus.cc:3055] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 13:12:28.208127 32144 raft_consensus.cc:740] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e69f3153d090418380c201dddfef4c60, State: Running, Role: LEADER
I20260709 13:12:28.209008 32144 consensus_queue.cc:260] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:28.210268 32144 raft_consensus.cc:3060] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [term 4 FOLLOWER]: Advancing to term 5
I20260709 13:12:28.212734 32337 raft_consensus.cc:2468] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31690560715140e6847173104a2b0f2e in term 5.
I20260709 13:12:28.213696 32037 leader_election.cc:304] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31690560715140e6847173104a2b0f2e, 91778255eb264fe6a6dc208998e849ba; no voters: 
I20260709 13:12:28.214440 32437 raft_consensus.cc:2804] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [term 5 FOLLOWER]: Leader election won for term 5
I20260709 13:12:28.215353 32437 raft_consensus.cc:697] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [term 5 LEADER]: Becoming Leader. State: Replica: 31690560715140e6847173104a2b0f2e, State: Running, Role: LEADER
I20260709 13:12:28.215600 32144 raft_consensus.cc:2468] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31690560715140e6847173104a2b0f2e in term 5.
I20260709 13:12:28.216136 32437 consensus_queue.cc:237] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:28.225512 31895 catalog_manager.cc:5697] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e reported cstate change: term changed from 4 to 5, leader changed from e69f3153d090418380c201dddfef4c60 (127.21.63.3) to 31690560715140e6847173104a2b0f2e (127.21.63.2). New cstate: current_term: 5 leader_uuid: "31690560715140e6847173104a2b0f2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:28.275218 32431 raft_consensus.cc:993] T 0dfa6288e543429c99109ebbd5a223b9 P e69f3153d090418380c201dddfef4c60: : Instructing follower 31690560715140e6847173104a2b0f2e to start an election
I20260709 13:12:28.275731 32445 raft_consensus.cc:1081] T 0dfa6288e543429c99109ebbd5a223b9 P e69f3153d090418380c201dddfef4c60 [term 1 LEADER]: Signalling peer 31690560715140e6847173104a2b0f2e to start an election
I20260709 13:12:28.277174 32068 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0dfa6288e543429c99109ebbd5a223b9"
dest_uuid: "31690560715140e6847173104a2b0f2e"
 from {username='slave'} at 127.0.0.1:36594
I20260709 13:12:28.277640 32068 raft_consensus.cc:493] T 0dfa6288e543429c99109ebbd5a223b9 P 31690560715140e6847173104a2b0f2e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:28.277964 32068 raft_consensus.cc:3060] T 0dfa6288e543429c99109ebbd5a223b9 P 31690560715140e6847173104a2b0f2e [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:28.283407 32068 raft_consensus.cc:515] T 0dfa6288e543429c99109ebbd5a223b9 P 31690560715140e6847173104a2b0f2e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:28.284821 32068 leader_election.cc:290] T 0dfa6288e543429c99109ebbd5a223b9 P 31690560715140e6847173104a2b0f2e [CANDIDATE]: Term 2 election: Requested vote from peers 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979), e69f3153d090418380c201dddfef4c60 (127.21.63.3:40639)
I20260709 13:12:28.285624 32337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dfa6288e543429c99109ebbd5a223b9" candidate_uuid: "31690560715140e6847173104a2b0f2e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "91778255eb264fe6a6dc208998e849ba"
I20260709 13:12:28.285944 32144 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0dfa6288e543429c99109ebbd5a223b9" candidate_uuid: "31690560715140e6847173104a2b0f2e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e69f3153d090418380c201dddfef4c60"
I20260709 13:12:28.286301 32337 raft_consensus.cc:3060] T 0dfa6288e543429c99109ebbd5a223b9 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:28.286509 32144 raft_consensus.cc:3055] T 0dfa6288e543429c99109ebbd5a223b9 P e69f3153d090418380c201dddfef4c60 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:28.286862 32144 raft_consensus.cc:740] T 0dfa6288e543429c99109ebbd5a223b9 P e69f3153d090418380c201dddfef4c60 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e69f3153d090418380c201dddfef4c60, State: Running, Role: LEADER
I20260709 13:12:28.287782 32144 consensus_queue.cc:260] T 0dfa6288e543429c99109ebbd5a223b9 P e69f3153d090418380c201dddfef4c60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:28.289096 32144 raft_consensus.cc:3060] T 0dfa6288e543429c99109ebbd5a223b9 P e69f3153d090418380c201dddfef4c60 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:28.293097 32337 raft_consensus.cc:2468] T 0dfa6288e543429c99109ebbd5a223b9 P 91778255eb264fe6a6dc208998e849ba [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31690560715140e6847173104a2b0f2e in term 2.
I20260709 13:12:28.294365 32037 leader_election.cc:304] T 0dfa6288e543429c99109ebbd5a223b9 P 31690560715140e6847173104a2b0f2e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31690560715140e6847173104a2b0f2e, 91778255eb264fe6a6dc208998e849ba; no voters: 
I20260709 13:12:28.295188 32437 raft_consensus.cc:2804] T 0dfa6288e543429c99109ebbd5a223b9 P 31690560715140e6847173104a2b0f2e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:28.295610 32437 raft_consensus.cc:697] T 0dfa6288e543429c99109ebbd5a223b9 P 31690560715140e6847173104a2b0f2e [term 2 LEADER]: Becoming Leader. State: Replica: 31690560715140e6847173104a2b0f2e, State: Running, Role: LEADER
I20260709 13:12:28.295941 32144 raft_consensus.cc:2468] T 0dfa6288e543429c99109ebbd5a223b9 P e69f3153d090418380c201dddfef4c60 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31690560715140e6847173104a2b0f2e in term 2.
I20260709 13:12:28.296518 32437 consensus_queue.cc:237] T 0dfa6288e543429c99109ebbd5a223b9 P 31690560715140e6847173104a2b0f2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } }
I20260709 13:12:28.302948 31895 catalog_manager.cc:5697] T 0dfa6288e543429c99109ebbd5a223b9 P 31690560715140e6847173104a2b0f2e reported cstate change: term changed from 1 to 2, leader changed from e69f3153d090418380c201dddfef4c60 (127.21.63.3) to 31690560715140e6847173104a2b0f2e (127.21.63.2). New cstate: current_term: 2 leader_uuid: "31690560715140e6847173104a2b0f2e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31690560715140e6847173104a2b0f2e" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 44689 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:28.700232 32337 raft_consensus.cc:1275] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba [term 5 FOLLOWER]: Refusing update from remote peer 31690560715140e6847173104a2b0f2e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:12:28.701601 32437 consensus_queue.cc:1048] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:28.713478 32144 raft_consensus.cc:1275] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [term 5 FOLLOWER]: Refusing update from remote peer 31690560715140e6847173104a2b0f2e: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 13:12:28.716120 32437 consensus_queue.cc:1048] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:28.756930 32337 raft_consensus.cc:1275] T 0dfa6288e543429c99109ebbd5a223b9 P 91778255eb264fe6a6dc208998e849ba [term 2 FOLLOWER]: Refusing update from remote peer 31690560715140e6847173104a2b0f2e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:28.758424 32437 consensus_queue.cc:1048] T 0dfa6288e543429c99109ebbd5a223b9 P 31690560715140e6847173104a2b0f2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "91778255eb264fe6a6dc208998e849ba" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 32979 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:28.767058 32144 raft_consensus.cc:1275] T 0dfa6288e543429c99109ebbd5a223b9 P e69f3153d090418380c201dddfef4c60 [term 2 FOLLOWER]: Refusing update from remote peer 31690560715140e6847173104a2b0f2e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:28.769107 32447 consensus_queue.cc:1048] T 0dfa6288e543429c99109ebbd5a223b9 P 31690560715140e6847173104a2b0f2e [LEADER]: Connected to new peer: Peer: permanent_uuid: "e69f3153d090418380c201dddfef4c60" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 40639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 13:12:28.868757 31943 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 13:12:29.066874 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:29.097424 21756 tablet_server.cc:179] TabletServer@127.21.63.1:32979 shutting down...
I20260709 13:12:29.117690 21756 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:12:29.118378 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:29.119050 21756 raft_consensus.cc:2243] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:29.119525 21756 raft_consensus.cc:2272] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 91778255eb264fe6a6dc208998e849ba [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:29.121492 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:29.122058 21756 raft_consensus.cc:2243] T 7b2f10befb2e474fbe821af8d5e92fd5 P 91778255eb264fe6a6dc208998e849ba [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:29.122678 21756 raft_consensus.cc:2272] T 7b2f10befb2e474fbe821af8d5e92fd5 P 91778255eb264fe6a6dc208998e849ba [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:29.124869 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:29.125583 21756 raft_consensus.cc:2243] T 3a76c5023ae14f8ba25de09e3a9b60de P 91778255eb264fe6a6dc208998e849ba [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:29.126127 21756 raft_consensus.cc:2272] T 3a76c5023ae14f8ba25de09e3a9b60de P 91778255eb264fe6a6dc208998e849ba [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:29.128048 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:29.128722 21756 raft_consensus.cc:2243] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:29.129179 21756 raft_consensus.cc:2272] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 91778255eb264fe6a6dc208998e849ba [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:29.131351 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:29.131927 21756 raft_consensus.cc:2243] T f9a7e52d74da4cd79abac3858391ae77 P 91778255eb264fe6a6dc208998e849ba [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:29.132354 21756 raft_consensus.cc:2272] T f9a7e52d74da4cd79abac3858391ae77 P 91778255eb264fe6a6dc208998e849ba [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:29.134193 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:29.134724 21756 raft_consensus.cc:2243] T b40f21aa6dd54f64bed8e716a77df4d6 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:29.135119 21756 raft_consensus.cc:2272] T b40f21aa6dd54f64bed8e716a77df4d6 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:29.137426 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:29.137971 21756 raft_consensus.cc:2243] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:29.138381 21756 raft_consensus.cc:2272] T d1fc79d86bee43c5b062ced8a090aab1 P 91778255eb264fe6a6dc208998e849ba [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:29.140236 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:29.140990 21756 raft_consensus.cc:2243] T d0a7cae5f3084d0ba50763cc87888571 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:29.141584 21756 raft_consensus.cc:2272] T d0a7cae5f3084d0ba50763cc87888571 P 91778255eb264fe6a6dc208998e849ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:29.143421 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:29.143911 21756 raft_consensus.cc:2243] T 709f871588d74579bab67c29aca59135 P 91778255eb264fe6a6dc208998e849ba [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:29.144317 21756 raft_consensus.cc:2272] T 709f871588d74579bab67c29aca59135 P 91778255eb264fe6a6dc208998e849ba [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:29.146059 21756 tablet_replica.cc:333] stopping tablet replica
I20260709 13:12:29.146533 21756 raft_consensus.cc:2243] T 0dfa6288e543429c99109ebbd5a223b9 P 91778255eb264fe6a6dc208998e849ba [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:29.146942 21756 raft_consensus.cc:2272] T 0dfa6288e543429c99109ebbd5a223b9 P 91778255eb264fe6a6dc208998e849ba [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:29.151222 32037 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 7 similar messages]
W20260709 13:12:29.154156 32037 consensus_peers.cc:597] T d0a7cae5f3084d0ba50763cc87888571 P 31690560715140e6847173104a2b0f2e -> Peer 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979): Couldn't send request to peer 91778255eb264fe6a6dc208998e849ba. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:29.179286 21756 tablet_server.cc:196] TabletServer@127.21.63.1:32979 shutdown complete.
I20260709 13:12:29.200644 21756 tablet_server.cc:179] TabletServer@127.21.63.2:0 shutting down...
I20260709 13:12:29.228611 21756 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:12:29.229462 21756 tablet_replica.cc:333] T 709f871588d74579bab67c29aca59135 P 31690560715140e6847173104a2b0f2e: stopping tablet replica
I20260709 13:12:29.230147 21756 raft_consensus.cc:2243] T 709f871588d74579bab67c29aca59135 P 31690560715140e6847173104a2b0f2e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:29.230922 21756 raft_consensus.cc:2272] T 709f871588d74579bab67c29aca59135 P 31690560715140e6847173104a2b0f2e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:29.233162 21756 tablet_replica.cc:333] T 0dfa6288e543429c99109ebbd5a223b9 P 31690560715140e6847173104a2b0f2e: stopping tablet replica
I20260709 13:12:29.233796 21756 raft_consensus.cc:2243] T 0dfa6288e543429c99109ebbd5a223b9 P 31690560715140e6847173104a2b0f2e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:29.234516 21756 raft_consensus.cc:2272] T 0dfa6288e543429c99109ebbd5a223b9 P 31690560715140e6847173104a2b0f2e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:29.236975 21756 tablet_replica.cc:333] T d1fc79d86bee43c5b062ced8a090aab1 P 31690560715140e6847173104a2b0f2e: stopping tablet replica
I20260709 13:12:29.237727 21756 raft_consensus.cc:2243] T d1fc79d86bee43c5b062ced8a090aab1 P 31690560715140e6847173104a2b0f2e [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:29.238492 21756 raft_consensus.cc:2272] T d1fc79d86bee43c5b062ced8a090aab1 P 31690560715140e6847173104a2b0f2e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:29.240576 21756 tablet_replica.cc:333] T b40f21aa6dd54f64bed8e716a77df4d6 P 31690560715140e6847173104a2b0f2e: stopping tablet replica
I20260709 13:12:29.241219 21756 raft_consensus.cc:2243] T b40f21aa6dd54f64bed8e716a77df4d6 P 31690560715140e6847173104a2b0f2e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:29.241644 21756 raft_consensus.cc:2272] T b40f21aa6dd54f64bed8e716a77df4d6 P 31690560715140e6847173104a2b0f2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:29.243691 21756 tablet_replica.cc:333] T f9a7e52d74da4cd79abac3858391ae77 P 31690560715140e6847173104a2b0f2e: stopping tablet replica
I20260709 13:12:29.244222 21756 raft_consensus.cc:2243] T f9a7e52d74da4cd79abac3858391ae77 P 31690560715140e6847173104a2b0f2e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:29.244619 21756 raft_consensus.cc:2272] T f9a7e52d74da4cd79abac3858391ae77 P 31690560715140e6847173104a2b0f2e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:29.246646 21756 tablet_replica.cc:333] T 7b2f10befb2e474fbe821af8d5e92fd5 P 31690560715140e6847173104a2b0f2e: stopping tablet replica
I20260709 13:12:29.247282 21756 raft_consensus.cc:2243] T 7b2f10befb2e474fbe821af8d5e92fd5 P 31690560715140e6847173104a2b0f2e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:29.247677 21756 raft_consensus.cc:2272] T 7b2f10befb2e474fbe821af8d5e92fd5 P 31690560715140e6847173104a2b0f2e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:29.249691 21756 tablet_replica.cc:333] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e: stopping tablet replica
I20260709 13:12:29.250196 21756 raft_consensus.cc:2243] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [term 5 LEADER]: Raft consensus shutting down.
I20260709 13:12:29.250945 21756 raft_consensus.cc:2272] T 5e568fd2b2a149aeadd6c64bc70af9b7 P 31690560715140e6847173104a2b0f2e [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:29.253328 21756 tablet_replica.cc:333] T 3a76c5023ae14f8ba25de09e3a9b60de P 31690560715140e6847173104a2b0f2e: stopping tablet replica
I20260709 13:12:29.253883 21756 raft_consensus.cc:2243] T 3a76c5023ae14f8ba25de09e3a9b60de P 31690560715140e6847173104a2b0f2e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:29.254302 21756 raft_consensus.cc:2272] T 3a76c5023ae14f8ba25de09e3a9b60de P 31690560715140e6847173104a2b0f2e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:29.256359 21756 tablet_replica.cc:333] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 31690560715140e6847173104a2b0f2e: stopping tablet replica
I20260709 13:12:29.256919 21756 raft_consensus.cc:2243] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 31690560715140e6847173104a2b0f2e [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:12:29.257651 21756 raft_consensus.cc:2272] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P 31690560715140e6847173104a2b0f2e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:29.259599 21756 tablet_replica.cc:333] T d0a7cae5f3084d0ba50763cc87888571 P 31690560715140e6847173104a2b0f2e: stopping tablet replica
I20260709 13:12:29.260102 21756 raft_consensus.cc:2243] T d0a7cae5f3084d0ba50763cc87888571 P 31690560715140e6847173104a2b0f2e [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:29.260905 21756 raft_consensus.cc:2272] T d0a7cae5f3084d0ba50763cc87888571 P 31690560715140e6847173104a2b0f2e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:29.293874 21756 tablet_server.cc:196] TabletServer@127.21.63.2:0 shutdown complete.
W20260709 13:12:29.304520 32111 consensus_peers.cc:597] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 -> Peer 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979): Couldn't send request to peer 91778255eb264fe6a6dc208998e849ba. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:32979: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:29.320797 21756 tablet_server.cc:179] TabletServer@127.21.63.3:0 shutting down...
W20260709 13:12:29.336119 32111 consensus_peers.cc:597] T 3a76c5023ae14f8ba25de09e3a9b60de P e69f3153d090418380c201dddfef4c60 -> Peer 31690560715140e6847173104a2b0f2e (127.21.63.2:44689): Couldn't send request to peer 31690560715140e6847173104a2b0f2e. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:44689: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:29.353255 21756 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:12:29.354040 21756 tablet_replica.cc:333] T 0dfa6288e543429c99109ebbd5a223b9 P e69f3153d090418380c201dddfef4c60: stopping tablet replica
I20260709 13:12:29.354750 21756 raft_consensus.cc:2243] T 0dfa6288e543429c99109ebbd5a223b9 P e69f3153d090418380c201dddfef4c60 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:29.355290 21756 raft_consensus.cc:2272] T 0dfa6288e543429c99109ebbd5a223b9 P e69f3153d090418380c201dddfef4c60 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:29.358036 21756 tablet_replica.cc:333] T 709f871588d74579bab67c29aca59135 P e69f3153d090418380c201dddfef4c60: stopping tablet replica
I20260709 13:12:29.358539 21756 raft_consensus.cc:2243] T 709f871588d74579bab67c29aca59135 P e69f3153d090418380c201dddfef4c60 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:29.358902 21756 raft_consensus.cc:2272] T 709f871588d74579bab67c29aca59135 P e69f3153d090418380c201dddfef4c60 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:29.360946 21756 tablet_replica.cc:333] T d0a7cae5f3084d0ba50763cc87888571 P e69f3153d090418380c201dddfef4c60: stopping tablet replica
I20260709 13:12:29.361537 21756 raft_consensus.cc:2243] T d0a7cae5f3084d0ba50763cc87888571 P e69f3153d090418380c201dddfef4c60 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:29.362015 21756 raft_consensus.cc:2272] T d0a7cae5f3084d0ba50763cc87888571 P e69f3153d090418380c201dddfef4c60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:29.365003 21756 tablet_replica.cc:333] T d1fc79d86bee43c5b062ced8a090aab1 P e69f3153d090418380c201dddfef4c60: stopping tablet replica
I20260709 13:12:29.365655 21756 raft_consensus.cc:2243] T d1fc79d86bee43c5b062ced8a090aab1 P e69f3153d090418380c201dddfef4c60 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:29.366276 21756 raft_consensus.cc:2272] T d1fc79d86bee43c5b062ced8a090aab1 P e69f3153d090418380c201dddfef4c60 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:29.366343 32111 consensus_peers.cc:597] T 7b2f10befb2e474fbe821af8d5e92fd5 P e69f3153d090418380c201dddfef4c60 -> Peer 91778255eb264fe6a6dc208998e849ba (127.21.63.1:32979): Couldn't send request to peer 91778255eb264fe6a6dc208998e849ba. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:32979: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:29.368940 21756 tablet_replica.cc:333] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60: stopping tablet replica
I20260709 13:12:29.369714 21756 raft_consensus.cc:2243] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:12:29.370481 21756 raft_consensus.cc:2272] T f9a7e52d74da4cd79abac3858391ae77 P e69f3153d090418380c201dddfef4c60 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:29.372392 21756 tablet_replica.cc:333] T b40f21aa6dd54f64bed8e716a77df4d6 P e69f3153d090418380c201dddfef4c60: stopping tablet replica
I20260709 13:12:29.373014 21756 raft_consensus.cc:2243] T b40f21aa6dd54f64bed8e716a77df4d6 P e69f3153d090418380c201dddfef4c60 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:29.373791 21756 raft_consensus.cc:2272] T b40f21aa6dd54f64bed8e716a77df4d6 P e69f3153d090418380c201dddfef4c60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:29.375850 21756 tablet_replica.cc:333] T 7b2f10befb2e474fbe821af8d5e92fd5 P e69f3153d090418380c201dddfef4c60: stopping tablet replica
I20260709 13:12:29.376358 21756 raft_consensus.cc:2243] T 7b2f10befb2e474fbe821af8d5e92fd5 P e69f3153d090418380c201dddfef4c60 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:29.377219 21756 raft_consensus.cc:2272] T 7b2f10befb2e474fbe821af8d5e92fd5 P e69f3153d090418380c201dddfef4c60 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:29.379312 21756 tablet_replica.cc:333] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60: stopping tablet replica
I20260709 13:12:29.379853 21756 raft_consensus.cc:2243] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:29.380275 21756 raft_consensus.cc:2272] T 5e568fd2b2a149aeadd6c64bc70af9b7 P e69f3153d090418380c201dddfef4c60 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:29.383095 21756 tablet_replica.cc:333] T 3a76c5023ae14f8ba25de09e3a9b60de P e69f3153d090418380c201dddfef4c60: stopping tablet replica
I20260709 13:12:29.383611 21756 raft_consensus.cc:2243] T 3a76c5023ae14f8ba25de09e3a9b60de P e69f3153d090418380c201dddfef4c60 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:29.384335 21756 raft_consensus.cc:2272] T 3a76c5023ae14f8ba25de09e3a9b60de P e69f3153d090418380c201dddfef4c60 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:29.386258 21756 tablet_replica.cc:333] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60: stopping tablet replica
I20260709 13:12:29.386737 21756 raft_consensus.cc:2243] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:29.387135 21756 raft_consensus.cc:2272] T 5ea7beb6ca5f4de4b64c0e8a9e9d99a7 P e69f3153d090418380c201dddfef4c60 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:29.418022 21756 tablet_server.cc:196] TabletServer@127.21.63.3:0 shutdown complete.
I20260709 13:12:29.442920 21756 master.cc:562] Master@127.21.63.62:36925 shutting down...
I20260709 13:12:29.470716 21756 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 33e978ec2bde4a4a89c7f981d4853921 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:29.471264 21756 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 33e978ec2bde4a4a89c7f981d4853921 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:29.471555 21756 tablet_replica.cc:333] T 00000000000000000000000000000000 P 33e978ec2bde4a4a89c7f981d4853921: stopping tablet replica
I20260709 13:12:29.492245 21756 master.cc:584] Master@127.21.63.62:36925 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (12833 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 13:12:29.534598 21756 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.63.62:41781
I20260709 13:12:29.535658 21756 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:12:29.540678 32468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:29.540711 32467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:29.542259 32470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:29.543298 21756 server_base.cc:1061] running on GCE node
I20260709 13:12:29.544322 21756 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:12:29.544487 21756 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:12:29.544920 21756 hybrid_clock.cc:648] HybridClock initialized: now 1783602749544688 us; error 0 us; skew 500 ppm
I20260709 13:12:29.545464 21756 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:12:29.547648 21756 webserver.cc:533] Webserver started at http://127.21.63.62:36703/ using document root <none> and password file <none>
I20260709 13:12:29.548143 21756 fs_manager.cc:362] Metadata directory not provided
I20260709 13:12:29.548327 21756 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:12:29.548614 21756 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:12:29.549727 21756 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602429420501-21756-0/minicluster-data/master-0-root/instance:
uuid: "58083de319b6453c99a5d54e73a10f9d"
format_stamp: "Formatted at 2026-07-09 13:12:29 on dist-test-slave-c6mt"
I20260709 13:12:29.554090 21756 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:12:29.557713 32475 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:29.558605 21756 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 13:12:29.558909 21756 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602429420501-21756-0/minicluster-data/master-0-root
uuid: "58083de319b6453c99a5d54e73a10f9d"
format_stamp: "Formatted at 2026-07-09 13:12:29 on dist-test-slave-c6mt"
I20260709 13:12:29.559185 21756 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602429420501-21756-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602429420501-21756-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602429420501-21756-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:12:29.572907 21756 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:12:29.573992 21756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:12:29.613636 21756 rpc_server.cc:307] RPC server started. Bound to: 127.21.63.62:41781
I20260709 13:12:29.613736 32526 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.63.62:41781 every 8 connection(s)
I20260709 13:12:29.617488 32527 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:29.629410 32527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58083de319b6453c99a5d54e73a10f9d: Bootstrap starting.
I20260709 13:12:29.634152 32527 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 58083de319b6453c99a5d54e73a10f9d: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:29.638268 32527 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58083de319b6453c99a5d54e73a10f9d: No bootstrap required, opened a new log
I20260709 13:12:29.640384 32527 raft_consensus.cc:359] T 00000000000000000000000000000000 P 58083de319b6453c99a5d54e73a10f9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58083de319b6453c99a5d54e73a10f9d" member_type: VOTER }
I20260709 13:12:29.640836 32527 raft_consensus.cc:385] T 00000000000000000000000000000000 P 58083de319b6453c99a5d54e73a10f9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:29.641052 32527 raft_consensus.cc:740] T 00000000000000000000000000000000 P 58083de319b6453c99a5d54e73a10f9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58083de319b6453c99a5d54e73a10f9d, State: Initialized, Role: FOLLOWER
I20260709 13:12:29.641692 32527 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58083de319b6453c99a5d54e73a10f9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58083de319b6453c99a5d54e73a10f9d" member_type: VOTER }
I20260709 13:12:29.642144 32527 raft_consensus.cc:399] T 00000000000000000000000000000000 P 58083de319b6453c99a5d54e73a10f9d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:12:29.642370 32527 raft_consensus.cc:493] T 00000000000000000000000000000000 P 58083de319b6453c99a5d54e73a10f9d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:12:29.642630 32527 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 58083de319b6453c99a5d54e73a10f9d [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:29.647210 32527 raft_consensus.cc:515] T 00000000000000000000000000000000 P 58083de319b6453c99a5d54e73a10f9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58083de319b6453c99a5d54e73a10f9d" member_type: VOTER }
I20260709 13:12:29.647699 32527 leader_election.cc:304] T 00000000000000000000000000000000 P 58083de319b6453c99a5d54e73a10f9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 58083de319b6453c99a5d54e73a10f9d; no voters: 
I20260709 13:12:29.648895 32527 leader_election.cc:290] T 00000000000000000000000000000000 P 58083de319b6453c99a5d54e73a10f9d [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:12:29.649296 32530 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 58083de319b6453c99a5d54e73a10f9d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:29.650760 32530 raft_consensus.cc:697] T 00000000000000000000000000000000 P 58083de319b6453c99a5d54e73a10f9d [term 1 LEADER]: Becoming Leader. State: Replica: 58083de319b6453c99a5d54e73a10f9d, State: Running, Role: LEADER
I20260709 13:12:29.651504 32530 consensus_queue.cc:237] T 00000000000000000000000000000000 P 58083de319b6453c99a5d54e73a10f9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58083de319b6453c99a5d54e73a10f9d" member_type: VOTER }
I20260709 13:12:29.651690 32527 sys_catalog.cc:565] T 00000000000000000000000000000000 P 58083de319b6453c99a5d54e73a10f9d [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:12:29.656556 32532 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58083de319b6453c99a5d54e73a10f9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58083de319b6453c99a5d54e73a10f9d. Latest consensus state: current_term: 1 leader_uuid: "58083de319b6453c99a5d54e73a10f9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58083de319b6453c99a5d54e73a10f9d" member_type: VOTER } }
I20260709 13:12:29.657639 32532 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58083de319b6453c99a5d54e73a10f9d [sys.catalog]: This master's current role is: LEADER
I20260709 13:12:29.659106 32531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58083de319b6453c99a5d54e73a10f9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "58083de319b6453c99a5d54e73a10f9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58083de319b6453c99a5d54e73a10f9d" member_type: VOTER } }
I20260709 13:12:29.660008 32531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58083de319b6453c99a5d54e73a10f9d [sys.catalog]: This master's current role is: LEADER
I20260709 13:12:29.662442 32538 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:12:29.671036 32538 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:12:29.672114 21756 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:12:29.680073 32538 catalog_manager.cc:1383] Generated new cluster ID: a0a39a1e7afa4f6088d47a322b79ccf3
I20260709 13:12:29.680372 32538 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:12:29.695827 32538 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:12:29.697271 32538 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:12:29.714671 32538 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 58083de319b6453c99a5d54e73a10f9d: Generated new TSK 0
I20260709 13:12:29.715340 32538 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:12:29.738890 21756 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:12:29.745213 32548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:29.746320 32549 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:29.749244 32551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:29.749794 21756 server_base.cc:1061] running on GCE node
I20260709 13:12:29.750715 21756 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:12:29.750941 21756 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:12:29.751104 21756 hybrid_clock.cc:648] HybridClock initialized: now 1783602749751085 us; error 0 us; skew 500 ppm
I20260709 13:12:29.751641 21756 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:12:29.754148 21756 webserver.cc:533] Webserver started at http://127.21.63.1:35399/ using document root <none> and password file <none>
I20260709 13:12:29.754616 21756 fs_manager.cc:362] Metadata directory not provided
I20260709 13:12:29.754820 21756 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:12:29.755090 21756 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:12:29.756138 21756 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602429420501-21756-0/minicluster-data/ts-0-root/instance:
uuid: "00d7cee1aea54375980b10dbdf4c4b79"
format_stamp: "Formatted at 2026-07-09 13:12:29 on dist-test-slave-c6mt"
I20260709 13:12:29.760622 21756 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.002s
I20260709 13:12:29.764415 32556 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:29.765301 21756 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:12:29.765619 21756 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602429420501-21756-0/minicluster-data/ts-0-root
uuid: "00d7cee1aea54375980b10dbdf4c4b79"
format_stamp: "Formatted at 2026-07-09 13:12:29 on dist-test-slave-c6mt"
I20260709 13:12:29.765933 21756 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602429420501-21756-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602429420501-21756-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602429420501-21756-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:12:29.811348 21756 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:12:29.812534 21756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:12:29.814054 21756 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:12:29.816295 21756 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:12:29.816519 21756 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:29.816776 21756 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:12:29.816944 21756 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:29.859841 21756 rpc_server.cc:307] RPC server started. Bound to: 127.21.63.1:34359
I20260709 13:12:29.859954 32618 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.63.1:34359 every 8 connection(s)
I20260709 13:12:29.865247 21756 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:12:29.873162 32623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:29.875371 32624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:29.878958 32626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:29.879786 32619 heartbeater.cc:344] Connected to a master server at 127.21.63.62:41781
I20260709 13:12:29.880324 32619 heartbeater.cc:461] Registering TS with master...
I20260709 13:12:29.880570 21756 server_base.cc:1061] running on GCE node
I20260709 13:12:29.881322 32619 heartbeater.cc:507] Master 127.21.63.62:41781 requested a full tablet report, sending...
I20260709 13:12:29.881711 21756 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:12:29.881947 21756 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:12:29.882197 21756 hybrid_clock.cc:648] HybridClock initialized: now 1783602749882172 us; error 0 us; skew 500 ppm
I20260709 13:12:29.883165 21756 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:12:29.884205 32492 ts_manager.cc:194] Registered new tserver with Master: 00d7cee1aea54375980b10dbdf4c4b79 (127.21.63.1:34359)
I20260709 13:12:29.886152 21756 webserver.cc:533] Webserver started at http://127.21.63.2:45673/ using document root <none> and password file <none>
I20260709 13:12:29.886726 32492 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57518
I20260709 13:12:29.886855 21756 fs_manager.cc:362] Metadata directory not provided
I20260709 13:12:29.887290 21756 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:12:29.887750 21756 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:12:29.889006 21756 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602429420501-21756-0/minicluster-data/ts-1-root/instance:
uuid: "bbe610f3a51e4732a48c941905e222f1"
format_stamp: "Formatted at 2026-07-09 13:12:29 on dist-test-slave-c6mt"
I20260709 13:12:29.893635 21756 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.001s	sys 0.003s
I20260709 13:12:29.897332 32631 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:29.898118 21756 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.002s
I20260709 13:12:29.898451 21756 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602429420501-21756-0/minicluster-data/ts-1-root
uuid: "bbe610f3a51e4732a48c941905e222f1"
format_stamp: "Formatted at 2026-07-09 13:12:29 on dist-test-slave-c6mt"
I20260709 13:12:29.898747 21756 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602429420501-21756-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602429420501-21756-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602429420501-21756-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:12:29.916108 21756 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:12:29.917479 21756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:12:29.919032 21756 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:12:29.922062 21756 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:12:29.922308 21756 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:29.922570 21756 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:12:29.922749 21756 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:29.968075 21756 rpc_server.cc:307] RPC server started. Bound to: 127.21.63.2:40017
I20260709 13:12:29.968185 32693 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.63.2:40017 every 8 connection(s)
I20260709 13:12:29.974323 21756 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:12:29.981047 32697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:29.981829 32698 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:29.985989 21756 server_base.cc:1061] running on GCE node
I20260709 13:12:29.987186 32694 heartbeater.cc:344] Connected to a master server at 127.21.63.62:41781
I20260709 13:12:29.987644 32694 heartbeater.cc:461] Registering TS with master...
W20260709 13:12:29.987967 32701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:29.988511 32694 heartbeater.cc:507] Master 127.21.63.62:41781 requested a full tablet report, sending...
I20260709 13:12:29.989099 21756 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:12:29.989363 21756 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:12:29.989564 21756 hybrid_clock.cc:648] HybridClock initialized: now 1783602749989546 us; error 0 us; skew 500 ppm
I20260709 13:12:29.990193 21756 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:12:29.990693 32492 ts_manager.cc:194] Registered new tserver with Master: bbe610f3a51e4732a48c941905e222f1 (127.21.63.2:40017)
I20260709 13:12:29.992264 32492 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57534
I20260709 13:12:29.992831 21756 webserver.cc:533] Webserver started at http://127.21.63.3:33529/ using document root <none> and password file <none>
I20260709 13:12:29.993386 21756 fs_manager.cc:362] Metadata directory not provided
I20260709 13:12:29.993609 21756 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:12:29.993902 21756 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:12:29.995059 21756 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602429420501-21756-0/minicluster-data/ts-2-root/instance:
uuid: "da3c6ff682704489a1fbaa0aadae6171"
format_stamp: "Formatted at 2026-07-09 13:12:29 on dist-test-slave-c6mt"
I20260709 13:12:29.999894 21756 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 13:12:30.003603 32705 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:30.004522 21756 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.002s	sys 0.000s
I20260709 13:12:30.004902 21756 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602429420501-21756-0/minicluster-data/ts-2-root
uuid: "da3c6ff682704489a1fbaa0aadae6171"
format_stamp: "Formatted at 2026-07-09 13:12:29 on dist-test-slave-c6mt"
I20260709 13:12:30.005261 21756 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602429420501-21756-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602429420501-21756-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783602429420501-21756-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:12:30.021196 21756 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:12:30.022387 21756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:12:30.023869 21756 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:12:30.026203 21756 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:12:30.026432 21756 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:30.026674 21756 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:12:30.026841 21756 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:30.071007 21756 rpc_server.cc:307] RPC server started. Bound to: 127.21.63.3:37001
I20260709 13:12:30.071167 32767 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.63.3:37001 every 8 connection(s)
I20260709 13:12:30.085887   300 heartbeater.cc:344] Connected to a master server at 127.21.63.62:41781
I20260709 13:12:30.086388   300 heartbeater.cc:461] Registering TS with master...
I20260709 13:12:30.087149   300 heartbeater.cc:507] Master 127.21.63.62:41781 requested a full tablet report, sending...
I20260709 13:12:30.089202 32492 ts_manager.cc:194] Registered new tserver with Master: da3c6ff682704489a1fbaa0aadae6171 (127.21.63.3:37001)
I20260709 13:12:30.089318 21756 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014801432s
I20260709 13:12:30.090731 32492 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:57548
I20260709 13:12:30.115504 32492 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57560:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:12:30.117791 32492 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:12:30.166528 32584 tablet_service.cc:1511] Processing CreateTablet for tablet 9ae94e8fb1e743b6a91213360360ff6d (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=e44b1976b36f428ba371dcc645719b49]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:12:30.167129 32583 tablet_service.cc:1511] Processing CreateTablet for tablet 8c5a000256b54521b6c4e697a81b2b4f (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=e44b1976b36f428ba371dcc645719b49]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:12:30.168077 32584 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ae94e8fb1e743b6a91213360360ff6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:30.164003 32733 tablet_service.cc:1511] Processing CreateTablet for tablet 9ae94e8fb1e743b6a91213360360ff6d (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=e44b1976b36f428ba371dcc645719b49]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:12:30.170580 32733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ae94e8fb1e743b6a91213360360ff6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:30.171532 32583 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c5a000256b54521b6c4e697a81b2b4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:30.172945 32732 tablet_service.cc:1511] Processing CreateTablet for tablet 8c5a000256b54521b6c4e697a81b2b4f (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=e44b1976b36f428ba371dcc645719b49]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:12:30.175488 32732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c5a000256b54521b6c4e697a81b2b4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:30.181447 32659 tablet_service.cc:1511] Processing CreateTablet for tablet 9ae94e8fb1e743b6a91213360360ff6d (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=e44b1976b36f428ba371dcc645719b49]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:12:30.181870 32658 tablet_service.cc:1511] Processing CreateTablet for tablet 8c5a000256b54521b6c4e697a81b2b4f (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=e44b1976b36f428ba371dcc645719b49]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:12:30.182921 32659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9ae94e8fb1e743b6a91213360360ff6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:30.183535 32658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c5a000256b54521b6c4e697a81b2b4f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:30.214432   317 tablet_bootstrap.cc:492] T 9ae94e8fb1e743b6a91213360360ff6d P da3c6ff682704489a1fbaa0aadae6171: Bootstrap starting.
I20260709 13:12:30.214471   315 tablet_bootstrap.cc:492] T 9ae94e8fb1e743b6a91213360360ff6d P bbe610f3a51e4732a48c941905e222f1: Bootstrap starting.
I20260709 13:12:30.215241   316 tablet_bootstrap.cc:492] T 8c5a000256b54521b6c4e697a81b2b4f P 00d7cee1aea54375980b10dbdf4c4b79: Bootstrap starting.
I20260709 13:12:30.220607   315 tablet_bootstrap.cc:654] T 9ae94e8fb1e743b6a91213360360ff6d P bbe610f3a51e4732a48c941905e222f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.222059   317 tablet_bootstrap.cc:654] T 9ae94e8fb1e743b6a91213360360ff6d P da3c6ff682704489a1fbaa0aadae6171: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.222637   316 tablet_bootstrap.cc:654] T 8c5a000256b54521b6c4e697a81b2b4f P 00d7cee1aea54375980b10dbdf4c4b79: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.227646   315 tablet_bootstrap.cc:492] T 9ae94e8fb1e743b6a91213360360ff6d P bbe610f3a51e4732a48c941905e222f1: No bootstrap required, opened a new log
I20260709 13:12:30.228360   315 ts_tablet_manager.cc:1403] T 9ae94e8fb1e743b6a91213360360ff6d P bbe610f3a51e4732a48c941905e222f1: Time spent bootstrapping tablet: real 0.014s	user 0.009s	sys 0.005s
I20260709 13:12:30.230587   316 tablet_bootstrap.cc:492] T 8c5a000256b54521b6c4e697a81b2b4f P 00d7cee1aea54375980b10dbdf4c4b79: No bootstrap required, opened a new log
I20260709 13:12:30.231153   316 ts_tablet_manager.cc:1403] T 8c5a000256b54521b6c4e697a81b2b4f P 00d7cee1aea54375980b10dbdf4c4b79: Time spent bootstrapping tablet: real 0.016s	user 0.010s	sys 0.004s
I20260709 13:12:30.231465   317 tablet_bootstrap.cc:492] T 9ae94e8fb1e743b6a91213360360ff6d P da3c6ff682704489a1fbaa0aadae6171: No bootstrap required, opened a new log
I20260709 13:12:30.231544   315 raft_consensus.cc:359] T 9ae94e8fb1e743b6a91213360360ff6d P bbe610f3a51e4732a48c941905e222f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } } peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } }
I20260709 13:12:30.232123   317 ts_tablet_manager.cc:1403] T 9ae94e8fb1e743b6a91213360360ff6d P da3c6ff682704489a1fbaa0aadae6171: Time spent bootstrapping tablet: real 0.018s	user 0.017s	sys 0.000s
I20260709 13:12:30.232465   315 raft_consensus.cc:385] T 9ae94e8fb1e743b6a91213360360ff6d P bbe610f3a51e4732a48c941905e222f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.232833   315 raft_consensus.cc:740] T 9ae94e8fb1e743b6a91213360360ff6d P bbe610f3a51e4732a48c941905e222f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbe610f3a51e4732a48c941905e222f1, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.233613   315 consensus_queue.cc:260] T 9ae94e8fb1e743b6a91213360360ff6d P bbe610f3a51e4732a48c941905e222f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } } peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } }
I20260709 13:12:30.234089   316 raft_consensus.cc:359] T 8c5a000256b54521b6c4e697a81b2b4f P 00d7cee1aea54375980b10dbdf4c4b79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } } peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } }
I20260709 13:12:30.234947   316 raft_consensus.cc:385] T 8c5a000256b54521b6c4e697a81b2b4f P 00d7cee1aea54375980b10dbdf4c4b79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.235296   316 raft_consensus.cc:740] T 8c5a000256b54521b6c4e697a81b2b4f P 00d7cee1aea54375980b10dbdf4c4b79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00d7cee1aea54375980b10dbdf4c4b79, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.235276   317 raft_consensus.cc:359] T 9ae94e8fb1e743b6a91213360360ff6d P da3c6ff682704489a1fbaa0aadae6171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } } peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } }
I20260709 13:12:30.236011 32694 heartbeater.cc:499] Master 127.21.63.62:41781 was elected leader, sending a full tablet report...
I20260709 13:12:30.236454   316 consensus_queue.cc:260] T 8c5a000256b54521b6c4e697a81b2b4f P 00d7cee1aea54375980b10dbdf4c4b79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } } peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } }
I20260709 13:12:30.237475   315 ts_tablet_manager.cc:1434] T 9ae94e8fb1e743b6a91213360360ff6d P bbe610f3a51e4732a48c941905e222f1: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.004s
I20260709 13:12:30.238694   315 tablet_bootstrap.cc:492] T 8c5a000256b54521b6c4e697a81b2b4f P bbe610f3a51e4732a48c941905e222f1: Bootstrap starting.
I20260709 13:12:30.239151 32619 heartbeater.cc:499] Master 127.21.63.62:41781 was elected leader, sending a full tablet report...
I20260709 13:12:30.239590   316 ts_tablet_manager.cc:1434] T 8c5a000256b54521b6c4e697a81b2b4f P 00d7cee1aea54375980b10dbdf4c4b79: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.004s
I20260709 13:12:30.240628   316 tablet_bootstrap.cc:492] T 9ae94e8fb1e743b6a91213360360ff6d P 00d7cee1aea54375980b10dbdf4c4b79: Bootstrap starting.
I20260709 13:12:30.245692   315 tablet_bootstrap.cc:654] T 8c5a000256b54521b6c4e697a81b2b4f P bbe610f3a51e4732a48c941905e222f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.236239   317 raft_consensus.cc:385] T 9ae94e8fb1e743b6a91213360360ff6d P da3c6ff682704489a1fbaa0aadae6171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.246698   317 raft_consensus.cc:740] T 9ae94e8fb1e743b6a91213360360ff6d P da3c6ff682704489a1fbaa0aadae6171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3c6ff682704489a1fbaa0aadae6171, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.247655   317 consensus_queue.cc:260] T 9ae94e8fb1e743b6a91213360360ff6d P da3c6ff682704489a1fbaa0aadae6171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } } peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } }
I20260709 13:12:30.249831   316 tablet_bootstrap.cc:654] T 9ae94e8fb1e743b6a91213360360ff6d P 00d7cee1aea54375980b10dbdf4c4b79: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.257843   317 ts_tablet_manager.cc:1434] T 9ae94e8fb1e743b6a91213360360ff6d P da3c6ff682704489a1fbaa0aadae6171: Time spent starting tablet: real 0.025s	user 0.010s	sys 0.004s
I20260709 13:12:30.259660   315 tablet_bootstrap.cc:492] T 8c5a000256b54521b6c4e697a81b2b4f P bbe610f3a51e4732a48c941905e222f1: No bootstrap required, opened a new log
I20260709 13:12:30.260581   315 ts_tablet_manager.cc:1403] T 8c5a000256b54521b6c4e697a81b2b4f P bbe610f3a51e4732a48c941905e222f1: Time spent bootstrapping tablet: real 0.022s	user 0.008s	sys 0.011s
I20260709 13:12:30.261152   300 heartbeater.cc:499] Master 127.21.63.62:41781 was elected leader, sending a full tablet report...
I20260709 13:12:30.262231   317 tablet_bootstrap.cc:492] T 8c5a000256b54521b6c4e697a81b2b4f P da3c6ff682704489a1fbaa0aadae6171: Bootstrap starting.
I20260709 13:12:30.263621   316 tablet_bootstrap.cc:492] T 9ae94e8fb1e743b6a91213360360ff6d P 00d7cee1aea54375980b10dbdf4c4b79: No bootstrap required, opened a new log
I20260709 13:12:30.264218   316 ts_tablet_manager.cc:1403] T 9ae94e8fb1e743b6a91213360360ff6d P 00d7cee1aea54375980b10dbdf4c4b79: Time spent bootstrapping tablet: real 0.024s	user 0.020s	sys 0.000s
I20260709 13:12:30.266742   315 raft_consensus.cc:359] T 8c5a000256b54521b6c4e697a81b2b4f P bbe610f3a51e4732a48c941905e222f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } } peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } }
I20260709 13:12:30.267016   316 raft_consensus.cc:359] T 9ae94e8fb1e743b6a91213360360ff6d P 00d7cee1aea54375980b10dbdf4c4b79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } } peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } }
I20260709 13:12:30.267583   315 raft_consensus.cc:385] T 8c5a000256b54521b6c4e697a81b2b4f P bbe610f3a51e4732a48c941905e222f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.267746   316 raft_consensus.cc:385] T 9ae94e8fb1e743b6a91213360360ff6d P 00d7cee1aea54375980b10dbdf4c4b79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.268025   315 raft_consensus.cc:740] T 8c5a000256b54521b6c4e697a81b2b4f P bbe610f3a51e4732a48c941905e222f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbe610f3a51e4732a48c941905e222f1, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.268105   316 raft_consensus.cc:740] T 9ae94e8fb1e743b6a91213360360ff6d P 00d7cee1aea54375980b10dbdf4c4b79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00d7cee1aea54375980b10dbdf4c4b79, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.269017   315 consensus_queue.cc:260] T 8c5a000256b54521b6c4e697a81b2b4f P bbe610f3a51e4732a48c941905e222f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } } peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } }
I20260709 13:12:30.269301   316 consensus_queue.cc:260] T 9ae94e8fb1e743b6a91213360360ff6d P 00d7cee1aea54375980b10dbdf4c4b79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } } peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } }
I20260709 13:12:30.271503   315 ts_tablet_manager.cc:1434] T 8c5a000256b54521b6c4e697a81b2b4f P bbe610f3a51e4732a48c941905e222f1: Time spent starting tablet: real 0.010s	user 0.000s	sys 0.006s
I20260709 13:12:30.271654   316 ts_tablet_manager.cc:1434] T 9ae94e8fb1e743b6a91213360360ff6d P 00d7cee1aea54375980b10dbdf4c4b79: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.003s
I20260709 13:12:30.272750   317 tablet_bootstrap.cc:654] T 8c5a000256b54521b6c4e697a81b2b4f P da3c6ff682704489a1fbaa0aadae6171: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.280530   317 tablet_bootstrap.cc:492] T 8c5a000256b54521b6c4e697a81b2b4f P da3c6ff682704489a1fbaa0aadae6171: No bootstrap required, opened a new log
I20260709 13:12:30.281169   317 ts_tablet_manager.cc:1403] T 8c5a000256b54521b6c4e697a81b2b4f P da3c6ff682704489a1fbaa0aadae6171: Time spent bootstrapping tablet: real 0.019s	user 0.005s	sys 0.007s
I20260709 13:12:30.283974   317 raft_consensus.cc:359] T 8c5a000256b54521b6c4e697a81b2b4f P da3c6ff682704489a1fbaa0aadae6171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } } peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } }
I20260709 13:12:30.284741   317 raft_consensus.cc:385] T 8c5a000256b54521b6c4e697a81b2b4f P da3c6ff682704489a1fbaa0aadae6171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.285024   317 raft_consensus.cc:740] T 8c5a000256b54521b6c4e697a81b2b4f P da3c6ff682704489a1fbaa0aadae6171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3c6ff682704489a1fbaa0aadae6171, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.285665   317 consensus_queue.cc:260] T 8c5a000256b54521b6c4e697a81b2b4f P da3c6ff682704489a1fbaa0aadae6171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } } peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } }
I20260709 13:12:30.287701   317 ts_tablet_manager.cc:1434] T 8c5a000256b54521b6c4e697a81b2b4f P da3c6ff682704489a1fbaa0aadae6171: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:12:30.288273   324 raft_consensus.cc:493] T 9ae94e8fb1e743b6a91213360360ff6d P da3c6ff682704489a1fbaa0aadae6171 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.288904   324 raft_consensus.cc:515] T 9ae94e8fb1e743b6a91213360360ff6d P da3c6ff682704489a1fbaa0aadae6171 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } } peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } }
I20260709 13:12:30.291868   324 leader_election.cc:290] T 9ae94e8fb1e743b6a91213360360ff6d P da3c6ff682704489a1fbaa0aadae6171 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 00d7cee1aea54375980b10dbdf4c4b79 (127.21.63.1:34359), bbe610f3a51e4732a48c941905e222f1 (127.21.63.2:40017)
I20260709 13:12:30.304005 32594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae94e8fb1e743b6a91213360360ff6d" candidate_uuid: "da3c6ff682704489a1fbaa0aadae6171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00d7cee1aea54375980b10dbdf4c4b79" is_pre_election: true
I20260709 13:12:30.304920 32594 raft_consensus.cc:2468] T 9ae94e8fb1e743b6a91213360360ff6d P 00d7cee1aea54375980b10dbdf4c4b79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3c6ff682704489a1fbaa0aadae6171 in term 0.
I20260709 13:12:30.305241 32669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae94e8fb1e743b6a91213360360ff6d" candidate_uuid: "da3c6ff682704489a1fbaa0aadae6171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbe610f3a51e4732a48c941905e222f1" is_pre_election: true
I20260709 13:12:30.306003 32669 raft_consensus.cc:2468] T 9ae94e8fb1e743b6a91213360360ff6d P bbe610f3a51e4732a48c941905e222f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3c6ff682704489a1fbaa0aadae6171 in term 0.
I20260709 13:12:30.306337 32706 leader_election.cc:304] T 9ae94e8fb1e743b6a91213360360ff6d P da3c6ff682704489a1fbaa0aadae6171 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00d7cee1aea54375980b10dbdf4c4b79, da3c6ff682704489a1fbaa0aadae6171; no voters: 
I20260709 13:12:30.307193   324 raft_consensus.cc:2804] T 9ae94e8fb1e743b6a91213360360ff6d P da3c6ff682704489a1fbaa0aadae6171 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:30.307581   324 raft_consensus.cc:493] T 9ae94e8fb1e743b6a91213360360ff6d P da3c6ff682704489a1fbaa0aadae6171 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:30.307893   324 raft_consensus.cc:3060] T 9ae94e8fb1e743b6a91213360360ff6d P da3c6ff682704489a1fbaa0aadae6171 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.313755   324 raft_consensus.cc:515] T 9ae94e8fb1e743b6a91213360360ff6d P da3c6ff682704489a1fbaa0aadae6171 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } } peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } }
I20260709 13:12:30.315390   324 leader_election.cc:290] T 9ae94e8fb1e743b6a91213360360ff6d P da3c6ff682704489a1fbaa0aadae6171 [CANDIDATE]: Term 1 election: Requested vote from peers 00d7cee1aea54375980b10dbdf4c4b79 (127.21.63.1:34359), bbe610f3a51e4732a48c941905e222f1 (127.21.63.2:40017)
I20260709 13:12:30.316442 32594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae94e8fb1e743b6a91213360360ff6d" candidate_uuid: "da3c6ff682704489a1fbaa0aadae6171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00d7cee1aea54375980b10dbdf4c4b79"
I20260709 13:12:30.316732 32669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae94e8fb1e743b6a91213360360ff6d" candidate_uuid: "da3c6ff682704489a1fbaa0aadae6171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbe610f3a51e4732a48c941905e222f1"
I20260709 13:12:30.316995 32594 raft_consensus.cc:3060] T 9ae94e8fb1e743b6a91213360360ff6d P 00d7cee1aea54375980b10dbdf4c4b79 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.317377 32669 raft_consensus.cc:3060] T 9ae94e8fb1e743b6a91213360360ff6d P bbe610f3a51e4732a48c941905e222f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.322372 32594 raft_consensus.cc:2468] T 9ae94e8fb1e743b6a91213360360ff6d P 00d7cee1aea54375980b10dbdf4c4b79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3c6ff682704489a1fbaa0aadae6171 in term 1.
I20260709 13:12:30.322775 32669 raft_consensus.cc:2468] T 9ae94e8fb1e743b6a91213360360ff6d P bbe610f3a51e4732a48c941905e222f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3c6ff682704489a1fbaa0aadae6171 in term 1.
I20260709 13:12:30.323693 32706 leader_election.cc:304] T 9ae94e8fb1e743b6a91213360360ff6d P da3c6ff682704489a1fbaa0aadae6171 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00d7cee1aea54375980b10dbdf4c4b79, da3c6ff682704489a1fbaa0aadae6171; no voters: 
I20260709 13:12:30.324554   324 raft_consensus.cc:2804] T 9ae94e8fb1e743b6a91213360360ff6d P da3c6ff682704489a1fbaa0aadae6171 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:30.326294   324 raft_consensus.cc:697] T 9ae94e8fb1e743b6a91213360360ff6d P da3c6ff682704489a1fbaa0aadae6171 [term 1 LEADER]: Becoming Leader. State: Replica: da3c6ff682704489a1fbaa0aadae6171, State: Running, Role: LEADER
I20260709 13:12:30.327132   324 consensus_queue.cc:237] T 9ae94e8fb1e743b6a91213360360ff6d P da3c6ff682704489a1fbaa0aadae6171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } } peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } }
I20260709 13:12:30.334569 32492 catalog_manager.cc:5697] T 9ae94e8fb1e743b6a91213360360ff6d P da3c6ff682704489a1fbaa0aadae6171 reported cstate change: term changed from 0 to 1, leader changed from <none> to da3c6ff682704489a1fbaa0aadae6171 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "da3c6ff682704489a1fbaa0aadae6171" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:30.441514   324 raft_consensus.cc:493] T 8c5a000256b54521b6c4e697a81b2b4f P da3c6ff682704489a1fbaa0aadae6171 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.442140   324 raft_consensus.cc:515] T 8c5a000256b54521b6c4e697a81b2b4f P da3c6ff682704489a1fbaa0aadae6171 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } } peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } }
I20260709 13:12:30.444016   324 leader_election.cc:290] T 8c5a000256b54521b6c4e697a81b2b4f P da3c6ff682704489a1fbaa0aadae6171 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 00d7cee1aea54375980b10dbdf4c4b79 (127.21.63.1:34359), bbe610f3a51e4732a48c941905e222f1 (127.21.63.2:40017)
I20260709 13:12:30.444793 32594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c5a000256b54521b6c4e697a81b2b4f" candidate_uuid: "da3c6ff682704489a1fbaa0aadae6171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00d7cee1aea54375980b10dbdf4c4b79" is_pre_election: true
I20260709 13:12:30.445003 32669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c5a000256b54521b6c4e697a81b2b4f" candidate_uuid: "da3c6ff682704489a1fbaa0aadae6171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbe610f3a51e4732a48c941905e222f1" is_pre_election: true
I20260709 13:12:30.445500 32594 raft_consensus.cc:2468] T 8c5a000256b54521b6c4e697a81b2b4f P 00d7cee1aea54375980b10dbdf4c4b79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3c6ff682704489a1fbaa0aadae6171 in term 0.
I20260709 13:12:30.445596 32669 raft_consensus.cc:2468] T 8c5a000256b54521b6c4e697a81b2b4f P bbe610f3a51e4732a48c941905e222f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3c6ff682704489a1fbaa0aadae6171 in term 0.
I20260709 13:12:30.446556 32706 leader_election.cc:304] T 8c5a000256b54521b6c4e697a81b2b4f P da3c6ff682704489a1fbaa0aadae6171 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00d7cee1aea54375980b10dbdf4c4b79, da3c6ff682704489a1fbaa0aadae6171; no voters: 
I20260709 13:12:30.447214   324 raft_consensus.cc:2804] T 8c5a000256b54521b6c4e697a81b2b4f P da3c6ff682704489a1fbaa0aadae6171 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:30.447588   324 raft_consensus.cc:493] T 8c5a000256b54521b6c4e697a81b2b4f P da3c6ff682704489a1fbaa0aadae6171 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:30.447839   324 raft_consensus.cc:3060] T 8c5a000256b54521b6c4e697a81b2b4f P da3c6ff682704489a1fbaa0aadae6171 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.452787   324 raft_consensus.cc:515] T 8c5a000256b54521b6c4e697a81b2b4f P da3c6ff682704489a1fbaa0aadae6171 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } } peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } }
I20260709 13:12:30.454519   324 leader_election.cc:290] T 8c5a000256b54521b6c4e697a81b2b4f P da3c6ff682704489a1fbaa0aadae6171 [CANDIDATE]: Term 1 election: Requested vote from peers 00d7cee1aea54375980b10dbdf4c4b79 (127.21.63.1:34359), bbe610f3a51e4732a48c941905e222f1 (127.21.63.2:40017)
I20260709 13:12:30.455206 32594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c5a000256b54521b6c4e697a81b2b4f" candidate_uuid: "da3c6ff682704489a1fbaa0aadae6171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00d7cee1aea54375980b10dbdf4c4b79"
I20260709 13:12:30.455592 32669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c5a000256b54521b6c4e697a81b2b4f" candidate_uuid: "da3c6ff682704489a1fbaa0aadae6171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbe610f3a51e4732a48c941905e222f1"
I20260709 13:12:30.455920 32594 raft_consensus.cc:3060] T 8c5a000256b54521b6c4e697a81b2b4f P 00d7cee1aea54375980b10dbdf4c4b79 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.456208 32669 raft_consensus.cc:3060] T 8c5a000256b54521b6c4e697a81b2b4f P bbe610f3a51e4732a48c941905e222f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.463061 32669 raft_consensus.cc:2468] T 8c5a000256b54521b6c4e697a81b2b4f P bbe610f3a51e4732a48c941905e222f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3c6ff682704489a1fbaa0aadae6171 in term 1.
I20260709 13:12:30.463061 32594 raft_consensus.cc:2468] T 8c5a000256b54521b6c4e697a81b2b4f P 00d7cee1aea54375980b10dbdf4c4b79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3c6ff682704489a1fbaa0aadae6171 in term 1.
I20260709 13:12:30.464453 32706 leader_election.cc:304] T 8c5a000256b54521b6c4e697a81b2b4f P da3c6ff682704489a1fbaa0aadae6171 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00d7cee1aea54375980b10dbdf4c4b79, da3c6ff682704489a1fbaa0aadae6171; no voters: 
I20260709 13:12:30.465193   324 raft_consensus.cc:2804] T 8c5a000256b54521b6c4e697a81b2b4f P da3c6ff682704489a1fbaa0aadae6171 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:30.465677   324 raft_consensus.cc:697] T 8c5a000256b54521b6c4e697a81b2b4f P da3c6ff682704489a1fbaa0aadae6171 [term 1 LEADER]: Becoming Leader. State: Replica: da3c6ff682704489a1fbaa0aadae6171, State: Running, Role: LEADER
I20260709 13:12:30.466460   324 consensus_queue.cc:237] T 8c5a000256b54521b6c4e697a81b2b4f P da3c6ff682704489a1fbaa0aadae6171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } } peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } }
I20260709 13:12:30.473524 32492 catalog_manager.cc:5697] T 8c5a000256b54521b6c4e697a81b2b4f P da3c6ff682704489a1fbaa0aadae6171 reported cstate change: term changed from 0 to 1, leader changed from <none> to da3c6ff682704489a1fbaa0aadae6171 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "da3c6ff682704489a1fbaa0aadae6171" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:30.567060 32492 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57574:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:12:30.569032 32492 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:12:30.591228 32583 tablet_service.cc:1511] Processing CreateTablet for tablet b9eb1f9645fa430d9a24dca067540bbd (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=e6dee541749f4468bf184eaa306e6f7e]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:12:30.592537 32583 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9eb1f9645fa430d9a24dca067540bbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:30.592418 32659 tablet_service.cc:1511] Processing CreateTablet for tablet b9eb1f9645fa430d9a24dca067540bbd (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=e6dee541749f4468bf184eaa306e6f7e]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:12:30.593819 32659 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9eb1f9645fa430d9a24dca067540bbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:30.595397 32733 tablet_service.cc:1511] Processing CreateTablet for tablet b9eb1f9645fa430d9a24dca067540bbd (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=e6dee541749f4468bf184eaa306e6f7e]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 13:12:30.596628 32733 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9eb1f9645fa430d9a24dca067540bbd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:30.598304 32732 tablet_service.cc:1511] Processing CreateTablet for tablet 986d36f58fbc438aa9ae156bdb87e69c (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=e6dee541749f4468bf184eaa306e6f7e]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:12:30.599689 32732 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 986d36f58fbc438aa9ae156bdb87e69c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:30.600098 32584 tablet_service.cc:1511] Processing CreateTablet for tablet 986d36f58fbc438aa9ae156bdb87e69c (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=e6dee541749f4468bf184eaa306e6f7e]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:12:30.601555 32584 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 986d36f58fbc438aa9ae156bdb87e69c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:30.598332 32658 tablet_service.cc:1511] Processing CreateTablet for tablet 986d36f58fbc438aa9ae156bdb87e69c (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=e6dee541749f4468bf184eaa306e6f7e]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 13:12:30.605470 32658 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 986d36f58fbc438aa9ae156bdb87e69c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:30.618006   316 tablet_bootstrap.cc:492] T b9eb1f9645fa430d9a24dca067540bbd P 00d7cee1aea54375980b10dbdf4c4b79: Bootstrap starting.
I20260709 13:12:30.623929   317 tablet_bootstrap.cc:492] T 986d36f58fbc438aa9ae156bdb87e69c P da3c6ff682704489a1fbaa0aadae6171: Bootstrap starting.
I20260709 13:12:30.631079   317 tablet_bootstrap.cc:654] T 986d36f58fbc438aa9ae156bdb87e69c P da3c6ff682704489a1fbaa0aadae6171: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.632251   316 tablet_bootstrap.cc:654] T b9eb1f9645fa430d9a24dca067540bbd P 00d7cee1aea54375980b10dbdf4c4b79: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.637825   315 tablet_bootstrap.cc:492] T b9eb1f9645fa430d9a24dca067540bbd P bbe610f3a51e4732a48c941905e222f1: Bootstrap starting.
I20260709 13:12:30.643226   317 tablet_bootstrap.cc:492] T 986d36f58fbc438aa9ae156bdb87e69c P da3c6ff682704489a1fbaa0aadae6171: No bootstrap required, opened a new log
I20260709 13:12:30.643606   316 tablet_bootstrap.cc:492] T b9eb1f9645fa430d9a24dca067540bbd P 00d7cee1aea54375980b10dbdf4c4b79: No bootstrap required, opened a new log
I20260709 13:12:30.643855   317 ts_tablet_manager.cc:1403] T 986d36f58fbc438aa9ae156bdb87e69c P da3c6ff682704489a1fbaa0aadae6171: Time spent bootstrapping tablet: real 0.020s	user 0.011s	sys 0.004s
I20260709 13:12:30.644194   316 ts_tablet_manager.cc:1403] T b9eb1f9645fa430d9a24dca067540bbd P 00d7cee1aea54375980b10dbdf4c4b79: Time spent bootstrapping tablet: real 0.026s	user 0.003s	sys 0.010s
I20260709 13:12:30.644764   315 tablet_bootstrap.cc:654] T b9eb1f9645fa430d9a24dca067540bbd P bbe610f3a51e4732a48c941905e222f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.646771   316 raft_consensus.cc:359] T b9eb1f9645fa430d9a24dca067540bbd P 00d7cee1aea54375980b10dbdf4c4b79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } } peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } }
I20260709 13:12:30.647428   316 raft_consensus.cc:385] T b9eb1f9645fa430d9a24dca067540bbd P 00d7cee1aea54375980b10dbdf4c4b79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.647799   316 raft_consensus.cc:740] T b9eb1f9645fa430d9a24dca067540bbd P 00d7cee1aea54375980b10dbdf4c4b79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00d7cee1aea54375980b10dbdf4c4b79, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.648263   317 raft_consensus.cc:359] T 986d36f58fbc438aa9ae156bdb87e69c P da3c6ff682704489a1fbaa0aadae6171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } } peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } }
I20260709 13:12:30.649642   317 raft_consensus.cc:385] T 986d36f58fbc438aa9ae156bdb87e69c P da3c6ff682704489a1fbaa0aadae6171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.648762   316 consensus_queue.cc:260] T b9eb1f9645fa430d9a24dca067540bbd P 00d7cee1aea54375980b10dbdf4c4b79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } } peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } }
I20260709 13:12:30.650017   317 raft_consensus.cc:740] T 986d36f58fbc438aa9ae156bdb87e69c P da3c6ff682704489a1fbaa0aadae6171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3c6ff682704489a1fbaa0aadae6171, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.650972   317 consensus_queue.cc:260] T 986d36f58fbc438aa9ae156bdb87e69c P da3c6ff682704489a1fbaa0aadae6171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } } peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } }
I20260709 13:12:30.651439   315 tablet_bootstrap.cc:492] T b9eb1f9645fa430d9a24dca067540bbd P bbe610f3a51e4732a48c941905e222f1: No bootstrap required, opened a new log
I20260709 13:12:30.651631   316 ts_tablet_manager.cc:1434] T b9eb1f9645fa430d9a24dca067540bbd P 00d7cee1aea54375980b10dbdf4c4b79: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 13:12:30.652110   315 ts_tablet_manager.cc:1403] T b9eb1f9645fa430d9a24dca067540bbd P bbe610f3a51e4732a48c941905e222f1: Time spent bootstrapping tablet: real 0.014s	user 0.005s	sys 0.009s
I20260709 13:12:30.653823   317 ts_tablet_manager.cc:1434] T 986d36f58fbc438aa9ae156bdb87e69c P da3c6ff682704489a1fbaa0aadae6171: Time spent starting tablet: real 0.009s	user 0.002s	sys 0.004s
I20260709 13:12:30.654863   317 tablet_bootstrap.cc:492] T b9eb1f9645fa430d9a24dca067540bbd P da3c6ff682704489a1fbaa0aadae6171: Bootstrap starting.
I20260709 13:12:30.656133   315 raft_consensus.cc:359] T b9eb1f9645fa430d9a24dca067540bbd P bbe610f3a51e4732a48c941905e222f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } } peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } }
I20260709 13:12:30.656721   315 raft_consensus.cc:385] T b9eb1f9645fa430d9a24dca067540bbd P bbe610f3a51e4732a48c941905e222f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.657037   315 raft_consensus.cc:740] T b9eb1f9645fa430d9a24dca067540bbd P bbe610f3a51e4732a48c941905e222f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbe610f3a51e4732a48c941905e222f1, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.657141   316 tablet_bootstrap.cc:492] T 986d36f58fbc438aa9ae156bdb87e69c P 00d7cee1aea54375980b10dbdf4c4b79: Bootstrap starting.
I20260709 13:12:30.657919   315 consensus_queue.cc:260] T b9eb1f9645fa430d9a24dca067540bbd P bbe610f3a51e4732a48c941905e222f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } } peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } }
I20260709 13:12:30.660118   315 ts_tablet_manager.cc:1434] T b9eb1f9645fa430d9a24dca067540bbd P bbe610f3a51e4732a48c941905e222f1: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:12:30.661206   315 tablet_bootstrap.cc:492] T 986d36f58fbc438aa9ae156bdb87e69c P bbe610f3a51e4732a48c941905e222f1: Bootstrap starting.
I20260709 13:12:30.664902   317 tablet_bootstrap.cc:654] T b9eb1f9645fa430d9a24dca067540bbd P da3c6ff682704489a1fbaa0aadae6171: Neither blocks nor log segments found. Creating new log.
W20260709 13:12:30.665364 32540 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:12:30.665696   316 tablet_bootstrap.cc:654] T 986d36f58fbc438aa9ae156bdb87e69c P 00d7cee1aea54375980b10dbdf4c4b79: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.667713   315 tablet_bootstrap.cc:654] T 986d36f58fbc438aa9ae156bdb87e69c P bbe610f3a51e4732a48c941905e222f1: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:30.677647   316 tablet_bootstrap.cc:492] T 986d36f58fbc438aa9ae156bdb87e69c P 00d7cee1aea54375980b10dbdf4c4b79: No bootstrap required, opened a new log
I20260709 13:12:30.678506   316 ts_tablet_manager.cc:1403] T 986d36f58fbc438aa9ae156bdb87e69c P 00d7cee1aea54375980b10dbdf4c4b79: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.008s
I20260709 13:12:30.682309   316 raft_consensus.cc:359] T 986d36f58fbc438aa9ae156bdb87e69c P 00d7cee1aea54375980b10dbdf4c4b79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } } peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } }
I20260709 13:12:30.683337   316 raft_consensus.cc:385] T 986d36f58fbc438aa9ae156bdb87e69c P 00d7cee1aea54375980b10dbdf4c4b79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.683753   316 raft_consensus.cc:740] T 986d36f58fbc438aa9ae156bdb87e69c P 00d7cee1aea54375980b10dbdf4c4b79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00d7cee1aea54375980b10dbdf4c4b79, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.684767   316 consensus_queue.cc:260] T 986d36f58fbc438aa9ae156bdb87e69c P 00d7cee1aea54375980b10dbdf4c4b79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } } peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } }
I20260709 13:12:30.688024   316 ts_tablet_manager.cc:1434] T 986d36f58fbc438aa9ae156bdb87e69c P 00d7cee1aea54375980b10dbdf4c4b79: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:12:30.699232   317 tablet_bootstrap.cc:492] T b9eb1f9645fa430d9a24dca067540bbd P da3c6ff682704489a1fbaa0aadae6171: No bootstrap required, opened a new log
I20260709 13:12:30.700032   317 ts_tablet_manager.cc:1403] T b9eb1f9645fa430d9a24dca067540bbd P da3c6ff682704489a1fbaa0aadae6171: Time spent bootstrapping tablet: real 0.045s	user 0.021s	sys 0.020s
I20260709 13:12:30.699227   315 tablet_bootstrap.cc:492] T 986d36f58fbc438aa9ae156bdb87e69c P bbe610f3a51e4732a48c941905e222f1: No bootstrap required, opened a new log
I20260709 13:12:30.700606   315 ts_tablet_manager.cc:1403] T 986d36f58fbc438aa9ae156bdb87e69c P bbe610f3a51e4732a48c941905e222f1: Time spent bootstrapping tablet: real 0.040s	user 0.027s	sys 0.011s
I20260709 13:12:30.702868   317 raft_consensus.cc:359] T b9eb1f9645fa430d9a24dca067540bbd P da3c6ff682704489a1fbaa0aadae6171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } } peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } }
I20260709 13:12:30.703536   317 raft_consensus.cc:385] T b9eb1f9645fa430d9a24dca067540bbd P da3c6ff682704489a1fbaa0aadae6171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.703845   317 raft_consensus.cc:740] T b9eb1f9645fa430d9a24dca067540bbd P da3c6ff682704489a1fbaa0aadae6171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3c6ff682704489a1fbaa0aadae6171, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.703691   315 raft_consensus.cc:359] T 986d36f58fbc438aa9ae156bdb87e69c P bbe610f3a51e4732a48c941905e222f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } } peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } }
I20260709 13:12:30.704695   315 raft_consensus.cc:385] T 986d36f58fbc438aa9ae156bdb87e69c P bbe610f3a51e4732a48c941905e222f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:30.705049   315 raft_consensus.cc:740] T 986d36f58fbc438aa9ae156bdb87e69c P bbe610f3a51e4732a48c941905e222f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbe610f3a51e4732a48c941905e222f1, State: Initialized, Role: FOLLOWER
I20260709 13:12:30.704970   317 consensus_queue.cc:260] T b9eb1f9645fa430d9a24dca067540bbd P da3c6ff682704489a1fbaa0aadae6171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } } peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } }
I20260709 13:12:30.707106   317 ts_tablet_manager.cc:1434] T b9eb1f9645fa430d9a24dca067540bbd P da3c6ff682704489a1fbaa0aadae6171: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 13:12:30.707721   315 consensus_queue.cc:260] T 986d36f58fbc438aa9ae156bdb87e69c P bbe610f3a51e4732a48c941905e222f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } } peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } }
I20260709 13:12:30.711026   315 ts_tablet_manager.cc:1434] T 986d36f58fbc438aa9ae156bdb87e69c P bbe610f3a51e4732a48c941905e222f1: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:12:30.735380   324 raft_consensus.cc:493] T b9eb1f9645fa430d9a24dca067540bbd P da3c6ff682704489a1fbaa0aadae6171 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.735813   324 raft_consensus.cc:515] T b9eb1f9645fa430d9a24dca067540bbd P da3c6ff682704489a1fbaa0aadae6171 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } } peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } }
I20260709 13:12:30.737602   324 leader_election.cc:290] T b9eb1f9645fa430d9a24dca067540bbd P da3c6ff682704489a1fbaa0aadae6171 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 00d7cee1aea54375980b10dbdf4c4b79 (127.21.63.1:34359), bbe610f3a51e4732a48c941905e222f1 (127.21.63.2:40017)
I20260709 13:12:30.738294 32594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9eb1f9645fa430d9a24dca067540bbd" candidate_uuid: "da3c6ff682704489a1fbaa0aadae6171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00d7cee1aea54375980b10dbdf4c4b79" is_pre_election: true
I20260709 13:12:30.738602 32669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9eb1f9645fa430d9a24dca067540bbd" candidate_uuid: "da3c6ff682704489a1fbaa0aadae6171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbe610f3a51e4732a48c941905e222f1" is_pre_election: true
I20260709 13:12:30.739012 32594 raft_consensus.cc:2468] T b9eb1f9645fa430d9a24dca067540bbd P 00d7cee1aea54375980b10dbdf4c4b79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3c6ff682704489a1fbaa0aadae6171 in term 0.
I20260709 13:12:30.739215 32669 raft_consensus.cc:2468] T b9eb1f9645fa430d9a24dca067540bbd P bbe610f3a51e4732a48c941905e222f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da3c6ff682704489a1fbaa0aadae6171 in term 0.
I20260709 13:12:30.740108 32706 leader_election.cc:304] T b9eb1f9645fa430d9a24dca067540bbd P da3c6ff682704489a1fbaa0aadae6171 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00d7cee1aea54375980b10dbdf4c4b79, da3c6ff682704489a1fbaa0aadae6171; no voters: 
I20260709 13:12:30.740779   324 raft_consensus.cc:2804] T b9eb1f9645fa430d9a24dca067540bbd P da3c6ff682704489a1fbaa0aadae6171 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:30.741132   324 raft_consensus.cc:493] T b9eb1f9645fa430d9a24dca067540bbd P da3c6ff682704489a1fbaa0aadae6171 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:30.741454   324 raft_consensus.cc:3060] T b9eb1f9645fa430d9a24dca067540bbd P da3c6ff682704489a1fbaa0aadae6171 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.746948   324 raft_consensus.cc:515] T b9eb1f9645fa430d9a24dca067540bbd P da3c6ff682704489a1fbaa0aadae6171 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } } peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } }
I20260709 13:12:30.748553   324 leader_election.cc:290] T b9eb1f9645fa430d9a24dca067540bbd P da3c6ff682704489a1fbaa0aadae6171 [CANDIDATE]: Term 1 election: Requested vote from peers 00d7cee1aea54375980b10dbdf4c4b79 (127.21.63.1:34359), bbe610f3a51e4732a48c941905e222f1 (127.21.63.2:40017)
I20260709 13:12:30.749204 32594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9eb1f9645fa430d9a24dca067540bbd" candidate_uuid: "da3c6ff682704489a1fbaa0aadae6171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00d7cee1aea54375980b10dbdf4c4b79"
I20260709 13:12:30.749562 32669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9eb1f9645fa430d9a24dca067540bbd" candidate_uuid: "da3c6ff682704489a1fbaa0aadae6171" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbe610f3a51e4732a48c941905e222f1"
I20260709 13:12:30.749851 32594 raft_consensus.cc:3060] T b9eb1f9645fa430d9a24dca067540bbd P 00d7cee1aea54375980b10dbdf4c4b79 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.750149 32669 raft_consensus.cc:3060] T b9eb1f9645fa430d9a24dca067540bbd P bbe610f3a51e4732a48c941905e222f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.756776 32669 raft_consensus.cc:2468] T b9eb1f9645fa430d9a24dca067540bbd P bbe610f3a51e4732a48c941905e222f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3c6ff682704489a1fbaa0aadae6171 in term 1.
I20260709 13:12:30.756776 32594 raft_consensus.cc:2468] T b9eb1f9645fa430d9a24dca067540bbd P 00d7cee1aea54375980b10dbdf4c4b79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3c6ff682704489a1fbaa0aadae6171 in term 1.
I20260709 13:12:30.757938 32706 leader_election.cc:304] T b9eb1f9645fa430d9a24dca067540bbd P da3c6ff682704489a1fbaa0aadae6171 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00d7cee1aea54375980b10dbdf4c4b79, da3c6ff682704489a1fbaa0aadae6171; no voters: 
I20260709 13:12:30.758592   324 raft_consensus.cc:2804] T b9eb1f9645fa430d9a24dca067540bbd P da3c6ff682704489a1fbaa0aadae6171 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:30.759060   324 raft_consensus.cc:697] T b9eb1f9645fa430d9a24dca067540bbd P da3c6ff682704489a1fbaa0aadae6171 [term 1 LEADER]: Becoming Leader. State: Replica: da3c6ff682704489a1fbaa0aadae6171, State: Running, Role: LEADER
I20260709 13:12:30.759878   324 consensus_queue.cc:237] T b9eb1f9645fa430d9a24dca067540bbd P da3c6ff682704489a1fbaa0aadae6171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } } peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } }
I20260709 13:12:30.766755 32492 catalog_manager.cc:5697] T b9eb1f9645fa430d9a24dca067540bbd P da3c6ff682704489a1fbaa0aadae6171 reported cstate change: term changed from 0 to 1, leader changed from <none> to da3c6ff682704489a1fbaa0aadae6171 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "da3c6ff682704489a1fbaa0aadae6171" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:30.805151   322 raft_consensus.cc:493] T 986d36f58fbc438aa9ae156bdb87e69c P 00d7cee1aea54375980b10dbdf4c4b79 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:30.805658   322 raft_consensus.cc:515] T 986d36f58fbc438aa9ae156bdb87e69c P 00d7cee1aea54375980b10dbdf4c4b79 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } } peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } }
I20260709 13:12:30.807621   322 leader_election.cc:290] T 986d36f58fbc438aa9ae156bdb87e69c P 00d7cee1aea54375980b10dbdf4c4b79 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bbe610f3a51e4732a48c941905e222f1 (127.21.63.2:40017), da3c6ff682704489a1fbaa0aadae6171 (127.21.63.3:37001)
I20260709 13:12:30.818043 32669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d36f58fbc438aa9ae156bdb87e69c" candidate_uuid: "00d7cee1aea54375980b10dbdf4c4b79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbe610f3a51e4732a48c941905e222f1" is_pre_election: true
I20260709 13:12:30.818734 32669 raft_consensus.cc:2468] T 986d36f58fbc438aa9ae156bdb87e69c P bbe610f3a51e4732a48c941905e222f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00d7cee1aea54375980b10dbdf4c4b79 in term 0.
I20260709 13:12:30.819716 32557 leader_election.cc:304] T 986d36f58fbc438aa9ae156bdb87e69c P 00d7cee1aea54375980b10dbdf4c4b79 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00d7cee1aea54375980b10dbdf4c4b79, bbe610f3a51e4732a48c941905e222f1; no voters: 
I20260709 13:12:30.819849 32743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d36f58fbc438aa9ae156bdb87e69c" candidate_uuid: "00d7cee1aea54375980b10dbdf4c4b79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3c6ff682704489a1fbaa0aadae6171" is_pre_election: true
I20260709 13:12:30.820511 32743 raft_consensus.cc:2468] T 986d36f58fbc438aa9ae156bdb87e69c P da3c6ff682704489a1fbaa0aadae6171 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 00d7cee1aea54375980b10dbdf4c4b79 in term 0.
I20260709 13:12:30.820585   322 raft_consensus.cc:2804] T 986d36f58fbc438aa9ae156bdb87e69c P 00d7cee1aea54375980b10dbdf4c4b79 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:30.821022   322 raft_consensus.cc:493] T 986d36f58fbc438aa9ae156bdb87e69c P 00d7cee1aea54375980b10dbdf4c4b79 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:30.821364   322 raft_consensus.cc:3060] T 986d36f58fbc438aa9ae156bdb87e69c P 00d7cee1aea54375980b10dbdf4c4b79 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.826459   322 raft_consensus.cc:515] T 986d36f58fbc438aa9ae156bdb87e69c P 00d7cee1aea54375980b10dbdf4c4b79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } } peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } }
I20260709 13:12:30.828138   322 leader_election.cc:290] T 986d36f58fbc438aa9ae156bdb87e69c P 00d7cee1aea54375980b10dbdf4c4b79 [CANDIDATE]: Term 1 election: Requested vote from peers bbe610f3a51e4732a48c941905e222f1 (127.21.63.2:40017), da3c6ff682704489a1fbaa0aadae6171 (127.21.63.3:37001)
I20260709 13:12:30.829021 32669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d36f58fbc438aa9ae156bdb87e69c" candidate_uuid: "00d7cee1aea54375980b10dbdf4c4b79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbe610f3a51e4732a48c941905e222f1"
I20260709 13:12:30.829316 32743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d36f58fbc438aa9ae156bdb87e69c" candidate_uuid: "00d7cee1aea54375980b10dbdf4c4b79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da3c6ff682704489a1fbaa0aadae6171"
I20260709 13:12:30.829648 32669 raft_consensus.cc:3060] T 986d36f58fbc438aa9ae156bdb87e69c P bbe610f3a51e4732a48c941905e222f1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.830029 32743 raft_consensus.cc:3060] T 986d36f58fbc438aa9ae156bdb87e69c P da3c6ff682704489a1fbaa0aadae6171 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:30.834823 32669 raft_consensus.cc:2468] T 986d36f58fbc438aa9ae156bdb87e69c P bbe610f3a51e4732a48c941905e222f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00d7cee1aea54375980b10dbdf4c4b79 in term 1.
I20260709 13:12:30.835547 32743 raft_consensus.cc:2468] T 986d36f58fbc438aa9ae156bdb87e69c P da3c6ff682704489a1fbaa0aadae6171 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00d7cee1aea54375980b10dbdf4c4b79 in term 1.
I20260709 13:12:30.835862 32557 leader_election.cc:304] T 986d36f58fbc438aa9ae156bdb87e69c P 00d7cee1aea54375980b10dbdf4c4b79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00d7cee1aea54375980b10dbdf4c4b79, bbe610f3a51e4732a48c941905e222f1; no voters: 
I20260709 13:12:30.836717   322 raft_consensus.cc:2804] T 986d36f58fbc438aa9ae156bdb87e69c P 00d7cee1aea54375980b10dbdf4c4b79 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:30.838058   322 raft_consensus.cc:697] T 986d36f58fbc438aa9ae156bdb87e69c P 00d7cee1aea54375980b10dbdf4c4b79 [term 1 LEADER]: Becoming Leader. State: Replica: 00d7cee1aea54375980b10dbdf4c4b79, State: Running, Role: LEADER
I20260709 13:12:30.839535   322 consensus_queue.cc:237] T 986d36f58fbc438aa9ae156bdb87e69c P 00d7cee1aea54375980b10dbdf4c4b79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } } peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } }
I20260709 13:12:30.846401 32492 catalog_manager.cc:5697] T 986d36f58fbc438aa9ae156bdb87e69c P 00d7cee1aea54375980b10dbdf4c4b79 reported cstate change: term changed from 0 to 1, leader changed from <none> to 00d7cee1aea54375980b10dbdf4c4b79 (127.21.63.1). New cstate: current_term: 1 leader_uuid: "00d7cee1aea54375980b10dbdf4c4b79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:30.894733   324 consensus_queue.cc:1048] T 9ae94e8fb1e743b6a91213360360ff6d P da3c6ff682704489a1fbaa0aadae6171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:30.903770 32743 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9ae94e8fb1e743b6a91213360360ff6d"
dest_uuid: "da3c6ff682704489a1fbaa0aadae6171"
mode: GRACEFUL
new_leader_uuid: "00d7cee1aea54375980b10dbdf4c4b79"
 from {username='slave'} at 127.0.0.1:50318
I20260709 13:12:30.911046 32743 raft_consensus.cc:606] T 9ae94e8fb1e743b6a91213360360ff6d P da3c6ff682704489a1fbaa0aadae6171 [term 1 LEADER]: Received request to transfer leadership to TS 00d7cee1aea54375980b10dbdf4c4b79
I20260709 13:12:30.915768   324 consensus_queue.cc:1048] T 9ae94e8fb1e743b6a91213360360ff6d P da3c6ff682704489a1fbaa0aadae6171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:30.922746 32743 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8c5a000256b54521b6c4e697a81b2b4f"
dest_uuid: "da3c6ff682704489a1fbaa0aadae6171"
mode: GRACEFUL
new_leader_uuid: "bbe610f3a51e4732a48c941905e222f1"
 from {username='slave'} at 127.0.0.1:50318
I20260709 13:12:30.923353 32743 raft_consensus.cc:606] T 8c5a000256b54521b6c4e697a81b2b4f P da3c6ff682704489a1fbaa0aadae6171 [term 1 LEADER]: Received request to transfer leadership to TS bbe610f3a51e4732a48c941905e222f1
I20260709 13:12:30.975019 32743 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b9eb1f9645fa430d9a24dca067540bbd"
dest_uuid: "da3c6ff682704489a1fbaa0aadae6171"
mode: GRACEFUL
new_leader_uuid: "00d7cee1aea54375980b10dbdf4c4b79"
 from {username='slave'} at 127.0.0.1:50318
I20260709 13:12:30.976997 32743 raft_consensus.cc:606] T b9eb1f9645fa430d9a24dca067540bbd P da3c6ff682704489a1fbaa0aadae6171 [term 1 LEADER]: Received request to transfer leadership to TS 00d7cee1aea54375980b10dbdf4c4b79
I20260709 13:12:31.004680   353 consensus_queue.cc:1048] T 8c5a000256b54521b6c4e697a81b2b4f P da3c6ff682704489a1fbaa0aadae6171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:31.023322   324 consensus_queue.cc:1048] T 8c5a000256b54521b6c4e697a81b2b4f P da3c6ff682704489a1fbaa0aadae6171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:31.038340   324 raft_consensus.cc:993] T 8c5a000256b54521b6c4e697a81b2b4f P da3c6ff682704489a1fbaa0aadae6171: : Instructing follower bbe610f3a51e4732a48c941905e222f1 to start an election
I20260709 13:12:31.038985   358 raft_consensus.cc:1081] T 8c5a000256b54521b6c4e697a81b2b4f P da3c6ff682704489a1fbaa0aadae6171 [term 1 LEADER]: Signalling peer bbe610f3a51e4732a48c941905e222f1 to start an election
I20260709 13:12:31.041018 32669 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8c5a000256b54521b6c4e697a81b2b4f"
dest_uuid: "bbe610f3a51e4732a48c941905e222f1"
 from {username='slave'} at 127.0.0.1:51056
I20260709 13:12:31.041596 32669 raft_consensus.cc:493] T 8c5a000256b54521b6c4e697a81b2b4f P bbe610f3a51e4732a48c941905e222f1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:31.042063 32669 raft_consensus.cc:3060] T 8c5a000256b54521b6c4e697a81b2b4f P bbe610f3a51e4732a48c941905e222f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:31.052225 32669 raft_consensus.cc:515] T 8c5a000256b54521b6c4e697a81b2b4f P bbe610f3a51e4732a48c941905e222f1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } } peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } }
I20260709 13:12:31.055254 32669 leader_election.cc:290] T 8c5a000256b54521b6c4e697a81b2b4f P bbe610f3a51e4732a48c941905e222f1 [CANDIDATE]: Term 2 election: Requested vote from peers 00d7cee1aea54375980b10dbdf4c4b79 (127.21.63.1:34359), da3c6ff682704489a1fbaa0aadae6171 (127.21.63.3:37001)
I20260709 13:12:31.070518 32594 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c5a000256b54521b6c4e697a81b2b4f" candidate_uuid: "bbe610f3a51e4732a48c941905e222f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "00d7cee1aea54375980b10dbdf4c4b79"
I20260709 13:12:31.071110 32594 raft_consensus.cc:3060] T 8c5a000256b54521b6c4e697a81b2b4f P 00d7cee1aea54375980b10dbdf4c4b79 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:31.072502 32743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c5a000256b54521b6c4e697a81b2b4f" candidate_uuid: "bbe610f3a51e4732a48c941905e222f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "da3c6ff682704489a1fbaa0aadae6171"
I20260709 13:12:31.073318 32743 raft_consensus.cc:3055] T 8c5a000256b54521b6c4e697a81b2b4f P da3c6ff682704489a1fbaa0aadae6171 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:31.073664 32743 raft_consensus.cc:740] T 8c5a000256b54521b6c4e697a81b2b4f P da3c6ff682704489a1fbaa0aadae6171 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: da3c6ff682704489a1fbaa0aadae6171, State: Running, Role: LEADER
I20260709 13:12:31.074929 32743 consensus_queue.cc:260] T 8c5a000256b54521b6c4e697a81b2b4f P da3c6ff682704489a1fbaa0aadae6171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } } peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } }
I20260709 13:12:31.076530 32743 raft_consensus.cc:3060] T 8c5a000256b54521b6c4e697a81b2b4f P da3c6ff682704489a1fbaa0aadae6171 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:31.084010 32594 raft_consensus.cc:2468] T 8c5a000256b54521b6c4e697a81b2b4f P 00d7cee1aea54375980b10dbdf4c4b79 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbe610f3a51e4732a48c941905e222f1 in term 2.
I20260709 13:12:31.085354 32632 leader_election.cc:304] T 8c5a000256b54521b6c4e697a81b2b4f P bbe610f3a51e4732a48c941905e222f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00d7cee1aea54375980b10dbdf4c4b79, bbe610f3a51e4732a48c941905e222f1; no voters: 
I20260709 13:12:31.086314   321 raft_consensus.cc:2804] T 8c5a000256b54521b6c4e697a81b2b4f P bbe610f3a51e4732a48c941905e222f1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:31.087507   321 raft_consensus.cc:697] T 8c5a000256b54521b6c4e697a81b2b4f P bbe610f3a51e4732a48c941905e222f1 [term 2 LEADER]: Becoming Leader. State: Replica: bbe610f3a51e4732a48c941905e222f1, State: Running, Role: LEADER
I20260709 13:12:31.087504 32743 raft_consensus.cc:2468] T 8c5a000256b54521b6c4e697a81b2b4f P da3c6ff682704489a1fbaa0aadae6171 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbe610f3a51e4732a48c941905e222f1 in term 2.
I20260709 13:12:31.088896   321 consensus_queue.cc:237] T 8c5a000256b54521b6c4e697a81b2b4f P bbe610f3a51e4732a48c941905e222f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } } peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } }
I20260709 13:12:31.095846 32490 catalog_manager.cc:5697] T 8c5a000256b54521b6c4e697a81b2b4f P bbe610f3a51e4732a48c941905e222f1 reported cstate change: term changed from 1 to 2, leader changed from da3c6ff682704489a1fbaa0aadae6171 (127.21.63.3) to bbe610f3a51e4732a48c941905e222f1 (127.21.63.2). New cstate: current_term: 2 leader_uuid: "bbe610f3a51e4732a48c941905e222f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:31.165820   358 consensus_queue.cc:1048] T b9eb1f9645fa430d9a24dca067540bbd P da3c6ff682704489a1fbaa0aadae6171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:31.172751   324 raft_consensus.cc:993] T b9eb1f9645fa430d9a24dca067540bbd P da3c6ff682704489a1fbaa0aadae6171: : Instructing follower 00d7cee1aea54375980b10dbdf4c4b79 to start an election
I20260709 13:12:31.173314   353 raft_consensus.cc:1081] T b9eb1f9645fa430d9a24dca067540bbd P da3c6ff682704489a1fbaa0aadae6171 [term 1 LEADER]: Signalling peer 00d7cee1aea54375980b10dbdf4c4b79 to start an election
I20260709 13:12:31.175437 32593 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b9eb1f9645fa430d9a24dca067540bbd"
dest_uuid: "00d7cee1aea54375980b10dbdf4c4b79"
 from {username='slave'} at 127.0.0.1:47142
I20260709 13:12:31.179093 32593 raft_consensus.cc:493] T b9eb1f9645fa430d9a24dca067540bbd P 00d7cee1aea54375980b10dbdf4c4b79 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:31.179116   353 consensus_queue.cc:1048] T b9eb1f9645fa430d9a24dca067540bbd P da3c6ff682704489a1fbaa0aadae6171 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:31.179605 32593 raft_consensus.cc:3060] T b9eb1f9645fa430d9a24dca067540bbd P 00d7cee1aea54375980b10dbdf4c4b79 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:31.190106 32593 raft_consensus.cc:515] T b9eb1f9645fa430d9a24dca067540bbd P 00d7cee1aea54375980b10dbdf4c4b79 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } } peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } }
I20260709 13:12:31.196033 32593 leader_election.cc:290] T b9eb1f9645fa430d9a24dca067540bbd P 00d7cee1aea54375980b10dbdf4c4b79 [CANDIDATE]: Term 2 election: Requested vote from peers bbe610f3a51e4732a48c941905e222f1 (127.21.63.2:40017), da3c6ff682704489a1fbaa0aadae6171 (127.21.63.3:37001)
I20260709 13:12:31.197494 32743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9eb1f9645fa430d9a24dca067540bbd" candidate_uuid: "00d7cee1aea54375980b10dbdf4c4b79" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "da3c6ff682704489a1fbaa0aadae6171"
I20260709 13:12:31.198261 32743 raft_consensus.cc:3055] T b9eb1f9645fa430d9a24dca067540bbd P da3c6ff682704489a1fbaa0aadae6171 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:31.198621 32743 raft_consensus.cc:740] T b9eb1f9645fa430d9a24dca067540bbd P da3c6ff682704489a1fbaa0aadae6171 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: da3c6ff682704489a1fbaa0aadae6171, State: Running, Role: LEADER
I20260709 13:12:31.199635 32743 consensus_queue.cc:260] T b9eb1f9645fa430d9a24dca067540bbd P da3c6ff682704489a1fbaa0aadae6171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } } peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } }
I20260709 13:12:31.201159 32743 raft_consensus.cc:3060] T b9eb1f9645fa430d9a24dca067540bbd P da3c6ff682704489a1fbaa0aadae6171 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:31.201898 32669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9eb1f9645fa430d9a24dca067540bbd" candidate_uuid: "00d7cee1aea54375980b10dbdf4c4b79" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bbe610f3a51e4732a48c941905e222f1"
I20260709 13:12:31.202540 32669 raft_consensus.cc:3060] T b9eb1f9645fa430d9a24dca067540bbd P bbe610f3a51e4732a48c941905e222f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:31.210175 32743 raft_consensus.cc:2468] T b9eb1f9645fa430d9a24dca067540bbd P da3c6ff682704489a1fbaa0aadae6171 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00d7cee1aea54375980b10dbdf4c4b79 in term 2.
I20260709 13:12:31.211668 32557 leader_election.cc:304] T b9eb1f9645fa430d9a24dca067540bbd P 00d7cee1aea54375980b10dbdf4c4b79 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00d7cee1aea54375980b10dbdf4c4b79, da3c6ff682704489a1fbaa0aadae6171; no voters: 
I20260709 13:12:31.211673 32669 raft_consensus.cc:2468] T b9eb1f9645fa430d9a24dca067540bbd P bbe610f3a51e4732a48c941905e222f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00d7cee1aea54375980b10dbdf4c4b79 in term 2.
I20260709 13:12:31.212898   322 raft_consensus.cc:2804] T b9eb1f9645fa430d9a24dca067540bbd P 00d7cee1aea54375980b10dbdf4c4b79 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:31.213305   322 raft_consensus.cc:697] T b9eb1f9645fa430d9a24dca067540bbd P 00d7cee1aea54375980b10dbdf4c4b79 [term 2 LEADER]: Becoming Leader. State: Replica: 00d7cee1aea54375980b10dbdf4c4b79, State: Running, Role: LEADER
I20260709 13:12:31.214286   322 consensus_queue.cc:237] T b9eb1f9645fa430d9a24dca067540bbd P 00d7cee1aea54375980b10dbdf4c4b79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } } peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } }
I20260709 13:12:31.222011 32492 catalog_manager.cc:5697] T b9eb1f9645fa430d9a24dca067540bbd P 00d7cee1aea54375980b10dbdf4c4b79 reported cstate change: term changed from 1 to 2, leader changed from da3c6ff682704489a1fbaa0aadae6171 (127.21.63.3) to 00d7cee1aea54375980b10dbdf4c4b79 (127.21.63.1). New cstate: current_term: 2 leader_uuid: "00d7cee1aea54375980b10dbdf4c4b79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:31.329145   322 consensus_queue.cc:1048] T 986d36f58fbc438aa9ae156bdb87e69c P 00d7cee1aea54375980b10dbdf4c4b79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:31.342289   322 consensus_queue.cc:1048] T 986d36f58fbc438aa9ae156bdb87e69c P 00d7cee1aea54375980b10dbdf4c4b79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:31.422813   358 raft_consensus.cc:993] T 9ae94e8fb1e743b6a91213360360ff6d P da3c6ff682704489a1fbaa0aadae6171: : Instructing follower 00d7cee1aea54375980b10dbdf4c4b79 to start an election
I20260709 13:12:31.423357   353 raft_consensus.cc:1081] T 9ae94e8fb1e743b6a91213360360ff6d P da3c6ff682704489a1fbaa0aadae6171 [term 1 LEADER]: Signalling peer 00d7cee1aea54375980b10dbdf4c4b79 to start an election
I20260709 13:12:31.424984 32593 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9ae94e8fb1e743b6a91213360360ff6d"
dest_uuid: "00d7cee1aea54375980b10dbdf4c4b79"
 from {username='slave'} at 127.0.0.1:47142
I20260709 13:12:31.425438 32593 raft_consensus.cc:493] T 9ae94e8fb1e743b6a91213360360ff6d P 00d7cee1aea54375980b10dbdf4c4b79 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:31.425714 32593 raft_consensus.cc:3060] T 9ae94e8fb1e743b6a91213360360ff6d P 00d7cee1aea54375980b10dbdf4c4b79 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:31.430975 32593 raft_consensus.cc:515] T 9ae94e8fb1e743b6a91213360360ff6d P 00d7cee1aea54375980b10dbdf4c4b79 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } } peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } }
I20260709 13:12:31.432631 32593 leader_election.cc:290] T 9ae94e8fb1e743b6a91213360360ff6d P 00d7cee1aea54375980b10dbdf4c4b79 [CANDIDATE]: Term 2 election: Requested vote from peers da3c6ff682704489a1fbaa0aadae6171 (127.21.63.3:37001), bbe610f3a51e4732a48c941905e222f1 (127.21.63.2:40017)
I20260709 13:12:31.433471 32743 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae94e8fb1e743b6a91213360360ff6d" candidate_uuid: "00d7cee1aea54375980b10dbdf4c4b79" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "da3c6ff682704489a1fbaa0aadae6171"
I20260709 13:12:31.433827 32669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9ae94e8fb1e743b6a91213360360ff6d" candidate_uuid: "00d7cee1aea54375980b10dbdf4c4b79" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bbe610f3a51e4732a48c941905e222f1"
I20260709 13:12:31.434108 32743 raft_consensus.cc:3055] T 9ae94e8fb1e743b6a91213360360ff6d P da3c6ff682704489a1fbaa0aadae6171 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:31.434393 32669 raft_consensus.cc:3060] T 9ae94e8fb1e743b6a91213360360ff6d P bbe610f3a51e4732a48c941905e222f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:31.434526 32743 raft_consensus.cc:740] T 9ae94e8fb1e743b6a91213360360ff6d P da3c6ff682704489a1fbaa0aadae6171 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: da3c6ff682704489a1fbaa0aadae6171, State: Running, Role: LEADER
I20260709 13:12:31.435456 32743 consensus_queue.cc:260] T 9ae94e8fb1e743b6a91213360360ff6d P da3c6ff682704489a1fbaa0aadae6171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } } peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } }
I20260709 13:12:31.436993 32743 raft_consensus.cc:3060] T 9ae94e8fb1e743b6a91213360360ff6d P da3c6ff682704489a1fbaa0aadae6171 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:31.440994 32669 raft_consensus.cc:2468] T 9ae94e8fb1e743b6a91213360360ff6d P bbe610f3a51e4732a48c941905e222f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00d7cee1aea54375980b10dbdf4c4b79 in term 2.
I20260709 13:12:31.442353 32557 leader_election.cc:304] T 9ae94e8fb1e743b6a91213360360ff6d P 00d7cee1aea54375980b10dbdf4c4b79 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00d7cee1aea54375980b10dbdf4c4b79, bbe610f3a51e4732a48c941905e222f1; no voters: 
I20260709 13:12:31.443221   322 raft_consensus.cc:2804] T 9ae94e8fb1e743b6a91213360360ff6d P 00d7cee1aea54375980b10dbdf4c4b79 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:31.443709   322 raft_consensus.cc:697] T 9ae94e8fb1e743b6a91213360360ff6d P 00d7cee1aea54375980b10dbdf4c4b79 [term 2 LEADER]: Becoming Leader. State: Replica: 00d7cee1aea54375980b10dbdf4c4b79, State: Running, Role: LEADER
I20260709 13:12:31.444100 32743 raft_consensus.cc:2468] T 9ae94e8fb1e743b6a91213360360ff6d P da3c6ff682704489a1fbaa0aadae6171 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 00d7cee1aea54375980b10dbdf4c4b79 in term 2.
I20260709 13:12:31.444947   322 consensus_queue.cc:237] T 9ae94e8fb1e743b6a91213360360ff6d P 00d7cee1aea54375980b10dbdf4c4b79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } } peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } }
I20260709 13:12:31.452257 32492 catalog_manager.cc:5697] T 9ae94e8fb1e743b6a91213360360ff6d P 00d7cee1aea54375980b10dbdf4c4b79 reported cstate change: term changed from 1 to 2, leader changed from da3c6ff682704489a1fbaa0aadae6171 (127.21.63.3) to 00d7cee1aea54375980b10dbdf4c4b79 (127.21.63.1). New cstate: current_term: 2 leader_uuid: "00d7cee1aea54375980b10dbdf4c4b79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:31.526865 32743 raft_consensus.cc:1275] T 8c5a000256b54521b6c4e697a81b2b4f P da3c6ff682704489a1fbaa0aadae6171 [term 2 FOLLOWER]: Refusing update from remote peer bbe610f3a51e4732a48c941905e222f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:31.528252   321 consensus_queue.cc:1048] T 8c5a000256b54521b6c4e697a81b2b4f P bbe610f3a51e4732a48c941905e222f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:31.536602 32593 raft_consensus.cc:1275] T 8c5a000256b54521b6c4e697a81b2b4f P 00d7cee1aea54375980b10dbdf4c4b79 [term 2 FOLLOWER]: Refusing update from remote peer bbe610f3a51e4732a48c941905e222f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:31.538595   363 consensus_queue.cc:1048] T 8c5a000256b54521b6c4e697a81b2b4f P bbe610f3a51e4732a48c941905e222f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:31.706808 32743 raft_consensus.cc:1275] T b9eb1f9645fa430d9a24dca067540bbd P da3c6ff682704489a1fbaa0aadae6171 [term 2 FOLLOWER]: Refusing update from remote peer 00d7cee1aea54375980b10dbdf4c4b79: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:31.708297   322 consensus_queue.cc:1048] T b9eb1f9645fa430d9a24dca067540bbd P 00d7cee1aea54375980b10dbdf4c4b79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:31.717005 32669 raft_consensus.cc:1275] T b9eb1f9645fa430d9a24dca067540bbd P bbe610f3a51e4732a48c941905e222f1 [term 2 FOLLOWER]: Refusing update from remote peer 00d7cee1aea54375980b10dbdf4c4b79: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:31.718413   347 consensus_queue.cc:1048] T b9eb1f9645fa430d9a24dca067540bbd P 00d7cee1aea54375980b10dbdf4c4b79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:31.743718 21756 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 13:12:31.745237 21756 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 13:12:31.745602 21756 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 13:12:31.758517 32593 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "986d36f58fbc438aa9ae156bdb87e69c"
dest_uuid: "00d7cee1aea54375980b10dbdf4c4b79"
mode: GRACEFUL
new_leader_uuid: "da3c6ff682704489a1fbaa0aadae6171"
 from {username='slave'} at 127.0.0.1:47160
I20260709 13:12:31.759075 32593 raft_consensus.cc:606] T 986d36f58fbc438aa9ae156bdb87e69c P 00d7cee1aea54375980b10dbdf4c4b79 [term 1 LEADER]: Received request to transfer leadership to TS da3c6ff682704489a1fbaa0aadae6171
I20260709 13:12:31.760278 21756 auto_leader_rebalancer.cc:452] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 13:12:31.916056 32669 raft_consensus.cc:1275] T 9ae94e8fb1e743b6a91213360360ff6d P bbe610f3a51e4732a48c941905e222f1 [term 2 FOLLOWER]: Refusing update from remote peer 00d7cee1aea54375980b10dbdf4c4b79: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:31.917730   369 consensus_queue.cc:1048] T 9ae94e8fb1e743b6a91213360360ff6d P 00d7cee1aea54375980b10dbdf4c4b79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:31.926815 32743 raft_consensus.cc:1275] T 9ae94e8fb1e743b6a91213360360ff6d P da3c6ff682704489a1fbaa0aadae6171 [term 2 FOLLOWER]: Refusing update from remote peer 00d7cee1aea54375980b10dbdf4c4b79: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:31.928218   373 consensus_queue.cc:1048] T 9ae94e8fb1e743b6a91213360360ff6d P 00d7cee1aea54375980b10dbdf4c4b79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:31.961112   347 raft_consensus.cc:993] T 986d36f58fbc438aa9ae156bdb87e69c P 00d7cee1aea54375980b10dbdf4c4b79: : Instructing follower da3c6ff682704489a1fbaa0aadae6171 to start an election
I20260709 13:12:31.961778   370 raft_consensus.cc:1081] T 986d36f58fbc438aa9ae156bdb87e69c P 00d7cee1aea54375980b10dbdf4c4b79 [term 1 LEADER]: Signalling peer da3c6ff682704489a1fbaa0aadae6171 to start an election
I20260709 13:12:31.963496 32743 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "986d36f58fbc438aa9ae156bdb87e69c"
dest_uuid: "da3c6ff682704489a1fbaa0aadae6171"
 from {username='slave'} at 127.0.0.1:50302
I20260709 13:12:31.964124 32743 raft_consensus.cc:493] T 986d36f58fbc438aa9ae156bdb87e69c P da3c6ff682704489a1fbaa0aadae6171 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:31.964471 32743 raft_consensus.cc:3060] T 986d36f58fbc438aa9ae156bdb87e69c P da3c6ff682704489a1fbaa0aadae6171 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:31.970098 32743 raft_consensus.cc:515] T 986d36f58fbc438aa9ae156bdb87e69c P da3c6ff682704489a1fbaa0aadae6171 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } } peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } }
I20260709 13:12:31.971853 32743 leader_election.cc:290] T 986d36f58fbc438aa9ae156bdb87e69c P da3c6ff682704489a1fbaa0aadae6171 [CANDIDATE]: Term 2 election: Requested vote from peers 00d7cee1aea54375980b10dbdf4c4b79 (127.21.63.1:34359), bbe610f3a51e4732a48c941905e222f1 (127.21.63.2:40017)
I20260709 13:12:31.972977 32669 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d36f58fbc438aa9ae156bdb87e69c" candidate_uuid: "da3c6ff682704489a1fbaa0aadae6171" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bbe610f3a51e4732a48c941905e222f1"
I20260709 13:12:31.973183 32593 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "986d36f58fbc438aa9ae156bdb87e69c" candidate_uuid: "da3c6ff682704489a1fbaa0aadae6171" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "00d7cee1aea54375980b10dbdf4c4b79"
I20260709 13:12:31.973590 32669 raft_consensus.cc:3060] T 986d36f58fbc438aa9ae156bdb87e69c P bbe610f3a51e4732a48c941905e222f1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:31.973878 32593 raft_consensus.cc:3055] T 986d36f58fbc438aa9ae156bdb87e69c P 00d7cee1aea54375980b10dbdf4c4b79 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:31.974305 32593 raft_consensus.cc:740] T 986d36f58fbc438aa9ae156bdb87e69c P 00d7cee1aea54375980b10dbdf4c4b79 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 00d7cee1aea54375980b10dbdf4c4b79, State: Running, Role: LEADER
I20260709 13:12:31.975245 32593 consensus_queue.cc:260] T 986d36f58fbc438aa9ae156bdb87e69c P 00d7cee1aea54375980b10dbdf4c4b79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } } peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } }
I20260709 13:12:31.976590 32593 raft_consensus.cc:3060] T 986d36f58fbc438aa9ae156bdb87e69c P 00d7cee1aea54375980b10dbdf4c4b79 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:31.979367 32669 raft_consensus.cc:2468] T 986d36f58fbc438aa9ae156bdb87e69c P bbe610f3a51e4732a48c941905e222f1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3c6ff682704489a1fbaa0aadae6171 in term 2.
I20260709 13:12:31.980558 32706 leader_election.cc:304] T 986d36f58fbc438aa9ae156bdb87e69c P da3c6ff682704489a1fbaa0aadae6171 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bbe610f3a51e4732a48c941905e222f1, da3c6ff682704489a1fbaa0aadae6171; no voters: 
I20260709 13:12:31.981419   353 raft_consensus.cc:2804] T 986d36f58fbc438aa9ae156bdb87e69c P da3c6ff682704489a1fbaa0aadae6171 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:31.982479   353 raft_consensus.cc:697] T 986d36f58fbc438aa9ae156bdb87e69c P da3c6ff682704489a1fbaa0aadae6171 [term 2 LEADER]: Becoming Leader. State: Replica: da3c6ff682704489a1fbaa0aadae6171, State: Running, Role: LEADER
I20260709 13:12:31.983366 32593 raft_consensus.cc:2468] T 986d36f58fbc438aa9ae156bdb87e69c P 00d7cee1aea54375980b10dbdf4c4b79 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da3c6ff682704489a1fbaa0aadae6171 in term 2.
I20260709 13:12:31.983629   353 consensus_queue.cc:237] T 986d36f58fbc438aa9ae156bdb87e69c P da3c6ff682704489a1fbaa0aadae6171 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } } peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } }
I20260709 13:12:31.991292 32491 catalog_manager.cc:5697] T 986d36f58fbc438aa9ae156bdb87e69c P da3c6ff682704489a1fbaa0aadae6171 reported cstate change: term changed from 1 to 2, leader changed from 00d7cee1aea54375980b10dbdf4c4b79 (127.21.63.1) to da3c6ff682704489a1fbaa0aadae6171 (127.21.63.3). New cstate: current_term: 2 leader_uuid: "da3c6ff682704489a1fbaa0aadae6171" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00d7cee1aea54375980b10dbdf4c4b79" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbe610f3a51e4732a48c941905e222f1" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 40017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "da3c6ff682704489a1fbaa0aadae6171" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 37001 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:32.115355 21756 tablet_server.cc:179] TabletServer@127.21.63.1:0 shutting down...
I20260709 13:12:32.141793 21756 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:12:32.142729 21756 tablet_replica.cc:333] T b9eb1f9645fa430d9a24dca067540bbd P 00d7cee1aea54375980b10dbdf4c4b79: stopping tablet replica
I20260709 13:12:32.143503 21756 raft_consensus.cc:2243] T b9eb1f9645fa430d9a24dca067540bbd P 00d7cee1aea54375980b10dbdf4c4b79 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:32.144364 21756 raft_consensus.cc:2272] T b9eb1f9645fa430d9a24dca067540bbd P 00d7cee1aea54375980b10dbdf4c4b79 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:32.147562 21756 tablet_replica.cc:333] T 986d36f58fbc438aa9ae156bdb87e69c P 00d7cee1aea54375980b10dbdf4c4b79: stopping tablet replica
I20260709 13:12:32.148198 21756 raft_consensus.cc:2243] T 986d36f58fbc438aa9ae156bdb87e69c P 00d7cee1aea54375980b10dbdf4c4b79 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:32.148777 21756 raft_consensus.cc:2272] T 986d36f58fbc438aa9ae156bdb87e69c P 00d7cee1aea54375980b10dbdf4c4b79 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:32.151866 21756 tablet_replica.cc:333] T 9ae94e8fb1e743b6a91213360360ff6d P 00d7cee1aea54375980b10dbdf4c4b79: stopping tablet replica
I20260709 13:12:32.152395 21756 raft_consensus.cc:2243] T 9ae94e8fb1e743b6a91213360360ff6d P 00d7cee1aea54375980b10dbdf4c4b79 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:32.153306 21756 raft_consensus.cc:2272] T 9ae94e8fb1e743b6a91213360360ff6d P 00d7cee1aea54375980b10dbdf4c4b79 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:32.155902 21756 tablet_replica.cc:333] T 8c5a000256b54521b6c4e697a81b2b4f P 00d7cee1aea54375980b10dbdf4c4b79: stopping tablet replica
I20260709 13:12:32.156407 21756 raft_consensus.cc:2243] T 8c5a000256b54521b6c4e697a81b2b4f P 00d7cee1aea54375980b10dbdf4c4b79 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:32.156885 21756 raft_consensus.cc:2272] T 8c5a000256b54521b6c4e697a81b2b4f P 00d7cee1aea54375980b10dbdf4c4b79 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:32.183732 21756 tablet_server.cc:196] TabletServer@127.21.63.1:0 shutdown complete.
I20260709 13:12:32.200968 21756 tablet_server.cc:179] TabletServer@127.21.63.2:0 shutting down...
I20260709 13:12:32.225131 21756 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:12:32.225909 21756 tablet_replica.cc:333] T b9eb1f9645fa430d9a24dca067540bbd P bbe610f3a51e4732a48c941905e222f1: stopping tablet replica
I20260709 13:12:32.226588 21756 raft_consensus.cc:2243] T b9eb1f9645fa430d9a24dca067540bbd P bbe610f3a51e4732a48c941905e222f1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:32.227063 21756 raft_consensus.cc:2272] T b9eb1f9645fa430d9a24dca067540bbd P bbe610f3a51e4732a48c941905e222f1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:32.229943 21756 tablet_replica.cc:333] T 986d36f58fbc438aa9ae156bdb87e69c P bbe610f3a51e4732a48c941905e222f1: stopping tablet replica
I20260709 13:12:32.230511 21756 raft_consensus.cc:2243] T 986d36f58fbc438aa9ae156bdb87e69c P bbe610f3a51e4732a48c941905e222f1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:32.230945 21756 raft_consensus.cc:2272] T 986d36f58fbc438aa9ae156bdb87e69c P bbe610f3a51e4732a48c941905e222f1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:32.233754 21756 tablet_replica.cc:333] T 8c5a000256b54521b6c4e697a81b2b4f P bbe610f3a51e4732a48c941905e222f1: stopping tablet replica
I20260709 13:12:32.234268 21756 raft_consensus.cc:2243] T 8c5a000256b54521b6c4e697a81b2b4f P bbe610f3a51e4732a48c941905e222f1 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:32.235018 21756 raft_consensus.cc:2272] T 8c5a000256b54521b6c4e697a81b2b4f P bbe610f3a51e4732a48c941905e222f1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:32.237622 21756 tablet_replica.cc:333] T 9ae94e8fb1e743b6a91213360360ff6d P bbe610f3a51e4732a48c941905e222f1: stopping tablet replica
I20260709 13:12:32.238142 21756 raft_consensus.cc:2243] T 9ae94e8fb1e743b6a91213360360ff6d P bbe610f3a51e4732a48c941905e222f1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:32.238639 21756 raft_consensus.cc:2272] T 9ae94e8fb1e743b6a91213360360ff6d P bbe610f3a51e4732a48c941905e222f1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:32.264333 21756 tablet_server.cc:196] TabletServer@127.21.63.2:0 shutdown complete.
I20260709 13:12:32.282187 21756 tablet_server.cc:179] TabletServer@127.21.63.3:0 shutting down...
I20260709 13:12:32.304288 21756 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:12:32.305075 21756 tablet_replica.cc:333] T b9eb1f9645fa430d9a24dca067540bbd P da3c6ff682704489a1fbaa0aadae6171: stopping tablet replica
I20260709 13:12:32.305670 21756 raft_consensus.cc:2243] T b9eb1f9645fa430d9a24dca067540bbd P da3c6ff682704489a1fbaa0aadae6171 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:32.306157 21756 raft_consensus.cc:2272] T b9eb1f9645fa430d9a24dca067540bbd P da3c6ff682704489a1fbaa0aadae6171 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:32.308912 21756 tablet_replica.cc:333] T 986d36f58fbc438aa9ae156bdb87e69c P da3c6ff682704489a1fbaa0aadae6171: stopping tablet replica
I20260709 13:12:32.309528 21756 raft_consensus.cc:2243] T 986d36f58fbc438aa9ae156bdb87e69c P da3c6ff682704489a1fbaa0aadae6171 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:32.310161 21756 pending_rounds.cc:70] T 986d36f58fbc438aa9ae156bdb87e69c P da3c6ff682704489a1fbaa0aadae6171: Trying to abort 1 pending ops.
I20260709 13:12:32.310355 21756 pending_rounds.cc:77] T 986d36f58fbc438aa9ae156bdb87e69c P da3c6ff682704489a1fbaa0aadae6171: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305636872135905280 op_type: NO_OP noop_request { }
I20260709 13:12:32.310642 21756 raft_consensus.cc:2889] T 986d36f58fbc438aa9ae156bdb87e69c P da3c6ff682704489a1fbaa0aadae6171 [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 13:12:32.310921 21756 raft_consensus.cc:2272] T 986d36f58fbc438aa9ae156bdb87e69c P da3c6ff682704489a1fbaa0aadae6171 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:32.313740 21756 tablet_replica.cc:333] T 8c5a000256b54521b6c4e697a81b2b4f P da3c6ff682704489a1fbaa0aadae6171: stopping tablet replica
I20260709 13:12:32.314250 21756 raft_consensus.cc:2243] T 8c5a000256b54521b6c4e697a81b2b4f P da3c6ff682704489a1fbaa0aadae6171 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:32.314677 21756 raft_consensus.cc:2272] T 8c5a000256b54521b6c4e697a81b2b4f P da3c6ff682704489a1fbaa0aadae6171 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:32.317332 21756 tablet_replica.cc:333] T 9ae94e8fb1e743b6a91213360360ff6d P da3c6ff682704489a1fbaa0aadae6171: stopping tablet replica
I20260709 13:12:32.317819 21756 raft_consensus.cc:2243] T 9ae94e8fb1e743b6a91213360360ff6d P da3c6ff682704489a1fbaa0aadae6171 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:32.318229 21756 raft_consensus.cc:2272] T 9ae94e8fb1e743b6a91213360360ff6d P da3c6ff682704489a1fbaa0aadae6171 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:32.332374 21756 tablet_server.cc:196] TabletServer@127.21.63.3:0 shutdown complete.
I20260709 13:12:32.348294 21756 master.cc:562] Master@127.21.63.62:41781 shutting down...
I20260709 13:12:32.369511 21756 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 58083de319b6453c99a5d54e73a10f9d [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:32.370128 21756 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 58083de319b6453c99a5d54e73a10f9d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:32.370517 21756 tablet_replica.cc:333] T 00000000000000000000000000000000 P 58083de319b6453c99a5d54e73a10f9d: stopping tablet replica
I20260709 13:12:32.390218 21756 master.cc:584] Master@127.21.63.62:41781 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (2892 ms)
[----------] 6 tests from LeaderRebalancerTest (322836 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 13:12:32.428438 21756 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.63.62:35237
I20260709 13:12:32.429713 21756 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:12:32.435248   382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:32.435388   381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:32.436733   384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:32.438090 21756 server_base.cc:1061] running on GCE node
I20260709 13:12:32.438920 21756 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:12:32.439149 21756 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:12:32.439275 21756 hybrid_clock.cc:648] HybridClock initialized: now 1783602752439262 us; error 0 us; skew 500 ppm
I20260709 13:12:32.439740 21756 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:12:32.442225 21756 webserver.cc:533] Webserver started at http://127.21.63.62:35723/ using document root <none> and password file <none>
I20260709 13:12:32.442664 21756 fs_manager.cc:362] Metadata directory not provided
I20260709 13:12:32.442816 21756 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:12:32.443023 21756 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:12:32.444061 21756 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602429420501-21756-0/minicluster-data/master-0-root/instance:
uuid: "5a0083fc91ec45318745e6e9cd5b8bcf"
format_stamp: "Formatted at 2026-07-09 13:12:32 on dist-test-slave-c6mt"
I20260709 13:12:32.448261 21756 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.002s
I20260709 13:12:32.451625   389 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:32.452472 21756 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.000s	sys 0.003s
I20260709 13:12:32.452833 21756 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602429420501-21756-0/minicluster-data/master-0-root
uuid: "5a0083fc91ec45318745e6e9cd5b8bcf"
format_stamp: "Formatted at 2026-07-09 13:12:32 on dist-test-slave-c6mt"
I20260709 13:12:32.453193 21756 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602429420501-21756-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602429420501-21756-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602429420501-21756-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:12:32.479621 21756 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:12:32.481050 21756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:12:32.521252 21756 rpc_server.cc:307] RPC server started. Bound to: 127.21.63.62:35237
I20260709 13:12:32.521368   440 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.63.62:35237 every 8 connection(s)
I20260709 13:12:32.525588   441 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:32.538252   441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a0083fc91ec45318745e6e9cd5b8bcf: Bootstrap starting.
I20260709 13:12:32.543700   441 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a0083fc91ec45318745e6e9cd5b8bcf: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:32.548610   441 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a0083fc91ec45318745e6e9cd5b8bcf: No bootstrap required, opened a new log
I20260709 13:12:32.551030   441 raft_consensus.cc:359] T 00000000000000000000000000000000 P 5a0083fc91ec45318745e6e9cd5b8bcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a0083fc91ec45318745e6e9cd5b8bcf" member_type: VOTER }
I20260709 13:12:32.551430   441 raft_consensus.cc:385] T 00000000000000000000000000000000 P 5a0083fc91ec45318745e6e9cd5b8bcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:32.551714   441 raft_consensus.cc:740] T 00000000000000000000000000000000 P 5a0083fc91ec45318745e6e9cd5b8bcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a0083fc91ec45318745e6e9cd5b8bcf, State: Initialized, Role: FOLLOWER
I20260709 13:12:32.552464   441 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a0083fc91ec45318745e6e9cd5b8bcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a0083fc91ec45318745e6e9cd5b8bcf" member_type: VOTER }
I20260709 13:12:32.552981   441 raft_consensus.cc:399] T 00000000000000000000000000000000 P 5a0083fc91ec45318745e6e9cd5b8bcf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:12:32.553231   441 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5a0083fc91ec45318745e6e9cd5b8bcf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:12:32.553540   441 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5a0083fc91ec45318745e6e9cd5b8bcf [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:32.558914   441 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5a0083fc91ec45318745e6e9cd5b8bcf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a0083fc91ec45318745e6e9cd5b8bcf" member_type: VOTER }
I20260709 13:12:32.559491   441 leader_election.cc:304] T 00000000000000000000000000000000 P 5a0083fc91ec45318745e6e9cd5b8bcf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a0083fc91ec45318745e6e9cd5b8bcf; no voters: 
I20260709 13:12:32.560721   441 leader_election.cc:290] T 00000000000000000000000000000000 P 5a0083fc91ec45318745e6e9cd5b8bcf [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:12:32.561185   445 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 5a0083fc91ec45318745e6e9cd5b8bcf [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:32.562546   445 raft_consensus.cc:697] T 00000000000000000000000000000000 P 5a0083fc91ec45318745e6e9cd5b8bcf [term 1 LEADER]: Becoming Leader. State: Replica: 5a0083fc91ec45318745e6e9cd5b8bcf, State: Running, Role: LEADER
I20260709 13:12:32.563243   445 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a0083fc91ec45318745e6e9cd5b8bcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a0083fc91ec45318745e6e9cd5b8bcf" member_type: VOTER }
I20260709 13:12:32.563941   441 sys_catalog.cc:565] T 00000000000000000000000000000000 P 5a0083fc91ec45318745e6e9cd5b8bcf [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:12:32.566090   447 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a0083fc91ec45318745e6e9cd5b8bcf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a0083fc91ec45318745e6e9cd5b8bcf. Latest consensus state: current_term: 1 leader_uuid: "5a0083fc91ec45318745e6e9cd5b8bcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a0083fc91ec45318745e6e9cd5b8bcf" member_type: VOTER } }
I20260709 13:12:32.566098   446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a0083fc91ec45318745e6e9cd5b8bcf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a0083fc91ec45318745e6e9cd5b8bcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a0083fc91ec45318745e6e9cd5b8bcf" member_type: VOTER } }
I20260709 13:12:32.566862   447 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a0083fc91ec45318745e6e9cd5b8bcf [sys.catalog]: This master's current role is: LEADER
I20260709 13:12:32.566912   446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a0083fc91ec45318745e6e9cd5b8bcf [sys.catalog]: This master's current role is: LEADER
I20260709 13:12:32.571521   450 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:12:32.578393   450 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:12:32.579394 21756 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:12:32.587504   450 catalog_manager.cc:1383] Generated new cluster ID: f0fc510e1de041bdaf94b6523e6f240a
I20260709 13:12:32.587795   450 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:12:32.607360   450 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:12:32.608886   450 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:12:32.636795   450 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 5a0083fc91ec45318745e6e9cd5b8bcf: Generated new TSK 0
I20260709 13:12:32.637436   450 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:12:32.645740 21756 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:12:32.651836   463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:32.653180   464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:32.654384   466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:32.655670 21756 server_base.cc:1061] running on GCE node
I20260709 13:12:32.656534 21756 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:12:32.656802 21756 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:12:32.656980 21756 hybrid_clock.cc:648] HybridClock initialized: now 1783602752656944 us; error 0 us; skew 500 ppm
I20260709 13:12:32.657567 21756 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:12:32.659729 21756 webserver.cc:533] Webserver started at http://127.21.63.1:33669/ using document root <none> and password file <none>
I20260709 13:12:32.660259 21756 fs_manager.cc:362] Metadata directory not provided
I20260709 13:12:32.660487 21756 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:12:32.660794 21756 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:12:32.662014 21756 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602429420501-21756-0/minicluster-data/ts-0-root/instance:
uuid: "f35a78aeac02496d88fa800153754b0a"
format_stamp: "Formatted at 2026-07-09 13:12:32 on dist-test-slave-c6mt"
I20260709 13:12:32.666605 21756 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:12:32.669898   471 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:32.670754 21756 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:12:32.671115 21756 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602429420501-21756-0/minicluster-data/ts-0-root
uuid: "f35a78aeac02496d88fa800153754b0a"
format_stamp: "Formatted at 2026-07-09 13:12:32 on dist-test-slave-c6mt"
I20260709 13:12:32.671533 21756 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602429420501-21756-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602429420501-21756-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602429420501-21756-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:12:32.689122 21756 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:12:32.690289 21756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:12:32.691639 21756 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:12:32.693994 21756 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:12:32.694162 21756 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:32.694461 21756 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:12:32.694621 21756 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:32.737304 21756 rpc_server.cc:307] RPC server started. Bound to: 127.21.63.1:34881
I20260709 13:12:32.737411   533 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.63.1:34881 every 8 connection(s)
I20260709 13:12:32.742594 21756 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:12:32.749747   538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:32.749840   539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:32.753811   541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:32.753914 21756 server_base.cc:1061] running on GCE node
I20260709 13:12:32.755014   534 heartbeater.cc:344] Connected to a master server at 127.21.63.62:35237
I20260709 13:12:32.755175 21756 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:12:32.755421   534 heartbeater.cc:461] Registering TS with master...
W20260709 13:12:32.755506 21756 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:12:32.755885 21756 hybrid_clock.cc:648] HybridClock initialized: now 1783602752755868 us; error 0 us; skew 500 ppm
I20260709 13:12:32.756356   534 heartbeater.cc:507] Master 127.21.63.62:35237 requested a full tablet report, sending...
I20260709 13:12:32.756495 21756 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:12:32.758615   406 ts_manager.cc:194] Registered new tserver with Master: f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881)
I20260709 13:12:32.759289 21756 webserver.cc:533] Webserver started at http://127.21.63.2:34507/ using document root <none> and password file <none>
I20260709 13:12:32.759814 21756 fs_manager.cc:362] Metadata directory not provided
I20260709 13:12:32.760021 21756 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:12:32.760277 21756 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:12:32.760602   406 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49878
I20260709 13:12:32.761552 21756 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602429420501-21756-0/minicluster-data/ts-1-root/instance:
uuid: "15c6b88567e44da395f245725ffe7f04"
format_stamp: "Formatted at 2026-07-09 13:12:32 on dist-test-slave-c6mt"
I20260709 13:12:32.766324 21756 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 13:12:32.769812   546 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:32.770586 21756 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:12:32.770907 21756 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602429420501-21756-0/minicluster-data/ts-1-root
uuid: "15c6b88567e44da395f245725ffe7f04"
format_stamp: "Formatted at 2026-07-09 13:12:32 on dist-test-slave-c6mt"
I20260709 13:12:32.771214 21756 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602429420501-21756-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602429420501-21756-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602429420501-21756-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:12:32.794374 21756 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:12:32.795667 21756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:12:32.797355 21756 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:12:32.799685 21756 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:12:32.799933 21756 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:32.800197 21756 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:12:32.800391 21756 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:32.842090 21756 rpc_server.cc:307] RPC server started. Bound to: 127.21.63.2:38181
I20260709 13:12:32.842209   608 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.63.2:38181 every 8 connection(s)
I20260709 13:12:32.847561 21756 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:12:32.854969   612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:32.858083   613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:32.858139 21756 server_base.cc:1061] running on GCE node
W20260709 13:12:32.859877   615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:32.861354 21756 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:12:32.861636 21756 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:12:32.861845 21756 hybrid_clock.cc:648] HybridClock initialized: now 1783602752861803 us; error 0 us; skew 500 ppm
I20260709 13:12:32.862493 21756 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:12:32.862449   609 heartbeater.cc:344] Connected to a master server at 127.21.63.62:35237
I20260709 13:12:32.863124   609 heartbeater.cc:461] Registering TS with master...
I20260709 13:12:32.864061   609 heartbeater.cc:507] Master 127.21.63.62:35237 requested a full tablet report, sending...
I20260709 13:12:32.865510 21756 webserver.cc:533] Webserver started at http://127.21.63.3:41139/ using document root <none> and password file <none>
I20260709 13:12:32.866108 21756 fs_manager.cc:362] Metadata directory not provided
I20260709 13:12:32.866319 21756 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:12:32.866212   406 ts_manager.cc:194] Registered new tserver with Master: 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181)
I20260709 13:12:32.866766 21756 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:12:32.867853   406 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49884
I20260709 13:12:32.867933 21756 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602429420501-21756-0/minicluster-data/ts-2-root/instance:
uuid: "7d13b2cad49e4853831a427132e294d3"
format_stamp: "Formatted at 2026-07-09 13:12:32 on dist-test-slave-c6mt"
I20260709 13:12:32.872608 21756 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 13:12:32.876402   620 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:32.877326 21756 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:12:32.877658 21756 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602429420501-21756-0/minicluster-data/ts-2-root
uuid: "7d13b2cad49e4853831a427132e294d3"
format_stamp: "Formatted at 2026-07-09 13:12:32 on dist-test-slave-c6mt"
I20260709 13:12:32.877921 21756 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602429420501-21756-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602429420501-21756-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783602429420501-21756-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:12:32.900425 21756 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:12:32.901711 21756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:12:32.903358 21756 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:12:32.905738 21756 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:12:32.905972 21756 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:32.906232 21756 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:12:32.906412 21756 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:32.948474 21756 rpc_server.cc:307] RPC server started. Bound to: 127.21.63.3:42695
I20260709 13:12:32.948647   682 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.63.3:42695 every 8 connection(s)
I20260709 13:12:32.963014   683 heartbeater.cc:344] Connected to a master server at 127.21.63.62:35237
I20260709 13:12:32.963457   683 heartbeater.cc:461] Registering TS with master...
I20260709 13:12:32.964243   683 heartbeater.cc:507] Master 127.21.63.62:35237 requested a full tablet report, sending...
I20260709 13:12:32.966396   406 ts_manager.cc:194] Registered new tserver with Master: 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695)
I20260709 13:12:32.966853 21756 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014599391s
I20260709 13:12:32.967957   406 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49888
I20260709 13:12:32.994742   406 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49896:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:12:32.997830   406 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:12:33.086612   499 tablet_service.cc:1511] Processing CreateTablet for tablet a13e0fbeefb142f6bf766c1509cd1d93 (DEFAULT_TABLE table=test-workload [id=c5662d0cce42400296dbe5758ce8f702]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:12:33.088284   499 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a13e0fbeefb142f6bf766c1509cd1d93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:33.087968   496 tablet_service.cc:1511] Processing CreateTablet for tablet 93236792b8284277a935d6e569d4c96b (DEFAULT_TABLE table=test-workload [id=c5662d0cce42400296dbe5758ce8f702]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:12:33.090396   497 tablet_service.cc:1511] Processing CreateTablet for tablet e6a5dbc01970406599bc229548544689 (DEFAULT_TABLE table=test-workload [id=c5662d0cce42400296dbe5758ce8f702]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:12:33.091375   496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93236792b8284277a935d6e569d4c96b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:33.092077   497 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6a5dbc01970406599bc229548544689. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:33.090920   572 tablet_service.cc:1511] Processing CreateTablet for tablet e6a5dbc01970406599bc229548544689 (DEFAULT_TABLE table=test-workload [id=c5662d0cce42400296dbe5758ce8f702]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:12:33.093206   572 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6a5dbc01970406599bc229548544689. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:33.094002   574 tablet_service.cc:1511] Processing CreateTablet for tablet a13e0fbeefb142f6bf766c1509cd1d93 (DEFAULT_TABLE table=test-workload [id=c5662d0cce42400296dbe5758ce8f702]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:12:33.094678   570 tablet_service.cc:1511] Processing CreateTablet for tablet 6e5c1404cee54b96a9c3a8c024493223 (DEFAULT_TABLE table=test-workload [id=c5662d0cce42400296dbe5758ce8f702]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:12:33.095381   574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a13e0fbeefb142f6bf766c1509cd1d93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:33.096088   570 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e5c1404cee54b96a9c3a8c024493223. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:33.088469   498 tablet_service.cc:1511] Processing CreateTablet for tablet bf7b037540fc46d098bf7e1888e9c682 (DEFAULT_TABLE table=test-workload [id=c5662d0cce42400296dbe5758ce8f702]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:12:33.097496   498 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf7b037540fc46d098bf7e1888e9c682. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:33.097558   573 tablet_service.cc:1511] Processing CreateTablet for tablet bf7b037540fc46d098bf7e1888e9c682 (DEFAULT_TABLE table=test-workload [id=c5662d0cce42400296dbe5758ce8f702]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:12:33.099128   573 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf7b037540fc46d098bf7e1888e9c682. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:33.092765   571 tablet_service.cc:1511] Processing CreateTablet for tablet 93236792b8284277a935d6e569d4c96b (DEFAULT_TABLE table=test-workload [id=c5662d0cce42400296dbe5758ce8f702]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:12:33.089948   495 tablet_service.cc:1511] Processing CreateTablet for tablet 6e5c1404cee54b96a9c3a8c024493223 (DEFAULT_TABLE table=test-workload [id=c5662d0cce42400296dbe5758ce8f702]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:12:33.101482   571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93236792b8284277a935d6e569d4c96b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:33.101518   495 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e5c1404cee54b96a9c3a8c024493223. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:33.107780   494 tablet_service.cc:1511] Processing CreateTablet for tablet 3723f5ab8f974d1892ce711b33fac1dd (DEFAULT_TABLE table=test-workload [id=c5662d0cce42400296dbe5758ce8f702]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:12:33.109313   494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3723f5ab8f974d1892ce711b33fac1dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:33.113976   493 tablet_service.cc:1511] Processing CreateTablet for tablet 82ac3d8dfa714f66a589173f6a9eb004 (DEFAULT_TABLE table=test-workload [id=c5662d0cce42400296dbe5758ce8f702]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:12:33.114269   569 tablet_service.cc:1511] Processing CreateTablet for tablet 3723f5ab8f974d1892ce711b33fac1dd (DEFAULT_TABLE table=test-workload [id=c5662d0cce42400296dbe5758ce8f702]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:12:33.115410   493 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82ac3d8dfa714f66a589173f6a9eb004. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:33.115638   569 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3723f5ab8f974d1892ce711b33fac1dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:33.123631   568 tablet_service.cc:1511] Processing CreateTablet for tablet 82ac3d8dfa714f66a589173f6a9eb004 (DEFAULT_TABLE table=test-workload [id=c5662d0cce42400296dbe5758ce8f702]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:12:33.125419   568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82ac3d8dfa714f66a589173f6a9eb004. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:33.127192   492 tablet_service.cc:1511] Processing CreateTablet for tablet 6653d3e90150480daf2486b331be3130 (DEFAULT_TABLE table=test-workload [id=c5662d0cce42400296dbe5758ce8f702]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:12:33.128643   492 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6653d3e90150480daf2486b331be3130. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:33.129956   567 tablet_service.cc:1511] Processing CreateTablet for tablet 6653d3e90150480daf2486b331be3130 (DEFAULT_TABLE table=test-workload [id=c5662d0cce42400296dbe5758ce8f702]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:12:33.131441   567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6653d3e90150480daf2486b331be3130. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:33.133807   491 tablet_service.cc:1511] Processing CreateTablet for tablet 3b24ef3f665449c288b2fcd28c288d27 (DEFAULT_TABLE table=test-workload [id=c5662d0cce42400296dbe5758ce8f702]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:12:33.135660   491 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b24ef3f665449c288b2fcd28c288d27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:33.149909   699 tablet_bootstrap.cc:492] T bf7b037540fc46d098bf7e1888e9c682 P 15c6b88567e44da395f245725ffe7f04: Bootstrap starting.
I20260709 13:12:33.159375   573 tablet_service.cc:1511] Processing CreateTablet for tablet 3b24ef3f665449c288b2fcd28c288d27 (DEFAULT_TABLE table=test-workload [id=c5662d0cce42400296dbe5758ce8f702]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:12:33.160851   573 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b24ef3f665449c288b2fcd28c288d27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:33.194926   698 tablet_bootstrap.cc:492] T e6a5dbc01970406599bc229548544689 P f35a78aeac02496d88fa800153754b0a: Bootstrap starting.
I20260709 13:12:33.204435   699 tablet_bootstrap.cc:654] T bf7b037540fc46d098bf7e1888e9c682 P 15c6b88567e44da395f245725ffe7f04: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:33.237236   646 tablet_service.cc:1511] Processing CreateTablet for tablet e6a5dbc01970406599bc229548544689 (DEFAULT_TABLE table=test-workload [id=c5662d0cce42400296dbe5758ce8f702]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:12:33.238765   646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e6a5dbc01970406599bc229548544689. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:33.233811   648 tablet_service.cc:1511] Processing CreateTablet for tablet a13e0fbeefb142f6bf766c1509cd1d93 (DEFAULT_TABLE table=test-workload [id=c5662d0cce42400296dbe5758ce8f702]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:12:33.241433   648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a13e0fbeefb142f6bf766c1509cd1d93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:33.244354   698 tablet_bootstrap.cc:654] T e6a5dbc01970406599bc229548544689 P f35a78aeac02496d88fa800153754b0a: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:33.235743   647 tablet_service.cc:1511] Processing CreateTablet for tablet bf7b037540fc46d098bf7e1888e9c682 (DEFAULT_TABLE table=test-workload [id=c5662d0cce42400296dbe5758ce8f702]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:12:33.245903   647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bf7b037540fc46d098bf7e1888e9c682. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:33.246055   642 tablet_service.cc:1511] Processing CreateTablet for tablet 82ac3d8dfa714f66a589173f6a9eb004 (DEFAULT_TABLE table=test-workload [id=c5662d0cce42400296dbe5758ce8f702]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:12:33.247641   642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 82ac3d8dfa714f66a589173f6a9eb004. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:33.250317   643 tablet_service.cc:1511] Processing CreateTablet for tablet 3723f5ab8f974d1892ce711b33fac1dd (DEFAULT_TABLE table=test-workload [id=c5662d0cce42400296dbe5758ce8f702]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:12:33.252283   643 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3723f5ab8f974d1892ce711b33fac1dd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:33.254391   644 tablet_service.cc:1511] Processing CreateTablet for tablet 6e5c1404cee54b96a9c3a8c024493223 (DEFAULT_TABLE table=test-workload [id=c5662d0cce42400296dbe5758ce8f702]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:12:33.257499   645 tablet_service.cc:1511] Processing CreateTablet for tablet 93236792b8284277a935d6e569d4c96b (DEFAULT_TABLE table=test-workload [id=c5662d0cce42400296dbe5758ce8f702]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:12:33.262073   641 tablet_service.cc:1511] Processing CreateTablet for tablet 6653d3e90150480daf2486b331be3130 (DEFAULT_TABLE table=test-workload [id=c5662d0cce42400296dbe5758ce8f702]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:12:33.263484   641 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6653d3e90150480daf2486b331be3130. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:33.265116   645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93236792b8284277a935d6e569d4c96b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:33.266698   644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6e5c1404cee54b96a9c3a8c024493223. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:33.266513   640 tablet_service.cc:1511] Processing CreateTablet for tablet 3b24ef3f665449c288b2fcd28c288d27 (DEFAULT_TABLE table=test-workload [id=c5662d0cce42400296dbe5758ce8f702]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:12:33.268234   640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b24ef3f665449c288b2fcd28c288d27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:33.341205   699 tablet_bootstrap.cc:492] T bf7b037540fc46d098bf7e1888e9c682 P 15c6b88567e44da395f245725ffe7f04: No bootstrap required, opened a new log
I20260709 13:12:33.341802   699 ts_tablet_manager.cc:1403] T bf7b037540fc46d098bf7e1888e9c682 P 15c6b88567e44da395f245725ffe7f04: Time spent bootstrapping tablet: real 0.192s	user 0.035s	sys 0.027s
I20260709 13:12:33.342742   698 tablet_bootstrap.cc:492] T e6a5dbc01970406599bc229548544689 P f35a78aeac02496d88fa800153754b0a: No bootstrap required, opened a new log
I20260709 13:12:33.343250   698 ts_tablet_manager.cc:1403] T e6a5dbc01970406599bc229548544689 P f35a78aeac02496d88fa800153754b0a: Time spent bootstrapping tablet: real 0.149s	user 0.041s	sys 0.047s
I20260709 13:12:33.344678   699 raft_consensus.cc:359] T bf7b037540fc46d098bf7e1888e9c682 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.345781   699 raft_consensus.cc:385] T bf7b037540fc46d098bf7e1888e9c682 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:33.346230   699 raft_consensus.cc:740] T bf7b037540fc46d098bf7e1888e9c682 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15c6b88567e44da395f245725ffe7f04, State: Initialized, Role: FOLLOWER
I20260709 13:12:33.347637   699 consensus_queue.cc:260] T bf7b037540fc46d098bf7e1888e9c682 P 15c6b88567e44da395f245725ffe7f04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.346132   698 raft_consensus.cc:359] T e6a5dbc01970406599bc229548544689 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.350324   698 raft_consensus.cc:385] T e6a5dbc01970406599bc229548544689 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:33.350962   698 raft_consensus.cc:740] T e6a5dbc01970406599bc229548544689 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f35a78aeac02496d88fa800153754b0a, State: Initialized, Role: FOLLOWER
I20260709 13:12:33.351794   698 consensus_queue.cc:260] T e6a5dbc01970406599bc229548544689 P f35a78aeac02496d88fa800153754b0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.352383   702 tablet_bootstrap.cc:492] T 82ac3d8dfa714f66a589173f6a9eb004 P 7d13b2cad49e4853831a427132e294d3: Bootstrap starting.
I20260709 13:12:33.356307   609 heartbeater.cc:499] Master 127.21.63.62:35237 was elected leader, sending a full tablet report...
I20260709 13:12:33.357366   699 ts_tablet_manager.cc:1434] T bf7b037540fc46d098bf7e1888e9c682 P 15c6b88567e44da395f245725ffe7f04: Time spent starting tablet: real 0.015s	user 0.011s	sys 0.004s
I20260709 13:12:33.358438   699 tablet_bootstrap.cc:492] T 6e5c1404cee54b96a9c3a8c024493223 P 15c6b88567e44da395f245725ffe7f04: Bootstrap starting.
I20260709 13:12:33.362516   534 heartbeater.cc:499] Master 127.21.63.62:35237 was elected leader, sending a full tablet report...
I20260709 13:12:33.363046   698 ts_tablet_manager.cc:1434] T e6a5dbc01970406599bc229548544689 P f35a78aeac02496d88fa800153754b0a: Time spent starting tablet: real 0.020s	user 0.017s	sys 0.000s
I20260709 13:12:33.363852   702 tablet_bootstrap.cc:654] T 82ac3d8dfa714f66a589173f6a9eb004 P 7d13b2cad49e4853831a427132e294d3: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:33.364356   698 tablet_bootstrap.cc:492] T a13e0fbeefb142f6bf766c1509cd1d93 P f35a78aeac02496d88fa800153754b0a: Bootstrap starting.
I20260709 13:12:33.372820   698 tablet_bootstrap.cc:654] T a13e0fbeefb142f6bf766c1509cd1d93 P f35a78aeac02496d88fa800153754b0a: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:33.375887   699 tablet_bootstrap.cc:654] T 6e5c1404cee54b96a9c3a8c024493223 P 15c6b88567e44da395f245725ffe7f04: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:33.386041   702 tablet_bootstrap.cc:492] T 82ac3d8dfa714f66a589173f6a9eb004 P 7d13b2cad49e4853831a427132e294d3: No bootstrap required, opened a new log
I20260709 13:12:33.386669   702 ts_tablet_manager.cc:1403] T 82ac3d8dfa714f66a589173f6a9eb004 P 7d13b2cad49e4853831a427132e294d3: Time spent bootstrapping tablet: real 0.035s	user 0.017s	sys 0.013s
I20260709 13:12:33.387900   698 tablet_bootstrap.cc:492] T a13e0fbeefb142f6bf766c1509cd1d93 P f35a78aeac02496d88fa800153754b0a: No bootstrap required, opened a new log
I20260709 13:12:33.388511   698 ts_tablet_manager.cc:1403] T a13e0fbeefb142f6bf766c1509cd1d93 P f35a78aeac02496d88fa800153754b0a: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.004s
I20260709 13:12:33.389690   702 raft_consensus.cc:359] T 82ac3d8dfa714f66a589173f6a9eb004 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.390533   702 raft_consensus.cc:385] T 82ac3d8dfa714f66a589173f6a9eb004 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:33.390885   702 raft_consensus.cc:740] T 82ac3d8dfa714f66a589173f6a9eb004 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d13b2cad49e4853831a427132e294d3, State: Initialized, Role: FOLLOWER
I20260709 13:12:33.391445   698 raft_consensus.cc:359] T a13e0fbeefb142f6bf766c1509cd1d93 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } }
I20260709 13:12:33.392262   698 raft_consensus.cc:385] T a13e0fbeefb142f6bf766c1509cd1d93 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:33.392619   698 raft_consensus.cc:740] T a13e0fbeefb142f6bf766c1509cd1d93 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f35a78aeac02496d88fa800153754b0a, State: Initialized, Role: FOLLOWER
I20260709 13:12:33.391831   702 consensus_queue.cc:260] T 82ac3d8dfa714f66a589173f6a9eb004 P 7d13b2cad49e4853831a427132e294d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.394418   698 consensus_queue.cc:260] T a13e0fbeefb142f6bf766c1509cd1d93 P f35a78aeac02496d88fa800153754b0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } }
I20260709 13:12:33.397485   698 ts_tablet_manager.cc:1434] T a13e0fbeefb142f6bf766c1509cd1d93 P f35a78aeac02496d88fa800153754b0a: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:12:33.398844   698 tablet_bootstrap.cc:492] T 82ac3d8dfa714f66a589173f6a9eb004 P f35a78aeac02496d88fa800153754b0a: Bootstrap starting.
I20260709 13:12:33.398957   699 tablet_bootstrap.cc:492] T 6e5c1404cee54b96a9c3a8c024493223 P 15c6b88567e44da395f245725ffe7f04: No bootstrap required, opened a new log
I20260709 13:12:33.399710   699 ts_tablet_manager.cc:1403] T 6e5c1404cee54b96a9c3a8c024493223 P 15c6b88567e44da395f245725ffe7f04: Time spent bootstrapping tablet: real 0.042s	user 0.008s	sys 0.018s
I20260709 13:12:33.403617   699 raft_consensus.cc:359] T 6e5c1404cee54b96a9c3a8c024493223 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.404446   699 raft_consensus.cc:385] T 6e5c1404cee54b96a9c3a8c024493223 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:33.404879   699 raft_consensus.cc:740] T 6e5c1404cee54b96a9c3a8c024493223 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15c6b88567e44da395f245725ffe7f04, State: Initialized, Role: FOLLOWER
I20260709 13:12:33.405918   699 consensus_queue.cc:260] T 6e5c1404cee54b96a9c3a8c024493223 P 15c6b88567e44da395f245725ffe7f04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.408394   698 tablet_bootstrap.cc:654] T 82ac3d8dfa714f66a589173f6a9eb004 P f35a78aeac02496d88fa800153754b0a: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:33.410840   703 raft_consensus.cc:493] T bf7b037540fc46d098bf7e1888e9c682 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:33.411459   703 raft_consensus.cc:515] T bf7b037540fc46d098bf7e1888e9c682 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.417610   683 heartbeater.cc:499] Master 127.21.63.62:35237 was elected leader, sending a full tablet report...
I20260709 13:12:33.417711   703 leader_election.cc:290] T bf7b037540fc46d098bf7e1888e9c682 P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881), 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695)
I20260709 13:12:33.418540   702 ts_tablet_manager.cc:1434] T 82ac3d8dfa714f66a589173f6a9eb004 P 7d13b2cad49e4853831a427132e294d3: Time spent starting tablet: real 0.031s	user 0.016s	sys 0.012s
I20260709 13:12:33.420099   702 tablet_bootstrap.cc:492] T bf7b037540fc46d098bf7e1888e9c682 P 7d13b2cad49e4853831a427132e294d3: Bootstrap starting.
I20260709 13:12:33.423259   699 ts_tablet_manager.cc:1434] T 6e5c1404cee54b96a9c3a8c024493223 P 15c6b88567e44da395f245725ffe7f04: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 13:12:33.424997   698 tablet_bootstrap.cc:492] T 82ac3d8dfa714f66a589173f6a9eb004 P f35a78aeac02496d88fa800153754b0a: No bootstrap required, opened a new log
I20260709 13:12:33.425642   698 ts_tablet_manager.cc:1403] T 82ac3d8dfa714f66a589173f6a9eb004 P f35a78aeac02496d88fa800153754b0a: Time spent bootstrapping tablet: real 0.027s	user 0.020s	sys 0.006s
I20260709 13:12:33.427729   702 tablet_bootstrap.cc:654] T bf7b037540fc46d098bf7e1888e9c682 P 7d13b2cad49e4853831a427132e294d3: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:33.428759   698 raft_consensus.cc:359] T 82ac3d8dfa714f66a589173f6a9eb004 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.429864   698 raft_consensus.cc:385] T 82ac3d8dfa714f66a589173f6a9eb004 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:33.430321   698 raft_consensus.cc:740] T 82ac3d8dfa714f66a589173f6a9eb004 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f35a78aeac02496d88fa800153754b0a, State: Initialized, Role: FOLLOWER
I20260709 13:12:33.432634   699 tablet_bootstrap.cc:492] T 82ac3d8dfa714f66a589173f6a9eb004 P 15c6b88567e44da395f245725ffe7f04: Bootstrap starting.
I20260709 13:12:33.432581   698 consensus_queue.cc:260] T 82ac3d8dfa714f66a589173f6a9eb004 P f35a78aeac02496d88fa800153754b0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.435544   698 ts_tablet_manager.cc:1434] T 82ac3d8dfa714f66a589173f6a9eb004 P f35a78aeac02496d88fa800153754b0a: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:12:33.436806   698 tablet_bootstrap.cc:492] T 6e5c1404cee54b96a9c3a8c024493223 P f35a78aeac02496d88fa800153754b0a: Bootstrap starting.
I20260709 13:12:33.453511   698 tablet_bootstrap.cc:654] T 6e5c1404cee54b96a9c3a8c024493223 P f35a78aeac02496d88fa800153754b0a: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:33.458808   699 tablet_bootstrap.cc:654] T 82ac3d8dfa714f66a589173f6a9eb004 P 15c6b88567e44da395f245725ffe7f04: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:33.463917   509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf7b037540fc46d098bf7e1888e9c682" candidate_uuid: "15c6b88567e44da395f245725ffe7f04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f35a78aeac02496d88fa800153754b0a" is_pre_election: true
I20260709 13:12:33.465181   704 raft_consensus.cc:493] T e6a5dbc01970406599bc229548544689 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:33.465472   702 tablet_bootstrap.cc:492] T bf7b037540fc46d098bf7e1888e9c682 P 7d13b2cad49e4853831a427132e294d3: No bootstrap required, opened a new log
I20260709 13:12:33.465737   704 raft_consensus.cc:515] T e6a5dbc01970406599bc229548544689 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.466001   702 ts_tablet_manager.cc:1403] T bf7b037540fc46d098bf7e1888e9c682 P 7d13b2cad49e4853831a427132e294d3: Time spent bootstrapping tablet: real 0.046s	user 0.025s	sys 0.020s
W20260709 13:12:33.466012   550 leader_election.cc:343] T bf7b037540fc46d098bf7e1888e9c682 P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:33.469429   702 raft_consensus.cc:359] T bf7b037540fc46d098bf7e1888e9c682 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.470181   702 raft_consensus.cc:385] T bf7b037540fc46d098bf7e1888e9c682 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:33.470505   702 raft_consensus.cc:740] T bf7b037540fc46d098bf7e1888e9c682 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d13b2cad49e4853831a427132e294d3, State: Initialized, Role: FOLLOWER
I20260709 13:12:33.471171   699 tablet_bootstrap.cc:492] T 82ac3d8dfa714f66a589173f6a9eb004 P 15c6b88567e44da395f245725ffe7f04: No bootstrap required, opened a new log
I20260709 13:12:33.471664   702 consensus_queue.cc:260] T bf7b037540fc46d098bf7e1888e9c682 P 7d13b2cad49e4853831a427132e294d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.472276   699 ts_tablet_manager.cc:1403] T 82ac3d8dfa714f66a589173f6a9eb004 P 15c6b88567e44da395f245725ffe7f04: Time spent bootstrapping tablet: real 0.045s	user 0.017s	sys 0.004s
I20260709 13:12:33.474265   702 ts_tablet_manager.cc:1434] T bf7b037540fc46d098bf7e1888e9c682 P 7d13b2cad49e4853831a427132e294d3: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:12:33.475574   702 tablet_bootstrap.cc:492] T e6a5dbc01970406599bc229548544689 P 7d13b2cad49e4853831a427132e294d3: Bootstrap starting.
I20260709 13:12:33.476406   699 raft_consensus.cc:359] T 82ac3d8dfa714f66a589173f6a9eb004 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.483752   702 tablet_bootstrap.cc:654] T e6a5dbc01970406599bc229548544689 P 7d13b2cad49e4853831a427132e294d3: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:33.489189   699 raft_consensus.cc:385] T 82ac3d8dfa714f66a589173f6a9eb004 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:33.490044   699 raft_consensus.cc:740] T 82ac3d8dfa714f66a589173f6a9eb004 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15c6b88567e44da395f245725ffe7f04, State: Initialized, Role: FOLLOWER
I20260709 13:12:33.484404   704 leader_election.cc:290] T e6a5dbc01970406599bc229548544689 P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181), 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695)
I20260709 13:12:33.489188   708 raft_consensus.cc:493] T bf7b037540fc46d098bf7e1888e9c682 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:33.490892   699 consensus_queue.cc:260] T 82ac3d8dfa714f66a589173f6a9eb004 P 15c6b88567e44da395f245725ffe7f04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.491374   708 raft_consensus.cc:515] T bf7b037540fc46d098bf7e1888e9c682 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.493342   699 ts_tablet_manager.cc:1434] T 82ac3d8dfa714f66a589173f6a9eb004 P 15c6b88567e44da395f245725ffe7f04: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.003s
I20260709 13:12:33.498106   699 tablet_bootstrap.cc:492] T 3723f5ab8f974d1892ce711b33fac1dd P 15c6b88567e44da395f245725ffe7f04: Bootstrap starting.
I20260709 13:12:33.498543   708 leader_election.cc:290] T bf7b037540fc46d098bf7e1888e9c682 P 7d13b2cad49e4853831a427132e294d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881), 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181)
I20260709 13:12:33.507867   699 tablet_bootstrap.cc:654] T 3723f5ab8f974d1892ce711b33fac1dd P 15c6b88567e44da395f245725ffe7f04: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:33.519199   658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf7b037540fc46d098bf7e1888e9c682" candidate_uuid: "15c6b88567e44da395f245725ffe7f04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d13b2cad49e4853831a427132e294d3" is_pre_election: true
I20260709 13:12:33.520131   658 raft_consensus.cc:2468] T bf7b037540fc46d098bf7e1888e9c682 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15c6b88567e44da395f245725ffe7f04 in term 0.
I20260709 13:12:33.521893   550 leader_election.cc:304] T bf7b037540fc46d098bf7e1888e9c682 P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 15c6b88567e44da395f245725ffe7f04, 7d13b2cad49e4853831a427132e294d3; no voters: f35a78aeac02496d88fa800153754b0a
I20260709 13:12:33.522751   698 tablet_bootstrap.cc:492] T 6e5c1404cee54b96a9c3a8c024493223 P f35a78aeac02496d88fa800153754b0a: No bootstrap required, opened a new log
I20260709 13:12:33.523150   703 raft_consensus.cc:2804] T bf7b037540fc46d098bf7e1888e9c682 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:33.523430   698 ts_tablet_manager.cc:1403] T 6e5c1404cee54b96a9c3a8c024493223 P f35a78aeac02496d88fa800153754b0a: Time spent bootstrapping tablet: real 0.087s	user 0.025s	sys 0.020s
I20260709 13:12:33.523576   703 raft_consensus.cc:493] T bf7b037540fc46d098bf7e1888e9c682 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:33.523937   703 raft_consensus.cc:3060] T bf7b037540fc46d098bf7e1888e9c682 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:33.527374   698 raft_consensus.cc:359] T 6e5c1404cee54b96a9c3a8c024493223 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.528246   698 raft_consensus.cc:385] T 6e5c1404cee54b96a9c3a8c024493223 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:33.528379   702 tablet_bootstrap.cc:492] T e6a5dbc01970406599bc229548544689 P 7d13b2cad49e4853831a427132e294d3: No bootstrap required, opened a new log
I20260709 13:12:33.528787   698 raft_consensus.cc:740] T 6e5c1404cee54b96a9c3a8c024493223 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f35a78aeac02496d88fa800153754b0a, State: Initialized, Role: FOLLOWER
I20260709 13:12:33.529345   702 ts_tablet_manager.cc:1403] T e6a5dbc01970406599bc229548544689 P 7d13b2cad49e4853831a427132e294d3: Time spent bootstrapping tablet: real 0.054s	user 0.018s	sys 0.023s
I20260709 13:12:33.529855   698 consensus_queue.cc:260] T 6e5c1404cee54b96a9c3a8c024493223 P f35a78aeac02496d88fa800153754b0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.532243   698 ts_tablet_manager.cc:1434] T 6e5c1404cee54b96a9c3a8c024493223 P f35a78aeac02496d88fa800153754b0a: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.002s
I20260709 13:12:33.533056   702 raft_consensus.cc:359] T e6a5dbc01970406599bc229548544689 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.533797   698 tablet_bootstrap.cc:492] T 3b24ef3f665449c288b2fcd28c288d27 P f35a78aeac02496d88fa800153754b0a: Bootstrap starting.
I20260709 13:12:33.534106   702 raft_consensus.cc:385] T e6a5dbc01970406599bc229548544689 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:33.535434   702 raft_consensus.cc:740] T e6a5dbc01970406599bc229548544689 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d13b2cad49e4853831a427132e294d3, State: Initialized, Role: FOLLOWER
I20260709 13:12:33.533192   703 raft_consensus.cc:515] T bf7b037540fc46d098bf7e1888e9c682 P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.536441   702 consensus_queue.cc:260] T e6a5dbc01970406599bc229548544689 P 7d13b2cad49e4853831a427132e294d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.537602   703 leader_election.cc:290] T bf7b037540fc46d098bf7e1888e9c682 P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 1 election: Requested vote from peers f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881), 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695)
I20260709 13:12:33.538285   509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf7b037540fc46d098bf7e1888e9c682" candidate_uuid: "15c6b88567e44da395f245725ffe7f04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f35a78aeac02496d88fa800153754b0a"
I20260709 13:12:33.538466   584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6a5dbc01970406599bc229548544689" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15c6b88567e44da395f245725ffe7f04" is_pre_election: true
W20260709 13:12:33.540035   550 leader_election.cc:343] T bf7b037540fc46d098bf7e1888e9c682 P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:33.540841   702 ts_tablet_manager.cc:1434] T e6a5dbc01970406599bc229548544689 P 7d13b2cad49e4853831a427132e294d3: Time spent starting tablet: real 0.011s	user 0.001s	sys 0.007s
I20260709 13:12:33.541391   658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf7b037540fc46d098bf7e1888e9c682" candidate_uuid: "15c6b88567e44da395f245725ffe7f04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d13b2cad49e4853831a427132e294d3"
W20260709 13:12:33.541692   472 leader_election.cc:343] T e6a5dbc01970406599bc229548544689 P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:33.541873   702 tablet_bootstrap.cc:492] T 6653d3e90150480daf2486b331be3130 P 7d13b2cad49e4853831a427132e294d3: Bootstrap starting.
I20260709 13:12:33.541915   658 raft_consensus.cc:3060] T bf7b037540fc46d098bf7e1888e9c682 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:33.550235   702 tablet_bootstrap.cc:654] T 6653d3e90150480daf2486b331be3130 P 7d13b2cad49e4853831a427132e294d3: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:33.551427   658 raft_consensus.cc:2468] T bf7b037540fc46d098bf7e1888e9c682 P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15c6b88567e44da395f245725ffe7f04 in term 1.
I20260709 13:12:33.552377   698 tablet_bootstrap.cc:654] T 3b24ef3f665449c288b2fcd28c288d27 P f35a78aeac02496d88fa800153754b0a: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:33.552814   550 leader_election.cc:304] T bf7b037540fc46d098bf7e1888e9c682 P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 15c6b88567e44da395f245725ffe7f04, 7d13b2cad49e4853831a427132e294d3; no voters: f35a78aeac02496d88fa800153754b0a
I20260709 13:12:33.553920   703 raft_consensus.cc:2804] T bf7b037540fc46d098bf7e1888e9c682 P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:33.569111   657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6a5dbc01970406599bc229548544689" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d13b2cad49e4853831a427132e294d3" is_pre_election: true
I20260709 13:12:33.569890   657 raft_consensus.cc:2468] T e6a5dbc01970406599bc229548544689 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f35a78aeac02496d88fa800153754b0a in term 0.
I20260709 13:12:33.570668   702 tablet_bootstrap.cc:492] T 6653d3e90150480daf2486b331be3130 P 7d13b2cad49e4853831a427132e294d3: No bootstrap required, opened a new log
I20260709 13:12:33.571247   702 ts_tablet_manager.cc:1403] T 6653d3e90150480daf2486b331be3130 P 7d13b2cad49e4853831a427132e294d3: Time spent bootstrapping tablet: real 0.030s	user 0.010s	sys 0.009s
I20260709 13:12:33.571439   475 leader_election.cc:304] T e6a5dbc01970406599bc229548544689 P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d13b2cad49e4853831a427132e294d3, f35a78aeac02496d88fa800153754b0a; no voters: 15c6b88567e44da395f245725ffe7f04
I20260709 13:12:33.572746   704 raft_consensus.cc:2804] T e6a5dbc01970406599bc229548544689 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:33.573103   704 raft_consensus.cc:493] T e6a5dbc01970406599bc229548544689 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:33.573463   704 raft_consensus.cc:3060] T e6a5dbc01970406599bc229548544689 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:12:33.574661   455 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:12:33.574541   702 raft_consensus.cc:359] T 6653d3e90150480daf2486b331be3130 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.575361   702 raft_consensus.cc:385] T 6653d3e90150480daf2486b331be3130 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:33.575700   702 raft_consensus.cc:740] T 6653d3e90150480daf2486b331be3130 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d13b2cad49e4853831a427132e294d3, State: Initialized, Role: FOLLOWER
I20260709 13:12:33.576695   702 consensus_queue.cc:260] T 6653d3e90150480daf2486b331be3130 P 7d13b2cad49e4853831a427132e294d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.582885   704 raft_consensus.cc:515] T e6a5dbc01970406599bc229548544689 P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.584596   702 ts_tablet_manager.cc:1434] T 6653d3e90150480daf2486b331be3130 P 7d13b2cad49e4853831a427132e294d3: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:12:33.586517   584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6a5dbc01970406599bc229548544689" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15c6b88567e44da395f245725ffe7f04"
W20260709 13:12:33.588244   472 leader_election.cc:343] T e6a5dbc01970406599bc229548544689 P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:33.590648   657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e6a5dbc01970406599bc229548544689" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d13b2cad49e4853831a427132e294d3"
I20260709 13:12:33.591200   657 raft_consensus.cc:3060] T e6a5dbc01970406599bc229548544689 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:33.593660   702 tablet_bootstrap.cc:492] T 3b24ef3f665449c288b2fcd28c288d27 P 7d13b2cad49e4853831a427132e294d3: Bootstrap starting.
I20260709 13:12:33.599045   657 raft_consensus.cc:2468] T e6a5dbc01970406599bc229548544689 P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f35a78aeac02496d88fa800153754b0a in term 1.
I20260709 13:12:33.601403   475 leader_election.cc:304] T e6a5dbc01970406599bc229548544689 P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d13b2cad49e4853831a427132e294d3, f35a78aeac02496d88fa800153754b0a; no voters: 15c6b88567e44da395f245725ffe7f04
I20260709 13:12:33.602617   702 tablet_bootstrap.cc:654] T 3b24ef3f665449c288b2fcd28c288d27 P 7d13b2cad49e4853831a427132e294d3: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:33.603404   726 raft_consensus.cc:2804] T e6a5dbc01970406599bc229548544689 P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:33.611975   728 raft_consensus.cc:493] T a13e0fbeefb142f6bf766c1509cd1d93 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:33.615064   702 tablet_bootstrap.cc:492] T 3b24ef3f665449c288b2fcd28c288d27 P 7d13b2cad49e4853831a427132e294d3: No bootstrap required, opened a new log
I20260709 13:12:33.614812   728 raft_consensus.cc:515] T a13e0fbeefb142f6bf766c1509cd1d93 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } }
I20260709 13:12:33.612567   726 raft_consensus.cc:697] T e6a5dbc01970406599bc229548544689 P f35a78aeac02496d88fa800153754b0a [term 1 LEADER]: Becoming Leader. State: Replica: f35a78aeac02496d88fa800153754b0a, State: Running, Role: LEADER
I20260709 13:12:33.615665   702 ts_tablet_manager.cc:1403] T 3b24ef3f665449c288b2fcd28c288d27 P 7d13b2cad49e4853831a427132e294d3: Time spent bootstrapping tablet: real 0.022s	user 0.009s	sys 0.010s
I20260709 13:12:33.620004   704 leader_election.cc:290] T e6a5dbc01970406599bc229548544689 P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 election: Requested vote from peers 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181), 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695)
I20260709 13:12:33.621505   703 raft_consensus.cc:697] T bf7b037540fc46d098bf7e1888e9c682 P 15c6b88567e44da395f245725ffe7f04 [term 1 LEADER]: Becoming Leader. State: Replica: 15c6b88567e44da395f245725ffe7f04, State: Running, Role: LEADER
I20260709 13:12:33.622313   657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a13e0fbeefb142f6bf766c1509cd1d93" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d13b2cad49e4853831a427132e294d3" is_pre_election: true
I20260709 13:12:33.635306   584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a13e0fbeefb142f6bf766c1509cd1d93" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15c6b88567e44da395f245725ffe7f04" is_pre_election: true
W20260709 13:12:33.636734   472 leader_election.cc:343] T a13e0fbeefb142f6bf766c1509cd1d93 P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:12:33.637517   475 leader_election.cc:343] T a13e0fbeefb142f6bf766c1509cd1d93 P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:33.637934   475 leader_election.cc:304] T a13e0fbeefb142f6bf766c1509cd1d93 P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f35a78aeac02496d88fa800153754b0a; no voters: 15c6b88567e44da395f245725ffe7f04, 7d13b2cad49e4853831a427132e294d3
I20260709 13:12:33.623549   726 consensus_queue.cc:237] T e6a5dbc01970406599bc229548544689 P f35a78aeac02496d88fa800153754b0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.638443   702 raft_consensus.cc:359] T 3b24ef3f665449c288b2fcd28c288d27 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.639494   704 raft_consensus.cc:2749] T a13e0fbeefb142f6bf766c1509cd1d93 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:33.639810   702 raft_consensus.cc:385] T 3b24ef3f665449c288b2fcd28c288d27 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:33.632923   728 leader_election.cc:290] T a13e0fbeefb142f6bf766c1509cd1d93 P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695), 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181)
I20260709 13:12:33.640337   702 raft_consensus.cc:740] T 3b24ef3f665449c288b2fcd28c288d27 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d13b2cad49e4853831a427132e294d3, State: Initialized, Role: FOLLOWER
I20260709 13:12:33.642139   703 consensus_queue.cc:237] T bf7b037540fc46d098bf7e1888e9c682 P 15c6b88567e44da395f245725ffe7f04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.642627   702 consensus_queue.cc:260] T 3b24ef3f665449c288b2fcd28c288d27 P 7d13b2cad49e4853831a427132e294d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.647330   702 ts_tablet_manager.cc:1434] T 3b24ef3f665449c288b2fcd28c288d27 P 7d13b2cad49e4853831a427132e294d3: Time spent starting tablet: real 0.014s	user 0.005s	sys 0.003s
I20260709 13:12:33.650353   702 tablet_bootstrap.cc:492] T 3723f5ab8f974d1892ce711b33fac1dd P 7d13b2cad49e4853831a427132e294d3: Bootstrap starting.
I20260709 13:12:33.655087   708 raft_consensus.cc:493] T 3b24ef3f665449c288b2fcd28c288d27 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:33.655659   708 raft_consensus.cc:515] T 3b24ef3f665449c288b2fcd28c288d27 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.657069   509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf7b037540fc46d098bf7e1888e9c682" candidate_uuid: "7d13b2cad49e4853831a427132e294d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f35a78aeac02496d88fa800153754b0a" is_pre_election: true
I20260709 13:12:33.657920   708 leader_election.cc:290] T 3b24ef3f665449c288b2fcd28c288d27 P 7d13b2cad49e4853831a427132e294d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881), 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181)
I20260709 13:12:33.660262   584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf7b037540fc46d098bf7e1888e9c682" candidate_uuid: "7d13b2cad49e4853831a427132e294d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15c6b88567e44da395f245725ffe7f04" is_pre_election: true
I20260709 13:12:33.660650   708 raft_consensus.cc:493] T 6653d3e90150480daf2486b331be3130 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:33.661218   698 tablet_bootstrap.cc:492] T 3b24ef3f665449c288b2fcd28c288d27 P f35a78aeac02496d88fa800153754b0a: No bootstrap required, opened a new log
I20260709 13:12:33.662498   698 ts_tablet_manager.cc:1403] T 3b24ef3f665449c288b2fcd28c288d27 P f35a78aeac02496d88fa800153754b0a: Time spent bootstrapping tablet: real 0.129s	user 0.051s	sys 0.022s
I20260709 13:12:33.662443   509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24ef3f665449c288b2fcd28c288d27" candidate_uuid: "7d13b2cad49e4853831a427132e294d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f35a78aeac02496d88fa800153754b0a" is_pre_election: true
I20260709 13:12:33.661871   583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24ef3f665449c288b2fcd28c288d27" candidate_uuid: "7d13b2cad49e4853831a427132e294d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15c6b88567e44da395f245725ffe7f04" is_pre_election: true
W20260709 13:12:33.660065   624 leader_election.cc:343] T bf7b037540fc46d098bf7e1888e9c682 P 7d13b2cad49e4853831a427132e294d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:33.668896   708 raft_consensus.cc:515] T 6653d3e90150480daf2486b331be3130 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.669703   698 raft_consensus.cc:359] T 3b24ef3f665449c288b2fcd28c288d27 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.671150   698 raft_consensus.cc:385] T 3b24ef3f665449c288b2fcd28c288d27 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:33.657996   702 tablet_bootstrap.cc:654] T 3723f5ab8f974d1892ce711b33fac1dd P 7d13b2cad49e4853831a427132e294d3: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:33.672289   508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6653d3e90150480daf2486b331be3130" candidate_uuid: "7d13b2cad49e4853831a427132e294d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f35a78aeac02496d88fa800153754b0a" is_pre_election: true
I20260709 13:12:33.671543   621 leader_election.cc:304] T bf7b037540fc46d098bf7e1888e9c682 P 7d13b2cad49e4853831a427132e294d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d13b2cad49e4853831a427132e294d3; no voters: 15c6b88567e44da395f245725ffe7f04, f35a78aeac02496d88fa800153754b0a
I20260709 13:12:33.674233   699 tablet_bootstrap.cc:492] T 3723f5ab8f974d1892ce711b33fac1dd P 15c6b88567e44da395f245725ffe7f04: No bootstrap required, opened a new log
W20260709 13:12:33.675314   621 leader_election.cc:343] T 3b24ef3f665449c288b2fcd28c288d27 P 7d13b2cad49e4853831a427132e294d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:12:33.675516   624 leader_election.cc:343] T 6653d3e90150480daf2486b331be3130 P 7d13b2cad49e4853831a427132e294d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:33.676064   699 ts_tablet_manager.cc:1403] T 3723f5ab8f974d1892ce711b33fac1dd P 15c6b88567e44da395f245725ffe7f04: Time spent bootstrapping tablet: real 0.178s	user 0.024s	sys 0.060s
I20260709 13:12:33.671576   698 raft_consensus.cc:740] T 3b24ef3f665449c288b2fcd28c288d27 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f35a78aeac02496d88fa800153754b0a, State: Initialized, Role: FOLLOWER
I20260709 13:12:33.678530   698 consensus_queue.cc:260] T 3b24ef3f665449c288b2fcd28c288d27 P f35a78aeac02496d88fa800153754b0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.676604   583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6653d3e90150480daf2486b331be3130" candidate_uuid: "7d13b2cad49e4853831a427132e294d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15c6b88567e44da395f245725ffe7f04" is_pre_election: true
I20260709 13:12:33.680907   509 raft_consensus.cc:2468] T 3b24ef3f665449c288b2fcd28c288d27 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d13b2cad49e4853831a427132e294d3 in term 0.
I20260709 13:12:33.680626   699 raft_consensus.cc:359] T 3723f5ab8f974d1892ce711b33fac1dd P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.681944   699 raft_consensus.cc:385] T 3723f5ab8f974d1892ce711b33fac1dd P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:33.682328   699 raft_consensus.cc:740] T 3723f5ab8f974d1892ce711b33fac1dd P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15c6b88567e44da395f245725ffe7f04, State: Initialized, Role: FOLLOWER
I20260709 13:12:33.683164   698 ts_tablet_manager.cc:1434] T 3b24ef3f665449c288b2fcd28c288d27 P f35a78aeac02496d88fa800153754b0a: Time spent starting tablet: real 0.020s	user 0.006s	sys 0.004s
I20260709 13:12:33.683423   624 leader_election.cc:304] T 3b24ef3f665449c288b2fcd28c288d27 P 7d13b2cad49e4853831a427132e294d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d13b2cad49e4853831a427132e294d3, f35a78aeac02496d88fa800153754b0a; no voters: 15c6b88567e44da395f245725ffe7f04
W20260709 13:12:33.683647   621 leader_election.cc:343] T 6653d3e90150480daf2486b331be3130 P 7d13b2cad49e4853831a427132e294d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:33.684087   621 leader_election.cc:304] T 6653d3e90150480daf2486b331be3130 P 7d13b2cad49e4853831a427132e294d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d13b2cad49e4853831a427132e294d3; no voters: 15c6b88567e44da395f245725ffe7f04, f35a78aeac02496d88fa800153754b0a
I20260709 13:12:33.684756   698 tablet_bootstrap.cc:492] T 6653d3e90150480daf2486b331be3130 P f35a78aeac02496d88fa800153754b0a: Bootstrap starting.
I20260709 13:12:33.672322   708 leader_election.cc:290] T 6653d3e90150480daf2486b331be3130 P 7d13b2cad49e4853831a427132e294d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881), 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181)
I20260709 13:12:33.686291   708 raft_consensus.cc:2749] T bf7b037540fc46d098bf7e1888e9c682 P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:33.676609   728 raft_consensus.cc:493] T 6e5c1404cee54b96a9c3a8c024493223 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:33.685025   699 consensus_queue.cc:260] T 3723f5ab8f974d1892ce711b33fac1dd P 15c6b88567e44da395f245725ffe7f04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.687040   728 raft_consensus.cc:515] T 6e5c1404cee54b96a9c3a8c024493223 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.687034   708 raft_consensus.cc:2804] T 3b24ef3f665449c288b2fcd28c288d27 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:33.689347   708 raft_consensus.cc:493] T 3b24ef3f665449c288b2fcd28c288d27 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:33.689849   708 raft_consensus.cc:3060] T 3b24ef3f665449c288b2fcd28c288d27 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:33.690815   699 ts_tablet_manager.cc:1434] T 3723f5ab8f974d1892ce711b33fac1dd P 15c6b88567e44da395f245725ffe7f04: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 13:12:33.692770   699 tablet_bootstrap.cc:492] T 93236792b8284277a935d6e569d4c96b P 15c6b88567e44da395f245725ffe7f04: Bootstrap starting.
I20260709 13:12:33.697117   405 catalog_manager.cc:5697] T e6a5dbc01970406599bc229548544689 P f35a78aeac02496d88fa800153754b0a reported cstate change: term changed from 0 to 1, leader changed from <none> to f35a78aeac02496d88fa800153754b0a (127.21.63.1). New cstate: current_term: 1 leader_uuid: "f35a78aeac02496d88fa800153754b0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:33.705822   404 catalog_manager.cc:5697] T bf7b037540fc46d098bf7e1888e9c682 P 15c6b88567e44da395f245725ffe7f04 reported cstate change: term changed from 0 to 1, leader changed from <none> to 15c6b88567e44da395f245725ffe7f04 (127.21.63.2). New cstate: current_term: 1 leader_uuid: "15c6b88567e44da395f245725ffe7f04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:33.708491   702 tablet_bootstrap.cc:492] T 3723f5ab8f974d1892ce711b33fac1dd P 7d13b2cad49e4853831a427132e294d3: No bootstrap required, opened a new log
I20260709 13:12:33.709180   702 ts_tablet_manager.cc:1403] T 3723f5ab8f974d1892ce711b33fac1dd P 7d13b2cad49e4853831a427132e294d3: Time spent bootstrapping tablet: real 0.059s	user 0.014s	sys 0.000s
I20260709 13:12:33.709275   732 raft_consensus.cc:493] T 82ac3d8dfa714f66a589173f6a9eb004 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:33.709970   698 tablet_bootstrap.cc:654] T 6653d3e90150480daf2486b331be3130 P f35a78aeac02496d88fa800153754b0a: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:33.710750   583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e5c1404cee54b96a9c3a8c024493223" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15c6b88567e44da395f245725ffe7f04" is_pre_election: true
I20260709 13:12:33.712914   657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e5c1404cee54b96a9c3a8c024493223" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d13b2cad49e4853831a427132e294d3" is_pre_election: true
I20260709 13:12:33.712822   702 raft_consensus.cc:359] T 3723f5ab8f974d1892ce711b33fac1dd P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.713506   728 leader_election.cc:290] T 6e5c1404cee54b96a9c3a8c024493223 P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181), 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695)
I20260709 13:12:33.708446   735 raft_consensus.cc:2749] T 6653d3e90150480daf2486b331be3130 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:33.717267   702 raft_consensus.cc:385] T 3723f5ab8f974d1892ce711b33fac1dd P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:33.717628   702 raft_consensus.cc:740] T 3723f5ab8f974d1892ce711b33fac1dd P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d13b2cad49e4853831a427132e294d3, State: Initialized, Role: FOLLOWER
I20260709 13:12:33.719827   702 consensus_queue.cc:260] T 3723f5ab8f974d1892ce711b33fac1dd P 7d13b2cad49e4853831a427132e294d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
W20260709 13:12:33.720340   475 leader_election.cc:343] T 6e5c1404cee54b96a9c3a8c024493223 P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:33.725638   699 tablet_bootstrap.cc:654] T 93236792b8284277a935d6e569d4c96b P 15c6b88567e44da395f245725ffe7f04: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:33.729808   702 ts_tablet_manager.cc:1434] T 3723f5ab8f974d1892ce711b33fac1dd P 7d13b2cad49e4853831a427132e294d3: Time spent starting tablet: real 0.020s	user 0.009s	sys 0.000s
I20260709 13:12:33.731197   702 tablet_bootstrap.cc:492] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3: Bootstrap starting.
I20260709 13:12:33.714859   708 raft_consensus.cc:515] T 3b24ef3f665449c288b2fcd28c288d27 P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.710021   732 raft_consensus.cc:515] T 82ac3d8dfa714f66a589173f6a9eb004 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.716758   583 raft_consensus.cc:2468] T 6e5c1404cee54b96a9c3a8c024493223 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f35a78aeac02496d88fa800153754b0a in term 0.
I20260709 13:12:33.735908   472 leader_election.cc:304] T 6e5c1404cee54b96a9c3a8c024493223 P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 15c6b88567e44da395f245725ffe7f04, f35a78aeac02496d88fa800153754b0a; no voters: 7d13b2cad49e4853831a427132e294d3
I20260709 13:12:33.736083   732 leader_election.cc:290] T 82ac3d8dfa714f66a589173f6a9eb004 P 7d13b2cad49e4853831a427132e294d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881), 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181)
I20260709 13:12:33.737937   509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ac3d8dfa714f66a589173f6a9eb004" candidate_uuid: "7d13b2cad49e4853831a427132e294d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f35a78aeac02496d88fa800153754b0a" is_pre_election: true
I20260709 13:12:33.736138   698 tablet_bootstrap.cc:492] T 6653d3e90150480daf2486b331be3130 P f35a78aeac02496d88fa800153754b0a: No bootstrap required, opened a new log
I20260709 13:12:33.738683   703 raft_consensus.cc:493] T 82ac3d8dfa714f66a589173f6a9eb004 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:33.739320   698 ts_tablet_manager.cc:1403] T 6653d3e90150480daf2486b331be3130 P f35a78aeac02496d88fa800153754b0a: Time spent bootstrapping tablet: real 0.055s	user 0.015s	sys 0.011s
I20260709 13:12:33.739259   703 raft_consensus.cc:515] T 82ac3d8dfa714f66a589173f6a9eb004 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.738722   509 raft_consensus.cc:2468] T 82ac3d8dfa714f66a589173f6a9eb004 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d13b2cad49e4853831a427132e294d3 in term 0.
I20260709 13:12:33.741400   703 leader_election.cc:290] T 82ac3d8dfa714f66a589173f6a9eb004 P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881), 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695)
I20260709 13:12:33.741474   624 leader_election.cc:304] T 82ac3d8dfa714f66a589173f6a9eb004 P 7d13b2cad49e4853831a427132e294d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d13b2cad49e4853831a427132e294d3, f35a78aeac02496d88fa800153754b0a; no voters: 
I20260709 13:12:33.741940   583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ac3d8dfa714f66a589173f6a9eb004" candidate_uuid: "7d13b2cad49e4853831a427132e294d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15c6b88567e44da395f245725ffe7f04" is_pre_election: true
I20260709 13:12:33.742568   583 raft_consensus.cc:2468] T 82ac3d8dfa714f66a589173f6a9eb004 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d13b2cad49e4853831a427132e294d3 in term 0.
I20260709 13:12:33.742938   732 raft_consensus.cc:2804] T 82ac3d8dfa714f66a589173f6a9eb004 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:33.743321   732 raft_consensus.cc:493] T 82ac3d8dfa714f66a589173f6a9eb004 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:33.743309   657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ac3d8dfa714f66a589173f6a9eb004" candidate_uuid: "15c6b88567e44da395f245725ffe7f04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d13b2cad49e4853831a427132e294d3" is_pre_election: true
I20260709 13:12:33.743922   732 raft_consensus.cc:3060] T 82ac3d8dfa714f66a589173f6a9eb004 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:33.744807   509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ac3d8dfa714f66a589173f6a9eb004" candidate_uuid: "15c6b88567e44da395f245725ffe7f04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f35a78aeac02496d88fa800153754b0a" is_pre_election: true
I20260709 13:12:33.745792   702 tablet_bootstrap.cc:654] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:33.743429   728 raft_consensus.cc:2804] T 6e5c1404cee54b96a9c3a8c024493223 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:33.746835   509 raft_consensus.cc:2468] T 82ac3d8dfa714f66a589173f6a9eb004 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15c6b88567e44da395f245725ffe7f04 in term 0.
I20260709 13:12:33.746973   728 raft_consensus.cc:493] T 6e5c1404cee54b96a9c3a8c024493223 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:33.747344   728 raft_consensus.cc:3060] T 6e5c1404cee54b96a9c3a8c024493223 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:33.748278   550 leader_election.cc:304] T 82ac3d8dfa714f66a589173f6a9eb004 P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15c6b88567e44da395f245725ffe7f04, f35a78aeac02496d88fa800153754b0a; no voters: 
I20260709 13:12:33.749209   703 raft_consensus.cc:2804] T 82ac3d8dfa714f66a589173f6a9eb004 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:33.749577   703 raft_consensus.cc:493] T 82ac3d8dfa714f66a589173f6a9eb004 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:33.749910   703 raft_consensus.cc:3060] T 82ac3d8dfa714f66a589173f6a9eb004 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:33.752961   732 raft_consensus.cc:515] T 82ac3d8dfa714f66a589173f6a9eb004 P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.754148   657 raft_consensus.cc:2393] T 82ac3d8dfa714f66a589173f6a9eb004 P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 15c6b88567e44da395f245725ffe7f04 in current term 1: Already voted for candidate 7d13b2cad49e4853831a427132e294d3 in this term.
I20260709 13:12:33.742583   698 raft_consensus.cc:359] T 6653d3e90150480daf2486b331be3130 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.756152   698 raft_consensus.cc:385] T 6653d3e90150480daf2486b331be3130 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:33.756464   698 raft_consensus.cc:740] T 6653d3e90150480daf2486b331be3130 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f35a78aeac02496d88fa800153754b0a, State: Initialized, Role: FOLLOWER
I20260709 13:12:33.759486   732 leader_election.cc:290] T 82ac3d8dfa714f66a589173f6a9eb004 P 7d13b2cad49e4853831a427132e294d3 [CANDIDATE]: Term 1 election: Requested vote from peers f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881), 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181)
I20260709 13:12:33.759860   708 leader_election.cc:290] T 3b24ef3f665449c288b2fcd28c288d27 P 7d13b2cad49e4853831a427132e294d3 [CANDIDATE]: Term 1 election: Requested vote from peers f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881), 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181)
I20260709 13:12:33.759565   509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24ef3f665449c288b2fcd28c288d27" candidate_uuid: "7d13b2cad49e4853831a427132e294d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f35a78aeac02496d88fa800153754b0a"
I20260709 13:12:33.760999   583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ac3d8dfa714f66a589173f6a9eb004" candidate_uuid: "7d13b2cad49e4853831a427132e294d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15c6b88567e44da395f245725ffe7f04"
I20260709 13:12:33.761447   584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24ef3f665449c288b2fcd28c288d27" candidate_uuid: "7d13b2cad49e4853831a427132e294d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15c6b88567e44da395f245725ffe7f04"
I20260709 13:12:33.765242   508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ac3d8dfa714f66a589173f6a9eb004" candidate_uuid: "7d13b2cad49e4853831a427132e294d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f35a78aeac02496d88fa800153754b0a"
I20260709 13:12:33.766848   508 raft_consensus.cc:3060] T 82ac3d8dfa714f66a589173f6a9eb004 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:33.767218   703 raft_consensus.cc:515] T 82ac3d8dfa714f66a589173f6a9eb004 P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
W20260709 13:12:33.767704   621 leader_election.cc:343] T 3b24ef3f665449c288b2fcd28c288d27 P 7d13b2cad49e4853831a427132e294d3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:33.768617   583 raft_consensus.cc:2393] T 82ac3d8dfa714f66a589173f6a9eb004 P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 7d13b2cad49e4853831a427132e294d3 in current term 1: Already voted for candidate 15c6b88567e44da395f245725ffe7f04 in this term.
I20260709 13:12:33.766160   509 raft_consensus.cc:3060] T 3b24ef3f665449c288b2fcd28c288d27 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:33.769819   698 consensus_queue.cc:260] T 6653d3e90150480daf2486b331be3130 P f35a78aeac02496d88fa800153754b0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.771150   507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ac3d8dfa714f66a589173f6a9eb004" candidate_uuid: "15c6b88567e44da395f245725ffe7f04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f35a78aeac02496d88fa800153754b0a"
I20260709 13:12:33.773391   703 leader_election.cc:290] T 82ac3d8dfa714f66a589173f6a9eb004 P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 1 election: Requested vote from peers f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881), 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695)
I20260709 13:12:33.774156   698 ts_tablet_manager.cc:1434] T 6653d3e90150480daf2486b331be3130 P f35a78aeac02496d88fa800153754b0a: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.000s
I20260709 13:12:33.774928   657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ac3d8dfa714f66a589173f6a9eb004" candidate_uuid: "15c6b88567e44da395f245725ffe7f04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d13b2cad49e4853831a427132e294d3"
I20260709 13:12:33.776175   657 raft_consensus.cc:2393] T 82ac3d8dfa714f66a589173f6a9eb004 P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 15c6b88567e44da395f245725ffe7f04 in current term 1: Already voted for candidate 7d13b2cad49e4853831a427132e294d3 in this term.
I20260709 13:12:33.777495   728 raft_consensus.cc:515] T 6e5c1404cee54b96a9c3a8c024493223 P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.777949   550 leader_election.cc:304] T 82ac3d8dfa714f66a589173f6a9eb004 P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 15c6b88567e44da395f245725ffe7f04; no voters: 7d13b2cad49e4853831a427132e294d3, f35a78aeac02496d88fa800153754b0a
I20260709 13:12:33.779129   703 raft_consensus.cc:2749] T 82ac3d8dfa714f66a589173f6a9eb004 P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:12:33.775321   698 tablet_bootstrap.cc:492] T 93236792b8284277a935d6e569d4c96b P f35a78aeac02496d88fa800153754b0a: Bootstrap starting.
I20260709 13:12:33.780390   583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e5c1404cee54b96a9c3a8c024493223" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15c6b88567e44da395f245725ffe7f04"
I20260709 13:12:33.781015   583 raft_consensus.cc:3060] T 6e5c1404cee54b96a9c3a8c024493223 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:33.786929   698 tablet_bootstrap.cc:654] T 93236792b8284277a935d6e569d4c96b P f35a78aeac02496d88fa800153754b0a: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:33.787307   728 leader_election.cc:290] T 6e5c1404cee54b96a9c3a8c024493223 P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 election: Requested vote from peers 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181), 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695)
I20260709 13:12:33.788379   657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e5c1404cee54b96a9c3a8c024493223" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d13b2cad49e4853831a427132e294d3"
W20260709 13:12:33.789839   475 leader_election.cc:343] T 6e5c1404cee54b96a9c3a8c024493223 P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:33.792321   583 raft_consensus.cc:2468] T 6e5c1404cee54b96a9c3a8c024493223 P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f35a78aeac02496d88fa800153754b0a in term 1.
I20260709 13:12:33.796020   472 leader_election.cc:304] T 6e5c1404cee54b96a9c3a8c024493223 P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 15c6b88567e44da395f245725ffe7f04, f35a78aeac02496d88fa800153754b0a; no voters: 7d13b2cad49e4853831a427132e294d3
I20260709 13:12:33.797127   728 raft_consensus.cc:2804] T 6e5c1404cee54b96a9c3a8c024493223 P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:33.797619   728 raft_consensus.cc:697] T 6e5c1404cee54b96a9c3a8c024493223 P f35a78aeac02496d88fa800153754b0a [term 1 LEADER]: Becoming Leader. State: Replica: f35a78aeac02496d88fa800153754b0a, State: Running, Role: LEADER
I20260709 13:12:33.798434   728 consensus_queue.cc:237] T 6e5c1404cee54b96a9c3a8c024493223 P f35a78aeac02496d88fa800153754b0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.799820   702 tablet_bootstrap.cc:492] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3: No bootstrap required, opened a new log
I20260709 13:12:33.800536   702 ts_tablet_manager.cc:1403] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3: Time spent bootstrapping tablet: real 0.070s	user 0.015s	sys 0.010s
I20260709 13:12:33.796399   699 tablet_bootstrap.cc:492] T 93236792b8284277a935d6e569d4c96b P 15c6b88567e44da395f245725ffe7f04: No bootstrap required, opened a new log
I20260709 13:12:33.801846   699 ts_tablet_manager.cc:1403] T 93236792b8284277a935d6e569d4c96b P 15c6b88567e44da395f245725ffe7f04: Time spent bootstrapping tablet: real 0.109s	user 0.021s	sys 0.016s
I20260709 13:12:33.804713   702 raft_consensus.cc:359] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } }
I20260709 13:12:33.805817   702 raft_consensus.cc:385] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:33.806185   702 raft_consensus.cc:740] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d13b2cad49e4853831a427132e294d3, State: Initialized, Role: FOLLOWER
I20260709 13:12:33.808538   702 consensus_queue.cc:260] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } }
I20260709 13:12:33.811808   702 ts_tablet_manager.cc:1434] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.003s
I20260709 13:12:33.813050   702 tablet_bootstrap.cc:492] T 93236792b8284277a935d6e569d4c96b P 7d13b2cad49e4853831a427132e294d3: Bootstrap starting.
I20260709 13:12:33.813403   699 raft_consensus.cc:359] T 93236792b8284277a935d6e569d4c96b P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.814265   699 raft_consensus.cc:385] T 93236792b8284277a935d6e569d4c96b P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:33.814584   699 raft_consensus.cc:740] T 93236792b8284277a935d6e569d4c96b P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15c6b88567e44da395f245725ffe7f04, State: Initialized, Role: FOLLOWER
I20260709 13:12:33.814927   698 tablet_bootstrap.cc:492] T 93236792b8284277a935d6e569d4c96b P f35a78aeac02496d88fa800153754b0a: No bootstrap required, opened a new log
I20260709 13:12:33.815536   698 ts_tablet_manager.cc:1403] T 93236792b8284277a935d6e569d4c96b P f35a78aeac02496d88fa800153754b0a: Time spent bootstrapping tablet: real 0.040s	user 0.015s	sys 0.012s
I20260709 13:12:33.815439   699 consensus_queue.cc:260] T 93236792b8284277a935d6e569d4c96b P 15c6b88567e44da395f245725ffe7f04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.819365   698 raft_consensus.cc:359] T 93236792b8284277a935d6e569d4c96b P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.820251   698 raft_consensus.cc:385] T 93236792b8284277a935d6e569d4c96b P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:33.820612   698 raft_consensus.cc:740] T 93236792b8284277a935d6e569d4c96b P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f35a78aeac02496d88fa800153754b0a, State: Initialized, Role: FOLLOWER
I20260709 13:12:33.821530   702 tablet_bootstrap.cc:654] T 93236792b8284277a935d6e569d4c96b P 7d13b2cad49e4853831a427132e294d3: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:33.822304   698 consensus_queue.cc:260] T 93236792b8284277a935d6e569d4c96b P f35a78aeac02496d88fa800153754b0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.825487   699 ts_tablet_manager.cc:1434] T 93236792b8284277a935d6e569d4c96b P 15c6b88567e44da395f245725ffe7f04: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 13:12:33.825616   698 ts_tablet_manager.cc:1434] T 93236792b8284277a935d6e569d4c96b P f35a78aeac02496d88fa800153754b0a: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:12:33.826723   699 tablet_bootstrap.cc:492] T 6653d3e90150480daf2486b331be3130 P 15c6b88567e44da395f245725ffe7f04: Bootstrap starting.
I20260709 13:12:33.827401   698 tablet_bootstrap.cc:492] T bf7b037540fc46d098bf7e1888e9c682 P f35a78aeac02496d88fa800153754b0a: Bootstrap starting.
I20260709 13:12:33.792321   508 raft_consensus.cc:2468] T 82ac3d8dfa714f66a589173f6a9eb004 P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d13b2cad49e4853831a427132e294d3 in term 1.
I20260709 13:12:33.830708   703 raft_consensus.cc:493] T 3723f5ab8f974d1892ce711b33fac1dd P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:33.831247   703 raft_consensus.cc:515] T 3723f5ab8f974d1892ce711b33fac1dd P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.833499   624 leader_election.cc:304] T 82ac3d8dfa714f66a589173f6a9eb004 P 7d13b2cad49e4853831a427132e294d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d13b2cad49e4853831a427132e294d3, f35a78aeac02496d88fa800153754b0a; no voters: 15c6b88567e44da395f245725ffe7f04
I20260709 13:12:33.833845   509 raft_consensus.cc:2468] T 3b24ef3f665449c288b2fcd28c288d27 P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d13b2cad49e4853831a427132e294d3 in term 1.
I20260709 13:12:33.834774   703 leader_election.cc:290] T 3723f5ab8f974d1892ce711b33fac1dd P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881), 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695)
I20260709 13:12:33.835103   657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3723f5ab8f974d1892ce711b33fac1dd" candidate_uuid: "15c6b88567e44da395f245725ffe7f04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d13b2cad49e4853831a427132e294d3" is_pre_election: true
I20260709 13:12:33.837412   708 raft_consensus.cc:2804] T 82ac3d8dfa714f66a589173f6a9eb004 P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:33.841377   508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3723f5ab8f974d1892ce711b33fac1dd" candidate_uuid: "15c6b88567e44da395f245725ffe7f04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f35a78aeac02496d88fa800153754b0a" is_pre_election: true
I20260709 13:12:33.841827   657 raft_consensus.cc:2468] T 3723f5ab8f974d1892ce711b33fac1dd P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15c6b88567e44da395f245725ffe7f04 in term 0.
W20260709 13:12:33.844894   550 leader_election.cc:343] T 3723f5ab8f974d1892ce711b33fac1dd P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:33.845604   708 raft_consensus.cc:697] T 82ac3d8dfa714f66a589173f6a9eb004 P 7d13b2cad49e4853831a427132e294d3 [term 1 LEADER]: Becoming Leader. State: Replica: 7d13b2cad49e4853831a427132e294d3, State: Running, Role: LEADER
I20260709 13:12:33.846786   708 consensus_queue.cc:237] T 82ac3d8dfa714f66a589173f6a9eb004 P 7d13b2cad49e4853831a427132e294d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.847534   698 tablet_bootstrap.cc:654] T bf7b037540fc46d098bf7e1888e9c682 P f35a78aeac02496d88fa800153754b0a: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:33.845467   624 leader_election.cc:304] T 3b24ef3f665449c288b2fcd28c288d27 P 7d13b2cad49e4853831a427132e294d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d13b2cad49e4853831a427132e294d3, f35a78aeac02496d88fa800153754b0a; no voters: 15c6b88567e44da395f245725ffe7f04
I20260709 13:12:33.849867   732 raft_consensus.cc:2804] T 3b24ef3f665449c288b2fcd28c288d27 P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:33.850446   732 raft_consensus.cc:697] T 3b24ef3f665449c288b2fcd28c288d27 P 7d13b2cad49e4853831a427132e294d3 [term 1 LEADER]: Becoming Leader. State: Replica: 7d13b2cad49e4853831a427132e294d3, State: Running, Role: LEADER
I20260709 13:12:33.851425   732 consensus_queue.cc:237] T 3b24ef3f665449c288b2fcd28c288d27 P 7d13b2cad49e4853831a427132e294d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.851958   728 raft_consensus.cc:493] T 93236792b8284277a935d6e569d4c96b P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:33.852140   699 tablet_bootstrap.cc:654] T 6653d3e90150480daf2486b331be3130 P 15c6b88567e44da395f245725ffe7f04: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:33.852485   728 raft_consensus.cc:515] T 93236792b8284277a935d6e569d4c96b P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.853423   698 tablet_bootstrap.cc:492] T bf7b037540fc46d098bf7e1888e9c682 P f35a78aeac02496d88fa800153754b0a: No bootstrap required, opened a new log
I20260709 13:12:33.853950   698 ts_tablet_manager.cc:1403] T bf7b037540fc46d098bf7e1888e9c682 P f35a78aeac02496d88fa800153754b0a: Time spent bootstrapping tablet: real 0.027s	user 0.014s	sys 0.000s
I20260709 13:12:33.854061   550 leader_election.cc:304] T 3723f5ab8f974d1892ce711b33fac1dd P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 15c6b88567e44da395f245725ffe7f04, 7d13b2cad49e4853831a427132e294d3; no voters: f35a78aeac02496d88fa800153754b0a
I20260709 13:12:33.854704   728 leader_election.cc:290] T 93236792b8284277a935d6e569d4c96b P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181), 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695)
I20260709 13:12:33.855276   703 raft_consensus.cc:2804] T 3723f5ab8f974d1892ce711b33fac1dd P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:33.855480   583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93236792b8284277a935d6e569d4c96b" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15c6b88567e44da395f245725ffe7f04" is_pre_election: true
I20260709 13:12:33.855818   703 raft_consensus.cc:493] T 3723f5ab8f974d1892ce711b33fac1dd P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:33.856158   583 raft_consensus.cc:2468] T 93236792b8284277a935d6e569d4c96b P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f35a78aeac02496d88fa800153754b0a in term 0.
I20260709 13:12:33.856184   657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93236792b8284277a935d6e569d4c96b" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d13b2cad49e4853831a427132e294d3" is_pre_election: true
I20260709 13:12:33.856287   703 raft_consensus.cc:3060] T 3723f5ab8f974d1892ce711b33fac1dd P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:33.857795   472 leader_election.cc:304] T 93236792b8284277a935d6e569d4c96b P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15c6b88567e44da395f245725ffe7f04, f35a78aeac02496d88fa800153754b0a; no voters: 
I20260709 13:12:33.859429   728 raft_consensus.cc:2804] T 93236792b8284277a935d6e569d4c96b P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:33.859871   728 raft_consensus.cc:493] T 93236792b8284277a935d6e569d4c96b P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:33.860263   728 raft_consensus.cc:3060] T 93236792b8284277a935d6e569d4c96b P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:33.861464   698 raft_consensus.cc:359] T bf7b037540fc46d098bf7e1888e9c682 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.862246   698 raft_consensus.cc:385] T bf7b037540fc46d098bf7e1888e9c682 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:33.862557   698 raft_consensus.cc:740] T bf7b037540fc46d098bf7e1888e9c682 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f35a78aeac02496d88fa800153754b0a, State: Initialized, Role: FOLLOWER
I20260709 13:12:33.863474   698 consensus_queue.cc:260] T bf7b037540fc46d098bf7e1888e9c682 P f35a78aeac02496d88fa800153754b0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.866051   698 ts_tablet_manager.cc:1434] T bf7b037540fc46d098bf7e1888e9c682 P f35a78aeac02496d88fa800153754b0a: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:12:33.865177   703 raft_consensus.cc:515] T 3723f5ab8f974d1892ce711b33fac1dd P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.867154   698 tablet_bootstrap.cc:492] T 3723f5ab8f974d1892ce711b33fac1dd P f35a78aeac02496d88fa800153754b0a: Bootstrap starting.
I20260709 13:12:33.867697   728 raft_consensus.cc:515] T 93236792b8284277a935d6e569d4c96b P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
W20260709 13:12:33.869377   475 leader_election.cc:343] T 93236792b8284277a935d6e569d4c96b P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:33.869313   508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3723f5ab8f974d1892ce711b33fac1dd" candidate_uuid: "15c6b88567e44da395f245725ffe7f04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f35a78aeac02496d88fa800153754b0a"
I20260709 13:12:33.870038   728 leader_election.cc:290] T 93236792b8284277a935d6e569d4c96b P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 election: Requested vote from peers 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181), 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695)
I20260709 13:12:33.870551   657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3723f5ab8f974d1892ce711b33fac1dd" candidate_uuid: "15c6b88567e44da395f245725ffe7f04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d13b2cad49e4853831a427132e294d3"
I20260709 13:12:33.871146   657 raft_consensus.cc:3060] T 3723f5ab8f974d1892ce711b33fac1dd P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:33.871405   658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93236792b8284277a935d6e569d4c96b" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d13b2cad49e4853831a427132e294d3"
W20260709 13:12:33.873023   475 leader_election.cc:343] T 93236792b8284277a935d6e569d4c96b P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:33.875370   698 tablet_bootstrap.cc:654] T 3723f5ab8f974d1892ce711b33fac1dd P f35a78aeac02496d88fa800153754b0a: Neither blocks nor log segments found. Creating new log.
W20260709 13:12:33.878146   550 leader_election.cc:343] T 3723f5ab8f974d1892ce711b33fac1dd P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:33.878224   583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93236792b8284277a935d6e569d4c96b" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15c6b88567e44da395f245725ffe7f04"
I20260709 13:12:33.879238   583 raft_consensus.cc:3060] T 93236792b8284277a935d6e569d4c96b P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:33.882691   657 raft_consensus.cc:2468] T 3723f5ab8f974d1892ce711b33fac1dd P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15c6b88567e44da395f245725ffe7f04 in term 1.
I20260709 13:12:33.884063   550 leader_election.cc:304] T 3723f5ab8f974d1892ce711b33fac1dd P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 15c6b88567e44da395f245725ffe7f04, 7d13b2cad49e4853831a427132e294d3; no voters: f35a78aeac02496d88fa800153754b0a
I20260709 13:12:33.885085   724 raft_consensus.cc:2804] T 3723f5ab8f974d1892ce711b33fac1dd P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:33.886219   724 raft_consensus.cc:697] T 3723f5ab8f974d1892ce711b33fac1dd P 15c6b88567e44da395f245725ffe7f04 [term 1 LEADER]: Becoming Leader. State: Replica: 15c6b88567e44da395f245725ffe7f04, State: Running, Role: LEADER
I20260709 13:12:33.887257   724 consensus_queue.cc:237] T 3723f5ab8f974d1892ce711b33fac1dd P 15c6b88567e44da395f245725ffe7f04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.889372   583 raft_consensus.cc:2468] T 93236792b8284277a935d6e569d4c96b P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f35a78aeac02496d88fa800153754b0a in term 1.
I20260709 13:12:33.891302   698 tablet_bootstrap.cc:492] T 3723f5ab8f974d1892ce711b33fac1dd P f35a78aeac02496d88fa800153754b0a: No bootstrap required, opened a new log
I20260709 13:12:33.891728   703 leader_election.cc:290] T 3723f5ab8f974d1892ce711b33fac1dd P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 1 election: Requested vote from peers f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881), 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695)
I20260709 13:12:33.891891   698 ts_tablet_manager.cc:1403] T 3723f5ab8f974d1892ce711b33fac1dd P f35a78aeac02496d88fa800153754b0a: Time spent bootstrapping tablet: real 0.025s	user 0.014s	sys 0.004s
I20260709 13:12:33.899377   699 tablet_bootstrap.cc:492] T 6653d3e90150480daf2486b331be3130 P 15c6b88567e44da395f245725ffe7f04: No bootstrap required, opened a new log
I20260709 13:12:33.900082   699 ts_tablet_manager.cc:1403] T 6653d3e90150480daf2486b331be3130 P 15c6b88567e44da395f245725ffe7f04: Time spent bootstrapping tablet: real 0.074s	user 0.015s	sys 0.003s
I20260709 13:12:33.900511   698 raft_consensus.cc:359] T 3723f5ab8f974d1892ce711b33fac1dd P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.901279   698 raft_consensus.cc:385] T 3723f5ab8f974d1892ce711b33fac1dd P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:33.901691   698 raft_consensus.cc:740] T 3723f5ab8f974d1892ce711b33fac1dd P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f35a78aeac02496d88fa800153754b0a, State: Initialized, Role: FOLLOWER
I20260709 13:12:33.902559   698 consensus_queue.cc:260] T 3723f5ab8f974d1892ce711b33fac1dd P f35a78aeac02496d88fa800153754b0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.903962   699 raft_consensus.cc:359] T 6653d3e90150480daf2486b331be3130 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.904832   698 ts_tablet_manager.cc:1434] T 3723f5ab8f974d1892ce711b33fac1dd P f35a78aeac02496d88fa800153754b0a: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 13:12:33.904847   699 raft_consensus.cc:385] T 6653d3e90150480daf2486b331be3130 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:33.905474   699 raft_consensus.cc:740] T 6653d3e90150480daf2486b331be3130 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15c6b88567e44da395f245725ffe7f04, State: Initialized, Role: FOLLOWER
I20260709 13:12:33.906255   699 consensus_queue.cc:260] T 6653d3e90150480daf2486b331be3130 P 15c6b88567e44da395f245725ffe7f04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:33.908980   699 ts_tablet_manager.cc:1434] T 6653d3e90150480daf2486b331be3130 P 15c6b88567e44da395f245725ffe7f04: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 13:12:33.910091   699 tablet_bootstrap.cc:492] T 3b24ef3f665449c288b2fcd28c288d27 P 15c6b88567e44da395f245725ffe7f04: Bootstrap starting.
I20260709 13:12:33.955323   702 tablet_bootstrap.cc:492] T 93236792b8284277a935d6e569d4c96b P 7d13b2cad49e4853831a427132e294d3: No bootstrap required, opened a new log
I20260709 13:12:33.956034   702 ts_tablet_manager.cc:1403] T 93236792b8284277a935d6e569d4c96b P 7d13b2cad49e4853831a427132e294d3: Time spent bootstrapping tablet: real 0.143s	user 0.011s	sys 0.009s
I20260709 13:12:33.994050   405 catalog_manager.cc:5697] T 6e5c1404cee54b96a9c3a8c024493223 P f35a78aeac02496d88fa800153754b0a reported cstate change: term changed from 0 to 1, leader changed from <none> to f35a78aeac02496d88fa800153754b0a (127.21.63.1). New cstate: current_term: 1 leader_uuid: "f35a78aeac02496d88fa800153754b0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:33.999290   472 leader_election.cc:304] T 93236792b8284277a935d6e569d4c96b P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 15c6b88567e44da395f245725ffe7f04, f35a78aeac02496d88fa800153754b0a; no voters: 7d13b2cad49e4853831a427132e294d3
I20260709 13:12:34.027716   699 tablet_bootstrap.cc:654] T 3b24ef3f665449c288b2fcd28c288d27 P 15c6b88567e44da395f245725ffe7f04: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:34.033413   699 tablet_bootstrap.cc:492] T 3b24ef3f665449c288b2fcd28c288d27 P 15c6b88567e44da395f245725ffe7f04: No bootstrap required, opened a new log
I20260709 13:12:34.034034   699 ts_tablet_manager.cc:1403] T 3b24ef3f665449c288b2fcd28c288d27 P 15c6b88567e44da395f245725ffe7f04: Time spent bootstrapping tablet: real 0.124s	user 0.011s	sys 0.007s
I20260709 13:12:34.038120   699 raft_consensus.cc:359] T 3b24ef3f665449c288b2fcd28c288d27 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:34.038837   699 raft_consensus.cc:385] T 3b24ef3f665449c288b2fcd28c288d27 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:34.039105   699 raft_consensus.cc:740] T 3b24ef3f665449c288b2fcd28c288d27 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15c6b88567e44da395f245725ffe7f04, State: Initialized, Role: FOLLOWER
I20260709 13:12:34.040014   699 consensus_queue.cc:260] T 3b24ef3f665449c288b2fcd28c288d27 P 15c6b88567e44da395f245725ffe7f04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:34.041115   728 raft_consensus.cc:493] T 6653d3e90150480daf2486b331be3130 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:34.041738   728 raft_consensus.cc:515] T 6653d3e90150480daf2486b331be3130 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:34.042542   699 ts_tablet_manager.cc:1434] T 3b24ef3f665449c288b2fcd28c288d27 P 15c6b88567e44da395f245725ffe7f04: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 13:12:34.043637   699 tablet_bootstrap.cc:492] T a13e0fbeefb142f6bf766c1509cd1d93 P 15c6b88567e44da395f245725ffe7f04: Bootstrap starting.
I20260709 13:12:34.043959   728 leader_election.cc:290] T 6653d3e90150480daf2486b331be3130 P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181), 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695)
I20260709 13:12:34.044548   728 raft_consensus.cc:2804] T 93236792b8284277a935d6e569d4c96b P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:34.045008   728 raft_consensus.cc:697] T 93236792b8284277a935d6e569d4c96b P f35a78aeac02496d88fa800153754b0a [term 1 LEADER]: Becoming Leader. State: Replica: f35a78aeac02496d88fa800153754b0a, State: Running, Role: LEADER
I20260709 13:12:34.045907   728 consensus_queue.cc:237] T 93236792b8284277a935d6e569d4c96b P f35a78aeac02496d88fa800153754b0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:34.052896   699 tablet_bootstrap.cc:654] T a13e0fbeefb142f6bf766c1509cd1d93 P 15c6b88567e44da395f245725ffe7f04: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:34.061574   702 raft_consensus.cc:359] T 93236792b8284277a935d6e569d4c96b P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:34.062378   702 raft_consensus.cc:385] T 93236792b8284277a935d6e569d4c96b P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:34.062738   702 raft_consensus.cc:740] T 93236792b8284277a935d6e569d4c96b P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d13b2cad49e4853831a427132e294d3, State: Initialized, Role: FOLLOWER
I20260709 13:12:34.063663   702 consensus_queue.cc:260] T 93236792b8284277a935d6e569d4c96b P 7d13b2cad49e4853831a427132e294d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:34.064222   699 tablet_bootstrap.cc:492] T a13e0fbeefb142f6bf766c1509cd1d93 P 15c6b88567e44da395f245725ffe7f04: No bootstrap required, opened a new log
I20260709 13:12:34.064792   699 ts_tablet_manager.cc:1403] T a13e0fbeefb142f6bf766c1509cd1d93 P 15c6b88567e44da395f245725ffe7f04: Time spent bootstrapping tablet: real 0.021s	user 0.012s	sys 0.007s
I20260709 13:12:34.067394   699 raft_consensus.cc:359] T a13e0fbeefb142f6bf766c1509cd1d93 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } }
I20260709 13:12:34.067922   699 raft_consensus.cc:385] T a13e0fbeefb142f6bf766c1509cd1d93 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:34.068203   699 raft_consensus.cc:740] T a13e0fbeefb142f6bf766c1509cd1d93 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15c6b88567e44da395f245725ffe7f04, State: Initialized, Role: FOLLOWER
I20260709 13:12:34.069168   699 consensus_queue.cc:260] T a13e0fbeefb142f6bf766c1509cd1d93 P 15c6b88567e44da395f245725ffe7f04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } }
I20260709 13:12:34.071189   699 ts_tablet_manager.cc:1434] T a13e0fbeefb142f6bf766c1509cd1d93 P 15c6b88567e44da395f245725ffe7f04: Time spent starting tablet: real 0.006s	user 0.001s	sys 0.003s
I20260709 13:12:34.071964   699 tablet_bootstrap.cc:492] T e6a5dbc01970406599bc229548544689 P 15c6b88567e44da395f245725ffe7f04: Bootstrap starting.
I20260709 13:12:34.073753   702 ts_tablet_manager.cc:1434] T 93236792b8284277a935d6e569d4c96b P 7d13b2cad49e4853831a427132e294d3: Time spent starting tablet: real 0.117s	user 0.010s	sys 0.000s
I20260709 13:12:34.074869   702 tablet_bootstrap.cc:492] T 6e5c1404cee54b96a9c3a8c024493223 P 7d13b2cad49e4853831a427132e294d3: Bootstrap starting.
I20260709 13:12:34.078033   699 tablet_bootstrap.cc:654] T e6a5dbc01970406599bc229548544689 P 15c6b88567e44da395f245725ffe7f04: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:34.080725   702 tablet_bootstrap.cc:654] T 6e5c1404cee54b96a9c3a8c024493223 P 7d13b2cad49e4853831a427132e294d3: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:34.082978   699 tablet_bootstrap.cc:492] T e6a5dbc01970406599bc229548544689 P 15c6b88567e44da395f245725ffe7f04: No bootstrap required, opened a new log
I20260709 13:12:34.083461   699 ts_tablet_manager.cc:1403] T e6a5dbc01970406599bc229548544689 P 15c6b88567e44da395f245725ffe7f04: Time spent bootstrapping tablet: real 0.012s	user 0.004s	sys 0.004s
I20260709 13:12:34.086323   699 raft_consensus.cc:359] T e6a5dbc01970406599bc229548544689 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:34.086757   702 tablet_bootstrap.cc:492] T 6e5c1404cee54b96a9c3a8c024493223 P 7d13b2cad49e4853831a427132e294d3: No bootstrap required, opened a new log
I20260709 13:12:34.086997   699 raft_consensus.cc:385] T e6a5dbc01970406599bc229548544689 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:34.087368   699 raft_consensus.cc:740] T e6a5dbc01970406599bc229548544689 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15c6b88567e44da395f245725ffe7f04, State: Initialized, Role: FOLLOWER
I20260709 13:12:34.087426   702 ts_tablet_manager.cc:1403] T 6e5c1404cee54b96a9c3a8c024493223 P 7d13b2cad49e4853831a427132e294d3: Time spent bootstrapping tablet: real 0.013s	user 0.011s	sys 0.000s
I20260709 13:12:34.088135   699 consensus_queue.cc:260] T e6a5dbc01970406599bc229548544689 P 15c6b88567e44da395f245725ffe7f04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:34.090066   699 ts_tablet_manager.cc:1434] T e6a5dbc01970406599bc229548544689 P 15c6b88567e44da395f245725ffe7f04: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:12:34.091414   702 raft_consensus.cc:359] T 6e5c1404cee54b96a9c3a8c024493223 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:34.092128   702 raft_consensus.cc:385] T 6e5c1404cee54b96a9c3a8c024493223 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:34.092382   702 raft_consensus.cc:740] T 6e5c1404cee54b96a9c3a8c024493223 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d13b2cad49e4853831a427132e294d3, State: Initialized, Role: FOLLOWER
I20260709 13:12:34.093076   702 consensus_queue.cc:260] T 6e5c1404cee54b96a9c3a8c024493223 P 7d13b2cad49e4853831a427132e294d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:34.095422   702 ts_tablet_manager.cc:1434] T 6e5c1404cee54b96a9c3a8c024493223 P 7d13b2cad49e4853831a427132e294d3: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 13:12:34.096997   726 raft_consensus.cc:493] T bf7b037540fc46d098bf7e1888e9c682 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:34.097541   726 raft_consensus.cc:515] T bf7b037540fc46d098bf7e1888e9c682 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:34.098456   728 consensus_queue.cc:1048] T e6a5dbc01970406599bc229548544689 P f35a78aeac02496d88fa800153754b0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:34.099697   726 leader_election.cc:290] T bf7b037540fc46d098bf7e1888e9c682 P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181), 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695)
I20260709 13:12:34.100757   584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf7b037540fc46d098bf7e1888e9c682" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15c6b88567e44da395f245725ffe7f04" is_pre_election: true
I20260709 13:12:34.102993   658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf7b037540fc46d098bf7e1888e9c682" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d13b2cad49e4853831a427132e294d3" is_pre_election: true
I20260709 13:12:34.103981   658 raft_consensus.cc:2393] T bf7b037540fc46d098bf7e1888e9c682 P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f35a78aeac02496d88fa800153754b0a in current term 1: Already voted for candidate 15c6b88567e44da395f245725ffe7f04 in this term.
I20260709 13:12:34.108371   475 leader_election.cc:304] T bf7b037540fc46d098bf7e1888e9c682 P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f35a78aeac02496d88fa800153754b0a; no voters: 15c6b88567e44da395f245725ffe7f04, 7d13b2cad49e4853831a427132e294d3
I20260709 13:12:34.109620   508 raft_consensus.cc:3060] T bf7b037540fc46d098bf7e1888e9c682 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:34.112983   732 raft_consensus.cc:493] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:34.113547   732 raft_consensus.cc:515] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } }
I20260709 13:12:34.114526   583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6653d3e90150480daf2486b331be3130" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15c6b88567e44da395f245725ffe7f04" is_pre_election: true
I20260709 13:12:34.115347   583 raft_consensus.cc:2468] T 6653d3e90150480daf2486b331be3130 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f35a78aeac02496d88fa800153754b0a in term 0.
I20260709 13:12:34.115840   732 leader_election.cc:290] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881), 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181)
I20260709 13:12:34.117396   472 leader_election.cc:304] T 6653d3e90150480daf2486b331be3130 P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15c6b88567e44da395f245725ffe7f04, f35a78aeac02496d88fa800153754b0a; no voters: 
I20260709 13:12:34.118319   728 raft_consensus.cc:2804] T 6653d3e90150480daf2486b331be3130 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:34.118573   657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6653d3e90150480daf2486b331be3130" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d13b2cad49e4853831a427132e294d3" is_pre_election: true
I20260709 13:12:34.118726   728 raft_consensus.cc:493] T 6653d3e90150480daf2486b331be3130 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:34.119269   657 raft_consensus.cc:2468] T 6653d3e90150480daf2486b331be3130 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f35a78aeac02496d88fa800153754b0a in term 0.
I20260709 13:12:34.118687   509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a13e0fbeefb142f6bf766c1509cd1d93" candidate_uuid: "7d13b2cad49e4853831a427132e294d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f35a78aeac02496d88fa800153754b0a" is_pre_election: true
I20260709 13:12:34.120023   509 raft_consensus.cc:2468] T a13e0fbeefb142f6bf766c1509cd1d93 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d13b2cad49e4853831a427132e294d3 in term 0.
I20260709 13:12:34.121115   583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a13e0fbeefb142f6bf766c1509cd1d93" candidate_uuid: "7d13b2cad49e4853831a427132e294d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15c6b88567e44da395f245725ffe7f04" is_pre_election: true
I20260709 13:12:34.119313   728 raft_consensus.cc:3060] T 6653d3e90150480daf2486b331be3130 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:34.121934   624 leader_election.cc:304] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d13b2cad49e4853831a427132e294d3, f35a78aeac02496d88fa800153754b0a; no voters: 
I20260709 13:12:34.122135   583 raft_consensus.cc:2468] T a13e0fbeefb142f6bf766c1509cd1d93 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d13b2cad49e4853831a427132e294d3 in term 0.
I20260709 13:12:34.123392   732 raft_consensus.cc:2804] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:34.123772   732 raft_consensus.cc:493] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:34.124107   732 raft_consensus.cc:3060] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:34.129345   728 raft_consensus.cc:515] T 6653d3e90150480daf2486b331be3130 P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:34.132398   584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6653d3e90150480daf2486b331be3130" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15c6b88567e44da395f245725ffe7f04"
I20260709 13:12:34.133013   584 raft_consensus.cc:3060] T 6653d3e90150480daf2486b331be3130 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:34.135610   404 catalog_manager.cc:5697] T 3723f5ab8f974d1892ce711b33fac1dd P 15c6b88567e44da395f245725ffe7f04 reported cstate change: term changed from 0 to 1, leader changed from <none> to 15c6b88567e44da395f245725ffe7f04 (127.21.63.2). New cstate: current_term: 1 leader_uuid: "15c6b88567e44da395f245725ffe7f04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:34.137085   732 raft_consensus.cc:515] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } }
I20260709 13:12:34.140961   509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a13e0fbeefb142f6bf766c1509cd1d93" candidate_uuid: "7d13b2cad49e4853831a427132e294d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f35a78aeac02496d88fa800153754b0a"
I20260709 13:12:34.141536   509 raft_consensus.cc:3060] T a13e0fbeefb142f6bf766c1509cd1d93 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:34.141618   732 leader_election.cc:290] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3 [CANDIDATE]: Term 1 election: Requested vote from peers f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881), 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181)
I20260709 13:12:34.148600   509 raft_consensus.cc:2468] T a13e0fbeefb142f6bf766c1509cd1d93 P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d13b2cad49e4853831a427132e294d3 in term 1.
I20260709 13:12:34.150068   624 leader_election.cc:304] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d13b2cad49e4853831a427132e294d3, f35a78aeac02496d88fa800153754b0a; no voters: 
I20260709 13:12:34.152149   732 raft_consensus.cc:2804] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:34.152776   732 raft_consensus.cc:697] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3 [term 1 LEADER]: Becoming Leader. State: Replica: 7d13b2cad49e4853831a427132e294d3, State: Running, Role: LEADER
I20260709 13:12:34.152913   658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6653d3e90150480daf2486b331be3130" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d13b2cad49e4853831a427132e294d3"
I20260709 13:12:34.153913   658 raft_consensus.cc:3060] T 6653d3e90150480daf2486b331be3130 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:34.154057   732 consensus_queue.cc:237] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } }
I20260709 13:12:34.153184   728 leader_election.cc:290] T 6653d3e90150480daf2486b331be3130 P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 election: Requested vote from peers 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181), 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695)
I20260709 13:12:34.140898   584 raft_consensus.cc:2468] T 6653d3e90150480daf2486b331be3130 P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f35a78aeac02496d88fa800153754b0a in term 1.
I20260709 13:12:34.159472   472 leader_election.cc:304] T 6653d3e90150480daf2486b331be3130 P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15c6b88567e44da395f245725ffe7f04, f35a78aeac02496d88fa800153754b0a; no voters: 
I20260709 13:12:34.160434   728 raft_consensus.cc:2804] T 6653d3e90150480daf2486b331be3130 P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:34.161120   728 raft_consensus.cc:697] T 6653d3e90150480daf2486b331be3130 P f35a78aeac02496d88fa800153754b0a [term 1 LEADER]: Becoming Leader. State: Replica: f35a78aeac02496d88fa800153754b0a, State: Running, Role: LEADER
I20260709 13:12:34.161942   728 consensus_queue.cc:237] T 6653d3e90150480daf2486b331be3130 P f35a78aeac02496d88fa800153754b0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:34.162851   724 consensus_queue.cc:1048] T bf7b037540fc46d098bf7e1888e9c682 P 15c6b88567e44da395f245725ffe7f04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:34.165607   726 raft_consensus.cc:2749] T bf7b037540fc46d098bf7e1888e9c682 P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:34.168309   724 consensus_queue.cc:1048] T bf7b037540fc46d098bf7e1888e9c682 P 15c6b88567e44da395f245725ffe7f04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:34.173676   583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a13e0fbeefb142f6bf766c1509cd1d93" candidate_uuid: "7d13b2cad49e4853831a427132e294d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15c6b88567e44da395f245725ffe7f04"
I20260709 13:12:34.175460   583 raft_consensus.cc:3060] T a13e0fbeefb142f6bf766c1509cd1d93 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:34.184046   658 raft_consensus.cc:2468] T 6653d3e90150480daf2486b331be3130 P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f35a78aeac02496d88fa800153754b0a in term 1.
I20260709 13:12:34.199108   584 raft_consensus.cc:3060] T e6a5dbc01970406599bc229548544689 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:34.203848   703 raft_consensus.cc:493] T 3b24ef3f665449c288b2fcd28c288d27 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:34.204591   703 raft_consensus.cc:515] T 3b24ef3f665449c288b2fcd28c288d27 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:34.208240   726 consensus_queue.cc:1048] T e6a5dbc01970406599bc229548544689 P f35a78aeac02496d88fa800153754b0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:34.208519   509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24ef3f665449c288b2fcd28c288d27" candidate_uuid: "15c6b88567e44da395f245725ffe7f04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f35a78aeac02496d88fa800153754b0a" is_pre_election: true
I20260709 13:12:34.208565   703 leader_election.cc:290] T 3b24ef3f665449c288b2fcd28c288d27 P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881), 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695)
I20260709 13:12:34.209446   509 raft_consensus.cc:2393] T 3b24ef3f665449c288b2fcd28c288d27 P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 15c6b88567e44da395f245725ffe7f04 in current term 1: Already voted for candidate 7d13b2cad49e4853831a427132e294d3 in this term.
I20260709 13:12:34.210995   656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b24ef3f665449c288b2fcd28c288d27" candidate_uuid: "15c6b88567e44da395f245725ffe7f04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d13b2cad49e4853831a427132e294d3" is_pre_election: true
I20260709 13:12:34.212793   550 leader_election.cc:304] T 3b24ef3f665449c288b2fcd28c288d27 P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 15c6b88567e44da395f245725ffe7f04; no voters: 7d13b2cad49e4853831a427132e294d3, f35a78aeac02496d88fa800153754b0a
I20260709 13:12:34.214414   703 raft_consensus.cc:3060] T 3b24ef3f665449c288b2fcd28c288d27 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:34.203567   404 catalog_manager.cc:5697] T 93236792b8284277a935d6e569d4c96b P f35a78aeac02496d88fa800153754b0a reported cstate change: term changed from 0 to 1, leader changed from <none> to f35a78aeac02496d88fa800153754b0a (127.21.63.1). New cstate: current_term: 1 leader_uuid: "f35a78aeac02496d88fa800153754b0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:34.247442   657 raft_consensus.cc:3060] T 6e5c1404cee54b96a9c3a8c024493223 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:34.255524   583 raft_consensus.cc:2468] T a13e0fbeefb142f6bf766c1509cd1d93 P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d13b2cad49e4853831a427132e294d3 in term 1.
I20260709 13:12:34.256192   405 catalog_manager.cc:5697] T 6653d3e90150480daf2486b331be3130 P f35a78aeac02496d88fa800153754b0a reported cstate change: term changed from 0 to 1, leader changed from <none> to f35a78aeac02496d88fa800153754b0a (127.21.63.1). New cstate: current_term: 1 leader_uuid: "f35a78aeac02496d88fa800153754b0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:34.259814   704 consensus_queue.cc:1048] T 6e5c1404cee54b96a9c3a8c024493223 P f35a78aeac02496d88fa800153754b0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:34.268062   703 raft_consensus.cc:2749] T 3b24ef3f665449c288b2fcd28c288d27 P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:34.269068   732 raft_consensus.cc:493] T 93236792b8284277a935d6e569d4c96b P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:34.269783   732 raft_consensus.cc:515] T 93236792b8284277a935d6e569d4c96b P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:34.272975   732 leader_election.cc:290] T 93236792b8284277a935d6e569d4c96b P 7d13b2cad49e4853831a427132e294d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881), 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181)
I20260709 13:12:34.273622   509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93236792b8284277a935d6e569d4c96b" candidate_uuid: "7d13b2cad49e4853831a427132e294d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f35a78aeac02496d88fa800153754b0a" is_pre_election: true
I20260709 13:12:34.276090   584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93236792b8284277a935d6e569d4c96b" candidate_uuid: "7d13b2cad49e4853831a427132e294d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15c6b88567e44da395f245725ffe7f04" is_pre_election: true
I20260709 13:12:34.277171   584 raft_consensus.cc:2393] T 93236792b8284277a935d6e569d4c96b P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d13b2cad49e4853831a427132e294d3 in current term 1: Already voted for candidate f35a78aeac02496d88fa800153754b0a in this term.
I20260709 13:12:34.278407   621 leader_election.cc:304] T 93236792b8284277a935d6e569d4c96b P 7d13b2cad49e4853831a427132e294d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d13b2cad49e4853831a427132e294d3; no voters: 15c6b88567e44da395f245725ffe7f04, f35a78aeac02496d88fa800153754b0a
I20260709 13:12:34.277091   406 catalog_manager.cc:5697] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d13b2cad49e4853831a427132e294d3 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "7d13b2cad49e4853831a427132e294d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:34.281579   406 catalog_manager.cc:5697] T 82ac3d8dfa714f66a589173f6a9eb004 P 7d13b2cad49e4853831a427132e294d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d13b2cad49e4853831a427132e294d3 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "7d13b2cad49e4853831a427132e294d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:34.283429   406 catalog_manager.cc:5697] T 3b24ef3f665449c288b2fcd28c288d27 P 7d13b2cad49e4853831a427132e294d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d13b2cad49e4853831a427132e294d3 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "7d13b2cad49e4853831a427132e294d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:34.286725   728 consensus_queue.cc:1048] T 6e5c1404cee54b96a9c3a8c024493223 P f35a78aeac02496d88fa800153754b0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:34.287078   708 raft_consensus.cc:3060] T 93236792b8284277a935d6e569d4c96b P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:34.299342   708 raft_consensus.cc:2749] T 93236792b8284277a935d6e569d4c96b P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:34.306895   708 consensus_queue.cc:1048] T 82ac3d8dfa714f66a589173f6a9eb004 P 7d13b2cad49e4853831a427132e294d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:34.325093   708 consensus_queue.cc:1048] T 82ac3d8dfa714f66a589173f6a9eb004 P 7d13b2cad49e4853831a427132e294d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:34.369082   703 consensus_queue.cc:1048] T 3723f5ab8f974d1892ce711b33fac1dd P 15c6b88567e44da395f245725ffe7f04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:34.372733   656 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a13e0fbeefb142f6bf766c1509cd1d93"
dest_uuid: "7d13b2cad49e4853831a427132e294d3"
mode: GRACEFUL
new_leader_uuid: "f35a78aeac02496d88fa800153754b0a"
 from {username='slave'} at 127.0.0.1:33240
I20260709 13:12:34.373353   656 raft_consensus.cc:606] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3 [term 1 LEADER]: Received request to transfer leadership to TS f35a78aeac02496d88fa800153754b0a
I20260709 13:12:34.379050   708 consensus_queue.cc:1048] T 3b24ef3f665449c288b2fcd28c288d27 P 7d13b2cad49e4853831a427132e294d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:34.384814   509 raft_consensus.cc:3060] T 3723f5ab8f974d1892ce711b33fac1dd P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:34.395746   584 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bf7b037540fc46d098bf7e1888e9c682"
dest_uuid: "15c6b88567e44da395f245725ffe7f04"
mode: GRACEFUL
new_leader_uuid: "f35a78aeac02496d88fa800153754b0a"
 from {username='slave'} at 127.0.0.1:60484
I20260709 13:12:34.396394   584 raft_consensus.cc:606] T bf7b037540fc46d098bf7e1888e9c682 P 15c6b88567e44da395f245725ffe7f04 [term 1 LEADER]: Received request to transfer leadership to TS f35a78aeac02496d88fa800153754b0a
I20260709 13:12:34.397548   724 consensus_queue.cc:1048] T 3723f5ab8f974d1892ce711b33fac1dd P 15c6b88567e44da395f245725ffe7f04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:34.414837   732 consensus_queue.cc:1048] T 3b24ef3f665449c288b2fcd28c288d27 P 7d13b2cad49e4853831a427132e294d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:34.427438   509 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6e5c1404cee54b96a9c3a8c024493223"
dest_uuid: "f35a78aeac02496d88fa800153754b0a"
mode: GRACEFUL
new_leader_uuid: "15c6b88567e44da395f245725ffe7f04"
 from {username='slave'} at 127.0.0.1:58040
I20260709 13:12:34.428130   509 raft_consensus.cc:606] T 6e5c1404cee54b96a9c3a8c024493223 P f35a78aeac02496d88fa800153754b0a [term 1 LEADER]: Received request to transfer leadership to TS 15c6b88567e44da395f245725ffe7f04
I20260709 13:12:34.436655   656 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "82ac3d8dfa714f66a589173f6a9eb004"
dest_uuid: "7d13b2cad49e4853831a427132e294d3"
mode: GRACEFUL
new_leader_uuid: "15c6b88567e44da395f245725ffe7f04"
 from {username='slave'} at 127.0.0.1:33240
I20260709 13:12:34.437366   656 raft_consensus.cc:606] T 82ac3d8dfa714f66a589173f6a9eb004 P 7d13b2cad49e4853831a427132e294d3 [term 1 LEADER]: Received request to transfer leadership to TS 15c6b88567e44da395f245725ffe7f04
I20260709 13:12:34.443068   509 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6653d3e90150480daf2486b331be3130"
dest_uuid: "f35a78aeac02496d88fa800153754b0a"
mode: GRACEFUL
new_leader_uuid: "15c6b88567e44da395f245725ffe7f04"
 from {username='slave'} at 127.0.0.1:58040
I20260709 13:12:34.443543   509 raft_consensus.cc:606] T 6653d3e90150480daf2486b331be3130 P f35a78aeac02496d88fa800153754b0a [term 1 LEADER]: Received request to transfer leadership to TS 15c6b88567e44da395f245725ffe7f04
I20260709 13:12:34.545480   732 consensus_queue.cc:1048] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:34.557932   708 consensus_queue.cc:1048] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:34.559998   704 consensus_queue.cc:1048] T 93236792b8284277a935d6e569d4c96b P f35a78aeac02496d88fa800153754b0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:34.566653   732 raft_consensus.cc:993] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3: : Instructing follower f35a78aeac02496d88fa800153754b0a to start an election
I20260709 13:12:34.567090   732 raft_consensus.cc:1081] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3 [term 1 LEADER]: Signalling peer f35a78aeac02496d88fa800153754b0a to start an election
I20260709 13:12:34.568969   509 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a13e0fbeefb142f6bf766c1509cd1d93"
dest_uuid: "f35a78aeac02496d88fa800153754b0a"
 from {username='slave'} at 127.0.0.1:58024
I20260709 13:12:34.569525   509 raft_consensus.cc:493] T a13e0fbeefb142f6bf766c1509cd1d93 P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:34.569891   509 raft_consensus.cc:3060] T a13e0fbeefb142f6bf766c1509cd1d93 P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:34.574570   704 consensus_queue.cc:1048] T 93236792b8284277a935d6e569d4c96b P f35a78aeac02496d88fa800153754b0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:34.584604   509 raft_consensus.cc:515] T a13e0fbeefb142f6bf766c1509cd1d93 P f35a78aeac02496d88fa800153754b0a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } }
I20260709 13:12:34.588945   509 leader_election.cc:290] T a13e0fbeefb142f6bf766c1509cd1d93 P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 2 election: Requested vote from peers 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695), 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181)
I20260709 13:12:34.591208   657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a13e0fbeefb142f6bf766c1509cd1d93" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7d13b2cad49e4853831a427132e294d3"
I20260709 13:12:34.591884   657 raft_consensus.cc:3055] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:34.592196   657 raft_consensus.cc:740] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7d13b2cad49e4853831a427132e294d3, State: Running, Role: LEADER
I20260709 13:12:34.593026   657 consensus_queue.cc:260] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } }
I20260709 13:12:34.594329   657 raft_consensus.cc:3060] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:34.599548   728 consensus_queue.cc:1048] T 6653d3e90150480daf2486b331be3130 P f35a78aeac02496d88fa800153754b0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:34.599937   583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a13e0fbeefb142f6bf766c1509cd1d93" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "15c6b88567e44da395f245725ffe7f04"
I20260709 13:12:34.600600   583 raft_consensus.cc:3060] T a13e0fbeefb142f6bf766c1509cd1d93 P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:34.605517   657 raft_consensus.cc:2468] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f35a78aeac02496d88fa800153754b0a in term 2.
I20260709 13:12:34.607190   475 leader_election.cc:304] T a13e0fbeefb142f6bf766c1509cd1d93 P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d13b2cad49e4853831a427132e294d3, f35a78aeac02496d88fa800153754b0a; no voters: 
I20260709 13:12:34.608070   728 raft_consensus.cc:2804] T a13e0fbeefb142f6bf766c1509cd1d93 P f35a78aeac02496d88fa800153754b0a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:34.608110   583 raft_consensus.cc:2468] T a13e0fbeefb142f6bf766c1509cd1d93 P 15c6b88567e44da395f245725ffe7f04 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f35a78aeac02496d88fa800153754b0a in term 2.
I20260709 13:12:34.608697   728 raft_consensus.cc:697] T a13e0fbeefb142f6bf766c1509cd1d93 P f35a78aeac02496d88fa800153754b0a [term 2 LEADER]: Becoming Leader. State: Replica: f35a78aeac02496d88fa800153754b0a, State: Running, Role: LEADER
I20260709 13:12:34.609579   728 consensus_queue.cc:237] T a13e0fbeefb142f6bf766c1509cd1d93 P f35a78aeac02496d88fa800153754b0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } }
I20260709 13:12:34.615919   728 consensus_queue.cc:1048] T 6653d3e90150480daf2486b331be3130 P f35a78aeac02496d88fa800153754b0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:34.619122   404 catalog_manager.cc:5697] T a13e0fbeefb142f6bf766c1509cd1d93 P f35a78aeac02496d88fa800153754b0a reported cstate change: term changed from 1 to 2, leader changed from 7d13b2cad49e4853831a427132e294d3 (127.21.63.3) to f35a78aeac02496d88fa800153754b0a (127.21.63.1). New cstate: current_term: 2 leader_uuid: "f35a78aeac02496d88fa800153754b0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:34.626674   704 raft_consensus.cc:993] T 6653d3e90150480daf2486b331be3130 P f35a78aeac02496d88fa800153754b0a: : Instructing follower 15c6b88567e44da395f245725ffe7f04 to start an election
I20260709 13:12:34.627257   704 raft_consensus.cc:1081] T 6653d3e90150480daf2486b331be3130 P f35a78aeac02496d88fa800153754b0a [term 1 LEADER]: Signalling peer 15c6b88567e44da395f245725ffe7f04 to start an election
I20260709 13:12:34.629186   583 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6653d3e90150480daf2486b331be3130"
dest_uuid: "15c6b88567e44da395f245725ffe7f04"
 from {username='slave'} at 127.0.0.1:60470
I20260709 13:12:34.629792   583 raft_consensus.cc:493] T 6653d3e90150480daf2486b331be3130 P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:34.630153   583 raft_consensus.cc:3060] T 6653d3e90150480daf2486b331be3130 P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:34.636127   583 raft_consensus.cc:515] T 6653d3e90150480daf2486b331be3130 P 15c6b88567e44da395f245725ffe7f04 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:34.638162   583 leader_election.cc:290] T 6653d3e90150480daf2486b331be3130 P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 2 election: Requested vote from peers f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881), 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695)
I20260709 13:12:34.639070   509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6653d3e90150480daf2486b331be3130" candidate_uuid: "15c6b88567e44da395f245725ffe7f04" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f35a78aeac02496d88fa800153754b0a"
I20260709 13:12:34.640281   509 raft_consensus.cc:3055] T 6653d3e90150480daf2486b331be3130 P f35a78aeac02496d88fa800153754b0a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:34.640612   509 raft_consensus.cc:740] T 6653d3e90150480daf2486b331be3130 P f35a78aeac02496d88fa800153754b0a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f35a78aeac02496d88fa800153754b0a, State: Running, Role: LEADER
I20260709 13:12:34.641476   509 consensus_queue.cc:260] T 6653d3e90150480daf2486b331be3130 P f35a78aeac02496d88fa800153754b0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:34.642588   509 raft_consensus.cc:3060] T 6653d3e90150480daf2486b331be3130 P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:34.639309   656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6653d3e90150480daf2486b331be3130" candidate_uuid: "15c6b88567e44da395f245725ffe7f04" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7d13b2cad49e4853831a427132e294d3"
I20260709 13:12:34.643533   656 raft_consensus.cc:3060] T 6653d3e90150480daf2486b331be3130 P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:34.647871   509 raft_consensus.cc:2468] T 6653d3e90150480daf2486b331be3130 P f35a78aeac02496d88fa800153754b0a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15c6b88567e44da395f245725ffe7f04 in term 2.
I20260709 13:12:34.649163   656 raft_consensus.cc:2468] T 6653d3e90150480daf2486b331be3130 P 7d13b2cad49e4853831a427132e294d3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15c6b88567e44da395f245725ffe7f04 in term 2.
I20260709 13:12:34.649196   550 leader_election.cc:304] T 6653d3e90150480daf2486b331be3130 P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15c6b88567e44da395f245725ffe7f04, f35a78aeac02496d88fa800153754b0a; no voters: 
I20260709 13:12:34.650238   724 raft_consensus.cc:2804] T 6653d3e90150480daf2486b331be3130 P 15c6b88567e44da395f245725ffe7f04 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:34.650657   724 raft_consensus.cc:697] T 6653d3e90150480daf2486b331be3130 P 15c6b88567e44da395f245725ffe7f04 [term 2 LEADER]: Becoming Leader. State: Replica: 15c6b88567e44da395f245725ffe7f04, State: Running, Role: LEADER
I20260709 13:12:34.651383   724 consensus_queue.cc:237] T 6653d3e90150480daf2486b331be3130 P 15c6b88567e44da395f245725ffe7f04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:34.657727   404 catalog_manager.cc:5697] T 6653d3e90150480daf2486b331be3130 P 15c6b88567e44da395f245725ffe7f04 reported cstate change: term changed from 1 to 2, leader changed from f35a78aeac02496d88fa800153754b0a (127.21.63.1) to 15c6b88567e44da395f245725ffe7f04 (127.21.63.2). New cstate: current_term: 2 leader_uuid: "15c6b88567e44da395f245725ffe7f04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:34.766031   749 raft_consensus.cc:993] T bf7b037540fc46d098bf7e1888e9c682 P 15c6b88567e44da395f245725ffe7f04: : Instructing follower f35a78aeac02496d88fa800153754b0a to start an election
I20260709 13:12:34.766562   724 raft_consensus.cc:1081] T bf7b037540fc46d098bf7e1888e9c682 P 15c6b88567e44da395f245725ffe7f04 [term 1 LEADER]: Signalling peer f35a78aeac02496d88fa800153754b0a to start an election
I20260709 13:12:34.768034   509 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bf7b037540fc46d098bf7e1888e9c682"
dest_uuid: "f35a78aeac02496d88fa800153754b0a"
 from {username='slave'} at 127.0.0.1:58018
I20260709 13:12:34.768666   509 raft_consensus.cc:493] T bf7b037540fc46d098bf7e1888e9c682 P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:34.769112   509 raft_consensus.cc:3060] T bf7b037540fc46d098bf7e1888e9c682 P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:34.771451   732 raft_consensus.cc:993] T 82ac3d8dfa714f66a589173f6a9eb004 P 7d13b2cad49e4853831a427132e294d3: : Instructing follower 15c6b88567e44da395f245725ffe7f04 to start an election
I20260709 13:12:34.771968   735 raft_consensus.cc:1081] T 82ac3d8dfa714f66a589173f6a9eb004 P 7d13b2cad49e4853831a427132e294d3 [term 1 LEADER]: Signalling peer 15c6b88567e44da395f245725ffe7f04 to start an election
I20260709 13:12:34.773597   583 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "82ac3d8dfa714f66a589173f6a9eb004"
dest_uuid: "15c6b88567e44da395f245725ffe7f04"
 from {username='slave'} at 127.0.0.1:60472
I20260709 13:12:34.774214   583 raft_consensus.cc:493] T 82ac3d8dfa714f66a589173f6a9eb004 P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:34.774621   583 raft_consensus.cc:3060] T 82ac3d8dfa714f66a589173f6a9eb004 P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:34.777325   509 raft_consensus.cc:515] T bf7b037540fc46d098bf7e1888e9c682 P f35a78aeac02496d88fa800153754b0a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:34.780203   509 leader_election.cc:290] T bf7b037540fc46d098bf7e1888e9c682 P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 2 election: Requested vote from peers 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181), 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695)
I20260709 13:12:34.780476   582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf7b037540fc46d098bf7e1888e9c682" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "15c6b88567e44da395f245725ffe7f04"
I20260709 13:12:34.781163   582 raft_consensus.cc:3055] T bf7b037540fc46d098bf7e1888e9c682 P 15c6b88567e44da395f245725ffe7f04 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:34.781280   656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bf7b037540fc46d098bf7e1888e9c682" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7d13b2cad49e4853831a427132e294d3"
I20260709 13:12:34.781519   582 raft_consensus.cc:740] T bf7b037540fc46d098bf7e1888e9c682 P 15c6b88567e44da395f245725ffe7f04 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 15c6b88567e44da395f245725ffe7f04, State: Running, Role: LEADER
I20260709 13:12:34.782327   582 consensus_queue.cc:260] T bf7b037540fc46d098bf7e1888e9c682 P 15c6b88567e44da395f245725ffe7f04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:34.782506   656 raft_consensus.cc:3060] T bf7b037540fc46d098bf7e1888e9c682 P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:34.782708   583 raft_consensus.cc:515] T 82ac3d8dfa714f66a589173f6a9eb004 P 15c6b88567e44da395f245725ffe7f04 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:34.783550   582 raft_consensus.cc:3060] T bf7b037540fc46d098bf7e1888e9c682 P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:34.784919   583 leader_election.cc:290] T 82ac3d8dfa714f66a589173f6a9eb004 P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 2 election: Requested vote from peers f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881), 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695)
I20260709 13:12:34.785871   657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ac3d8dfa714f66a589173f6a9eb004" candidate_uuid: "15c6b88567e44da395f245725ffe7f04" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7d13b2cad49e4853831a427132e294d3"
I20260709 13:12:34.786098   509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "82ac3d8dfa714f66a589173f6a9eb004" candidate_uuid: "15c6b88567e44da395f245725ffe7f04" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f35a78aeac02496d88fa800153754b0a"
I20260709 13:12:34.786463   657 raft_consensus.cc:3055] T 82ac3d8dfa714f66a589173f6a9eb004 P 7d13b2cad49e4853831a427132e294d3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:34.786690   509 raft_consensus.cc:3060] T 82ac3d8dfa714f66a589173f6a9eb004 P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:34.786859   657 raft_consensus.cc:740] T 82ac3d8dfa714f66a589173f6a9eb004 P 7d13b2cad49e4853831a427132e294d3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7d13b2cad49e4853831a427132e294d3, State: Running, Role: LEADER
I20260709 13:12:34.787792   657 consensus_queue.cc:260] T 82ac3d8dfa714f66a589173f6a9eb004 P 7d13b2cad49e4853831a427132e294d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:34.789065   657 raft_consensus.cc:3060] T 82ac3d8dfa714f66a589173f6a9eb004 P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:34.790431   656 raft_consensus.cc:2468] T bf7b037540fc46d098bf7e1888e9c682 P 7d13b2cad49e4853831a427132e294d3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f35a78aeac02496d88fa800153754b0a in term 2.
I20260709 13:12:34.791652   475 leader_election.cc:304] T bf7b037540fc46d098bf7e1888e9c682 P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d13b2cad49e4853831a427132e294d3, f35a78aeac02496d88fa800153754b0a; no voters: 
I20260709 13:12:34.792397   704 raft_consensus.cc:2804] T bf7b037540fc46d098bf7e1888e9c682 P f35a78aeac02496d88fa800153754b0a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:34.792927   704 raft_consensus.cc:697] T bf7b037540fc46d098bf7e1888e9c682 P f35a78aeac02496d88fa800153754b0a [term 2 LEADER]: Becoming Leader. State: Replica: f35a78aeac02496d88fa800153754b0a, State: Running, Role: LEADER
I20260709 13:12:34.793642   704 consensus_queue.cc:237] T bf7b037540fc46d098bf7e1888e9c682 P f35a78aeac02496d88fa800153754b0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:34.794270   509 raft_consensus.cc:2468] T 82ac3d8dfa714f66a589173f6a9eb004 P f35a78aeac02496d88fa800153754b0a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15c6b88567e44da395f245725ffe7f04 in term 2.
I20260709 13:12:34.794270   582 raft_consensus.cc:2468] T bf7b037540fc46d098bf7e1888e9c682 P 15c6b88567e44da395f245725ffe7f04 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f35a78aeac02496d88fa800153754b0a in term 2.
I20260709 13:12:34.796434   550 leader_election.cc:304] T 82ac3d8dfa714f66a589173f6a9eb004 P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15c6b88567e44da395f245725ffe7f04, f35a78aeac02496d88fa800153754b0a; no voters: 
I20260709 13:12:34.797236   724 raft_consensus.cc:2804] T 82ac3d8dfa714f66a589173f6a9eb004 P 15c6b88567e44da395f245725ffe7f04 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:34.797710   724 raft_consensus.cc:697] T 82ac3d8dfa714f66a589173f6a9eb004 P 15c6b88567e44da395f245725ffe7f04 [term 2 LEADER]: Becoming Leader. State: Replica: 15c6b88567e44da395f245725ffe7f04, State: Running, Role: LEADER
I20260709 13:12:34.798497   724 consensus_queue.cc:237] T 82ac3d8dfa714f66a589173f6a9eb004 P 15c6b88567e44da395f245725ffe7f04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:34.801112   657 raft_consensus.cc:2468] T 82ac3d8dfa714f66a589173f6a9eb004 P 7d13b2cad49e4853831a427132e294d3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15c6b88567e44da395f245725ffe7f04 in term 2.
I20260709 13:12:34.801256   404 catalog_manager.cc:5697] T bf7b037540fc46d098bf7e1888e9c682 P f35a78aeac02496d88fa800153754b0a reported cstate change: term changed from 1 to 2, leader changed from 15c6b88567e44da395f245725ffe7f04 (127.21.63.2) to f35a78aeac02496d88fa800153754b0a (127.21.63.1). New cstate: current_term: 2 leader_uuid: "f35a78aeac02496d88fa800153754b0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:34.806020   405 catalog_manager.cc:5697] T 82ac3d8dfa714f66a589173f6a9eb004 P 15c6b88567e44da395f245725ffe7f04 reported cstate change: term changed from 1 to 2, leader changed from 7d13b2cad49e4853831a427132e294d3 (127.21.63.3) to 15c6b88567e44da395f245725ffe7f04 (127.21.63.2). New cstate: current_term: 2 leader_uuid: "15c6b88567e44da395f245725ffe7f04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:34.884824   728 raft_consensus.cc:993] T 6e5c1404cee54b96a9c3a8c024493223 P f35a78aeac02496d88fa800153754b0a: : Instructing follower 15c6b88567e44da395f245725ffe7f04 to start an election
I20260709 13:12:34.885383   704 raft_consensus.cc:1081] T 6e5c1404cee54b96a9c3a8c024493223 P f35a78aeac02496d88fa800153754b0a [term 1 LEADER]: Signalling peer 15c6b88567e44da395f245725ffe7f04 to start an election
I20260709 13:12:34.887815   582 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6e5c1404cee54b96a9c3a8c024493223"
dest_uuid: "15c6b88567e44da395f245725ffe7f04"
 from {username='slave'} at 127.0.0.1:60470
I20260709 13:12:34.888388   582 raft_consensus.cc:493] T 6e5c1404cee54b96a9c3a8c024493223 P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:34.888654   582 raft_consensus.cc:3060] T 6e5c1404cee54b96a9c3a8c024493223 P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:34.895188   582 raft_consensus.cc:515] T 6e5c1404cee54b96a9c3a8c024493223 P 15c6b88567e44da395f245725ffe7f04 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:34.897112   582 leader_election.cc:290] T 6e5c1404cee54b96a9c3a8c024493223 P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 2 election: Requested vote from peers f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881), 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695)
I20260709 13:12:34.898082   509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e5c1404cee54b96a9c3a8c024493223" candidate_uuid: "15c6b88567e44da395f245725ffe7f04" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f35a78aeac02496d88fa800153754b0a"
I20260709 13:12:34.898324   657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6e5c1404cee54b96a9c3a8c024493223" candidate_uuid: "15c6b88567e44da395f245725ffe7f04" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7d13b2cad49e4853831a427132e294d3"
I20260709 13:12:34.898846   509 raft_consensus.cc:3055] T 6e5c1404cee54b96a9c3a8c024493223 P f35a78aeac02496d88fa800153754b0a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:34.898921   657 raft_consensus.cc:3060] T 6e5c1404cee54b96a9c3a8c024493223 P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:34.899204   509 raft_consensus.cc:740] T 6e5c1404cee54b96a9c3a8c024493223 P f35a78aeac02496d88fa800153754b0a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f35a78aeac02496d88fa800153754b0a, State: Running, Role: LEADER
I20260709 13:12:34.900000   509 consensus_queue.cc:260] T 6e5c1404cee54b96a9c3a8c024493223 P f35a78aeac02496d88fa800153754b0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:34.901326   509 raft_consensus.cc:3060] T 6e5c1404cee54b96a9c3a8c024493223 P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:34.904433   657 raft_consensus.cc:2468] T 6e5c1404cee54b96a9c3a8c024493223 P 7d13b2cad49e4853831a427132e294d3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15c6b88567e44da395f245725ffe7f04 in term 2.
I20260709 13:12:34.905772   550 leader_election.cc:304] T 6e5c1404cee54b96a9c3a8c024493223 P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15c6b88567e44da395f245725ffe7f04, 7d13b2cad49e4853831a427132e294d3; no voters: 
I20260709 13:12:34.906608   724 raft_consensus.cc:2804] T 6e5c1404cee54b96a9c3a8c024493223 P 15c6b88567e44da395f245725ffe7f04 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:34.907080   724 raft_consensus.cc:697] T 6e5c1404cee54b96a9c3a8c024493223 P 15c6b88567e44da395f245725ffe7f04 [term 2 LEADER]: Becoming Leader. State: Replica: 15c6b88567e44da395f245725ffe7f04, State: Running, Role: LEADER
I20260709 13:12:34.907305   509 raft_consensus.cc:2468] T 6e5c1404cee54b96a9c3a8c024493223 P f35a78aeac02496d88fa800153754b0a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15c6b88567e44da395f245725ffe7f04 in term 2.
I20260709 13:12:34.907919   724 consensus_queue.cc:237] T 6e5c1404cee54b96a9c3a8c024493223 P 15c6b88567e44da395f245725ffe7f04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:34.914682   404 catalog_manager.cc:5697] T 6e5c1404cee54b96a9c3a8c024493223 P 15c6b88567e44da395f245725ffe7f04 reported cstate change: term changed from 1 to 2, leader changed from f35a78aeac02496d88fa800153754b0a (127.21.63.1) to 15c6b88567e44da395f245725ffe7f04 (127.21.63.2). New cstate: current_term: 2 leader_uuid: "15c6b88567e44da395f245725ffe7f04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } health_report { overall_health: UNKNOWN } } }
W20260709 13:12:34.963666   437 debug-util.cc:398] Leaking SignalData structure 0x7b0800067220 after lost signal to thread 21759
W20260709 13:12:34.964834   437 debug-util.cc:398] Leaking SignalData structure 0x7b080016eee0 after lost signal to thread 440
I20260709 13:12:35.040942   657 raft_consensus.cc:1275] T 6653d3e90150480daf2486b331be3130 P 7d13b2cad49e4853831a427132e294d3 [term 2 FOLLOWER]: Refusing update from remote peer 15c6b88567e44da395f245725ffe7f04: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:35.040942   582 raft_consensus.cc:1275] T a13e0fbeefb142f6bf766c1509cd1d93 P 15c6b88567e44da395f245725ffe7f04 [term 2 FOLLOWER]: Refusing update from remote peer f35a78aeac02496d88fa800153754b0a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:35.042793   787 consensus_queue.cc:1048] T 6653d3e90150480daf2486b331be3130 P 15c6b88567e44da395f245725ffe7f04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:35.042932   704 consensus_queue.cc:1048] T a13e0fbeefb142f6bf766c1509cd1d93 P f35a78aeac02496d88fa800153754b0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:35.052624   509 raft_consensus.cc:1275] T 6653d3e90150480daf2486b331be3130 P f35a78aeac02496d88fa800153754b0a [term 2 FOLLOWER]: Refusing update from remote peer 15c6b88567e44da395f245725ffe7f04: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:35.057281   749 consensus_queue.cc:1048] T 6653d3e90150480daf2486b331be3130 P 15c6b88567e44da395f245725ffe7f04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:35.060508   657 raft_consensus.cc:1275] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3 [term 2 FOLLOWER]: Refusing update from remote peer f35a78aeac02496d88fa800153754b0a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:35.062615   728 consensus_queue.cc:1048] T a13e0fbeefb142f6bf766c1509cd1d93 P f35a78aeac02496d88fa800153754b0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:35.276914   509 raft_consensus.cc:1275] T 82ac3d8dfa714f66a589173f6a9eb004 P f35a78aeac02496d88fa800153754b0a [term 2 FOLLOWER]: Refusing update from remote peer 15c6b88567e44da395f245725ffe7f04: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:35.278671   724 consensus_queue.cc:1048] T 82ac3d8dfa714f66a589173f6a9eb004 P 15c6b88567e44da395f245725ffe7f04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:35.286681   657 raft_consensus.cc:1275] T 82ac3d8dfa714f66a589173f6a9eb004 P 7d13b2cad49e4853831a427132e294d3 [term 2 FOLLOWER]: Refusing update from remote peer 15c6b88567e44da395f245725ffe7f04: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:35.292447   787 consensus_queue.cc:1048] T 82ac3d8dfa714f66a589173f6a9eb004 P 15c6b88567e44da395f245725ffe7f04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:35.349167   657 raft_consensus.cc:1275] T bf7b037540fc46d098bf7e1888e9c682 P 7d13b2cad49e4853831a427132e294d3 [term 2 FOLLOWER]: Refusing update from remote peer f35a78aeac02496d88fa800153754b0a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:35.350747   704 consensus_queue.cc:1048] T bf7b037540fc46d098bf7e1888e9c682 P f35a78aeac02496d88fa800153754b0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:35.360069   582 raft_consensus.cc:1275] T bf7b037540fc46d098bf7e1888e9c682 P 15c6b88567e44da395f245725ffe7f04 [term 2 FOLLOWER]: Refusing update from remote peer f35a78aeac02496d88fa800153754b0a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:35.361836   785 consensus_queue.cc:1048] T bf7b037540fc46d098bf7e1888e9c682 P f35a78aeac02496d88fa800153754b0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:35.482831   657 raft_consensus.cc:1275] T 6e5c1404cee54b96a9c3a8c024493223 P 7d13b2cad49e4853831a427132e294d3 [term 2 FOLLOWER]: Refusing update from remote peer 15c6b88567e44da395f245725ffe7f04: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:35.484330   787 consensus_queue.cc:1048] T 6e5c1404cee54b96a9c3a8c024493223 P 15c6b88567e44da395f245725ffe7f04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:35.494603   509 raft_consensus.cc:1275] T 6e5c1404cee54b96a9c3a8c024493223 P f35a78aeac02496d88fa800153754b0a [term 2 FOLLOWER]: Refusing update from remote peer 15c6b88567e44da395f245725ffe7f04: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:35.496260   724 consensus_queue.cc:1048] T 6e5c1404cee54b96a9c3a8c024493223 P 15c6b88567e44da395f245725ffe7f04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:37.476841   406 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40368:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:12:37.480196   406 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:12:37.547791   494 tablet_service.cc:1511] Processing CreateTablet for tablet bedab716985143c7a40f88c69d71ecd4 (DEFAULT_TABLE table=soft_deleted_table [id=9258462a82ed4b239a497b7531d655f0]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:12:37.549224   494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bedab716985143c7a40f88c69d71ecd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:37.555255   572 tablet_service.cc:1511] Processing CreateTablet for tablet bedab716985143c7a40f88c69d71ecd4 (DEFAULT_TABLE table=soft_deleted_table [id=9258462a82ed4b239a497b7531d655f0]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:12:37.556627   572 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bedab716985143c7a40f88c69d71ecd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:37.567226   642 tablet_service.cc:1511] Processing CreateTablet for tablet bedab716985143c7a40f88c69d71ecd4 (DEFAULT_TABLE table=soft_deleted_table [id=9258462a82ed4b239a497b7531d655f0]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:12:37.568532   642 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bedab716985143c7a40f88c69d71ecd4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:37.570277   574 tablet_service.cc:1511] Processing CreateTablet for tablet b9b003cc8908492ab10df382680b5141 (DEFAULT_TABLE table=soft_deleted_table [id=9258462a82ed4b239a497b7531d655f0]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:12:37.571599   574 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9b003cc8908492ab10df382680b5141. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:37.574663   805 tablet_bootstrap.cc:492] T bedab716985143c7a40f88c69d71ecd4 P f35a78aeac02496d88fa800153754b0a: Bootstrap starting.
I20260709 13:12:37.575111   492 tablet_service.cc:1511] Processing CreateTablet for tablet b9b003cc8908492ab10df382680b5141 (DEFAULT_TABLE table=soft_deleted_table [id=9258462a82ed4b239a497b7531d655f0]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:12:37.576969   492 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9b003cc8908492ab10df382680b5141. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:37.577395   644 tablet_service.cc:1511] Processing CreateTablet for tablet b9b003cc8908492ab10df382680b5141 (DEFAULT_TABLE table=soft_deleted_table [id=9258462a82ed4b239a497b7531d655f0]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:12:37.578971   644 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b9b003cc8908492ab10df382680b5141. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:37.583269   494 tablet_service.cc:1511] Processing CreateTablet for tablet 9082b9270cf74ed5b72b7bc4b0166836 (DEFAULT_TABLE table=soft_deleted_table [id=9258462a82ed4b239a497b7531d655f0]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:12:37.584919   494 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9082b9270cf74ed5b72b7bc4b0166836. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:37.588755   573 tablet_service.cc:1511] Processing CreateTablet for tablet 9082b9270cf74ed5b72b7bc4b0166836 (DEFAULT_TABLE table=soft_deleted_table [id=9258462a82ed4b239a497b7531d655f0]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:12:37.590167   573 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9082b9270cf74ed5b72b7bc4b0166836. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:37.593591   647 tablet_service.cc:1511] Processing CreateTablet for tablet 9082b9270cf74ed5b72b7bc4b0166836 (DEFAULT_TABLE table=soft_deleted_table [id=9258462a82ed4b239a497b7531d655f0]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:12:37.595765   498 tablet_service.cc:1511] Processing CreateTablet for tablet 78d4c60d0e4b42ef9e2bc081980888af (DEFAULT_TABLE table=soft_deleted_table [id=9258462a82ed4b239a497b7531d655f0]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:12:37.597168   498 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78d4c60d0e4b42ef9e2bc081980888af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:37.598452   567 tablet_service.cc:1511] Processing CreateTablet for tablet 78d4c60d0e4b42ef9e2bc081980888af (DEFAULT_TABLE table=soft_deleted_table [id=9258462a82ed4b239a497b7531d655f0]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:12:37.599243   647 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9082b9270cf74ed5b72b7bc4b0166836. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:37.599732   567 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78d4c60d0e4b42ef9e2bc081980888af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:37.604029   571 tablet_service.cc:1511] Processing CreateTablet for tablet d7f398f030d840cab518d8996f2bdbce (DEFAULT_TABLE table=soft_deleted_table [id=9258462a82ed4b239a497b7531d655f0]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:12:37.605294   571 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7f398f030d840cab518d8996f2bdbce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:37.606877   805 tablet_bootstrap.cc:654] T bedab716985143c7a40f88c69d71ecd4 P f35a78aeac02496d88fa800153754b0a: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:37.608716   648 tablet_service.cc:1511] Processing CreateTablet for tablet d7f398f030d840cab518d8996f2bdbce (DEFAULT_TABLE table=soft_deleted_table [id=9258462a82ed4b239a497b7531d655f0]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:12:37.600567   645 tablet_service.cc:1511] Processing CreateTablet for tablet 78d4c60d0e4b42ef9e2bc081980888af (DEFAULT_TABLE table=soft_deleted_table [id=9258462a82ed4b239a497b7531d655f0]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:12:37.602381   496 tablet_service.cc:1511] Processing CreateTablet for tablet d7f398f030d840cab518d8996f2bdbce (DEFAULT_TABLE table=soft_deleted_table [id=9258462a82ed4b239a497b7531d655f0]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:12:37.610347   496 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7f398f030d840cab518d8996f2bdbce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:37.612363   569 tablet_service.cc:1511] Processing CreateTablet for tablet f75ad0645c22439c9a0611d214158d01 (DEFAULT_TABLE table=soft_deleted_table [id=9258462a82ed4b239a497b7531d655f0]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:12:37.613602   569 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f75ad0645c22439c9a0611d214158d01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:37.613638   491 tablet_service.cc:1511] Processing CreateTablet for tablet f75ad0645c22439c9a0611d214158d01 (DEFAULT_TABLE table=soft_deleted_table [id=9258462a82ed4b239a497b7531d655f0]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:12:37.614449   643 tablet_service.cc:1511] Processing CreateTablet for tablet f75ad0645c22439c9a0611d214158d01 (DEFAULT_TABLE table=soft_deleted_table [id=9258462a82ed4b239a497b7531d655f0]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:12:37.615041   491 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f75ad0645c22439c9a0611d214158d01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:37.616767   568 tablet_service.cc:1511] Processing CreateTablet for tablet 16c6d1765fc145348e0a4f703fe6abdf (DEFAULT_TABLE table=soft_deleted_table [id=9258462a82ed4b239a497b7531d655f0]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:12:37.617324   495 tablet_service.cc:1511] Processing CreateTablet for tablet 16c6d1765fc145348e0a4f703fe6abdf (DEFAULT_TABLE table=soft_deleted_table [id=9258462a82ed4b239a497b7531d655f0]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:12:37.617957   568 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16c6d1765fc145348e0a4f703fe6abdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:37.618543   495 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16c6d1765fc145348e0a4f703fe6abdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:37.621197   570 tablet_service.cc:1511] Processing CreateTablet for tablet def0acc1a0684b9b86755e1c34191d91 (DEFAULT_TABLE table=soft_deleted_table [id=9258462a82ed4b239a497b7531d655f0]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:12:37.622464   570 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet def0acc1a0684b9b86755e1c34191d91. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:12:37.625452   455 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:12:37.626219   648 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7f398f030d840cab518d8996f2bdbce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:37.627970   645 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 78d4c60d0e4b42ef9e2bc081980888af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:37.629638   493 tablet_service.cc:1511] Processing CreateTablet for tablet def0acc1a0684b9b86755e1c34191d91 (DEFAULT_TABLE table=soft_deleted_table [id=9258462a82ed4b239a497b7531d655f0]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:12:37.630990   493 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet def0acc1a0684b9b86755e1c34191d91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:37.633019   643 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f75ad0645c22439c9a0611d214158d01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:37.641377   640 tablet_service.cc:1511] Processing CreateTablet for tablet 16c6d1765fc145348e0a4f703fe6abdf (DEFAULT_TABLE table=soft_deleted_table [id=9258462a82ed4b239a497b7531d655f0]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:12:37.642964   640 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 16c6d1765fc145348e0a4f703fe6abdf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:37.664367   646 tablet_service.cc:1511] Processing CreateTablet for tablet def0acc1a0684b9b86755e1c34191d91 (DEFAULT_TABLE table=soft_deleted_table [id=9258462a82ed4b239a497b7531d655f0]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:12:37.665812   646 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet def0acc1a0684b9b86755e1c34191d91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:37.699043   570 tablet_service.cc:1511] Processing CreateTablet for tablet 42ebb32ae777484fab5e26a137cbd4f6 (DEFAULT_TABLE table=soft_deleted_table [id=9258462a82ed4b239a497b7531d655f0]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:12:37.701051   570 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42ebb32ae777484fab5e26a137cbd4f6. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:12:37.759824   472 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:38181, user_credentials={real_user=slave}} blocked reactor thread for 102284us
I20260709 13:12:37.773877   643 tablet_service.cc:1511] Processing CreateTablet for tablet 42ebb32ae777484fab5e26a137cbd4f6 (DEFAULT_TABLE table=soft_deleted_table [id=9258462a82ed4b239a497b7531d655f0]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:12:37.775517   643 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42ebb32ae777484fab5e26a137cbd4f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:37.776894   493 tablet_service.cc:1511] Processing CreateTablet for tablet 42ebb32ae777484fab5e26a137cbd4f6 (DEFAULT_TABLE table=soft_deleted_table [id=9258462a82ed4b239a497b7531d655f0]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:12:37.780498   493 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 42ebb32ae777484fab5e26a137cbd4f6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:37.783300   805 tablet_bootstrap.cc:492] T bedab716985143c7a40f88c69d71ecd4 P f35a78aeac02496d88fa800153754b0a: No bootstrap required, opened a new log
I20260709 13:12:37.784108   805 ts_tablet_manager.cc:1403] T bedab716985143c7a40f88c69d71ecd4 P f35a78aeac02496d88fa800153754b0a: Time spent bootstrapping tablet: real 0.210s	user 0.010s	sys 0.005s
I20260709 13:12:37.789824   807 tablet_bootstrap.cc:492] T bedab716985143c7a40f88c69d71ecd4 P 15c6b88567e44da395f245725ffe7f04: Bootstrap starting.
I20260709 13:12:37.796267   805 raft_consensus.cc:359] T bedab716985143c7a40f88c69d71ecd4 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:37.801637   805 raft_consensus.cc:385] T bedab716985143c7a40f88c69d71ecd4 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:37.801998   805 raft_consensus.cc:740] T bedab716985143c7a40f88c69d71ecd4 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f35a78aeac02496d88fa800153754b0a, State: Initialized, Role: FOLLOWER
I20260709 13:12:37.802868   805 consensus_queue.cc:260] T bedab716985143c7a40f88c69d71ecd4 P f35a78aeac02496d88fa800153754b0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:37.805510   805 ts_tablet_manager.cc:1434] T bedab716985143c7a40f88c69d71ecd4 P f35a78aeac02496d88fa800153754b0a: Time spent starting tablet: real 0.021s	user 0.010s	sys 0.000s
I20260709 13:12:37.807564   805 tablet_bootstrap.cc:492] T b9b003cc8908492ab10df382680b5141 P f35a78aeac02496d88fa800153754b0a: Bootstrap starting.
I20260709 13:12:37.814946   807 tablet_bootstrap.cc:654] T bedab716985143c7a40f88c69d71ecd4 P 15c6b88567e44da395f245725ffe7f04: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:37.815204   805 tablet_bootstrap.cc:654] T b9b003cc8908492ab10df382680b5141 P f35a78aeac02496d88fa800153754b0a: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:37.821687   809 tablet_bootstrap.cc:492] T 78d4c60d0e4b42ef9e2bc081980888af P 7d13b2cad49e4853831a427132e294d3: Bootstrap starting.
I20260709 13:12:37.828370   805 tablet_bootstrap.cc:492] T b9b003cc8908492ab10df382680b5141 P f35a78aeac02496d88fa800153754b0a: No bootstrap required, opened a new log
I20260709 13:12:37.829341   805 ts_tablet_manager.cc:1403] T b9b003cc8908492ab10df382680b5141 P f35a78aeac02496d88fa800153754b0a: Time spent bootstrapping tablet: real 0.022s	user 0.014s	sys 0.000s
I20260709 13:12:37.830802   809 tablet_bootstrap.cc:654] T 78d4c60d0e4b42ef9e2bc081980888af P 7d13b2cad49e4853831a427132e294d3: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:37.831281   807 tablet_bootstrap.cc:492] T bedab716985143c7a40f88c69d71ecd4 P 15c6b88567e44da395f245725ffe7f04: No bootstrap required, opened a new log
I20260709 13:12:37.831899   807 ts_tablet_manager.cc:1403] T bedab716985143c7a40f88c69d71ecd4 P 15c6b88567e44da395f245725ffe7f04: Time spent bootstrapping tablet: real 0.042s	user 0.014s	sys 0.001s
I20260709 13:12:37.832902   805 raft_consensus.cc:359] T b9b003cc8908492ab10df382680b5141 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:37.833853   805 raft_consensus.cc:385] T b9b003cc8908492ab10df382680b5141 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:37.834278   805 raft_consensus.cc:740] T b9b003cc8908492ab10df382680b5141 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f35a78aeac02496d88fa800153754b0a, State: Initialized, Role: FOLLOWER
I20260709 13:12:37.835073   807 raft_consensus.cc:359] T bedab716985143c7a40f88c69d71ecd4 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:37.836054   807 raft_consensus.cc:385] T bedab716985143c7a40f88c69d71ecd4 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:37.835595   805 consensus_queue.cc:260] T b9b003cc8908492ab10df382680b5141 P f35a78aeac02496d88fa800153754b0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:37.836444   807 raft_consensus.cc:740] T bedab716985143c7a40f88c69d71ecd4 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15c6b88567e44da395f245725ffe7f04, State: Initialized, Role: FOLLOWER
I20260709 13:12:37.843106   805 ts_tablet_manager.cc:1434] T b9b003cc8908492ab10df382680b5141 P f35a78aeac02496d88fa800153754b0a: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 13:12:37.844929   805 tablet_bootstrap.cc:492] T 9082b9270cf74ed5b72b7bc4b0166836 P f35a78aeac02496d88fa800153754b0a: Bootstrap starting.
I20260709 13:12:37.846236   809 tablet_bootstrap.cc:492] T 78d4c60d0e4b42ef9e2bc081980888af P 7d13b2cad49e4853831a427132e294d3: No bootstrap required, opened a new log
I20260709 13:12:37.846871   809 ts_tablet_manager.cc:1403] T 78d4c60d0e4b42ef9e2bc081980888af P 7d13b2cad49e4853831a427132e294d3: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.006s
I20260709 13:12:37.850157   809 raft_consensus.cc:359] T 78d4c60d0e4b42ef9e2bc081980888af P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:37.851027   809 raft_consensus.cc:385] T 78d4c60d0e4b42ef9e2bc081980888af P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:37.850889   807 consensus_queue.cc:260] T bedab716985143c7a40f88c69d71ecd4 P 15c6b88567e44da395f245725ffe7f04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:37.851475   809 raft_consensus.cc:740] T 78d4c60d0e4b42ef9e2bc081980888af P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d13b2cad49e4853831a427132e294d3, State: Initialized, Role: FOLLOWER
I20260709 13:12:37.852447   809 consensus_queue.cc:260] T 78d4c60d0e4b42ef9e2bc081980888af P 7d13b2cad49e4853831a427132e294d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:37.853190   805 tablet_bootstrap.cc:654] T 9082b9270cf74ed5b72b7bc4b0166836 P f35a78aeac02496d88fa800153754b0a: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:37.855058   809 ts_tablet_manager.cc:1434] T 78d4c60d0e4b42ef9e2bc081980888af P 7d13b2cad49e4853831a427132e294d3: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:12:37.861127   809 tablet_bootstrap.cc:492] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3: Bootstrap starting.
I20260709 13:12:37.867895   809 tablet_bootstrap.cc:654] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:37.856992   807 ts_tablet_manager.cc:1434] T bedab716985143c7a40f88c69d71ecd4 P 15c6b88567e44da395f245725ffe7f04: Time spent starting tablet: real 0.025s	user 0.007s	sys 0.000s
I20260709 13:12:37.870898   807 tablet_bootstrap.cc:492] T def0acc1a0684b9b86755e1c34191d91 P 15c6b88567e44da395f245725ffe7f04: Bootstrap starting.
I20260709 13:12:37.875607   809 tablet_bootstrap.cc:492] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3: No bootstrap required, opened a new log
I20260709 13:12:37.876250   809 ts_tablet_manager.cc:1403] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3: Time spent bootstrapping tablet: real 0.015s	user 0.009s	sys 0.003s
I20260709 13:12:37.879670   809 raft_consensus.cc:359] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:37.880468   809 raft_consensus.cc:385] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:37.880818   809 raft_consensus.cc:740] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d13b2cad49e4853831a427132e294d3, State: Initialized, Role: FOLLOWER
I20260709 13:12:37.881772   809 consensus_queue.cc:260] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:37.884413   809 ts_tablet_manager.cc:1434] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 13:12:37.882344   805 tablet_bootstrap.cc:492] T 9082b9270cf74ed5b72b7bc4b0166836 P f35a78aeac02496d88fa800153754b0a: No bootstrap required, opened a new log
I20260709 13:12:37.886236   805 ts_tablet_manager.cc:1403] T 9082b9270cf74ed5b72b7bc4b0166836 P f35a78aeac02496d88fa800153754b0a: Time spent bootstrapping tablet: real 0.042s	user 0.007s	sys 0.005s
I20260709 13:12:37.888984   809 tablet_bootstrap.cc:492] T b9b003cc8908492ab10df382680b5141 P 7d13b2cad49e4853831a427132e294d3: Bootstrap starting.
I20260709 13:12:37.890486   807 tablet_bootstrap.cc:654] T def0acc1a0684b9b86755e1c34191d91 P 15c6b88567e44da395f245725ffe7f04: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:37.889714   805 raft_consensus.cc:359] T 9082b9270cf74ed5b72b7bc4b0166836 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:37.892946   805 raft_consensus.cc:385] T 9082b9270cf74ed5b72b7bc4b0166836 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:37.893368   805 raft_consensus.cc:740] T 9082b9270cf74ed5b72b7bc4b0166836 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f35a78aeac02496d88fa800153754b0a, State: Initialized, Role: FOLLOWER
I20260709 13:12:37.894258   805 consensus_queue.cc:260] T 9082b9270cf74ed5b72b7bc4b0166836 P f35a78aeac02496d88fa800153754b0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:37.898316   809 tablet_bootstrap.cc:654] T b9b003cc8908492ab10df382680b5141 P 7d13b2cad49e4853831a427132e294d3: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:37.904349   805 ts_tablet_manager.cc:1434] T 9082b9270cf74ed5b72b7bc4b0166836 P f35a78aeac02496d88fa800153754b0a: Time spent starting tablet: real 0.018s	user 0.001s	sys 0.005s
I20260709 13:12:37.906376   805 tablet_bootstrap.cc:492] T d7f398f030d840cab518d8996f2bdbce P f35a78aeac02496d88fa800153754b0a: Bootstrap starting.
I20260709 13:12:37.912276   807 tablet_bootstrap.cc:492] T def0acc1a0684b9b86755e1c34191d91 P 15c6b88567e44da395f245725ffe7f04: No bootstrap required, opened a new log
I20260709 13:12:37.912878   807 ts_tablet_manager.cc:1403] T def0acc1a0684b9b86755e1c34191d91 P 15c6b88567e44da395f245725ffe7f04: Time spent bootstrapping tablet: real 0.042s	user 0.014s	sys 0.000s
I20260709 13:12:37.917335   807 raft_consensus.cc:359] T def0acc1a0684b9b86755e1c34191d91 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:37.918150   807 raft_consensus.cc:385] T def0acc1a0684b9b86755e1c34191d91 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:37.918509   807 raft_consensus.cc:740] T def0acc1a0684b9b86755e1c34191d91 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15c6b88567e44da395f245725ffe7f04, State: Initialized, Role: FOLLOWER
I20260709 13:12:37.919456   807 consensus_queue.cc:260] T def0acc1a0684b9b86755e1c34191d91 P 15c6b88567e44da395f245725ffe7f04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:37.920233   805 tablet_bootstrap.cc:654] T d7f398f030d840cab518d8996f2bdbce P f35a78aeac02496d88fa800153754b0a: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:37.922072   807 ts_tablet_manager.cc:1434] T def0acc1a0684b9b86755e1c34191d91 P 15c6b88567e44da395f245725ffe7f04: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 13:12:37.924535   807 tablet_bootstrap.cc:492] T f75ad0645c22439c9a0611d214158d01 P 15c6b88567e44da395f245725ffe7f04: Bootstrap starting.
I20260709 13:12:37.938706   805 tablet_bootstrap.cc:492] T d7f398f030d840cab518d8996f2bdbce P f35a78aeac02496d88fa800153754b0a: No bootstrap required, opened a new log
I20260709 13:12:37.939481   805 ts_tablet_manager.cc:1403] T d7f398f030d840cab518d8996f2bdbce P f35a78aeac02496d88fa800153754b0a: Time spent bootstrapping tablet: real 0.033s	user 0.009s	sys 0.010s
I20260709 13:12:37.940430   809 tablet_bootstrap.cc:492] T b9b003cc8908492ab10df382680b5141 P 7d13b2cad49e4853831a427132e294d3: No bootstrap required, opened a new log
I20260709 13:12:37.940976   809 ts_tablet_manager.cc:1403] T b9b003cc8908492ab10df382680b5141 P 7d13b2cad49e4853831a427132e294d3: Time spent bootstrapping tablet: real 0.052s	user 0.017s	sys 0.014s
I20260709 13:12:37.943080   805 raft_consensus.cc:359] T d7f398f030d840cab518d8996f2bdbce P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:37.931803   807 tablet_bootstrap.cc:654] T f75ad0645c22439c9a0611d214158d01 P 15c6b88567e44da395f245725ffe7f04: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:37.943933   805 raft_consensus.cc:385] T d7f398f030d840cab518d8996f2bdbce P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:37.944440   805 raft_consensus.cc:740] T d7f398f030d840cab518d8996f2bdbce P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f35a78aeac02496d88fa800153754b0a, State: Initialized, Role: FOLLOWER
I20260709 13:12:37.943964   809 raft_consensus.cc:359] T b9b003cc8908492ab10df382680b5141 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:37.945540   809 raft_consensus.cc:385] T b9b003cc8908492ab10df382680b5141 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:37.945379   805 consensus_queue.cc:260] T d7f398f030d840cab518d8996f2bdbce P f35a78aeac02496d88fa800153754b0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:37.946029   809 raft_consensus.cc:740] T b9b003cc8908492ab10df382680b5141 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d13b2cad49e4853831a427132e294d3, State: Initialized, Role: FOLLOWER
I20260709 13:12:37.947042   809 consensus_queue.cc:260] T b9b003cc8908492ab10df382680b5141 P 7d13b2cad49e4853831a427132e294d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:37.949628   809 ts_tablet_manager.cc:1434] T b9b003cc8908492ab10df382680b5141 P 7d13b2cad49e4853831a427132e294d3: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.003s
I20260709 13:12:37.950733   809 tablet_bootstrap.cc:492] T bedab716985143c7a40f88c69d71ecd4 P 7d13b2cad49e4853831a427132e294d3: Bootstrap starting.
I20260709 13:12:37.955765   805 ts_tablet_manager.cc:1434] T d7f398f030d840cab518d8996f2bdbce P f35a78aeac02496d88fa800153754b0a: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 13:12:37.958033   809 tablet_bootstrap.cc:654] T bedab716985143c7a40f88c69d71ecd4 P 7d13b2cad49e4853831a427132e294d3: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:37.959853   805 tablet_bootstrap.cc:492] T 78d4c60d0e4b42ef9e2bc081980888af P f35a78aeac02496d88fa800153754b0a: Bootstrap starting.
I20260709 13:12:37.967703   805 tablet_bootstrap.cc:654] T 78d4c60d0e4b42ef9e2bc081980888af P f35a78aeac02496d88fa800153754b0a: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:37.975337   805 tablet_bootstrap.cc:492] T 78d4c60d0e4b42ef9e2bc081980888af P f35a78aeac02496d88fa800153754b0a: No bootstrap required, opened a new log
I20260709 13:12:37.976046   805 ts_tablet_manager.cc:1403] T 78d4c60d0e4b42ef9e2bc081980888af P f35a78aeac02496d88fa800153754b0a: Time spent bootstrapping tablet: real 0.016s	user 0.009s	sys 0.004s
I20260709 13:12:37.979295   805 raft_consensus.cc:359] T 78d4c60d0e4b42ef9e2bc081980888af P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:37.980278   805 raft_consensus.cc:385] T 78d4c60d0e4b42ef9e2bc081980888af P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:37.980700   805 raft_consensus.cc:740] T 78d4c60d0e4b42ef9e2bc081980888af P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f35a78aeac02496d88fa800153754b0a, State: Initialized, Role: FOLLOWER
I20260709 13:12:37.981660   805 consensus_queue.cc:260] T 78d4c60d0e4b42ef9e2bc081980888af P f35a78aeac02496d88fa800153754b0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:37.986479   807 tablet_bootstrap.cc:492] T f75ad0645c22439c9a0611d214158d01 P 15c6b88567e44da395f245725ffe7f04: No bootstrap required, opened a new log
I20260709 13:12:37.986685   805 ts_tablet_manager.cc:1434] T 78d4c60d0e4b42ef9e2bc081980888af P f35a78aeac02496d88fa800153754b0a: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.004s
I20260709 13:12:37.987195   807 ts_tablet_manager.cc:1403] T f75ad0645c22439c9a0611d214158d01 P 15c6b88567e44da395f245725ffe7f04: Time spent bootstrapping tablet: real 0.063s	user 0.012s	sys 0.000s
I20260709 13:12:37.988106   805 tablet_bootstrap.cc:492] T 16c6d1765fc145348e0a4f703fe6abdf P f35a78aeac02496d88fa800153754b0a: Bootstrap starting.
I20260709 13:12:37.990602   807 raft_consensus.cc:359] T f75ad0645c22439c9a0611d214158d01 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:37.991369   807 raft_consensus.cc:385] T f75ad0645c22439c9a0611d214158d01 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:37.991698   807 raft_consensus.cc:740] T f75ad0645c22439c9a0611d214158d01 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15c6b88567e44da395f245725ffe7f04, State: Initialized, Role: FOLLOWER
I20260709 13:12:37.992573   807 consensus_queue.cc:260] T f75ad0645c22439c9a0611d214158d01 P 15c6b88567e44da395f245725ffe7f04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:37.996537   807 ts_tablet_manager.cc:1434] T f75ad0645c22439c9a0611d214158d01 P 15c6b88567e44da395f245725ffe7f04: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.000s
I20260709 13:12:37.997648   807 tablet_bootstrap.cc:492] T 9082b9270cf74ed5b72b7bc4b0166836 P 15c6b88567e44da395f245725ffe7f04: Bootstrap starting.
I20260709 13:12:38.004483   807 tablet_bootstrap.cc:654] T 9082b9270cf74ed5b72b7bc4b0166836 P 15c6b88567e44da395f245725ffe7f04: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:38.008533   809 tablet_bootstrap.cc:492] T bedab716985143c7a40f88c69d71ecd4 P 7d13b2cad49e4853831a427132e294d3: No bootstrap required, opened a new log
I20260709 13:12:38.008906   805 tablet_bootstrap.cc:654] T 16c6d1765fc145348e0a4f703fe6abdf P f35a78aeac02496d88fa800153754b0a: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:38.009219   809 ts_tablet_manager.cc:1403] T bedab716985143c7a40f88c69d71ecd4 P 7d13b2cad49e4853831a427132e294d3: Time spent bootstrapping tablet: real 0.059s	user 0.018s	sys 0.016s
I20260709 13:12:38.013482   809 raft_consensus.cc:359] T bedab716985143c7a40f88c69d71ecd4 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.014618   809 raft_consensus.cc:385] T bedab716985143c7a40f88c69d71ecd4 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:38.015126   809 raft_consensus.cc:740] T bedab716985143c7a40f88c69d71ecd4 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d13b2cad49e4853831a427132e294d3, State: Initialized, Role: FOLLOWER
I20260709 13:12:38.016506   809 consensus_queue.cc:260] T bedab716985143c7a40f88c69d71ecd4 P 7d13b2cad49e4853831a427132e294d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.018201   805 tablet_bootstrap.cc:492] T 16c6d1765fc145348e0a4f703fe6abdf P f35a78aeac02496d88fa800153754b0a: No bootstrap required, opened a new log
I20260709 13:12:38.018894   805 ts_tablet_manager.cc:1403] T 16c6d1765fc145348e0a4f703fe6abdf P f35a78aeac02496d88fa800153754b0a: Time spent bootstrapping tablet: real 0.031s	user 0.002s	sys 0.015s
I20260709 13:12:38.022439   805 raft_consensus.cc:359] T 16c6d1765fc145348e0a4f703fe6abdf P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.026469   805 raft_consensus.cc:385] T 16c6d1765fc145348e0a4f703fe6abdf P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:38.025564   809 ts_tablet_manager.cc:1434] T bedab716985143c7a40f88c69d71ecd4 P 7d13b2cad49e4853831a427132e294d3: Time spent starting tablet: real 0.016s	user 0.005s	sys 0.004s
I20260709 13:12:38.026868   805 raft_consensus.cc:740] T 16c6d1765fc145348e0a4f703fe6abdf P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f35a78aeac02496d88fa800153754b0a, State: Initialized, Role: FOLLOWER
I20260709 13:12:38.028981   809 tablet_bootstrap.cc:492] T f75ad0645c22439c9a0611d214158d01 P 7d13b2cad49e4853831a427132e294d3: Bootstrap starting.
I20260709 13:12:38.029181   805 consensus_queue.cc:260] T 16c6d1765fc145348e0a4f703fe6abdf P f35a78aeac02496d88fa800153754b0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.026082   807 tablet_bootstrap.cc:492] T 9082b9270cf74ed5b72b7bc4b0166836 P 15c6b88567e44da395f245725ffe7f04: No bootstrap required, opened a new log
I20260709 13:12:38.031066   807 ts_tablet_manager.cc:1403] T 9082b9270cf74ed5b72b7bc4b0166836 P 15c6b88567e44da395f245725ffe7f04: Time spent bootstrapping tablet: real 0.034s	user 0.016s	sys 0.007s
I20260709 13:12:38.032091   805 ts_tablet_manager.cc:1434] T 16c6d1765fc145348e0a4f703fe6abdf P f35a78aeac02496d88fa800153754b0a: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:12:38.033308   805 tablet_bootstrap.cc:492] T def0acc1a0684b9b86755e1c34191d91 P f35a78aeac02496d88fa800153754b0a: Bootstrap starting.
I20260709 13:12:38.034569   807 raft_consensus.cc:359] T 9082b9270cf74ed5b72b7bc4b0166836 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.035377   807 raft_consensus.cc:385] T 9082b9270cf74ed5b72b7bc4b0166836 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:38.035692   807 raft_consensus.cc:740] T 9082b9270cf74ed5b72b7bc4b0166836 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15c6b88567e44da395f245725ffe7f04, State: Initialized, Role: FOLLOWER
I20260709 13:12:38.042476   807 consensus_queue.cc:260] T 9082b9270cf74ed5b72b7bc4b0166836 P 15c6b88567e44da395f245725ffe7f04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.045310   807 ts_tablet_manager.cc:1434] T 9082b9270cf74ed5b72b7bc4b0166836 P 15c6b88567e44da395f245725ffe7f04: Time spent starting tablet: real 0.014s	user 0.004s	sys 0.004s
I20260709 13:12:38.049335   785 raft_consensus.cc:493] T b9b003cc8908492ab10df382680b5141 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:38.049901   785 raft_consensus.cc:515] T b9b003cc8908492ab10df382680b5141 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.052096   785 leader_election.cc:290] T b9b003cc8908492ab10df382680b5141 P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181), 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695)
I20260709 13:12:38.040827   805 tablet_bootstrap.cc:654] T def0acc1a0684b9b86755e1c34191d91 P f35a78aeac02496d88fa800153754b0a: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:38.054839   657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9b003cc8908492ab10df382680b5141" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d13b2cad49e4853831a427132e294d3" is_pre_election: true
I20260709 13:12:38.055732   657 raft_consensus.cc:2468] T b9b003cc8908492ab10df382680b5141 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f35a78aeac02496d88fa800153754b0a in term 0.
I20260709 13:12:38.055848   582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9b003cc8908492ab10df382680b5141" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15c6b88567e44da395f245725ffe7f04" is_pre_election: true
I20260709 13:12:38.048151   809 tablet_bootstrap.cc:654] T f75ad0645c22439c9a0611d214158d01 P 7d13b2cad49e4853831a427132e294d3: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:38.057246   475 leader_election.cc:304] T b9b003cc8908492ab10df382680b5141 P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d13b2cad49e4853831a427132e294d3, f35a78aeac02496d88fa800153754b0a; no voters: 
W20260709 13:12:38.058089   472 leader_election.cc:343] T b9b003cc8908492ab10df382680b5141 P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:38.059104   785 raft_consensus.cc:2804] T b9b003cc8908492ab10df382680b5141 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:38.059581   785 raft_consensus.cc:493] T b9b003cc8908492ab10df382680b5141 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:38.060285   807 tablet_bootstrap.cc:492] T b9b003cc8908492ab10df382680b5141 P 15c6b88567e44da395f245725ffe7f04: Bootstrap starting.
I20260709 13:12:38.061272   785 raft_consensus.cc:3060] T b9b003cc8908492ab10df382680b5141 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:38.068245   807 tablet_bootstrap.cc:654] T b9b003cc8908492ab10df382680b5141 P 15c6b88567e44da395f245725ffe7f04: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:38.072304   809 tablet_bootstrap.cc:492] T f75ad0645c22439c9a0611d214158d01 P 7d13b2cad49e4853831a427132e294d3: No bootstrap required, opened a new log
I20260709 13:12:38.073114   809 ts_tablet_manager.cc:1403] T f75ad0645c22439c9a0611d214158d01 P 7d13b2cad49e4853831a427132e294d3: Time spent bootstrapping tablet: real 0.044s	user 0.008s	sys 0.006s
I20260709 13:12:38.076244   809 raft_consensus.cc:359] T f75ad0645c22439c9a0611d214158d01 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.079624   809 raft_consensus.cc:385] T f75ad0645c22439c9a0611d214158d01 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:38.080021   809 raft_consensus.cc:740] T f75ad0645c22439c9a0611d214158d01 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d13b2cad49e4853831a427132e294d3, State: Initialized, Role: FOLLOWER
I20260709 13:12:38.080967   809 consensus_queue.cc:260] T f75ad0645c22439c9a0611d214158d01 P 7d13b2cad49e4853831a427132e294d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.083842   785 raft_consensus.cc:515] T b9b003cc8908492ab10df382680b5141 P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.086699   809 ts_tablet_manager.cc:1434] T f75ad0645c22439c9a0611d214158d01 P 7d13b2cad49e4853831a427132e294d3: Time spent starting tablet: real 0.013s	user 0.000s	sys 0.005s
I20260709 13:12:38.087837   809 tablet_bootstrap.cc:492] T def0acc1a0684b9b86755e1c34191d91 P 7d13b2cad49e4853831a427132e294d3: Bootstrap starting.
I20260709 13:12:38.089035   806 raft_consensus.cc:493] T 78d4c60d0e4b42ef9e2bc081980888af P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:38.089758   806 raft_consensus.cc:515] T 78d4c60d0e4b42ef9e2bc081980888af P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.093366   582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d4c60d0e4b42ef9e2bc081980888af" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15c6b88567e44da395f245725ffe7f04" is_pre_election: true
W20260709 13:12:38.094822   472 leader_election.cc:343] T 78d4c60d0e4b42ef9e2bc081980888af P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:38.095220   657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d4c60d0e4b42ef9e2bc081980888af" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d13b2cad49e4853831a427132e294d3" is_pre_election: true
I20260709 13:12:38.096164   657 raft_consensus.cc:2468] T 78d4c60d0e4b42ef9e2bc081980888af P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f35a78aeac02496d88fa800153754b0a in term 0.
I20260709 13:12:38.098071   475 leader_election.cc:304] T 78d4c60d0e4b42ef9e2bc081980888af P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d13b2cad49e4853831a427132e294d3, f35a78aeac02496d88fa800153754b0a; no voters: 15c6b88567e44da395f245725ffe7f04
I20260709 13:12:38.098727   582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9b003cc8908492ab10df382680b5141" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15c6b88567e44da395f245725ffe7f04"
I20260709 13:12:38.098930   785 leader_election.cc:290] T b9b003cc8908492ab10df382680b5141 P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 election: Requested vote from peers 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181), 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695)
I20260709 13:12:38.099299   806 leader_election.cc:290] T 78d4c60d0e4b42ef9e2bc081980888af P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181), 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695)
I20260709 13:12:38.099502   785 raft_consensus.cc:2804] T 78d4c60d0e4b42ef9e2bc081980888af P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:38.099959   785 raft_consensus.cc:493] T 78d4c60d0e4b42ef9e2bc081980888af P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 13:12:38.100224   472 leader_election.cc:343] T b9b003cc8908492ab10df382680b5141 P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:38.100306   785 raft_consensus.cc:3060] T 78d4c60d0e4b42ef9e2bc081980888af P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:38.103910   807 tablet_bootstrap.cc:492] T b9b003cc8908492ab10df382680b5141 P 15c6b88567e44da395f245725ffe7f04: No bootstrap required, opened a new log
I20260709 13:12:38.104485   807 ts_tablet_manager.cc:1403] T b9b003cc8908492ab10df382680b5141 P 15c6b88567e44da395f245725ffe7f04: Time spent bootstrapping tablet: real 0.044s	user 0.008s	sys 0.008s
I20260709 13:12:38.106284   805 tablet_bootstrap.cc:492] T def0acc1a0684b9b86755e1c34191d91 P f35a78aeac02496d88fa800153754b0a: No bootstrap required, opened a new log
I20260709 13:12:38.106799   805 ts_tablet_manager.cc:1403] T def0acc1a0684b9b86755e1c34191d91 P f35a78aeac02496d88fa800153754b0a: Time spent bootstrapping tablet: real 0.074s	user 0.020s	sys 0.034s
I20260709 13:12:38.108542   809 tablet_bootstrap.cc:654] T def0acc1a0684b9b86755e1c34191d91 P 7d13b2cad49e4853831a427132e294d3: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:38.109784   805 raft_consensus.cc:359] T def0acc1a0684b9b86755e1c34191d91 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.109476   807 raft_consensus.cc:359] T b9b003cc8908492ab10df382680b5141 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.110514   805 raft_consensus.cc:385] T def0acc1a0684b9b86755e1c34191d91 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:38.110613   807 raft_consensus.cc:385] T b9b003cc8908492ab10df382680b5141 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:38.110893   805 raft_consensus.cc:740] T def0acc1a0684b9b86755e1c34191d91 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f35a78aeac02496d88fa800153754b0a, State: Initialized, Role: FOLLOWER
I20260709 13:12:38.111130   807 raft_consensus.cc:740] T b9b003cc8908492ab10df382680b5141 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15c6b88567e44da395f245725ffe7f04, State: Initialized, Role: FOLLOWER
I20260709 13:12:38.112202   657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9b003cc8908492ab10df382680b5141" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d13b2cad49e4853831a427132e294d3"
I20260709 13:12:38.113049   657 raft_consensus.cc:3060] T b9b003cc8908492ab10df382680b5141 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:38.113407   804 raft_consensus.cc:493] T f75ad0645c22439c9a0611d214158d01 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:38.113936   804 raft_consensus.cc:515] T f75ad0645c22439c9a0611d214158d01 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.116707   804 leader_election.cc:290] T f75ad0645c22439c9a0611d214158d01 P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881), 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695)
I20260709 13:12:38.119968   656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f75ad0645c22439c9a0611d214158d01" candidate_uuid: "15c6b88567e44da395f245725ffe7f04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d13b2cad49e4853831a427132e294d3" is_pre_election: true
I20260709 13:12:38.121701   807 consensus_queue.cc:260] T b9b003cc8908492ab10df382680b5141 P 15c6b88567e44da395f245725ffe7f04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.120770   657 raft_consensus.cc:2468] T b9b003cc8908492ab10df382680b5141 P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f35a78aeac02496d88fa800153754b0a in term 1.
I20260709 13:12:38.122275   804 raft_consensus.cc:493] T def0acc1a0684b9b86755e1c34191d91 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:38.120131   509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f75ad0645c22439c9a0611d214158d01" candidate_uuid: "15c6b88567e44da395f245725ffe7f04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f35a78aeac02496d88fa800153754b0a" is_pre_election: true
I20260709 13:12:38.122326   656 raft_consensus.cc:2468] T f75ad0645c22439c9a0611d214158d01 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15c6b88567e44da395f245725ffe7f04 in term 0.
I20260709 13:12:38.111825   805 consensus_queue.cc:260] T def0acc1a0684b9b86755e1c34191d91 P f35a78aeac02496d88fa800153754b0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.124464   785 raft_consensus.cc:515] T 78d4c60d0e4b42ef9e2bc081980888af P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.125980   804 raft_consensus.cc:515] T def0acc1a0684b9b86755e1c34191d91 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.127326   550 leader_election.cc:304] T f75ad0645c22439c9a0611d214158d01 P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15c6b88567e44da395f245725ffe7f04, 7d13b2cad49e4853831a427132e294d3; no voters: 
I20260709 13:12:38.129206   827 raft_consensus.cc:2804] T f75ad0645c22439c9a0611d214158d01 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:38.129587   827 raft_consensus.cc:493] T f75ad0645c22439c9a0611d214158d01 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:38.129917   827 raft_consensus.cc:3060] T f75ad0645c22439c9a0611d214158d01 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:38.129961   805 ts_tablet_manager.cc:1434] T def0acc1a0684b9b86755e1c34191d91 P f35a78aeac02496d88fa800153754b0a: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.002s
I20260709 13:12:38.131273   805 tablet_bootstrap.cc:492] T f75ad0645c22439c9a0611d214158d01 P f35a78aeac02496d88fa800153754b0a: Bootstrap starting.
I20260709 13:12:38.134775   475 leader_election.cc:304] T b9b003cc8908492ab10df382680b5141 P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d13b2cad49e4853831a427132e294d3, f35a78aeac02496d88fa800153754b0a; no voters: 15c6b88567e44da395f245725ffe7f04
I20260709 13:12:38.135712   826 raft_consensus.cc:2804] T b9b003cc8908492ab10df382680b5141 P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:38.136188   826 raft_consensus.cc:697] T b9b003cc8908492ab10df382680b5141 P f35a78aeac02496d88fa800153754b0a [term 1 LEADER]: Becoming Leader. State: Replica: f35a78aeac02496d88fa800153754b0a, State: Running, Role: LEADER
I20260709 13:12:38.137313   826 consensus_queue.cc:237] T b9b003cc8908492ab10df382680b5141 P f35a78aeac02496d88fa800153754b0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.138149   827 raft_consensus.cc:515] T f75ad0645c22439c9a0611d214158d01 P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.145779   806 raft_consensus.cc:493] T bedab716985143c7a40f88c69d71ecd4 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:38.146235   807 ts_tablet_manager.cc:1434] T b9b003cc8908492ab10df382680b5141 P 15c6b88567e44da395f245725ffe7f04: Time spent starting tablet: real 0.041s	user 0.006s	sys 0.001s
I20260709 13:12:38.146350   806 raft_consensus.cc:515] T bedab716985143c7a40f88c69d71ecd4 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.146957   804 leader_election.cc:290] T def0acc1a0684b9b86755e1c34191d91 P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881), 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695)
I20260709 13:12:38.147526   807 tablet_bootstrap.cc:492] T d7f398f030d840cab518d8996f2bdbce P 15c6b88567e44da395f245725ffe7f04: Bootstrap starting.
I20260709 13:12:38.148571   806 leader_election.cc:290] T bedab716985143c7a40f88c69d71ecd4 P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181), 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695)
I20260709 13:12:38.150345   582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedab716985143c7a40f88c69d71ecd4" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15c6b88567e44da395f245725ffe7f04" is_pre_election: true
I20260709 13:12:38.150790   785 leader_election.cc:290] T 78d4c60d0e4b42ef9e2bc081980888af P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 election: Requested vote from peers 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181), 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695)
I20260709 13:12:38.151090   582 raft_consensus.cc:2468] T bedab716985143c7a40f88c69d71ecd4 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f35a78aeac02496d88fa800153754b0a in term 0.
I20260709 13:12:38.140560   827 leader_election.cc:290] T f75ad0645c22439c9a0611d214158d01 P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 1 election: Requested vote from peers f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881), 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695)
I20260709 13:12:38.153523   472 leader_election.cc:304] T bedab716985143c7a40f88c69d71ecd4 P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15c6b88567e44da395f245725ffe7f04, f35a78aeac02496d88fa800153754b0a; no voters: 
I20260709 13:12:38.154091   582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d4c60d0e4b42ef9e2bc081980888af" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15c6b88567e44da395f245725ffe7f04"
W20260709 13:12:38.155620   472 leader_election.cc:343] T 78d4c60d0e4b42ef9e2bc081980888af P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:38.157410   657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedab716985143c7a40f88c69d71ecd4" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d13b2cad49e4853831a427132e294d3" is_pre_election: true
I20260709 13:12:38.157792   785 raft_consensus.cc:2804] T bedab716985143c7a40f88c69d71ecd4 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:38.158066   657 raft_consensus.cc:2468] T bedab716985143c7a40f88c69d71ecd4 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f35a78aeac02496d88fa800153754b0a in term 0.
I20260709 13:12:38.158264   785 raft_consensus.cc:493] T bedab716985143c7a40f88c69d71ecd4 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:38.158823   785 raft_consensus.cc:3060] T bedab716985143c7a40f88c69d71ecd4 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:38.159452   809 tablet_bootstrap.cc:492] T def0acc1a0684b9b86755e1c34191d91 P 7d13b2cad49e4853831a427132e294d3: No bootstrap required, opened a new log
I20260709 13:12:38.160224   809 ts_tablet_manager.cc:1403] T def0acc1a0684b9b86755e1c34191d91 P 7d13b2cad49e4853831a427132e294d3: Time spent bootstrapping tablet: real 0.073s	user 0.006s	sys 0.006s
I20260709 13:12:38.160931   657 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d4c60d0e4b42ef9e2bc081980888af" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d13b2cad49e4853831a427132e294d3"
I20260709 13:12:38.161574   657 raft_consensus.cc:3060] T 78d4c60d0e4b42ef9e2bc081980888af P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:38.163597   809 raft_consensus.cc:359] T def0acc1a0684b9b86755e1c34191d91 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.164363   809 raft_consensus.cc:385] T def0acc1a0684b9b86755e1c34191d91 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:38.164656   809 raft_consensus.cc:740] T def0acc1a0684b9b86755e1c34191d91 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d13b2cad49e4853831a427132e294d3, State: Initialized, Role: FOLLOWER
I20260709 13:12:38.165488   809 consensus_queue.cc:260] T def0acc1a0684b9b86755e1c34191d91 P 7d13b2cad49e4853831a427132e294d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.167783   807 tablet_bootstrap.cc:654] T d7f398f030d840cab518d8996f2bdbce P 15c6b88567e44da395f245725ffe7f04: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:38.167955   809 ts_tablet_manager.cc:1434] T def0acc1a0684b9b86755e1c34191d91 P 7d13b2cad49e4853831a427132e294d3: Time spent starting tablet: real 0.007s	user 0.003s	sys 0.004s
I20260709 13:12:38.169101   809 tablet_bootstrap.cc:492] T 9082b9270cf74ed5b72b7bc4b0166836 P 7d13b2cad49e4853831a427132e294d3: Bootstrap starting.
I20260709 13:12:38.170424   805 tablet_bootstrap.cc:654] T f75ad0645c22439c9a0611d214158d01 P f35a78aeac02496d88fa800153754b0a: Neither blocks nor log segments found. Creating new log.
W20260709 13:12:38.171578   550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42695, user_credentials={real_user=slave}} blocked reactor thread for 44357.7us
I20260709 13:12:38.173647   656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f75ad0645c22439c9a0611d214158d01" candidate_uuid: "15c6b88567e44da395f245725ffe7f04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d13b2cad49e4853831a427132e294d3"
I20260709 13:12:38.174190   509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f75ad0645c22439c9a0611d214158d01" candidate_uuid: "15c6b88567e44da395f245725ffe7f04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f35a78aeac02496d88fa800153754b0a"
I20260709 13:12:38.174291   656 raft_consensus.cc:3060] T f75ad0645c22439c9a0611d214158d01 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:38.177980   809 tablet_bootstrap.cc:654] T 9082b9270cf74ed5b72b7bc4b0166836 P 7d13b2cad49e4853831a427132e294d3: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:38.178656   806 raft_consensus.cc:493] T 9082b9270cf74ed5b72b7bc4b0166836 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:38.179167   806 raft_consensus.cc:515] T 9082b9270cf74ed5b72b7bc4b0166836 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.185281   582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9082b9270cf74ed5b72b7bc4b0166836" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15c6b88567e44da395f245725ffe7f04" is_pre_election: true
I20260709 13:12:38.185978   582 raft_consensus.cc:2468] T 9082b9270cf74ed5b72b7bc4b0166836 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f35a78aeac02496d88fa800153754b0a in term 0.
I20260709 13:12:38.187315   472 leader_election.cc:304] T 9082b9270cf74ed5b72b7bc4b0166836 P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15c6b88567e44da395f245725ffe7f04, f35a78aeac02496d88fa800153754b0a; no voters: 
I20260709 13:12:38.188890   658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "def0acc1a0684b9b86755e1c34191d91" candidate_uuid: "15c6b88567e44da395f245725ffe7f04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d13b2cad49e4853831a427132e294d3" is_pre_election: true
I20260709 13:12:38.189635   658 raft_consensus.cc:2468] T def0acc1a0684b9b86755e1c34191d91 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15c6b88567e44da395f245725ffe7f04 in term 0.
I20260709 13:12:38.192292   805 tablet_bootstrap.cc:492] T f75ad0645c22439c9a0611d214158d01 P f35a78aeac02496d88fa800153754b0a: No bootstrap required, opened a new log
I20260709 13:12:38.192888   805 ts_tablet_manager.cc:1403] T f75ad0645c22439c9a0611d214158d01 P f35a78aeac02496d88fa800153754b0a: Time spent bootstrapping tablet: real 0.062s	user 0.011s	sys 0.001s
I20260709 13:12:38.187220   656 raft_consensus.cc:2468] T f75ad0645c22439c9a0611d214158d01 P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15c6b88567e44da395f245725ffe7f04 in term 1.
I20260709 13:12:38.196772   735 raft_consensus.cc:493] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:38.197366   735 raft_consensus.cc:515] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.198333   833 raft_consensus.cc:2804] T 9082b9270cf74ed5b72b7bc4b0166836 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:38.198890   833 raft_consensus.cc:493] T 9082b9270cf74ed5b72b7bc4b0166836 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:38.199304   833 raft_consensus.cc:3060] T 9082b9270cf74ed5b72b7bc4b0166836 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:38.201174   735 leader_election.cc:290] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881), 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181)
I20260709 13:12:38.201207   509 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "def0acc1a0684b9b86755e1c34191d91" candidate_uuid: "15c6b88567e44da395f245725ffe7f04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f35a78aeac02496d88fa800153754b0a" is_pre_election: true
I20260709 13:12:38.202281   509 raft_consensus.cc:2468] T def0acc1a0684b9b86755e1c34191d91 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15c6b88567e44da395f245725ffe7f04 in term 0.
I20260709 13:12:38.202991   508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16c6d1765fc145348e0a4f703fe6abdf" candidate_uuid: "7d13b2cad49e4853831a427132e294d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f35a78aeac02496d88fa800153754b0a" is_pre_election: true
I20260709 13:12:38.203661   508 raft_consensus.cc:2468] T 16c6d1765fc145348e0a4f703fe6abdf P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7d13b2cad49e4853831a427132e294d3 in term 0.
I20260709 13:12:38.203565   785 raft_consensus.cc:515] T bedab716985143c7a40f88c69d71ecd4 P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.204272   582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16c6d1765fc145348e0a4f703fe6abdf" candidate_uuid: "7d13b2cad49e4853831a427132e294d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15c6b88567e44da395f245725ffe7f04" is_pre_election: true
W20260709 13:12:38.205708   621 leader_election.cc:343] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:38.206979   809 tablet_bootstrap.cc:492] T 9082b9270cf74ed5b72b7bc4b0166836 P 7d13b2cad49e4853831a427132e294d3: No bootstrap required, opened a new log
I20260709 13:12:38.206890   624 leader_election.cc:304] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d13b2cad49e4853831a427132e294d3, f35a78aeac02496d88fa800153754b0a; no voters: 15c6b88567e44da395f245725ffe7f04
I20260709 13:12:38.207741   809 ts_tablet_manager.cc:1403] T 9082b9270cf74ed5b72b7bc4b0166836 P 7d13b2cad49e4853831a427132e294d3: Time spent bootstrapping tablet: real 0.039s	user 0.011s	sys 0.002s
I20260709 13:12:38.208220   735 raft_consensus.cc:2804] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:38.208799   735 raft_consensus.cc:493] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:38.209214   735 raft_consensus.cc:3060] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:38.211791   809 raft_consensus.cc:359] T 9082b9270cf74ed5b72b7bc4b0166836 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.212793   809 raft_consensus.cc:385] T 9082b9270cf74ed5b72b7bc4b0166836 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:38.213145   809 raft_consensus.cc:740] T 9082b9270cf74ed5b72b7bc4b0166836 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d13b2cad49e4853831a427132e294d3, State: Initialized, Role: FOLLOWER
W20260709 13:12:38.214255   550 leader_election.cc:343] T f75ad0645c22439c9a0611d214158d01 P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:12:38.221683   550 leader_election.cc:343] T f75ad0645c22439c9a0611d214158d01 P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:38.214080   809 consensus_queue.cc:260] T 9082b9270cf74ed5b72b7bc4b0166836 P 7d13b2cad49e4853831a427132e294d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.222503   807 tablet_bootstrap.cc:492] T d7f398f030d840cab518d8996f2bdbce P 15c6b88567e44da395f245725ffe7f04: No bootstrap required, opened a new log
I20260709 13:12:38.223109   807 ts_tablet_manager.cc:1403] T d7f398f030d840cab518d8996f2bdbce P 15c6b88567e44da395f245725ffe7f04: Time spent bootstrapping tablet: real 0.076s	user 0.012s	sys 0.007s
I20260709 13:12:38.215605   806 leader_election.cc:290] T 9082b9270cf74ed5b72b7bc4b0166836 P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181), 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695)
I20260709 13:12:38.217306   656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9082b9270cf74ed5b72b7bc4b0166836" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d13b2cad49e4853831a427132e294d3" is_pre_election: true
I20260709 13:12:38.226454   656 raft_consensus.cc:2468] T 9082b9270cf74ed5b72b7bc4b0166836 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f35a78aeac02496d88fa800153754b0a in term 0.
I20260709 13:12:38.217758   785 leader_election.cc:290] T bedab716985143c7a40f88c69d71ecd4 P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 election: Requested vote from peers 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181), 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695)
I20260709 13:12:38.218734   582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedab716985143c7a40f88c69d71ecd4" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15c6b88567e44da395f245725ffe7f04"
I20260709 13:12:38.219590   658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bedab716985143c7a40f88c69d71ecd4" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d13b2cad49e4853831a427132e294d3"
I20260709 13:12:38.227582   658 raft_consensus.cc:3060] T bedab716985143c7a40f88c69d71ecd4 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:38.228714   657 raft_consensus.cc:2468] T 78d4c60d0e4b42ef9e2bc081980888af P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f35a78aeac02496d88fa800153754b0a in term 1.
I20260709 13:12:38.196233   805 raft_consensus.cc:359] T f75ad0645c22439c9a0611d214158d01 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.231016   805 raft_consensus.cc:385] T f75ad0645c22439c9a0611d214158d01 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:38.231678   805 raft_consensus.cc:740] T f75ad0645c22439c9a0611d214158d01 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f35a78aeac02496d88fa800153754b0a, State: Initialized, Role: FOLLOWER
I20260709 13:12:38.230530   582 raft_consensus.cc:3060] T bedab716985143c7a40f88c69d71ecd4 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:38.237203   806 raft_consensus.cc:493] T d7f398f030d840cab518d8996f2bdbce P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:38.237735   806 raft_consensus.cc:515] T d7f398f030d840cab518d8996f2bdbce P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.239732   806 leader_election.cc:290] T d7f398f030d840cab518d8996f2bdbce P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181), 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695)
I20260709 13:12:38.222538   550 leader_election.cc:304] T def0acc1a0684b9b86755e1c34191d91 P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15c6b88567e44da395f245725ffe7f04, f35a78aeac02496d88fa800153754b0a; no voters: 
I20260709 13:12:38.242146   550 leader_election.cc:304] T f75ad0645c22439c9a0611d214158d01 P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 15c6b88567e44da395f245725ffe7f04, 7d13b2cad49e4853831a427132e294d3; no voters: f35a78aeac02496d88fa800153754b0a
I20260709 13:12:38.242992   827 raft_consensus.cc:2804] T def0acc1a0684b9b86755e1c34191d91 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:38.243363   827 raft_consensus.cc:493] T def0acc1a0684b9b86755e1c34191d91 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:38.243714   827 raft_consensus.cc:3060] T def0acc1a0684b9b86755e1c34191d91 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:38.243484   829 raft_consensus.cc:2804] T f75ad0645c22439c9a0611d214158d01 P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:38.247066   829 raft_consensus.cc:697] T f75ad0645c22439c9a0611d214158d01 P 15c6b88567e44da395f245725ffe7f04 [term 1 LEADER]: Becoming Leader. State: Replica: 15c6b88567e44da395f245725ffe7f04, State: Running, Role: LEADER
I20260709 13:12:38.248327   829 consensus_queue.cc:237] T f75ad0645c22439c9a0611d214158d01 P 15c6b88567e44da395f245725ffe7f04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.239951   582 raft_consensus.cc:2468] T bedab716985143c7a40f88c69d71ecd4 P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f35a78aeac02496d88fa800153754b0a in term 1.
I20260709 13:12:38.250065   809 ts_tablet_manager.cc:1434] T 9082b9270cf74ed5b72b7bc4b0166836 P 7d13b2cad49e4853831a427132e294d3: Time spent starting tablet: real 0.042s	user 0.001s	sys 0.012s
I20260709 13:12:38.245194   658 raft_consensus.cc:2468] T bedab716985143c7a40f88c69d71ecd4 P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f35a78aeac02496d88fa800153754b0a in term 1.
I20260709 13:12:38.252539   805 consensus_queue.cc:260] T f75ad0645c22439c9a0611d214158d01 P f35a78aeac02496d88fa800153754b0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.253819   735 raft_consensus.cc:515] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.254119   582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7f398f030d840cab518d8996f2bdbce" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15c6b88567e44da395f245725ffe7f04" is_pre_election: true
I20260709 13:12:38.256680   805 ts_tablet_manager.cc:1434] T f75ad0645c22439c9a0611d214158d01 P f35a78aeac02496d88fa800153754b0a: Time spent starting tablet: real 0.063s	user 0.006s	sys 0.000s
I20260709 13:12:38.254767   472 leader_election.cc:304] T bedab716985143c7a40f88c69d71ecd4 P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15c6b88567e44da395f245725ffe7f04, f35a78aeac02496d88fa800153754b0a; no voters: 
I20260709 13:12:38.257980   805 tablet_bootstrap.cc:492] T 42ebb32ae777484fab5e26a137cbd4f6 P f35a78aeac02496d88fa800153754b0a: Bootstrap starting.
I20260709 13:12:38.256399   833 raft_consensus.cc:515] T 9082b9270cf74ed5b72b7bc4b0166836 P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.262223   806 raft_consensus.cc:2804] T bedab716985143c7a40f88c69d71ecd4 P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:38.262895   806 raft_consensus.cc:697] T bedab716985143c7a40f88c69d71ecd4 P f35a78aeac02496d88fa800153754b0a [term 1 LEADER]: Becoming Leader. State: Replica: f35a78aeac02496d88fa800153754b0a, State: Running, Role: LEADER
I20260709 13:12:38.263913   833 leader_election.cc:290] T 9082b9270cf74ed5b72b7bc4b0166836 P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 election: Requested vote from peers 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181), 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695)
I20260709 13:12:38.263729   806 consensus_queue.cc:237] T bedab716985143c7a40f88c69d71ecd4 P f35a78aeac02496d88fa800153754b0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.261344   406 catalog_manager.cc:5697] T b9b003cc8908492ab10df382680b5141 P f35a78aeac02496d88fa800153754b0a reported cstate change: term changed from 0 to 1, leader changed from <none> to f35a78aeac02496d88fa800153754b0a (127.21.63.1). New cstate: current_term: 1 leader_uuid: "f35a78aeac02496d88fa800153754b0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:38.269049   735 leader_election.cc:290] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 [CANDIDATE]: Term 1 election: Requested vote from peers f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881), 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181)
I20260709 13:12:38.270040   809 tablet_bootstrap.cc:492] T d7f398f030d840cab518d8996f2bdbce P 7d13b2cad49e4853831a427132e294d3: Bootstrap starting.
I20260709 13:12:38.270294   584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16c6d1765fc145348e0a4f703fe6abdf" candidate_uuid: "7d13b2cad49e4853831a427132e294d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15c6b88567e44da395f245725ffe7f04"
I20260709 13:12:38.271356   805 tablet_bootstrap.cc:654] T 42ebb32ae777484fab5e26a137cbd4f6 P f35a78aeac02496d88fa800153754b0a: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:38.257602   827 raft_consensus.cc:515] T def0acc1a0684b9b86755e1c34191d91 P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
W20260709 13:12:38.271870   621 leader_election.cc:343] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:38.271878   658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7f398f030d840cab518d8996f2bdbce" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d13b2cad49e4853831a427132e294d3" is_pre_election: true
I20260709 13:12:38.250582   807 raft_consensus.cc:359] T d7f398f030d840cab518d8996f2bdbce P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.276187   807 raft_consensus.cc:385] T d7f398f030d840cab518d8996f2bdbce P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:38.276475   584 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9082b9270cf74ed5b72b7bc4b0166836" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15c6b88567e44da395f245725ffe7f04"
I20260709 13:12:38.276997   807 raft_consensus.cc:740] T d7f398f030d840cab518d8996f2bdbce P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15c6b88567e44da395f245725ffe7f04, State: Initialized, Role: FOLLOWER
I20260709 13:12:38.277133   584 raft_consensus.cc:3060] T 9082b9270cf74ed5b72b7bc4b0166836 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:38.275391   475 leader_election.cc:304] T 78d4c60d0e4b42ef9e2bc081980888af P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d13b2cad49e4853831a427132e294d3, f35a78aeac02496d88fa800153754b0a; no voters: 15c6b88567e44da395f245725ffe7f04
I20260709 13:12:38.277930   807 consensus_queue.cc:260] T d7f398f030d840cab518d8996f2bdbce P 15c6b88567e44da395f245725ffe7f04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.277931   827 leader_election.cc:290] T def0acc1a0684b9b86755e1c34191d91 P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 1 election: Requested vote from peers f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881), 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695)
I20260709 13:12:38.279248   508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "def0acc1a0684b9b86755e1c34191d91" candidate_uuid: "15c6b88567e44da395f245725ffe7f04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f35a78aeac02496d88fa800153754b0a"
I20260709 13:12:38.279562   785 raft_consensus.cc:2804] T 78d4c60d0e4b42ef9e2bc081980888af P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:38.285171   508 raft_consensus.cc:3060] T def0acc1a0684b9b86755e1c34191d91 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:38.285284   785 raft_consensus.cc:697] T 78d4c60d0e4b42ef9e2bc081980888af P f35a78aeac02496d88fa800153754b0a [term 1 LEADER]: Becoming Leader. State: Replica: f35a78aeac02496d88fa800153754b0a, State: Running, Role: LEADER
I20260709 13:12:38.286783   807 ts_tablet_manager.cc:1434] T d7f398f030d840cab518d8996f2bdbce P 15c6b88567e44da395f245725ffe7f04: Time spent starting tablet: real 0.063s	user 0.003s	sys 0.004s
I20260709 13:12:38.286680   785 consensus_queue.cc:237] T 78d4c60d0e4b42ef9e2bc081980888af P f35a78aeac02496d88fa800153754b0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.291289   807 tablet_bootstrap.cc:492] T 78d4c60d0e4b42ef9e2bc081980888af P 15c6b88567e44da395f245725ffe7f04: Bootstrap starting.
I20260709 13:12:38.295569   584 raft_consensus.cc:2468] T 9082b9270cf74ed5b72b7bc4b0166836 P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f35a78aeac02496d88fa800153754b0a in term 1.
I20260709 13:12:38.296906   472 leader_election.cc:304] T 9082b9270cf74ed5b72b7bc4b0166836 P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15c6b88567e44da395f245725ffe7f04, f35a78aeac02496d88fa800153754b0a; no voters: 
I20260709 13:12:38.299503   508 raft_consensus.cc:2468] T def0acc1a0684b9b86755e1c34191d91 P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15c6b88567e44da395f245725ffe7f04 in term 1.
I20260709 13:12:38.301007   550 leader_election.cc:304] T def0acc1a0684b9b86755e1c34191d91 P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15c6b88567e44da395f245725ffe7f04, f35a78aeac02496d88fa800153754b0a; no voters: 
I20260709 13:12:38.301908   829 raft_consensus.cc:2804] T def0acc1a0684b9b86755e1c34191d91 P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:38.302770   829 raft_consensus.cc:697] T def0acc1a0684b9b86755e1c34191d91 P 15c6b88567e44da395f245725ffe7f04 [term 1 LEADER]: Becoming Leader. State: Replica: 15c6b88567e44da395f245725ffe7f04, State: Running, Role: LEADER
I20260709 13:12:38.303901   829 consensus_queue.cc:237] T def0acc1a0684b9b86755e1c34191d91 P 15c6b88567e44da395f245725ffe7f04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.304885   826 raft_consensus.cc:2804] T 9082b9270cf74ed5b72b7bc4b0166836 P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:38.305466   826 raft_consensus.cc:697] T 9082b9270cf74ed5b72b7bc4b0166836 P f35a78aeac02496d88fa800153754b0a [term 1 LEADER]: Becoming Leader. State: Replica: f35a78aeac02496d88fa800153754b0a, State: Running, Role: LEADER
I20260709 13:12:38.306371   826 consensus_queue.cc:237] T 9082b9270cf74ed5b72b7bc4b0166836 P f35a78aeac02496d88fa800153754b0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.308458   805 tablet_bootstrap.cc:492] T 42ebb32ae777484fab5e26a137cbd4f6 P f35a78aeac02496d88fa800153754b0a: No bootstrap required, opened a new log
I20260709 13:12:38.309036   805 ts_tablet_manager.cc:1403] T 42ebb32ae777484fab5e26a137cbd4f6 P f35a78aeac02496d88fa800153754b0a: Time spent bootstrapping tablet: real 0.052s	user 0.014s	sys 0.006s
I20260709 13:12:38.311968   805 raft_consensus.cc:359] T 42ebb32ae777484fab5e26a137cbd4f6 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.312729   805 raft_consensus.cc:385] T 42ebb32ae777484fab5e26a137cbd4f6 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:38.313009   805 raft_consensus.cc:740] T 42ebb32ae777484fab5e26a137cbd4f6 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f35a78aeac02496d88fa800153754b0a, State: Initialized, Role: FOLLOWER
I20260709 13:12:38.313687   805 consensus_queue.cc:260] T 42ebb32ae777484fab5e26a137cbd4f6 P f35a78aeac02496d88fa800153754b0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.304152   403 catalog_manager.cc:5697] T f75ad0645c22439c9a0611d214158d01 P 15c6b88567e44da395f245725ffe7f04 reported cstate change: term changed from 0 to 1, leader changed from <none> to 15c6b88567e44da395f245725ffe7f04 (127.21.63.2). New cstate: current_term: 1 leader_uuid: "15c6b88567e44da395f245725ffe7f04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:38.287986   582 raft_consensus.cc:2468] T d7f398f030d840cab518d8996f2bdbce P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f35a78aeac02496d88fa800153754b0a in term 0.
I20260709 13:12:38.293118   658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "def0acc1a0684b9b86755e1c34191d91" candidate_uuid: "15c6b88567e44da395f245725ffe7f04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d13b2cad49e4853831a427132e294d3"
I20260709 13:12:38.317042   658 raft_consensus.cc:3060] T def0acc1a0684b9b86755e1c34191d91 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:12:38.281208   475 leader_election.cc:343] T d7f398f030d840cab518d8996f2bdbce P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:38.316143   807 tablet_bootstrap.cc:654] T 78d4c60d0e4b42ef9e2bc081980888af P 15c6b88567e44da395f245725ffe7f04: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:38.328872   472 leader_election.cc:304] T d7f398f030d840cab518d8996f2bdbce P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 15c6b88567e44da395f245725ffe7f04, f35a78aeac02496d88fa800153754b0a; no voters: 7d13b2cad49e4853831a427132e294d3
I20260709 13:12:38.315717   805 ts_tablet_manager.cc:1434] T 42ebb32ae777484fab5e26a137cbd4f6 P f35a78aeac02496d88fa800153754b0a: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:12:38.330076   806 raft_consensus.cc:2804] T d7f398f030d840cab518d8996f2bdbce P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:38.329456   658 raft_consensus.cc:2468] T def0acc1a0684b9b86755e1c34191d91 P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15c6b88567e44da395f245725ffe7f04 in term 1.
I20260709 13:12:38.330646   806 raft_consensus.cc:493] T d7f398f030d840cab518d8996f2bdbce P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:38.331272   806 raft_consensus.cc:3060] T d7f398f030d840cab518d8996f2bdbce P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:12:38.335209   475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:42695, user_credentials={real_user=slave}} blocked reactor thread for 54031.9us
I20260709 13:12:38.337069   658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9082b9270cf74ed5b72b7bc4b0166836" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d13b2cad49e4853831a427132e294d3"
I20260709 13:12:38.337697   658 raft_consensus.cc:3060] T 9082b9270cf74ed5b72b7bc4b0166836 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:38.342767   809 tablet_bootstrap.cc:654] T d7f398f030d840cab518d8996f2bdbce P 7d13b2cad49e4853831a427132e294d3: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:38.343266   508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16c6d1765fc145348e0a4f703fe6abdf" candidate_uuid: "7d13b2cad49e4853831a427132e294d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f35a78aeac02496d88fa800153754b0a"
I20260709 13:12:38.343961   508 raft_consensus.cc:3060] T 16c6d1765fc145348e0a4f703fe6abdf P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:38.349474   807 tablet_bootstrap.cc:492] T 78d4c60d0e4b42ef9e2bc081980888af P 15c6b88567e44da395f245725ffe7f04: No bootstrap required, opened a new log
I20260709 13:12:38.350011   807 ts_tablet_manager.cc:1403] T 78d4c60d0e4b42ef9e2bc081980888af P 15c6b88567e44da395f245725ffe7f04: Time spent bootstrapping tablet: real 0.059s	user 0.011s	sys 0.009s
I20260709 13:12:38.353207   508 raft_consensus.cc:2468] T 16c6d1765fc145348e0a4f703fe6abdf P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d13b2cad49e4853831a427132e294d3 in term 1.
I20260709 13:12:38.354791   658 raft_consensus.cc:2468] T 9082b9270cf74ed5b72b7bc4b0166836 P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f35a78aeac02496d88fa800153754b0a in term 1.
I20260709 13:12:38.355367   624 leader_election.cc:304] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7d13b2cad49e4853831a427132e294d3, f35a78aeac02496d88fa800153754b0a; no voters: 15c6b88567e44da395f245725ffe7f04
I20260709 13:12:38.353333   807 raft_consensus.cc:359] T 78d4c60d0e4b42ef9e2bc081980888af P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.356098   807 raft_consensus.cc:385] T 78d4c60d0e4b42ef9e2bc081980888af P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:38.356264   735 raft_consensus.cc:2804] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:38.356467   807 raft_consensus.cc:740] T 78d4c60d0e4b42ef9e2bc081980888af P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15c6b88567e44da395f245725ffe7f04, State: Initialized, Role: FOLLOWER
I20260709 13:12:38.356980   735 raft_consensus.cc:697] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 [term 1 LEADER]: Becoming Leader. State: Replica: 7d13b2cad49e4853831a427132e294d3, State: Running, Role: LEADER
I20260709 13:12:38.357910   807 consensus_queue.cc:260] T 78d4c60d0e4b42ef9e2bc081980888af P 15c6b88567e44da395f245725ffe7f04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.358067   735 consensus_queue.cc:237] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.360381   807 ts_tablet_manager.cc:1434] T 78d4c60d0e4b42ef9e2bc081980888af P 15c6b88567e44da395f245725ffe7f04: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 13:12:38.361493   807 tablet_bootstrap.cc:492] T 16c6d1765fc145348e0a4f703fe6abdf P 15c6b88567e44da395f245725ffe7f04: Bootstrap starting.
I20260709 13:12:38.368585   807 tablet_bootstrap.cc:654] T 16c6d1765fc145348e0a4f703fe6abdf P 15c6b88567e44da395f245725ffe7f04: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:38.373639   806 raft_consensus.cc:515] T d7f398f030d840cab518d8996f2bdbce P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.376747   582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7f398f030d840cab518d8996f2bdbce" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15c6b88567e44da395f245725ffe7f04"
I20260709 13:12:38.377351   582 raft_consensus.cc:3060] T d7f398f030d840cab518d8996f2bdbce P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:38.379854   403 catalog_manager.cc:5697] T def0acc1a0684b9b86755e1c34191d91 P 15c6b88567e44da395f245725ffe7f04 reported cstate change: term changed from 0 to 1, leader changed from <none> to 15c6b88567e44da395f245725ffe7f04 (127.21.63.2). New cstate: current_term: 1 leader_uuid: "15c6b88567e44da395f245725ffe7f04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:38.380820   807 tablet_bootstrap.cc:492] T 16c6d1765fc145348e0a4f703fe6abdf P 15c6b88567e44da395f245725ffe7f04: No bootstrap required, opened a new log
I20260709 13:12:38.381579   807 ts_tablet_manager.cc:1403] T 16c6d1765fc145348e0a4f703fe6abdf P 15c6b88567e44da395f245725ffe7f04: Time spent bootstrapping tablet: real 0.020s	user 0.011s	sys 0.007s
I20260709 13:12:38.384987   807 raft_consensus.cc:359] T 16c6d1765fc145348e0a4f703fe6abdf P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.385725   807 raft_consensus.cc:385] T 16c6d1765fc145348e0a4f703fe6abdf P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:38.386014   807 raft_consensus.cc:740] T 16c6d1765fc145348e0a4f703fe6abdf P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15c6b88567e44da395f245725ffe7f04, State: Initialized, Role: FOLLOWER
I20260709 13:12:38.385236   582 raft_consensus.cc:2468] T d7f398f030d840cab518d8996f2bdbce P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f35a78aeac02496d88fa800153754b0a in term 1.
I20260709 13:12:38.386860   807 consensus_queue.cc:260] T 16c6d1765fc145348e0a4f703fe6abdf P 15c6b88567e44da395f245725ffe7f04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.388149   472 leader_election.cc:304] T d7f398f030d840cab518d8996f2bdbce P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15c6b88567e44da395f245725ffe7f04, f35a78aeac02496d88fa800153754b0a; no voters: 
I20260709 13:12:38.389165   807 ts_tablet_manager.cc:1434] T 16c6d1765fc145348e0a4f703fe6abdf P 15c6b88567e44da395f245725ffe7f04: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:12:38.389151   826 raft_consensus.cc:2804] T d7f398f030d840cab518d8996f2bdbce P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:38.390321   807 tablet_bootstrap.cc:492] T 42ebb32ae777484fab5e26a137cbd4f6 P 15c6b88567e44da395f245725ffe7f04: Bootstrap starting.
I20260709 13:12:38.390223   826 raft_consensus.cc:697] T d7f398f030d840cab518d8996f2bdbce P f35a78aeac02496d88fa800153754b0a [term 1 LEADER]: Becoming Leader. State: Replica: f35a78aeac02496d88fa800153754b0a, State: Running, Role: LEADER
I20260709 13:12:38.391426   826 consensus_queue.cc:237] T d7f398f030d840cab518d8996f2bdbce P f35a78aeac02496d88fa800153754b0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.392848   658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7f398f030d840cab518d8996f2bdbce" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d13b2cad49e4853831a427132e294d3"
I20260709 13:12:38.392418   406 catalog_manager.cc:5697] T 78d4c60d0e4b42ef9e2bc081980888af P f35a78aeac02496d88fa800153754b0a reported cstate change: term changed from 0 to 1, leader changed from <none> to f35a78aeac02496d88fa800153754b0a (127.21.63.1). New cstate: current_term: 1 leader_uuid: "f35a78aeac02496d88fa800153754b0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:38.393198   806 leader_election.cc:290] T d7f398f030d840cab518d8996f2bdbce P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 election: Requested vote from peers 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181), 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695)
I20260709 13:12:38.393956   829 raft_consensus.cc:493] T b9b003cc8908492ab10df382680b5141 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:38.394860   829 raft_consensus.cc:515] T b9b003cc8908492ab10df382680b5141 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.394038   406 catalog_manager.cc:5697] T bedab716985143c7a40f88c69d71ecd4 P f35a78aeac02496d88fa800153754b0a reported cstate change: term changed from 0 to 1, leader changed from <none> to f35a78aeac02496d88fa800153754b0a (127.21.63.1). New cstate: current_term: 1 leader_uuid: "f35a78aeac02496d88fa800153754b0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:38.397341   829 leader_election.cc:290] T b9b003cc8908492ab10df382680b5141 P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881), 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695)
I20260709 13:12:38.397711   406 catalog_manager.cc:5697] T 9082b9270cf74ed5b72b7bc4b0166836 P f35a78aeac02496d88fa800153754b0a reported cstate change: term changed from 0 to 1, leader changed from <none> to f35a78aeac02496d88fa800153754b0a (127.21.63.1). New cstate: current_term: 1 leader_uuid: "f35a78aeac02496d88fa800153754b0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } health_report { overall_health: UNKNOWN } } }
W20260709 13:12:38.398346   475 leader_election.cc:343] T d7f398f030d840cab518d8996f2bdbce P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:38.398761   658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9b003cc8908492ab10df382680b5141" candidate_uuid: "15c6b88567e44da395f245725ffe7f04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d13b2cad49e4853831a427132e294d3" is_pre_election: true
I20260709 13:12:38.399569   658 raft_consensus.cc:2393] T b9b003cc8908492ab10df382680b5141 P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 15c6b88567e44da395f245725ffe7f04 in current term 1: Already voted for candidate f35a78aeac02496d88fa800153754b0a in this term.
I20260709 13:12:38.397790   508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b9b003cc8908492ab10df382680b5141" candidate_uuid: "15c6b88567e44da395f245725ffe7f04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f35a78aeac02496d88fa800153754b0a" is_pre_election: true
I20260709 13:12:38.407279   550 leader_election.cc:304] T b9b003cc8908492ab10df382680b5141 P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 15c6b88567e44da395f245725ffe7f04; no voters: 7d13b2cad49e4853831a427132e294d3, f35a78aeac02496d88fa800153754b0a
I20260709 13:12:38.408620   829 raft_consensus.cc:3060] T b9b003cc8908492ab10df382680b5141 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:38.411494   404 catalog_manager.cc:5697] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7d13b2cad49e4853831a427132e294d3 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "7d13b2cad49e4853831a427132e294d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:38.416469   829 raft_consensus.cc:2749] T b9b003cc8908492ab10df382680b5141 P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:38.419875   809 tablet_bootstrap.cc:492] T d7f398f030d840cab518d8996f2bdbce P 7d13b2cad49e4853831a427132e294d3: No bootstrap required, opened a new log
I20260709 13:12:38.420394   809 ts_tablet_manager.cc:1403] T d7f398f030d840cab518d8996f2bdbce P 7d13b2cad49e4853831a427132e294d3: Time spent bootstrapping tablet: real 0.151s	user 0.002s	sys 0.014s
I20260709 13:12:38.423215   809 raft_consensus.cc:359] T d7f398f030d840cab518d8996f2bdbce P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.424228   809 raft_consensus.cc:385] T d7f398f030d840cab518d8996f2bdbce P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:38.424480   809 raft_consensus.cc:740] T d7f398f030d840cab518d8996f2bdbce P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d13b2cad49e4853831a427132e294d3, State: Initialized, Role: FOLLOWER
I20260709 13:12:38.421653   807 tablet_bootstrap.cc:654] T 42ebb32ae777484fab5e26a137cbd4f6 P 15c6b88567e44da395f245725ffe7f04: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:38.425210   809 consensus_queue.cc:260] T d7f398f030d840cab518d8996f2bdbce P 7d13b2cad49e4853831a427132e294d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.434465   809 ts_tablet_manager.cc:1434] T d7f398f030d840cab518d8996f2bdbce P 7d13b2cad49e4853831a427132e294d3: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 13:12:38.435950   809 tablet_bootstrap.cc:492] T 42ebb32ae777484fab5e26a137cbd4f6 P 7d13b2cad49e4853831a427132e294d3: Bootstrap starting.
I20260709 13:12:38.441380   809 tablet_bootstrap.cc:654] T 42ebb32ae777484fab5e26a137cbd4f6 P 7d13b2cad49e4853831a427132e294d3: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:38.431923   406 catalog_manager.cc:5697] T d7f398f030d840cab518d8996f2bdbce P f35a78aeac02496d88fa800153754b0a reported cstate change: term changed from 0 to 1, leader changed from <none> to f35a78aeac02496d88fa800153754b0a (127.21.63.1). New cstate: current_term: 1 leader_uuid: "f35a78aeac02496d88fa800153754b0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:38.445319   807 tablet_bootstrap.cc:492] T 42ebb32ae777484fab5e26a137cbd4f6 P 15c6b88567e44da395f245725ffe7f04: No bootstrap required, opened a new log
I20260709 13:12:38.445817   807 ts_tablet_manager.cc:1403] T 42ebb32ae777484fab5e26a137cbd4f6 P 15c6b88567e44da395f245725ffe7f04: Time spent bootstrapping tablet: real 0.056s	user 0.009s	sys 0.001s
I20260709 13:12:38.447794   807 raft_consensus.cc:359] T 42ebb32ae777484fab5e26a137cbd4f6 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.448302   807 raft_consensus.cc:385] T 42ebb32ae777484fab5e26a137cbd4f6 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:38.448678   807 raft_consensus.cc:740] T 42ebb32ae777484fab5e26a137cbd4f6 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15c6b88567e44da395f245725ffe7f04, State: Initialized, Role: FOLLOWER
I20260709 13:12:38.449494   807 consensus_queue.cc:260] T 42ebb32ae777484fab5e26a137cbd4f6 P 15c6b88567e44da395f245725ffe7f04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.451668   807 ts_tablet_manager.cc:1434] T 42ebb32ae777484fab5e26a137cbd4f6 P 15c6b88567e44da395f245725ffe7f04: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 13:12:38.457304   809 tablet_bootstrap.cc:492] T 42ebb32ae777484fab5e26a137cbd4f6 P 7d13b2cad49e4853831a427132e294d3: No bootstrap required, opened a new log
I20260709 13:12:38.457898   809 ts_tablet_manager.cc:1403] T 42ebb32ae777484fab5e26a137cbd4f6 P 7d13b2cad49e4853831a427132e294d3: Time spent bootstrapping tablet: real 0.022s	user 0.010s	sys 0.002s
I20260709 13:12:38.460636   809 raft_consensus.cc:359] T 42ebb32ae777484fab5e26a137cbd4f6 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.461403   809 raft_consensus.cc:385] T 42ebb32ae777484fab5e26a137cbd4f6 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:38.461743   809 raft_consensus.cc:740] T 42ebb32ae777484fab5e26a137cbd4f6 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d13b2cad49e4853831a427132e294d3, State: Initialized, Role: FOLLOWER
I20260709 13:12:38.462486   809 consensus_queue.cc:260] T 42ebb32ae777484fab5e26a137cbd4f6 P 7d13b2cad49e4853831a427132e294d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.464519   809 ts_tablet_manager.cc:1434] T 42ebb32ae777484fab5e26a137cbd4f6 P 7d13b2cad49e4853831a427132e294d3: Time spent starting tablet: real 0.006s	user 0.000s	sys 0.003s
I20260709 13:12:38.466771   735 raft_consensus.cc:493] T d7f398f030d840cab518d8996f2bdbce P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:38.467290   735 raft_consensus.cc:515] T d7f398f030d840cab518d8996f2bdbce P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.469381   735 leader_election.cc:290] T d7f398f030d840cab518d8996f2bdbce P 7d13b2cad49e4853831a427132e294d3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881), 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181)
I20260709 13:12:38.470541   508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7f398f030d840cab518d8996f2bdbce" candidate_uuid: "7d13b2cad49e4853831a427132e294d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f35a78aeac02496d88fa800153754b0a" is_pre_election: true
I20260709 13:12:38.470882   582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7f398f030d840cab518d8996f2bdbce" candidate_uuid: "7d13b2cad49e4853831a427132e294d3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15c6b88567e44da395f245725ffe7f04" is_pre_election: true
I20260709 13:12:38.471778   582 raft_consensus.cc:2393] T d7f398f030d840cab518d8996f2bdbce P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 7d13b2cad49e4853831a427132e294d3 in current term 1: Already voted for candidate f35a78aeac02496d88fa800153754b0a in this term.
I20260709 13:12:38.472842   621 leader_election.cc:304] T d7f398f030d840cab518d8996f2bdbce P 7d13b2cad49e4853831a427132e294d3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7d13b2cad49e4853831a427132e294d3; no voters: 15c6b88567e44da395f245725ffe7f04, f35a78aeac02496d88fa800153754b0a
I20260709 13:12:38.473680   735 raft_consensus.cc:3060] T d7f398f030d840cab518d8996f2bdbce P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:38.481909   735 raft_consensus.cc:2749] T d7f398f030d840cab518d8996f2bdbce P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:38.545017   829 raft_consensus.cc:493] T 42ebb32ae777484fab5e26a137cbd4f6 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:38.545684   829 raft_consensus.cc:515] T 42ebb32ae777484fab5e26a137cbd4f6 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.548000   829 leader_election.cc:290] T 42ebb32ae777484fab5e26a137cbd4f6 P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881), 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695)
I20260709 13:12:38.548799   508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42ebb32ae777484fab5e26a137cbd4f6" candidate_uuid: "15c6b88567e44da395f245725ffe7f04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f35a78aeac02496d88fa800153754b0a" is_pre_election: true
I20260709 13:12:38.549189   658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42ebb32ae777484fab5e26a137cbd4f6" candidate_uuid: "15c6b88567e44da395f245725ffe7f04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d13b2cad49e4853831a427132e294d3" is_pre_election: true
I20260709 13:12:38.549417   508 raft_consensus.cc:2468] T 42ebb32ae777484fab5e26a137cbd4f6 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15c6b88567e44da395f245725ffe7f04 in term 0.
I20260709 13:12:38.550002   658 raft_consensus.cc:2468] T 42ebb32ae777484fab5e26a137cbd4f6 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 15c6b88567e44da395f245725ffe7f04 in term 0.
I20260709 13:12:38.550534   550 leader_election.cc:304] T 42ebb32ae777484fab5e26a137cbd4f6 P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15c6b88567e44da395f245725ffe7f04, f35a78aeac02496d88fa800153754b0a; no voters: 
I20260709 13:12:38.551378   829 raft_consensus.cc:2804] T 42ebb32ae777484fab5e26a137cbd4f6 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:38.551776   829 raft_consensus.cc:493] T 42ebb32ae777484fab5e26a137cbd4f6 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:38.552125   829 raft_consensus.cc:3060] T 42ebb32ae777484fab5e26a137cbd4f6 P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:38.560076   829 raft_consensus.cc:515] T 42ebb32ae777484fab5e26a137cbd4f6 P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.562259   829 leader_election.cc:290] T 42ebb32ae777484fab5e26a137cbd4f6 P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 1 election: Requested vote from peers f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881), 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695)
I20260709 13:12:38.563126   508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42ebb32ae777484fab5e26a137cbd4f6" candidate_uuid: "15c6b88567e44da395f245725ffe7f04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f35a78aeac02496d88fa800153754b0a"
I20260709 13:12:38.563293   658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42ebb32ae777484fab5e26a137cbd4f6" candidate_uuid: "15c6b88567e44da395f245725ffe7f04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d13b2cad49e4853831a427132e294d3"
I20260709 13:12:38.563819   508 raft_consensus.cc:3060] T 42ebb32ae777484fab5e26a137cbd4f6 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:38.563894   658 raft_consensus.cc:3060] T 42ebb32ae777484fab5e26a137cbd4f6 P 7d13b2cad49e4853831a427132e294d3 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:38.569136   658 raft_consensus.cc:2468] T 42ebb32ae777484fab5e26a137cbd4f6 P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15c6b88567e44da395f245725ffe7f04 in term 1.
I20260709 13:12:38.570130   550 leader_election.cc:304] T 42ebb32ae777484fab5e26a137cbd4f6 P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15c6b88567e44da395f245725ffe7f04, 7d13b2cad49e4853831a427132e294d3; no voters: 
I20260709 13:12:38.570806   829 raft_consensus.cc:2804] T 42ebb32ae777484fab5e26a137cbd4f6 P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:38.571241   829 raft_consensus.cc:697] T 42ebb32ae777484fab5e26a137cbd4f6 P 15c6b88567e44da395f245725ffe7f04 [term 1 LEADER]: Becoming Leader. State: Replica: 15c6b88567e44da395f245725ffe7f04, State: Running, Role: LEADER
I20260709 13:12:38.571897   508 raft_consensus.cc:2468] T 42ebb32ae777484fab5e26a137cbd4f6 P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15c6b88567e44da395f245725ffe7f04 in term 1.
I20260709 13:12:38.572362   829 consensus_queue.cc:237] T 42ebb32ae777484fab5e26a137cbd4f6 P 15c6b88567e44da395f245725ffe7f04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.587663   406 catalog_manager.cc:5697] T 42ebb32ae777484fab5e26a137cbd4f6 P 15c6b88567e44da395f245725ffe7f04 reported cstate change: term changed from 0 to 1, leader changed from <none> to 15c6b88567e44da395f245725ffe7f04 (127.21.63.2). New cstate: current_term: 1 leader_uuid: "15c6b88567e44da395f245725ffe7f04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:38.659898   826 raft_consensus.cc:493] T f75ad0645c22439c9a0611d214158d01 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:38.660547   826 raft_consensus.cc:515] T f75ad0645c22439c9a0611d214158d01 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.662869   826 leader_election.cc:290] T f75ad0645c22439c9a0611d214158d01 P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181), 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695)
I20260709 13:12:38.667917   582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f75ad0645c22439c9a0611d214158d01" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "15c6b88567e44da395f245725ffe7f04" is_pre_election: true
I20260709 13:12:38.665014   658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f75ad0645c22439c9a0611d214158d01" candidate_uuid: "f35a78aeac02496d88fa800153754b0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d13b2cad49e4853831a427132e294d3" is_pre_election: true
I20260709 13:12:38.672394   658 raft_consensus.cc:2393] T f75ad0645c22439c9a0611d214158d01 P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f35a78aeac02496d88fa800153754b0a in current term 1: Already voted for candidate 15c6b88567e44da395f245725ffe7f04 in this term.
I20260709 13:12:38.673838   475 leader_election.cc:304] T f75ad0645c22439c9a0611d214158d01 P f35a78aeac02496d88fa800153754b0a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f35a78aeac02496d88fa800153754b0a; no voters: 15c6b88567e44da395f245725ffe7f04, 7d13b2cad49e4853831a427132e294d3
I20260709 13:12:38.674803   826 raft_consensus.cc:3060] T f75ad0645c22439c9a0611d214158d01 P f35a78aeac02496d88fa800153754b0a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:38.680665   826 raft_consensus.cc:2749] T f75ad0645c22439c9a0611d214158d01 P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:38.689565   829 raft_consensus.cc:493] T 78d4c60d0e4b42ef9e2bc081980888af P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:38.690114   829 raft_consensus.cc:515] T 78d4c60d0e4b42ef9e2bc081980888af P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.691808   829 leader_election.cc:290] T 78d4c60d0e4b42ef9e2bc081980888af P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881), 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695)
I20260709 13:12:38.692728   508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d4c60d0e4b42ef9e2bc081980888af" candidate_uuid: "15c6b88567e44da395f245725ffe7f04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f35a78aeac02496d88fa800153754b0a" is_pre_election: true
I20260709 13:12:38.693156   658 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d4c60d0e4b42ef9e2bc081980888af" candidate_uuid: "15c6b88567e44da395f245725ffe7f04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d13b2cad49e4853831a427132e294d3" is_pre_election: true
I20260709 13:12:38.693956   658 raft_consensus.cc:2393] T 78d4c60d0e4b42ef9e2bc081980888af P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 15c6b88567e44da395f245725ffe7f04 in current term 1: Already voted for candidate f35a78aeac02496d88fa800153754b0a in this term.
I20260709 13:12:38.694844   550 leader_election.cc:304] T 78d4c60d0e4b42ef9e2bc081980888af P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 15c6b88567e44da395f245725ffe7f04; no voters: 7d13b2cad49e4853831a427132e294d3, f35a78aeac02496d88fa800153754b0a
I20260709 13:12:38.695600   829 raft_consensus.cc:3060] T 78d4c60d0e4b42ef9e2bc081980888af P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:38.704249   826 consensus_queue.cc:1048] T b9b003cc8908492ab10df382680b5141 P f35a78aeac02496d88fa800153754b0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:38.704499   829 raft_consensus.cc:2749] T 78d4c60d0e4b42ef9e2bc081980888af P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:38.716195   826 consensus_queue.cc:1048] T bedab716985143c7a40f88c69d71ecd4 P f35a78aeac02496d88fa800153754b0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:38.720043   827 consensus_queue.cc:1048] T f75ad0645c22439c9a0611d214158d01 P 15c6b88567e44da395f245725ffe7f04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:38.721457   827 consensus_queue.cc:1048] T def0acc1a0684b9b86755e1c34191d91 P 15c6b88567e44da395f245725ffe7f04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:38.733564   827 raft_consensus.cc:493] T 16c6d1765fc145348e0a4f703fe6abdf P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:38.734166   806 consensus_queue.cc:1048] T 78d4c60d0e4b42ef9e2bc081980888af P f35a78aeac02496d88fa800153754b0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:38.734164   827 raft_consensus.cc:515] T 16c6d1765fc145348e0a4f703fe6abdf P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:38.737380   827 leader_election.cc:290] T 16c6d1765fc145348e0a4f703fe6abdf P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881), 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695)
I20260709 13:12:38.744900   507 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d7f398f030d840cab518d8996f2bdbce"
dest_uuid: "f35a78aeac02496d88fa800153754b0a"
mode: GRACEFUL
new_leader_uuid: "15c6b88567e44da395f245725ffe7f04"
 from {username='slave'} at 127.0.0.1:58040
I20260709 13:12:38.745483   507 raft_consensus.cc:606] T d7f398f030d840cab518d8996f2bdbce P f35a78aeac02496d88fa800153754b0a [term 1 LEADER]: Received request to transfer leadership to TS 15c6b88567e44da395f245725ffe7f04
I20260709 13:12:38.745847   826 consensus_queue.cc:1048] T b9b003cc8908492ab10df382680b5141 P f35a78aeac02496d88fa800153754b0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:38.754571   507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16c6d1765fc145348e0a4f703fe6abdf" candidate_uuid: "15c6b88567e44da395f245725ffe7f04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f35a78aeac02496d88fa800153754b0a" is_pre_election: true
I20260709 13:12:38.755594   507 raft_consensus.cc:2393] T 16c6d1765fc145348e0a4f703fe6abdf P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 15c6b88567e44da395f245725ffe7f04 in current term 1: Already voted for candidate 7d13b2cad49e4853831a427132e294d3 in this term.
I20260709 13:12:38.755736   735 consensus_queue.cc:1048] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:38.757346   658 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "16c6d1765fc145348e0a4f703fe6abdf"
dest_uuid: "7d13b2cad49e4853831a427132e294d3"
mode: GRACEFUL
new_leader_uuid: "15c6b88567e44da395f245725ffe7f04"
 from {username='slave'} at 127.0.0.1:33240
I20260709 13:12:38.758981   655 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16c6d1765fc145348e0a4f703fe6abdf" candidate_uuid: "15c6b88567e44da395f245725ffe7f04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d13b2cad49e4853831a427132e294d3" is_pre_election: true
I20260709 13:12:38.764742   550 leader_election.cc:304] T 16c6d1765fc145348e0a4f703fe6abdf P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 15c6b88567e44da395f245725ffe7f04; no voters: 7d13b2cad49e4853831a427132e294d3, f35a78aeac02496d88fa800153754b0a
I20260709 13:12:38.767027   829 raft_consensus.cc:3060] T 16c6d1765fc145348e0a4f703fe6abdf P 15c6b88567e44da395f245725ffe7f04 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:38.770148   836 consensus_queue.cc:1048] T 9082b9270cf74ed5b72b7bc4b0166836 P f35a78aeac02496d88fa800153754b0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:38.778232   827 consensus_queue.cc:1048] T def0acc1a0684b9b86755e1c34191d91 P 15c6b88567e44da395f245725ffe7f04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:38.757961   658 raft_consensus.cc:606] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 [term 1 LEADER]: Received request to transfer leadership to TS 15c6b88567e44da395f245725ffe7f04
I20260709 13:12:38.850603   806 consensus_queue.cc:1048] T 9082b9270cf74ed5b72b7bc4b0166836 P f35a78aeac02496d88fa800153754b0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:38.851531   829 raft_consensus.cc:2749] T 16c6d1765fc145348e0a4f703fe6abdf P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:38.855500   735 consensus_queue.cc:1048] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:38.858417   806 consensus_queue.cc:1048] T bedab716985143c7a40f88c69d71ecd4 P f35a78aeac02496d88fa800153754b0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:38.862627   836 consensus_queue.cc:1048] T 78d4c60d0e4b42ef9e2bc081980888af P f35a78aeac02496d88fa800153754b0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 13:12:38.877907   550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:34881, user_credentials={real_user=slave}} blocked reactor thread for 49978.1us
W20260709 13:12:38.961715   550 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:34881, user_credentials={real_user=slave}} blocked reactor thread for 60982us
I20260709 13:12:38.963619   827 consensus_queue.cc:1048] T f75ad0645c22439c9a0611d214158d01 P 15c6b88567e44da395f245725ffe7f04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:38.996964   582 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "42ebb32ae777484fab5e26a137cbd4f6"
dest_uuid: "15c6b88567e44da395f245725ffe7f04"
mode: GRACEFUL
new_leader_uuid: "7d13b2cad49e4853831a427132e294d3"
 from {username='slave'} at 127.0.0.1:60484
I20260709 13:12:38.997634   582 raft_consensus.cc:606] T 42ebb32ae777484fab5e26a137cbd4f6 P 15c6b88567e44da395f245725ffe7f04 [term 1 LEADER]: Received request to transfer leadership to TS 7d13b2cad49e4853831a427132e294d3
I20260709 13:12:39.008817   785 consensus_queue.cc:1048] T d7f398f030d840cab518d8996f2bdbce P f35a78aeac02496d88fa800153754b0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:39.008859   735 raft_consensus.cc:993] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3: : Instructing follower 15c6b88567e44da395f245725ffe7f04 to start an election
I20260709 13:12:39.012871   735 raft_consensus.cc:1081] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 [term 1 LEADER]: Signalling peer 15c6b88567e44da395f245725ffe7f04 to start an election
I20260709 13:12:39.025795   579 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "16c6d1765fc145348e0a4f703fe6abdf"
dest_uuid: "15c6b88567e44da395f245725ffe7f04"
 from {username='slave'} at 127.0.0.1:60472
I20260709 13:12:39.026428   579 raft_consensus.cc:493] T 16c6d1765fc145348e0a4f703fe6abdf P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:39.026820   579 raft_consensus.cc:3060] T 16c6d1765fc145348e0a4f703fe6abdf P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:39.039088   579 raft_consensus.cc:515] T 16c6d1765fc145348e0a4f703fe6abdf P 15c6b88567e44da395f245725ffe7f04 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:39.040620   785 consensus_queue.cc:1048] T d7f398f030d840cab518d8996f2bdbce P f35a78aeac02496d88fa800153754b0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:39.046806   507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16c6d1765fc145348e0a4f703fe6abdf" candidate_uuid: "15c6b88567e44da395f245725ffe7f04" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f35a78aeac02496d88fa800153754b0a"
I20260709 13:12:39.047453   507 raft_consensus.cc:3060] T 16c6d1765fc145348e0a4f703fe6abdf P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:39.055749   507 raft_consensus.cc:2468] T 16c6d1765fc145348e0a4f703fe6abdf P f35a78aeac02496d88fa800153754b0a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15c6b88567e44da395f245725ffe7f04 in term 2.
I20260709 13:12:39.066691   550 leader_election.cc:304] T 16c6d1765fc145348e0a4f703fe6abdf P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15c6b88567e44da395f245725ffe7f04, f35a78aeac02496d88fa800153754b0a; no voters: 
I20260709 13:12:39.067772   882 raft_consensus.cc:2804] T 16c6d1765fc145348e0a4f703fe6abdf P 15c6b88567e44da395f245725ffe7f04 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:39.068292   882 raft_consensus.cc:697] T 16c6d1765fc145348e0a4f703fe6abdf P 15c6b88567e44da395f245725ffe7f04 [term 2 LEADER]: Becoming Leader. State: Replica: 15c6b88567e44da395f245725ffe7f04, State: Running, Role: LEADER
I20260709 13:12:39.069231   882 consensus_queue.cc:237] T 16c6d1765fc145348e0a4f703fe6abdf P 15c6b88567e44da395f245725ffe7f04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:39.071113   579 leader_election.cc:290] T 16c6d1765fc145348e0a4f703fe6abdf P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 2 election: Requested vote from peers f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881), 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695)
I20260709 13:12:39.072093   656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16c6d1765fc145348e0a4f703fe6abdf" candidate_uuid: "15c6b88567e44da395f245725ffe7f04" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7d13b2cad49e4853831a427132e294d3"
I20260709 13:12:39.080106   656 raft_consensus.cc:3055] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:39.080466   656 raft_consensus.cc:740] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 7d13b2cad49e4853831a427132e294d3, State: Running, Role: LEADER
I20260709 13:12:39.081305   656 consensus_queue.cc:260] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:39.082523   656 raft_consensus.cc:3060] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:39.084213   806 raft_consensus.cc:993] T d7f398f030d840cab518d8996f2bdbce P f35a78aeac02496d88fa800153754b0a: : Instructing follower 15c6b88567e44da395f245725ffe7f04 to start an election
I20260709 13:12:39.084836   836 raft_consensus.cc:1081] T d7f398f030d840cab518d8996f2bdbce P f35a78aeac02496d88fa800153754b0a [term 1 LEADER]: Signalling peer 15c6b88567e44da395f245725ffe7f04 to start an election
I20260709 13:12:39.086541   582 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d7f398f030d840cab518d8996f2bdbce"
dest_uuid: "15c6b88567e44da395f245725ffe7f04"
 from {username='slave'} at 127.0.0.1:60470
I20260709 13:12:39.087101   582 raft_consensus.cc:493] T d7f398f030d840cab518d8996f2bdbce P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:39.087463   582 raft_consensus.cc:3060] T d7f398f030d840cab518d8996f2bdbce P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:39.087867   882 consensus_queue.cc:1048] T 42ebb32ae777484fab5e26a137cbd4f6 P 15c6b88567e44da395f245725ffe7f04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:39.097381   656 raft_consensus.cc:2468] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15c6b88567e44da395f245725ffe7f04 in term 2.
I20260709 13:12:39.099897   582 raft_consensus.cc:515] T d7f398f030d840cab518d8996f2bdbce P 15c6b88567e44da395f245725ffe7f04 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:39.102933   508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7f398f030d840cab518d8996f2bdbce" candidate_uuid: "15c6b88567e44da395f245725ffe7f04" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f35a78aeac02496d88fa800153754b0a"
I20260709 13:12:39.103571   508 raft_consensus.cc:3055] T d7f398f030d840cab518d8996f2bdbce P f35a78aeac02496d88fa800153754b0a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:39.103920   508 raft_consensus.cc:740] T d7f398f030d840cab518d8996f2bdbce P f35a78aeac02496d88fa800153754b0a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f35a78aeac02496d88fa800153754b0a, State: Running, Role: LEADER
I20260709 13:12:39.103996   582 leader_election.cc:290] T d7f398f030d840cab518d8996f2bdbce P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 2 election: Requested vote from peers f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881), 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695)
I20260709 13:12:39.104233   656 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7f398f030d840cab518d8996f2bdbce" candidate_uuid: "15c6b88567e44da395f245725ffe7f04" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "7d13b2cad49e4853831a427132e294d3"
I20260709 13:12:39.104920   656 raft_consensus.cc:3060] T d7f398f030d840cab518d8996f2bdbce P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:39.104800   508 consensus_queue.cc:260] T d7f398f030d840cab518d8996f2bdbce P f35a78aeac02496d88fa800153754b0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:39.106201   508 raft_consensus.cc:3060] T d7f398f030d840cab518d8996f2bdbce P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:39.112224   656 raft_consensus.cc:2468] T d7f398f030d840cab518d8996f2bdbce P 7d13b2cad49e4853831a427132e294d3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15c6b88567e44da395f245725ffe7f04 in term 2.
I20260709 13:12:39.117633   550 leader_election.cc:304] T d7f398f030d840cab518d8996f2bdbce P 15c6b88567e44da395f245725ffe7f04 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15c6b88567e44da395f245725ffe7f04, 7d13b2cad49e4853831a427132e294d3; no voters: 
I20260709 13:12:39.119447   871 raft_consensus.cc:2804] T d7f398f030d840cab518d8996f2bdbce P 15c6b88567e44da395f245725ffe7f04 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:39.119944   871 raft_consensus.cc:697] T d7f398f030d840cab518d8996f2bdbce P 15c6b88567e44da395f245725ffe7f04 [term 2 LEADER]: Becoming Leader. State: Replica: 15c6b88567e44da395f245725ffe7f04, State: Running, Role: LEADER
I20260709 13:12:39.123593   871 consensus_queue.cc:237] T d7f398f030d840cab518d8996f2bdbce P 15c6b88567e44da395f245725ffe7f04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:39.125651   868 consensus_queue.cc:1048] T 42ebb32ae777484fab5e26a137cbd4f6 P 15c6b88567e44da395f245725ffe7f04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:39.135224   405 catalog_manager.cc:5697] T d7f398f030d840cab518d8996f2bdbce P 15c6b88567e44da395f245725ffe7f04 reported cstate change: term changed from 1 to 2, leader changed from f35a78aeac02496d88fa800153754b0a (127.21.63.1) to 15c6b88567e44da395f245725ffe7f04 (127.21.63.2). New cstate: current_term: 2 leader_uuid: "15c6b88567e44da395f245725ffe7f04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:39.136847   405 catalog_manager.cc:5697] T 16c6d1765fc145348e0a4f703fe6abdf P 15c6b88567e44da395f245725ffe7f04 reported cstate change: term changed from 1 to 2, leader changed from 7d13b2cad49e4853831a427132e294d3 (127.21.63.3) to 15c6b88567e44da395f245725ffe7f04 (127.21.63.2). New cstate: current_term: 2 leader_uuid: "15c6b88567e44da395f245725ffe7f04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:39.123452   508 raft_consensus.cc:2468] T d7f398f030d840cab518d8996f2bdbce P f35a78aeac02496d88fa800153754b0a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 15c6b88567e44da395f245725ffe7f04 in term 2.
I20260709 13:12:39.153918   871 raft_consensus.cc:993] T 42ebb32ae777484fab5e26a137cbd4f6 P 15c6b88567e44da395f245725ffe7f04: : Instructing follower 7d13b2cad49e4853831a427132e294d3 to start an election
I20260709 13:12:39.154366   871 raft_consensus.cc:1081] T 42ebb32ae777484fab5e26a137cbd4f6 P 15c6b88567e44da395f245725ffe7f04 [term 1 LEADER]: Signalling peer 7d13b2cad49e4853831a427132e294d3 to start an election
I20260709 13:12:39.156092   656 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "42ebb32ae777484fab5e26a137cbd4f6"
dest_uuid: "7d13b2cad49e4853831a427132e294d3"
 from {username='slave'} at 127.0.0.1:33216
I20260709 13:12:39.156872   656 raft_consensus.cc:493] T 42ebb32ae777484fab5e26a137cbd4f6 P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:39.157325   656 raft_consensus.cc:3060] T 42ebb32ae777484fab5e26a137cbd4f6 P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:39.164419   656 raft_consensus.cc:515] T 42ebb32ae777484fab5e26a137cbd4f6 P 7d13b2cad49e4853831a427132e294d3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:39.167132   508 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42ebb32ae777484fab5e26a137cbd4f6" candidate_uuid: "7d13b2cad49e4853831a427132e294d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f35a78aeac02496d88fa800153754b0a"
I20260709 13:12:39.167780   508 raft_consensus.cc:3060] T 42ebb32ae777484fab5e26a137cbd4f6 P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:39.167966   582 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "42ebb32ae777484fab5e26a137cbd4f6" candidate_uuid: "7d13b2cad49e4853831a427132e294d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "15c6b88567e44da395f245725ffe7f04"
I20260709 13:12:39.168599   582 raft_consensus.cc:3055] T 42ebb32ae777484fab5e26a137cbd4f6 P 15c6b88567e44da395f245725ffe7f04 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:39.168979   582 raft_consensus.cc:740] T 42ebb32ae777484fab5e26a137cbd4f6 P 15c6b88567e44da395f245725ffe7f04 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 15c6b88567e44da395f245725ffe7f04, State: Running, Role: LEADER
I20260709 13:12:39.169523   656 leader_election.cc:290] T 42ebb32ae777484fab5e26a137cbd4f6 P 7d13b2cad49e4853831a427132e294d3 [CANDIDATE]: Term 2 election: Requested vote from peers f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881), 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181)
I20260709 13:12:39.169813   582 consensus_queue.cc:260] T 42ebb32ae777484fab5e26a137cbd4f6 P 15c6b88567e44da395f245725ffe7f04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:39.171033   582 raft_consensus.cc:3060] T 42ebb32ae777484fab5e26a137cbd4f6 P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:39.176859   508 raft_consensus.cc:2468] T 42ebb32ae777484fab5e26a137cbd4f6 P f35a78aeac02496d88fa800153754b0a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d13b2cad49e4853831a427132e294d3 in term 2.
I20260709 13:12:39.180204   582 raft_consensus.cc:2468] T 42ebb32ae777484fab5e26a137cbd4f6 P 15c6b88567e44da395f245725ffe7f04 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d13b2cad49e4853831a427132e294d3 in term 2.
I20260709 13:12:39.180430   624 leader_election.cc:304] T 42ebb32ae777484fab5e26a137cbd4f6 P 7d13b2cad49e4853831a427132e294d3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d13b2cad49e4853831a427132e294d3, f35a78aeac02496d88fa800153754b0a; no voters: 
I20260709 13:12:39.181603   878 raft_consensus.cc:2804] T 42ebb32ae777484fab5e26a137cbd4f6 P 7d13b2cad49e4853831a427132e294d3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:39.182104   878 raft_consensus.cc:697] T 42ebb32ae777484fab5e26a137cbd4f6 P 7d13b2cad49e4853831a427132e294d3 [term 2 LEADER]: Becoming Leader. State: Replica: 7d13b2cad49e4853831a427132e294d3, State: Running, Role: LEADER
I20260709 13:12:39.182938   878 consensus_queue.cc:237] T 42ebb32ae777484fab5e26a137cbd4f6 P 7d13b2cad49e4853831a427132e294d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:39.192025   405 catalog_manager.cc:5697] T 42ebb32ae777484fab5e26a137cbd4f6 P 7d13b2cad49e4853831a427132e294d3 reported cstate change: term changed from 1 to 2, leader changed from 15c6b88567e44da395f245725ffe7f04 (127.21.63.2) to 7d13b2cad49e4853831a427132e294d3 (127.21.63.3). New cstate: current_term: 2 leader_uuid: "7d13b2cad49e4853831a427132e294d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:39.456077   656 raft_consensus.cc:1275] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 [term 2 FOLLOWER]: Refusing update from remote peer 15c6b88567e44da395f245725ffe7f04: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:39.457559   868 consensus_queue.cc:1048] T 16c6d1765fc145348e0a4f703fe6abdf P 15c6b88567e44da395f245725ffe7f04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:39.470638   508 raft_consensus.cc:1275] T 16c6d1765fc145348e0a4f703fe6abdf P f35a78aeac02496d88fa800153754b0a [term 2 FOLLOWER]: Refusing update from remote peer 15c6b88567e44da395f245725ffe7f04: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:39.472014   870 consensus_queue.cc:1048] T 16c6d1765fc145348e0a4f703fe6abdf P 15c6b88567e44da395f245725ffe7f04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:39.629987   508 raft_consensus.cc:1275] T d7f398f030d840cab518d8996f2bdbce P f35a78aeac02496d88fa800153754b0a [term 2 FOLLOWER]: Refusing update from remote peer 15c6b88567e44da395f245725ffe7f04: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:39.631246   871 consensus_queue.cc:1048] T d7f398f030d840cab518d8996f2bdbce P 15c6b88567e44da395f245725ffe7f04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:39.639365   656 raft_consensus.cc:1275] T d7f398f030d840cab518d8996f2bdbce P 7d13b2cad49e4853831a427132e294d3 [term 2 FOLLOWER]: Refusing update from remote peer 15c6b88567e44da395f245725ffe7f04: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:39.639367   508 raft_consensus.cc:1275] T 42ebb32ae777484fab5e26a137cbd4f6 P f35a78aeac02496d88fa800153754b0a [term 2 FOLLOWER]: Refusing update from remote peer 7d13b2cad49e4853831a427132e294d3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:39.643815   871 consensus_queue.cc:1048] T d7f398f030d840cab518d8996f2bdbce P 15c6b88567e44da395f245725ffe7f04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:39.644716   878 consensus_queue.cc:1048] T 42ebb32ae777484fab5e26a137cbd4f6 P 7d13b2cad49e4853831a427132e294d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:39.666723   582 raft_consensus.cc:1275] T 42ebb32ae777484fab5e26a137cbd4f6 P 15c6b88567e44da395f245725ffe7f04 [term 2 FOLLOWER]: Refusing update from remote peer 7d13b2cad49e4853831a427132e294d3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:39.668299   735 consensus_queue.cc:1048] T 42ebb32ae777484fab5e26a137cbd4f6 P 7d13b2cad49e4853831a427132e294d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:42.002930   405 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40368:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:12:42.027491   405 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:40368:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:12:42.062635 21756 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:12:42.184860   584 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "16c6d1765fc145348e0a4f703fe6abdf"
dest_uuid: "15c6b88567e44da395f245725ffe7f04"
mode: GRACEFUL
new_leader_uuid: "7d13b2cad49e4853831a427132e294d3"
 from {username='slave'} at 127.0.0.1:42786
I20260709 13:12:42.188413   584 raft_consensus.cc:606] T 16c6d1765fc145348e0a4f703fe6abdf P 15c6b88567e44da395f245725ffe7f04 [term 2 LEADER]: Received request to transfer leadership to TS 7d13b2cad49e4853831a427132e294d3
W20260709 13:12:42.275416   932 log.cc:927] Time spent T 78d4c60d0e4b42ef9e2bc081980888af P 7d13b2cad49e4853831a427132e294d3: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 13:12:42.283547   943 tablet.cc:1724] T 9082b9270cf74ed5b72b7bc4b0166836 P f35a78aeac02496d88fa800153754b0a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:12:42.349797   535 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:9082b9270cf74ed5b72b7bc4b0166836)
I20260709 13:12:42.353271   506 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "78d4c60d0e4b42ef9e2bc081980888af"
dest_uuid: "f35a78aeac02496d88fa800153754b0a"
mode: GRACEFUL
new_leader_uuid: "7d13b2cad49e4853831a427132e294d3"
 from {username='slave'} at 127.0.0.1:45172
W20260709 13:12:42.360642   535 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:9082b9270cf74ed5b72b7bc4b0166836)
W20260709 13:12:42.368742   535 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:9082b9270cf74ed5b72b7bc4b0166836)
W20260709 13:12:42.370869   938 log.cc:927] Time spent T bedab716985143c7a40f88c69d71ecd4 P 15c6b88567e44da395f245725ffe7f04: Append to log took a long time: real 0.101s	user 0.001s	sys 0.001s
W20260709 13:12:42.371091   944 log.cc:927] Time spent T b9b003cc8908492ab10df382680b5141 P 15c6b88567e44da395f245725ffe7f04: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 13:12:42.441740   959 tablet.cc:1724] T f75ad0645c22439c9a0611d214158d01 P 15c6b88567e44da395f245725ffe7f04: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:42.447639   953 tablet.cc:1724] T 78d4c60d0e4b42ef9e2bc081980888af P f35a78aeac02496d88fa800153754b0a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:42.463495   506 raft_consensus.cc:606] T 78d4c60d0e4b42ef9e2bc081980888af P f35a78aeac02496d88fa800153754b0a [term 1 LEADER]: Received request to transfer leadership to TS 7d13b2cad49e4853831a427132e294d3
I20260709 13:12:42.320894   948 tablet.cc:1724] T 9082b9270cf74ed5b72b7bc4b0166836 P 15c6b88567e44da395f245725ffe7f04: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:12:42.483698   472 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:38181, user_credentials={real_user=slave}} blocked reactor thread for 101838us
I20260709 13:12:42.493044   936 raft_consensus.cc:993] T 16c6d1765fc145348e0a4f703fe6abdf P 15c6b88567e44da395f245725ffe7f04: : Instructing follower 7d13b2cad49e4853831a427132e294d3 to start an election
W20260709 13:12:42.493340   475 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.3:42695, user_credentials={real_user=slave}} blocked reactor thread for 108302us
I20260709 13:12:42.493559   936 raft_consensus.cc:1081] T 16c6d1765fc145348e0a4f703fe6abdf P 15c6b88567e44da395f245725ffe7f04 [term 2 LEADER]: Signalling peer 7d13b2cad49e4853831a427132e294d3 to start an election
I20260709 13:12:42.496752   653 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "16c6d1765fc145348e0a4f703fe6abdf"
dest_uuid: "7d13b2cad49e4853831a427132e294d3"
 from {username='slave'} at 127.0.0.1:33216
I20260709 13:12:42.497459   653 raft_consensus.cc:493] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:42.497951   653 raft_consensus.cc:3060] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:42.456498   963 tablet.cc:1724] T f75ad0645c22439c9a0611d214158d01 P 7d13b2cad49e4853831a427132e294d3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:42.519783 21756 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 13:12:42.524216 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:42.529129   964 tablet.cc:1724] T 42ebb32ae777484fab5e26a137cbd4f6 P 7d13b2cad49e4853831a427132e294d3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:42.531065   966 tablet.cc:1724] T 42ebb32ae777484fab5e26a137cbd4f6 P 15c6b88567e44da395f245725ffe7f04: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:42.546988   951 raft_consensus.cc:993] T 78d4c60d0e4b42ef9e2bc081980888af P f35a78aeac02496d88fa800153754b0a: : Instructing follower 7d13b2cad49e4853831a427132e294d3 to start an election
I20260709 13:12:42.547590   951 raft_consensus.cc:1081] T 78d4c60d0e4b42ef9e2bc081980888af P f35a78aeac02496d88fa800153754b0a [term 1 LEADER]: Signalling peer 7d13b2cad49e4853831a427132e294d3 to start an election
I20260709 13:12:42.546234   958 tablet.cc:1724] T bedab716985143c7a40f88c69d71ecd4 P f35a78aeac02496d88fa800153754b0a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:42.570158   580 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3723f5ab8f974d1892ce711b33fac1dd"
dest_uuid: "15c6b88567e44da395f245725ffe7f04"
mode: GRACEFUL
new_leader_uuid: "7d13b2cad49e4853831a427132e294d3"
 from {username='slave'} at 127.0.0.1:42786
I20260709 13:12:42.570870   580 raft_consensus.cc:606] T 3723f5ab8f974d1892ce711b33fac1dd P 15c6b88567e44da395f245725ffe7f04 [term 1 LEADER]: Received request to transfer leadership to TS 7d13b2cad49e4853831a427132e294d3
I20260709 13:12:42.574870   967 tablet.cc:1724] T def0acc1a0684b9b86755e1c34191d91 P 15c6b88567e44da395f245725ffe7f04: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:42.577346   654 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "78d4c60d0e4b42ef9e2bc081980888af"
dest_uuid: "7d13b2cad49e4853831a427132e294d3"
 from {username='slave'} at 127.0.0.1:33230
I20260709 13:12:42.577894   509 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a13e0fbeefb142f6bf766c1509cd1d93"
dest_uuid: "f35a78aeac02496d88fa800153754b0a"
mode: GRACEFUL
new_leader_uuid: "7d13b2cad49e4853831a427132e294d3"
 from {username='slave'} at 127.0.0.1:45172
I20260709 13:12:42.578534   509 raft_consensus.cc:606] T a13e0fbeefb142f6bf766c1509cd1d93 P f35a78aeac02496d88fa800153754b0a [term 2 LEADER]: Received request to transfer leadership to TS 7d13b2cad49e4853831a427132e294d3
I20260709 13:12:42.580161 21756 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:12:42.580907 21756 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:12:42.581230   968 tablet.cc:1724] T b9b003cc8908492ab10df382680b5141 P f35a78aeac02496d88fa800153754b0a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:42.585785   653 raft_consensus.cc:515] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:42.586971   652 raft_consensus.cc:1240] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 [term 3 FOLLOWER]: Rejecting Update request from peer 15c6b88567e44da395f245725ffe7f04 for earlier term 2. Current term is 3. Ops: []
I20260709 13:12:42.590934   942 consensus_queue.cc:1059] T 16c6d1765fc145348e0a4f703fe6abdf P 15c6b88567e44da395f245725ffe7f04 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 }, Status: INVALID_TERM, Last received: 2.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 13:12:42.591912   942 raft_consensus.cc:3055] T 16c6d1765fc145348e0a4f703fe6abdf P 15c6b88567e44da395f245725ffe7f04 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:42.592296   942 raft_consensus.cc:740] T 16c6d1765fc145348e0a4f703fe6abdf P 15c6b88567e44da395f245725ffe7f04 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 15c6b88567e44da395f245725ffe7f04, State: Running, Role: LEADER
I20260709 13:12:42.573643   943 tablet.cc:1724] T f75ad0645c22439c9a0611d214158d01 P f35a78aeac02496d88fa800153754b0a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:42.593456   942 consensus_queue.cc:260] T 16c6d1765fc145348e0a4f703fe6abdf P 15c6b88567e44da395f245725ffe7f04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:42.577982   654 raft_consensus.cc:493] T 78d4c60d0e4b42ef9e2bc081980888af P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:42.594906   942 raft_consensus.cc:3060] T 16c6d1765fc145348e0a4f703fe6abdf P 15c6b88567e44da395f245725ffe7f04 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:42.594973   654 raft_consensus.cc:3060] T 78d4c60d0e4b42ef9e2bc081980888af P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:42.597214   972 tablet.cc:1724] T 9082b9270cf74ed5b72b7bc4b0166836 P 7d13b2cad49e4853831a427132e294d3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:42.602008   973 tablet.cc:1724] T bedab716985143c7a40f88c69d71ecd4 P 7d13b2cad49e4853831a427132e294d3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:42.593420   963 tablet.cc:1724] T def0acc1a0684b9b86755e1c34191d91 P 7d13b2cad49e4853831a427132e294d3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:42.613107   948 tablet.cc:1724] T bedab716985143c7a40f88c69d71ecd4 P 15c6b88567e44da395f245725ffe7f04: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:42.629333   653 leader_election.cc:290] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 [CANDIDATE]: Term 3 election: Requested vote from peers f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881), 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181)
I20260709 13:12:42.630038   580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16c6d1765fc145348e0a4f703fe6abdf" candidate_uuid: "7d13b2cad49e4853831a427132e294d3" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "15c6b88567e44da395f245725ffe7f04"
I20260709 13:12:42.631172   505 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "16c6d1765fc145348e0a4f703fe6abdf" candidate_uuid: "7d13b2cad49e4853831a427132e294d3" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "f35a78aeac02496d88fa800153754b0a"
I20260709 13:12:42.631908   505 raft_consensus.cc:3060] T 16c6d1765fc145348e0a4f703fe6abdf P f35a78aeac02496d88fa800153754b0a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:42.639200   654 raft_consensus.cc:515] T 78d4c60d0e4b42ef9e2bc081980888af P 7d13b2cad49e4853831a427132e294d3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:42.652160   583 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d4c60d0e4b42ef9e2bc081980888af" candidate_uuid: "7d13b2cad49e4853831a427132e294d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "15c6b88567e44da395f245725ffe7f04"
I20260709 13:12:42.652493   507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "78d4c60d0e4b42ef9e2bc081980888af" candidate_uuid: "7d13b2cad49e4853831a427132e294d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "f35a78aeac02496d88fa800153754b0a"
I20260709 13:12:42.652910   583 raft_consensus.cc:3060] T 78d4c60d0e4b42ef9e2bc081980888af P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:42.651018   654 leader_election.cc:290] T 78d4c60d0e4b42ef9e2bc081980888af P 7d13b2cad49e4853831a427132e294d3 [CANDIDATE]: Term 2 election: Requested vote from peers f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881), 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181)
I20260709 13:12:42.653102   507 raft_consensus.cc:3055] T 78d4c60d0e4b42ef9e2bc081980888af P f35a78aeac02496d88fa800153754b0a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:42.650369   964 tablet.cc:1724] T 78d4c60d0e4b42ef9e2bc081980888af P 7d13b2cad49e4853831a427132e294d3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:42.661036   507 raft_consensus.cc:740] T 78d4c60d0e4b42ef9e2bc081980888af P f35a78aeac02496d88fa800153754b0a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f35a78aeac02496d88fa800153754b0a, State: Running, Role: LEADER
I20260709 13:12:42.659783   953 tablet.cc:1724] T def0acc1a0684b9b86755e1c34191d91 P f35a78aeac02496d88fa800153754b0a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:42.664372   507 consensus_queue.cc:260] T 78d4c60d0e4b42ef9e2bc081980888af P f35a78aeac02496d88fa800153754b0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:42.665910   507 raft_consensus.cc:3060] T 78d4c60d0e4b42ef9e2bc081980888af P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:42.633977   959 tablet.cc:1724] T 16c6d1765fc145348e0a4f703fe6abdf P 15c6b88567e44da395f245725ffe7f04: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:42.654232   943 tablet.cc:1724] T 42ebb32ae777484fab5e26a137cbd4f6 P f35a78aeac02496d88fa800153754b0a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:42.709048   967 tablet.cc:1724] T b9b003cc8908492ab10df382680b5141 P 15c6b88567e44da395f245725ffe7f04: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:42.711341   958 tablet.cc:1724] T 16c6d1765fc145348e0a4f703fe6abdf P f35a78aeac02496d88fa800153754b0a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:42.712311   583 raft_consensus.cc:2468] T 78d4c60d0e4b42ef9e2bc081980888af P 15c6b88567e44da395f245725ffe7f04 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d13b2cad49e4853831a427132e294d3 in term 2.
I20260709 13:12:42.718286   621 leader_election.cc:304] T 78d4c60d0e4b42ef9e2bc081980888af P 7d13b2cad49e4853831a427132e294d3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 15c6b88567e44da395f245725ffe7f04, 7d13b2cad49e4853831a427132e294d3; no voters: 
I20260709 13:12:42.721601   950 raft_consensus.cc:2804] T 78d4c60d0e4b42ef9e2bc081980888af P 7d13b2cad49e4853831a427132e294d3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:42.722142   950 raft_consensus.cc:697] T 78d4c60d0e4b42ef9e2bc081980888af P 7d13b2cad49e4853831a427132e294d3 [term 2 LEADER]: Becoming Leader. State: Replica: 7d13b2cad49e4853831a427132e294d3, State: Running, Role: LEADER
I20260709 13:12:42.723153   950 consensus_queue.cc:237] T 78d4c60d0e4b42ef9e2bc081980888af P 7d13b2cad49e4853831a427132e294d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:42.729526   948 tablet.cc:1724] T 78d4c60d0e4b42ef9e2bc081980888af P 15c6b88567e44da395f245725ffe7f04: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:42.737533   973 tablet.cc:1724] T b9b003cc8908492ab10df382680b5141 P 7d13b2cad49e4853831a427132e294d3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:42.737789   968 tablet.cc:1724] T d7f398f030d840cab518d8996f2bdbce P f35a78aeac02496d88fa800153754b0a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:42.753968   505 raft_consensus.cc:2468] T 16c6d1765fc145348e0a4f703fe6abdf P f35a78aeac02496d88fa800153754b0a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d13b2cad49e4853831a427132e294d3 in term 3.
I20260709 13:12:42.755391   624 leader_election.cc:304] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d13b2cad49e4853831a427132e294d3, f35a78aeac02496d88fa800153754b0a; no voters: 
I20260709 13:12:42.756311   580 raft_consensus.cc:2468] T 16c6d1765fc145348e0a4f703fe6abdf P 15c6b88567e44da395f245725ffe7f04 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d13b2cad49e4853831a427132e294d3 in term 3.
I20260709 13:12:42.756469   950 raft_consensus.cc:2804] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:42.756008   963 tablet.cc:1724] T d7f398f030d840cab518d8996f2bdbce P 7d13b2cad49e4853831a427132e294d3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:42.759447   950 raft_consensus.cc:697] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 [term 3 LEADER]: Becoming Leader. State: Replica: 7d13b2cad49e4853831a427132e294d3, State: Running, Role: LEADER
I20260709 13:12:42.760550   950 consensus_queue.cc:237] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:42.743681   967 tablet.cc:1724] T d7f398f030d840cab518d8996f2bdbce P 15c6b88567e44da395f245725ffe7f04: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:42.780444   942 raft_consensus.cc:993] T 3723f5ab8f974d1892ce711b33fac1dd P 15c6b88567e44da395f245725ffe7f04: : Instructing follower 7d13b2cad49e4853831a427132e294d3 to start an election
I20260709 13:12:42.781071   942 raft_consensus.cc:1081] T 3723f5ab8f974d1892ce711b33fac1dd P 15c6b88567e44da395f245725ffe7f04 [term 1 LEADER]: Signalling peer 7d13b2cad49e4853831a427132e294d3 to start an election
I20260709 13:12:42.783005   507 raft_consensus.cc:2468] T 78d4c60d0e4b42ef9e2bc081980888af P f35a78aeac02496d88fa800153754b0a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d13b2cad49e4853831a427132e294d3 in term 2.
I20260709 13:12:42.783366   654 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3723f5ab8f974d1892ce711b33fac1dd"
dest_uuid: "7d13b2cad49e4853831a427132e294d3"
 from {username='slave'} at 127.0.0.1:33216
I20260709 13:12:42.784884   654 raft_consensus.cc:493] T 3723f5ab8f974d1892ce711b33fac1dd P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:42.785228   654 raft_consensus.cc:3060] T 3723f5ab8f974d1892ce711b33fac1dd P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:42.792238   654 raft_consensus.cc:515] T 3723f5ab8f974d1892ce711b33fac1dd P 7d13b2cad49e4853831a427132e294d3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:42.800153   406 catalog_manager.cc:5697] T 78d4c60d0e4b42ef9e2bc081980888af P 7d13b2cad49e4853831a427132e294d3 reported cstate change: term changed from 1 to 2, leader changed from f35a78aeac02496d88fa800153754b0a (127.21.63.1) to 7d13b2cad49e4853831a427132e294d3 (127.21.63.3). New cstate: current_term: 2 leader_uuid: "7d13b2cad49e4853831a427132e294d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:42.810442   507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3723f5ab8f974d1892ce711b33fac1dd" candidate_uuid: "7d13b2cad49e4853831a427132e294d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f35a78aeac02496d88fa800153754b0a"
I20260709 13:12:42.809995   406 catalog_manager.cc:5697] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 reported cstate change: term changed from 2 to 3, leader changed from 15c6b88567e44da395f245725ffe7f04 (127.21.63.2) to 7d13b2cad49e4853831a427132e294d3 (127.21.63.3). New cstate: current_term: 3 leader_uuid: "7d13b2cad49e4853831a427132e294d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:42.811154   507 raft_consensus.cc:3060] T 3723f5ab8f974d1892ce711b33fac1dd P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:42.811595   406 catalog_manager.cc:5789] TS 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695) does not have the latest schema for tablet 16c6d1765fc145348e0a4f703fe6abdf (table soft_deleted_table [id=9258462a82ed4b239a497b7531d655f0]). Expected version 1 got 0
I20260709 13:12:42.820046   507 raft_consensus.cc:2468] T 3723f5ab8f974d1892ce711b33fac1dd P f35a78aeac02496d88fa800153754b0a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d13b2cad49e4853831a427132e294d3 in term 2.
I20260709 13:12:42.821818   654 leader_election.cc:290] T 3723f5ab8f974d1892ce711b33fac1dd P 7d13b2cad49e4853831a427132e294d3 [CANDIDATE]: Term 2 election: Requested vote from peers f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881), 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181)
I20260709 13:12:42.822224   624 leader_election.cc:304] T 3723f5ab8f974d1892ce711b33fac1dd P 7d13b2cad49e4853831a427132e294d3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d13b2cad49e4853831a427132e294d3, f35a78aeac02496d88fa800153754b0a; no voters: 
I20260709 13:12:42.823776   950 raft_consensus.cc:2804] T 3723f5ab8f974d1892ce711b33fac1dd P 7d13b2cad49e4853831a427132e294d3 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:42.824242   580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3723f5ab8f974d1892ce711b33fac1dd" candidate_uuid: "7d13b2cad49e4853831a427132e294d3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "15c6b88567e44da395f245725ffe7f04"
I20260709 13:12:42.824281   950 raft_consensus.cc:697] T 3723f5ab8f974d1892ce711b33fac1dd P 7d13b2cad49e4853831a427132e294d3 [term 2 LEADER]: Becoming Leader. State: Replica: 7d13b2cad49e4853831a427132e294d3, State: Running, Role: LEADER
I20260709 13:12:42.824946   580 raft_consensus.cc:3055] T 3723f5ab8f974d1892ce711b33fac1dd P 15c6b88567e44da395f245725ffe7f04 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:42.825289   580 raft_consensus.cc:740] T 3723f5ab8f974d1892ce711b33fac1dd P 15c6b88567e44da395f245725ffe7f04 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 15c6b88567e44da395f245725ffe7f04, State: Running, Role: LEADER
I20260709 13:12:42.825649   950 consensus_queue.cc:237] T 3723f5ab8f974d1892ce711b33fac1dd P 7d13b2cad49e4853831a427132e294d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:42.826324   580 consensus_queue.cc:260] T 3723f5ab8f974d1892ce711b33fac1dd P 15c6b88567e44da395f245725ffe7f04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } }
I20260709 13:12:42.827711   580 raft_consensus.cc:3060] T 3723f5ab8f974d1892ce711b33fac1dd P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:42.838315   580 raft_consensus.cc:2468] T 3723f5ab8f974d1892ce711b33fac1dd P 15c6b88567e44da395f245725ffe7f04 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d13b2cad49e4853831a427132e294d3 in term 2.
I20260709 13:12:42.843217   405 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 5a0083fc91ec45318745e6e9cd5b8bcf: soft_deleted_table [id=9258462a82ed4b239a497b7531d655f0] alter complete (version 1)
I20260709 13:12:42.860114   403 catalog_manager.cc:5697] T 3723f5ab8f974d1892ce711b33fac1dd P 7d13b2cad49e4853831a427132e294d3 reported cstate change: term changed from 1 to 2, leader changed from 15c6b88567e44da395f245725ffe7f04 (127.21.63.2) to 7d13b2cad49e4853831a427132e294d3 (127.21.63.3). New cstate: current_term: 2 leader_uuid: "7d13b2cad49e4853831a427132e294d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:42.869765   951 raft_consensus.cc:993] T a13e0fbeefb142f6bf766c1509cd1d93 P f35a78aeac02496d88fa800153754b0a: : Instructing follower 7d13b2cad49e4853831a427132e294d3 to start an election
I20260709 13:12:42.870250   951 raft_consensus.cc:1081] T a13e0fbeefb142f6bf766c1509cd1d93 P f35a78aeac02496d88fa800153754b0a [term 2 LEADER]: Signalling peer 7d13b2cad49e4853831a427132e294d3 to start an election
I20260709 13:12:42.871811   654 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a13e0fbeefb142f6bf766c1509cd1d93"
dest_uuid: "7d13b2cad49e4853831a427132e294d3"
 from {username='slave'} at 127.0.0.1:33230
I20260709 13:12:42.872303   654 raft_consensus.cc:493] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:42.872646   654 raft_consensus.cc:3060] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:42.877912   654 raft_consensus.cc:515] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } }
I20260709 13:12:42.879412   654 leader_election.cc:290] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3 [CANDIDATE]: Term 3 election: Requested vote from peers f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881), 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181)
I20260709 13:12:42.880347   507 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a13e0fbeefb142f6bf766c1509cd1d93" candidate_uuid: "7d13b2cad49e4853831a427132e294d3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f35a78aeac02496d88fa800153754b0a"
I20260709 13:12:42.880983   507 raft_consensus.cc:3055] T a13e0fbeefb142f6bf766c1509cd1d93 P f35a78aeac02496d88fa800153754b0a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:42.880687   580 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a13e0fbeefb142f6bf766c1509cd1d93" candidate_uuid: "7d13b2cad49e4853831a427132e294d3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "15c6b88567e44da395f245725ffe7f04"
I20260709 13:12:42.881219   507 raft_consensus.cc:740] T a13e0fbeefb142f6bf766c1509cd1d93 P f35a78aeac02496d88fa800153754b0a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f35a78aeac02496d88fa800153754b0a, State: Running, Role: LEADER
I20260709 13:12:42.881513   580 raft_consensus.cc:3060] T a13e0fbeefb142f6bf766c1509cd1d93 P 15c6b88567e44da395f245725ffe7f04 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:42.881929   507 consensus_queue.cc:260] T a13e0fbeefb142f6bf766c1509cd1d93 P f35a78aeac02496d88fa800153754b0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } }
I20260709 13:12:42.882799   507 raft_consensus.cc:3060] T a13e0fbeefb142f6bf766c1509cd1d93 P f35a78aeac02496d88fa800153754b0a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:42.889714   507 raft_consensus.cc:2468] T a13e0fbeefb142f6bf766c1509cd1d93 P f35a78aeac02496d88fa800153754b0a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d13b2cad49e4853831a427132e294d3 in term 3.
I20260709 13:12:42.890813   580 raft_consensus.cc:2468] T a13e0fbeefb142f6bf766c1509cd1d93 P 15c6b88567e44da395f245725ffe7f04 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7d13b2cad49e4853831a427132e294d3 in term 3.
I20260709 13:12:42.891108   624 leader_election.cc:304] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7d13b2cad49e4853831a427132e294d3, f35a78aeac02496d88fa800153754b0a; no voters: 
I20260709 13:12:42.892102   950 raft_consensus.cc:2804] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:42.892684   950 raft_consensus.cc:697] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3 [term 3 LEADER]: Becoming Leader. State: Replica: 7d13b2cad49e4853831a427132e294d3, State: Running, Role: LEADER
I20260709 13:12:42.893743   950 consensus_queue.cc:237] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } }
I20260709 13:12:42.903928   403 catalog_manager.cc:5697] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3 reported cstate change: term changed from 2 to 3, leader changed from f35a78aeac02496d88fa800153754b0a (127.21.63.1) to 7d13b2cad49e4853831a427132e294d3 (127.21.63.3). New cstate: current_term: 3 leader_uuid: "7d13b2cad49e4853831a427132e294d3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d13b2cad49e4853831a427132e294d3" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 42695 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:43.163436   580 raft_consensus.cc:1275] T 78d4c60d0e4b42ef9e2bc081980888af P 15c6b88567e44da395f245725ffe7f04 [term 2 FOLLOWER]: Refusing update from remote peer 7d13b2cad49e4853831a427132e294d3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:12:43.165035   956 consensus_queue.cc:1048] T 78d4c60d0e4b42ef9e2bc081980888af P 7d13b2cad49e4853831a427132e294d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:43.174451   507 raft_consensus.cc:1275] T 78d4c60d0e4b42ef9e2bc081980888af P f35a78aeac02496d88fa800153754b0a [term 2 FOLLOWER]: Refusing update from remote peer 7d13b2cad49e4853831a427132e294d3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 13:12:43.176170   950 consensus_queue.cc:1048] T 78d4c60d0e4b42ef9e2bc081980888af P 7d13b2cad49e4853831a427132e294d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:43.188697   406 catalog_manager.cc:5789] TS 7d13b2cad49e4853831a427132e294d3 (127.21.63.3:42695) does not have the latest schema for tablet 16c6d1765fc145348e0a4f703fe6abdf (table soft_deleted_table [id=9258462a82ed4b239a497b7531d655f0]). Expected version 1 got 0
I20260709 13:12:43.207666   507 raft_consensus.cc:1275] T 3723f5ab8f974d1892ce711b33fac1dd P f35a78aeac02496d88fa800153754b0a [term 2 FOLLOWER]: Refusing update from remote peer 7d13b2cad49e4853831a427132e294d3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:43.209093   950 consensus_queue.cc:1048] T 3723f5ab8f974d1892ce711b33fac1dd P 7d13b2cad49e4853831a427132e294d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:43.218637   580 raft_consensus.cc:1275] T 3723f5ab8f974d1892ce711b33fac1dd P 15c6b88567e44da395f245725ffe7f04 [term 2 FOLLOWER]: Refusing update from remote peer 7d13b2cad49e4853831a427132e294d3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:43.221158   956 consensus_queue.cc:1048] T 3723f5ab8f974d1892ce711b33fac1dd P 7d13b2cad49e4853831a427132e294d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:43.242846   507 raft_consensus.cc:1275] T 16c6d1765fc145348e0a4f703fe6abdf P f35a78aeac02496d88fa800153754b0a [term 3 FOLLOWER]: Refusing update from remote peer 7d13b2cad49e4853831a427132e294d3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:12:43.244393   956 consensus_queue.cc:1048] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:43.250444   973 tablet.cc:1724] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:43.252722   580 raft_consensus.cc:1275] T 16c6d1765fc145348e0a4f703fe6abdf P 15c6b88567e44da395f245725ffe7f04 [term 3 FOLLOWER]: Refusing update from remote peer 7d13b2cad49e4853831a427132e294d3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 13:12:43.255097   950 consensus_queue.cc:1048] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 13:12:43.286597   973 tablet.cc:1719] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3: Skipping requested alter to schema version 1, tablet already version 1
I20260709 13:12:43.288455   967 tablet.cc:1719] T 16c6d1765fc145348e0a4f703fe6abdf P 15c6b88567e44da395f245725ffe7f04: Skipping requested alter to schema version 1, tablet already version 1
I20260709 13:12:43.294787   986 tablet.cc:1719] T 16c6d1765fc145348e0a4f703fe6abdf P f35a78aeac02496d88fa800153754b0a: Skipping requested alter to schema version 1, tablet already version 1
I20260709 13:12:43.341112   507 raft_consensus.cc:1275] T a13e0fbeefb142f6bf766c1509cd1d93 P f35a78aeac02496d88fa800153754b0a [term 3 FOLLOWER]: Refusing update from remote peer 7d13b2cad49e4853831a427132e294d3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:43.342411   950 consensus_queue.cc:1048] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f35a78aeac02496d88fa800153754b0a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 34881 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:43.352254   580 raft_consensus.cc:1275] T a13e0fbeefb142f6bf766c1509cd1d93 P 15c6b88567e44da395f245725ffe7f04 [term 3 FOLLOWER]: Refusing update from remote peer 7d13b2cad49e4853831a427132e294d3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:43.353629   950 consensus_queue.cc:1048] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "15c6b88567e44da395f245725ffe7f04" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 38181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:43.642895 21756 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 13:12:43.647956 21756 auto_leader_rebalancer.cc:452] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 13:12:43.648445 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:43.653710 21756 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 13:12:43.654276 21756 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:12:44.686143 21756 tablet_server.cc:179] TabletServer@127.21.63.1:0 shutting down...
W20260709 13:12:44.714309   550 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 3 similar messages]
W20260709 13:12:44.718874   550 consensus_peers.cc:597] T 6653d3e90150480daf2486b331be3130 P 15c6b88567e44da395f245725ffe7f04 -> Peer f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881): Couldn't send request to peer f35a78aeac02496d88fa800153754b0a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:44.720427   550 consensus_peers.cc:597] T 6e5c1404cee54b96a9c3a8c024493223 P 15c6b88567e44da395f245725ffe7f04 -> Peer f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881): Couldn't send request to peer f35a78aeac02496d88fa800153754b0a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:44.729661 21756 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:12:44.730695 21756 tablet_replica.cc:333] T 42ebb32ae777484fab5e26a137cbd4f6 P f35a78aeac02496d88fa800153754b0a: stopping tablet replica
I20260709 13:12:44.731693 21756 raft_consensus.cc:2243] T 42ebb32ae777484fab5e26a137cbd4f6 P f35a78aeac02496d88fa800153754b0a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:44.732213 21756 raft_consensus.cc:2272] T 42ebb32ae777484fab5e26a137cbd4f6 P f35a78aeac02496d88fa800153754b0a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:44.734797 21756 tablet_replica.cc:333] T f75ad0645c22439c9a0611d214158d01 P f35a78aeac02496d88fa800153754b0a: stopping tablet replica
I20260709 13:12:44.735512 21756 raft_consensus.cc:2243] T f75ad0645c22439c9a0611d214158d01 P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:44.736037 21756 raft_consensus.cc:2272] T f75ad0645c22439c9a0611d214158d01 P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:44.738471 21756 tablet_replica.cc:333] T 78d4c60d0e4b42ef9e2bc081980888af P f35a78aeac02496d88fa800153754b0a: stopping tablet replica
I20260709 13:12:44.739001 21756 raft_consensus.cc:2243] T 78d4c60d0e4b42ef9e2bc081980888af P f35a78aeac02496d88fa800153754b0a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:44.739382 21756 raft_consensus.cc:2272] T 78d4c60d0e4b42ef9e2bc081980888af P f35a78aeac02496d88fa800153754b0a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:44.741796 21756 tablet_replica.cc:333] T 16c6d1765fc145348e0a4f703fe6abdf P f35a78aeac02496d88fa800153754b0a: stopping tablet replica
I20260709 13:12:44.742285 21756 raft_consensus.cc:2243] T 16c6d1765fc145348e0a4f703fe6abdf P f35a78aeac02496d88fa800153754b0a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:44.742683 21756 raft_consensus.cc:2272] T 16c6d1765fc145348e0a4f703fe6abdf P f35a78aeac02496d88fa800153754b0a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:44.744601 21756 tablet_replica.cc:333] T d7f398f030d840cab518d8996f2bdbce P f35a78aeac02496d88fa800153754b0a: stopping tablet replica
I20260709 13:12:44.745354 21756 raft_consensus.cc:2243] T d7f398f030d840cab518d8996f2bdbce P f35a78aeac02496d88fa800153754b0a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:44.745766 21756 raft_consensus.cc:2272] T d7f398f030d840cab518d8996f2bdbce P f35a78aeac02496d88fa800153754b0a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:44.747772 21756 tablet_replica.cc:333] T def0acc1a0684b9b86755e1c34191d91 P f35a78aeac02496d88fa800153754b0a: stopping tablet replica
I20260709 13:12:44.748281 21756 raft_consensus.cc:2243] T def0acc1a0684b9b86755e1c34191d91 P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:44.748652 21756 raft_consensus.cc:2272] T def0acc1a0684b9b86755e1c34191d91 P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:44.751004 21756 tablet_replica.cc:333] T 9082b9270cf74ed5b72b7bc4b0166836 P f35a78aeac02496d88fa800153754b0a: stopping tablet replica
I20260709 13:12:44.751926 21756 raft_consensus.cc:2243] T 9082b9270cf74ed5b72b7bc4b0166836 P f35a78aeac02496d88fa800153754b0a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:44.752880 21756 raft_consensus.cc:2272] T 9082b9270cf74ed5b72b7bc4b0166836 P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:44.755571 21756 tablet_replica.cc:333] T b9b003cc8908492ab10df382680b5141 P f35a78aeac02496d88fa800153754b0a: stopping tablet replica
W20260709 13:12:44.755667   550 consensus_peers.cc:597] T 82ac3d8dfa714f66a589173f6a9eb004 P 15c6b88567e44da395f245725ffe7f04 -> Peer f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881): Couldn't send request to peer f35a78aeac02496d88fa800153754b0a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:44.756242 21756 raft_consensus.cc:2243] T b9b003cc8908492ab10df382680b5141 P f35a78aeac02496d88fa800153754b0a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:44.757272 21756 raft_consensus.cc:2272] T b9b003cc8908492ab10df382680b5141 P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:44.760100 21756 tablet_replica.cc:333] T e6a5dbc01970406599bc229548544689 P f35a78aeac02496d88fa800153754b0a: stopping tablet replica
I20260709 13:12:44.760821 21756 raft_consensus.cc:2243] T e6a5dbc01970406599bc229548544689 P f35a78aeac02496d88fa800153754b0a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:44.761603 21756 raft_consensus.cc:2272] T e6a5dbc01970406599bc229548544689 P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:44.763638 21756 tablet_replica.cc:333] T bedab716985143c7a40f88c69d71ecd4 P f35a78aeac02496d88fa800153754b0a: stopping tablet replica
I20260709 13:12:44.764199 21756 raft_consensus.cc:2243] T bedab716985143c7a40f88c69d71ecd4 P f35a78aeac02496d88fa800153754b0a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:44.765156 21756 raft_consensus.cc:2272] T bedab716985143c7a40f88c69d71ecd4 P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:44.767692 21756 tablet_replica.cc:333] T 3723f5ab8f974d1892ce711b33fac1dd P f35a78aeac02496d88fa800153754b0a: stopping tablet replica
I20260709 13:12:44.768237 21756 raft_consensus.cc:2243] T 3723f5ab8f974d1892ce711b33fac1dd P f35a78aeac02496d88fa800153754b0a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:44.768728 21756 raft_consensus.cc:2272] T 3723f5ab8f974d1892ce711b33fac1dd P f35a78aeac02496d88fa800153754b0a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:44.770640 21756 tablet_replica.cc:333] T 93236792b8284277a935d6e569d4c96b P f35a78aeac02496d88fa800153754b0a: stopping tablet replica
I20260709 13:12:44.771152 21756 raft_consensus.cc:2243] T 93236792b8284277a935d6e569d4c96b P f35a78aeac02496d88fa800153754b0a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:44.771898 21756 raft_consensus.cc:2272] T 93236792b8284277a935d6e569d4c96b P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:44.774672 21756 tablet_replica.cc:333] T 6653d3e90150480daf2486b331be3130 P f35a78aeac02496d88fa800153754b0a: stopping tablet replica
I20260709 13:12:44.775326 21756 raft_consensus.cc:2243] T 6653d3e90150480daf2486b331be3130 P f35a78aeac02496d88fa800153754b0a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:44.775820 21756 raft_consensus.cc:2272] T 6653d3e90150480daf2486b331be3130 P f35a78aeac02496d88fa800153754b0a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:44.777853 21756 tablet_replica.cc:333] T 6e5c1404cee54b96a9c3a8c024493223 P f35a78aeac02496d88fa800153754b0a: stopping tablet replica
I20260709 13:12:44.778385 21756 raft_consensus.cc:2243] T 6e5c1404cee54b96a9c3a8c024493223 P f35a78aeac02496d88fa800153754b0a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:44.778879 21756 raft_consensus.cc:2272] T 6e5c1404cee54b96a9c3a8c024493223 P f35a78aeac02496d88fa800153754b0a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:44.781335 21756 tablet_replica.cc:333] T 3b24ef3f665449c288b2fcd28c288d27 P f35a78aeac02496d88fa800153754b0a: stopping tablet replica
I20260709 13:12:44.782078 21756 raft_consensus.cc:2243] T 3b24ef3f665449c288b2fcd28c288d27 P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:44.782565 21756 raft_consensus.cc:2272] T 3b24ef3f665449c288b2fcd28c288d27 P f35a78aeac02496d88fa800153754b0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:44.784948 21756 tablet_replica.cc:333] T 82ac3d8dfa714f66a589173f6a9eb004 P f35a78aeac02496d88fa800153754b0a: stopping tablet replica
W20260709 13:12:44.785322   624 consensus_peers.cc:597] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3 -> Peer f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881): Couldn't send request to peer f35a78aeac02496d88fa800153754b0a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:44.785519 21756 raft_consensus.cc:2243] T 82ac3d8dfa714f66a589173f6a9eb004 P f35a78aeac02496d88fa800153754b0a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:44.786121 21756 raft_consensus.cc:2272] T 82ac3d8dfa714f66a589173f6a9eb004 P f35a78aeac02496d88fa800153754b0a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:44.788134 21756 tablet_replica.cc:333] T bf7b037540fc46d098bf7e1888e9c682 P f35a78aeac02496d88fa800153754b0a: stopping tablet replica
I20260709 13:12:44.788645 21756 raft_consensus.cc:2243] T bf7b037540fc46d098bf7e1888e9c682 P f35a78aeac02496d88fa800153754b0a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:44.789491 21756 raft_consensus.cc:2272] T bf7b037540fc46d098bf7e1888e9c682 P f35a78aeac02496d88fa800153754b0a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:44.791468 21756 tablet_replica.cc:333] T a13e0fbeefb142f6bf766c1509cd1d93 P f35a78aeac02496d88fa800153754b0a: stopping tablet replica
I20260709 13:12:44.791973 21756 raft_consensus.cc:2243] T a13e0fbeefb142f6bf766c1509cd1d93 P f35a78aeac02496d88fa800153754b0a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:44.792371 21756 raft_consensus.cc:2272] T a13e0fbeefb142f6bf766c1509cd1d93 P f35a78aeac02496d88fa800153754b0a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:44.834704 21756 tablet_server.cc:196] TabletServer@127.21.63.1:0 shutdown complete.
W20260709 13:12:44.878125   550 consensus_peers.cc:597] T d7f398f030d840cab518d8996f2bdbce P 15c6b88567e44da395f245725ffe7f04 -> Peer f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881): Couldn't send request to peer f35a78aeac02496d88fa800153754b0a. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:34881: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:44.887207 21756 tablet_server.cc:179] TabletServer@127.21.63.2:0 shutting down...
W20260709 13:12:44.891201   550 consensus_peers.cc:597] T f75ad0645c22439c9a0611d214158d01 P 15c6b88567e44da395f245725ffe7f04 -> Peer f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881): Couldn't send request to peer f35a78aeac02496d88fa800153754b0a. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:34881: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:44.926967 21756 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:12:44.927867 21756 tablet_replica.cc:333] T 42ebb32ae777484fab5e26a137cbd4f6 P 15c6b88567e44da395f245725ffe7f04: stopping tablet replica
I20260709 13:12:44.929059 21756 raft_consensus.cc:2243] T 42ebb32ae777484fab5e26a137cbd4f6 P 15c6b88567e44da395f245725ffe7f04 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:44.929759 21756 raft_consensus.cc:2272] T 42ebb32ae777484fab5e26a137cbd4f6 P 15c6b88567e44da395f245725ffe7f04 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:44.931836 21756 tablet_replica.cc:333] T 78d4c60d0e4b42ef9e2bc081980888af P 15c6b88567e44da395f245725ffe7f04: stopping tablet replica
I20260709 13:12:44.932360 21756 raft_consensus.cc:2243] T 78d4c60d0e4b42ef9e2bc081980888af P 15c6b88567e44da395f245725ffe7f04 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:44.932823 21756 raft_consensus.cc:2272] T 78d4c60d0e4b42ef9e2bc081980888af P 15c6b88567e44da395f245725ffe7f04 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:44.934610 21756 tablet_replica.cc:333] T 16c6d1765fc145348e0a4f703fe6abdf P 15c6b88567e44da395f245725ffe7f04: stopping tablet replica
I20260709 13:12:44.935133 21756 raft_consensus.cc:2243] T 16c6d1765fc145348e0a4f703fe6abdf P 15c6b88567e44da395f245725ffe7f04 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:44.935571 21756 raft_consensus.cc:2272] T 16c6d1765fc145348e0a4f703fe6abdf P 15c6b88567e44da395f245725ffe7f04 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:44.937471 21756 tablet_replica.cc:333] T d7f398f030d840cab518d8996f2bdbce P 15c6b88567e44da395f245725ffe7f04: stopping tablet replica
I20260709 13:12:44.937951 21756 raft_consensus.cc:2243] T d7f398f030d840cab518d8996f2bdbce P 15c6b88567e44da395f245725ffe7f04 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:44.938665 21756 raft_consensus.cc:2272] T d7f398f030d840cab518d8996f2bdbce P 15c6b88567e44da395f245725ffe7f04 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:44.940449 21756 tablet_replica.cc:333] T b9b003cc8908492ab10df382680b5141 P 15c6b88567e44da395f245725ffe7f04: stopping tablet replica
I20260709 13:12:44.941022 21756 raft_consensus.cc:2243] T b9b003cc8908492ab10df382680b5141 P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:44.941409 21756 raft_consensus.cc:2272] T b9b003cc8908492ab10df382680b5141 P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:44.943174 21756 tablet_replica.cc:333] T f75ad0645c22439c9a0611d214158d01 P 15c6b88567e44da395f245725ffe7f04: stopping tablet replica
I20260709 13:12:44.943656 21756 raft_consensus.cc:2243] T f75ad0645c22439c9a0611d214158d01 P 15c6b88567e44da395f245725ffe7f04 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:44.944319 21756 raft_consensus.cc:2272] T f75ad0645c22439c9a0611d214158d01 P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:44.946079 21756 tablet_replica.cc:333] T 9082b9270cf74ed5b72b7bc4b0166836 P 15c6b88567e44da395f245725ffe7f04: stopping tablet replica
I20260709 13:12:44.946543 21756 raft_consensus.cc:2243] T 9082b9270cf74ed5b72b7bc4b0166836 P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:44.946905 21756 raft_consensus.cc:2272] T 9082b9270cf74ed5b72b7bc4b0166836 P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:44.948640 21756 tablet_replica.cc:333] T def0acc1a0684b9b86755e1c34191d91 P 15c6b88567e44da395f245725ffe7f04: stopping tablet replica
I20260709 13:12:44.949157 21756 raft_consensus.cc:2243] T def0acc1a0684b9b86755e1c34191d91 P 15c6b88567e44da395f245725ffe7f04 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:44.949822 21756 raft_consensus.cc:2272] T def0acc1a0684b9b86755e1c34191d91 P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:44.951763 21756 tablet_replica.cc:333] T e6a5dbc01970406599bc229548544689 P 15c6b88567e44da395f245725ffe7f04: stopping tablet replica
I20260709 13:12:44.952179 21756 raft_consensus.cc:2243] T e6a5dbc01970406599bc229548544689 P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:44.952554 21756 raft_consensus.cc:2272] T e6a5dbc01970406599bc229548544689 P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:44.955149 21756 tablet_replica.cc:333] T bedab716985143c7a40f88c69d71ecd4 P 15c6b88567e44da395f245725ffe7f04: stopping tablet replica
I20260709 13:12:44.956048 21756 raft_consensus.cc:2243] T bedab716985143c7a40f88c69d71ecd4 P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:44.956558 21756 raft_consensus.cc:2272] T bedab716985143c7a40f88c69d71ecd4 P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:44.958096   621 consensus_peers.cc:597] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 -> Peer 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181): Couldn't send request to peer 15c6b88567e44da395f245725ffe7f04. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:44.959105 21756 tablet_replica.cc:333] T a13e0fbeefb142f6bf766c1509cd1d93 P 15c6b88567e44da395f245725ffe7f04: stopping tablet replica
I20260709 13:12:44.959667 21756 raft_consensus.cc:2243] T a13e0fbeefb142f6bf766c1509cd1d93 P 15c6b88567e44da395f245725ffe7f04 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:44.960054 21756 raft_consensus.cc:2272] T a13e0fbeefb142f6bf766c1509cd1d93 P 15c6b88567e44da395f245725ffe7f04 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:44.962083 21756 tablet_replica.cc:333] T 6e5c1404cee54b96a9c3a8c024493223 P 15c6b88567e44da395f245725ffe7f04: stopping tablet replica
I20260709 13:12:44.962550 21756 raft_consensus.cc:2243] T 6e5c1404cee54b96a9c3a8c024493223 P 15c6b88567e44da395f245725ffe7f04 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:44.963289 21756 raft_consensus.cc:2272] T 6e5c1404cee54b96a9c3a8c024493223 P 15c6b88567e44da395f245725ffe7f04 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:44.965937 21756 tablet_replica.cc:333] T 3b24ef3f665449c288b2fcd28c288d27 P 15c6b88567e44da395f245725ffe7f04: stopping tablet replica
I20260709 13:12:44.966503 21756 raft_consensus.cc:2243] T 3b24ef3f665449c288b2fcd28c288d27 P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:44.966984 21756 raft_consensus.cc:2272] T 3b24ef3f665449c288b2fcd28c288d27 P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:44.969353 21756 tablet_replica.cc:333] T 6653d3e90150480daf2486b331be3130 P 15c6b88567e44da395f245725ffe7f04: stopping tablet replica
I20260709 13:12:44.969897 21756 raft_consensus.cc:2243] T 6653d3e90150480daf2486b331be3130 P 15c6b88567e44da395f245725ffe7f04 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:44.970865 21756 raft_consensus.cc:2272] T 6653d3e90150480daf2486b331be3130 P 15c6b88567e44da395f245725ffe7f04 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:44.972886 21756 tablet_replica.cc:333] T 93236792b8284277a935d6e569d4c96b P 15c6b88567e44da395f245725ffe7f04: stopping tablet replica
I20260709 13:12:44.973527 21756 raft_consensus.cc:2243] T 93236792b8284277a935d6e569d4c96b P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:44.973995 21756 raft_consensus.cc:2272] T 93236792b8284277a935d6e569d4c96b P 15c6b88567e44da395f245725ffe7f04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:44.976095 21756 tablet_replica.cc:333] T 3723f5ab8f974d1892ce711b33fac1dd P 15c6b88567e44da395f245725ffe7f04: stopping tablet replica
I20260709 13:12:44.976598 21756 raft_consensus.cc:2243] T 3723f5ab8f974d1892ce711b33fac1dd P 15c6b88567e44da395f245725ffe7f04 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:44.977077 21756 raft_consensus.cc:2272] T 3723f5ab8f974d1892ce711b33fac1dd P 15c6b88567e44da395f245725ffe7f04 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:44.979079 21756 tablet_replica.cc:333] T 82ac3d8dfa714f66a589173f6a9eb004 P 15c6b88567e44da395f245725ffe7f04: stopping tablet replica
I20260709 13:12:44.979585 21756 raft_consensus.cc:2243] T 82ac3d8dfa714f66a589173f6a9eb004 P 15c6b88567e44da395f245725ffe7f04 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:44.980418 21756 raft_consensus.cc:2272] T 82ac3d8dfa714f66a589173f6a9eb004 P 15c6b88567e44da395f245725ffe7f04 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:44.982560 21756 tablet_replica.cc:333] T bf7b037540fc46d098bf7e1888e9c682 P 15c6b88567e44da395f245725ffe7f04: stopping tablet replica
I20260709 13:12:44.983047 21756 raft_consensus.cc:2243] T bf7b037540fc46d098bf7e1888e9c682 P 15c6b88567e44da395f245725ffe7f04 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:44.983484 21756 raft_consensus.cc:2272] T bf7b037540fc46d098bf7e1888e9c682 P 15c6b88567e44da395f245725ffe7f04 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:44.996181   624 consensus_peers.cc:597] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 -> Peer f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881): Couldn't send request to peer f35a78aeac02496d88fa800153754b0a. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:34881: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:45.015995 21756 tablet_server.cc:196] TabletServer@127.21.63.2:0 shutdown complete.
W20260709 13:12:45.049872   624 consensus_peers.cc:597] T 42ebb32ae777484fab5e26a137cbd4f6 P 7d13b2cad49e4853831a427132e294d3 -> Peer f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881): Couldn't send request to peer f35a78aeac02496d88fa800153754b0a. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:34881: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:45.052145 21756 tablet_server.cc:179] TabletServer@127.21.63.3:0 shutting down...
W20260709 13:12:45.058944   621 consensus_peers.cc:597] T 3b24ef3f665449c288b2fcd28c288d27 P 7d13b2cad49e4853831a427132e294d3 -> Peer 15c6b88567e44da395f245725ffe7f04 (127.21.63.2:38181): Couldn't send request to peer 15c6b88567e44da395f245725ffe7f04. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:38181: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 13:12:45.066869   624 consensus_peers.cc:597] T 3b24ef3f665449c288b2fcd28c288d27 P 7d13b2cad49e4853831a427132e294d3 -> Peer f35a78aeac02496d88fa800153754b0a (127.21.63.1:34881): Couldn't send request to peer f35a78aeac02496d88fa800153754b0a. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.1:34881: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:45.086862 21756 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:12:45.087728 21756 tablet_replica.cc:333] T 42ebb32ae777484fab5e26a137cbd4f6 P 7d13b2cad49e4853831a427132e294d3: stopping tablet replica
I20260709 13:12:45.088893 21756 raft_consensus.cc:2243] T 42ebb32ae777484fab5e26a137cbd4f6 P 7d13b2cad49e4853831a427132e294d3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:45.089838 21756 raft_consensus.cc:2272] T 42ebb32ae777484fab5e26a137cbd4f6 P 7d13b2cad49e4853831a427132e294d3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:45.091779 21756 tablet_replica.cc:333] T f75ad0645c22439c9a0611d214158d01 P 7d13b2cad49e4853831a427132e294d3: stopping tablet replica
I20260709 13:12:45.092274 21756 raft_consensus.cc:2243] T f75ad0645c22439c9a0611d214158d01 P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:45.092779 21756 raft_consensus.cc:2272] T f75ad0645c22439c9a0611d214158d01 P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:45.094643 21756 tablet_replica.cc:333] T 9082b9270cf74ed5b72b7bc4b0166836 P 7d13b2cad49e4853831a427132e294d3: stopping tablet replica
I20260709 13:12:45.095129 21756 raft_consensus.cc:2243] T 9082b9270cf74ed5b72b7bc4b0166836 P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:45.095566 21756 raft_consensus.cc:2272] T 9082b9270cf74ed5b72b7bc4b0166836 P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:45.097545 21756 tablet_replica.cc:333] T def0acc1a0684b9b86755e1c34191d91 P 7d13b2cad49e4853831a427132e294d3: stopping tablet replica
I20260709 13:12:45.098060 21756 raft_consensus.cc:2243] T def0acc1a0684b9b86755e1c34191d91 P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:45.098434 21756 raft_consensus.cc:2272] T def0acc1a0684b9b86755e1c34191d91 P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:45.100287 21756 tablet_replica.cc:333] T b9b003cc8908492ab10df382680b5141 P 7d13b2cad49e4853831a427132e294d3: stopping tablet replica
I20260709 13:12:45.100857 21756 raft_consensus.cc:2243] T b9b003cc8908492ab10df382680b5141 P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:45.101365 21756 raft_consensus.cc:2272] T b9b003cc8908492ab10df382680b5141 P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:45.103358 21756 tablet_replica.cc:333] T d7f398f030d840cab518d8996f2bdbce P 7d13b2cad49e4853831a427132e294d3: stopping tablet replica
I20260709 13:12:45.103828 21756 raft_consensus.cc:2243] T d7f398f030d840cab518d8996f2bdbce P 7d13b2cad49e4853831a427132e294d3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:45.104185 21756 raft_consensus.cc:2272] T d7f398f030d840cab518d8996f2bdbce P 7d13b2cad49e4853831a427132e294d3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:45.106010 21756 tablet_replica.cc:333] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3: stopping tablet replica
I20260709 13:12:45.106554 21756 raft_consensus.cc:2243] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:12:45.107236 21756 raft_consensus.cc:2272] T 16c6d1765fc145348e0a4f703fe6abdf P 7d13b2cad49e4853831a427132e294d3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:45.109246 21756 tablet_replica.cc:333] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3: stopping tablet replica
I20260709 13:12:45.109728 21756 raft_consensus.cc:2243] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:12:45.110541 21756 raft_consensus.cc:2272] T a13e0fbeefb142f6bf766c1509cd1d93 P 7d13b2cad49e4853831a427132e294d3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:45.112473 21756 tablet_replica.cc:333] T 93236792b8284277a935d6e569d4c96b P 7d13b2cad49e4853831a427132e294d3: stopping tablet replica
I20260709 13:12:45.113019 21756 raft_consensus.cc:2243] T 93236792b8284277a935d6e569d4c96b P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:45.113443 21756 raft_consensus.cc:2272] T 93236792b8284277a935d6e569d4c96b P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:45.115271 21756 tablet_replica.cc:333] T 3723f5ab8f974d1892ce711b33fac1dd P 7d13b2cad49e4853831a427132e294d3: stopping tablet replica
I20260709 13:12:45.115720 21756 raft_consensus.cc:2243] T 3723f5ab8f974d1892ce711b33fac1dd P 7d13b2cad49e4853831a427132e294d3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:45.116374 21756 raft_consensus.cc:2272] T 3723f5ab8f974d1892ce711b33fac1dd P 7d13b2cad49e4853831a427132e294d3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:45.118279 21756 tablet_replica.cc:333] T 6653d3e90150480daf2486b331be3130 P 7d13b2cad49e4853831a427132e294d3: stopping tablet replica
I20260709 13:12:45.118793 21756 raft_consensus.cc:2243] T 6653d3e90150480daf2486b331be3130 P 7d13b2cad49e4853831a427132e294d3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:45.119171 21756 raft_consensus.cc:2272] T 6653d3e90150480daf2486b331be3130 P 7d13b2cad49e4853831a427132e294d3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:45.121042 21756 tablet_replica.cc:333] T 6e5c1404cee54b96a9c3a8c024493223 P 7d13b2cad49e4853831a427132e294d3: stopping tablet replica
I20260709 13:12:45.121718 21756 raft_consensus.cc:2243] T 6e5c1404cee54b96a9c3a8c024493223 P 7d13b2cad49e4853831a427132e294d3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:45.122213 21756 raft_consensus.cc:2272] T 6e5c1404cee54b96a9c3a8c024493223 P 7d13b2cad49e4853831a427132e294d3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:45.124172 21756 tablet_replica.cc:333] T 3b24ef3f665449c288b2fcd28c288d27 P 7d13b2cad49e4853831a427132e294d3: stopping tablet replica
I20260709 13:12:45.124637 21756 raft_consensus.cc:2243] T 3b24ef3f665449c288b2fcd28c288d27 P 7d13b2cad49e4853831a427132e294d3 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:45.125422 21756 raft_consensus.cc:2272] T 3b24ef3f665449c288b2fcd28c288d27 P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:45.127295 21756 tablet_replica.cc:333] T 78d4c60d0e4b42ef9e2bc081980888af P 7d13b2cad49e4853831a427132e294d3: stopping tablet replica
I20260709 13:12:45.127733 21756 raft_consensus.cc:2243] T 78d4c60d0e4b42ef9e2bc081980888af P 7d13b2cad49e4853831a427132e294d3 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:45.128379 21756 raft_consensus.cc:2272] T 78d4c60d0e4b42ef9e2bc081980888af P 7d13b2cad49e4853831a427132e294d3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:45.130218 21756 tablet_replica.cc:333] T 82ac3d8dfa714f66a589173f6a9eb004 P 7d13b2cad49e4853831a427132e294d3: stopping tablet replica
I20260709 13:12:45.130647 21756 raft_consensus.cc:2243] T 82ac3d8dfa714f66a589173f6a9eb004 P 7d13b2cad49e4853831a427132e294d3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:45.131021 21756 raft_consensus.cc:2272] T 82ac3d8dfa714f66a589173f6a9eb004 P 7d13b2cad49e4853831a427132e294d3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:45.132948 21756 tablet_replica.cc:333] T bf7b037540fc46d098bf7e1888e9c682 P 7d13b2cad49e4853831a427132e294d3: stopping tablet replica
I20260709 13:12:45.133353 21756 raft_consensus.cc:2243] T bf7b037540fc46d098bf7e1888e9c682 P 7d13b2cad49e4853831a427132e294d3 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:45.133713 21756 raft_consensus.cc:2272] T bf7b037540fc46d098bf7e1888e9c682 P 7d13b2cad49e4853831a427132e294d3 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:45.135483 21756 tablet_replica.cc:333] T bedab716985143c7a40f88c69d71ecd4 P 7d13b2cad49e4853831a427132e294d3: stopping tablet replica
I20260709 13:12:45.135882 21756 raft_consensus.cc:2243] T bedab716985143c7a40f88c69d71ecd4 P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:45.136236 21756 raft_consensus.cc:2272] T bedab716985143c7a40f88c69d71ecd4 P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:45.138077 21756 tablet_replica.cc:333] T e6a5dbc01970406599bc229548544689 P 7d13b2cad49e4853831a427132e294d3: stopping tablet replica
I20260709 13:12:45.138484 21756 raft_consensus.cc:2243] T e6a5dbc01970406599bc229548544689 P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:45.138854 21756 raft_consensus.cc:2272] T e6a5dbc01970406599bc229548544689 P 7d13b2cad49e4853831a427132e294d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:45.178215 21756 tablet_server.cc:196] TabletServer@127.21.63.3:0 shutdown complete.
I20260709 13:12:45.213513 21756 master.cc:562] Master@127.21.63.62:35237 shutting down...
I20260709 13:12:45.237149 21756 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 5a0083fc91ec45318745e6e9cd5b8bcf [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:45.237785 21756 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 5a0083fc91ec45318745e6e9cd5b8bcf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:45.238075 21756 tablet_replica.cc:333] T 00000000000000000000000000000000 P 5a0083fc91ec45318745e6e9cd5b8bcf: stopping tablet replica
I20260709 13:12:45.281711 21756 master.cc:584] Master@127.21.63.62:35237 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (12907 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 13:12:45.335453 21756 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.21.63.62:41855
I20260709 13:12:45.336578 21756 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:12:45.342783   997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:45.342803   996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:45.343446   999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:45.343819 21756 server_base.cc:1061] running on GCE node
I20260709 13:12:45.344991 21756 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:12:45.345170 21756 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:12:45.345394 21756 hybrid_clock.cc:648] HybridClock initialized: now 1783602765345364 us; error 0 us; skew 500 ppm
I20260709 13:12:45.345925 21756 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:12:45.348219 21756 webserver.cc:533] Webserver started at http://127.21.63.62:37313/ using document root <none> and password file <none>
I20260709 13:12:45.348881 21756 fs_manager.cc:362] Metadata directory not provided
I20260709 13:12:45.349064 21756 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:12:45.349354 21756 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:12:45.350446 21756 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602429420501-21756-0/minicluster-data/master-0-root/instance:
uuid: "31065877c4af477d9b2e726230936891"
format_stamp: "Formatted at 2026-07-09 13:12:45 on dist-test-slave-c6mt"
I20260709 13:12:45.354816 21756 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.004s	sys 0.001s
I20260709 13:12:45.358249  1004 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:45.359045 21756 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:12:45.359390 21756 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602429420501-21756-0/minicluster-data/master-0-root
uuid: "31065877c4af477d9b2e726230936891"
format_stamp: "Formatted at 2026-07-09 13:12:45 on dist-test-slave-c6mt"
I20260709 13:12:45.359675 21756 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602429420501-21756-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602429420501-21756-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602429420501-21756-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:12:45.374065 21756 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:12:45.375273 21756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:12:45.417438 21756 rpc_server.cc:307] RPC server started. Bound to: 127.21.63.62:41855
I20260709 13:12:45.417551  1055 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.63.62:41855 every 8 connection(s)
I20260709 13:12:45.421391  1056 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:45.432008  1056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31065877c4af477d9b2e726230936891: Bootstrap starting.
I20260709 13:12:45.437425  1056 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31065877c4af477d9b2e726230936891: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:45.441847  1056 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31065877c4af477d9b2e726230936891: No bootstrap required, opened a new log
I20260709 13:12:45.444080  1056 raft_consensus.cc:359] T 00000000000000000000000000000000 P 31065877c4af477d9b2e726230936891 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31065877c4af477d9b2e726230936891" member_type: VOTER }
I20260709 13:12:45.444459  1056 raft_consensus.cc:385] T 00000000000000000000000000000000 P 31065877c4af477d9b2e726230936891 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:45.444777  1056 raft_consensus.cc:740] T 00000000000000000000000000000000 P 31065877c4af477d9b2e726230936891 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31065877c4af477d9b2e726230936891, State: Initialized, Role: FOLLOWER
I20260709 13:12:45.445379  1056 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31065877c4af477d9b2e726230936891 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31065877c4af477d9b2e726230936891" member_type: VOTER }
I20260709 13:12:45.445817  1056 raft_consensus.cc:399] T 00000000000000000000000000000000 P 31065877c4af477d9b2e726230936891 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 13:12:45.446030  1056 raft_consensus.cc:493] T 00000000000000000000000000000000 P 31065877c4af477d9b2e726230936891 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 13:12:45.446333  1056 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 31065877c4af477d9b2e726230936891 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:45.451233  1056 raft_consensus.cc:515] T 00000000000000000000000000000000 P 31065877c4af477d9b2e726230936891 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31065877c4af477d9b2e726230936891" member_type: VOTER }
I20260709 13:12:45.451786  1056 leader_election.cc:304] T 00000000000000000000000000000000 P 31065877c4af477d9b2e726230936891 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31065877c4af477d9b2e726230936891; no voters: 
I20260709 13:12:45.453068  1056 leader_election.cc:290] T 00000000000000000000000000000000 P 31065877c4af477d9b2e726230936891 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 13:12:45.453444  1059 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 31065877c4af477d9b2e726230936891 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:45.454932  1059 raft_consensus.cc:697] T 00000000000000000000000000000000 P 31065877c4af477d9b2e726230936891 [term 1 LEADER]: Becoming Leader. State: Replica: 31065877c4af477d9b2e726230936891, State: Running, Role: LEADER
I20260709 13:12:45.455636  1059 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31065877c4af477d9b2e726230936891 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31065877c4af477d9b2e726230936891" member_type: VOTER }
I20260709 13:12:45.455937  1056 sys_catalog.cc:565] T 00000000000000000000000000000000 P 31065877c4af477d9b2e726230936891 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 13:12:45.460724  1060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31065877c4af477d9b2e726230936891 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "31065877c4af477d9b2e726230936891" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31065877c4af477d9b2e726230936891" member_type: VOTER } }
I20260709 13:12:45.460825  1061 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31065877c4af477d9b2e726230936891 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31065877c4af477d9b2e726230936891. Latest consensus state: current_term: 1 leader_uuid: "31065877c4af477d9b2e726230936891" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31065877c4af477d9b2e726230936891" member_type: VOTER } }
I20260709 13:12:45.461525  1060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31065877c4af477d9b2e726230936891 [sys.catalog]: This master's current role is: LEADER
I20260709 13:12:45.461781  1061 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31065877c4af477d9b2e726230936891 [sys.catalog]: This master's current role is: LEADER
I20260709 13:12:45.468413  1066 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 13:12:45.476933  1066 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 13:12:45.478202 21756 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 13:12:45.487139  1066 catalog_manager.cc:1383] Generated new cluster ID: 9987e5c20f5f4a35ae10df82fc12ff80
I20260709 13:12:45.487432  1066 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 13:12:45.510809  1066 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 13:12:45.512202  1066 catalog_manager.cc:1540] Loading token signing keys...
I20260709 13:12:45.530033  1066 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 31065877c4af477d9b2e726230936891: Generated new TSK 0
I20260709 13:12:45.530661  1066 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 13:12:45.545141 21756 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:12:45.551083  1077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:45.551961  1078 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:45.554088 21756 server_base.cc:1061] running on GCE node
W20260709 13:12:45.554538  1080 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:45.555420 21756 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 13:12:45.555646 21756 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:12:45.555819 21756 hybrid_clock.cc:648] HybridClock initialized: now 1783602765555797 us; error 0 us; skew 500 ppm
I20260709 13:12:45.556309 21756 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:12:45.558727 21756 webserver.cc:533] Webserver started at http://127.21.63.1:42235/ using document root <none> and password file <none>
I20260709 13:12:45.559185 21756 fs_manager.cc:362] Metadata directory not provided
I20260709 13:12:45.559383 21756 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:12:45.559635 21756 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:12:45.560765 21756 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602429420501-21756-0/minicluster-data/ts-0-root/instance:
uuid: "22ec8c2439d14f139d5330c8ec45133a"
format_stamp: "Formatted at 2026-07-09 13:12:45 on dist-test-slave-c6mt"
I20260709 13:12:45.564949 21756 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 13:12:45.567997  1085 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:45.568786 21756 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.001s
I20260709 13:12:45.569119 21756 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602429420501-21756-0/minicluster-data/ts-0-root
uuid: "22ec8c2439d14f139d5330c8ec45133a"
format_stamp: "Formatted at 2026-07-09 13:12:45 on dist-test-slave-c6mt"
I20260709 13:12:45.569397 21756 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602429420501-21756-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602429420501-21756-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602429420501-21756-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:12:45.606392 21756 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:12:45.607614 21756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:12:45.609043 21756 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:12:45.611263 21756 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:12:45.611479 21756 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:45.611708 21756 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:12:45.611886 21756 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:45.652371 21756 rpc_server.cc:307] RPC server started. Bound to: 127.21.63.1:36943
I20260709 13:12:45.652468  1147 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.63.1:36943 every 8 connection(s)
I20260709 13:12:45.657640 21756 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:12:45.664589  1152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:45.666608  1153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:45.667646  1155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:45.669334 21756 server_base.cc:1061] running on GCE node
I20260709 13:12:45.670792  1148 heartbeater.cc:344] Connected to a master server at 127.21.63.62:41855
I20260709 13:12:45.670848 21756 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:12:45.671254  1148 heartbeater.cc:461] Registering TS with master...
W20260709 13:12:45.671265 21756 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:12:45.671669 21756 hybrid_clock.cc:648] HybridClock initialized: now 1783602765671649 us; error 0 us; skew 500 ppm
I20260709 13:12:45.672169  1148 heartbeater.cc:507] Master 127.21.63.62:41855 requested a full tablet report, sending...
I20260709 13:12:45.672259 21756 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:12:45.674569  1021 ts_manager.cc:194] Registered new tserver with Master: 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943)
I20260709 13:12:45.675145 21756 webserver.cc:533] Webserver started at http://127.21.63.2:33519/ using document root <none> and password file <none>
I20260709 13:12:45.675683 21756 fs_manager.cc:362] Metadata directory not provided
I20260709 13:12:45.675894 21756 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:12:45.676151 21756 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:12:45.676636  1021 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49940
I20260709 13:12:45.677448 21756 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602429420501-21756-0/minicluster-data/ts-1-root/instance:
uuid: "bbcbed9750ae416bab78287348e9e196"
format_stamp: "Formatted at 2026-07-09 13:12:45 on dist-test-slave-c6mt"
I20260709 13:12:45.682832 21756 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.002s	sys 0.003s
I20260709 13:12:45.686406  1160 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:45.687211 21756 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.001s	sys 0.000s
I20260709 13:12:45.687526 21756 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602429420501-21756-0/minicluster-data/ts-1-root
uuid: "bbcbed9750ae416bab78287348e9e196"
format_stamp: "Formatted at 2026-07-09 13:12:45 on dist-test-slave-c6mt"
I20260709 13:12:45.687855 21756 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602429420501-21756-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602429420501-21756-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602429420501-21756-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:12:45.703460 21756 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:12:45.704656 21756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:12:45.706138 21756 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:12:45.708246 21756 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:12:45.708465 21756 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:45.708750 21756 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:12:45.708937 21756 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:45.748348 21756 rpc_server.cc:307] RPC server started. Bound to: 127.21.63.2:34515
I20260709 13:12:45.748482  1222 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.63.2:34515 every 8 connection(s)
I20260709 13:12:45.753315 21756 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 13:12:45.760507  1226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:45.761689  1227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 13:12:45.763728  1229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 13:12:45.764755 21756 server_base.cc:1061] running on GCE node
I20260709 13:12:45.765808 21756 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 13:12:45.765818  1223 heartbeater.cc:344] Connected to a master server at 127.21.63.62:41855
W20260709 13:12:45.766034 21756 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 13:12:45.766456 21756 hybrid_clock.cc:648] HybridClock initialized: now 1783602765766431 us; error 0 us; skew 500 ppm
I20260709 13:12:45.766489  1223 heartbeater.cc:461] Registering TS with master...
I20260709 13:12:45.767104 21756 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 13:12:45.767455  1223 heartbeater.cc:507] Master 127.21.63.62:41855 requested a full tablet report, sending...
I20260709 13:12:45.769868 21756 webserver.cc:533] Webserver started at http://127.21.63.3:40229/ using document root <none> and password file <none>
I20260709 13:12:45.769987  1021 ts_manager.cc:194] Registered new tserver with Master: bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515)
I20260709 13:12:45.770489 21756 fs_manager.cc:362] Metadata directory not provided
I20260709 13:12:45.770763 21756 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 13:12:45.771031 21756 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 13:12:45.771767  1021 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49950
I20260709 13:12:45.772272 21756 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602429420501-21756-0/minicluster-data/ts-2-root/instance:
uuid: "efe0ab937a564199977b910104824f30"
format_stamp: "Formatted at 2026-07-09 13:12:45 on dist-test-slave-c6mt"
I20260709 13:12:45.776803 21756 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.003s	sys 0.001s
I20260709 13:12:45.780189  1234 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:45.781044 21756 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 13:12:45.781307 21756 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602429420501-21756-0/minicluster-data/ts-2-root
uuid: "efe0ab937a564199977b910104824f30"
format_stamp: "Formatted at 2026-07-09 13:12:45 on dist-test-slave-c6mt"
I20260709 13:12:45.781553 21756 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602429420501-21756-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602429420501-21756-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskxKLX8Y/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783602429420501-21756-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 13:12:45.789870 21756 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 13:12:45.790956 21756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 13:12:45.792313 21756 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 13:12:45.794534 21756 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 13:12:45.794703 21756 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:45.794994 21756 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 13:12:45.795142 21756 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 13:12:45.834753 21756 rpc_server.cc:307] RPC server started. Bound to: 127.21.63.3:33527
I20260709 13:12:45.834913  1296 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.21.63.3:33527 every 8 connection(s)
I20260709 13:12:45.848901  1297 heartbeater.cc:344] Connected to a master server at 127.21.63.62:41855
I20260709 13:12:45.849390  1297 heartbeater.cc:461] Registering TS with master...
I20260709 13:12:45.850169  1297 heartbeater.cc:507] Master 127.21.63.62:41855 requested a full tablet report, sending...
I20260709 13:12:45.852212  1021 ts_manager.cc:194] Registered new tserver with Master: efe0ab937a564199977b910104824f30 (127.21.63.3:33527)
I20260709 13:12:45.852465 21756 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014747614s
I20260709 13:12:45.854310  1021 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:49954
I20260709 13:12:45.878572  1021 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49962:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:12:45.881311  1021 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:12:45.951237  1188 tablet_service.cc:1511] Processing CreateTablet for tablet 89400744bdfc474a805d19fe866d401b (DEFAULT_TABLE table=test-workload [id=e8e53ae718c24ddda6ca186cc525903b]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:12:45.952822  1188 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89400744bdfc474a805d19fe866d401b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:45.954879  1260 tablet_service.cc:1511] Processing CreateTablet for tablet 9d35b87a150449d5b2adefeb4b241fd2 (DEFAULT_TABLE table=test-workload [id=e8e53ae718c24ddda6ca186cc525903b]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:12:45.956465  1260 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d35b87a150449d5b2adefeb4b241fd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:45.956717  1259 tablet_service.cc:1511] Processing CreateTablet for tablet 8de23e2629b2451b900a09bfe12dbf86 (DEFAULT_TABLE table=test-workload [id=e8e53ae718c24ddda6ca186cc525903b]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:12:45.957983  1187 tablet_service.cc:1511] Processing CreateTablet for tablet 8ad7ce1d2ca94ba2bc0a82302a9eee4e (DEFAULT_TABLE table=test-workload [id=e8e53ae718c24ddda6ca186cc525903b]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:12:45.959378  1187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ad7ce1d2ca94ba2bc0a82302a9eee4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:45.954974  1262 tablet_service.cc:1511] Processing CreateTablet for tablet 89400744bdfc474a805d19fe866d401b (DEFAULT_TABLE table=test-workload [id=e8e53ae718c24ddda6ca186cc525903b]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:12:45.961323  1186 tablet_service.cc:1511] Processing CreateTablet for tablet 9d35b87a150449d5b2adefeb4b241fd2 (DEFAULT_TABLE table=test-workload [id=e8e53ae718c24ddda6ca186cc525903b]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:12:45.962651  1186 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d35b87a150449d5b2adefeb4b241fd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:45.959010  1261 tablet_service.cc:1511] Processing CreateTablet for tablet 8ad7ce1d2ca94ba2bc0a82302a9eee4e (DEFAULT_TABLE table=test-workload [id=e8e53ae718c24ddda6ca186cc525903b]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:12:45.963138  1258 tablet_service.cc:1511] Processing CreateTablet for tablet fd271e857f3f41c6bcc5d54aae05f985 (DEFAULT_TABLE table=test-workload [id=e8e53ae718c24ddda6ca186cc525903b]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:12:45.967526  1185 tablet_service.cc:1511] Processing CreateTablet for tablet 8de23e2629b2451b900a09bfe12dbf86 (DEFAULT_TABLE table=test-workload [id=e8e53ae718c24ddda6ca186cc525903b]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:12:45.967689  1184 tablet_service.cc:1511] Processing CreateTablet for tablet fd271e857f3f41c6bcc5d54aae05f985 (DEFAULT_TABLE table=test-workload [id=e8e53ae718c24ddda6ca186cc525903b]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:12:45.958024  1259 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8de23e2629b2451b900a09bfe12dbf86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:45.975961  1183 tablet_service.cc:1511] Processing CreateTablet for tablet 250a7a602e924c57b99ecbd2bf900bcc (DEFAULT_TABLE table=test-workload [id=e8e53ae718c24ddda6ca186cc525903b]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:12:45.976866  1257 tablet_service.cc:1511] Processing CreateTablet for tablet 250a7a602e924c57b99ecbd2bf900bcc (DEFAULT_TABLE table=test-workload [id=e8e53ae718c24ddda6ca186cc525903b]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:12:45.978916  1182 tablet_service.cc:1511] Processing CreateTablet for tablet 57aa843159154feb87f476bac96b2dba (DEFAULT_TABLE table=test-workload [id=e8e53ae718c24ddda6ca186cc525903b]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:12:45.981237  1256 tablet_service.cc:1511] Processing CreateTablet for tablet 57aa843159154feb87f476bac96b2dba (DEFAULT_TABLE table=test-workload [id=e8e53ae718c24ddda6ca186cc525903b]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:12:45.985073  1181 tablet_service.cc:1511] Processing CreateTablet for tablet 98d5edd9b01941898accf5cf97879ab4 (DEFAULT_TABLE table=test-workload [id=e8e53ae718c24ddda6ca186cc525903b]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:12:45.990502  1255 tablet_service.cc:1511] Processing CreateTablet for tablet 98d5edd9b01941898accf5cf97879ab4 (DEFAULT_TABLE table=test-workload [id=e8e53ae718c24ddda6ca186cc525903b]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:12:45.992447  1255 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98d5edd9b01941898accf5cf97879ab4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:45.992599  1180 tablet_service.cc:1511] Processing CreateTablet for tablet af54d168945b40d7b514668ce80da735 (DEFAULT_TABLE table=test-workload [id=e8e53ae718c24ddda6ca186cc525903b]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:12:45.994841  1254 tablet_service.cc:1511] Processing CreateTablet for tablet af54d168945b40d7b514668ce80da735 (DEFAULT_TABLE table=test-workload [id=e8e53ae718c24ddda6ca186cc525903b]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:12:46.000444  1254 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af54d168945b40d7b514668ce80da735. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:46.013070  1185 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8de23e2629b2451b900a09bfe12dbf86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:46.014459  1261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ad7ce1d2ca94ba2bc0a82302a9eee4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:46.021087  1262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89400744bdfc474a805d19fe866d401b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:46.025698  1258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd271e857f3f41c6bcc5d54aae05f985. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:46.033782  1182 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57aa843159154feb87f476bac96b2dba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:46.037146  1184 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd271e857f3f41c6bcc5d54aae05f985. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:46.041121  1183 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 250a7a602e924c57b99ecbd2bf900bcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:46.044251  1256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57aa843159154feb87f476bac96b2dba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:46.045248  1181 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98d5edd9b01941898accf5cf97879ab4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:46.052389  1113 tablet_service.cc:1511] Processing CreateTablet for tablet 89400744bdfc474a805d19fe866d401b (DEFAULT_TABLE table=test-workload [id=e8e53ae718c24ddda6ca186cc525903b]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:12:46.053791  1113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 89400744bdfc474a805d19fe866d401b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:46.053653  1110 tablet_service.cc:1511] Processing CreateTablet for tablet 8de23e2629b2451b900a09bfe12dbf86 (DEFAULT_TABLE table=test-workload [id=e8e53ae718c24ddda6ca186cc525903b]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:12:46.054469  1180 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af54d168945b40d7b514668ce80da735. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:46.054965  1110 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8de23e2629b2451b900a09bfe12dbf86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:46.058374  1112 tablet_service.cc:1511] Processing CreateTablet for tablet 8ad7ce1d2ca94ba2bc0a82302a9eee4e (DEFAULT_TABLE table=test-workload [id=e8e53ae718c24ddda6ca186cc525903b]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:12:46.059712  1112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ad7ce1d2ca94ba2bc0a82302a9eee4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:46.066176  1257 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 250a7a602e924c57b99ecbd2bf900bcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:46.069108  1111 tablet_service.cc:1511] Processing CreateTablet for tablet 9d35b87a150449d5b2adefeb4b241fd2 (DEFAULT_TABLE table=test-workload [id=e8e53ae718c24ddda6ca186cc525903b]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:12:46.070521  1111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9d35b87a150449d5b2adefeb4b241fd2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:46.075055  1109 tablet_service.cc:1511] Processing CreateTablet for tablet fd271e857f3f41c6bcc5d54aae05f985 (DEFAULT_TABLE table=test-workload [id=e8e53ae718c24ddda6ca186cc525903b]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:12:46.076426  1109 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd271e857f3f41c6bcc5d54aae05f985. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:46.078125  1108 tablet_service.cc:1511] Processing CreateTablet for tablet 250a7a602e924c57b99ecbd2bf900bcc (DEFAULT_TABLE table=test-workload [id=e8e53ae718c24ddda6ca186cc525903b]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:12:46.089640  1108 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 250a7a602e924c57b99ecbd2bf900bcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:46.079981  1107 tablet_service.cc:1511] Processing CreateTablet for tablet 57aa843159154feb87f476bac96b2dba (DEFAULT_TABLE table=test-workload [id=e8e53ae718c24ddda6ca186cc525903b]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:12:46.091449  1107 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57aa843159154feb87f476bac96b2dba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:46.082190  1106 tablet_service.cc:1511] Processing CreateTablet for tablet 98d5edd9b01941898accf5cf97879ab4 (DEFAULT_TABLE table=test-workload [id=e8e53ae718c24ddda6ca186cc525903b]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:12:46.097559  1106 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 98d5edd9b01941898accf5cf97879ab4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:46.084555  1105 tablet_service.cc:1511] Processing CreateTablet for tablet af54d168945b40d7b514668ce80da735 (DEFAULT_TABLE table=test-workload [id=e8e53ae718c24ddda6ca186cc525903b]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:12:46.105595  1105 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af54d168945b40d7b514668ce80da735. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:46.123432  1312 tablet_bootstrap.cc:492] T 89400744bdfc474a805d19fe866d401b P bbcbed9750ae416bab78287348e9e196: Bootstrap starting.
I20260709 13:12:46.121264  1313 tablet_bootstrap.cc:492] T 9d35b87a150449d5b2adefeb4b241fd2 P efe0ab937a564199977b910104824f30: Bootstrap starting.
I20260709 13:12:46.171919  1313 tablet_bootstrap.cc:654] T 9d35b87a150449d5b2adefeb4b241fd2 P efe0ab937a564199977b910104824f30: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:46.185463  1314 tablet_bootstrap.cc:492] T 9d35b87a150449d5b2adefeb4b241fd2 P 22ec8c2439d14f139d5330c8ec45133a: Bootstrap starting.
I20260709 13:12:46.188797  1313 tablet_bootstrap.cc:492] T 9d35b87a150449d5b2adefeb4b241fd2 P efe0ab937a564199977b910104824f30: No bootstrap required, opened a new log
I20260709 13:12:46.189543  1313 ts_tablet_manager.cc:1403] T 9d35b87a150449d5b2adefeb4b241fd2 P efe0ab937a564199977b910104824f30: Time spent bootstrapping tablet: real 0.124s	user 0.003s	sys 0.014s
I20260709 13:12:46.190467  1312 tablet_bootstrap.cc:654] T 89400744bdfc474a805d19fe866d401b P bbcbed9750ae416bab78287348e9e196: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:46.193354  1314 tablet_bootstrap.cc:654] T 9d35b87a150449d5b2adefeb4b241fd2 P 22ec8c2439d14f139d5330c8ec45133a: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:46.192919  1313 raft_consensus.cc:359] T 9d35b87a150449d5b2adefeb4b241fd2 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.193806  1313 raft_consensus.cc:385] T 9d35b87a150449d5b2adefeb4b241fd2 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:46.194144  1313 raft_consensus.cc:740] T 9d35b87a150449d5b2adefeb4b241fd2 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe0ab937a564199977b910104824f30, State: Initialized, Role: FOLLOWER
I20260709 13:12:46.195416  1312 tablet_bootstrap.cc:492] T 89400744bdfc474a805d19fe866d401b P bbcbed9750ae416bab78287348e9e196: No bootstrap required, opened a new log
I20260709 13:12:46.195112  1313 consensus_queue.cc:260] T 9d35b87a150449d5b2adefeb4b241fd2 P efe0ab937a564199977b910104824f30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.195917  1312 ts_tablet_manager.cc:1403] T 89400744bdfc474a805d19fe866d401b P bbcbed9750ae416bab78287348e9e196: Time spent bootstrapping tablet: real 0.073s	user 0.011s	sys 0.000s
I20260709 13:12:46.197579  1297 heartbeater.cc:499] Master 127.21.63.62:41855 was elected leader, sending a full tablet report...
I20260709 13:12:46.198118  1313 ts_tablet_manager.cc:1434] T 9d35b87a150449d5b2adefeb4b241fd2 P efe0ab937a564199977b910104824f30: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260709 13:12:46.199069  1314 tablet_bootstrap.cc:492] T 9d35b87a150449d5b2adefeb4b241fd2 P 22ec8c2439d14f139d5330c8ec45133a: No bootstrap required, opened a new log
I20260709 13:12:46.199137  1313 tablet_bootstrap.cc:492] T af54d168945b40d7b514668ce80da735 P efe0ab937a564199977b910104824f30: Bootstrap starting.
I20260709 13:12:46.198953  1312 raft_consensus.cc:359] T 89400744bdfc474a805d19fe866d401b P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } } peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } }
I20260709 13:12:46.199848  1312 raft_consensus.cc:385] T 89400744bdfc474a805d19fe866d401b P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:46.199936  1314 ts_tablet_manager.cc:1403] T 9d35b87a150449d5b2adefeb4b241fd2 P 22ec8c2439d14f139d5330c8ec45133a: Time spent bootstrapping tablet: real 0.015s	user 0.005s	sys 0.008s
I20260709 13:12:46.200184  1312 raft_consensus.cc:740] T 89400744bdfc474a805d19fe866d401b P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbcbed9750ae416bab78287348e9e196, State: Initialized, Role: FOLLOWER
I20260709 13:12:46.201145  1312 consensus_queue.cc:260] T 89400744bdfc474a805d19fe866d401b P bbcbed9750ae416bab78287348e9e196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } } peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } }
I20260709 13:12:46.203536  1314 raft_consensus.cc:359] T 9d35b87a150449d5b2adefeb4b241fd2 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.204627  1314 raft_consensus.cc:385] T 9d35b87a150449d5b2adefeb4b241fd2 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:46.205107  1314 raft_consensus.cc:740] T 9d35b87a150449d5b2adefeb4b241fd2 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22ec8c2439d14f139d5330c8ec45133a, State: Initialized, Role: FOLLOWER
I20260709 13:12:46.205747  1313 tablet_bootstrap.cc:654] T af54d168945b40d7b514668ce80da735 P efe0ab937a564199977b910104824f30: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:46.206183  1314 consensus_queue.cc:260] T 9d35b87a150449d5b2adefeb4b241fd2 P 22ec8c2439d14f139d5330c8ec45133a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.208897  1223 heartbeater.cc:499] Master 127.21.63.62:41855 was elected leader, sending a full tablet report...
I20260709 13:12:46.209414  1312 ts_tablet_manager.cc:1434] T 89400744bdfc474a805d19fe866d401b P bbcbed9750ae416bab78287348e9e196: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.008s
I20260709 13:12:46.210387  1312 tablet_bootstrap.cc:492] T 9d35b87a150449d5b2adefeb4b241fd2 P bbcbed9750ae416bab78287348e9e196: Bootstrap starting.
I20260709 13:12:46.216012  1313 tablet_bootstrap.cc:492] T af54d168945b40d7b514668ce80da735 P efe0ab937a564199977b910104824f30: No bootstrap required, opened a new log
I20260709 13:12:46.216594  1313 ts_tablet_manager.cc:1403] T af54d168945b40d7b514668ce80da735 P efe0ab937a564199977b910104824f30: Time spent bootstrapping tablet: real 0.018s	user 0.004s	sys 0.009s
I20260709 13:12:46.219558  1313 raft_consensus.cc:359] T af54d168945b40d7b514668ce80da735 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.220276  1313 raft_consensus.cc:385] T af54d168945b40d7b514668ce80da735 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:46.220567  1313 raft_consensus.cc:740] T af54d168945b40d7b514668ce80da735 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe0ab937a564199977b910104824f30, State: Initialized, Role: FOLLOWER
I20260709 13:12:46.221088  1312 tablet_bootstrap.cc:654] T 9d35b87a150449d5b2adefeb4b241fd2 P bbcbed9750ae416bab78287348e9e196: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:46.221361  1313 consensus_queue.cc:260] T af54d168945b40d7b514668ce80da735 P efe0ab937a564199977b910104824f30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.224031  1313 ts_tablet_manager.cc:1434] T af54d168945b40d7b514668ce80da735 P efe0ab937a564199977b910104824f30: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:12:46.225063  1313 tablet_bootstrap.cc:492] T 98d5edd9b01941898accf5cf97879ab4 P efe0ab937a564199977b910104824f30: Bootstrap starting.
I20260709 13:12:46.225004  1148 heartbeater.cc:499] Master 127.21.63.62:41855 was elected leader, sending a full tablet report...
I20260709 13:12:46.225638  1314 ts_tablet_manager.cc:1434] T 9d35b87a150449d5b2adefeb4b241fd2 P 22ec8c2439d14f139d5330c8ec45133a: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.010s
I20260709 13:12:46.226723  1314 tablet_bootstrap.cc:492] T af54d168945b40d7b514668ce80da735 P 22ec8c2439d14f139d5330c8ec45133a: Bootstrap starting.
I20260709 13:12:46.231489  1313 tablet_bootstrap.cc:654] T 98d5edd9b01941898accf5cf97879ab4 P efe0ab937a564199977b910104824f30: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:46.233237  1314 tablet_bootstrap.cc:654] T af54d168945b40d7b514668ce80da735 P 22ec8c2439d14f139d5330c8ec45133a: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:46.243635  1313 tablet_bootstrap.cc:492] T 98d5edd9b01941898accf5cf97879ab4 P efe0ab937a564199977b910104824f30: No bootstrap required, opened a new log
I20260709 13:12:46.244170  1313 ts_tablet_manager.cc:1403] T 98d5edd9b01941898accf5cf97879ab4 P efe0ab937a564199977b910104824f30: Time spent bootstrapping tablet: real 0.019s	user 0.006s	sys 0.012s
I20260709 13:12:46.245126  1312 tablet_bootstrap.cc:492] T 9d35b87a150449d5b2adefeb4b241fd2 P bbcbed9750ae416bab78287348e9e196: No bootstrap required, opened a new log
I20260709 13:12:46.245330  1314 tablet_bootstrap.cc:492] T af54d168945b40d7b514668ce80da735 P 22ec8c2439d14f139d5330c8ec45133a: No bootstrap required, opened a new log
I20260709 13:12:46.245661  1312 ts_tablet_manager.cc:1403] T 9d35b87a150449d5b2adefeb4b241fd2 P bbcbed9750ae416bab78287348e9e196: Time spent bootstrapping tablet: real 0.035s	user 0.009s	sys 0.009s
I20260709 13:12:46.246076  1314 ts_tablet_manager.cc:1403] T af54d168945b40d7b514668ce80da735 P 22ec8c2439d14f139d5330c8ec45133a: Time spent bootstrapping tablet: real 0.020s	user 0.019s	sys 0.000s
I20260709 13:12:46.247076  1313 raft_consensus.cc:359] T 98d5edd9b01941898accf5cf97879ab4 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.247831  1313 raft_consensus.cc:385] T 98d5edd9b01941898accf5cf97879ab4 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:46.248116  1313 raft_consensus.cc:740] T 98d5edd9b01941898accf5cf97879ab4 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe0ab937a564199977b910104824f30, State: Initialized, Role: FOLLOWER
I20260709 13:12:46.248715  1312 raft_consensus.cc:359] T 9d35b87a150449d5b2adefeb4b241fd2 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.249171  1313 consensus_queue.cc:260] T 98d5edd9b01941898accf5cf97879ab4 P efe0ab937a564199977b910104824f30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.249611  1312 raft_consensus.cc:385] T 9d35b87a150449d5b2adefeb4b241fd2 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:46.249387  1314 raft_consensus.cc:359] T af54d168945b40d7b514668ce80da735 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.250413  1314 raft_consensus.cc:385] T af54d168945b40d7b514668ce80da735 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:46.250964  1314 raft_consensus.cc:740] T af54d168945b40d7b514668ce80da735 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22ec8c2439d14f139d5330c8ec45133a, State: Initialized, Role: FOLLOWER
I20260709 13:12:46.251487  1313 ts_tablet_manager.cc:1434] T 98d5edd9b01941898accf5cf97879ab4 P efe0ab937a564199977b910104824f30: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.001s
I20260709 13:12:46.250099  1312 raft_consensus.cc:740] T 9d35b87a150449d5b2adefeb4b241fd2 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbcbed9750ae416bab78287348e9e196, State: Initialized, Role: FOLLOWER
I20260709 13:12:46.252516  1312 consensus_queue.cc:260] T 9d35b87a150449d5b2adefeb4b241fd2 P bbcbed9750ae416bab78287348e9e196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.253504  1313 tablet_bootstrap.cc:492] T 8de23e2629b2451b900a09bfe12dbf86 P efe0ab937a564199977b910104824f30: Bootstrap starting.
I20260709 13:12:46.251776  1314 consensus_queue.cc:260] T af54d168945b40d7b514668ce80da735 P 22ec8c2439d14f139d5330c8ec45133a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.255164  1312 ts_tablet_manager.cc:1434] T 9d35b87a150449d5b2adefeb4b241fd2 P bbcbed9750ae416bab78287348e9e196: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 13:12:46.255419  1314 ts_tablet_manager.cc:1434] T af54d168945b40d7b514668ce80da735 P 22ec8c2439d14f139d5330c8ec45133a: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 13:12:46.256319  1312 tablet_bootstrap.cc:492] T 8de23e2629b2451b900a09bfe12dbf86 P bbcbed9750ae416bab78287348e9e196: Bootstrap starting.
I20260709 13:12:46.256448  1314 tablet_bootstrap.cc:492] T 98d5edd9b01941898accf5cf97879ab4 P 22ec8c2439d14f139d5330c8ec45133a: Bootstrap starting.
I20260709 13:12:46.260962  1313 tablet_bootstrap.cc:654] T 8de23e2629b2451b900a09bfe12dbf86 P efe0ab937a564199977b910104824f30: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:46.263661  1314 tablet_bootstrap.cc:654] T 98d5edd9b01941898accf5cf97879ab4 P 22ec8c2439d14f139d5330c8ec45133a: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:46.266652  1312 tablet_bootstrap.cc:654] T 8de23e2629b2451b900a09bfe12dbf86 P bbcbed9750ae416bab78287348e9e196: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:46.278918  1313 tablet_bootstrap.cc:492] T 8de23e2629b2451b900a09bfe12dbf86 P efe0ab937a564199977b910104824f30: No bootstrap required, opened a new log
I20260709 13:12:46.279412  1314 tablet_bootstrap.cc:492] T 98d5edd9b01941898accf5cf97879ab4 P 22ec8c2439d14f139d5330c8ec45133a: No bootstrap required, opened a new log
I20260709 13:12:46.279912  1313 ts_tablet_manager.cc:1403] T 8de23e2629b2451b900a09bfe12dbf86 P efe0ab937a564199977b910104824f30: Time spent bootstrapping tablet: real 0.027s	user 0.020s	sys 0.005s
I20260709 13:12:46.279999  1314 ts_tablet_manager.cc:1403] T 98d5edd9b01941898accf5cf97879ab4 P 22ec8c2439d14f139d5330c8ec45133a: Time spent bootstrapping tablet: real 0.024s	user 0.011s	sys 0.005s
I20260709 13:12:46.281464  1312 tablet_bootstrap.cc:492] T 8de23e2629b2451b900a09bfe12dbf86 P bbcbed9750ae416bab78287348e9e196: No bootstrap required, opened a new log
I20260709 13:12:46.282051  1312 ts_tablet_manager.cc:1403] T 8de23e2629b2451b900a09bfe12dbf86 P bbcbed9750ae416bab78287348e9e196: Time spent bootstrapping tablet: real 0.026s	user 0.016s	sys 0.002s
I20260709 13:12:46.282902  1314 raft_consensus.cc:359] T 98d5edd9b01941898accf5cf97879ab4 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.283226  1313 raft_consensus.cc:359] T 8de23e2629b2451b900a09bfe12dbf86 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.283802  1314 raft_consensus.cc:385] T 98d5edd9b01941898accf5cf97879ab4 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:46.283918  1313 raft_consensus.cc:385] T 8de23e2629b2451b900a09bfe12dbf86 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:46.284180  1314 raft_consensus.cc:740] T 98d5edd9b01941898accf5cf97879ab4 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22ec8c2439d14f139d5330c8ec45133a, State: Initialized, Role: FOLLOWER
I20260709 13:12:46.284299  1313 raft_consensus.cc:740] T 8de23e2629b2451b900a09bfe12dbf86 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe0ab937a564199977b910104824f30, State: Initialized, Role: FOLLOWER
I20260709 13:12:46.285060  1314 consensus_queue.cc:260] T 98d5edd9b01941898accf5cf97879ab4 P 22ec8c2439d14f139d5330c8ec45133a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.285198  1313 consensus_queue.cc:260] T 8de23e2629b2451b900a09bfe12dbf86 P efe0ab937a564199977b910104824f30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.285522  1312 raft_consensus.cc:359] T 8de23e2629b2451b900a09bfe12dbf86 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.286450  1312 raft_consensus.cc:385] T 8de23e2629b2451b900a09bfe12dbf86 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:46.286803  1312 raft_consensus.cc:740] T 8de23e2629b2451b900a09bfe12dbf86 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbcbed9750ae416bab78287348e9e196, State: Initialized, Role: FOLLOWER
I20260709 13:12:46.287328  1314 ts_tablet_manager.cc:1434] T 98d5edd9b01941898accf5cf97879ab4 P 22ec8c2439d14f139d5330c8ec45133a: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 13:12:46.287592  1312 consensus_queue.cc:260] T 8de23e2629b2451b900a09bfe12dbf86 P bbcbed9750ae416bab78287348e9e196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.288610  1314 tablet_bootstrap.cc:492] T 57aa843159154feb87f476bac96b2dba P 22ec8c2439d14f139d5330c8ec45133a: Bootstrap starting.
I20260709 13:12:46.289830  1313 ts_tablet_manager.cc:1434] T 8de23e2629b2451b900a09bfe12dbf86 P efe0ab937a564199977b910104824f30: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.002s
I20260709 13:12:46.290860  1313 tablet_bootstrap.cc:492] T 250a7a602e924c57b99ecbd2bf900bcc P efe0ab937a564199977b910104824f30: Bootstrap starting.
I20260709 13:12:46.291936  1312 ts_tablet_manager.cc:1434] T 8de23e2629b2451b900a09bfe12dbf86 P bbcbed9750ae416bab78287348e9e196: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.001s
I20260709 13:12:46.293080  1312 tablet_bootstrap.cc:492] T fd271e857f3f41c6bcc5d54aae05f985 P bbcbed9750ae416bab78287348e9e196: Bootstrap starting.
I20260709 13:12:46.295609  1314 tablet_bootstrap.cc:654] T 57aa843159154feb87f476bac96b2dba P 22ec8c2439d14f139d5330c8ec45133a: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:46.297365  1313 tablet_bootstrap.cc:654] T 250a7a602e924c57b99ecbd2bf900bcc P efe0ab937a564199977b910104824f30: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:46.299365  1312 tablet_bootstrap.cc:654] T fd271e857f3f41c6bcc5d54aae05f985 P bbcbed9750ae416bab78287348e9e196: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:46.307406  1312 tablet_bootstrap.cc:492] T fd271e857f3f41c6bcc5d54aae05f985 P bbcbed9750ae416bab78287348e9e196: No bootstrap required, opened a new log
I20260709 13:12:46.307879  1312 ts_tablet_manager.cc:1403] T fd271e857f3f41c6bcc5d54aae05f985 P bbcbed9750ae416bab78287348e9e196: Time spent bootstrapping tablet: real 0.015s	user 0.011s	sys 0.000s
I20260709 13:12:46.310340  1314 tablet_bootstrap.cc:492] T 57aa843159154feb87f476bac96b2dba P 22ec8c2439d14f139d5330c8ec45133a: No bootstrap required, opened a new log
I20260709 13:12:46.311947  1314 ts_tablet_manager.cc:1403] T 57aa843159154feb87f476bac96b2dba P 22ec8c2439d14f139d5330c8ec45133a: Time spent bootstrapping tablet: real 0.024s	user 0.007s	sys 0.012s
I20260709 13:12:46.310567  1312 raft_consensus.cc:359] T fd271e857f3f41c6bcc5d54aae05f985 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.313050  1312 raft_consensus.cc:385] T fd271e857f3f41c6bcc5d54aae05f985 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:46.313119  1313 tablet_bootstrap.cc:492] T 250a7a602e924c57b99ecbd2bf900bcc P efe0ab937a564199977b910104824f30: No bootstrap required, opened a new log
I20260709 13:12:46.313481  1312 raft_consensus.cc:740] T fd271e857f3f41c6bcc5d54aae05f985 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbcbed9750ae416bab78287348e9e196, State: Initialized, Role: FOLLOWER
I20260709 13:12:46.313727  1313 ts_tablet_manager.cc:1403] T 250a7a602e924c57b99ecbd2bf900bcc P efe0ab937a564199977b910104824f30: Time spent bootstrapping tablet: real 0.023s	user 0.022s	sys 0.000s
I20260709 13:12:46.314452  1312 consensus_queue.cc:260] T fd271e857f3f41c6bcc5d54aae05f985 P bbcbed9750ae416bab78287348e9e196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.315152  1314 raft_consensus.cc:359] T 57aa843159154feb87f476bac96b2dba P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.316071  1314 raft_consensus.cc:385] T 57aa843159154feb87f476bac96b2dba P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:46.316425  1314 raft_consensus.cc:740] T 57aa843159154feb87f476bac96b2dba P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22ec8c2439d14f139d5330c8ec45133a, State: Initialized, Role: FOLLOWER
I20260709 13:12:46.316819  1313 raft_consensus.cc:359] T 250a7a602e924c57b99ecbd2bf900bcc P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.317559  1313 raft_consensus.cc:385] T 250a7a602e924c57b99ecbd2bf900bcc P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:46.317245  1314 consensus_queue.cc:260] T 57aa843159154feb87f476bac96b2dba P 22ec8c2439d14f139d5330c8ec45133a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.317885  1313 raft_consensus.cc:740] T 250a7a602e924c57b99ecbd2bf900bcc P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe0ab937a564199977b910104824f30, State: Initialized, Role: FOLLOWER
I20260709 13:12:46.319799  1314 ts_tablet_manager.cc:1434] T 57aa843159154feb87f476bac96b2dba P 22ec8c2439d14f139d5330c8ec45133a: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 13:12:46.318745  1313 consensus_queue.cc:260] T 250a7a602e924c57b99ecbd2bf900bcc P efe0ab937a564199977b910104824f30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.320981  1314 tablet_bootstrap.cc:492] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P 22ec8c2439d14f139d5330c8ec45133a: Bootstrap starting.
I20260709 13:12:46.322332  1313 ts_tablet_manager.cc:1434] T 250a7a602e924c57b99ecbd2bf900bcc P efe0ab937a564199977b910104824f30: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:12:46.322494  1312 ts_tablet_manager.cc:1434] T fd271e857f3f41c6bcc5d54aae05f985 P bbcbed9750ae416bab78287348e9e196: Time spent starting tablet: real 0.014s	user 0.002s	sys 0.005s
I20260709 13:12:46.323412  1313 tablet_bootstrap.cc:492] T 89400744bdfc474a805d19fe866d401b P efe0ab937a564199977b910104824f30: Bootstrap starting.
I20260709 13:12:46.323585  1312 tablet_bootstrap.cc:492] T 250a7a602e924c57b99ecbd2bf900bcc P bbcbed9750ae416bab78287348e9e196: Bootstrap starting.
I20260709 13:12:46.327674  1314 tablet_bootstrap.cc:654] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P 22ec8c2439d14f139d5330c8ec45133a: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:46.330093  1312 tablet_bootstrap.cc:654] T 250a7a602e924c57b99ecbd2bf900bcc P bbcbed9750ae416bab78287348e9e196: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:46.330386  1313 tablet_bootstrap.cc:654] T 89400744bdfc474a805d19fe866d401b P efe0ab937a564199977b910104824f30: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:46.341675  1312 tablet_bootstrap.cc:492] T 250a7a602e924c57b99ecbd2bf900bcc P bbcbed9750ae416bab78287348e9e196: No bootstrap required, opened a new log
I20260709 13:12:46.342167  1312 ts_tablet_manager.cc:1403] T 250a7a602e924c57b99ecbd2bf900bcc P bbcbed9750ae416bab78287348e9e196: Time spent bootstrapping tablet: real 0.019s	user 0.016s	sys 0.000s
I20260709 13:12:46.345098  1312 raft_consensus.cc:359] T 250a7a602e924c57b99ecbd2bf900bcc P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.345844  1312 raft_consensus.cc:385] T 250a7a602e924c57b99ecbd2bf900bcc P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:46.346143  1312 raft_consensus.cc:740] T 250a7a602e924c57b99ecbd2bf900bcc P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbcbed9750ae416bab78287348e9e196, State: Initialized, Role: FOLLOWER
I20260709 13:12:46.348249  1312 consensus_queue.cc:260] T 250a7a602e924c57b99ecbd2bf900bcc P bbcbed9750ae416bab78287348e9e196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.349467  1318 raft_consensus.cc:493] T 8de23e2629b2451b900a09bfe12dbf86 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:46.350096  1313 tablet_bootstrap.cc:492] T 89400744bdfc474a805d19fe866d401b P efe0ab937a564199977b910104824f30: No bootstrap required, opened a new log
I20260709 13:12:46.349998  1318 raft_consensus.cc:515] T 8de23e2629b2451b900a09bfe12dbf86 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.350807  1313 ts_tablet_manager.cc:1403] T 89400744bdfc474a805d19fe866d401b P efe0ab937a564199977b910104824f30: Time spent bootstrapping tablet: real 0.028s	user 0.010s	sys 0.015s
I20260709 13:12:46.353518  1312 ts_tablet_manager.cc:1434] T 250a7a602e924c57b99ecbd2bf900bcc P bbcbed9750ae416bab78287348e9e196: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.001s
I20260709 13:12:46.354620  1312 tablet_bootstrap.cc:492] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P bbcbed9750ae416bab78287348e9e196: Bootstrap starting.
I20260709 13:12:46.354874  1313 raft_consensus.cc:359] T 89400744bdfc474a805d19fe866d401b P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } } peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } }
I20260709 13:12:46.356014  1313 raft_consensus.cc:385] T 89400744bdfc474a805d19fe866d401b P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:46.356382  1313 raft_consensus.cc:740] T 89400744bdfc474a805d19fe866d401b P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe0ab937a564199977b910104824f30, State: Initialized, Role: FOLLOWER
I20260709 13:12:46.357199  1321 raft_consensus.cc:493] T 98d5edd9b01941898accf5cf97879ab4 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:46.357378  1313 consensus_queue.cc:260] T 89400744bdfc474a805d19fe866d401b P efe0ab937a564199977b910104824f30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } } peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } }
I20260709 13:12:46.357897  1321 raft_consensus.cc:515] T 98d5edd9b01941898accf5cf97879ab4 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.363137  1312 tablet_bootstrap.cc:654] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P bbcbed9750ae416bab78287348e9e196: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:46.373205  1318 leader_election.cc:290] T 8de23e2629b2451b900a09bfe12dbf86 P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943), bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515)
I20260709 13:12:46.364202  1321 leader_election.cc:290] T 98d5edd9b01941898accf5cf97879ab4 P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515), efe0ab937a564199977b910104824f30 (127.21.63.3:33527)
I20260709 13:12:46.415619  1198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8de23e2629b2451b900a09bfe12dbf86" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbcbed9750ae416bab78287348e9e196" is_pre_election: true
I20260709 13:12:46.416064  1321 raft_consensus.cc:493] T af54d168945b40d7b514668ce80da735 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:46.416049  1197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d5edd9b01941898accf5cf97879ab4" candidate_uuid: "22ec8c2439d14f139d5330c8ec45133a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbcbed9750ae416bab78287348e9e196" is_pre_election: true
I20260709 13:12:46.416390  1198 raft_consensus.cc:2468] T 8de23e2629b2451b900a09bfe12dbf86 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efe0ab937a564199977b910104824f30 in term 0.
I20260709 13:12:46.417084  1321 raft_consensus.cc:515] T af54d168945b40d7b514668ce80da735 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.418493  1237 leader_election.cc:304] T 8de23e2629b2451b900a09bfe12dbf86 P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bbcbed9750ae416bab78287348e9e196, efe0ab937a564199977b910104824f30; no voters: 
I20260709 13:12:46.419592  1321 leader_election.cc:290] T af54d168945b40d7b514668ce80da735 P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515), efe0ab937a564199977b910104824f30 (127.21.63.3:33527)
I20260709 13:12:46.420046  1318 raft_consensus.cc:2804] T 8de23e2629b2451b900a09bfe12dbf86 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:46.420483  1318 raft_consensus.cc:493] T 8de23e2629b2451b900a09bfe12dbf86 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:46.420881  1318 raft_consensus.cc:3060] T 8de23e2629b2451b900a09bfe12dbf86 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:12:46.424266  1088 leader_election.cc:343] T 98d5edd9b01941898accf5cf97879ab4 P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:46.426265  1197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af54d168945b40d7b514668ce80da735" candidate_uuid: "22ec8c2439d14f139d5330c8ec45133a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbcbed9750ae416bab78287348e9e196" is_pre_election: true
W20260709 13:12:46.427718  1088 leader_election.cc:343] T af54d168945b40d7b514668ce80da735 P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:46.430056  1318 raft_consensus.cc:515] T 8de23e2629b2451b900a09bfe12dbf86 P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.433476  1197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8de23e2629b2451b900a09bfe12dbf86" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbcbed9750ae416bab78287348e9e196"
I20260709 13:12:46.434303  1197 raft_consensus.cc:3060] T 8de23e2629b2451b900a09bfe12dbf86 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:46.442529  1197 raft_consensus.cc:2468] T 8de23e2629b2451b900a09bfe12dbf86 P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe0ab937a564199977b910104824f30 in term 1.
I20260709 13:12:46.443692  1341 raft_consensus.cc:493] T af54d168945b40d7b514668ce80da735 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:46.443934  1237 leader_election.cc:304] T 8de23e2629b2451b900a09bfe12dbf86 P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bbcbed9750ae416bab78287348e9e196, efe0ab937a564199977b910104824f30; no voters: 
I20260709 13:12:46.445851  1312 tablet_bootstrap.cc:492] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P bbcbed9750ae416bab78287348e9e196: No bootstrap required, opened a new log
I20260709 13:12:46.446446  1123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8de23e2629b2451b900a09bfe12dbf86" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22ec8c2439d14f139d5330c8ec45133a" is_pre_election: true
I20260709 13:12:46.449312  1318 leader_election.cc:290] T 8de23e2629b2451b900a09bfe12dbf86 P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 election: Requested vote from peers 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943), bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515)
I20260709 13:12:46.449218  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8de23e2629b2451b900a09bfe12dbf86" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22ec8c2439d14f139d5330c8ec45133a"
I20260709 13:12:46.449278  1312 ts_tablet_manager.cc:1403] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P bbcbed9750ae416bab78287348e9e196: Time spent bootstrapping tablet: real 0.095s	user 0.019s	sys 0.028s
W20260709 13:12:46.450359  1235 leader_election.cc:343] T 8de23e2629b2451b900a09bfe12dbf86 P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:46.451653  1318 raft_consensus.cc:493] T 250a7a602e924c57b99ecbd2bf900bcc P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:46.452160  1318 raft_consensus.cc:515] T 250a7a602e924c57b99ecbd2bf900bcc P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
W20260709 13:12:46.453631  1235 leader_election.cc:343] T 8de23e2629b2451b900a09bfe12dbf86 P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:46.455554  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250a7a602e924c57b99ecbd2bf900bcc" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22ec8c2439d14f139d5330c8ec45133a" is_pre_election: true
W20260709 13:12:46.457835  1235 leader_election.cc:343] T 250a7a602e924c57b99ecbd2bf900bcc P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:46.454429  1318 leader_election.cc:290] T 250a7a602e924c57b99ecbd2bf900bcc P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943), bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515)
I20260709 13:12:46.458984  1318 raft_consensus.cc:2804] T 8de23e2629b2451b900a09bfe12dbf86 P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:46.459412  1312 raft_consensus.cc:359] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.460834  1312 raft_consensus.cc:385] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:46.461277  1312 raft_consensus.cc:740] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbcbed9750ae416bab78287348e9e196, State: Initialized, Role: FOLLOWER
I20260709 13:12:46.444319  1341 raft_consensus.cc:515] T af54d168945b40d7b514668ce80da735 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.462551  1197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250a7a602e924c57b99ecbd2bf900bcc" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbcbed9750ae416bab78287348e9e196" is_pre_election: true
I20260709 13:12:46.463461  1197 raft_consensus.cc:2468] T 250a7a602e924c57b99ecbd2bf900bcc P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efe0ab937a564199977b910104824f30 in term 0.
I20260709 13:12:46.462615  1312 consensus_queue.cc:260] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P bbcbed9750ae416bab78287348e9e196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.463805  1341 leader_election.cc:290] T af54d168945b40d7b514668ce80da735 P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943), bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515)
I20260709 13:12:46.465669  1198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af54d168945b40d7b514668ce80da735" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbcbed9750ae416bab78287348e9e196" is_pre_election: true
I20260709 13:12:46.466797  1312 ts_tablet_manager.cc:1434] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P bbcbed9750ae416bab78287348e9e196: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.003s
I20260709 13:12:46.467343  1237 leader_election.cc:304] T 250a7a602e924c57b99ecbd2bf900bcc P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bbcbed9750ae416bab78287348e9e196, efe0ab937a564199977b910104824f30; no voters: 22ec8c2439d14f139d5330c8ec45133a
W20260709 13:12:46.468740  1237 leader_election.cc:343] T af54d168945b40d7b514668ce80da735 P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:46.459501  1318 raft_consensus.cc:697] T 8de23e2629b2451b900a09bfe12dbf86 P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Becoming Leader. State: Replica: efe0ab937a564199977b910104824f30, State: Running, Role: LEADER
W20260709 13:12:46.470712  1069 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:12:46.468377  1312 tablet_bootstrap.cc:492] T 57aa843159154feb87f476bac96b2dba P bbcbed9750ae416bab78287348e9e196: Bootstrap starting.
I20260709 13:12:46.471321  1342 raft_consensus.cc:2804] T 250a7a602e924c57b99ecbd2bf900bcc P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:46.472033  1342 raft_consensus.cc:493] T 250a7a602e924c57b99ecbd2bf900bcc P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:46.472405  1342 raft_consensus.cc:3060] T 250a7a602e924c57b99ecbd2bf900bcc P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:46.472601  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af54d168945b40d7b514668ce80da735" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22ec8c2439d14f139d5330c8ec45133a" is_pre_election: true
I20260709 13:12:46.473366  1122 raft_consensus.cc:2468] T af54d168945b40d7b514668ce80da735 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efe0ab937a564199977b910104824f30 in term 0.
I20260709 13:12:46.476086  1272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d5edd9b01941898accf5cf97879ab4" candidate_uuid: "22ec8c2439d14f139d5330c8ec45133a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe0ab937a564199977b910104824f30" is_pre_election: true
I20260709 13:12:46.475871  1318 consensus_queue.cc:237] T 8de23e2629b2451b900a09bfe12dbf86 P efe0ab937a564199977b910104824f30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.476837  1272 raft_consensus.cc:2468] T 98d5edd9b01941898accf5cf97879ab4 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22ec8c2439d14f139d5330c8ec45133a in term 0.
I20260709 13:12:46.476936  1314 tablet_bootstrap.cc:492] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P 22ec8c2439d14f139d5330c8ec45133a: No bootstrap required, opened a new log
I20260709 13:12:46.477814  1314 ts_tablet_manager.cc:1403] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P 22ec8c2439d14f139d5330c8ec45133a: Time spent bootstrapping tablet: real 0.157s	user 0.049s	sys 0.035s
I20260709 13:12:46.478159  1319 raft_consensus.cc:493] T fd271e857f3f41c6bcc5d54aae05f985 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:46.478796  1319 raft_consensus.cc:515] T fd271e857f3f41c6bcc5d54aae05f985 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.481851  1314 raft_consensus.cc:359] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.482862  1314 raft_consensus.cc:385] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:46.483224  1314 raft_consensus.cc:740] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22ec8c2439d14f139d5330c8ec45133a, State: Initialized, Role: FOLLOWER
I20260709 13:12:46.484454  1314 consensus_queue.cc:260] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P 22ec8c2439d14f139d5330c8ec45133a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.486683  1319 leader_election.cc:290] T fd271e857f3f41c6bcc5d54aae05f985 P bbcbed9750ae416bab78287348e9e196 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943), efe0ab937a564199977b910104824f30 (127.21.63.3:33527)
I20260709 13:12:46.494328  1321 raft_consensus.cc:493] T 9d35b87a150449d5b2adefeb4b241fd2 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:46.494819  1321 raft_consensus.cc:515] T 9d35b87a150449d5b2adefeb4b241fd2 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.496317  1272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af54d168945b40d7b514668ce80da735" candidate_uuid: "22ec8c2439d14f139d5330c8ec45133a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe0ab937a564199977b910104824f30" is_pre_election: true
I20260709 13:12:46.497069  1272 raft_consensus.cc:2468] T af54d168945b40d7b514668ce80da735 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22ec8c2439d14f139d5330c8ec45133a in term 0.
I20260709 13:12:46.498554  1235 leader_election.cc:304] T af54d168945b40d7b514668ce80da735 P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 22ec8c2439d14f139d5330c8ec45133a, efe0ab937a564199977b910104824f30; no voters: bbcbed9750ae416bab78287348e9e196
I20260709 13:12:46.499524  1318 raft_consensus.cc:2804] T af54d168945b40d7b514668ce80da735 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:46.499994  1318 raft_consensus.cc:493] T af54d168945b40d7b514668ce80da735 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:46.500458  1318 raft_consensus.cc:3060] T af54d168945b40d7b514668ce80da735 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:46.502997  1313 ts_tablet_manager.cc:1434] T 89400744bdfc474a805d19fe866d401b P efe0ab937a564199977b910104824f30: Time spent starting tablet: real 0.152s	user 0.038s	sys 0.102s
I20260709 13:12:46.490574  1312 tablet_bootstrap.cc:654] T 57aa843159154feb87f476bac96b2dba P bbcbed9750ae416bab78287348e9e196: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:46.504132  1313 tablet_bootstrap.cc:492] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P efe0ab937a564199977b910104824f30: Bootstrap starting.
I20260709 13:12:46.507057  1021 catalog_manager.cc:5697] T 8de23e2629b2451b900a09bfe12dbf86 P efe0ab937a564199977b910104824f30 reported cstate change: term changed from 0 to 1, leader changed from <none> to efe0ab937a564199977b910104824f30 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "efe0ab937a564199977b910104824f30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:46.508970  1319 raft_consensus.cc:493] T 89400744bdfc474a805d19fe866d401b P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:46.509390  1321 leader_election.cc:290] T 9d35b87a150449d5b2adefeb4b241fd2 P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515), efe0ab937a564199977b910104824f30 (127.21.63.3:33527)
I20260709 13:12:46.510797  1087 leader_election.cc:304] T 98d5edd9b01941898accf5cf97879ab4 P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 22ec8c2439d14f139d5330c8ec45133a, efe0ab937a564199977b910104824f30; no voters: bbcbed9750ae416bab78287348e9e196
I20260709 13:12:46.511309  1272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d35b87a150449d5b2adefeb4b241fd2" candidate_uuid: "22ec8c2439d14f139d5330c8ec45133a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe0ab937a564199977b910104824f30" is_pre_election: true
I20260709 13:12:46.512032  1197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d35b87a150449d5b2adefeb4b241fd2" candidate_uuid: "22ec8c2439d14f139d5330c8ec45133a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbcbed9750ae416bab78287348e9e196" is_pre_election: true
I20260709 13:12:46.498569  1342 raft_consensus.cc:515] T 250a7a602e924c57b99ecbd2bf900bcc P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.515283  1319 raft_consensus.cc:515] T 89400744bdfc474a805d19fe866d401b P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } } peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } }
I20260709 13:12:46.516508  1314 ts_tablet_manager.cc:1434] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P 22ec8c2439d14f139d5330c8ec45133a: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.002s
I20260709 13:12:46.517117  1272 raft_consensus.cc:2468] T 9d35b87a150449d5b2adefeb4b241fd2 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22ec8c2439d14f139d5330c8ec45133a in term 0.
I20260709 13:12:46.517825  1321 raft_consensus.cc:2804] T 98d5edd9b01941898accf5cf97879ab4 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:46.520035  1321 raft_consensus.cc:493] T 98d5edd9b01941898accf5cf97879ab4 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:46.520475  1314 tablet_bootstrap.cc:492] T 89400744bdfc474a805d19fe866d401b P 22ec8c2439d14f139d5330c8ec45133a: Bootstrap starting.
I20260709 13:12:46.518512  1197 raft_consensus.cc:2468] T 9d35b87a150449d5b2adefeb4b241fd2 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22ec8c2439d14f139d5330c8ec45133a in term 0.
I20260709 13:12:46.521966  1088 leader_election.cc:304] T 9d35b87a150449d5b2adefeb4b241fd2 P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22ec8c2439d14f139d5330c8ec45133a, bbcbed9750ae416bab78287348e9e196; no voters: 
I20260709 13:12:46.522217  1319 leader_election.cc:290] T 89400744bdfc474a805d19fe866d401b P bbcbed9750ae416bab78287348e9e196 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers efe0ab937a564199977b910104824f30 (127.21.63.3:33527), 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943)
I20260709 13:12:46.522678  1197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250a7a602e924c57b99ecbd2bf900bcc" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbcbed9750ae416bab78287348e9e196"
I20260709 13:12:46.522989  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250a7a602e924c57b99ecbd2bf900bcc" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22ec8c2439d14f139d5330c8ec45133a"
I20260709 13:12:46.523250  1197 raft_consensus.cc:3060] T 250a7a602e924c57b99ecbd2bf900bcc P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:46.520417  1321 raft_consensus.cc:3060] T 98d5edd9b01941898accf5cf97879ab4 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:46.527050  1342 leader_election.cc:290] T 250a7a602e924c57b99ecbd2bf900bcc P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 election: Requested vote from peers 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943), bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515)
W20260709 13:12:46.528734  1235 leader_election.cc:343] T 250a7a602e924c57b99ecbd2bf900bcc P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:46.531077  1314 tablet_bootstrap.cc:654] T 89400744bdfc474a805d19fe866d401b P 22ec8c2439d14f139d5330c8ec45133a: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:46.534960  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd271e857f3f41c6bcc5d54aae05f985" candidate_uuid: "bbcbed9750ae416bab78287348e9e196" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22ec8c2439d14f139d5330c8ec45133a" is_pre_election: true
W20260709 13:12:46.537143  1161 leader_election.cc:343] T fd271e857f3f41c6bcc5d54aae05f985 P bbcbed9750ae416bab78287348e9e196 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:46.535315  1123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89400744bdfc474a805d19fe866d401b" candidate_uuid: "bbcbed9750ae416bab78287348e9e196" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22ec8c2439d14f139d5330c8ec45133a" is_pre_election: true
I20260709 13:12:46.538506  1197 raft_consensus.cc:2468] T 250a7a602e924c57b99ecbd2bf900bcc P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe0ab937a564199977b910104824f30 in term 1.
W20260709 13:12:46.538905  1161 leader_election.cc:343] T 89400744bdfc474a805d19fe866d401b P bbcbed9750ae416bab78287348e9e196 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:46.537393  1313 tablet_bootstrap.cc:654] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P efe0ab937a564199977b910104824f30: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:46.539774  1318 raft_consensus.cc:515] T af54d168945b40d7b514668ce80da735 P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.541352  1237 leader_election.cc:304] T 250a7a602e924c57b99ecbd2bf900bcc P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bbcbed9750ae416bab78287348e9e196, efe0ab937a564199977b910104824f30; no voters: 22ec8c2439d14f139d5330c8ec45133a
I20260709 13:12:46.549454  1271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89400744bdfc474a805d19fe866d401b" candidate_uuid: "bbcbed9750ae416bab78287348e9e196" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe0ab937a564199977b910104824f30" is_pre_election: true
I20260709 13:12:46.550643  1272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd271e857f3f41c6bcc5d54aae05f985" candidate_uuid: "bbcbed9750ae416bab78287348e9e196" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe0ab937a564199977b910104824f30" is_pre_election: true
I20260709 13:12:46.566457  1123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af54d168945b40d7b514668ce80da735" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22ec8c2439d14f139d5330c8ec45133a"
I20260709 13:12:46.565879  1321 raft_consensus.cc:515] T 98d5edd9b01941898accf5cf97879ab4 P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.565780  1318 leader_election.cc:290] T af54d168945b40d7b514668ce80da735 P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 election: Requested vote from peers 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943), bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515)
I20260709 13:12:46.567206  1123 raft_consensus.cc:3060] T af54d168945b40d7b514668ce80da735 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Advancing to term 1
W20260709 13:12:46.564961  1162 leader_election.cc:343] T fd271e857f3f41c6bcc5d54aae05f985 P bbcbed9750ae416bab78287348e9e196 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efe0ab937a564199977b910104824f30 (127.21.63.3:33527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:46.566020  1271 raft_consensus.cc:2468] T 89400744bdfc474a805d19fe866d401b P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bbcbed9750ae416bab78287348e9e196 in term 0.
I20260709 13:12:46.568707  1162 leader_election.cc:304] T fd271e857f3f41c6bcc5d54aae05f985 P bbcbed9750ae416bab78287348e9e196 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bbcbed9750ae416bab78287348e9e196; no voters: 22ec8c2439d14f139d5330c8ec45133a, efe0ab937a564199977b910104824f30
I20260709 13:12:46.569020  1346 raft_consensus.cc:2804] T 9d35b87a150449d5b2adefeb4b241fd2 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:46.569459  1346 raft_consensus.cc:493] T 9d35b87a150449d5b2adefeb4b241fd2 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:46.569777  1346 raft_consensus.cc:3060] T 9d35b87a150449d5b2adefeb4b241fd2 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:46.570127  1162 leader_election.cc:304] T 89400744bdfc474a805d19fe866d401b P bbcbed9750ae416bab78287348e9e196 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: bbcbed9750ae416bab78287348e9e196, efe0ab937a564199977b910104824f30; no voters: 22ec8c2439d14f139d5330c8ec45133a
W20260709 13:12:46.562754  1087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:33527, user_credentials={real_user=slave}} blocked reactor thread for 52043us
I20260709 13:12:46.573751  1087 leader_election.cc:304] T af54d168945b40d7b514668ce80da735 P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 22ec8c2439d14f139d5330c8ec45133a, efe0ab937a564199977b910104824f30; no voters: bbcbed9750ae416bab78287348e9e196
I20260709 13:12:46.575242  1314 tablet_bootstrap.cc:492] T 89400744bdfc474a805d19fe866d401b P 22ec8c2439d14f139d5330c8ec45133a: No bootstrap required, opened a new log
I20260709 13:12:46.575783  1314 ts_tablet_manager.cc:1403] T 89400744bdfc474a805d19fe866d401b P 22ec8c2439d14f139d5330c8ec45133a: Time spent bootstrapping tablet: real 0.056s	user 0.011s	sys 0.031s
I20260709 13:12:46.571143  1318 raft_consensus.cc:2804] T 250a7a602e924c57b99ecbd2bf900bcc P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:46.571414  1319 raft_consensus.cc:2749] T fd271e857f3f41c6bcc5d54aae05f985 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:46.576993  1318 raft_consensus.cc:697] T 250a7a602e924c57b99ecbd2bf900bcc P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Becoming Leader. State: Replica: efe0ab937a564199977b910104824f30, State: Running, Role: LEADER
I20260709 13:12:46.577358  1319 raft_consensus.cc:2804] T 89400744bdfc474a805d19fe866d401b P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:46.578037  1319 raft_consensus.cc:493] T 89400744bdfc474a805d19fe866d401b P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:46.578676  1319 raft_consensus.cc:3060] T 89400744bdfc474a805d19fe866d401b P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:46.578101  1318 consensus_queue.cc:237] T 250a7a602e924c57b99ecbd2bf900bcc P efe0ab937a564199977b910104824f30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.579311  1314 raft_consensus.cc:359] T 89400744bdfc474a805d19fe866d401b P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } } peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } }
I20260709 13:12:46.580400  1314 raft_consensus.cc:385] T 89400744bdfc474a805d19fe866d401b P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:46.580880  1314 raft_consensus.cc:740] T 89400744bdfc474a805d19fe866d401b P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22ec8c2439d14f139d5330c8ec45133a, State: Initialized, Role: FOLLOWER
I20260709 13:12:46.581534  1313 tablet_bootstrap.cc:492] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P efe0ab937a564199977b910104824f30: No bootstrap required, opened a new log
I20260709 13:12:46.582369  1313 ts_tablet_manager.cc:1403] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P efe0ab937a564199977b910104824f30: Time spent bootstrapping tablet: real 0.078s	user 0.014s	sys 0.007s
I20260709 13:12:46.582013  1314 consensus_queue.cc:260] T 89400744bdfc474a805d19fe866d401b P 22ec8c2439d14f139d5330c8ec45133a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } } peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } }
I20260709 13:12:46.583500  1197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af54d168945b40d7b514668ce80da735" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbcbed9750ae416bab78287348e9e196"
I20260709 13:12:46.591054  1321 leader_election.cc:290] T 98d5edd9b01941898accf5cf97879ab4 P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 1 election: Requested vote from peers bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515), efe0ab937a564199977b910104824f30 (127.21.63.3:33527)
I20260709 13:12:46.591456  1198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d5edd9b01941898accf5cf97879ab4" candidate_uuid: "22ec8c2439d14f139d5330c8ec45133a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbcbed9750ae416bab78287348e9e196"
W20260709 13:12:46.593168  1237 leader_election.cc:343] T af54d168945b40d7b514668ce80da735 P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515): Illegal state: must be running to vote when last-logged opid is not known
W20260709 13:12:46.593531  1088 leader_election.cc:343] T 98d5edd9b01941898accf5cf97879ab4 P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:46.596544  1312 tablet_bootstrap.cc:492] T 57aa843159154feb87f476bac96b2dba P bbcbed9750ae416bab78287348e9e196: No bootstrap required, opened a new log
I20260709 13:12:46.597111  1312 ts_tablet_manager.cc:1403] T 57aa843159154feb87f476bac96b2dba P bbcbed9750ae416bab78287348e9e196: Time spent bootstrapping tablet: real 0.129s	user 0.032s	sys 0.022s
I20260709 13:12:46.598498  1313 raft_consensus.cc:359] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.599004  1346 raft_consensus.cc:515] T 9d35b87a150449d5b2adefeb4b241fd2 P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.599555  1313 raft_consensus.cc:385] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:46.600461  1313 raft_consensus.cc:740] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe0ab937a564199977b910104824f30, State: Initialized, Role: FOLLOWER
I20260709 13:12:46.601742  1313 consensus_queue.cc:260] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P efe0ab937a564199977b910104824f30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.610857  1342 raft_consensus.cc:493] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:46.611513  1342 raft_consensus.cc:515] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.614212  1346 leader_election.cc:290] T 9d35b87a150449d5b2adefeb4b241fd2 P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 1 election: Requested vote from peers bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515), efe0ab937a564199977b910104824f30 (127.21.63.3:33527)
W20260709 13:12:46.615669  1162 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:33527, user_credentials={real_user=slave}} blocked reactor thread for 45627us
I20260709 13:12:46.616289  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad7ce1d2ca94ba2bc0a82302a9eee4e" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22ec8c2439d14f139d5330c8ec45133a" is_pre_election: true
I20260709 13:12:46.615857  1123 raft_consensus.cc:2468] T af54d168945b40d7b514668ce80da735 P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe0ab937a564199977b910104824f30 in term 1.
I20260709 13:12:46.616729  1198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad7ce1d2ca94ba2bc0a82302a9eee4e" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbcbed9750ae416bab78287348e9e196" is_pre_election: true
I20260709 13:12:46.617009  1342 leader_election.cc:290] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943), bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515)
I20260709 13:12:46.617473  1198 raft_consensus.cc:2468] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efe0ab937a564199977b910104824f30 in term 0.
I20260709 13:12:46.617980  1122 raft_consensus.cc:2468] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efe0ab937a564199977b910104824f30 in term 0.
I20260709 13:12:46.618320  1321 raft_consensus.cc:2764] T af54d168945b40d7b514668ce80da735 P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 13:12:46.615705  1197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d35b87a150449d5b2adefeb4b241fd2" candidate_uuid: "22ec8c2439d14f139d5330c8ec45133a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbcbed9750ae416bab78287348e9e196"
I20260709 13:12:46.618398  1319 raft_consensus.cc:515] T 89400744bdfc474a805d19fe866d401b P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } } peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } }
I20260709 13:12:46.620183  1312 raft_consensus.cc:359] T 57aa843159154feb87f476bac96b2dba P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.622822  1312 raft_consensus.cc:385] T 57aa843159154feb87f476bac96b2dba P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:46.623271  1312 raft_consensus.cc:740] T 57aa843159154feb87f476bac96b2dba P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbcbed9750ae416bab78287348e9e196, State: Initialized, Role: FOLLOWER
I20260709 13:12:46.624449  1312 consensus_queue.cc:260] T 57aa843159154feb87f476bac96b2dba P bbcbed9750ae416bab78287348e9e196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.625761  1237 leader_election.cc:304] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bbcbed9750ae416bab78287348e9e196, efe0ab937a564199977b910104824f30; no voters: 
I20260709 13:12:46.627213  1342 raft_consensus.cc:2804] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:46.627683  1342 raft_consensus.cc:493] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:46.628119  1342 raft_consensus.cc:3060] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:46.632375  1312 ts_tablet_manager.cc:1434] T 57aa843159154feb87f476bac96b2dba P bbcbed9750ae416bab78287348e9e196: Time spent starting tablet: real 0.035s	user 0.007s	sys 0.000s
I20260709 13:12:46.621326  1197 raft_consensus.cc:3060] T 9d35b87a150449d5b2adefeb4b241fd2 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:46.633996  1312 tablet_bootstrap.cc:492] T 98d5edd9b01941898accf5cf97879ab4 P bbcbed9750ae416bab78287348e9e196: Bootstrap starting.
I20260709 13:12:46.625311  1235 leader_election.cc:304] T af54d168945b40d7b514668ce80da735 P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 22ec8c2439d14f139d5330c8ec45133a, efe0ab937a564199977b910104824f30; no voters: bbcbed9750ae416bab78287348e9e196
I20260709 13:12:46.639489  1336 raft_consensus.cc:2804] T af54d168945b40d7b514668ce80da735 P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:46.640204  1336 raft_consensus.cc:697] T af54d168945b40d7b514668ce80da735 P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Becoming Leader. State: Replica: efe0ab937a564199977b910104824f30, State: Running, Role: LEADER
I20260709 13:12:46.641291  1336 consensus_queue.cc:237] T af54d168945b40d7b514668ce80da735 P efe0ab937a564199977b910104824f30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.644732  1312 tablet_bootstrap.cc:654] T 98d5edd9b01941898accf5cf97879ab4 P bbcbed9750ae416bab78287348e9e196: Neither blocks nor log segments found. Creating new log.
W20260709 13:12:46.619467  1087 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.21.63.3:33527, user_credentials={real_user=slave}} blocked reactor thread for 45850.7us
I20260709 13:12:46.650537  1321 raft_consensus.cc:493] T 57aa843159154feb87f476bac96b2dba P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:46.650879  1271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d5edd9b01941898accf5cf97879ab4" candidate_uuid: "22ec8c2439d14f139d5330c8ec45133a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe0ab937a564199977b910104824f30"
I20260709 13:12:46.651003  1321 raft_consensus.cc:515] T 57aa843159154feb87f476bac96b2dba P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.651566  1271 raft_consensus.cc:3060] T 98d5edd9b01941898accf5cf97879ab4 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:46.651494  1272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9d35b87a150449d5b2adefeb4b241fd2" candidate_uuid: "22ec8c2439d14f139d5330c8ec45133a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe0ab937a564199977b910104824f30"
I20260709 13:12:46.652561  1272 raft_consensus.cc:3060] T 9d35b87a150449d5b2adefeb4b241fd2 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:46.653645  1313 ts_tablet_manager.cc:1434] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P efe0ab937a564199977b910104824f30: Time spent starting tablet: real 0.071s	user 0.011s	sys 0.000s
I20260709 13:12:46.654134  1198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57aa843159154feb87f476bac96b2dba" candidate_uuid: "22ec8c2439d14f139d5330c8ec45133a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbcbed9750ae416bab78287348e9e196" is_pre_election: true
I20260709 13:12:46.654754  1313 tablet_bootstrap.cc:492] T fd271e857f3f41c6bcc5d54aae05f985 P efe0ab937a564199977b910104824f30: Bootstrap starting.
I20260709 13:12:46.654778  1198 raft_consensus.cc:2468] T 57aa843159154feb87f476bac96b2dba P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22ec8c2439d14f139d5330c8ec45133a in term 0.
I20260709 13:12:46.656579  1088 leader_election.cc:304] T 57aa843159154feb87f476bac96b2dba P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22ec8c2439d14f139d5330c8ec45133a, bbcbed9750ae416bab78287348e9e196; no voters: 
I20260709 13:12:46.657648  1356 raft_consensus.cc:2804] T 57aa843159154feb87f476bac96b2dba P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:46.658036  1356 raft_consensus.cc:493] T 57aa843159154feb87f476bac96b2dba P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:46.658326  1356 raft_consensus.cc:3060] T 57aa843159154feb87f476bac96b2dba P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:46.659695  1271 raft_consensus.cc:2468] T 98d5edd9b01941898accf5cf97879ab4 P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22ec8c2439d14f139d5330c8ec45133a in term 1.
I20260709 13:12:46.660400  1272 raft_consensus.cc:2468] T 9d35b87a150449d5b2adefeb4b241fd2 P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22ec8c2439d14f139d5330c8ec45133a in term 1.
I20260709 13:12:46.663911  1342 raft_consensus.cc:515] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.665202  1321 leader_election.cc:290] T 57aa843159154feb87f476bac96b2dba P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515), efe0ab937a564199977b910104824f30 (127.21.63.3:33527)
I20260709 13:12:46.668009  1314 ts_tablet_manager.cc:1434] T 89400744bdfc474a805d19fe866d401b P 22ec8c2439d14f139d5330c8ec45133a: Time spent starting tablet: real 0.092s	user 0.016s	sys 0.003s
I20260709 13:12:46.670753  1313 tablet_bootstrap.cc:654] T fd271e857f3f41c6bcc5d54aae05f985 P efe0ab937a564199977b910104824f30: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:46.671096  1197 raft_consensus.cc:2468] T 9d35b87a150449d5b2adefeb4b241fd2 P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22ec8c2439d14f139d5330c8ec45133a in term 1.
I20260709 13:12:46.672549  1314 tablet_bootstrap.cc:492] T 8de23e2629b2451b900a09bfe12dbf86 P 22ec8c2439d14f139d5330c8ec45133a: Bootstrap starting.
I20260709 13:12:46.674157  1088 leader_election.cc:304] T 9d35b87a150449d5b2adefeb4b241fd2 P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22ec8c2439d14f139d5330c8ec45133a, bbcbed9750ae416bab78287348e9e196; no voters: 
I20260709 13:12:46.675890  1087 leader_election.cc:304] T 98d5edd9b01941898accf5cf97879ab4 P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 22ec8c2439d14f139d5330c8ec45133a, efe0ab937a564199977b910104824f30; no voters: bbcbed9750ae416bab78287348e9e196
I20260709 13:12:46.678092  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad7ce1d2ca94ba2bc0a82302a9eee4e" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22ec8c2439d14f139d5330c8ec45133a"
I20260709 13:12:46.678752  1122 raft_consensus.cc:3060] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:46.679365  1352 raft_consensus.cc:2804] T 9d35b87a150449d5b2adefeb4b241fd2 P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:46.680073  1352 raft_consensus.cc:697] T 9d35b87a150449d5b2adefeb4b241fd2 P 22ec8c2439d14f139d5330c8ec45133a [term 1 LEADER]: Becoming Leader. State: Replica: 22ec8c2439d14f139d5330c8ec45133a, State: Running, Role: LEADER
I20260709 13:12:46.681043  1321 raft_consensus.cc:2804] T 98d5edd9b01941898accf5cf97879ab4 P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:46.681190  1352 consensus_queue.cc:237] T 9d35b87a150449d5b2adefeb4b241fd2 P 22ec8c2439d14f139d5330c8ec45133a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.684983  1272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57aa843159154feb87f476bac96b2dba" candidate_uuid: "22ec8c2439d14f139d5330c8ec45133a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe0ab937a564199977b910104824f30" is_pre_election: true
W20260709 13:12:46.686586  1087 leader_election.cc:343] T 57aa843159154feb87f476bac96b2dba P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer efe0ab937a564199977b910104824f30 (127.21.63.3:33527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:46.689260  1319 leader_election.cc:290] T 89400744bdfc474a805d19fe866d401b P bbcbed9750ae416bab78287348e9e196 [CANDIDATE]: Term 1 election: Requested vote from peers efe0ab937a564199977b910104824f30 (127.21.63.3:33527), 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943)
I20260709 13:12:46.696292  1342 leader_election.cc:290] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 election: Requested vote from peers 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943), bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515)
I20260709 13:12:46.696646  1321 raft_consensus.cc:697] T 98d5edd9b01941898accf5cf97879ab4 P 22ec8c2439d14f139d5330c8ec45133a [term 1 LEADER]: Becoming Leader. State: Replica: 22ec8c2439d14f139d5330c8ec45133a, State: Running, Role: LEADER
I20260709 13:12:46.701517  1272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89400744bdfc474a805d19fe866d401b" candidate_uuid: "bbcbed9750ae416bab78287348e9e196" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe0ab937a564199977b910104824f30"
I20260709 13:12:46.701838  1197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad7ce1d2ca94ba2bc0a82302a9eee4e" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbcbed9750ae416bab78287348e9e196"
I20260709 13:12:46.702186  1272 raft_consensus.cc:3060] T 89400744bdfc474a805d19fe866d401b P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:46.702482  1197 raft_consensus.cc:3060] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:46.703058  1314 tablet_bootstrap.cc:654] T 8de23e2629b2451b900a09bfe12dbf86 P 22ec8c2439d14f139d5330c8ec45133a: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:46.697502  1123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89400744bdfc474a805d19fe866d401b" candidate_uuid: "bbcbed9750ae416bab78287348e9e196" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22ec8c2439d14f139d5330c8ec45133a"
I20260709 13:12:46.698635  1356 raft_consensus.cc:515] T 57aa843159154feb87f476bac96b2dba P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.706102  1123 raft_consensus.cc:3060] T 89400744bdfc474a805d19fe866d401b P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:46.708617  1198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57aa843159154feb87f476bac96b2dba" candidate_uuid: "22ec8c2439d14f139d5330c8ec45133a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbcbed9750ae416bab78287348e9e196"
I20260709 13:12:46.697795  1321 consensus_queue.cc:237] T 98d5edd9b01941898accf5cf97879ab4 P 22ec8c2439d14f139d5330c8ec45133a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.709224  1198 raft_consensus.cc:3060] T 57aa843159154feb87f476bac96b2dba P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:46.709836  1197 raft_consensus.cc:2468] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe0ab937a564199977b910104824f30 in term 1.
I20260709 13:12:46.711169  1237 leader_election.cc:304] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bbcbed9750ae416bab78287348e9e196, efe0ab937a564199977b910104824f30; no voters: 
I20260709 13:12:46.712035  1342 raft_consensus.cc:2804] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:46.712479  1342 raft_consensus.cc:697] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Becoming Leader. State: Replica: efe0ab937a564199977b910104824f30, State: Running, Role: LEADER
I20260709 13:12:46.712517  1272 raft_consensus.cc:2468] T 89400744bdfc474a805d19fe866d401b P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbcbed9750ae416bab78287348e9e196 in term 1.
I20260709 13:12:46.713310  1123 raft_consensus.cc:2468] T 89400744bdfc474a805d19fe866d401b P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbcbed9750ae416bab78287348e9e196 in term 1.
I20260709 13:12:46.713500  1342 consensus_queue.cc:237] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P efe0ab937a564199977b910104824f30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.713893  1162 leader_election.cc:304] T 89400744bdfc474a805d19fe866d401b P bbcbed9750ae416bab78287348e9e196 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bbcbed9750ae416bab78287348e9e196, efe0ab937a564199977b910104824f30; no voters: 
I20260709 13:12:46.714649  1312 tablet_bootstrap.cc:492] T 98d5edd9b01941898accf5cf97879ab4 P bbcbed9750ae416bab78287348e9e196: No bootstrap required, opened a new log
I20260709 13:12:46.715124  1312 ts_tablet_manager.cc:1403] T 98d5edd9b01941898accf5cf97879ab4 P bbcbed9750ae416bab78287348e9e196: Time spent bootstrapping tablet: real 0.081s	user 0.018s	sys 0.021s
I20260709 13:12:46.715133  1319 raft_consensus.cc:2804] T 89400744bdfc474a805d19fe866d401b P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:46.715765  1319 raft_consensus.cc:697] T 89400744bdfc474a805d19fe866d401b P bbcbed9750ae416bab78287348e9e196 [term 1 LEADER]: Becoming Leader. State: Replica: bbcbed9750ae416bab78287348e9e196, State: Running, Role: LEADER
I20260709 13:12:46.725389  1312 raft_consensus.cc:359] T 98d5edd9b01941898accf5cf97879ab4 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.728147  1312 raft_consensus.cc:385] T 98d5edd9b01941898accf5cf97879ab4 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:46.728428  1312 raft_consensus.cc:740] T 98d5edd9b01941898accf5cf97879ab4 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbcbed9750ae416bab78287348e9e196, State: Initialized, Role: FOLLOWER
I20260709 13:12:46.729447  1312 consensus_queue.cc:260] T 98d5edd9b01941898accf5cf97879ab4 P bbcbed9750ae416bab78287348e9e196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.716531  1319 consensus_queue.cc:237] T 89400744bdfc474a805d19fe866d401b P bbcbed9750ae416bab78287348e9e196 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } } peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } }
I20260709 13:12:46.728848  1122 raft_consensus.cc:2468] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe0ab937a564199977b910104824f30 in term 1.
I20260709 13:12:46.734302  1272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57aa843159154feb87f476bac96b2dba" candidate_uuid: "22ec8c2439d14f139d5330c8ec45133a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe0ab937a564199977b910104824f30"
W20260709 13:12:46.735788  1087 leader_election.cc:343] T 57aa843159154feb87f476bac96b2dba P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer efe0ab937a564199977b910104824f30 (127.21.63.3:33527): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:46.739246  1198 raft_consensus.cc:2468] T 57aa843159154feb87f476bac96b2dba P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22ec8c2439d14f139d5330c8ec45133a in term 1.
I20260709 13:12:46.730542  1356 leader_election.cc:290] T 57aa843159154feb87f476bac96b2dba P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 1 election: Requested vote from peers bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515), efe0ab937a564199977b910104824f30 (127.21.63.3:33527)
I20260709 13:12:46.740864  1088 leader_election.cc:304] T 57aa843159154feb87f476bac96b2dba P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 22ec8c2439d14f139d5330c8ec45133a, bbcbed9750ae416bab78287348e9e196; no voters: efe0ab937a564199977b910104824f30
I20260709 13:12:46.747079  1312 ts_tablet_manager.cc:1434] T 98d5edd9b01941898accf5cf97879ab4 P bbcbed9750ae416bab78287348e9e196: Time spent starting tablet: real 0.032s	user 0.006s	sys 0.001s
I20260709 13:12:46.748380  1312 tablet_bootstrap.cc:492] T af54d168945b40d7b514668ce80da735 P bbcbed9750ae416bab78287348e9e196: Bootstrap starting.
I20260709 13:12:46.745484  1356 raft_consensus.cc:2804] T 57aa843159154feb87f476bac96b2dba P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:46.753177  1356 raft_consensus.cc:697] T 57aa843159154feb87f476bac96b2dba P 22ec8c2439d14f139d5330c8ec45133a [term 1 LEADER]: Becoming Leader. State: Replica: 22ec8c2439d14f139d5330c8ec45133a, State: Running, Role: LEADER
I20260709 13:12:46.751899  1020 catalog_manager.cc:5697] T 9d35b87a150449d5b2adefeb4b241fd2 P 22ec8c2439d14f139d5330c8ec45133a reported cstate change: term changed from 0 to 1, leader changed from <none> to 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1). New cstate: current_term: 1 leader_uuid: "22ec8c2439d14f139d5330c8ec45133a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:46.754645  1314 tablet_bootstrap.cc:492] T 8de23e2629b2451b900a09bfe12dbf86 P 22ec8c2439d14f139d5330c8ec45133a: No bootstrap required, opened a new log
I20260709 13:12:46.754339  1356 consensus_queue.cc:237] T 57aa843159154feb87f476bac96b2dba P 22ec8c2439d14f139d5330c8ec45133a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.755203  1314 ts_tablet_manager.cc:1403] T 8de23e2629b2451b900a09bfe12dbf86 P 22ec8c2439d14f139d5330c8ec45133a: Time spent bootstrapping tablet: real 0.083s	user 0.013s	sys 0.008s
I20260709 13:12:46.755141  1020 catalog_manager.cc:5697] T 98d5edd9b01941898accf5cf97879ab4 P 22ec8c2439d14f139d5330c8ec45133a reported cstate change: term changed from 0 to 1, leader changed from <none> to 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1). New cstate: current_term: 1 leader_uuid: "22ec8c2439d14f139d5330c8ec45133a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:46.758450  1314 raft_consensus.cc:359] T 8de23e2629b2451b900a09bfe12dbf86 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.760977  1314 raft_consensus.cc:385] T 8de23e2629b2451b900a09bfe12dbf86 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:46.761430  1314 raft_consensus.cc:740] T 8de23e2629b2451b900a09bfe12dbf86 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22ec8c2439d14f139d5330c8ec45133a, State: Initialized, Role: FOLLOWER
I20260709 13:12:46.762321  1314 consensus_queue.cc:260] T 8de23e2629b2451b900a09bfe12dbf86 P 22ec8c2439d14f139d5330c8ec45133a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.767449  1314 ts_tablet_manager.cc:1434] T 8de23e2629b2451b900a09bfe12dbf86 P 22ec8c2439d14f139d5330c8ec45133a: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.001s
I20260709 13:12:46.768784  1313 tablet_bootstrap.cc:492] T fd271e857f3f41c6bcc5d54aae05f985 P efe0ab937a564199977b910104824f30: No bootstrap required, opened a new log
I20260709 13:12:46.769202  1314 tablet_bootstrap.cc:492] T 250a7a602e924c57b99ecbd2bf900bcc P 22ec8c2439d14f139d5330c8ec45133a: Bootstrap starting.
I20260709 13:12:46.769411  1313 ts_tablet_manager.cc:1403] T fd271e857f3f41c6bcc5d54aae05f985 P efe0ab937a564199977b910104824f30: Time spent bootstrapping tablet: real 0.115s	user 0.017s	sys 0.006s
I20260709 13:12:46.772313  1313 raft_consensus.cc:359] T fd271e857f3f41c6bcc5d54aae05f985 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.772981  1313 raft_consensus.cc:385] T fd271e857f3f41c6bcc5d54aae05f985 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:46.773265  1313 raft_consensus.cc:740] T fd271e857f3f41c6bcc5d54aae05f985 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe0ab937a564199977b910104824f30, State: Initialized, Role: FOLLOWER
I20260709 13:12:46.773902  1313 consensus_queue.cc:260] T fd271e857f3f41c6bcc5d54aae05f985 P efe0ab937a564199977b910104824f30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.774384  1019 catalog_manager.cc:5697] T 250a7a602e924c57b99ecbd2bf900bcc P efe0ab937a564199977b910104824f30 reported cstate change: term changed from 0 to 1, leader changed from <none> to efe0ab937a564199977b910104824f30 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "efe0ab937a564199977b910104824f30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:46.775970  1313 ts_tablet_manager.cc:1434] T fd271e857f3f41c6bcc5d54aae05f985 P efe0ab937a564199977b910104824f30: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:12:46.776755  1314 tablet_bootstrap.cc:654] T 250a7a602e924c57b99ecbd2bf900bcc P 22ec8c2439d14f139d5330c8ec45133a: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:46.776461  1019 catalog_manager.cc:5697] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P efe0ab937a564199977b910104824f30 reported cstate change: term changed from 0 to 1, leader changed from <none> to efe0ab937a564199977b910104824f30 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "efe0ab937a564199977b910104824f30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:46.776793  1313 tablet_bootstrap.cc:492] T 57aa843159154feb87f476bac96b2dba P efe0ab937a564199977b910104824f30: Bootstrap starting.
I20260709 13:12:46.779815  1019 catalog_manager.cc:5697] T af54d168945b40d7b514668ce80da735 P efe0ab937a564199977b910104824f30 reported cstate change: term changed from 0 to 1, leader changed from <none> to efe0ab937a564199977b910104824f30 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "efe0ab937a564199977b910104824f30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:46.781262  1312 tablet_bootstrap.cc:654] T af54d168945b40d7b514668ce80da735 P bbcbed9750ae416bab78287348e9e196: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:46.788190  1313 tablet_bootstrap.cc:654] T 57aa843159154feb87f476bac96b2dba P efe0ab937a564199977b910104824f30: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:46.791684  1314 tablet_bootstrap.cc:492] T 250a7a602e924c57b99ecbd2bf900bcc P 22ec8c2439d14f139d5330c8ec45133a: No bootstrap required, opened a new log
I20260709 13:12:46.792253  1314 ts_tablet_manager.cc:1403] T 250a7a602e924c57b99ecbd2bf900bcc P 22ec8c2439d14f139d5330c8ec45133a: Time spent bootstrapping tablet: real 0.023s	user 0.008s	sys 0.010s
I20260709 13:12:46.791975  1018 catalog_manager.cc:5697] T 89400744bdfc474a805d19fe866d401b P bbcbed9750ae416bab78287348e9e196 reported cstate change: term changed from 0 to 1, leader changed from <none> to bbcbed9750ae416bab78287348e9e196 (127.21.63.2). New cstate: current_term: 1 leader_uuid: "bbcbed9750ae416bab78287348e9e196" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:46.794508  1313 tablet_bootstrap.cc:492] T 57aa843159154feb87f476bac96b2dba P efe0ab937a564199977b910104824f30: No bootstrap required, opened a new log
I20260709 13:12:46.795085  1313 ts_tablet_manager.cc:1403] T 57aa843159154feb87f476bac96b2dba P efe0ab937a564199977b910104824f30: Time spent bootstrapping tablet: real 0.019s	user 0.005s	sys 0.004s
I20260709 13:12:46.795194  1314 raft_consensus.cc:359] T 250a7a602e924c57b99ecbd2bf900bcc P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.796018  1314 raft_consensus.cc:385] T 250a7a602e924c57b99ecbd2bf900bcc P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:46.796350  1314 raft_consensus.cc:740] T 250a7a602e924c57b99ecbd2bf900bcc P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22ec8c2439d14f139d5330c8ec45133a, State: Initialized, Role: FOLLOWER
I20260709 13:12:46.797226  1314 consensus_queue.cc:260] T 250a7a602e924c57b99ecbd2bf900bcc P 22ec8c2439d14f139d5330c8ec45133a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.797968  1313 raft_consensus.cc:359] T 57aa843159154feb87f476bac96b2dba P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.798758  1313 raft_consensus.cc:385] T 57aa843159154feb87f476bac96b2dba P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:46.799062  1313 raft_consensus.cc:740] T 57aa843159154feb87f476bac96b2dba P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe0ab937a564199977b910104824f30, State: Initialized, Role: FOLLOWER
I20260709 13:12:46.799914  1314 ts_tablet_manager.cc:1434] T 250a7a602e924c57b99ecbd2bf900bcc P 22ec8c2439d14f139d5330c8ec45133a: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 13:12:46.799789  1313 consensus_queue.cc:260] T 57aa843159154feb87f476bac96b2dba P efe0ab937a564199977b910104824f30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.801266  1314 tablet_bootstrap.cc:492] T fd271e857f3f41c6bcc5d54aae05f985 P 22ec8c2439d14f139d5330c8ec45133a: Bootstrap starting.
I20260709 13:12:46.802155  1313 ts_tablet_manager.cc:1434] T 57aa843159154feb87f476bac96b2dba P efe0ab937a564199977b910104824f30: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 13:12:46.794576  1312 tablet_bootstrap.cc:492] T af54d168945b40d7b514668ce80da735 P bbcbed9750ae416bab78287348e9e196: No bootstrap required, opened a new log
I20260709 13:12:46.803995  1312 ts_tablet_manager.cc:1403] T af54d168945b40d7b514668ce80da735 P bbcbed9750ae416bab78287348e9e196: Time spent bootstrapping tablet: real 0.056s	user 0.010s	sys 0.001s
I20260709 13:12:46.805015  1020 catalog_manager.cc:5697] T 57aa843159154feb87f476bac96b2dba P 22ec8c2439d14f139d5330c8ec45133a reported cstate change: term changed from 0 to 1, leader changed from <none> to 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1). New cstate: current_term: 1 leader_uuid: "22ec8c2439d14f139d5330c8ec45133a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:46.809477  1314 tablet_bootstrap.cc:654] T fd271e857f3f41c6bcc5d54aae05f985 P 22ec8c2439d14f139d5330c8ec45133a: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:46.806670  1312 raft_consensus.cc:359] T af54d168945b40d7b514668ce80da735 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.810899  1312 raft_consensus.cc:385] T af54d168945b40d7b514668ce80da735 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:46.811213  1312 raft_consensus.cc:740] T af54d168945b40d7b514668ce80da735 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbcbed9750ae416bab78287348e9e196, State: Initialized, Role: FOLLOWER
I20260709 13:12:46.811965  1312 consensus_queue.cc:260] T af54d168945b40d7b514668ce80da735 P bbcbed9750ae416bab78287348e9e196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.814042  1312 ts_tablet_manager.cc:1434] T af54d168945b40d7b514668ce80da735 P bbcbed9750ae416bab78287348e9e196: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 13:12:46.814793  1314 tablet_bootstrap.cc:492] T fd271e857f3f41c6bcc5d54aae05f985 P 22ec8c2439d14f139d5330c8ec45133a: No bootstrap required, opened a new log
I20260709 13:12:46.815301  1314 ts_tablet_manager.cc:1403] T fd271e857f3f41c6bcc5d54aae05f985 P 22ec8c2439d14f139d5330c8ec45133a: Time spent bootstrapping tablet: real 0.014s	user 0.010s	sys 0.000s
I20260709 13:12:46.816121  1319 raft_consensus.cc:493] T 98d5edd9b01941898accf5cf97879ab4 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:46.816576  1319 raft_consensus.cc:515] T 98d5edd9b01941898accf5cf97879ab4 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.819062  1319 leader_election.cc:290] T 98d5edd9b01941898accf5cf97879ab4 P bbcbed9750ae416bab78287348e9e196 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943), efe0ab937a564199977b910104824f30 (127.21.63.3:33527)
I20260709 13:12:46.819245  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d5edd9b01941898accf5cf97879ab4" candidate_uuid: "bbcbed9750ae416bab78287348e9e196" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22ec8c2439d14f139d5330c8ec45133a" is_pre_election: true
I20260709 13:12:46.819491  1314 raft_consensus.cc:359] T fd271e857f3f41c6bcc5d54aae05f985 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.820128  1314 raft_consensus.cc:385] T fd271e857f3f41c6bcc5d54aae05f985 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:46.820426  1314 raft_consensus.cc:740] T fd271e857f3f41c6bcc5d54aae05f985 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22ec8c2439d14f139d5330c8ec45133a, State: Initialized, Role: FOLLOWER
I20260709 13:12:46.821120  1314 consensus_queue.cc:260] T fd271e857f3f41c6bcc5d54aae05f985 P 22ec8c2439d14f139d5330c8ec45133a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.821898  1272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d5edd9b01941898accf5cf97879ab4" candidate_uuid: "bbcbed9750ae416bab78287348e9e196" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe0ab937a564199977b910104824f30" is_pre_election: true
I20260709 13:12:46.822678  1272 raft_consensus.cc:2393] T 98d5edd9b01941898accf5cf97879ab4 P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bbcbed9750ae416bab78287348e9e196 in current term 1: Already voted for candidate 22ec8c2439d14f139d5330c8ec45133a in this term.
I20260709 13:12:46.823856  1162 leader_election.cc:304] T 98d5edd9b01941898accf5cf97879ab4 P bbcbed9750ae416bab78287348e9e196 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bbcbed9750ae416bab78287348e9e196; no voters: 22ec8c2439d14f139d5330c8ec45133a, efe0ab937a564199977b910104824f30
I20260709 13:12:46.824864  1319 raft_consensus.cc:3060] T 98d5edd9b01941898accf5cf97879ab4 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:46.827819  1314 ts_tablet_manager.cc:1434] T fd271e857f3f41c6bcc5d54aae05f985 P 22ec8c2439d14f139d5330c8ec45133a: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 13:12:46.831957  1319 raft_consensus.cc:2749] T 98d5edd9b01941898accf5cf97879ab4 P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:46.913695  1319 raft_consensus.cc:493] T af54d168945b40d7b514668ce80da735 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:46.914745  1319 raft_consensus.cc:515] T af54d168945b40d7b514668ce80da735 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.916635  1319 leader_election.cc:290] T af54d168945b40d7b514668ce80da735 P bbcbed9750ae416bab78287348e9e196 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943), efe0ab937a564199977b910104824f30 (127.21.63.3:33527)
I20260709 13:12:46.917678  1272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af54d168945b40d7b514668ce80da735" candidate_uuid: "bbcbed9750ae416bab78287348e9e196" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe0ab937a564199977b910104824f30" is_pre_election: true
I20260709 13:12:46.917726  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af54d168945b40d7b514668ce80da735" candidate_uuid: "bbcbed9750ae416bab78287348e9e196" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22ec8c2439d14f139d5330c8ec45133a" is_pre_election: true
I20260709 13:12:46.918656  1122 raft_consensus.cc:2393] T af54d168945b40d7b514668ce80da735 P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bbcbed9750ae416bab78287348e9e196 in current term 1: Already voted for candidate efe0ab937a564199977b910104824f30 in this term.
I20260709 13:12:46.919665  1161 leader_election.cc:304] T af54d168945b40d7b514668ce80da735 P bbcbed9750ae416bab78287348e9e196 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bbcbed9750ae416bab78287348e9e196; no voters: 22ec8c2439d14f139d5330c8ec45133a, efe0ab937a564199977b910104824f30
I20260709 13:12:46.920459  1319 raft_consensus.cc:3060] T af54d168945b40d7b514668ce80da735 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:46.925374  1319 raft_consensus.cc:2749] T af54d168945b40d7b514668ce80da735 P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 13:12:46.959455  1122 raft_consensus.cc:3060] T 8de23e2629b2451b900a09bfe12dbf86 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:46.965927  1342 consensus_queue.cc:1048] T 8de23e2629b2451b900a09bfe12dbf86 P efe0ab937a564199977b910104824f30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:46.978075  1336 consensus_queue.cc:1048] T 8de23e2629b2451b900a09bfe12dbf86 P efe0ab937a564199977b910104824f30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:46.985697  1336 raft_consensus.cc:493] T fd271e857f3f41c6bcc5d54aae05f985 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:46.986131  1336 raft_consensus.cc:515] T fd271e857f3f41c6bcc5d54aae05f985 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.988054  1336 leader_election.cc:290] T fd271e857f3f41c6bcc5d54aae05f985 P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943), bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515)
I20260709 13:12:46.988826  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd271e857f3f41c6bcc5d54aae05f985" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22ec8c2439d14f139d5330c8ec45133a" is_pre_election: true
I20260709 13:12:46.989620  1122 raft_consensus.cc:2468] T fd271e857f3f41c6bcc5d54aae05f985 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efe0ab937a564199977b910104824f30 in term 0.
I20260709 13:12:46.989526  1198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd271e857f3f41c6bcc5d54aae05f985" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbcbed9750ae416bab78287348e9e196" is_pre_election: true
I20260709 13:12:46.990829  1198 raft_consensus.cc:2468] T fd271e857f3f41c6bcc5d54aae05f985 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efe0ab937a564199977b910104824f30 in term 0.
I20260709 13:12:46.990840  1235 leader_election.cc:304] T fd271e857f3f41c6bcc5d54aae05f985 P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22ec8c2439d14f139d5330c8ec45133a, efe0ab937a564199977b910104824f30; no voters: 
I20260709 13:12:46.991813  1336 raft_consensus.cc:2804] T fd271e857f3f41c6bcc5d54aae05f985 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:46.992183  1336 raft_consensus.cc:493] T fd271e857f3f41c6bcc5d54aae05f985 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:46.992516  1336 raft_consensus.cc:3060] T fd271e857f3f41c6bcc5d54aae05f985 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:46.998172  1336 raft_consensus.cc:515] T fd271e857f3f41c6bcc5d54aae05f985 P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:46.999692  1336 leader_election.cc:290] T fd271e857f3f41c6bcc5d54aae05f985 P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 election: Requested vote from peers 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943), bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515)
I20260709 13:12:47.000593  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd271e857f3f41c6bcc5d54aae05f985" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22ec8c2439d14f139d5330c8ec45133a"
I20260709 13:12:47.000820  1198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd271e857f3f41c6bcc5d54aae05f985" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbcbed9750ae416bab78287348e9e196"
I20260709 13:12:47.001317  1122 raft_consensus.cc:3060] T fd271e857f3f41c6bcc5d54aae05f985 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:47.001502  1198 raft_consensus.cc:3060] T fd271e857f3f41c6bcc5d54aae05f985 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:47.006394  1198 raft_consensus.cc:2468] T fd271e857f3f41c6bcc5d54aae05f985 P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe0ab937a564199977b910104824f30 in term 1.
I20260709 13:12:47.006531  1122 raft_consensus.cc:2468] T fd271e857f3f41c6bcc5d54aae05f985 P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe0ab937a564199977b910104824f30 in term 1.
I20260709 13:12:47.007535  1237 leader_election.cc:304] T fd271e857f3f41c6bcc5d54aae05f985 P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bbcbed9750ae416bab78287348e9e196, efe0ab937a564199977b910104824f30; no voters: 
I20260709 13:12:47.008368  1336 raft_consensus.cc:2804] T fd271e857f3f41c6bcc5d54aae05f985 P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:47.009070  1336 raft_consensus.cc:697] T fd271e857f3f41c6bcc5d54aae05f985 P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Becoming Leader. State: Replica: efe0ab937a564199977b910104824f30, State: Running, Role: LEADER
I20260709 13:12:47.009805  1336 consensus_queue.cc:237] T fd271e857f3f41c6bcc5d54aae05f985 P efe0ab937a564199977b910104824f30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:47.017381  1021 catalog_manager.cc:5697] T fd271e857f3f41c6bcc5d54aae05f985 P efe0ab937a564199977b910104824f30 reported cstate change: term changed from 0 to 1, leader changed from <none> to efe0ab937a564199977b910104824f30 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "efe0ab937a564199977b910104824f30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:47.037885  1336 consensus_queue.cc:1048] T af54d168945b40d7b514668ce80da735 P efe0ab937a564199977b910104824f30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:47.049093  1342 consensus_queue.cc:1048] T af54d168945b40d7b514668ce80da735 P efe0ab937a564199977b910104824f30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:47.053555  1336 consensus_queue.cc:1048] T 250a7a602e924c57b99ecbd2bf900bcc P efe0ab937a564199977b910104824f30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:47.066251  1122 raft_consensus.cc:3060] T 250a7a602e924c57b99ecbd2bf900bcc P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:47.074615  1198 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "89400744bdfc474a805d19fe866d401b"
dest_uuid: "bbcbed9750ae416bab78287348e9e196"
mode: GRACEFUL
new_leader_uuid: "22ec8c2439d14f139d5330c8ec45133a"
 from {username='slave'} at 127.0.0.1:60790
I20260709 13:12:47.075302  1198 raft_consensus.cc:606] T 89400744bdfc474a805d19fe866d401b P bbcbed9750ae416bab78287348e9e196 [term 1 LEADER]: Received request to transfer leadership to TS 22ec8c2439d14f139d5330c8ec45133a
I20260709 13:12:47.075729  1336 consensus_queue.cc:1048] T 250a7a602e924c57b99ecbd2bf900bcc P efe0ab937a564199977b910104824f30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:47.094090  1272 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8ad7ce1d2ca94ba2bc0a82302a9eee4e"
dest_uuid: "efe0ab937a564199977b910104824f30"
mode: GRACEFUL
new_leader_uuid: "22ec8c2439d14f139d5330c8ec45133a"
 from {username='slave'} at 127.0.0.1:52974
I20260709 13:12:47.094651  1272 raft_consensus.cc:606] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Received request to transfer leadership to TS 22ec8c2439d14f139d5330c8ec45133a
I20260709 13:12:47.103576  1272 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8de23e2629b2451b900a09bfe12dbf86"
dest_uuid: "efe0ab937a564199977b910104824f30"
mode: GRACEFUL
new_leader_uuid: "22ec8c2439d14f139d5330c8ec45133a"
 from {username='slave'} at 127.0.0.1:52974
I20260709 13:12:47.104081  1272 raft_consensus.cc:606] T 8de23e2629b2451b900a09bfe12dbf86 P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Received request to transfer leadership to TS 22ec8c2439d14f139d5330c8ec45133a
I20260709 13:12:47.109014  1272 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fd271e857f3f41c6bcc5d54aae05f985"
dest_uuid: "efe0ab937a564199977b910104824f30"
mode: GRACEFUL
new_leader_uuid: "bbcbed9750ae416bab78287348e9e196"
 from {username='slave'} at 127.0.0.1:52974
I20260709 13:12:47.109489  1272 raft_consensus.cc:606] T fd271e857f3f41c6bcc5d54aae05f985 P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Received request to transfer leadership to TS bbcbed9750ae416bab78287348e9e196
I20260709 13:12:47.114650  1272 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "250a7a602e924c57b99ecbd2bf900bcc"
dest_uuid: "efe0ab937a564199977b910104824f30"
mode: GRACEFUL
new_leader_uuid: "bbcbed9750ae416bab78287348e9e196"
 from {username='slave'} at 127.0.0.1:52974
I20260709 13:12:47.115119  1272 raft_consensus.cc:606] T 250a7a602e924c57b99ecbd2bf900bcc P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Received request to transfer leadership to TS bbcbed9750ae416bab78287348e9e196
I20260709 13:12:47.128122  1122 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "57aa843159154feb87f476bac96b2dba"
dest_uuid: "22ec8c2439d14f139d5330c8ec45133a"
mode: GRACEFUL
new_leader_uuid: "bbcbed9750ae416bab78287348e9e196"
 from {username='slave'} at 127.0.0.1:56378
I20260709 13:12:47.128652  1122 raft_consensus.cc:606] T 57aa843159154feb87f476bac96b2dba P 22ec8c2439d14f139d5330c8ec45133a [term 1 LEADER]: Received request to transfer leadership to TS bbcbed9750ae416bab78287348e9e196
I20260709 13:12:47.134142  1122 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "98d5edd9b01941898accf5cf97879ab4"
dest_uuid: "22ec8c2439d14f139d5330c8ec45133a"
mode: GRACEFUL
new_leader_uuid: "bbcbed9750ae416bab78287348e9e196"
 from {username='slave'} at 127.0.0.1:56378
I20260709 13:12:47.134806  1122 raft_consensus.cc:606] T 98d5edd9b01941898accf5cf97879ab4 P 22ec8c2439d14f139d5330c8ec45133a [term 1 LEADER]: Received request to transfer leadership to TS bbcbed9750ae416bab78287348e9e196
I20260709 13:12:47.135560  1356 consensus_queue.cc:1048] T 98d5edd9b01941898accf5cf97879ab4 P 22ec8c2439d14f139d5330c8ec45133a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:47.135744  1336 consensus_queue.cc:1048] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P efe0ab937a564199977b910104824f30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:47.143435  1352 raft_consensus.cc:993] T 98d5edd9b01941898accf5cf97879ab4 P 22ec8c2439d14f139d5330c8ec45133a: : Instructing follower bbcbed9750ae416bab78287348e9e196 to start an election
I20260709 13:12:47.143944  1352 raft_consensus.cc:1081] T 98d5edd9b01941898accf5cf97879ab4 P 22ec8c2439d14f139d5330c8ec45133a [term 1 LEADER]: Signalling peer bbcbed9750ae416bab78287348e9e196 to start an election
I20260709 13:12:47.146153  1196 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "98d5edd9b01941898accf5cf97879ab4"
dest_uuid: "bbcbed9750ae416bab78287348e9e196"
 from {username='slave'} at 127.0.0.1:50980
I20260709 13:12:47.148624  1196 raft_consensus.cc:493] T 98d5edd9b01941898accf5cf97879ab4 P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:47.149102  1196 raft_consensus.cc:3060] T 98d5edd9b01941898accf5cf97879ab4 P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:47.149825  1321 consensus_queue.cc:1048] T 98d5edd9b01941898accf5cf97879ab4 P 22ec8c2439d14f139d5330c8ec45133a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:47.159796  1196 raft_consensus.cc:515] T 98d5edd9b01941898accf5cf97879ab4 P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:47.160913  1341 consensus_queue.cc:1048] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P efe0ab937a564199977b910104824f30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:47.165943  1352 consensus_queue.cc:1048] T 9d35b87a150449d5b2adefeb4b241fd2 P 22ec8c2439d14f139d5330c8ec45133a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:47.168473  1196 leader_election.cc:290] T 98d5edd9b01941898accf5cf97879ab4 P bbcbed9750ae416bab78287348e9e196 [CANDIDATE]: Term 2 election: Requested vote from peers 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943), efe0ab937a564199977b910104824f30 (127.21.63.3:33527)
I20260709 13:12:47.169211  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d5edd9b01941898accf5cf97879ab4" candidate_uuid: "bbcbed9750ae416bab78287348e9e196" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "22ec8c2439d14f139d5330c8ec45133a"
I20260709 13:12:47.169818  1122 raft_consensus.cc:3055] T 98d5edd9b01941898accf5cf97879ab4 P 22ec8c2439d14f139d5330c8ec45133a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:47.170133  1122 raft_consensus.cc:740] T 98d5edd9b01941898accf5cf97879ab4 P 22ec8c2439d14f139d5330c8ec45133a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 22ec8c2439d14f139d5330c8ec45133a, State: Running, Role: LEADER
I20260709 13:12:47.170925  1122 consensus_queue.cc:260] T 98d5edd9b01941898accf5cf97879ab4 P 22ec8c2439d14f139d5330c8ec45133a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:47.171998  1122 raft_consensus.cc:3060] T 98d5edd9b01941898accf5cf97879ab4 P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:47.172964  1271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d5edd9b01941898accf5cf97879ab4" candidate_uuid: "bbcbed9750ae416bab78287348e9e196" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efe0ab937a564199977b910104824f30"
I20260709 13:12:47.173465  1271 raft_consensus.cc:3060] T 98d5edd9b01941898accf5cf97879ab4 P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:47.178817  1353 consensus_queue.cc:1048] T 89400744bdfc474a805d19fe866d401b P bbcbed9750ae416bab78287348e9e196 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:47.187173  1271 raft_consensus.cc:2468] T 98d5edd9b01941898accf5cf97879ab4 P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbcbed9750ae416bab78287348e9e196 in term 2.
I20260709 13:12:47.192147  1318 raft_consensus.cc:993] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P efe0ab937a564199977b910104824f30: : Instructing follower 22ec8c2439d14f139d5330c8ec45133a to start an election
I20260709 13:12:47.192770  1318 raft_consensus.cc:1081] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Signalling peer 22ec8c2439d14f139d5330c8ec45133a to start an election
I20260709 13:12:47.198521  1122 raft_consensus.cc:2468] T 98d5edd9b01941898accf5cf97879ab4 P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbcbed9750ae416bab78287348e9e196 in term 2.
I20260709 13:12:47.192297  1353 raft_consensus.cc:993] T 89400744bdfc474a805d19fe866d401b P bbcbed9750ae416bab78287348e9e196: : Instructing follower 22ec8c2439d14f139d5330c8ec45133a to start an election
I20260709 13:12:47.199926  1162 leader_election.cc:304] T 98d5edd9b01941898accf5cf97879ab4 P bbcbed9750ae416bab78287348e9e196 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bbcbed9750ae416bab78287348e9e196, efe0ab937a564199977b910104824f30; no voters: 
I20260709 13:12:47.200240  1121 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8ad7ce1d2ca94ba2bc0a82302a9eee4e"
dest_uuid: "22ec8c2439d14f139d5330c8ec45133a"
 from {username='slave'} at 127.0.0.1:51800
I20260709 13:12:47.200977  1353 raft_consensus.cc:1081] T 89400744bdfc474a805d19fe866d401b P bbcbed9750ae416bab78287348e9e196 [term 1 LEADER]: Signalling peer 22ec8c2439d14f139d5330c8ec45133a to start an election
I20260709 13:12:47.201051  1121 raft_consensus.cc:493] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:47.201670  1121 raft_consensus.cc:3060] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:47.202232  1353 raft_consensus.cc:2804] T 98d5edd9b01941898accf5cf97879ab4 P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:47.202728  1353 raft_consensus.cc:697] T 98d5edd9b01941898accf5cf97879ab4 P bbcbed9750ae416bab78287348e9e196 [term 2 LEADER]: Becoming Leader. State: Replica: bbcbed9750ae416bab78287348e9e196, State: Running, Role: LEADER
I20260709 13:12:47.203568  1353 consensus_queue.cc:237] T 98d5edd9b01941898accf5cf97879ab4 P bbcbed9750ae416bab78287348e9e196 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:47.205471  1356 consensus_queue.cc:1048] T 9d35b87a150449d5b2adefeb4b241fd2 P 22ec8c2439d14f139d5330c8ec45133a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:47.200395  1272 raft_consensus.cc:3060] T 57aa843159154feb87f476bac96b2dba P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:47.212982  1356 consensus_queue.cc:1048] T 57aa843159154feb87f476bac96b2dba P 22ec8c2439d14f139d5330c8ec45133a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:47.213129  1121 raft_consensus.cc:515] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:47.220005  1122 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "89400744bdfc474a805d19fe866d401b"
dest_uuid: "22ec8c2439d14f139d5330c8ec45133a"
 from {username='slave'} at 127.0.0.1:51804
I20260709 13:12:47.221233  1356 consensus_queue.cc:1048] T 57aa843159154feb87f476bac96b2dba P 22ec8c2439d14f139d5330c8ec45133a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:47.225041  1122 raft_consensus.cc:493] T 89400744bdfc474a805d19fe866d401b P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:47.225492  1122 raft_consensus.cc:3060] T 89400744bdfc474a805d19fe866d401b P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:47.222651  1196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad7ce1d2ca94ba2bc0a82302a9eee4e" candidate_uuid: "22ec8c2439d14f139d5330c8ec45133a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bbcbed9750ae416bab78287348e9e196"
I20260709 13:12:47.222992  1121 leader_election.cc:290] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 2 election: Requested vote from peers bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515), efe0ab937a564199977b910104824f30 (127.21.63.3:33527)
I20260709 13:12:47.223564  1272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ad7ce1d2ca94ba2bc0a82302a9eee4e" candidate_uuid: "22ec8c2439d14f139d5330c8ec45133a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efe0ab937a564199977b910104824f30"
I20260709 13:12:47.229070  1272 raft_consensus.cc:3055] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:47.229696  1196 raft_consensus.cc:3060] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:47.239404  1021 catalog_manager.cc:5697] T 98d5edd9b01941898accf5cf97879ab4 P bbcbed9750ae416bab78287348e9e196 reported cstate change: term changed from 1 to 2, leader changed from 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1) to bbcbed9750ae416bab78287348e9e196 (127.21.63.2). New cstate: current_term: 2 leader_uuid: "bbcbed9750ae416bab78287348e9e196" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:47.229437  1272 raft_consensus.cc:740] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efe0ab937a564199977b910104824f30, State: Running, Role: LEADER
I20260709 13:12:47.245215  1196 raft_consensus.cc:2468] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22ec8c2439d14f139d5330c8ec45133a in term 2.
I20260709 13:12:47.246075  1272 consensus_queue.cc:260] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P efe0ab937a564199977b910104824f30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:47.246464  1391 consensus_queue.cc:1048] T 89400744bdfc474a805d19fe866d401b P bbcbed9750ae416bab78287348e9e196 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:47.247548  1272 raft_consensus.cc:3060] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:47.244745  1122 raft_consensus.cc:515] T 89400744bdfc474a805d19fe866d401b P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } } peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } }
I20260709 13:12:47.249390  1088 leader_election.cc:304] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22ec8c2439d14f139d5330c8ec45133a, bbcbed9750ae416bab78287348e9e196; no voters: 
I20260709 13:12:47.250515  1321 raft_consensus.cc:2804] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:47.250990  1321 raft_consensus.cc:697] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P 22ec8c2439d14f139d5330c8ec45133a [term 2 LEADER]: Becoming Leader. State: Replica: 22ec8c2439d14f139d5330c8ec45133a, State: Running, Role: LEADER
I20260709 13:12:47.251794  1321 consensus_queue.cc:237] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P 22ec8c2439d14f139d5330c8ec45133a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:47.252408  1122 leader_election.cc:290] T 89400744bdfc474a805d19fe866d401b P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 2 election: Requested vote from peers bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515), efe0ab937a564199977b910104824f30 (127.21.63.3:33527)
I20260709 13:12:47.259105  1271 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89400744bdfc474a805d19fe866d401b" candidate_uuid: "22ec8c2439d14f139d5330c8ec45133a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efe0ab937a564199977b910104824f30"
I20260709 13:12:47.260821  1352 raft_consensus.cc:993] T 57aa843159154feb87f476bac96b2dba P 22ec8c2439d14f139d5330c8ec45133a: : Instructing follower bbcbed9750ae416bab78287348e9e196 to start an election
I20260709 13:12:47.261411  1346 raft_consensus.cc:1081] T 57aa843159154feb87f476bac96b2dba P 22ec8c2439d14f139d5330c8ec45133a [term 1 LEADER]: Signalling peer bbcbed9750ae416bab78287348e9e196 to start an election
I20260709 13:12:47.264166  1198 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "57aa843159154feb87f476bac96b2dba"
dest_uuid: "bbcbed9750ae416bab78287348e9e196"
 from {username='slave'} at 127.0.0.1:50980
I20260709 13:12:47.264809  1198 raft_consensus.cc:493] T 57aa843159154feb87f476bac96b2dba P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:47.265230  1198 raft_consensus.cc:3060] T 57aa843159154feb87f476bac96b2dba P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:47.265688  1196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89400744bdfc474a805d19fe866d401b" candidate_uuid: "22ec8c2439d14f139d5330c8ec45133a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bbcbed9750ae416bab78287348e9e196"
I20260709 13:12:47.266333  1196 raft_consensus.cc:3055] T 89400744bdfc474a805d19fe866d401b P bbcbed9750ae416bab78287348e9e196 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:47.266646  1196 raft_consensus.cc:740] T 89400744bdfc474a805d19fe866d401b P bbcbed9750ae416bab78287348e9e196 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bbcbed9750ae416bab78287348e9e196, State: Running, Role: LEADER
I20260709 13:12:47.267417  1196 consensus_queue.cc:260] T 89400744bdfc474a805d19fe866d401b P bbcbed9750ae416bab78287348e9e196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } } peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } }
I20260709 13:12:47.268532  1196 raft_consensus.cc:3060] T 89400744bdfc474a805d19fe866d401b P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:47.278929  1336 raft_consensus.cc:493] T 57aa843159154feb87f476bac96b2dba P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 22ec8c2439d14f139d5330c8ec45133a)
I20260709 13:12:47.279534  1336 raft_consensus.cc:515] T 57aa843159154feb87f476bac96b2dba P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:47.281656  1196 raft_consensus.cc:2468] T 89400744bdfc474a805d19fe866d401b P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22ec8c2439d14f139d5330c8ec45133a in term 2.
I20260709 13:12:47.282872  1088 leader_election.cc:304] T 89400744bdfc474a805d19fe866d401b P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 22ec8c2439d14f139d5330c8ec45133a, bbcbed9750ae416bab78287348e9e196; no voters: efe0ab937a564199977b910104824f30
I20260709 13:12:47.282902  1272 raft_consensus.cc:2468] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22ec8c2439d14f139d5330c8ec45133a in term 2.
I20260709 13:12:47.283936  1321 raft_consensus.cc:2804] T 89400744bdfc474a805d19fe866d401b P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:47.284521  1321 raft_consensus.cc:697] T 89400744bdfc474a805d19fe866d401b P 22ec8c2439d14f139d5330c8ec45133a [term 2 LEADER]: Becoming Leader. State: Replica: 22ec8c2439d14f139d5330c8ec45133a, State: Running, Role: LEADER
I20260709 13:12:47.284876  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57aa843159154feb87f476bac96b2dba" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "22ec8c2439d14f139d5330c8ec45133a" is_pre_election: true
I20260709 13:12:47.285204  1336 leader_election.cc:290] T 57aa843159154feb87f476bac96b2dba P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943), bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515)
I20260709 13:12:47.286451  1197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57aa843159154feb87f476bac96b2dba" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "bbcbed9750ae416bab78287348e9e196" is_pre_election: true
I20260709 13:12:47.286080  1321 consensus_queue.cc:237] T 89400744bdfc474a805d19fe866d401b P 22ec8c2439d14f139d5330c8ec45133a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } } peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } }
I20260709 13:12:47.287776  1237 leader_election.cc:304] T 57aa843159154feb87f476bac96b2dba P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: efe0ab937a564199977b910104824f30; no voters: 22ec8c2439d14f139d5330c8ec45133a, bbcbed9750ae416bab78287348e9e196
I20260709 13:12:47.288610  1336 raft_consensus.cc:2749] T 57aa843159154feb87f476bac96b2dba P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 13:12:47.289619  1019 catalog_manager.cc:5697] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P 22ec8c2439d14f139d5330c8ec45133a reported cstate change: term changed from 1 to 2, leader changed from efe0ab937a564199977b910104824f30 (127.21.63.3) to 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1). New cstate: current_term: 2 leader_uuid: "22ec8c2439d14f139d5330c8ec45133a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:47.290269  1198 raft_consensus.cc:515] T 57aa843159154feb87f476bac96b2dba P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:47.292245  1198 leader_election.cc:290] T 57aa843159154feb87f476bac96b2dba P bbcbed9750ae416bab78287348e9e196 [CANDIDATE]: Term 2 election: Requested vote from peers 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943), efe0ab937a564199977b910104824f30 (127.21.63.3:33527)
I20260709 13:12:47.293452  1269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57aa843159154feb87f476bac96b2dba" candidate_uuid: "bbcbed9750ae416bab78287348e9e196" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efe0ab937a564199977b910104824f30"
I20260709 13:12:47.294346  1269 raft_consensus.cc:3060] T 57aa843159154feb87f476bac96b2dba P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:47.296942  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57aa843159154feb87f476bac96b2dba" candidate_uuid: "bbcbed9750ae416bab78287348e9e196" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "22ec8c2439d14f139d5330c8ec45133a"
I20260709 13:12:47.297629  1122 raft_consensus.cc:3055] T 57aa843159154feb87f476bac96b2dba P 22ec8c2439d14f139d5330c8ec45133a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:47.297933  1122 raft_consensus.cc:740] T 57aa843159154feb87f476bac96b2dba P 22ec8c2439d14f139d5330c8ec45133a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 22ec8c2439d14f139d5330c8ec45133a, State: Running, Role: LEADER
I20260709 13:12:47.298837  1122 consensus_queue.cc:260] T 57aa843159154feb87f476bac96b2dba P 22ec8c2439d14f139d5330c8ec45133a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:47.300094  1122 raft_consensus.cc:3060] T 57aa843159154feb87f476bac96b2dba P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:47.301965  1269 raft_consensus.cc:2468] T 57aa843159154feb87f476bac96b2dba P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbcbed9750ae416bab78287348e9e196 in term 2.
I20260709 13:12:47.304337  1162 leader_election.cc:304] T 57aa843159154feb87f476bac96b2dba P bbcbed9750ae416bab78287348e9e196 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bbcbed9750ae416bab78287348e9e196, efe0ab937a564199977b910104824f30; no voters: 
I20260709 13:12:47.305107  1391 raft_consensus.cc:2804] T 57aa843159154feb87f476bac96b2dba P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:47.305546  1391 raft_consensus.cc:697] T 57aa843159154feb87f476bac96b2dba P bbcbed9750ae416bab78287348e9e196 [term 2 LEADER]: Becoming Leader. State: Replica: bbcbed9750ae416bab78287348e9e196, State: Running, Role: LEADER
I20260709 13:12:47.306753  1122 raft_consensus.cc:2468] T 57aa843159154feb87f476bac96b2dba P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbcbed9750ae416bab78287348e9e196 in term 2.
I20260709 13:12:47.306386  1391 consensus_queue.cc:237] T 57aa843159154feb87f476bac96b2dba P bbcbed9750ae416bab78287348e9e196 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:47.312901  1021 catalog_manager.cc:5697] T 89400744bdfc474a805d19fe866d401b P 22ec8c2439d14f139d5330c8ec45133a reported cstate change: term changed from 1 to 2, leader changed from bbcbed9750ae416bab78287348e9e196 (127.21.63.2) to 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1). New cstate: current_term: 2 leader_uuid: "22ec8c2439d14f139d5330c8ec45133a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:47.321089  1019 catalog_manager.cc:5697] T 57aa843159154feb87f476bac96b2dba P bbcbed9750ae416bab78287348e9e196 reported cstate change: term changed from 1 to 2, leader changed from 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1) to bbcbed9750ae416bab78287348e9e196 (127.21.63.2). New cstate: current_term: 2 leader_uuid: "bbcbed9750ae416bab78287348e9e196" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:47.454442  1318 raft_consensus.cc:993] T 8de23e2629b2451b900a09bfe12dbf86 P efe0ab937a564199977b910104824f30: : Instructing follower 22ec8c2439d14f139d5330c8ec45133a to start an election
I20260709 13:12:47.454907  1336 raft_consensus.cc:1081] T 8de23e2629b2451b900a09bfe12dbf86 P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Signalling peer 22ec8c2439d14f139d5330c8ec45133a to start an election
I20260709 13:12:47.456455  1122 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8de23e2629b2451b900a09bfe12dbf86"
dest_uuid: "22ec8c2439d14f139d5330c8ec45133a"
 from {username='slave'} at 127.0.0.1:51800
I20260709 13:12:47.456998  1122 raft_consensus.cc:493] T 8de23e2629b2451b900a09bfe12dbf86 P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:47.457336  1122 raft_consensus.cc:3060] T 8de23e2629b2451b900a09bfe12dbf86 P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:47.462783  1122 raft_consensus.cc:515] T 8de23e2629b2451b900a09bfe12dbf86 P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:47.464558  1122 leader_election.cc:290] T 8de23e2629b2451b900a09bfe12dbf86 P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 2 election: Requested vote from peers bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515), efe0ab937a564199977b910104824f30 (127.21.63.3:33527)
I20260709 13:12:47.465705  1198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8de23e2629b2451b900a09bfe12dbf86" candidate_uuid: "22ec8c2439d14f139d5330c8ec45133a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bbcbed9750ae416bab78287348e9e196"
I20260709 13:12:47.465889  1269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8de23e2629b2451b900a09bfe12dbf86" candidate_uuid: "22ec8c2439d14f139d5330c8ec45133a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efe0ab937a564199977b910104824f30"
I20260709 13:12:47.466267  1198 raft_consensus.cc:3060] T 8de23e2629b2451b900a09bfe12dbf86 P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:47.466504  1269 raft_consensus.cc:3055] T 8de23e2629b2451b900a09bfe12dbf86 P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:47.466876  1269 raft_consensus.cc:740] T 8de23e2629b2451b900a09bfe12dbf86 P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efe0ab937a564199977b910104824f30, State: Running, Role: LEADER
I20260709 13:12:47.467720  1269 consensus_queue.cc:260] T 8de23e2629b2451b900a09bfe12dbf86 P efe0ab937a564199977b910104824f30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:47.468818  1269 raft_consensus.cc:3060] T 8de23e2629b2451b900a09bfe12dbf86 P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:47.471992  1198 raft_consensus.cc:2468] T 8de23e2629b2451b900a09bfe12dbf86 P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22ec8c2439d14f139d5330c8ec45133a in term 2.
I20260709 13:12:47.473385  1088 leader_election.cc:304] T 8de23e2629b2451b900a09bfe12dbf86 P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22ec8c2439d14f139d5330c8ec45133a, bbcbed9750ae416bab78287348e9e196; no voters: 
I20260709 13:12:47.474334  1321 raft_consensus.cc:2804] T 8de23e2629b2451b900a09bfe12dbf86 P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:47.474459  1269 raft_consensus.cc:2468] T 8de23e2629b2451b900a09bfe12dbf86 P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22ec8c2439d14f139d5330c8ec45133a in term 2.
I20260709 13:12:47.474859  1321 raft_consensus.cc:697] T 8de23e2629b2451b900a09bfe12dbf86 P 22ec8c2439d14f139d5330c8ec45133a [term 2 LEADER]: Becoming Leader. State: Replica: 22ec8c2439d14f139d5330c8ec45133a, State: Running, Role: LEADER
I20260709 13:12:47.475723  1321 consensus_queue.cc:237] T 8de23e2629b2451b900a09bfe12dbf86 P 22ec8c2439d14f139d5330c8ec45133a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:47.483196  1019 catalog_manager.cc:5697] T 8de23e2629b2451b900a09bfe12dbf86 P 22ec8c2439d14f139d5330c8ec45133a reported cstate change: term changed from 1 to 2, leader changed from efe0ab937a564199977b910104824f30 (127.21.63.3) to 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1). New cstate: current_term: 2 leader_uuid: "22ec8c2439d14f139d5330c8ec45133a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:47.546324  1336 consensus_queue.cc:1048] T fd271e857f3f41c6bcc5d54aae05f985 P efe0ab937a564199977b910104824f30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:47.557050  1318 consensus_queue.cc:1048] T fd271e857f3f41c6bcc5d54aae05f985 P efe0ab937a564199977b910104824f30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:47.565140  1341 raft_consensus.cc:993] T fd271e857f3f41c6bcc5d54aae05f985 P efe0ab937a564199977b910104824f30: : Instructing follower bbcbed9750ae416bab78287348e9e196 to start an election
I20260709 13:12:47.565593  1318 raft_consensus.cc:1081] T fd271e857f3f41c6bcc5d54aae05f985 P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Signalling peer bbcbed9750ae416bab78287348e9e196 to start an election
I20260709 13:12:47.567054  1198 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fd271e857f3f41c6bcc5d54aae05f985"
dest_uuid: "bbcbed9750ae416bab78287348e9e196"
 from {username='slave'} at 127.0.0.1:50976
I20260709 13:12:47.567574  1198 raft_consensus.cc:493] T fd271e857f3f41c6bcc5d54aae05f985 P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:47.567921  1198 raft_consensus.cc:3060] T fd271e857f3f41c6bcc5d54aae05f985 P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:47.573050  1198 raft_consensus.cc:515] T fd271e857f3f41c6bcc5d54aae05f985 P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:47.575811  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd271e857f3f41c6bcc5d54aae05f985" candidate_uuid: "bbcbed9750ae416bab78287348e9e196" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "22ec8c2439d14f139d5330c8ec45133a"
I20260709 13:12:47.576438  1122 raft_consensus.cc:3060] T fd271e857f3f41c6bcc5d54aae05f985 P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:47.574857  1198 leader_election.cc:290] T fd271e857f3f41c6bcc5d54aae05f985 P bbcbed9750ae416bab78287348e9e196 [CANDIDATE]: Term 2 election: Requested vote from peers 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943), efe0ab937a564199977b910104824f30 (127.21.63.3:33527)
I20260709 13:12:47.579581  1269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd271e857f3f41c6bcc5d54aae05f985" candidate_uuid: "bbcbed9750ae416bab78287348e9e196" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efe0ab937a564199977b910104824f30"
I20260709 13:12:47.580238  1269 raft_consensus.cc:3055] T fd271e857f3f41c6bcc5d54aae05f985 P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:47.580545  1269 raft_consensus.cc:740] T fd271e857f3f41c6bcc5d54aae05f985 P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efe0ab937a564199977b910104824f30, State: Running, Role: LEADER
I20260709 13:12:47.581558  1269 consensus_queue.cc:260] T fd271e857f3f41c6bcc5d54aae05f985 P efe0ab937a564199977b910104824f30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:47.583633  1269 raft_consensus.cc:3060] T fd271e857f3f41c6bcc5d54aae05f985 P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:47.584235  1122 raft_consensus.cc:2468] T fd271e857f3f41c6bcc5d54aae05f985 P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbcbed9750ae416bab78287348e9e196 in term 2.
I20260709 13:12:47.585575  1161 leader_election.cc:304] T fd271e857f3f41c6bcc5d54aae05f985 P bbcbed9750ae416bab78287348e9e196 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22ec8c2439d14f139d5330c8ec45133a, bbcbed9750ae416bab78287348e9e196; no voters: 
I20260709 13:12:47.586277  1391 raft_consensus.cc:2804] T fd271e857f3f41c6bcc5d54aae05f985 P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:47.586659  1391 raft_consensus.cc:697] T fd271e857f3f41c6bcc5d54aae05f985 P bbcbed9750ae416bab78287348e9e196 [term 2 LEADER]: Becoming Leader. State: Replica: bbcbed9750ae416bab78287348e9e196, State: Running, Role: LEADER
I20260709 13:12:47.587438  1391 consensus_queue.cc:237] T fd271e857f3f41c6bcc5d54aae05f985 P bbcbed9750ae416bab78287348e9e196 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:47.589218  1269 raft_consensus.cc:2468] T fd271e857f3f41c6bcc5d54aae05f985 P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbcbed9750ae416bab78287348e9e196 in term 2.
I20260709 13:12:47.593811  1019 catalog_manager.cc:5697] T fd271e857f3f41c6bcc5d54aae05f985 P bbcbed9750ae416bab78287348e9e196 reported cstate change: term changed from 1 to 2, leader changed from efe0ab937a564199977b910104824f30 (127.21.63.3) to bbcbed9750ae416bab78287348e9e196 (127.21.63.2). New cstate: current_term: 2 leader_uuid: "bbcbed9750ae416bab78287348e9e196" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:47.633414  1122 raft_consensus.cc:1275] T 98d5edd9b01941898accf5cf97879ab4 P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Refusing update from remote peer bbcbed9750ae416bab78287348e9e196: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:47.634689  1391 consensus_queue.cc:1048] T 98d5edd9b01941898accf5cf97879ab4 P bbcbed9750ae416bab78287348e9e196 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:47.642625  1269 raft_consensus.cc:1275] T 98d5edd9b01941898accf5cf97879ab4 P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Refusing update from remote peer bbcbed9750ae416bab78287348e9e196: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:47.644387  1353 consensus_queue.cc:1048] T 98d5edd9b01941898accf5cf97879ab4 P bbcbed9750ae416bab78287348e9e196 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:47.676994  1318 raft_consensus.cc:993] T 250a7a602e924c57b99ecbd2bf900bcc P efe0ab937a564199977b910104824f30: : Instructing follower bbcbed9750ae416bab78287348e9e196 to start an election
I20260709 13:12:47.677523  1399 raft_consensus.cc:1081] T 250a7a602e924c57b99ecbd2bf900bcc P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Signalling peer bbcbed9750ae416bab78287348e9e196 to start an election
I20260709 13:12:47.679177  1197 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "250a7a602e924c57b99ecbd2bf900bcc"
dest_uuid: "bbcbed9750ae416bab78287348e9e196"
 from {username='slave'} at 127.0.0.1:50976
I20260709 13:12:47.679664  1197 raft_consensus.cc:493] T 250a7a602e924c57b99ecbd2bf900bcc P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:47.679994  1197 raft_consensus.cc:3060] T 250a7a602e924c57b99ecbd2bf900bcc P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:47.686420  1197 raft_consensus.cc:515] T 250a7a602e924c57b99ecbd2bf900bcc P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:47.687968  1197 leader_election.cc:290] T 250a7a602e924c57b99ecbd2bf900bcc P bbcbed9750ae416bab78287348e9e196 [CANDIDATE]: Term 2 election: Requested vote from peers 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943), efe0ab937a564199977b910104824f30 (127.21.63.3:33527)
I20260709 13:12:47.688964  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250a7a602e924c57b99ecbd2bf900bcc" candidate_uuid: "bbcbed9750ae416bab78287348e9e196" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "22ec8c2439d14f139d5330c8ec45133a"
I20260709 13:12:47.689221  1269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "250a7a602e924c57b99ecbd2bf900bcc" candidate_uuid: "bbcbed9750ae416bab78287348e9e196" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efe0ab937a564199977b910104824f30"
I20260709 13:12:47.689747  1122 raft_consensus.cc:3060] T 250a7a602e924c57b99ecbd2bf900bcc P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:47.689872  1269 raft_consensus.cc:3055] T 250a7a602e924c57b99ecbd2bf900bcc P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:47.690322  1269 raft_consensus.cc:740] T 250a7a602e924c57b99ecbd2bf900bcc P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efe0ab937a564199977b910104824f30, State: Running, Role: LEADER
I20260709 13:12:47.691253  1269 consensus_queue.cc:260] T 250a7a602e924c57b99ecbd2bf900bcc P efe0ab937a564199977b910104824f30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:47.692608  1269 raft_consensus.cc:3060] T 250a7a602e924c57b99ecbd2bf900bcc P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:47.697816  1122 raft_consensus.cc:2468] T 250a7a602e924c57b99ecbd2bf900bcc P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbcbed9750ae416bab78287348e9e196 in term 2.
I20260709 13:12:47.699571  1161 leader_election.cc:304] T 250a7a602e924c57b99ecbd2bf900bcc P bbcbed9750ae416bab78287348e9e196 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22ec8c2439d14f139d5330c8ec45133a, bbcbed9750ae416bab78287348e9e196; no voters: 
I20260709 13:12:47.700383  1353 raft_consensus.cc:2804] T 250a7a602e924c57b99ecbd2bf900bcc P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:47.700655  1269 raft_consensus.cc:2468] T 250a7a602e924c57b99ecbd2bf900bcc P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbcbed9750ae416bab78287348e9e196 in term 2.
I20260709 13:12:47.701094  1353 raft_consensus.cc:697] T 250a7a602e924c57b99ecbd2bf900bcc P bbcbed9750ae416bab78287348e9e196 [term 2 LEADER]: Becoming Leader. State: Replica: bbcbed9750ae416bab78287348e9e196, State: Running, Role: LEADER
I20260709 13:12:47.702255  1353 consensus_queue.cc:237] T 250a7a602e924c57b99ecbd2bf900bcc P bbcbed9750ae416bab78287348e9e196 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:47.703646  1197 raft_consensus.cc:1275] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Refusing update from remote peer 22ec8c2439d14f139d5330c8ec45133a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:47.705423  1321 consensus_queue.cc:1048] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P 22ec8c2439d14f139d5330c8ec45133a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:47.710104  1019 catalog_manager.cc:5697] T 250a7a602e924c57b99ecbd2bf900bcc P bbcbed9750ae416bab78287348e9e196 reported cstate change: term changed from 1 to 2, leader changed from efe0ab937a564199977b910104824f30 (127.21.63.3) to bbcbed9750ae416bab78287348e9e196 (127.21.63.2). New cstate: current_term: 2 leader_uuid: "bbcbed9750ae416bab78287348e9e196" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:47.713253  1269 raft_consensus.cc:1275] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Refusing update from remote peer 22ec8c2439d14f139d5330c8ec45133a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:47.714499  1321 consensus_queue.cc:1048] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P 22ec8c2439d14f139d5330c8ec45133a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:47.739850  1197 raft_consensus.cc:1275] T 89400744bdfc474a805d19fe866d401b P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Refusing update from remote peer 22ec8c2439d14f139d5330c8ec45133a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:47.741297  1321 consensus_queue.cc:1048] T 89400744bdfc474a805d19fe866d401b P 22ec8c2439d14f139d5330c8ec45133a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:47.750097  1269 raft_consensus.cc:3060] T 89400744bdfc474a805d19fe866d401b P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:47.757309  1269 raft_consensus.cc:1275] T 89400744bdfc474a805d19fe866d401b P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Refusing update from remote peer 22ec8c2439d14f139d5330c8ec45133a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:47.759029  1321 consensus_queue.cc:1048] T 89400744bdfc474a805d19fe866d401b P 22ec8c2439d14f139d5330c8ec45133a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:47.866873  1122 raft_consensus.cc:1275] T 57aa843159154feb87f476bac96b2dba P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Refusing update from remote peer bbcbed9750ae416bab78287348e9e196: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:47.868094  1353 consensus_queue.cc:1048] T 57aa843159154feb87f476bac96b2dba P bbcbed9750ae416bab78287348e9e196 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:47.875578  1269 raft_consensus.cc:1275] T 57aa843159154feb87f476bac96b2dba P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Refusing update from remote peer bbcbed9750ae416bab78287348e9e196: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:47.877498  1392 consensus_queue.cc:1048] T 57aa843159154feb87f476bac96b2dba P bbcbed9750ae416bab78287348e9e196 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:47.942725  1269 raft_consensus.cc:1275] T 8de23e2629b2451b900a09bfe12dbf86 P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Refusing update from remote peer 22ec8c2439d14f139d5330c8ec45133a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:47.943967  1321 consensus_queue.cc:1048] T 8de23e2629b2451b900a09bfe12dbf86 P 22ec8c2439d14f139d5330c8ec45133a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:47.952168  1197 raft_consensus.cc:1275] T 8de23e2629b2451b900a09bfe12dbf86 P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Refusing update from remote peer 22ec8c2439d14f139d5330c8ec45133a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:47.953744  1321 consensus_queue.cc:1048] T 8de23e2629b2451b900a09bfe12dbf86 P 22ec8c2439d14f139d5330c8ec45133a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:48.052999  1122 raft_consensus.cc:1275] T fd271e857f3f41c6bcc5d54aae05f985 P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Refusing update from remote peer bbcbed9750ae416bab78287348e9e196: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:48.054296  1392 consensus_queue.cc:1048] T fd271e857f3f41c6bcc5d54aae05f985 P bbcbed9750ae416bab78287348e9e196 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:48.061641  1269 raft_consensus.cc:1275] T fd271e857f3f41c6bcc5d54aae05f985 P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Refusing update from remote peer bbcbed9750ae416bab78287348e9e196: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:48.063186  1319 consensus_queue.cc:1048] T fd271e857f3f41c6bcc5d54aae05f985 P bbcbed9750ae416bab78287348e9e196 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:48.121613  1269 raft_consensus.cc:1275] T 250a7a602e924c57b99ecbd2bf900bcc P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Refusing update from remote peer bbcbed9750ae416bab78287348e9e196: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:48.123013  1319 consensus_queue.cc:1048] T 250a7a602e924c57b99ecbd2bf900bcc P bbcbed9750ae416bab78287348e9e196 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:48.131822  1122 raft_consensus.cc:1275] T 250a7a602e924c57b99ecbd2bf900bcc P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Refusing update from remote peer bbcbed9750ae416bab78287348e9e196: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:48.133637  1391 consensus_queue.cc:1048] T 250a7a602e924c57b99ecbd2bf900bcc P bbcbed9750ae416bab78287348e9e196 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:50.177502  1019 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59232:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 13:12:50.179636  1019 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 13:12:50.229167  1109 tablet_service.cc:1511] Processing CreateTablet for tablet 0be8303fce99487c9cb34506b3181363 (DEFAULT_TABLE table=soft_deleted_table [id=b518086dfff74502b43f9ea7c5d9299a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:12:50.229687  1181 tablet_service.cc:1511] Processing CreateTablet for tablet 0be8303fce99487c9cb34506b3181363 (DEFAULT_TABLE table=soft_deleted_table [id=b518086dfff74502b43f9ea7c5d9299a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:12:50.230834  1109 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0be8303fce99487c9cb34506b3181363. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:50.230975  1181 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0be8303fce99487c9cb34506b3181363. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:50.231945  1256 tablet_service.cc:1511] Processing CreateTablet for tablet 0be8303fce99487c9cb34506b3181363 (DEFAULT_TABLE table=soft_deleted_table [id=b518086dfff74502b43f9ea7c5d9299a]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 13:12:50.233184  1256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0be8303fce99487c9cb34506b3181363. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:50.233299  1108 tablet_service.cc:1511] Processing CreateTablet for tablet 4bf47f99c081476eabfaeb56515e0d79 (DEFAULT_TABLE table=soft_deleted_table [id=b518086dfff74502b43f9ea7c5d9299a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:12:50.234740  1108 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4bf47f99c081476eabfaeb56515e0d79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:50.246254  1180 tablet_service.cc:1511] Processing CreateTablet for tablet 4bf47f99c081476eabfaeb56515e0d79 (DEFAULT_TABLE table=soft_deleted_table [id=b518086dfff74502b43f9ea7c5d9299a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:12:50.247746  1180 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4bf47f99c081476eabfaeb56515e0d79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:50.250046  1258 tablet_service.cc:1511] Processing CreateTablet for tablet 4bf47f99c081476eabfaeb56515e0d79 (DEFAULT_TABLE table=soft_deleted_table [id=b518086dfff74502b43f9ea7c5d9299a]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 13:12:50.251523  1258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4bf47f99c081476eabfaeb56515e0d79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:50.256207  1413 tablet_bootstrap.cc:492] T 0be8303fce99487c9cb34506b3181363 P efe0ab937a564199977b910104824f30: Bootstrap starting.
I20260709 13:12:50.264082  1111 tablet_service.cc:1511] Processing CreateTablet for tablet 7ae6ea4d73e243a19c55a262c620bb7d (DEFAULT_TABLE table=soft_deleted_table [id=b518086dfff74502b43f9ea7c5d9299a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:12:50.265779  1111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ae6ea4d73e243a19c55a262c620bb7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:50.271296  1182 tablet_service.cc:1511] Processing CreateTablet for tablet 7ae6ea4d73e243a19c55a262c620bb7d (DEFAULT_TABLE table=soft_deleted_table [id=b518086dfff74502b43f9ea7c5d9299a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:12:50.272851  1182 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ae6ea4d73e243a19c55a262c620bb7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:50.273845  1256 tablet_service.cc:1511] Processing CreateTablet for tablet 7ae6ea4d73e243a19c55a262c620bb7d (DEFAULT_TABLE table=soft_deleted_table [id=b518086dfff74502b43f9ea7c5d9299a]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 13:12:50.275089  1256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ae6ea4d73e243a19c55a262c620bb7d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:50.264067  1413 tablet_bootstrap.cc:654] T 0be8303fce99487c9cb34506b3181363 P efe0ab937a564199977b910104824f30: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:50.287742  1113 tablet_service.cc:1511] Processing CreateTablet for tablet 249a9a4a171d491b9f64f9b263bbb91a (DEFAULT_TABLE table=soft_deleted_table [id=b518086dfff74502b43f9ea7c5d9299a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:12:50.289160  1113 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 249a9a4a171d491b9f64f9b263bbb91a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:50.296770  1415 tablet_bootstrap.cc:492] T 0be8303fce99487c9cb34506b3181363 P 22ec8c2439d14f139d5330c8ec45133a: Bootstrap starting.
I20260709 13:12:50.298027  1180 tablet_service.cc:1511] Processing CreateTablet for tablet 249a9a4a171d491b9f64f9b263bbb91a (DEFAULT_TABLE table=soft_deleted_table [id=b518086dfff74502b43f9ea7c5d9299a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:12:50.299487  1180 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 249a9a4a171d491b9f64f9b263bbb91a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:50.304270  1262 tablet_service.cc:1511] Processing CreateTablet for tablet 249a9a4a171d491b9f64f9b263bbb91a (DEFAULT_TABLE table=soft_deleted_table [id=b518086dfff74502b43f9ea7c5d9299a]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 13:12:50.305734  1262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 249a9a4a171d491b9f64f9b263bbb91a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:50.310345  1413 tablet_bootstrap.cc:492] T 0be8303fce99487c9cb34506b3181363 P efe0ab937a564199977b910104824f30: No bootstrap required, opened a new log
I20260709 13:12:50.310971  1413 ts_tablet_manager.cc:1403] T 0be8303fce99487c9cb34506b3181363 P efe0ab937a564199977b910104824f30: Time spent bootstrapping tablet: real 0.055s	user 0.000s	sys 0.014s
I20260709 13:12:50.313741  1109 tablet_service.cc:1511] Processing CreateTablet for tablet 976112eb8da2479e9a559399bee97dfc (DEFAULT_TABLE table=soft_deleted_table [id=b518086dfff74502b43f9ea7c5d9299a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:12:50.315382  1109 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 976112eb8da2479e9a559399bee97dfc. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 13:12:50.317704  1088 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.2:34515, user_credentials={real_user=slave}} blocked reactor thread for 45156.2us
I20260709 13:12:50.323254  1111 tablet_service.cc:1511] Processing CreateTablet for tablet 5e059664b8d64cc49d217bfcddfc2232 (DEFAULT_TABLE table=soft_deleted_table [id=b518086dfff74502b43f9ea7c5d9299a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:12:50.313839  1413 raft_consensus.cc:359] T 0be8303fce99487c9cb34506b3181363 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.324584  1111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e059664b8d64cc49d217bfcddfc2232. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:50.324862  1413 raft_consensus.cc:385] T 0be8303fce99487c9cb34506b3181363 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:50.325278  1413 raft_consensus.cc:740] T 0be8303fce99487c9cb34506b3181363 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe0ab937a564199977b910104824f30, State: Initialized, Role: FOLLOWER
I20260709 13:12:50.310390  1414 tablet_bootstrap.cc:492] T 0be8303fce99487c9cb34506b3181363 P bbcbed9750ae416bab78287348e9e196: Bootstrap starting.
I20260709 13:12:50.326117  1413 consensus_queue.cc:260] T 0be8303fce99487c9cb34506b3181363 P efe0ab937a564199977b910104824f30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.322042  1256 tablet_service.cc:1511] Processing CreateTablet for tablet 976112eb8da2479e9a559399bee97dfc (DEFAULT_TABLE table=soft_deleted_table [id=b518086dfff74502b43f9ea7c5d9299a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:12:50.328878  1413 ts_tablet_manager.cc:1434] T 0be8303fce99487c9cb34506b3181363 P efe0ab937a564199977b910104824f30: Time spent starting tablet: real 0.018s	user 0.004s	sys 0.004s
I20260709 13:12:50.330036  1256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 976112eb8da2479e9a559399bee97dfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:50.331364  1182 tablet_service.cc:1511] Processing CreateTablet for tablet 976112eb8da2479e9a559399bee97dfc (DEFAULT_TABLE table=soft_deleted_table [id=b518086dfff74502b43f9ea7c5d9299a]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 13:12:50.340010  1182 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 976112eb8da2479e9a559399bee97dfc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:50.337576  1415 tablet_bootstrap.cc:654] T 0be8303fce99487c9cb34506b3181363 P 22ec8c2439d14f139d5330c8ec45133a: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:50.337474  1181 tablet_service.cc:1511] Processing CreateTablet for tablet 5e059664b8d64cc49d217bfcddfc2232 (DEFAULT_TABLE table=soft_deleted_table [id=b518086dfff74502b43f9ea7c5d9299a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:12:50.356484  1181 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e059664b8d64cc49d217bfcddfc2232. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:50.338816  1414 tablet_bootstrap.cc:654] T 0be8303fce99487c9cb34506b3181363 P bbcbed9750ae416bab78287348e9e196: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:50.338502  1258 tablet_service.cc:1511] Processing CreateTablet for tablet 5e059664b8d64cc49d217bfcddfc2232 (DEFAULT_TABLE table=soft_deleted_table [id=b518086dfff74502b43f9ea7c5d9299a]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 13:12:50.362016  1258 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5e059664b8d64cc49d217bfcddfc2232. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:50.343474  1108 tablet_service.cc:1511] Processing CreateTablet for tablet fe6a0e06494f416e80a6ebbec27396bd (DEFAULT_TABLE table=soft_deleted_table [id=b518086dfff74502b43f9ea7c5d9299a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:12:50.365561  1108 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe6a0e06494f416e80a6ebbec27396bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:50.359941  1187 tablet_service.cc:1511] Processing CreateTablet for tablet fe6a0e06494f416e80a6ebbec27396bd (DEFAULT_TABLE table=soft_deleted_table [id=b518086dfff74502b43f9ea7c5d9299a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:12:50.369623  1187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe6a0e06494f416e80a6ebbec27396bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:50.332875  1413 tablet_bootstrap.cc:492] T 4bf47f99c081476eabfaeb56515e0d79 P efe0ab937a564199977b910104824f30: Bootstrap starting.
I20260709 13:12:50.380357  1415 tablet_bootstrap.cc:492] T 0be8303fce99487c9cb34506b3181363 P 22ec8c2439d14f139d5330c8ec45133a: No bootstrap required, opened a new log
I20260709 13:12:50.383121  1415 ts_tablet_manager.cc:1403] T 0be8303fce99487c9cb34506b3181363 P 22ec8c2439d14f139d5330c8ec45133a: Time spent bootstrapping tablet: real 0.087s	user 0.009s	sys 0.005s
W20260709 13:12:50.387153  1298 tablet.cc:2406] T 0be8303fce99487c9cb34506b3181363 P efe0ab937a564199977b910104824f30: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 13:12:50.388850  1413 tablet_bootstrap.cc:654] T 4bf47f99c081476eabfaeb56515e0d79 P efe0ab937a564199977b910104824f30: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:50.389863  1399 raft_consensus.cc:493] T 0be8303fce99487c9cb34506b3181363 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.390368  1399 raft_consensus.cc:515] T 0be8303fce99487c9cb34506b3181363 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.386623  1415 raft_consensus.cc:359] T 0be8303fce99487c9cb34506b3181363 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.392128  1415 raft_consensus.cc:385] T 0be8303fce99487c9cb34506b3181363 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:50.392354  1414 tablet_bootstrap.cc:492] T 0be8303fce99487c9cb34506b3181363 P bbcbed9750ae416bab78287348e9e196: No bootstrap required, opened a new log
I20260709 13:12:50.392481  1415 raft_consensus.cc:740] T 0be8303fce99487c9cb34506b3181363 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22ec8c2439d14f139d5330c8ec45133a, State: Initialized, Role: FOLLOWER
I20260709 13:12:50.392948  1414 ts_tablet_manager.cc:1403] T 0be8303fce99487c9cb34506b3181363 P bbcbed9750ae416bab78287348e9e196: Time spent bootstrapping tablet: real 0.083s	user 0.015s	sys 0.000s
I20260709 13:12:50.393553  1415 consensus_queue.cc:260] T 0be8303fce99487c9cb34506b3181363 P 22ec8c2439d14f139d5330c8ec45133a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.394594  1197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0be8303fce99487c9cb34506b3181363" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbcbed9750ae416bab78287348e9e196" is_pre_election: true
I20260709 13:12:50.396216  1415 ts_tablet_manager.cc:1434] T 0be8303fce99487c9cb34506b3181363 P 22ec8c2439d14f139d5330c8ec45133a: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 13:12:50.392848  1256 tablet_service.cc:1511] Processing CreateTablet for tablet fe6a0e06494f416e80a6ebbec27396bd (DEFAULT_TABLE table=soft_deleted_table [id=b518086dfff74502b43f9ea7c5d9299a]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 13:12:50.404383  1256 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fe6a0e06494f416e80a6ebbec27396bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:50.404420  1262 tablet_service.cc:1511] Processing CreateTablet for tablet 8e4161719e214f1a9836f352271dc985 (DEFAULT_TABLE table=soft_deleted_table [id=b518086dfff74502b43f9ea7c5d9299a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:12:50.406029  1262 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e4161719e214f1a9836f352271dc985. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:50.405926  1111 tablet_service.cc:1511] Processing CreateTablet for tablet ac170d34df16448ea4c965890a7d22a1 (DEFAULT_TABLE table=soft_deleted_table [id=b518086dfff74502b43f9ea7c5d9299a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:12:50.399065  1109 tablet_service.cc:1511] Processing CreateTablet for tablet 8e4161719e214f1a9836f352271dc985 (DEFAULT_TABLE table=soft_deleted_table [id=b518086dfff74502b43f9ea7c5d9299a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:12:50.408582  1414 raft_consensus.cc:359] T 0be8303fce99487c9cb34506b3181363 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.409391  1414 raft_consensus.cc:385] T 0be8303fce99487c9cb34506b3181363 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:50.409660  1414 raft_consensus.cc:740] T 0be8303fce99487c9cb34506b3181363 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbcbed9750ae416bab78287348e9e196, State: Initialized, Role: FOLLOWER
W20260709 13:12:50.399226  1237 leader_election.cc:343] T 0be8303fce99487c9cb34506b3181363 P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515): Illegal state: must be running to vote when last-logged opid is not known
I20260709 13:12:50.410271  1180 tablet_service.cc:1511] Processing CreateTablet for tablet ac170d34df16448ea4c965890a7d22a1 (DEFAULT_TABLE table=soft_deleted_table [id=b518086dfff74502b43f9ea7c5d9299a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:12:50.411716  1180 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac170d34df16448ea4c965890a7d22a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:50.410735  1414 consensus_queue.cc:260] T 0be8303fce99487c9cb34506b3181363 P bbcbed9750ae416bab78287348e9e196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.414060  1414 ts_tablet_manager.cc:1434] T 0be8303fce99487c9cb34506b3181363 P bbcbed9750ae416bab78287348e9e196: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 13:12:50.414909  1261 tablet_service.cc:1511] Processing CreateTablet for tablet ac170d34df16448ea4c965890a7d22a1 (DEFAULT_TABLE table=soft_deleted_table [id=b518086dfff74502b43f9ea7c5d9299a]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 13:12:50.416265  1261 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac170d34df16448ea4c965890a7d22a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:50.392791  1399 leader_election.cc:290] T 0be8303fce99487c9cb34506b3181363 P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943), bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515)
I20260709 13:12:50.418025  1187 tablet_service.cc:1511] Processing CreateTablet for tablet 8e4161719e214f1a9836f352271dc985 (DEFAULT_TABLE table=soft_deleted_table [id=b518086dfff74502b43f9ea7c5d9299a]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 13:12:50.428457  1414 tablet_bootstrap.cc:492] T 4bf47f99c081476eabfaeb56515e0d79 P bbcbed9750ae416bab78287348e9e196: Bootstrap starting.
I20260709 13:12:50.431034  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0be8303fce99487c9cb34506b3181363" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22ec8c2439d14f139d5330c8ec45133a" is_pre_election: true
I20260709 13:12:50.446084  1122 raft_consensus.cc:2468] T 0be8303fce99487c9cb34506b3181363 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efe0ab937a564199977b910104824f30 in term 0.
I20260709 13:12:50.432369  1111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ac170d34df16448ea4c965890a7d22a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:50.448125  1109 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e4161719e214f1a9836f352271dc985. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:50.448372  1235 leader_election.cc:304] T 0be8303fce99487c9cb34506b3181363 P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 22ec8c2439d14f139d5330c8ec45133a, efe0ab937a564199977b910104824f30; no voters: bbcbed9750ae416bab78287348e9e196
I20260709 13:12:50.449975  1399 raft_consensus.cc:2804] T 0be8303fce99487c9cb34506b3181363 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:50.450434  1399 raft_consensus.cc:493] T 0be8303fce99487c9cb34506b3181363 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:50.450799  1399 raft_consensus.cc:3060] T 0be8303fce99487c9cb34506b3181363 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:50.438452  1415 tablet_bootstrap.cc:492] T 7ae6ea4d73e243a19c55a262c620bb7d P 22ec8c2439d14f139d5330c8ec45133a: Bootstrap starting.
I20260709 13:12:50.462601  1399 raft_consensus.cc:515] T 0be8303fce99487c9cb34506b3181363 P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.468640  1414 tablet_bootstrap.cc:654] T 4bf47f99c081476eabfaeb56515e0d79 P bbcbed9750ae416bab78287348e9e196: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:50.474143  1415 tablet_bootstrap.cc:654] T 7ae6ea4d73e243a19c55a262c620bb7d P 22ec8c2439d14f139d5330c8ec45133a: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:50.477082  1187 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e4161719e214f1a9836f352271dc985. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 13:12:50.479177  1197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0be8303fce99487c9cb34506b3181363" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbcbed9750ae416bab78287348e9e196"
I20260709 13:12:50.493891  1197 raft_consensus.cc:3060] T 0be8303fce99487c9cb34506b3181363 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:50.479254  1399 leader_election.cc:290] T 0be8303fce99487c9cb34506b3181363 P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 election: Requested vote from peers 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943), bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515)
I20260709 13:12:50.476765  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0be8303fce99487c9cb34506b3181363" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22ec8c2439d14f139d5330c8ec45133a"
I20260709 13:12:50.497421  1122 raft_consensus.cc:3060] T 0be8303fce99487c9cb34506b3181363 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:50.504171  1122 raft_consensus.cc:2468] T 0be8303fce99487c9cb34506b3181363 P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe0ab937a564199977b910104824f30 in term 1.
I20260709 13:12:50.505617  1235 leader_election.cc:304] T 0be8303fce99487c9cb34506b3181363 P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22ec8c2439d14f139d5330c8ec45133a, efe0ab937a564199977b910104824f30; no voters: 
I20260709 13:12:50.506428  1197 raft_consensus.cc:2468] T 0be8303fce99487c9cb34506b3181363 P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe0ab937a564199977b910104824f30 in term 1.
I20260709 13:12:50.506565  1399 raft_consensus.cc:2804] T 0be8303fce99487c9cb34506b3181363 P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:50.507197  1413 tablet_bootstrap.cc:492] T 4bf47f99c081476eabfaeb56515e0d79 P efe0ab937a564199977b910104824f30: No bootstrap required, opened a new log
I20260709 13:12:50.507902  1413 ts_tablet_manager.cc:1403] T 4bf47f99c081476eabfaeb56515e0d79 P efe0ab937a564199977b910104824f30: Time spent bootstrapping tablet: real 0.175s	user 0.026s	sys 0.054s
I20260709 13:12:50.509146  1399 raft_consensus.cc:697] T 0be8303fce99487c9cb34506b3181363 P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Becoming Leader. State: Replica: efe0ab937a564199977b910104824f30, State: Running, Role: LEADER
I20260709 13:12:50.510161  1399 consensus_queue.cc:237] T 0be8303fce99487c9cb34506b3181363 P efe0ab937a564199977b910104824f30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.511363  1413 raft_consensus.cc:359] T 4bf47f99c081476eabfaeb56515e0d79 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.512097  1413 raft_consensus.cc:385] T 4bf47f99c081476eabfaeb56515e0d79 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:50.512395  1413 raft_consensus.cc:740] T 4bf47f99c081476eabfaeb56515e0d79 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe0ab937a564199977b910104824f30, State: Initialized, Role: FOLLOWER
W20260709 13:12:50.512684  1069 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 13:12:50.513123  1413 consensus_queue.cc:260] T 4bf47f99c081476eabfaeb56515e0d79 P efe0ab937a564199977b910104824f30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.515494  1415 tablet_bootstrap.cc:492] T 7ae6ea4d73e243a19c55a262c620bb7d P 22ec8c2439d14f139d5330c8ec45133a: No bootstrap required, opened a new log
I20260709 13:12:50.516575  1415 ts_tablet_manager.cc:1403] T 7ae6ea4d73e243a19c55a262c620bb7d P 22ec8c2439d14f139d5330c8ec45133a: Time spent bootstrapping tablet: real 0.079s	user 0.005s	sys 0.015s
I20260709 13:12:50.515511  1413 ts_tablet_manager.cc:1434] T 4bf47f99c081476eabfaeb56515e0d79 P efe0ab937a564199977b910104824f30: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.003s
I20260709 13:12:50.518865  1413 tablet_bootstrap.cc:492] T 7ae6ea4d73e243a19c55a262c620bb7d P efe0ab937a564199977b910104824f30: Bootstrap starting.
I20260709 13:12:50.520287  1415 raft_consensus.cc:359] T 7ae6ea4d73e243a19c55a262c620bb7d P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.521255  1415 raft_consensus.cc:385] T 7ae6ea4d73e243a19c55a262c620bb7d P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:50.521781  1415 raft_consensus.cc:740] T 7ae6ea4d73e243a19c55a262c620bb7d P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22ec8c2439d14f139d5330c8ec45133a, State: Initialized, Role: FOLLOWER
I20260709 13:12:50.522919  1415 consensus_queue.cc:260] T 7ae6ea4d73e243a19c55a262c620bb7d P 22ec8c2439d14f139d5330c8ec45133a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.525148  1019 catalog_manager.cc:5697] T 0be8303fce99487c9cb34506b3181363 P efe0ab937a564199977b910104824f30 reported cstate change: term changed from 0 to 1, leader changed from <none> to efe0ab937a564199977b910104824f30 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "efe0ab937a564199977b910104824f30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:50.527040  1414 tablet_bootstrap.cc:492] T 4bf47f99c081476eabfaeb56515e0d79 P bbcbed9750ae416bab78287348e9e196: No bootstrap required, opened a new log
I20260709 13:12:50.527148  1413 tablet_bootstrap.cc:654] T 7ae6ea4d73e243a19c55a262c620bb7d P efe0ab937a564199977b910104824f30: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:50.527611  1414 ts_tablet_manager.cc:1403] T 4bf47f99c081476eabfaeb56515e0d79 P bbcbed9750ae416bab78287348e9e196: Time spent bootstrapping tablet: real 0.099s	user 0.016s	sys 0.018s
I20260709 13:12:50.530720  1414 raft_consensus.cc:359] T 4bf47f99c081476eabfaeb56515e0d79 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.531394  1414 raft_consensus.cc:385] T 4bf47f99c081476eabfaeb56515e0d79 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:50.531679  1414 raft_consensus.cc:740] T 4bf47f99c081476eabfaeb56515e0d79 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbcbed9750ae416bab78287348e9e196, State: Initialized, Role: FOLLOWER
I20260709 13:12:50.532475  1414 consensus_queue.cc:260] T 4bf47f99c081476eabfaeb56515e0d79 P bbcbed9750ae416bab78287348e9e196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.533829  1415 ts_tablet_manager.cc:1434] T 7ae6ea4d73e243a19c55a262c620bb7d P 22ec8c2439d14f139d5330c8ec45133a: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:12:50.534932  1414 ts_tablet_manager.cc:1434] T 4bf47f99c081476eabfaeb56515e0d79 P bbcbed9750ae416bab78287348e9e196: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:12:50.535915  1414 tablet_bootstrap.cc:492] T 7ae6ea4d73e243a19c55a262c620bb7d P bbcbed9750ae416bab78287348e9e196: Bootstrap starting.
I20260709 13:12:50.536233  1415 tablet_bootstrap.cc:492] T 249a9a4a171d491b9f64f9b263bbb91a P 22ec8c2439d14f139d5330c8ec45133a: Bootstrap starting.
I20260709 13:12:50.542189  1414 tablet_bootstrap.cc:654] T 7ae6ea4d73e243a19c55a262c620bb7d P bbcbed9750ae416bab78287348e9e196: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:50.542949  1415 tablet_bootstrap.cc:654] T 249a9a4a171d491b9f64f9b263bbb91a P 22ec8c2439d14f139d5330c8ec45133a: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:50.548499  1413 tablet_bootstrap.cc:492] T 7ae6ea4d73e243a19c55a262c620bb7d P efe0ab937a564199977b910104824f30: No bootstrap required, opened a new log
I20260709 13:12:50.549078  1413 ts_tablet_manager.cc:1403] T 7ae6ea4d73e243a19c55a262c620bb7d P efe0ab937a564199977b910104824f30: Time spent bootstrapping tablet: real 0.030s	user 0.006s	sys 0.021s
I20260709 13:12:50.552295  1413 raft_consensus.cc:359] T 7ae6ea4d73e243a19c55a262c620bb7d P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.556533  1413 raft_consensus.cc:385] T 7ae6ea4d73e243a19c55a262c620bb7d P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:50.556918  1413 raft_consensus.cc:740] T 7ae6ea4d73e243a19c55a262c620bb7d P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe0ab937a564199977b910104824f30, State: Initialized, Role: FOLLOWER
I20260709 13:12:50.557610  1414 tablet_bootstrap.cc:492] T 7ae6ea4d73e243a19c55a262c620bb7d P bbcbed9750ae416bab78287348e9e196: No bootstrap required, opened a new log
I20260709 13:12:50.558178  1414 ts_tablet_manager.cc:1403] T 7ae6ea4d73e243a19c55a262c620bb7d P bbcbed9750ae416bab78287348e9e196: Time spent bootstrapping tablet: real 0.022s	user 0.012s	sys 0.000s
I20260709 13:12:50.559172  1413 consensus_queue.cc:260] T 7ae6ea4d73e243a19c55a262c620bb7d P efe0ab937a564199977b910104824f30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.559808  1415 tablet_bootstrap.cc:492] T 249a9a4a171d491b9f64f9b263bbb91a P 22ec8c2439d14f139d5330c8ec45133a: No bootstrap required, opened a new log
I20260709 13:12:50.560468  1415 ts_tablet_manager.cc:1403] T 249a9a4a171d491b9f64f9b263bbb91a P 22ec8c2439d14f139d5330c8ec45133a: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.000s
I20260709 13:12:50.560840  1414 raft_consensus.cc:359] T 7ae6ea4d73e243a19c55a262c620bb7d P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.561631  1414 raft_consensus.cc:385] T 7ae6ea4d73e243a19c55a262c620bb7d P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:50.561914  1414 raft_consensus.cc:740] T 7ae6ea4d73e243a19c55a262c620bb7d P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbcbed9750ae416bab78287348e9e196, State: Initialized, Role: FOLLOWER
I20260709 13:12:50.562021  1413 ts_tablet_manager.cc:1434] T 7ae6ea4d73e243a19c55a262c620bb7d P efe0ab937a564199977b910104824f30: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 13:12:50.562709  1414 consensus_queue.cc:260] T 7ae6ea4d73e243a19c55a262c620bb7d P bbcbed9750ae416bab78287348e9e196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.563155  1413 tablet_bootstrap.cc:492] T 249a9a4a171d491b9f64f9b263bbb91a P efe0ab937a564199977b910104824f30: Bootstrap starting.
I20260709 13:12:50.564239  1415 raft_consensus.cc:359] T 249a9a4a171d491b9f64f9b263bbb91a P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.565037  1414 ts_tablet_manager.cc:1434] T 7ae6ea4d73e243a19c55a262c620bb7d P bbcbed9750ae416bab78287348e9e196: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 13:12:50.565423  1415 raft_consensus.cc:385] T 249a9a4a171d491b9f64f9b263bbb91a P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:50.565860  1415 raft_consensus.cc:740] T 249a9a4a171d491b9f64f9b263bbb91a P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22ec8c2439d14f139d5330c8ec45133a, State: Initialized, Role: FOLLOWER
I20260709 13:12:50.566437  1414 tablet_bootstrap.cc:492] T 5e059664b8d64cc49d217bfcddfc2232 P bbcbed9750ae416bab78287348e9e196: Bootstrap starting.
I20260709 13:12:50.566722  1415 consensus_queue.cc:260] T 249a9a4a171d491b9f64f9b263bbb91a P 22ec8c2439d14f139d5330c8ec45133a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.569506  1415 ts_tablet_manager.cc:1434] T 249a9a4a171d491b9f64f9b263bbb91a P 22ec8c2439d14f139d5330c8ec45133a: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:12:50.570643  1415 tablet_bootstrap.cc:492] T 4bf47f99c081476eabfaeb56515e0d79 P 22ec8c2439d14f139d5330c8ec45133a: Bootstrap starting.
I20260709 13:12:50.573304  1413 tablet_bootstrap.cc:654] T 249a9a4a171d491b9f64f9b263bbb91a P efe0ab937a564199977b910104824f30: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:50.577849  1415 tablet_bootstrap.cc:654] T 4bf47f99c081476eabfaeb56515e0d79 P 22ec8c2439d14f139d5330c8ec45133a: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:50.582402  1414 tablet_bootstrap.cc:654] T 5e059664b8d64cc49d217bfcddfc2232 P bbcbed9750ae416bab78287348e9e196: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:50.586477  1413 tablet_bootstrap.cc:492] T 249a9a4a171d491b9f64f9b263bbb91a P efe0ab937a564199977b910104824f30: No bootstrap required, opened a new log
I20260709 13:12:50.587309  1413 ts_tablet_manager.cc:1403] T 249a9a4a171d491b9f64f9b263bbb91a P efe0ab937a564199977b910104824f30: Time spent bootstrapping tablet: real 0.024s	user 0.020s	sys 0.000s
I20260709 13:12:50.591609  1413 raft_consensus.cc:359] T 249a9a4a171d491b9f64f9b263bbb91a P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.593452  1413 raft_consensus.cc:385] T 249a9a4a171d491b9f64f9b263bbb91a P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:50.593822  1413 raft_consensus.cc:740] T 249a9a4a171d491b9f64f9b263bbb91a P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe0ab937a564199977b910104824f30, State: Initialized, Role: FOLLOWER
I20260709 13:12:50.594825  1414 tablet_bootstrap.cc:492] T 5e059664b8d64cc49d217bfcddfc2232 P bbcbed9750ae416bab78287348e9e196: No bootstrap required, opened a new log
I20260709 13:12:50.594785  1413 consensus_queue.cc:260] T 249a9a4a171d491b9f64f9b263bbb91a P efe0ab937a564199977b910104824f30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.595464  1414 ts_tablet_manager.cc:1403] T 5e059664b8d64cc49d217bfcddfc2232 P bbcbed9750ae416bab78287348e9e196: Time spent bootstrapping tablet: real 0.029s	user 0.017s	sys 0.000s
I20260709 13:12:50.595981  1415 tablet_bootstrap.cc:492] T 4bf47f99c081476eabfaeb56515e0d79 P 22ec8c2439d14f139d5330c8ec45133a: No bootstrap required, opened a new log
I20260709 13:12:50.596642  1415 ts_tablet_manager.cc:1403] T 4bf47f99c081476eabfaeb56515e0d79 P 22ec8c2439d14f139d5330c8ec45133a: Time spent bootstrapping tablet: real 0.026s	user 0.014s	sys 0.011s
I20260709 13:12:50.597602  1413 ts_tablet_manager.cc:1434] T 249a9a4a171d491b9f64f9b263bbb91a P efe0ab937a564199977b910104824f30: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 13:12:50.598841  1413 tablet_bootstrap.cc:492] T 976112eb8da2479e9a559399bee97dfc P efe0ab937a564199977b910104824f30: Bootstrap starting.
I20260709 13:12:50.598779  1414 raft_consensus.cc:359] T 5e059664b8d64cc49d217bfcddfc2232 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.599685  1414 raft_consensus.cc:385] T 5e059664b8d64cc49d217bfcddfc2232 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:50.600010  1414 raft_consensus.cc:740] T 5e059664b8d64cc49d217bfcddfc2232 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbcbed9750ae416bab78287348e9e196, State: Initialized, Role: FOLLOWER
I20260709 13:12:50.600036  1415 raft_consensus.cc:359] T 4bf47f99c081476eabfaeb56515e0d79 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.601032  1415 raft_consensus.cc:385] T 4bf47f99c081476eabfaeb56515e0d79 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:50.600979  1414 consensus_queue.cc:260] T 5e059664b8d64cc49d217bfcddfc2232 P bbcbed9750ae416bab78287348e9e196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.601558  1415 raft_consensus.cc:740] T 4bf47f99c081476eabfaeb56515e0d79 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22ec8c2439d14f139d5330c8ec45133a, State: Initialized, Role: FOLLOWER
I20260709 13:12:50.602581  1415 consensus_queue.cc:260] T 4bf47f99c081476eabfaeb56515e0d79 P 22ec8c2439d14f139d5330c8ec45133a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.603744  1414 ts_tablet_manager.cc:1434] T 5e059664b8d64cc49d217bfcddfc2232 P bbcbed9750ae416bab78287348e9e196: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.003s
I20260709 13:12:50.605997  1415 ts_tablet_manager.cc:1434] T 4bf47f99c081476eabfaeb56515e0d79 P 22ec8c2439d14f139d5330c8ec45133a: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 13:12:50.607180  1413 tablet_bootstrap.cc:654] T 976112eb8da2479e9a559399bee97dfc P efe0ab937a564199977b910104824f30: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:50.610975  1414 tablet_bootstrap.cc:492] T 249a9a4a171d491b9f64f9b263bbb91a P bbcbed9750ae416bab78287348e9e196: Bootstrap starting.
I20260709 13:12:50.607200  1415 tablet_bootstrap.cc:492] T 5e059664b8d64cc49d217bfcddfc2232 P 22ec8c2439d14f139d5330c8ec45133a: Bootstrap starting.
I20260709 13:12:50.622397  1415 tablet_bootstrap.cc:654] T 5e059664b8d64cc49d217bfcddfc2232 P 22ec8c2439d14f139d5330c8ec45133a: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:50.623133  1414 tablet_bootstrap.cc:654] T 249a9a4a171d491b9f64f9b263bbb91a P bbcbed9750ae416bab78287348e9e196: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:50.632912  1416 raft_consensus.cc:493] T 4bf47f99c081476eabfaeb56515e0d79 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.633543  1416 raft_consensus.cc:515] T 4bf47f99c081476eabfaeb56515e0d79 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.634938  1413 tablet_bootstrap.cc:492] T 976112eb8da2479e9a559399bee97dfc P efe0ab937a564199977b910104824f30: No bootstrap required, opened a new log
I20260709 13:12:50.635540  1413 ts_tablet_manager.cc:1403] T 976112eb8da2479e9a559399bee97dfc P efe0ab937a564199977b910104824f30: Time spent bootstrapping tablet: real 0.037s	user 0.033s	sys 0.000s
I20260709 13:12:50.635740  1415 tablet_bootstrap.cc:492] T 5e059664b8d64cc49d217bfcddfc2232 P 22ec8c2439d14f139d5330c8ec45133a: No bootstrap required, opened a new log
I20260709 13:12:50.636642  1415 ts_tablet_manager.cc:1403] T 5e059664b8d64cc49d217bfcddfc2232 P 22ec8c2439d14f139d5330c8ec45133a: Time spent bootstrapping tablet: real 0.030s	user 0.018s	sys 0.005s
I20260709 13:12:50.638082  1416 leader_election.cc:290] T 4bf47f99c081476eabfaeb56515e0d79 P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515), efe0ab937a564199977b910104824f30 (127.21.63.3:33527)
I20260709 13:12:50.637995  1269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bf47f99c081476eabfaeb56515e0d79" candidate_uuid: "22ec8c2439d14f139d5330c8ec45133a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe0ab937a564199977b910104824f30" is_pre_election: true
I20260709 13:12:50.639441  1269 raft_consensus.cc:2468] T 4bf47f99c081476eabfaeb56515e0d79 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22ec8c2439d14f139d5330c8ec45133a in term 0.
I20260709 13:12:50.638744  1391 raft_consensus.cc:493] T 4bf47f99c081476eabfaeb56515e0d79 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.640043  1413 raft_consensus.cc:359] T 976112eb8da2479e9a559399bee97dfc P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.637995  1197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bf47f99c081476eabfaeb56515e0d79" candidate_uuid: "22ec8c2439d14f139d5330c8ec45133a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbcbed9750ae416bab78287348e9e196" is_pre_election: true
I20260709 13:12:50.640866  1391 raft_consensus.cc:515] T 4bf47f99c081476eabfaeb56515e0d79 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.641844  1415 raft_consensus.cc:359] T 5e059664b8d64cc49d217bfcddfc2232 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.642681  1415 raft_consensus.cc:385] T 5e059664b8d64cc49d217bfcddfc2232 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:50.643085  1415 raft_consensus.cc:740] T 5e059664b8d64cc49d217bfcddfc2232 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22ec8c2439d14f139d5330c8ec45133a, State: Initialized, Role: FOLLOWER
I20260709 13:12:50.644325  1415 consensus_queue.cc:260] T 5e059664b8d64cc49d217bfcddfc2232 P 22ec8c2439d14f139d5330c8ec45133a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.640884  1413 raft_consensus.cc:385] T 976112eb8da2479e9a559399bee97dfc P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:50.645350  1413 raft_consensus.cc:740] T 976112eb8da2479e9a559399bee97dfc P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe0ab937a564199977b910104824f30, State: Initialized, Role: FOLLOWER
I20260709 13:12:50.645897  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bf47f99c081476eabfaeb56515e0d79" candidate_uuid: "bbcbed9750ae416bab78287348e9e196" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22ec8c2439d14f139d5330c8ec45133a" is_pre_election: true
I20260709 13:12:50.646576  1122 raft_consensus.cc:2468] T 4bf47f99c081476eabfaeb56515e0d79 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bbcbed9750ae416bab78287348e9e196 in term 0.
I20260709 13:12:50.646162  1413 consensus_queue.cc:260] T 976112eb8da2479e9a559399bee97dfc P efe0ab937a564199977b910104824f30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.648136  1087 leader_election.cc:304] T 4bf47f99c081476eabfaeb56515e0d79 P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22ec8c2439d14f139d5330c8ec45133a, efe0ab937a564199977b910104824f30; no voters: 
I20260709 13:12:50.649446  1161 leader_election.cc:304] T 4bf47f99c081476eabfaeb56515e0d79 P bbcbed9750ae416bab78287348e9e196 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22ec8c2439d14f139d5330c8ec45133a, bbcbed9750ae416bab78287348e9e196; no voters: 
I20260709 13:12:50.649972  1197 raft_consensus.cc:2468] T 4bf47f99c081476eabfaeb56515e0d79 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22ec8c2439d14f139d5330c8ec45133a in term 0.
I20260709 13:12:50.650588  1321 raft_consensus.cc:2804] T 4bf47f99c081476eabfaeb56515e0d79 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:50.651057  1321 raft_consensus.cc:493] T 4bf47f99c081476eabfaeb56515e0d79 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:50.651358  1414 tablet_bootstrap.cc:492] T 249a9a4a171d491b9f64f9b263bbb91a P bbcbed9750ae416bab78287348e9e196: No bootstrap required, opened a new log
I20260709 13:12:50.652051  1414 ts_tablet_manager.cc:1403] T 249a9a4a171d491b9f64f9b263bbb91a P bbcbed9750ae416bab78287348e9e196: Time spent bootstrapping tablet: real 0.041s	user 0.016s	sys 0.012s
I20260709 13:12:50.653721  1269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bf47f99c081476eabfaeb56515e0d79" candidate_uuid: "bbcbed9750ae416bab78287348e9e196" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe0ab937a564199977b910104824f30" is_pre_election: true
I20260709 13:12:50.654685  1269 raft_consensus.cc:2468] T 4bf47f99c081476eabfaeb56515e0d79 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bbcbed9750ae416bab78287348e9e196 in term 0.
I20260709 13:12:50.643209  1391 leader_election.cc:290] T 4bf47f99c081476eabfaeb56515e0d79 P bbcbed9750ae416bab78287348e9e196 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943), efe0ab937a564199977b910104824f30 (127.21.63.3:33527)
I20260709 13:12:50.655467  1414 raft_consensus.cc:359] T 249a9a4a171d491b9f64f9b263bbb91a P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.656122  1391 raft_consensus.cc:2804] T 4bf47f99c081476eabfaeb56515e0d79 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:50.656457  1414 raft_consensus.cc:385] T 249a9a4a171d491b9f64f9b263bbb91a P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:50.656621  1391 raft_consensus.cc:493] T 4bf47f99c081476eabfaeb56515e0d79 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:50.657063  1414 raft_consensus.cc:740] T 249a9a4a171d491b9f64f9b263bbb91a P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbcbed9750ae416bab78287348e9e196, State: Initialized, Role: FOLLOWER
I20260709 13:12:50.657356  1391 raft_consensus.cc:3060] T 4bf47f99c081476eabfaeb56515e0d79 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:50.658330  1414 consensus_queue.cc:260] T 249a9a4a171d491b9f64f9b263bbb91a P bbcbed9750ae416bab78287348e9e196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.651465  1321 raft_consensus.cc:3060] T 4bf47f99c081476eabfaeb56515e0d79 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:50.661538  1414 ts_tablet_manager.cc:1434] T 249a9a4a171d491b9f64f9b263bbb91a P bbcbed9750ae416bab78287348e9e196: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:12:50.662763  1414 tablet_bootstrap.cc:492] T fe6a0e06494f416e80a6ebbec27396bd P bbcbed9750ae416bab78287348e9e196: Bootstrap starting.
I20260709 13:12:50.648584  1415 ts_tablet_manager.cc:1434] T 5e059664b8d64cc49d217bfcddfc2232 P 22ec8c2439d14f139d5330c8ec45133a: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.003s
I20260709 13:12:50.649168  1413 ts_tablet_manager.cc:1434] T 976112eb8da2479e9a559399bee97dfc P efe0ab937a564199977b910104824f30: Time spent starting tablet: real 0.013s	user 0.010s	sys 0.000s
I20260709 13:12:50.668479  1415 tablet_bootstrap.cc:492] T 976112eb8da2479e9a559399bee97dfc P 22ec8c2439d14f139d5330c8ec45133a: Bootstrap starting.
I20260709 13:12:50.669050  1413 tablet_bootstrap.cc:492] T 5e059664b8d64cc49d217bfcddfc2232 P efe0ab937a564199977b910104824f30: Bootstrap starting.
I20260709 13:12:50.678119  1414 tablet_bootstrap.cc:654] T fe6a0e06494f416e80a6ebbec27396bd P bbcbed9750ae416bab78287348e9e196: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:50.678797  1391 raft_consensus.cc:515] T 4bf47f99c081476eabfaeb56515e0d79 P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.682312  1399 raft_consensus.cc:493] T 249a9a4a171d491b9f64f9b263bbb91a P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.682885  1399 raft_consensus.cc:515] T 249a9a4a171d491b9f64f9b263bbb91a P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.685235  1399 leader_election.cc:290] T 249a9a4a171d491b9f64f9b263bbb91a P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943), bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515)
I20260709 13:12:50.688462  1413 tablet_bootstrap.cc:654] T 5e059664b8d64cc49d217bfcddfc2232 P efe0ab937a564199977b910104824f30: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:50.690706  1197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "249a9a4a171d491b9f64f9b263bbb91a" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbcbed9750ae416bab78287348e9e196" is_pre_election: true
I20260709 13:12:50.690907  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "249a9a4a171d491b9f64f9b263bbb91a" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22ec8c2439d14f139d5330c8ec45133a" is_pre_election: true
I20260709 13:12:50.691452  1122 raft_consensus.cc:2468] T 249a9a4a171d491b9f64f9b263bbb91a P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efe0ab937a564199977b910104824f30 in term 0.
I20260709 13:12:50.691442  1197 raft_consensus.cc:2468] T 249a9a4a171d491b9f64f9b263bbb91a P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efe0ab937a564199977b910104824f30 in term 0.
I20260709 13:12:50.692968  1235 leader_election.cc:304] T 249a9a4a171d491b9f64f9b263bbb91a P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22ec8c2439d14f139d5330c8ec45133a, efe0ab937a564199977b910104824f30; no voters: 
I20260709 13:12:50.694739  1399 raft_consensus.cc:2804] T 249a9a4a171d491b9f64f9b263bbb91a P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:50.695149  1399 raft_consensus.cc:493] T 249a9a4a171d491b9f64f9b263bbb91a P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:50.695499  1399 raft_consensus.cc:3060] T 249a9a4a171d491b9f64f9b263bbb91a P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:50.697021  1321 raft_consensus.cc:515] T 4bf47f99c081476eabfaeb56515e0d79 P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.700737  1197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bf47f99c081476eabfaeb56515e0d79" candidate_uuid: "22ec8c2439d14f139d5330c8ec45133a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbcbed9750ae416bab78287348e9e196"
I20260709 13:12:50.701579  1197 raft_consensus.cc:2393] T 4bf47f99c081476eabfaeb56515e0d79 P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 22ec8c2439d14f139d5330c8ec45133a in current term 1: Already voted for candidate bbcbed9750ae416bab78287348e9e196 in this term.
I20260709 13:12:50.702541  1391 leader_election.cc:290] T 4bf47f99c081476eabfaeb56515e0d79 P bbcbed9750ae416bab78287348e9e196 [CANDIDATE]: Term 1 election: Requested vote from peers 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943), efe0ab937a564199977b910104824f30 (127.21.63.3:33527)
I20260709 13:12:50.703620  1269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bf47f99c081476eabfaeb56515e0d79" candidate_uuid: "bbcbed9750ae416bab78287348e9e196" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe0ab937a564199977b910104824f30"
I20260709 13:12:50.709761  1269 raft_consensus.cc:3060] T 4bf47f99c081476eabfaeb56515e0d79 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:50.717664  1269 raft_consensus.cc:2468] T 4bf47f99c081476eabfaeb56515e0d79 P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbcbed9750ae416bab78287348e9e196 in term 1.
I20260709 13:12:50.718976  1162 leader_election.cc:304] T 4bf47f99c081476eabfaeb56515e0d79 P bbcbed9750ae416bab78287348e9e196 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bbcbed9750ae416bab78287348e9e196, efe0ab937a564199977b910104824f30; no voters: 
I20260709 13:12:50.719925  1391 raft_consensus.cc:2804] T 4bf47f99c081476eabfaeb56515e0d79 P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:50.720451  1391 raft_consensus.cc:697] T 4bf47f99c081476eabfaeb56515e0d79 P bbcbed9750ae416bab78287348e9e196 [term 1 LEADER]: Becoming Leader. State: Replica: bbcbed9750ae416bab78287348e9e196, State: Running, Role: LEADER
I20260709 13:12:50.721344  1391 consensus_queue.cc:237] T 4bf47f99c081476eabfaeb56515e0d79 P bbcbed9750ae416bab78287348e9e196 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.704785  1321 leader_election.cc:290] T 4bf47f99c081476eabfaeb56515e0d79 P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 1 election: Requested vote from peers bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515), efe0ab937a564199977b910104824f30 (127.21.63.3:33527)
I20260709 13:12:50.705426  1272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bf47f99c081476eabfaeb56515e0d79" candidate_uuid: "22ec8c2439d14f139d5330c8ec45133a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe0ab937a564199977b910104824f30"
I20260709 13:12:50.707962  1415 tablet_bootstrap.cc:654] T 976112eb8da2479e9a559399bee97dfc P 22ec8c2439d14f139d5330c8ec45133a: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:50.723521  1272 raft_consensus.cc:2393] T 4bf47f99c081476eabfaeb56515e0d79 P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 22ec8c2439d14f139d5330c8ec45133a in current term 1: Already voted for candidate bbcbed9750ae416bab78287348e9e196 in this term.
I20260709 13:12:50.724942  1087 leader_election.cc:304] T 4bf47f99c081476eabfaeb56515e0d79 P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 22ec8c2439d14f139d5330c8ec45133a; no voters: bbcbed9750ae416bab78287348e9e196, efe0ab937a564199977b910104824f30
I20260709 13:12:50.725929  1416 raft_consensus.cc:2749] T 4bf47f99c081476eabfaeb56515e0d79 P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:12:50.729920  1399 raft_consensus.cc:515] T 249a9a4a171d491b9f64f9b263bbb91a P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.730263  1434 raft_consensus.cc:493] T 7ae6ea4d73e243a19c55a262c620bb7d P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.730967  1434 raft_consensus.cc:515] T 7ae6ea4d73e243a19c55a262c620bb7d P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.733088  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "249a9a4a171d491b9f64f9b263bbb91a" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22ec8c2439d14f139d5330c8ec45133a"
I20260709 13:12:50.733665  1122 raft_consensus.cc:3060] T 249a9a4a171d491b9f64f9b263bbb91a P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:50.735338  1423 raft_consensus.cc:493] T 7ae6ea4d73e243a19c55a262c620bb7d P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.735903  1423 raft_consensus.cc:515] T 7ae6ea4d73e243a19c55a262c620bb7d P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.736192  1121 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bf47f99c081476eabfaeb56515e0d79" candidate_uuid: "bbcbed9750ae416bab78287348e9e196" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22ec8c2439d14f139d5330c8ec45133a"
I20260709 13:12:50.736642  1123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae6ea4d73e243a19c55a262c620bb7d" candidate_uuid: "bbcbed9750ae416bab78287348e9e196" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22ec8c2439d14f139d5330c8ec45133a" is_pre_election: true
I20260709 13:12:50.736503  1269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae6ea4d73e243a19c55a262c620bb7d" candidate_uuid: "bbcbed9750ae416bab78287348e9e196" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe0ab937a564199977b910104824f30" is_pre_election: true
I20260709 13:12:50.738106  1197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "249a9a4a171d491b9f64f9b263bbb91a" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbcbed9750ae416bab78287348e9e196"
I20260709 13:12:50.738808  1197 raft_consensus.cc:3060] T 249a9a4a171d491b9f64f9b263bbb91a P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:50.737493  1269 raft_consensus.cc:2468] T 7ae6ea4d73e243a19c55a262c620bb7d P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bbcbed9750ae416bab78287348e9e196 in term 0.
I20260709 13:12:50.739893  1198 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae6ea4d73e243a19c55a262c620bb7d" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbcbed9750ae416bab78287348e9e196" is_pre_election: true
I20260709 13:12:50.737016  1121 raft_consensus.cc:2393] T 4bf47f99c081476eabfaeb56515e0d79 P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bbcbed9750ae416bab78287348e9e196 in current term 1: Already voted for candidate 22ec8c2439d14f139d5330c8ec45133a in this term.
I20260709 13:12:50.737444  1123 raft_consensus.cc:2468] T 7ae6ea4d73e243a19c55a262c620bb7d P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bbcbed9750ae416bab78287348e9e196 in term 0.
I20260709 13:12:50.738214  1423 leader_election.cc:290] T 7ae6ea4d73e243a19c55a262c620bb7d P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943), bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515)
I20260709 13:12:50.740629  1198 raft_consensus.cc:2468] T 7ae6ea4d73e243a19c55a262c620bb7d P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efe0ab937a564199977b910104824f30 in term 0.
I20260709 13:12:50.747603  1123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae6ea4d73e243a19c55a262c620bb7d" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22ec8c2439d14f139d5330c8ec45133a" is_pre_election: true
I20260709 13:12:50.748297  1123 raft_consensus.cc:2468] T 7ae6ea4d73e243a19c55a262c620bb7d P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efe0ab937a564199977b910104824f30 in term 0.
I20260709 13:12:50.749819  1235 leader_election.cc:304] T 7ae6ea4d73e243a19c55a262c620bb7d P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22ec8c2439d14f139d5330c8ec45133a, efe0ab937a564199977b910104824f30; no voters: 
I20260709 13:12:50.750176  1122 raft_consensus.cc:2468] T 249a9a4a171d491b9f64f9b263bbb91a P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe0ab937a564199977b910104824f30 in term 1.
I20260709 13:12:50.751173  1423 raft_consensus.cc:2804] T 7ae6ea4d73e243a19c55a262c620bb7d P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:50.751682  1423 raft_consensus.cc:493] T 7ae6ea4d73e243a19c55a262c620bb7d P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:50.752133  1423 raft_consensus.cc:3060] T 7ae6ea4d73e243a19c55a262c620bb7d P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:50.741081  1162 leader_election.cc:304] T 7ae6ea4d73e243a19c55a262c620bb7d P bbcbed9750ae416bab78287348e9e196 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bbcbed9750ae416bab78287348e9e196, efe0ab937a564199977b910104824f30; no voters: 
I20260709 13:12:50.741312  1434 leader_election.cc:290] T 7ae6ea4d73e243a19c55a262c620bb7d P bbcbed9750ae416bab78287348e9e196 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943), efe0ab937a564199977b910104824f30 (127.21.63.3:33527)
I20260709 13:12:50.757416  1235 leader_election.cc:304] T 249a9a4a171d491b9f64f9b263bbb91a P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22ec8c2439d14f139d5330c8ec45133a, efe0ab937a564199977b910104824f30; no voters: 
I20260709 13:12:50.758088  1434 raft_consensus.cc:2804] T 7ae6ea4d73e243a19c55a262c620bb7d P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:50.758496  1434 raft_consensus.cc:493] T 7ae6ea4d73e243a19c55a262c620bb7d P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:50.758910  1434 raft_consensus.cc:3060] T 7ae6ea4d73e243a19c55a262c620bb7d P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:50.746896  1197 raft_consensus.cc:2468] T 249a9a4a171d491b9f64f9b263bbb91a P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe0ab937a564199977b910104824f30 in term 1.
I20260709 13:12:50.761106  1019 catalog_manager.cc:5697] T 4bf47f99c081476eabfaeb56515e0d79 P bbcbed9750ae416bab78287348e9e196 reported cstate change: term changed from 0 to 1, leader changed from <none> to bbcbed9750ae416bab78287348e9e196 (127.21.63.2). New cstate: current_term: 1 leader_uuid: "bbcbed9750ae416bab78287348e9e196" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:50.764288  1399 leader_election.cc:290] T 249a9a4a171d491b9f64f9b263bbb91a P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 election: Requested vote from peers 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943), bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515)
I20260709 13:12:50.740847  1415 tablet_bootstrap.cc:492] T 976112eb8da2479e9a559399bee97dfc P 22ec8c2439d14f139d5330c8ec45133a: No bootstrap required, opened a new log
I20260709 13:12:50.764961  1399 raft_consensus.cc:2804] T 249a9a4a171d491b9f64f9b263bbb91a P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:50.765304  1415 ts_tablet_manager.cc:1403] T 976112eb8da2479e9a559399bee97dfc P 22ec8c2439d14f139d5330c8ec45133a: Time spent bootstrapping tablet: real 0.097s	user 0.016s	sys 0.000s
I20260709 13:12:50.765503  1399 raft_consensus.cc:697] T 249a9a4a171d491b9f64f9b263bbb91a P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Becoming Leader. State: Replica: efe0ab937a564199977b910104824f30, State: Running, Role: LEADER
I20260709 13:12:50.766530  1399 consensus_queue.cc:237] T 249a9a4a171d491b9f64f9b263bbb91a P efe0ab937a564199977b910104824f30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.758566  1413 tablet_bootstrap.cc:492] T 5e059664b8d64cc49d217bfcddfc2232 P efe0ab937a564199977b910104824f30: No bootstrap required, opened a new log
I20260709 13:12:50.769055  1413 ts_tablet_manager.cc:1403] T 5e059664b8d64cc49d217bfcddfc2232 P efe0ab937a564199977b910104824f30: Time spent bootstrapping tablet: real 0.101s	user 0.023s	sys 0.000s
I20260709 13:12:50.772013  1423 raft_consensus.cc:515] T 7ae6ea4d73e243a19c55a262c620bb7d P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.772409  1434 raft_consensus.cc:515] T 7ae6ea4d73e243a19c55a262c620bb7d P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.776548  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae6ea4d73e243a19c55a262c620bb7d" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22ec8c2439d14f139d5330c8ec45133a"
I20260709 13:12:50.777130  1122 raft_consensus.cc:3060] T 7ae6ea4d73e243a19c55a262c620bb7d P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:50.777724  1123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae6ea4d73e243a19c55a262c620bb7d" candidate_uuid: "bbcbed9750ae416bab78287348e9e196" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22ec8c2439d14f139d5330c8ec45133a"
I20260709 13:12:50.768759  1415 raft_consensus.cc:359] T 976112eb8da2479e9a559399bee97dfc P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.778708  1415 raft_consensus.cc:385] T 976112eb8da2479e9a559399bee97dfc P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:50.778820  1272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae6ea4d73e243a19c55a262c620bb7d" candidate_uuid: "bbcbed9750ae416bab78287348e9e196" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe0ab937a564199977b910104824f30"
I20260709 13:12:50.779071  1415 raft_consensus.cc:740] T 976112eb8da2479e9a559399bee97dfc P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22ec8c2439d14f139d5330c8ec45133a, State: Initialized, Role: FOLLOWER
I20260709 13:12:50.779719  1272 raft_consensus.cc:2393] T 7ae6ea4d73e243a19c55a262c620bb7d P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bbcbed9750ae416bab78287348e9e196 in current term 1: Already voted for candidate efe0ab937a564199977b910104824f30 in this term.
I20260709 13:12:50.780025  1415 consensus_queue.cc:260] T 976112eb8da2479e9a559399bee97dfc P 22ec8c2439d14f139d5330c8ec45133a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.781847  1161 leader_election.cc:304] T 7ae6ea4d73e243a19c55a262c620bb7d P bbcbed9750ae416bab78287348e9e196 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bbcbed9750ae416bab78287348e9e196; no voters: 22ec8c2439d14f139d5330c8ec45133a, efe0ab937a564199977b910104824f30
I20260709 13:12:50.782672  1415 ts_tablet_manager.cc:1434] T 976112eb8da2479e9a559399bee97dfc P 22ec8c2439d14f139d5330c8ec45133a: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 13:12:50.783274  1440 raft_consensus.cc:2749] T 7ae6ea4d73e243a19c55a262c620bb7d P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 13:12:50.783867  1415 tablet_bootstrap.cc:492] T fe6a0e06494f416e80a6ebbec27396bd P 22ec8c2439d14f139d5330c8ec45133a: Bootstrap starting.
I20260709 13:12:50.783942  1122 raft_consensus.cc:2468] T 7ae6ea4d73e243a19c55a262c620bb7d P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe0ab937a564199977b910104824f30 in term 1.
I20260709 13:12:50.785521  1235 leader_election.cc:304] T 7ae6ea4d73e243a19c55a262c620bb7d P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22ec8c2439d14f139d5330c8ec45133a, efe0ab937a564199977b910104824f30; no voters: 
I20260709 13:12:50.773793  1414 tablet_bootstrap.cc:492] T fe6a0e06494f416e80a6ebbec27396bd P bbcbed9750ae416bab78287348e9e196: No bootstrap required, opened a new log
I20260709 13:12:50.772619  1413 raft_consensus.cc:359] T 5e059664b8d64cc49d217bfcddfc2232 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.786842  1414 ts_tablet_manager.cc:1403] T fe6a0e06494f416e80a6ebbec27396bd P bbcbed9750ae416bab78287348e9e196: Time spent bootstrapping tablet: real 0.124s	user 0.027s	sys 0.031s
I20260709 13:12:50.786877  1413 raft_consensus.cc:385] T 5e059664b8d64cc49d217bfcddfc2232 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:50.787127  1434 leader_election.cc:290] T 7ae6ea4d73e243a19c55a262c620bb7d P bbcbed9750ae416bab78287348e9e196 [CANDIDATE]: Term 1 election: Requested vote from peers 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943), efe0ab937a564199977b910104824f30 (127.21.63.3:33527)
I20260709 13:12:50.787470  1413 raft_consensus.cc:740] T 5e059664b8d64cc49d217bfcddfc2232 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe0ab937a564199977b910104824f30, State: Initialized, Role: FOLLOWER
I20260709 13:12:50.788362  1413 consensus_queue.cc:260] T 5e059664b8d64cc49d217bfcddfc2232 P efe0ab937a564199977b910104824f30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.790385  1414 raft_consensus.cc:359] T fe6a0e06494f416e80a6ebbec27396bd P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.791174  1414 raft_consensus.cc:385] T fe6a0e06494f416e80a6ebbec27396bd P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:50.776993  1197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae6ea4d73e243a19c55a262c620bb7d" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbcbed9750ae416bab78287348e9e196"
I20260709 13:12:50.791530  1414 raft_consensus.cc:740] T fe6a0e06494f416e80a6ebbec27396bd P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbcbed9750ae416bab78287348e9e196, State: Initialized, Role: FOLLOWER
I20260709 13:12:50.792166  1197 raft_consensus.cc:2393] T 7ae6ea4d73e243a19c55a262c620bb7d P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate efe0ab937a564199977b910104824f30 in current term 1: Already voted for candidate bbcbed9750ae416bab78287348e9e196 in this term.
I20260709 13:12:50.792711  1415 tablet_bootstrap.cc:654] T fe6a0e06494f416e80a6ebbec27396bd P 22ec8c2439d14f139d5330c8ec45133a: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:50.792497  1414 consensus_queue.cc:260] T fe6a0e06494f416e80a6ebbec27396bd P bbcbed9750ae416bab78287348e9e196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.794155  1441 raft_consensus.cc:2804] T 7ae6ea4d73e243a19c55a262c620bb7d P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:50.795241  1414 ts_tablet_manager.cc:1434] T fe6a0e06494f416e80a6ebbec27396bd P bbcbed9750ae416bab78287348e9e196: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 13:12:50.795244  1423 leader_election.cc:290] T 7ae6ea4d73e243a19c55a262c620bb7d P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 election: Requested vote from peers 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943), bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515)
I20260709 13:12:50.796717  1414 tablet_bootstrap.cc:492] T ac170d34df16448ea4c965890a7d22a1 P bbcbed9750ae416bab78287348e9e196: Bootstrap starting.
I20260709 13:12:50.805995  1413 ts_tablet_manager.cc:1434] T 5e059664b8d64cc49d217bfcddfc2232 P efe0ab937a564199977b910104824f30: Time spent starting tablet: real 0.037s	user 0.011s	sys 0.002s
I20260709 13:12:50.806213  1414 tablet_bootstrap.cc:654] T ac170d34df16448ea4c965890a7d22a1 P bbcbed9750ae416bab78287348e9e196: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:50.807143  1413 tablet_bootstrap.cc:492] T fe6a0e06494f416e80a6ebbec27396bd P efe0ab937a564199977b910104824f30: Bootstrap starting.
I20260709 13:12:50.815928  1415 tablet_bootstrap.cc:492] T fe6a0e06494f416e80a6ebbec27396bd P 22ec8c2439d14f139d5330c8ec45133a: No bootstrap required, opened a new log
I20260709 13:12:50.816525  1415 ts_tablet_manager.cc:1403] T fe6a0e06494f416e80a6ebbec27396bd P 22ec8c2439d14f139d5330c8ec45133a: Time spent bootstrapping tablet: real 0.033s	user 0.010s	sys 0.002s
I20260709 13:12:50.819584  1415 raft_consensus.cc:359] T fe6a0e06494f416e80a6ebbec27396bd P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.820290  1415 raft_consensus.cc:385] T fe6a0e06494f416e80a6ebbec27396bd P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:50.820597  1415 raft_consensus.cc:740] T fe6a0e06494f416e80a6ebbec27396bd P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22ec8c2439d14f139d5330c8ec45133a, State: Initialized, Role: FOLLOWER
I20260709 13:12:50.820710  1414 tablet_bootstrap.cc:492] T ac170d34df16448ea4c965890a7d22a1 P bbcbed9750ae416bab78287348e9e196: No bootstrap required, opened a new log
I20260709 13:12:50.821167  1441 raft_consensus.cc:697] T 7ae6ea4d73e243a19c55a262c620bb7d P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Becoming Leader. State: Replica: efe0ab937a564199977b910104824f30, State: Running, Role: LEADER
I20260709 13:12:50.821452  1414 ts_tablet_manager.cc:1403] T ac170d34df16448ea4c965890a7d22a1 P bbcbed9750ae416bab78287348e9e196: Time spent bootstrapping tablet: real 0.025s	user 0.022s	sys 0.000s
I20260709 13:12:50.822254  1441 consensus_queue.cc:237] T 7ae6ea4d73e243a19c55a262c620bb7d P efe0ab937a564199977b910104824f30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.824575  1321 raft_consensus.cc:493] T 5e059664b8d64cc49d217bfcddfc2232 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.825116  1321 raft_consensus.cc:515] T 5e059664b8d64cc49d217bfcddfc2232 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.824800  1019 catalog_manager.cc:5697] T 249a9a4a171d491b9f64f9b263bbb91a P efe0ab937a564199977b910104824f30 reported cstate change: term changed from 0 to 1, leader changed from <none> to efe0ab937a564199977b910104824f30 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "efe0ab937a564199977b910104824f30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:50.825313  1414 raft_consensus.cc:359] T ac170d34df16448ea4c965890a7d22a1 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.826499  1414 raft_consensus.cc:385] T ac170d34df16448ea4c965890a7d22a1 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:50.827005  1414 raft_consensus.cc:740] T ac170d34df16448ea4c965890a7d22a1 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbcbed9750ae416bab78287348e9e196, State: Initialized, Role: FOLLOWER
I20260709 13:12:50.827093  1321 leader_election.cc:290] T 5e059664b8d64cc49d217bfcddfc2232 P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515), efe0ab937a564199977b910104824f30 (127.21.63.3:33527)
I20260709 13:12:50.827953  1414 consensus_queue.cc:260] T ac170d34df16448ea4c965890a7d22a1 P bbcbed9750ae416bab78287348e9e196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.829167  1197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e059664b8d64cc49d217bfcddfc2232" candidate_uuid: "22ec8c2439d14f139d5330c8ec45133a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbcbed9750ae416bab78287348e9e196" is_pre_election: true
I20260709 13:12:50.829886  1197 raft_consensus.cc:2468] T 5e059664b8d64cc49d217bfcddfc2232 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22ec8c2439d14f139d5330c8ec45133a in term 0.
I20260709 13:12:50.830824  1414 ts_tablet_manager.cc:1434] T ac170d34df16448ea4c965890a7d22a1 P bbcbed9750ae416bab78287348e9e196: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 13:12:50.831243  1088 leader_election.cc:304] T 5e059664b8d64cc49d217bfcddfc2232 P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22ec8c2439d14f139d5330c8ec45133a, bbcbed9750ae416bab78287348e9e196; no voters: 
I20260709 13:12:50.828331  1272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e059664b8d64cc49d217bfcddfc2232" candidate_uuid: "22ec8c2439d14f139d5330c8ec45133a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe0ab937a564199977b910104824f30" is_pre_election: true
I20260709 13:12:50.832118  1414 tablet_bootstrap.cc:492] T 976112eb8da2479e9a559399bee97dfc P bbcbed9750ae416bab78287348e9e196: Bootstrap starting.
I20260709 13:12:50.832337  1272 raft_consensus.cc:2468] T 5e059664b8d64cc49d217bfcddfc2232 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22ec8c2439d14f139d5330c8ec45133a in term 0.
I20260709 13:12:50.832857  1415 consensus_queue.cc:260] T fe6a0e06494f416e80a6ebbec27396bd P 22ec8c2439d14f139d5330c8ec45133a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.834111  1416 raft_consensus.cc:2804] T 5e059664b8d64cc49d217bfcddfc2232 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:50.834488  1416 raft_consensus.cc:493] T 5e059664b8d64cc49d217bfcddfc2232 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:50.834884  1416 raft_consensus.cc:3060] T 5e059664b8d64cc49d217bfcddfc2232 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:50.835148  1415 ts_tablet_manager.cc:1434] T fe6a0e06494f416e80a6ebbec27396bd P 22ec8c2439d14f139d5330c8ec45133a: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.000s
I20260709 13:12:50.836649  1415 tablet_bootstrap.cc:492] T ac170d34df16448ea4c965890a7d22a1 P 22ec8c2439d14f139d5330c8ec45133a: Bootstrap starting.
I20260709 13:12:50.840829  1414 tablet_bootstrap.cc:654] T 976112eb8da2479e9a559399bee97dfc P bbcbed9750ae416bab78287348e9e196: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:50.846146  1416 raft_consensus.cc:515] T 5e059664b8d64cc49d217bfcddfc2232 P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.848963  1197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e059664b8d64cc49d217bfcddfc2232" candidate_uuid: "22ec8c2439d14f139d5330c8ec45133a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbcbed9750ae416bab78287348e9e196"
I20260709 13:12:50.849577  1197 raft_consensus.cc:3060] T 5e059664b8d64cc49d217bfcddfc2232 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:50.852646  1415 tablet_bootstrap.cc:654] T ac170d34df16448ea4c965890a7d22a1 P 22ec8c2439d14f139d5330c8ec45133a: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:50.855877  1414 tablet_bootstrap.cc:492] T 976112eb8da2479e9a559399bee97dfc P bbcbed9750ae416bab78287348e9e196: No bootstrap required, opened a new log
I20260709 13:12:50.856587  1414 ts_tablet_manager.cc:1403] T 976112eb8da2479e9a559399bee97dfc P bbcbed9750ae416bab78287348e9e196: Time spent bootstrapping tablet: real 0.025s	user 0.013s	sys 0.007s
I20260709 13:12:50.859349  1414 raft_consensus.cc:359] T 976112eb8da2479e9a559399bee97dfc P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.859896  1414 raft_consensus.cc:385] T 976112eb8da2479e9a559399bee97dfc P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:50.860136  1414 raft_consensus.cc:740] T 976112eb8da2479e9a559399bee97dfc P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbcbed9750ae416bab78287348e9e196, State: Initialized, Role: FOLLOWER
I20260709 13:12:50.860929  1414 consensus_queue.cc:260] T 976112eb8da2479e9a559399bee97dfc P bbcbed9750ae416bab78287348e9e196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.862685  1414 ts_tablet_manager.cc:1434] T 976112eb8da2479e9a559399bee97dfc P bbcbed9750ae416bab78287348e9e196: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:12:50.863811  1414 tablet_bootstrap.cc:492] T 8e4161719e214f1a9836f352271dc985 P bbcbed9750ae416bab78287348e9e196: Bootstrap starting.
I20260709 13:12:50.865722  1197 raft_consensus.cc:2468] T 5e059664b8d64cc49d217bfcddfc2232 P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22ec8c2439d14f139d5330c8ec45133a in term 1.
I20260709 13:12:50.867386  1088 leader_election.cc:304] T 5e059664b8d64cc49d217bfcddfc2232 P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22ec8c2439d14f139d5330c8ec45133a, bbcbed9750ae416bab78287348e9e196; no voters: 
I20260709 13:12:50.868539  1321 raft_consensus.cc:2804] T 5e059664b8d64cc49d217bfcddfc2232 P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:50.869079  1414 tablet_bootstrap.cc:654] T 8e4161719e214f1a9836f352271dc985 P bbcbed9750ae416bab78287348e9e196: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:50.881520  1321 raft_consensus.cc:697] T 5e059664b8d64cc49d217bfcddfc2232 P 22ec8c2439d14f139d5330c8ec45133a [term 1 LEADER]: Becoming Leader. State: Replica: 22ec8c2439d14f139d5330c8ec45133a, State: Running, Role: LEADER
I20260709 13:12:50.882627  1321 consensus_queue.cc:237] T 5e059664b8d64cc49d217bfcddfc2232 P 22ec8c2439d14f139d5330c8ec45133a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.895345  1272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e059664b8d64cc49d217bfcddfc2232" candidate_uuid: "22ec8c2439d14f139d5330c8ec45133a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe0ab937a564199977b910104824f30"
I20260709 13:12:50.896059  1272 raft_consensus.cc:3060] T 5e059664b8d64cc49d217bfcddfc2232 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:50.904824  1416 leader_election.cc:290] T 5e059664b8d64cc49d217bfcddfc2232 P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 1 election: Requested vote from peers bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515), efe0ab937a564199977b910104824f30 (127.21.63.3:33527)
I20260709 13:12:50.907198  1272 raft_consensus.cc:2468] T 5e059664b8d64cc49d217bfcddfc2232 P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22ec8c2439d14f139d5330c8ec45133a in term 1.
I20260709 13:12:50.911566  1414 tablet_bootstrap.cc:492] T 8e4161719e214f1a9836f352271dc985 P bbcbed9750ae416bab78287348e9e196: No bootstrap required, opened a new log
I20260709 13:12:50.912139  1414 ts_tablet_manager.cc:1403] T 8e4161719e214f1a9836f352271dc985 P bbcbed9750ae416bab78287348e9e196: Time spent bootstrapping tablet: real 0.048s	user 0.016s	sys 0.012s
I20260709 13:12:50.914863  1413 tablet_bootstrap.cc:654] T fe6a0e06494f416e80a6ebbec27396bd P efe0ab937a564199977b910104824f30: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:50.915158  1414 raft_consensus.cc:359] T 8e4161719e214f1a9836f352271dc985 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.915958  1414 raft_consensus.cc:385] T 8e4161719e214f1a9836f352271dc985 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:50.916285  1414 raft_consensus.cc:740] T 8e4161719e214f1a9836f352271dc985 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbcbed9750ae416bab78287348e9e196, State: Initialized, Role: FOLLOWER
I20260709 13:12:50.916203  1018 catalog_manager.cc:5697] T 7ae6ea4d73e243a19c55a262c620bb7d P efe0ab937a564199977b910104824f30 reported cstate change: term changed from 0 to 1, leader changed from <none> to efe0ab937a564199977b910104824f30 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "efe0ab937a564199977b910104824f30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:50.917222  1414 consensus_queue.cc:260] T 8e4161719e214f1a9836f352271dc985 P bbcbed9750ae416bab78287348e9e196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.920008  1414 ts_tablet_manager.cc:1434] T 8e4161719e214f1a9836f352271dc985 P bbcbed9750ae416bab78287348e9e196: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 13:12:50.925503  1415 tablet_bootstrap.cc:492] T ac170d34df16448ea4c965890a7d22a1 P 22ec8c2439d14f139d5330c8ec45133a: No bootstrap required, opened a new log
I20260709 13:12:50.926033  1415 ts_tablet_manager.cc:1403] T ac170d34df16448ea4c965890a7d22a1 P 22ec8c2439d14f139d5330c8ec45133a: Time spent bootstrapping tablet: real 0.090s	user 0.034s	sys 0.019s
I20260709 13:12:50.927379  1413 tablet_bootstrap.cc:492] T fe6a0e06494f416e80a6ebbec27396bd P efe0ab937a564199977b910104824f30: No bootstrap required, opened a new log
I20260709 13:12:50.928025  1413 ts_tablet_manager.cc:1403] T fe6a0e06494f416e80a6ebbec27396bd P efe0ab937a564199977b910104824f30: Time spent bootstrapping tablet: real 0.121s	user 0.010s	sys 0.010s
I20260709 13:12:50.929203  1415 raft_consensus.cc:359] T ac170d34df16448ea4c965890a7d22a1 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.929755  1415 raft_consensus.cc:385] T ac170d34df16448ea4c965890a7d22a1 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:50.929967  1415 raft_consensus.cc:740] T ac170d34df16448ea4c965890a7d22a1 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22ec8c2439d14f139d5330c8ec45133a, State: Initialized, Role: FOLLOWER
I20260709 13:12:50.930707  1415 consensus_queue.cc:260] T ac170d34df16448ea4c965890a7d22a1 P 22ec8c2439d14f139d5330c8ec45133a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.932072  1019 catalog_manager.cc:5697] T 5e059664b8d64cc49d217bfcddfc2232 P 22ec8c2439d14f139d5330c8ec45133a reported cstate change: term changed from 0 to 1, leader changed from <none> to 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1). New cstate: current_term: 1 leader_uuid: "22ec8c2439d14f139d5330c8ec45133a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:50.932739  1415 ts_tablet_manager.cc:1434] T ac170d34df16448ea4c965890a7d22a1 P 22ec8c2439d14f139d5330c8ec45133a: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 13:12:50.934015  1415 tablet_bootstrap.cc:492] T 8e4161719e214f1a9836f352271dc985 P 22ec8c2439d14f139d5330c8ec45133a: Bootstrap starting.
I20260709 13:12:50.932729  1413 raft_consensus.cc:359] T fe6a0e06494f416e80a6ebbec27396bd P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.936409  1413 raft_consensus.cc:385] T fe6a0e06494f416e80a6ebbec27396bd P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:50.936853  1413 raft_consensus.cc:740] T fe6a0e06494f416e80a6ebbec27396bd P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe0ab937a564199977b910104824f30, State: Initialized, Role: FOLLOWER
I20260709 13:12:50.937794  1413 consensus_queue.cc:260] T fe6a0e06494f416e80a6ebbec27396bd P efe0ab937a564199977b910104824f30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.939173  1415 tablet_bootstrap.cc:654] T 8e4161719e214f1a9836f352271dc985 P 22ec8c2439d14f139d5330c8ec45133a: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:50.939879  1413 ts_tablet_manager.cc:1434] T fe6a0e06494f416e80a6ebbec27396bd P efe0ab937a564199977b910104824f30: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 13:12:50.941062  1413 tablet_bootstrap.cc:492] T 8e4161719e214f1a9836f352271dc985 P efe0ab937a564199977b910104824f30: Bootstrap starting.
I20260709 13:12:50.948997  1399 raft_consensus.cc:493] T fe6a0e06494f416e80a6ebbec27396bd P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.949594  1399 raft_consensus.cc:515] T fe6a0e06494f416e80a6ebbec27396bd P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.950356  1415 tablet_bootstrap.cc:492] T 8e4161719e214f1a9836f352271dc985 P 22ec8c2439d14f139d5330c8ec45133a: No bootstrap required, opened a new log
I20260709 13:12:50.950845  1415 ts_tablet_manager.cc:1403] T 8e4161719e214f1a9836f352271dc985 P 22ec8c2439d14f139d5330c8ec45133a: Time spent bootstrapping tablet: real 0.017s	user 0.010s	sys 0.004s
I20260709 13:12:50.951915  1399 leader_election.cc:290] T fe6a0e06494f416e80a6ebbec27396bd P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943), bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515)
I20260709 13:12:50.952497  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe6a0e06494f416e80a6ebbec27396bd" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22ec8c2439d14f139d5330c8ec45133a" is_pre_election: true
I20260709 13:12:50.953296  1122 raft_consensus.cc:2468] T fe6a0e06494f416e80a6ebbec27396bd P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efe0ab937a564199977b910104824f30 in term 0.
I20260709 13:12:50.953485  1415 raft_consensus.cc:359] T 8e4161719e214f1a9836f352271dc985 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.953683  1197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe6a0e06494f416e80a6ebbec27396bd" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbcbed9750ae416bab78287348e9e196" is_pre_election: true
I20260709 13:12:50.954311  1415 raft_consensus.cc:385] T 8e4161719e214f1a9836f352271dc985 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:50.954624  1415 raft_consensus.cc:740] T 8e4161719e214f1a9836f352271dc985 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22ec8c2439d14f139d5330c8ec45133a, State: Initialized, Role: FOLLOWER
I20260709 13:12:50.954559  1197 raft_consensus.cc:2468] T fe6a0e06494f416e80a6ebbec27396bd P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efe0ab937a564199977b910104824f30 in term 0.
I20260709 13:12:50.954826  1235 leader_election.cc:304] T fe6a0e06494f416e80a6ebbec27396bd P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22ec8c2439d14f139d5330c8ec45133a, efe0ab937a564199977b910104824f30; no voters: 
I20260709 13:12:50.955555  1415 consensus_queue.cc:260] T 8e4161719e214f1a9836f352271dc985 P 22ec8c2439d14f139d5330c8ec45133a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.956241  1399 raft_consensus.cc:2804] T fe6a0e06494f416e80a6ebbec27396bd P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:50.956625  1399 raft_consensus.cc:493] T fe6a0e06494f416e80a6ebbec27396bd P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:50.956980  1399 raft_consensus.cc:3060] T fe6a0e06494f416e80a6ebbec27396bd P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:50.957700  1415 ts_tablet_manager.cc:1434] T 8e4161719e214f1a9836f352271dc985 P 22ec8c2439d14f139d5330c8ec45133a: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.001s
I20260709 13:12:50.960523  1413 tablet_bootstrap.cc:654] T 8e4161719e214f1a9836f352271dc985 P efe0ab937a564199977b910104824f30: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:50.963406  1399 raft_consensus.cc:515] T fe6a0e06494f416e80a6ebbec27396bd P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.966084  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe6a0e06494f416e80a6ebbec27396bd" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22ec8c2439d14f139d5330c8ec45133a"
I20260709 13:12:50.966367  1399 leader_election.cc:290] T fe6a0e06494f416e80a6ebbec27396bd P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 election: Requested vote from peers 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943), bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515)
I20260709 13:12:50.966575  1197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe6a0e06494f416e80a6ebbec27396bd" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbcbed9750ae416bab78287348e9e196"
I20260709 13:12:50.966799  1122 raft_consensus.cc:3060] T fe6a0e06494f416e80a6ebbec27396bd P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:50.966951  1399 raft_consensus.cc:493] T 976112eb8da2479e9a559399bee97dfc P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:50.967161  1197 raft_consensus.cc:3060] T fe6a0e06494f416e80a6ebbec27396bd P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:50.967742  1399 raft_consensus.cc:515] T 976112eb8da2479e9a559399bee97dfc P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.970651  1123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "976112eb8da2479e9a559399bee97dfc" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22ec8c2439d14f139d5330c8ec45133a" is_pre_election: true
I20260709 13:12:50.971284  1123 raft_consensus.cc:2468] T 976112eb8da2479e9a559399bee97dfc P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efe0ab937a564199977b910104824f30 in term 0.
I20260709 13:12:50.972987  1235 leader_election.cc:304] T 976112eb8da2479e9a559399bee97dfc P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22ec8c2439d14f139d5330c8ec45133a, efe0ab937a564199977b910104824f30; no voters: 
I20260709 13:12:50.975500  1441 raft_consensus.cc:2804] T 976112eb8da2479e9a559399bee97dfc P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:50.975646  1197 raft_consensus.cc:2468] T fe6a0e06494f416e80a6ebbec27396bd P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe0ab937a564199977b910104824f30 in term 1.
I20260709 13:12:50.975903  1441 raft_consensus.cc:493] T 976112eb8da2479e9a559399bee97dfc P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:50.976962  1441 raft_consensus.cc:3060] T 976112eb8da2479e9a559399bee97dfc P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:50.976929  1237 leader_election.cc:304] T fe6a0e06494f416e80a6ebbec27396bd P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bbcbed9750ae416bab78287348e9e196, efe0ab937a564199977b910104824f30; no voters: 
I20260709 13:12:50.980333  1122 raft_consensus.cc:2468] T fe6a0e06494f416e80a6ebbec27396bd P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe0ab937a564199977b910104824f30 in term 1.
I20260709 13:12:50.984157  1442 raft_consensus.cc:2804] T fe6a0e06494f416e80a6ebbec27396bd P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:50.984735  1442 raft_consensus.cc:697] T fe6a0e06494f416e80a6ebbec27396bd P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Becoming Leader. State: Replica: efe0ab937a564199977b910104824f30, State: Running, Role: LEADER
I20260709 13:12:50.985548  1442 consensus_queue.cc:237] T fe6a0e06494f416e80a6ebbec27396bd P efe0ab937a564199977b910104824f30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.986336  1399 leader_election.cc:290] T 976112eb8da2479e9a559399bee97dfc P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943), bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515)
I20260709 13:12:50.987422  1413 tablet_bootstrap.cc:492] T 8e4161719e214f1a9836f352271dc985 P efe0ab937a564199977b910104824f30: No bootstrap required, opened a new log
I20260709 13:12:50.987579  1197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "976112eb8da2479e9a559399bee97dfc" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbcbed9750ae416bab78287348e9e196" is_pre_election: true
I20260709 13:12:50.987941  1413 ts_tablet_manager.cc:1403] T 8e4161719e214f1a9836f352271dc985 P efe0ab937a564199977b910104824f30: Time spent bootstrapping tablet: real 0.047s	user 0.016s	sys 0.004s
I20260709 13:12:50.988428  1197 raft_consensus.cc:2468] T 976112eb8da2479e9a559399bee97dfc P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efe0ab937a564199977b910104824f30 in term 0.
I20260709 13:12:50.989322  1441 raft_consensus.cc:515] T 976112eb8da2479e9a559399bee97dfc P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.991111  1413 raft_consensus.cc:359] T 8e4161719e214f1a9836f352271dc985 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.992286  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "976112eb8da2479e9a559399bee97dfc" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22ec8c2439d14f139d5330c8ec45133a"
I20260709 13:12:50.992422  1441 leader_election.cc:290] T 976112eb8da2479e9a559399bee97dfc P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 election: Requested vote from peers 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943), bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515)
I20260709 13:12:50.993006  1122 raft_consensus.cc:3060] T 976112eb8da2479e9a559399bee97dfc P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:50.992738  1197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "976112eb8da2479e9a559399bee97dfc" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbcbed9750ae416bab78287348e9e196"
I20260709 13:12:50.991864  1413 raft_consensus.cc:385] T 8e4161719e214f1a9836f352271dc985 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:50.995716  1197 raft_consensus.cc:3060] T 976112eb8da2479e9a559399bee97dfc P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:50.996054  1413 raft_consensus.cc:740] T 8e4161719e214f1a9836f352271dc985 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe0ab937a564199977b910104824f30, State: Initialized, Role: FOLLOWER
I20260709 13:12:50.996896  1413 consensus_queue.cc:260] T 8e4161719e214f1a9836f352271dc985 P efe0ab937a564199977b910104824f30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:50.998111  1019 catalog_manager.cc:5697] T fe6a0e06494f416e80a6ebbec27396bd P efe0ab937a564199977b910104824f30 reported cstate change: term changed from 0 to 1, leader changed from <none> to efe0ab937a564199977b910104824f30 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "efe0ab937a564199977b910104824f30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:50.998975  1413 ts_tablet_manager.cc:1434] T 8e4161719e214f1a9836f352271dc985 P efe0ab937a564199977b910104824f30: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.004s
I20260709 13:12:51.000247  1413 tablet_bootstrap.cc:492] T ac170d34df16448ea4c965890a7d22a1 P efe0ab937a564199977b910104824f30: Bootstrap starting.
I20260709 13:12:51.000362  1122 raft_consensus.cc:2468] T 976112eb8da2479e9a559399bee97dfc P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe0ab937a564199977b910104824f30 in term 1.
I20260709 13:12:51.001689  1235 leader_election.cc:304] T 976112eb8da2479e9a559399bee97dfc P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22ec8c2439d14f139d5330c8ec45133a, efe0ab937a564199977b910104824f30; no voters: 
I20260709 13:12:51.002590  1442 raft_consensus.cc:2804] T 976112eb8da2479e9a559399bee97dfc P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:51.003044  1197 raft_consensus.cc:2468] T 976112eb8da2479e9a559399bee97dfc P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe0ab937a564199977b910104824f30 in term 1.
I20260709 13:12:51.003337  1442 raft_consensus.cc:697] T 976112eb8da2479e9a559399bee97dfc P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Becoming Leader. State: Replica: efe0ab937a564199977b910104824f30, State: Running, Role: LEADER
I20260709 13:12:51.004343  1442 consensus_queue.cc:237] T 976112eb8da2479e9a559399bee97dfc P efe0ab937a564199977b910104824f30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:51.007217  1413 tablet_bootstrap.cc:654] T ac170d34df16448ea4c965890a7d22a1 P efe0ab937a564199977b910104824f30: Neither blocks nor log segments found. Creating new log.
I20260709 13:12:51.012185  1434 raft_consensus.cc:493] T 8e4161719e214f1a9836f352271dc985 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:51.012658  1413 tablet_bootstrap.cc:492] T ac170d34df16448ea4c965890a7d22a1 P efe0ab937a564199977b910104824f30: No bootstrap required, opened a new log
I20260709 13:12:51.013229  1413 ts_tablet_manager.cc:1403] T ac170d34df16448ea4c965890a7d22a1 P efe0ab937a564199977b910104824f30: Time spent bootstrapping tablet: real 0.013s	user 0.011s	sys 0.000s
I20260709 13:12:51.013132  1434 raft_consensus.cc:515] T 8e4161719e214f1a9836f352271dc985 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:51.015563  1413 raft_consensus.cc:359] T ac170d34df16448ea4c965890a7d22a1 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:51.016207  1413 raft_consensus.cc:385] T ac170d34df16448ea4c965890a7d22a1 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 13:12:51.016340  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4161719e214f1a9836f352271dc985" candidate_uuid: "bbcbed9750ae416bab78287348e9e196" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22ec8c2439d14f139d5330c8ec45133a" is_pre_election: true
I20260709 13:12:51.016711  1272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4161719e214f1a9836f352271dc985" candidate_uuid: "bbcbed9750ae416bab78287348e9e196" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe0ab937a564199977b910104824f30" is_pre_election: true
I20260709 13:12:51.016870  1413 raft_consensus.cc:740] T ac170d34df16448ea4c965890a7d22a1 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efe0ab937a564199977b910104824f30, State: Initialized, Role: FOLLOWER
I20260709 13:12:51.017292  1122 raft_consensus.cc:2468] T 8e4161719e214f1a9836f352271dc985 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bbcbed9750ae416bab78287348e9e196 in term 0.
I20260709 13:12:51.017448  1434 leader_election.cc:290] T 8e4161719e214f1a9836f352271dc985 P bbcbed9750ae416bab78287348e9e196 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943), efe0ab937a564199977b910104824f30 (127.21.63.3:33527)
I20260709 13:12:51.017560  1272 raft_consensus.cc:2468] T 8e4161719e214f1a9836f352271dc985 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bbcbed9750ae416bab78287348e9e196 in term 0.
I20260709 13:12:51.016186  1019 catalog_manager.cc:5697] T 976112eb8da2479e9a559399bee97dfc P efe0ab937a564199977b910104824f30 reported cstate change: term changed from 0 to 1, leader changed from <none> to efe0ab937a564199977b910104824f30 (127.21.63.3). New cstate: current_term: 1 leader_uuid: "efe0ab937a564199977b910104824f30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:51.017840  1413 consensus_queue.cc:260] T ac170d34df16448ea4c965890a7d22a1 P efe0ab937a564199977b910104824f30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:51.018625  1161 leader_election.cc:304] T 8e4161719e214f1a9836f352271dc985 P bbcbed9750ae416bab78287348e9e196 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22ec8c2439d14f139d5330c8ec45133a, bbcbed9750ae416bab78287348e9e196; no voters: 
I20260709 13:12:51.019526  1434 raft_consensus.cc:2804] T 8e4161719e214f1a9836f352271dc985 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:51.019878  1434 raft_consensus.cc:493] T 8e4161719e214f1a9836f352271dc985 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:51.020035  1413 ts_tablet_manager.cc:1434] T ac170d34df16448ea4c965890a7d22a1 P efe0ab937a564199977b910104824f30: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 13:12:51.020217  1434 raft_consensus.cc:3060] T 8e4161719e214f1a9836f352271dc985 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:51.025974  1434 raft_consensus.cc:515] T 8e4161719e214f1a9836f352271dc985 P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:51.027726  1434 leader_election.cc:290] T 8e4161719e214f1a9836f352271dc985 P bbcbed9750ae416bab78287348e9e196 [CANDIDATE]: Term 1 election: Requested vote from peers 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943), efe0ab937a564199977b910104824f30 (127.21.63.3:33527)
I20260709 13:12:51.028451  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4161719e214f1a9836f352271dc985" candidate_uuid: "bbcbed9750ae416bab78287348e9e196" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22ec8c2439d14f139d5330c8ec45133a"
I20260709 13:12:51.028790  1272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e4161719e214f1a9836f352271dc985" candidate_uuid: "bbcbed9750ae416bab78287348e9e196" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe0ab937a564199977b910104824f30"
I20260709 13:12:51.029114  1122 raft_consensus.cc:3060] T 8e4161719e214f1a9836f352271dc985 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:51.029449  1272 raft_consensus.cc:3060] T 8e4161719e214f1a9836f352271dc985 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:51.034017  1122 raft_consensus.cc:2468] T 8e4161719e214f1a9836f352271dc985 P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbcbed9750ae416bab78287348e9e196 in term 1.
I20260709 13:12:51.035035  1161 leader_election.cc:304] T 8e4161719e214f1a9836f352271dc985 P bbcbed9750ae416bab78287348e9e196 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22ec8c2439d14f139d5330c8ec45133a, bbcbed9750ae416bab78287348e9e196; no voters: 
I20260709 13:12:51.035398  1272 raft_consensus.cc:2468] T 8e4161719e214f1a9836f352271dc985 P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbcbed9750ae416bab78287348e9e196 in term 1.
I20260709 13:12:51.035892  1434 raft_consensus.cc:2804] T 8e4161719e214f1a9836f352271dc985 P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:51.036388  1434 raft_consensus.cc:697] T 8e4161719e214f1a9836f352271dc985 P bbcbed9750ae416bab78287348e9e196 [term 1 LEADER]: Becoming Leader. State: Replica: bbcbed9750ae416bab78287348e9e196, State: Running, Role: LEADER
I20260709 13:12:51.037185  1434 consensus_queue.cc:237] T 8e4161719e214f1a9836f352271dc985 P bbcbed9750ae416bab78287348e9e196 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:51.044947  1019 catalog_manager.cc:5697] T 8e4161719e214f1a9836f352271dc985 P bbcbed9750ae416bab78287348e9e196 reported cstate change: term changed from 0 to 1, leader changed from <none> to bbcbed9750ae416bab78287348e9e196 (127.21.63.2). New cstate: current_term: 1 leader_uuid: "bbcbed9750ae416bab78287348e9e196" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:51.051656  1442 consensus_queue.cc:1048] T 0be8303fce99487c9cb34506b3181363 P efe0ab937a564199977b910104824f30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:51.064110  1441 consensus_queue.cc:1048] T 0be8303fce99487c9cb34506b3181363 P efe0ab937a564199977b910104824f30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:51.126371  1454 raft_consensus.cc:493] T ac170d34df16448ea4c965890a7d22a1 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 13:12:51.128314  1454 raft_consensus.cc:515] T ac170d34df16448ea4c965890a7d22a1 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:51.132009  1454 leader_election.cc:290] T ac170d34df16448ea4c965890a7d22a1 P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515), efe0ab937a564199977b910104824f30 (127.21.63.3:33527)
I20260709 13:12:51.133934  1197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac170d34df16448ea4c965890a7d22a1" candidate_uuid: "22ec8c2439d14f139d5330c8ec45133a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbcbed9750ae416bab78287348e9e196" is_pre_election: true
I20260709 13:12:51.135082  1272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac170d34df16448ea4c965890a7d22a1" candidate_uuid: "22ec8c2439d14f139d5330c8ec45133a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe0ab937a564199977b910104824f30" is_pre_election: true
I20260709 13:12:51.135458  1197 raft_consensus.cc:2468] T ac170d34df16448ea4c965890a7d22a1 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22ec8c2439d14f139d5330c8ec45133a in term 0.
I20260709 13:12:51.136186  1272 raft_consensus.cc:2468] T ac170d34df16448ea4c965890a7d22a1 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22ec8c2439d14f139d5330c8ec45133a in term 0.
I20260709 13:12:51.137416  1088 leader_election.cc:304] T ac170d34df16448ea4c965890a7d22a1 P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22ec8c2439d14f139d5330c8ec45133a, bbcbed9750ae416bab78287348e9e196; no voters: 
I20260709 13:12:51.138603  1454 raft_consensus.cc:2804] T ac170d34df16448ea4c965890a7d22a1 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 13:12:51.139081  1454 raft_consensus.cc:493] T ac170d34df16448ea4c965890a7d22a1 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 13:12:51.139513  1454 raft_consensus.cc:3060] T ac170d34df16448ea4c965890a7d22a1 P 22ec8c2439d14f139d5330c8ec45133a [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:51.150765  1454 raft_consensus.cc:515] T ac170d34df16448ea4c965890a7d22a1 P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:51.152482  1454 leader_election.cc:290] T ac170d34df16448ea4c965890a7d22a1 P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 1 election: Requested vote from peers bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515), efe0ab937a564199977b910104824f30 (127.21.63.3:33527)
I20260709 13:12:51.153388  1197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac170d34df16448ea4c965890a7d22a1" candidate_uuid: "22ec8c2439d14f139d5330c8ec45133a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbcbed9750ae416bab78287348e9e196"
I20260709 13:12:51.153903  1197 raft_consensus.cc:3060] T ac170d34df16448ea4c965890a7d22a1 P bbcbed9750ae416bab78287348e9e196 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:51.153571  1272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac170d34df16448ea4c965890a7d22a1" candidate_uuid: "22ec8c2439d14f139d5330c8ec45133a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efe0ab937a564199977b910104824f30"
I20260709 13:12:51.154541  1272 raft_consensus.cc:3060] T ac170d34df16448ea4c965890a7d22a1 P efe0ab937a564199977b910104824f30 [term 0 FOLLOWER]: Advancing to term 1
I20260709 13:12:51.163704  1197 raft_consensus.cc:2468] T ac170d34df16448ea4c965890a7d22a1 P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22ec8c2439d14f139d5330c8ec45133a in term 1.
I20260709 13:12:51.165091  1088 leader_election.cc:304] T ac170d34df16448ea4c965890a7d22a1 P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22ec8c2439d14f139d5330c8ec45133a, bbcbed9750ae416bab78287348e9e196; no voters: 
I20260709 13:12:51.166424  1321 raft_consensus.cc:2804] T ac170d34df16448ea4c965890a7d22a1 P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Leader election won for term 1
I20260709 13:12:51.167201  1321 raft_consensus.cc:697] T ac170d34df16448ea4c965890a7d22a1 P 22ec8c2439d14f139d5330c8ec45133a [term 1 LEADER]: Becoming Leader. State: Replica: 22ec8c2439d14f139d5330c8ec45133a, State: Running, Role: LEADER
I20260709 13:12:51.169004  1321 consensus_queue.cc:237] T ac170d34df16448ea4c965890a7d22a1 P 22ec8c2439d14f139d5330c8ec45133a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:51.171408  1272 raft_consensus.cc:2468] T ac170d34df16448ea4c965890a7d22a1 P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22ec8c2439d14f139d5330c8ec45133a in term 1.
I20260709 13:12:51.182941  1019 catalog_manager.cc:5697] T ac170d34df16448ea4c965890a7d22a1 P 22ec8c2439d14f139d5330c8ec45133a reported cstate change: term changed from 0 to 1, leader changed from <none> to 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1). New cstate: current_term: 1 leader_uuid: "22ec8c2439d14f139d5330c8ec45133a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:51.203886  1441 consensus_queue.cc:1048] T 249a9a4a171d491b9f64f9b263bbb91a P efe0ab937a564199977b910104824f30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:51.217597  1441 consensus_queue.cc:1048] T 249a9a4a171d491b9f64f9b263bbb91a P efe0ab937a564199977b910104824f30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:51.230007  1441 consensus_queue.cc:1048] T 7ae6ea4d73e243a19c55a262c620bb7d P efe0ab937a564199977b910104824f30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:51.245287  1399 consensus_queue.cc:1048] T 7ae6ea4d73e243a19c55a262c620bb7d P efe0ab937a564199977b910104824f30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:51.273103  1321 consensus_queue.cc:1048] T 5e059664b8d64cc49d217bfcddfc2232 P 22ec8c2439d14f139d5330c8ec45133a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:51.288226  1321 consensus_queue.cc:1048] T 5e059664b8d64cc49d217bfcddfc2232 P 22ec8c2439d14f139d5330c8ec45133a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:51.299968  1434 consensus_queue.cc:1048] T 4bf47f99c081476eabfaeb56515e0d79 P bbcbed9750ae416bab78287348e9e196 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:51.301081  1440 consensus_queue.cc:1048] T 4bf47f99c081476eabfaeb56515e0d79 P bbcbed9750ae416bab78287348e9e196 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:51.384146  1269 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0be8303fce99487c9cb34506b3181363"
dest_uuid: "efe0ab937a564199977b910104824f30"
mode: GRACEFUL
new_leader_uuid: "22ec8c2439d14f139d5330c8ec45133a"
 from {username='slave'} at 127.0.0.1:52974
I20260709 13:12:51.385155  1269 raft_consensus.cc:606] T 0be8303fce99487c9cb34506b3181363 P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Received request to transfer leadership to TS 22ec8c2439d14f139d5330c8ec45133a
I20260709 13:12:51.387378  1399 consensus_queue.cc:1048] T fe6a0e06494f416e80a6ebbec27396bd P efe0ab937a564199977b910104824f30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:51.391754  1196 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4bf47f99c081476eabfaeb56515e0d79"
dest_uuid: "bbcbed9750ae416bab78287348e9e196"
mode: GRACEFUL
new_leader_uuid: "22ec8c2439d14f139d5330c8ec45133a"
 from {username='slave'} at 127.0.0.1:60790
I20260709 13:12:51.392446  1196 raft_consensus.cc:606] T 4bf47f99c081476eabfaeb56515e0d79 P bbcbed9750ae416bab78287348e9e196 [term 1 LEADER]: Received request to transfer leadership to TS 22ec8c2439d14f139d5330c8ec45133a
I20260709 13:12:51.400765  1441 consensus_queue.cc:1048] T fe6a0e06494f416e80a6ebbec27396bd P efe0ab937a564199977b910104824f30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:51.402148  1269 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7ae6ea4d73e243a19c55a262c620bb7d"
dest_uuid: "efe0ab937a564199977b910104824f30"
mode: GRACEFUL
new_leader_uuid: "22ec8c2439d14f139d5330c8ec45133a"
 from {username='slave'} at 127.0.0.1:52974
I20260709 13:12:51.402833  1269 raft_consensus.cc:606] T 7ae6ea4d73e243a19c55a262c620bb7d P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Received request to transfer leadership to TS 22ec8c2439d14f139d5330c8ec45133a
I20260709 13:12:51.410597  1269 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "249a9a4a171d491b9f64f9b263bbb91a"
dest_uuid: "efe0ab937a564199977b910104824f30"
mode: GRACEFUL
new_leader_uuid: "22ec8c2439d14f139d5330c8ec45133a"
 from {username='slave'} at 127.0.0.1:52974
I20260709 13:12:51.411266  1269 raft_consensus.cc:606] T 249a9a4a171d491b9f64f9b263bbb91a P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Received request to transfer leadership to TS 22ec8c2439d14f139d5330c8ec45133a
I20260709 13:12:51.420477  1269 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "976112eb8da2479e9a559399bee97dfc"
dest_uuid: "efe0ab937a564199977b910104824f30"
mode: GRACEFUL
new_leader_uuid: "bbcbed9750ae416bab78287348e9e196"
 from {username='slave'} at 127.0.0.1:52974
I20260709 13:12:51.421164  1269 raft_consensus.cc:606] T 976112eb8da2479e9a559399bee97dfc P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Received request to transfer leadership to TS bbcbed9750ae416bab78287348e9e196
I20260709 13:12:51.427713  1122 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5e059664b8d64cc49d217bfcddfc2232"
dest_uuid: "22ec8c2439d14f139d5330c8ec45133a"
mode: GRACEFUL
new_leader_uuid: "bbcbed9750ae416bab78287348e9e196"
 from {username='slave'} at 127.0.0.1:56378
I20260709 13:12:51.428191  1122 raft_consensus.cc:606] T 5e059664b8d64cc49d217bfcddfc2232 P 22ec8c2439d14f139d5330c8ec45133a [term 1 LEADER]: Received request to transfer leadership to TS bbcbed9750ae416bab78287348e9e196
I20260709 13:12:51.433964  1269 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fe6a0e06494f416e80a6ebbec27396bd"
dest_uuid: "efe0ab937a564199977b910104824f30"
mode: GRACEFUL
new_leader_uuid: "bbcbed9750ae416bab78287348e9e196"
 from {username='slave'} at 127.0.0.1:52974
I20260709 13:12:51.434546  1269 raft_consensus.cc:606] T fe6a0e06494f416e80a6ebbec27396bd P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Received request to transfer leadership to TS bbcbed9750ae416bab78287348e9e196
I20260709 13:12:51.442306  1122 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ac170d34df16448ea4c965890a7d22a1"
dest_uuid: "22ec8c2439d14f139d5330c8ec45133a"
mode: GRACEFUL
new_leader_uuid: "efe0ab937a564199977b910104824f30"
 from {username='slave'} at 127.0.0.1:56378
I20260709 13:12:51.442833  1122 raft_consensus.cc:606] T ac170d34df16448ea4c965890a7d22a1 P 22ec8c2439d14f139d5330c8ec45133a [term 1 LEADER]: Received request to transfer leadership to TS efe0ab937a564199977b910104824f30
I20260709 13:12:51.550966  1434 consensus_queue.cc:1048] T 8e4161719e214f1a9836f352271dc985 P bbcbed9750ae416bab78287348e9e196 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:51.557235  1441 consensus_queue.cc:1048] T 976112eb8da2479e9a559399bee97dfc P efe0ab937a564199977b910104824f30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:51.565974  1434 consensus_queue.cc:1048] T 8e4161719e214f1a9836f352271dc985 P bbcbed9750ae416bab78287348e9e196 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:51.580813  1441 consensus_queue.cc:1048] T 976112eb8da2479e9a559399bee97dfc P efe0ab937a564199977b910104824f30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:51.594028  1399 raft_consensus.cc:993] T 976112eb8da2479e9a559399bee97dfc P efe0ab937a564199977b910104824f30: : Instructing follower bbcbed9750ae416bab78287348e9e196 to start an election
I20260709 13:12:51.594511  1441 raft_consensus.cc:1081] T 976112eb8da2479e9a559399bee97dfc P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Signalling peer bbcbed9750ae416bab78287348e9e196 to start an election
I20260709 13:12:51.596228  1196 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "976112eb8da2479e9a559399bee97dfc"
dest_uuid: "bbcbed9750ae416bab78287348e9e196"
 from {username='slave'} at 127.0.0.1:50976
I20260709 13:12:51.596882  1196 raft_consensus.cc:493] T 976112eb8da2479e9a559399bee97dfc P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:51.597275  1196 raft_consensus.cc:3060] T 976112eb8da2479e9a559399bee97dfc P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:51.604385  1196 raft_consensus.cc:515] T 976112eb8da2479e9a559399bee97dfc P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:51.606383  1196 leader_election.cc:290] T 976112eb8da2479e9a559399bee97dfc P bbcbed9750ae416bab78287348e9e196 [CANDIDATE]: Term 2 election: Requested vote from peers 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943), efe0ab937a564199977b910104824f30 (127.21.63.3:33527)
I20260709 13:12:51.607367  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "976112eb8da2479e9a559399bee97dfc" candidate_uuid: "bbcbed9750ae416bab78287348e9e196" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "22ec8c2439d14f139d5330c8ec45133a"
I20260709 13:12:51.607988  1122 raft_consensus.cc:3060] T 976112eb8da2479e9a559399bee97dfc P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:51.609068  1269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "976112eb8da2479e9a559399bee97dfc" candidate_uuid: "bbcbed9750ae416bab78287348e9e196" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efe0ab937a564199977b910104824f30"
I20260709 13:12:51.609860  1269 raft_consensus.cc:3055] T 976112eb8da2479e9a559399bee97dfc P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:51.610198  1269 raft_consensus.cc:740] T 976112eb8da2479e9a559399bee97dfc P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efe0ab937a564199977b910104824f30, State: Running, Role: LEADER
I20260709 13:12:51.611064  1269 consensus_queue.cc:260] T 976112eb8da2479e9a559399bee97dfc P efe0ab937a564199977b910104824f30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:51.612473  1269 raft_consensus.cc:3060] T 976112eb8da2479e9a559399bee97dfc P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:51.615245  1122 raft_consensus.cc:2468] T 976112eb8da2479e9a559399bee97dfc P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbcbed9750ae416bab78287348e9e196 in term 2.
I20260709 13:12:51.616494  1161 leader_election.cc:304] T 976112eb8da2479e9a559399bee97dfc P bbcbed9750ae416bab78287348e9e196 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22ec8c2439d14f139d5330c8ec45133a, bbcbed9750ae416bab78287348e9e196; no voters: 
I20260709 13:12:51.617349  1434 raft_consensus.cc:2804] T 976112eb8da2479e9a559399bee97dfc P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:51.617869  1434 raft_consensus.cc:697] T 976112eb8da2479e9a559399bee97dfc P bbcbed9750ae416bab78287348e9e196 [term 2 LEADER]: Becoming Leader. State: Replica: bbcbed9750ae416bab78287348e9e196, State: Running, Role: LEADER
I20260709 13:12:51.618876  1269 raft_consensus.cc:2468] T 976112eb8da2479e9a559399bee97dfc P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbcbed9750ae416bab78287348e9e196 in term 2.
I20260709 13:12:51.618808  1434 consensus_queue.cc:237] T 976112eb8da2479e9a559399bee97dfc P bbcbed9750ae416bab78287348e9e196 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:51.625993  1019 catalog_manager.cc:5697] T 976112eb8da2479e9a559399bee97dfc P bbcbed9750ae416bab78287348e9e196 reported cstate change: term changed from 1 to 2, leader changed from efe0ab937a564199977b910104824f30 (127.21.63.3) to bbcbed9750ae416bab78287348e9e196 (127.21.63.2). New cstate: current_term: 2 leader_uuid: "bbcbed9750ae416bab78287348e9e196" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:51.673957  1399 raft_consensus.cc:993] T 0be8303fce99487c9cb34506b3181363 P efe0ab937a564199977b910104824f30: : Instructing follower 22ec8c2439d14f139d5330c8ec45133a to start an election
I20260709 13:12:51.674445  1441 raft_consensus.cc:1081] T 0be8303fce99487c9cb34506b3181363 P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Signalling peer 22ec8c2439d14f139d5330c8ec45133a to start an election
I20260709 13:12:51.676147  1122 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0be8303fce99487c9cb34506b3181363"
dest_uuid: "22ec8c2439d14f139d5330c8ec45133a"
 from {username='slave'} at 127.0.0.1:51800
I20260709 13:12:51.676721  1122 raft_consensus.cc:493] T 0be8303fce99487c9cb34506b3181363 P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:51.677073  1122 raft_consensus.cc:3060] T 0be8303fce99487c9cb34506b3181363 P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:51.681532  1122 raft_consensus.cc:515] T 0be8303fce99487c9cb34506b3181363 P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:51.682998  1122 leader_election.cc:290] T 0be8303fce99487c9cb34506b3181363 P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 2 election: Requested vote from peers bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515), efe0ab937a564199977b910104824f30 (127.21.63.3:33527)
I20260709 13:12:51.684047  1269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0be8303fce99487c9cb34506b3181363" candidate_uuid: "22ec8c2439d14f139d5330c8ec45133a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efe0ab937a564199977b910104824f30"
I20260709 13:12:51.684008  1196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0be8303fce99487c9cb34506b3181363" candidate_uuid: "22ec8c2439d14f139d5330c8ec45133a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bbcbed9750ae416bab78287348e9e196"
I20260709 13:12:51.684984  1269 raft_consensus.cc:3055] T 0be8303fce99487c9cb34506b3181363 P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:51.685096  1196 raft_consensus.cc:3060] T 0be8303fce99487c9cb34506b3181363 P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:51.685298  1269 raft_consensus.cc:740] T 0be8303fce99487c9cb34506b3181363 P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efe0ab937a564199977b910104824f30, State: Running, Role: LEADER
I20260709 13:12:51.686316  1269 consensus_queue.cc:260] T 0be8303fce99487c9cb34506b3181363 P efe0ab937a564199977b910104824f30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:51.687352  1269 raft_consensus.cc:3060] T 0be8303fce99487c9cb34506b3181363 P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:51.692449  1269 raft_consensus.cc:2468] T 0be8303fce99487c9cb34506b3181363 P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22ec8c2439d14f139d5330c8ec45133a in term 2.
I20260709 13:12:51.692562  1196 raft_consensus.cc:2468] T 0be8303fce99487c9cb34506b3181363 P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22ec8c2439d14f139d5330c8ec45133a in term 2.
I20260709 13:12:51.693914  1087 leader_election.cc:304] T 0be8303fce99487c9cb34506b3181363 P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22ec8c2439d14f139d5330c8ec45133a, efe0ab937a564199977b910104824f30; no voters: 
I20260709 13:12:51.694674  1321 raft_consensus.cc:2804] T 0be8303fce99487c9cb34506b3181363 P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:51.695178  1321 raft_consensus.cc:697] T 0be8303fce99487c9cb34506b3181363 P 22ec8c2439d14f139d5330c8ec45133a [term 2 LEADER]: Becoming Leader. State: Replica: 22ec8c2439d14f139d5330c8ec45133a, State: Running, Role: LEADER
I20260709 13:12:51.696025  1321 consensus_queue.cc:237] T 0be8303fce99487c9cb34506b3181363 P 22ec8c2439d14f139d5330c8ec45133a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:51.703217  1019 catalog_manager.cc:5697] T 0be8303fce99487c9cb34506b3181363 P 22ec8c2439d14f139d5330c8ec45133a reported cstate change: term changed from 1 to 2, leader changed from efe0ab937a564199977b910104824f30 (127.21.63.3) to 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1). New cstate: current_term: 2 leader_uuid: "22ec8c2439d14f139d5330c8ec45133a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:51.705518  1399 raft_consensus.cc:993] T 249a9a4a171d491b9f64f9b263bbb91a P efe0ab937a564199977b910104824f30: : Instructing follower 22ec8c2439d14f139d5330c8ec45133a to start an election
I20260709 13:12:51.706025  1441 raft_consensus.cc:1081] T 249a9a4a171d491b9f64f9b263bbb91a P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Signalling peer 22ec8c2439d14f139d5330c8ec45133a to start an election
I20260709 13:12:51.707739  1122 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "249a9a4a171d491b9f64f9b263bbb91a"
dest_uuid: "22ec8c2439d14f139d5330c8ec45133a"
 from {username='slave'} at 127.0.0.1:51800
I20260709 13:12:51.708194  1122 raft_consensus.cc:493] T 249a9a4a171d491b9f64f9b263bbb91a P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:51.708527  1122 raft_consensus.cc:3060] T 249a9a4a171d491b9f64f9b263bbb91a P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:51.714021  1122 raft_consensus.cc:515] T 249a9a4a171d491b9f64f9b263bbb91a P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:51.715369  1122 leader_election.cc:290] T 249a9a4a171d491b9f64f9b263bbb91a P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 2 election: Requested vote from peers bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515), efe0ab937a564199977b910104824f30 (127.21.63.3:33527)
I20260709 13:12:51.716315  1196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "249a9a4a171d491b9f64f9b263bbb91a" candidate_uuid: "22ec8c2439d14f139d5330c8ec45133a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bbcbed9750ae416bab78287348e9e196"
I20260709 13:12:51.716634  1269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "249a9a4a171d491b9f64f9b263bbb91a" candidate_uuid: "22ec8c2439d14f139d5330c8ec45133a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efe0ab937a564199977b910104824f30"
I20260709 13:12:51.717154  1196 raft_consensus.cc:3060] T 249a9a4a171d491b9f64f9b263bbb91a P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:51.717571  1269 raft_consensus.cc:3055] T 249a9a4a171d491b9f64f9b263bbb91a P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:51.717972  1269 raft_consensus.cc:740] T 249a9a4a171d491b9f64f9b263bbb91a P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efe0ab937a564199977b910104824f30, State: Running, Role: LEADER
I20260709 13:12:51.718961  1269 consensus_queue.cc:260] T 249a9a4a171d491b9f64f9b263bbb91a P efe0ab937a564199977b910104824f30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:51.720525  1269 raft_consensus.cc:3060] T 249a9a4a171d491b9f64f9b263bbb91a P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:51.725521  1269 raft_consensus.cc:2468] T 249a9a4a171d491b9f64f9b263bbb91a P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22ec8c2439d14f139d5330c8ec45133a in term 2.
I20260709 13:12:51.726104  1196 raft_consensus.cc:2468] T 249a9a4a171d491b9f64f9b263bbb91a P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22ec8c2439d14f139d5330c8ec45133a in term 2.
I20260709 13:12:51.726715  1087 leader_election.cc:304] T 249a9a4a171d491b9f64f9b263bbb91a P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22ec8c2439d14f139d5330c8ec45133a, efe0ab937a564199977b910104824f30; no voters: 
I20260709 13:12:51.727556  1321 raft_consensus.cc:2804] T 249a9a4a171d491b9f64f9b263bbb91a P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:51.728032  1321 raft_consensus.cc:697] T 249a9a4a171d491b9f64f9b263bbb91a P 22ec8c2439d14f139d5330c8ec45133a [term 2 LEADER]: Becoming Leader. State: Replica: 22ec8c2439d14f139d5330c8ec45133a, State: Running, Role: LEADER
I20260709 13:12:51.728885  1321 consensus_queue.cc:237] T 249a9a4a171d491b9f64f9b263bbb91a P 22ec8c2439d14f139d5330c8ec45133a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:51.734984  1019 catalog_manager.cc:5697] T 249a9a4a171d491b9f64f9b263bbb91a P 22ec8c2439d14f139d5330c8ec45133a reported cstate change: term changed from 1 to 2, leader changed from efe0ab937a564199977b910104824f30 (127.21.63.3) to 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1). New cstate: current_term: 2 leader_uuid: "22ec8c2439d14f139d5330c8ec45133a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:51.751884  1321 consensus_queue.cc:1048] T ac170d34df16448ea4c965890a7d22a1 P 22ec8c2439d14f139d5330c8ec45133a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 13:12:51.765164  1441 raft_consensus.cc:993] T 7ae6ea4d73e243a19c55a262c620bb7d P efe0ab937a564199977b910104824f30: : Instructing follower 22ec8c2439d14f139d5330c8ec45133a to start an election
I20260709 13:12:51.765688  1399 raft_consensus.cc:1081] T 7ae6ea4d73e243a19c55a262c620bb7d P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Signalling peer 22ec8c2439d14f139d5330c8ec45133a to start an election
I20260709 13:12:51.767614  1122 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7ae6ea4d73e243a19c55a262c620bb7d"
dest_uuid: "22ec8c2439d14f139d5330c8ec45133a"
 from {username='slave'} at 127.0.0.1:51800
I20260709 13:12:51.768167  1122 raft_consensus.cc:493] T 7ae6ea4d73e243a19c55a262c620bb7d P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:51.768566  1122 raft_consensus.cc:3060] T 7ae6ea4d73e243a19c55a262c620bb7d P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:51.768682  1321 consensus_queue.cc:1048] T ac170d34df16448ea4c965890a7d22a1 P 22ec8c2439d14f139d5330c8ec45133a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 13:12:51.777048  1122 raft_consensus.cc:515] T 7ae6ea4d73e243a19c55a262c620bb7d P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:51.779075  1122 leader_election.cc:290] T 7ae6ea4d73e243a19c55a262c620bb7d P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 2 election: Requested vote from peers bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515), efe0ab937a564199977b910104824f30 (127.21.63.3:33527)
I20260709 13:12:51.780270  1269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae6ea4d73e243a19c55a262c620bb7d" candidate_uuid: "22ec8c2439d14f139d5330c8ec45133a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efe0ab937a564199977b910104824f30"
I20260709 13:12:51.781152  1196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ae6ea4d73e243a19c55a262c620bb7d" candidate_uuid: "22ec8c2439d14f139d5330c8ec45133a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bbcbed9750ae416bab78287348e9e196"
I20260709 13:12:51.781677  1269 raft_consensus.cc:3055] T 7ae6ea4d73e243a19c55a262c620bb7d P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:51.781960  1196 raft_consensus.cc:3060] T 7ae6ea4d73e243a19c55a262c620bb7d P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:51.782274  1269 raft_consensus.cc:740] T 7ae6ea4d73e243a19c55a262c620bb7d P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efe0ab937a564199977b910104824f30, State: Running, Role: LEADER
I20260709 13:12:51.783623  1269 consensus_queue.cc:260] T 7ae6ea4d73e243a19c55a262c620bb7d P efe0ab937a564199977b910104824f30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:51.787395  1269 raft_consensus.cc:3060] T 7ae6ea4d73e243a19c55a262c620bb7d P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:51.787634  1454 raft_consensus.cc:993] T ac170d34df16448ea4c965890a7d22a1 P 22ec8c2439d14f139d5330c8ec45133a: : Instructing follower efe0ab937a564199977b910104824f30 to start an election
I20260709 13:12:51.788219  1454 raft_consensus.cc:1081] T ac170d34df16448ea4c965890a7d22a1 P 22ec8c2439d14f139d5330c8ec45133a [term 1 LEADER]: Signalling peer efe0ab937a564199977b910104824f30 to start an election
I20260709 13:12:51.790199  1196 raft_consensus.cc:2468] T 7ae6ea4d73e243a19c55a262c620bb7d P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22ec8c2439d14f139d5330c8ec45133a in term 2.
I20260709 13:12:51.790295  1272 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ac170d34df16448ea4c965890a7d22a1"
dest_uuid: "efe0ab937a564199977b910104824f30"
 from {username='slave'} at 127.0.0.1:37350
I20260709 13:12:51.791067  1272 raft_consensus.cc:493] T ac170d34df16448ea4c965890a7d22a1 P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:51.791442  1088 leader_election.cc:304] T 7ae6ea4d73e243a19c55a262c620bb7d P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22ec8c2439d14f139d5330c8ec45133a, bbcbed9750ae416bab78287348e9e196; no voters: 
I20260709 13:12:51.791458  1272 raft_consensus.cc:3060] T ac170d34df16448ea4c965890a7d22a1 P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:51.792286  1321 raft_consensus.cc:2804] T 7ae6ea4d73e243a19c55a262c620bb7d P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:51.793599  1321 raft_consensus.cc:697] T 7ae6ea4d73e243a19c55a262c620bb7d P 22ec8c2439d14f139d5330c8ec45133a [term 2 LEADER]: Becoming Leader. State: Replica: 22ec8c2439d14f139d5330c8ec45133a, State: Running, Role: LEADER
I20260709 13:12:51.793845  1269 raft_consensus.cc:2468] T 7ae6ea4d73e243a19c55a262c620bb7d P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22ec8c2439d14f139d5330c8ec45133a in term 2.
I20260709 13:12:51.794406  1321 consensus_queue.cc:237] T 7ae6ea4d73e243a19c55a262c620bb7d P 22ec8c2439d14f139d5330c8ec45133a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:51.799005  1272 raft_consensus.cc:515] T ac170d34df16448ea4c965890a7d22a1 P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:51.801638  1272 leader_election.cc:290] T ac170d34df16448ea4c965890a7d22a1 P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 2 election: Requested vote from peers 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943), bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515)
I20260709 13:12:51.803675  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac170d34df16448ea4c965890a7d22a1" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "22ec8c2439d14f139d5330c8ec45133a"
I20260709 13:12:51.804203  1122 raft_consensus.cc:3055] T ac170d34df16448ea4c965890a7d22a1 P 22ec8c2439d14f139d5330c8ec45133a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:51.803329  1018 catalog_manager.cc:5697] T 7ae6ea4d73e243a19c55a262c620bb7d P 22ec8c2439d14f139d5330c8ec45133a reported cstate change: term changed from 1 to 2, leader changed from efe0ab937a564199977b910104824f30 (127.21.63.3) to 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1). New cstate: current_term: 2 leader_uuid: "22ec8c2439d14f139d5330c8ec45133a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:51.804452  1122 raft_consensus.cc:740] T ac170d34df16448ea4c965890a7d22a1 P 22ec8c2439d14f139d5330c8ec45133a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 22ec8c2439d14f139d5330c8ec45133a, State: Running, Role: LEADER
I20260709 13:12:51.805753  1196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ac170d34df16448ea4c965890a7d22a1" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bbcbed9750ae416bab78287348e9e196"
I20260709 13:12:51.806363  1196 raft_consensus.cc:3060] T ac170d34df16448ea4c965890a7d22a1 P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:51.812067  1196 raft_consensus.cc:2468] T ac170d34df16448ea4c965890a7d22a1 P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe0ab937a564199977b910104824f30 in term 2.
I20260709 13:12:51.812745  1122 consensus_queue.cc:260] T ac170d34df16448ea4c965890a7d22a1 P 22ec8c2439d14f139d5330c8ec45133a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:51.813360  1237 leader_election.cc:304] T ac170d34df16448ea4c965890a7d22a1 P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bbcbed9750ae416bab78287348e9e196, efe0ab937a564199977b910104824f30; no voters: 
I20260709 13:12:51.813969  1122 raft_consensus.cc:3060] T ac170d34df16448ea4c965890a7d22a1 P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:51.814440  1442 raft_consensus.cc:2804] T ac170d34df16448ea4c965890a7d22a1 P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:51.814951  1442 raft_consensus.cc:697] T ac170d34df16448ea4c965890a7d22a1 P efe0ab937a564199977b910104824f30 [term 2 LEADER]: Becoming Leader. State: Replica: efe0ab937a564199977b910104824f30, State: Running, Role: LEADER
I20260709 13:12:51.815809  1442 consensus_queue.cc:237] T ac170d34df16448ea4c965890a7d22a1 P efe0ab937a564199977b910104824f30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:51.820256  1122 raft_consensus.cc:2468] T ac170d34df16448ea4c965890a7d22a1 P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe0ab937a564199977b910104824f30 in term 2.
I20260709 13:12:51.823881  1018 catalog_manager.cc:5697] T ac170d34df16448ea4c965890a7d22a1 P efe0ab937a564199977b910104824f30 reported cstate change: term changed from 1 to 2, leader changed from 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1) to efe0ab937a564199977b910104824f30 (127.21.63.3). New cstate: current_term: 2 leader_uuid: "efe0ab937a564199977b910104824f30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:51.899711  1321 raft_consensus.cc:993] T 5e059664b8d64cc49d217bfcddfc2232 P 22ec8c2439d14f139d5330c8ec45133a: : Instructing follower bbcbed9750ae416bab78287348e9e196 to start an election
I20260709 13:12:51.900269  1454 raft_consensus.cc:1081] T 5e059664b8d64cc49d217bfcddfc2232 P 22ec8c2439d14f139d5330c8ec45133a [term 1 LEADER]: Signalling peer bbcbed9750ae416bab78287348e9e196 to start an election
I20260709 13:12:51.902024  1196 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5e059664b8d64cc49d217bfcddfc2232"
dest_uuid: "bbcbed9750ae416bab78287348e9e196"
 from {username='slave'} at 127.0.0.1:50980
I20260709 13:12:51.902590  1196 raft_consensus.cc:493] T 5e059664b8d64cc49d217bfcddfc2232 P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:51.902887  1196 raft_consensus.cc:3060] T 5e059664b8d64cc49d217bfcddfc2232 P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:51.908651  1196 raft_consensus.cc:515] T 5e059664b8d64cc49d217bfcddfc2232 P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:51.910195  1196 leader_election.cc:290] T 5e059664b8d64cc49d217bfcddfc2232 P bbcbed9750ae416bab78287348e9e196 [CANDIDATE]: Term 2 election: Requested vote from peers 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943), efe0ab937a564199977b910104824f30 (127.21.63.3:33527)
I20260709 13:12:51.911034  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e059664b8d64cc49d217bfcddfc2232" candidate_uuid: "bbcbed9750ae416bab78287348e9e196" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "22ec8c2439d14f139d5330c8ec45133a"
I20260709 13:12:51.911398  1272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5e059664b8d64cc49d217bfcddfc2232" candidate_uuid: "bbcbed9750ae416bab78287348e9e196" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efe0ab937a564199977b910104824f30"
I20260709 13:12:51.911763  1122 raft_consensus.cc:3055] T 5e059664b8d64cc49d217bfcddfc2232 P 22ec8c2439d14f139d5330c8ec45133a [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:51.912058  1272 raft_consensus.cc:3060] T 5e059664b8d64cc49d217bfcddfc2232 P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:51.912156  1122 raft_consensus.cc:740] T 5e059664b8d64cc49d217bfcddfc2232 P 22ec8c2439d14f139d5330c8ec45133a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 22ec8c2439d14f139d5330c8ec45133a, State: Running, Role: LEADER
I20260709 13:12:51.913175  1122 consensus_queue.cc:260] T 5e059664b8d64cc49d217bfcddfc2232 P 22ec8c2439d14f139d5330c8ec45133a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:51.914548  1122 raft_consensus.cc:3060] T 5e059664b8d64cc49d217bfcddfc2232 P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:51.919552  1272 raft_consensus.cc:2468] T 5e059664b8d64cc49d217bfcddfc2232 P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbcbed9750ae416bab78287348e9e196 in term 2.
I20260709 13:12:51.920648  1442 raft_consensus.cc:993] T fe6a0e06494f416e80a6ebbec27396bd P efe0ab937a564199977b910104824f30: : Instructing follower bbcbed9750ae416bab78287348e9e196 to start an election
I20260709 13:12:51.920904  1162 leader_election.cc:304] T 5e059664b8d64cc49d217bfcddfc2232 P bbcbed9750ae416bab78287348e9e196 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bbcbed9750ae416bab78287348e9e196, efe0ab937a564199977b910104824f30; no voters: 
I20260709 13:12:51.921211  1441 raft_consensus.cc:1081] T fe6a0e06494f416e80a6ebbec27396bd P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Signalling peer bbcbed9750ae416bab78287348e9e196 to start an election
I20260709 13:12:51.921273  1122 raft_consensus.cc:2468] T 5e059664b8d64cc49d217bfcddfc2232 P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbcbed9750ae416bab78287348e9e196 in term 2.
I20260709 13:12:51.921976  1478 raft_consensus.cc:2804] T 5e059664b8d64cc49d217bfcddfc2232 P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:51.922465  1478 raft_consensus.cc:697] T 5e059664b8d64cc49d217bfcddfc2232 P bbcbed9750ae416bab78287348e9e196 [term 2 LEADER]: Becoming Leader. State: Replica: bbcbed9750ae416bab78287348e9e196, State: Running, Role: LEADER
I20260709 13:12:51.923171  1196 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fe6a0e06494f416e80a6ebbec27396bd"
dest_uuid: "bbcbed9750ae416bab78287348e9e196"
 from {username='slave'} at 127.0.0.1:50976
I20260709 13:12:51.923794  1196 raft_consensus.cc:493] T fe6a0e06494f416e80a6ebbec27396bd P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:51.923375  1478 consensus_queue.cc:237] T 5e059664b8d64cc49d217bfcddfc2232 P bbcbed9750ae416bab78287348e9e196 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:51.924230  1196 raft_consensus.cc:3060] T fe6a0e06494f416e80a6ebbec27396bd P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:51.930171  1434 raft_consensus.cc:993] T 4bf47f99c081476eabfaeb56515e0d79 P bbcbed9750ae416bab78287348e9e196: : Instructing follower 22ec8c2439d14f139d5330c8ec45133a to start an election
I20260709 13:12:51.930968  1434 raft_consensus.cc:1081] T 4bf47f99c081476eabfaeb56515e0d79 P bbcbed9750ae416bab78287348e9e196 [term 1 LEADER]: Signalling peer 22ec8c2439d14f139d5330c8ec45133a to start an election
I20260709 13:12:51.932380  1196 raft_consensus.cc:515] T fe6a0e06494f416e80a6ebbec27396bd P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:51.932653  1122 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4bf47f99c081476eabfaeb56515e0d79"
dest_uuid: "22ec8c2439d14f139d5330c8ec45133a"
 from {username='slave'} at 127.0.0.1:51804
I20260709 13:12:51.933382  1122 raft_consensus.cc:493] T 4bf47f99c081476eabfaeb56515e0d79 P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:51.932803  1018 catalog_manager.cc:5697] T 5e059664b8d64cc49d217bfcddfc2232 P bbcbed9750ae416bab78287348e9e196 reported cstate change: term changed from 1 to 2, leader changed from 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1) to bbcbed9750ae416bab78287348e9e196 (127.21.63.2). New cstate: current_term: 2 leader_uuid: "bbcbed9750ae416bab78287348e9e196" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:51.933765  1122 raft_consensus.cc:3060] T 4bf47f99c081476eabfaeb56515e0d79 P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:51.934764  1196 leader_election.cc:290] T fe6a0e06494f416e80a6ebbec27396bd P bbcbed9750ae416bab78287348e9e196 [CANDIDATE]: Term 2 election: Requested vote from peers 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943), efe0ab937a564199977b910104824f30 (127.21.63.3:33527)
I20260709 13:12:51.935899  1123 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe6a0e06494f416e80a6ebbec27396bd" candidate_uuid: "bbcbed9750ae416bab78287348e9e196" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "22ec8c2439d14f139d5330c8ec45133a"
I20260709 13:12:51.936496  1123 raft_consensus.cc:3060] T fe6a0e06494f416e80a6ebbec27396bd P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:51.940388  1122 raft_consensus.cc:515] T 4bf47f99c081476eabfaeb56515e0d79 P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:51.942394  1272 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fe6a0e06494f416e80a6ebbec27396bd" candidate_uuid: "bbcbed9750ae416bab78287348e9e196" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efe0ab937a564199977b910104824f30"
I20260709 13:12:51.943004  1272 raft_consensus.cc:3055] T fe6a0e06494f416e80a6ebbec27396bd P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:51.943825  1269 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bf47f99c081476eabfaeb56515e0d79" candidate_uuid: "22ec8c2439d14f139d5330c8ec45133a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "efe0ab937a564199977b910104824f30"
I20260709 13:12:51.944808  1272 raft_consensus.cc:740] T fe6a0e06494f416e80a6ebbec27396bd P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: efe0ab937a564199977b910104824f30, State: Running, Role: LEADER
I20260709 13:12:51.945076  1269 raft_consensus.cc:3060] T 4bf47f99c081476eabfaeb56515e0d79 P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:51.945819  1272 consensus_queue.cc:260] T fe6a0e06494f416e80a6ebbec27396bd P efe0ab937a564199977b910104824f30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:51.947098  1272 raft_consensus.cc:3060] T fe6a0e06494f416e80a6ebbec27396bd P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:51.950301  1196 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4bf47f99c081476eabfaeb56515e0d79" candidate_uuid: "22ec8c2439d14f139d5330c8ec45133a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "bbcbed9750ae416bab78287348e9e196"
I20260709 13:12:51.951840  1269 raft_consensus.cc:2468] T 4bf47f99c081476eabfaeb56515e0d79 P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22ec8c2439d14f139d5330c8ec45133a in term 2.
I20260709 13:12:51.951058  1196 raft_consensus.cc:3055] T 4bf47f99c081476eabfaeb56515e0d79 P bbcbed9750ae416bab78287348e9e196 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 13:12:51.952627  1196 raft_consensus.cc:740] T 4bf47f99c081476eabfaeb56515e0d79 P bbcbed9750ae416bab78287348e9e196 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: bbcbed9750ae416bab78287348e9e196, State: Running, Role: LEADER
I20260709 13:12:51.953070  1087 leader_election.cc:304] T 4bf47f99c081476eabfaeb56515e0d79 P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22ec8c2439d14f139d5330c8ec45133a, efe0ab937a564199977b910104824f30; no voters: 
I20260709 13:12:51.943892  1122 leader_election.cc:290] T 4bf47f99c081476eabfaeb56515e0d79 P 22ec8c2439d14f139d5330c8ec45133a [CANDIDATE]: Term 2 election: Requested vote from peers bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515), efe0ab937a564199977b910104824f30 (127.21.63.3:33527)
I20260709 13:12:51.953768  1196 consensus_queue.cc:260] T 4bf47f99c081476eabfaeb56515e0d79 P bbcbed9750ae416bab78287348e9e196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:51.944280  1123 raft_consensus.cc:2468] T fe6a0e06494f416e80a6ebbec27396bd P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbcbed9750ae416bab78287348e9e196 in term 2.
I20260709 13:12:51.955250  1196 raft_consensus.cc:3060] T 4bf47f99c081476eabfaeb56515e0d79 P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Advancing to term 2
I20260709 13:12:51.955417  1272 raft_consensus.cc:2468] T fe6a0e06494f416e80a6ebbec27396bd P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbcbed9750ae416bab78287348e9e196 in term 2.
I20260709 13:12:51.955945  1454 raft_consensus.cc:2804] T 4bf47f99c081476eabfaeb56515e0d79 P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:51.956550  1454 raft_consensus.cc:697] T 4bf47f99c081476eabfaeb56515e0d79 P 22ec8c2439d14f139d5330c8ec45133a [term 2 LEADER]: Becoming Leader. State: Replica: 22ec8c2439d14f139d5330c8ec45133a, State: Running, Role: LEADER
I20260709 13:12:51.956754  1162 leader_election.cc:304] T fe6a0e06494f416e80a6ebbec27396bd P bbcbed9750ae416bab78287348e9e196 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bbcbed9750ae416bab78287348e9e196, efe0ab937a564199977b910104824f30; no voters: 
I20260709 13:12:51.957752  1478 raft_consensus.cc:2804] T fe6a0e06494f416e80a6ebbec27396bd P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 13:12:51.957604  1454 consensus_queue.cc:237] T 4bf47f99c081476eabfaeb56515e0d79 P 22ec8c2439d14f139d5330c8ec45133a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:51.958257  1478 raft_consensus.cc:697] T fe6a0e06494f416e80a6ebbec27396bd P bbcbed9750ae416bab78287348e9e196 [term 2 LEADER]: Becoming Leader. State: Replica: bbcbed9750ae416bab78287348e9e196, State: Running, Role: LEADER
I20260709 13:12:51.959270  1478 consensus_queue.cc:237] T fe6a0e06494f416e80a6ebbec27396bd P bbcbed9750ae416bab78287348e9e196 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:51.964347  1196 raft_consensus.cc:2468] T 4bf47f99c081476eabfaeb56515e0d79 P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22ec8c2439d14f139d5330c8ec45133a in term 2.
I20260709 13:12:51.967774  1018 catalog_manager.cc:5697] T fe6a0e06494f416e80a6ebbec27396bd P bbcbed9750ae416bab78287348e9e196 reported cstate change: term changed from 1 to 2, leader changed from efe0ab937a564199977b910104824f30 (127.21.63.3) to bbcbed9750ae416bab78287348e9e196 (127.21.63.2). New cstate: current_term: 2 leader_uuid: "bbcbed9750ae416bab78287348e9e196" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:51.968847  1020 catalog_manager.cc:5697] T 4bf47f99c081476eabfaeb56515e0d79 P 22ec8c2439d14f139d5330c8ec45133a reported cstate change: term changed from 1 to 2, leader changed from bbcbed9750ae416bab78287348e9e196 (127.21.63.2) to 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1). New cstate: current_term: 2 leader_uuid: "22ec8c2439d14f139d5330c8ec45133a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:52.000742  1123 raft_consensus.cc:1275] T 976112eb8da2479e9a559399bee97dfc P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Refusing update from remote peer bbcbed9750ae416bab78287348e9e196: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:52.001979  1478 consensus_queue.cc:1048] T 976112eb8da2479e9a559399bee97dfc P bbcbed9750ae416bab78287348e9e196 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:52.012308  1272 raft_consensus.cc:1275] T 976112eb8da2479e9a559399bee97dfc P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Refusing update from remote peer bbcbed9750ae416bab78287348e9e196: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:52.014257  1478 consensus_queue.cc:1048] T 976112eb8da2479e9a559399bee97dfc P bbcbed9750ae416bab78287348e9e196 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:52.190546  1272 raft_consensus.cc:1275] T 0be8303fce99487c9cb34506b3181363 P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Refusing update from remote peer 22ec8c2439d14f139d5330c8ec45133a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:52.191879  1454 consensus_queue.cc:1048] T 0be8303fce99487c9cb34506b3181363 P 22ec8c2439d14f139d5330c8ec45133a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:52.199047  1196 raft_consensus.cc:1275] T 7ae6ea4d73e243a19c55a262c620bb7d P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Refusing update from remote peer 22ec8c2439d14f139d5330c8ec45133a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:52.201283  1321 consensus_queue.cc:1048] T 7ae6ea4d73e243a19c55a262c620bb7d P 22ec8c2439d14f139d5330c8ec45133a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:52.203567  1196 raft_consensus.cc:1275] T 0be8303fce99487c9cb34506b3181363 P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Refusing update from remote peer 22ec8c2439d14f139d5330c8ec45133a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:52.205302  1454 consensus_queue.cc:1048] T 0be8303fce99487c9cb34506b3181363 P 22ec8c2439d14f139d5330c8ec45133a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:52.210872  1269 raft_consensus.cc:1275] T 7ae6ea4d73e243a19c55a262c620bb7d P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Refusing update from remote peer 22ec8c2439d14f139d5330c8ec45133a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:52.212800  1321 consensus_queue.cc:1048] T 7ae6ea4d73e243a19c55a262c620bb7d P 22ec8c2439d14f139d5330c8ec45133a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:52.213094  1197 raft_consensus.cc:1275] T ac170d34df16448ea4c965890a7d22a1 P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Refusing update from remote peer efe0ab937a564199977b910104824f30: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:52.214653  1441 consensus_queue.cc:1048] T ac170d34df16448ea4c965890a7d22a1 P efe0ab937a564199977b910104824f30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:52.225044  1269 raft_consensus.cc:1275] T 249a9a4a171d491b9f64f9b263bbb91a P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Refusing update from remote peer 22ec8c2439d14f139d5330c8ec45133a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:52.227348  1123 raft_consensus.cc:1275] T ac170d34df16448ea4c965890a7d22a1 P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Refusing update from remote peer efe0ab937a564199977b910104824f30: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:52.232162  1450 consensus_queue.cc:1048] T 249a9a4a171d491b9f64f9b263bbb91a P 22ec8c2439d14f139d5330c8ec45133a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:52.235791  1441 consensus_queue.cc:1048] T ac170d34df16448ea4c965890a7d22a1 P efe0ab937a564199977b910104824f30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:52.251257  1197 raft_consensus.cc:1275] T 249a9a4a171d491b9f64f9b263bbb91a P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Refusing update from remote peer 22ec8c2439d14f139d5330c8ec45133a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:52.255300  1488 consensus_queue.cc:1048] T 249a9a4a171d491b9f64f9b263bbb91a P 22ec8c2439d14f139d5330c8ec45133a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 13:12:52.375279  1272 raft_consensus.cc:1275] T 5e059664b8d64cc49d217bfcddfc2232 P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Refusing update from remote peer bbcbed9750ae416bab78287348e9e196: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:52.376737  1440 consensus_queue.cc:1048] T 5e059664b8d64cc49d217bfcddfc2232 P bbcbed9750ae416bab78287348e9e196 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:52.386137  1123 raft_consensus.cc:1275] T 5e059664b8d64cc49d217bfcddfc2232 P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Refusing update from remote peer bbcbed9750ae416bab78287348e9e196: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:52.387385  1434 consensus_queue.cc:1048] T 5e059664b8d64cc49d217bfcddfc2232 P bbcbed9750ae416bab78287348e9e196 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:52.452234  1123 raft_consensus.cc:1275] T fe6a0e06494f416e80a6ebbec27396bd P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Refusing update from remote peer bbcbed9750ae416bab78287348e9e196: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:52.453601  1434 consensus_queue.cc:1048] T fe6a0e06494f416e80a6ebbec27396bd P bbcbed9750ae416bab78287348e9e196 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:52.456632  1272 raft_consensus.cc:1275] T 4bf47f99c081476eabfaeb56515e0d79 P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Refusing update from remote peer 22ec8c2439d14f139d5330c8ec45133a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:52.457873  1488 consensus_queue.cc:1048] T 4bf47f99c081476eabfaeb56515e0d79 P 22ec8c2439d14f139d5330c8ec45133a [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:52.470122  1197 raft_consensus.cc:1275] T 4bf47f99c081476eabfaeb56515e0d79 P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Refusing update from remote peer 22ec8c2439d14f139d5330c8ec45133a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:52.470955  1269 raft_consensus.cc:1275] T fe6a0e06494f416e80a6ebbec27396bd P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Refusing update from remote peer bbcbed9750ae416bab78287348e9e196: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 13:12:52.472718  1434 consensus_queue.cc:1048] T fe6a0e06494f416e80a6ebbec27396bd P bbcbed9750ae416bab78287348e9e196 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:52.473101  1450 consensus_queue.cc:1048] T 4bf47f99c081476eabfaeb56515e0d79 P 22ec8c2439d14f139d5330c8ec45133a [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 13:12:54.446014  1019 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59232:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 13:12:54.464766  1019 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:59232:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 13:12:54.621479 21756 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 13:12:54.655584  1545 tablet.cc:1724] T 0be8303fce99487c9cb34506b3181363 P 22ec8c2439d14f139d5330c8ec45133a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:54.691006  1537 tablet.cc:1724] T 0be8303fce99487c9cb34506b3181363 P efe0ab937a564199977b910104824f30: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:54.692157  1549 tablet.cc:1724] T fe6a0e06494f416e80a6ebbec27396bd P bbcbed9750ae416bab78287348e9e196: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:54.659328  1535 tablet.cc:1724] T 4bf47f99c081476eabfaeb56515e0d79 P 22ec8c2439d14f139d5330c8ec45133a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:54.703037  1547 tablet.cc:1724] T fe6a0e06494f416e80a6ebbec27396bd P 22ec8c2439d14f139d5330c8ec45133a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:54.753182  1548 tablet.cc:1724] T 249a9a4a171d491b9f64f9b263bbb91a P bbcbed9750ae416bab78287348e9e196: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:54.774614  1553 tablet.cc:1724] T 0be8303fce99487c9cb34506b3181363 P bbcbed9750ae416bab78287348e9e196: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:54.732998  1550 tablet.cc:1724] T 249a9a4a171d491b9f64f9b263bbb91a P 22ec8c2439d14f139d5330c8ec45133a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:54.820792  1551 tablet.cc:1724] T 249a9a4a171d491b9f64f9b263bbb91a P efe0ab937a564199977b910104824f30: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:12:54.810915  1161 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:36943, user_credentials={real_user=slave}} blocked reactor thread for 92639.1us
W20260709 13:12:54.921188  1564 log.cc:927] Time spent T 7ae6ea4d73e243a19c55a262c620bb7d P efe0ab937a564199977b910104824f30: Append to log took a long time: real 0.055s	user 0.000s	sys 0.003s
I20260709 13:12:54.908316  1561 tablet.cc:1724] T 4bf47f99c081476eabfaeb56515e0d79 P efe0ab937a564199977b910104824f30: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:54.914764  1548 tablet.cc:1724] T 4bf47f99c081476eabfaeb56515e0d79 P bbcbed9750ae416bab78287348e9e196: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:12:54.944769  1543 log.cc:927] Time spent T 8e4161719e214f1a9836f352271dc985 P bbcbed9750ae416bab78287348e9e196: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 13:12:54.944305  1549 tablet.cc:1724] T 976112eb8da2479e9a559399bee97dfc P bbcbed9750ae416bab78287348e9e196: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:54.872046  1567 tablet.cc:1724] T 5e059664b8d64cc49d217bfcddfc2232 P bbcbed9750ae416bab78287348e9e196: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:54.974186  1120 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "89400744bdfc474a805d19fe866d401b"
dest_uuid: "22ec8c2439d14f139d5330c8ec45133a"
mode: GRACEFUL
new_leader_uuid: "efe0ab937a564199977b910104824f30"
 from {username='slave'} at 127.0.0.1:56390
I20260709 13:12:54.975152  1120 raft_consensus.cc:606] T 89400744bdfc474a805d19fe866d401b P 22ec8c2439d14f139d5330c8ec45133a [term 2 LEADER]: Received request to transfer leadership to TS efe0ab937a564199977b910104824f30
I20260709 13:12:54.981145  1535 tablet.cc:1724] T 5e059664b8d64cc49d217bfcddfc2232 P 22ec8c2439d14f139d5330c8ec45133a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:54.986356  1545 tablet.cc:1724] T 7ae6ea4d73e243a19c55a262c620bb7d P 22ec8c2439d14f139d5330c8ec45133a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:55.021940  1551 tablet.cc:1724] T 976112eb8da2479e9a559399bee97dfc P efe0ab937a564199977b910104824f30: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:55.023921  1573 tablet.cc:1724] T ac170d34df16448ea4c965890a7d22a1 P efe0ab937a564199977b910104824f30: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:55.045498  1537 tablet.cc:1724] T fe6a0e06494f416e80a6ebbec27396bd P efe0ab937a564199977b910104824f30: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:55.077884  1567 tablet.cc:1724] T ac170d34df16448ea4c965890a7d22a1 P bbcbed9750ae416bab78287348e9e196: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:55.089715  1549 tablet.cc:1724] T 7ae6ea4d73e243a19c55a262c620bb7d P bbcbed9750ae416bab78287348e9e196: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 13:12:55.095309  1235 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.21.63.1:36943, user_credentials={real_user=slave}} blocked reactor thread for 93680.1us
I20260709 13:12:55.100081  1573 tablet.cc:1724] T 7ae6ea4d73e243a19c55a262c620bb7d P efe0ab937a564199977b910104824f30: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:55.105270  1561 tablet.cc:1724] T 8e4161719e214f1a9836f352271dc985 P efe0ab937a564199977b910104824f30: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:55.114365  1537 tablet.cc:1724] T 5e059664b8d64cc49d217bfcddfc2232 P efe0ab937a564199977b910104824f30: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:55.117972  1550 tablet.cc:1724] T 976112eb8da2479e9a559399bee97dfc P 22ec8c2439d14f139d5330c8ec45133a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:55.120927  1547 tablet.cc:1724] T ac170d34df16448ea4c965890a7d22a1 P 22ec8c2439d14f139d5330c8ec45133a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:55.129309  1553 tablet.cc:1724] T 8e4161719e214f1a9836f352271dc985 P bbcbed9750ae416bab78287348e9e196: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:55.147424  1545 tablet.cc:1724] T 8e4161719e214f1a9836f352271dc985 P 22ec8c2439d14f139d5330c8ec45133a: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 13:12:55.169433  1197 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "98d5edd9b01941898accf5cf97879ab4"
dest_uuid: "bbcbed9750ae416bab78287348e9e196"
mode: GRACEFUL
new_leader_uuid: "efe0ab937a564199977b910104824f30"
 from {username='slave'} at 127.0.0.1:60806
I20260709 13:12:55.169992  1197 raft_consensus.cc:606] T 98d5edd9b01941898accf5cf97879ab4 P bbcbed9750ae416bab78287348e9e196 [term 2 LEADER]: Received request to transfer leadership to TS efe0ab937a564199977b910104824f30
I20260709 13:12:55.171355 21756 auto_leader_rebalancer.cc:452] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 13:12:55.172000 21756 auto_leader_rebalancer.cc:527] All tables' leader rebalancing finished this round
I20260709 13:12:55.190603  1021 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 31065877c4af477d9b2e726230936891: soft_deleted_table [id=b518086dfff74502b43f9ea7c5d9299a] alter complete (version 1)
I20260709 13:12:55.293805  1536 raft_consensus.cc:993] T 89400744bdfc474a805d19fe866d401b P 22ec8c2439d14f139d5330c8ec45133a: : Instructing follower efe0ab937a564199977b910104824f30 to start an election
I20260709 13:12:55.294286  1536 raft_consensus.cc:1081] T 89400744bdfc474a805d19fe866d401b P 22ec8c2439d14f139d5330c8ec45133a [term 2 LEADER]: Signalling peer efe0ab937a564199977b910104824f30 to start an election
I20260709 13:12:55.295651  1268 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "89400744bdfc474a805d19fe866d401b"
dest_uuid: "efe0ab937a564199977b910104824f30"
 from {username='slave'} at 127.0.0.1:37350
I20260709 13:12:55.296370  1268 raft_consensus.cc:493] T 89400744bdfc474a805d19fe866d401b P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:55.296766  1268 raft_consensus.cc:3060] T 89400744bdfc474a805d19fe866d401b P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:55.303341  1268 raft_consensus.cc:515] T 89400744bdfc474a805d19fe866d401b P efe0ab937a564199977b910104824f30 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } } peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } }
I20260709 13:12:55.305224  1268 leader_election.cc:290] T 89400744bdfc474a805d19fe866d401b P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 3 election: Requested vote from peers bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515), 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943)
I20260709 13:12:55.305754  1197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89400744bdfc474a805d19fe866d401b" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bbcbed9750ae416bab78287348e9e196"
I20260709 13:12:55.306406  1197 raft_consensus.cc:3060] T 89400744bdfc474a805d19fe866d401b P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:55.306263  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "89400744bdfc474a805d19fe866d401b" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "22ec8c2439d14f139d5330c8ec45133a"
I20260709 13:12:55.306962  1122 raft_consensus.cc:3055] T 89400744bdfc474a805d19fe866d401b P 22ec8c2439d14f139d5330c8ec45133a [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:55.307225  1122 raft_consensus.cc:740] T 89400744bdfc474a805d19fe866d401b P 22ec8c2439d14f139d5330c8ec45133a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 22ec8c2439d14f139d5330c8ec45133a, State: Running, Role: LEADER
I20260709 13:12:55.307945  1122 consensus_queue.cc:260] T 89400744bdfc474a805d19fe866d401b P 22ec8c2439d14f139d5330c8ec45133a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } } peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } }
I20260709 13:12:55.308979  1122 raft_consensus.cc:3060] T 89400744bdfc474a805d19fe866d401b P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:55.311112  1197 raft_consensus.cc:2468] T 89400744bdfc474a805d19fe866d401b P bbcbed9750ae416bab78287348e9e196 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe0ab937a564199977b910104824f30 in term 3.
I20260709 13:12:55.312112  1237 leader_election.cc:304] T 89400744bdfc474a805d19fe866d401b P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bbcbed9750ae416bab78287348e9e196, efe0ab937a564199977b910104824f30; no voters: 
I20260709 13:12:55.312881  1500 raft_consensus.cc:2804] T 89400744bdfc474a805d19fe866d401b P efe0ab937a564199977b910104824f30 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:55.313265  1500 raft_consensus.cc:697] T 89400744bdfc474a805d19fe866d401b P efe0ab937a564199977b910104824f30 [term 3 LEADER]: Becoming Leader. State: Replica: efe0ab937a564199977b910104824f30, State: Running, Role: LEADER
I20260709 13:12:55.313848  1122 raft_consensus.cc:2468] T 89400744bdfc474a805d19fe866d401b P 22ec8c2439d14f139d5330c8ec45133a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe0ab937a564199977b910104824f30 in term 3.
I20260709 13:12:55.313982  1500 consensus_queue.cc:237] T 89400744bdfc474a805d19fe866d401b P efe0ab937a564199977b910104824f30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } } peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } }
I20260709 13:12:55.322897  1019 catalog_manager.cc:5697] T 89400744bdfc474a805d19fe866d401b P efe0ab937a564199977b910104824f30 reported cstate change: term changed from 2 to 3, leader changed from 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1) to efe0ab937a564199977b910104824f30 (127.21.63.3). New cstate: current_term: 3 leader_uuid: "efe0ab937a564199977b910104824f30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } health_report { overall_health: UNKNOWN } } }
I20260709 13:12:55.537598  1558 raft_consensus.cc:993] T 98d5edd9b01941898accf5cf97879ab4 P bbcbed9750ae416bab78287348e9e196: : Instructing follower efe0ab937a564199977b910104824f30 to start an election
I20260709 13:12:55.538148  1554 raft_consensus.cc:1081] T 98d5edd9b01941898accf5cf97879ab4 P bbcbed9750ae416bab78287348e9e196 [term 2 LEADER]: Signalling peer efe0ab937a564199977b910104824f30 to start an election
I20260709 13:12:55.539740  1267 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "98d5edd9b01941898accf5cf97879ab4"
dest_uuid: "efe0ab937a564199977b910104824f30"
 from {username='slave'} at 127.0.0.1:37354
I20260709 13:12:55.540246  1267 raft_consensus.cc:493] T 98d5edd9b01941898accf5cf97879ab4 P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 13:12:55.540557  1267 raft_consensus.cc:3060] T 98d5edd9b01941898accf5cf97879ab4 P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:55.546139  1267 raft_consensus.cc:515] T 98d5edd9b01941898accf5cf97879ab4 P efe0ab937a564199977b910104824f30 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:55.547753  1267 leader_election.cc:290] T 98d5edd9b01941898accf5cf97879ab4 P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 3 election: Requested vote from peers 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943), bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515)
I20260709 13:12:55.548504  1122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d5edd9b01941898accf5cf97879ab4" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "22ec8c2439d14f139d5330c8ec45133a"
I20260709 13:12:55.548766  1197 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "98d5edd9b01941898accf5cf97879ab4" candidate_uuid: "efe0ab937a564199977b910104824f30" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "bbcbed9750ae416bab78287348e9e196"
I20260709 13:12:55.549209  1122 raft_consensus.cc:3060] T 98d5edd9b01941898accf5cf97879ab4 P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:55.549297  1197 raft_consensus.cc:3055] T 98d5edd9b01941898accf5cf97879ab4 P bbcbed9750ae416bab78287348e9e196 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 13:12:55.549669  1197 raft_consensus.cc:740] T 98d5edd9b01941898accf5cf97879ab4 P bbcbed9750ae416bab78287348e9e196 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: bbcbed9750ae416bab78287348e9e196, State: Running, Role: LEADER
I20260709 13:12:55.550432  1197 consensus_queue.cc:260] T 98d5edd9b01941898accf5cf97879ab4 P bbcbed9750ae416bab78287348e9e196 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:55.551739  1197 raft_consensus.cc:3060] T 98d5edd9b01941898accf5cf97879ab4 P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Advancing to term 3
I20260709 13:12:55.556052  1122 raft_consensus.cc:2468] T 98d5edd9b01941898accf5cf97879ab4 P 22ec8c2439d14f139d5330c8ec45133a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe0ab937a564199977b910104824f30 in term 3.
I20260709 13:12:55.557191  1235 leader_election.cc:304] T 98d5edd9b01941898accf5cf97879ab4 P efe0ab937a564199977b910104824f30 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22ec8c2439d14f139d5330c8ec45133a, efe0ab937a564199977b910104824f30; no voters: 
I20260709 13:12:55.558040  1500 raft_consensus.cc:2804] T 98d5edd9b01941898accf5cf97879ab4 P efe0ab937a564199977b910104824f30 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 13:12:55.558212  1197 raft_consensus.cc:2468] T 98d5edd9b01941898accf5cf97879ab4 P bbcbed9750ae416bab78287348e9e196 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efe0ab937a564199977b910104824f30 in term 3.
I20260709 13:12:55.558512  1500 raft_consensus.cc:697] T 98d5edd9b01941898accf5cf97879ab4 P efe0ab937a564199977b910104824f30 [term 3 LEADER]: Becoming Leader. State: Replica: efe0ab937a564199977b910104824f30, State: Running, Role: LEADER
I20260709 13:12:55.559465  1500 consensus_queue.cc:237] T 98d5edd9b01941898accf5cf97879ab4 P efe0ab937a564199977b910104824f30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } }
I20260709 13:12:55.566732  1019 catalog_manager.cc:5697] T 98d5edd9b01941898accf5cf97879ab4 P efe0ab937a564199977b910104824f30 reported cstate change: term changed from 2 to 3, leader changed from bbcbed9750ae416bab78287348e9e196 (127.21.63.2) to efe0ab937a564199977b910104824f30 (127.21.63.3). New cstate: current_term: 3 leader_uuid: "efe0ab937a564199977b910104824f30" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efe0ab937a564199977b910104824f30" member_type: VOTER last_known_addr { host: "127.21.63.3" port: 33527 } health_report { overall_health: HEALTHY } } }
I20260709 13:12:55.743336  1197 raft_consensus.cc:1275] T 89400744bdfc474a805d19fe866d401b P bbcbed9750ae416bab78287348e9e196 [term 3 FOLLOWER]: Refusing update from remote peer efe0ab937a564199977b910104824f30: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:55.744627  1500 consensus_queue.cc:1048] T 89400744bdfc474a805d19fe866d401b P efe0ab937a564199977b910104824f30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:55.757261  1122 raft_consensus.cc:1275] T 89400744bdfc474a805d19fe866d401b P 22ec8c2439d14f139d5330c8ec45133a [term 3 FOLLOWER]: Refusing update from remote peer efe0ab937a564199977b910104824f30: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:55.758369  1500 consensus_queue.cc:1048] T 89400744bdfc474a805d19fe866d401b P efe0ab937a564199977b910104824f30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:56.158727  1197 raft_consensus.cc:1275] T 98d5edd9b01941898accf5cf97879ab4 P bbcbed9750ae416bab78287348e9e196 [term 3 FOLLOWER]: Refusing update from remote peer efe0ab937a564199977b910104824f30: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:56.160187  1500 consensus_queue.cc:1048] T 98d5edd9b01941898accf5cf97879ab4 P efe0ab937a564199977b910104824f30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbcbed9750ae416bab78287348e9e196" member_type: VOTER last_known_addr { host: "127.21.63.2" port: 34515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:56.171103  1122 raft_consensus.cc:1275] T 98d5edd9b01941898accf5cf97879ab4 P 22ec8c2439d14f139d5330c8ec45133a [term 3 FOLLOWER]: Refusing update from remote peer efe0ab937a564199977b910104824f30: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 13:12:56.172904  1588 consensus_queue.cc:1048] T 98d5edd9b01941898accf5cf97879ab4 P efe0ab937a564199977b910104824f30 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22ec8c2439d14f139d5330c8ec45133a" member_type: VOTER last_known_addr { host: "127.21.63.1" port: 36943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 13:12:56.237795 21756 tablet_server.cc:179] TabletServer@127.21.63.1:0 shutting down...
W20260709 13:12:56.261250  1235 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 10 similar messages]
W20260709 13:12:56.264581  1235 consensus_peers.cc:597] T 89400744bdfc474a805d19fe866d401b P efe0ab937a564199977b910104824f30 -> Peer 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943): Couldn't send request to peer 22ec8c2439d14f139d5330c8ec45133a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:56.275894 21756 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:12:56.277087 21756 tablet_replica.cc:333] T ac170d34df16448ea4c965890a7d22a1 P 22ec8c2439d14f139d5330c8ec45133a: stopping tablet replica
I20260709 13:12:56.278174 21756 raft_consensus.cc:2243] T ac170d34df16448ea4c965890a7d22a1 P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:56.278782 21756 raft_consensus.cc:2272] T ac170d34df16448ea4c965890a7d22a1 P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.282162 21756 tablet_replica.cc:333] T fe6a0e06494f416e80a6ebbec27396bd P 22ec8c2439d14f139d5330c8ec45133a: stopping tablet replica
I20260709 13:12:56.283162 21756 raft_consensus.cc:2243] T fe6a0e06494f416e80a6ebbec27396bd P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:56.283661 21756 raft_consensus.cc:2272] T fe6a0e06494f416e80a6ebbec27396bd P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:56.286021  1235 consensus_peers.cc:597] T af54d168945b40d7b514668ce80da735 P efe0ab937a564199977b910104824f30 -> Peer 22ec8c2439d14f139d5330c8ec45133a (127.21.63.1:36943): Couldn't send request to peer 22ec8c2439d14f139d5330c8ec45133a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:56.286164 21756 tablet_replica.cc:333] T 976112eb8da2479e9a559399bee97dfc P 22ec8c2439d14f139d5330c8ec45133a: stopping tablet replica
I20260709 13:12:56.286995 21756 raft_consensus.cc:2243] T 976112eb8da2479e9a559399bee97dfc P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:56.287463 21756 raft_consensus.cc:2272] T 976112eb8da2479e9a559399bee97dfc P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.289628 21756 tablet_replica.cc:333] T 8e4161719e214f1a9836f352271dc985 P 22ec8c2439d14f139d5330c8ec45133a: stopping tablet replica
I20260709 13:12:56.290238 21756 raft_consensus.cc:2243] T 8e4161719e214f1a9836f352271dc985 P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:56.290704 21756 raft_consensus.cc:2272] T 8e4161719e214f1a9836f352271dc985 P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.293058 21756 tablet_replica.cc:333] T 249a9a4a171d491b9f64f9b263bbb91a P 22ec8c2439d14f139d5330c8ec45133a: stopping tablet replica
I20260709 13:12:56.293694 21756 raft_consensus.cc:2243] T 249a9a4a171d491b9f64f9b263bbb91a P 22ec8c2439d14f139d5330c8ec45133a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:56.294458 21756 raft_consensus.cc:2272] T 249a9a4a171d491b9f64f9b263bbb91a P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.296564 21756 tablet_replica.cc:333] T 7ae6ea4d73e243a19c55a262c620bb7d P 22ec8c2439d14f139d5330c8ec45133a: stopping tablet replica
I20260709 13:12:56.297257 21756 raft_consensus.cc:2243] T 7ae6ea4d73e243a19c55a262c620bb7d P 22ec8c2439d14f139d5330c8ec45133a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:56.298038 21756 raft_consensus.cc:2272] T 7ae6ea4d73e243a19c55a262c620bb7d P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.300128 21756 tablet_replica.cc:333] T fd271e857f3f41c6bcc5d54aae05f985 P 22ec8c2439d14f139d5330c8ec45133a: stopping tablet replica
I20260709 13:12:56.300786 21756 raft_consensus.cc:2243] T fd271e857f3f41c6bcc5d54aae05f985 P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:56.301246 21756 raft_consensus.cc:2272] T fd271e857f3f41c6bcc5d54aae05f985 P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.303432 21756 tablet_replica.cc:333] T 4bf47f99c081476eabfaeb56515e0d79 P 22ec8c2439d14f139d5330c8ec45133a: stopping tablet replica
I20260709 13:12:56.304108 21756 raft_consensus.cc:2243] T 4bf47f99c081476eabfaeb56515e0d79 P 22ec8c2439d14f139d5330c8ec45133a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:56.304910 21756 raft_consensus.cc:2272] T 4bf47f99c081476eabfaeb56515e0d79 P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.307204 21756 tablet_replica.cc:333] T 250a7a602e924c57b99ecbd2bf900bcc P 22ec8c2439d14f139d5330c8ec45133a: stopping tablet replica
I20260709 13:12:56.307811 21756 raft_consensus.cc:2243] T 250a7a602e924c57b99ecbd2bf900bcc P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:56.308512 21756 raft_consensus.cc:2272] T 250a7a602e924c57b99ecbd2bf900bcc P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.310561 21756 tablet_replica.cc:333] T 9d35b87a150449d5b2adefeb4b241fd2 P 22ec8c2439d14f139d5330c8ec45133a: stopping tablet replica
I20260709 13:12:56.311116 21756 raft_consensus.cc:2243] T 9d35b87a150449d5b2adefeb4b241fd2 P 22ec8c2439d14f139d5330c8ec45133a [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:56.311895 21756 raft_consensus.cc:2272] T 9d35b87a150449d5b2adefeb4b241fd2 P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.314392 21756 tablet_replica.cc:333] T 57aa843159154feb87f476bac96b2dba P 22ec8c2439d14f139d5330c8ec45133a: stopping tablet replica
I20260709 13:12:56.314972 21756 raft_consensus.cc:2243] T 57aa843159154feb87f476bac96b2dba P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:56.315452 21756 raft_consensus.cc:2272] T 57aa843159154feb87f476bac96b2dba P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.317937 21756 tablet_replica.cc:333] T 0be8303fce99487c9cb34506b3181363 P 22ec8c2439d14f139d5330c8ec45133a: stopping tablet replica
I20260709 13:12:56.318531 21756 raft_consensus.cc:2243] T 0be8303fce99487c9cb34506b3181363 P 22ec8c2439d14f139d5330c8ec45133a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:56.319311 21756 raft_consensus.cc:2272] T 0be8303fce99487c9cb34506b3181363 P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.321293 21756 tablet_replica.cc:333] T af54d168945b40d7b514668ce80da735 P 22ec8c2439d14f139d5330c8ec45133a: stopping tablet replica
I20260709 13:12:56.321770 21756 raft_consensus.cc:2243] T af54d168945b40d7b514668ce80da735 P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:56.322212 21756 raft_consensus.cc:2272] T af54d168945b40d7b514668ce80da735 P 22ec8c2439d14f139d5330c8ec45133a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.325337 21756 tablet_replica.cc:333] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P 22ec8c2439d14f139d5330c8ec45133a: stopping tablet replica
I20260709 13:12:56.325903 21756 raft_consensus.cc:2243] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P 22ec8c2439d14f139d5330c8ec45133a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:56.326773 21756 raft_consensus.cc:2272] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.329257 21756 tablet_replica.cc:333] T 8de23e2629b2451b900a09bfe12dbf86 P 22ec8c2439d14f139d5330c8ec45133a: stopping tablet replica
I20260709 13:12:56.329830 21756 raft_consensus.cc:2243] T 8de23e2629b2451b900a09bfe12dbf86 P 22ec8c2439d14f139d5330c8ec45133a [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:56.330680 21756 raft_consensus.cc:2272] T 8de23e2629b2451b900a09bfe12dbf86 P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.332859 21756 tablet_replica.cc:333] T 89400744bdfc474a805d19fe866d401b P 22ec8c2439d14f139d5330c8ec45133a: stopping tablet replica
I20260709 13:12:56.333446 21756 raft_consensus.cc:2243] T 89400744bdfc474a805d19fe866d401b P 22ec8c2439d14f139d5330c8ec45133a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:56.333894 21756 raft_consensus.cc:2272] T 89400744bdfc474a805d19fe866d401b P 22ec8c2439d14f139d5330c8ec45133a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.336519 21756 tablet_replica.cc:333] T 5e059664b8d64cc49d217bfcddfc2232 P 22ec8c2439d14f139d5330c8ec45133a: stopping tablet replica
I20260709 13:12:56.337127 21756 raft_consensus.cc:2243] T 5e059664b8d64cc49d217bfcddfc2232 P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:56.337618 21756 raft_consensus.cc:2272] T 5e059664b8d64cc49d217bfcddfc2232 P 22ec8c2439d14f139d5330c8ec45133a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.339733 21756 tablet_replica.cc:333] T 98d5edd9b01941898accf5cf97879ab4 P 22ec8c2439d14f139d5330c8ec45133a: stopping tablet replica
I20260709 13:12:56.340301 21756 raft_consensus.cc:2243] T 98d5edd9b01941898accf5cf97879ab4 P 22ec8c2439d14f139d5330c8ec45133a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:56.340821 21756 raft_consensus.cc:2272] T 98d5edd9b01941898accf5cf97879ab4 P 22ec8c2439d14f139d5330c8ec45133a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.384482 21756 tablet_server.cc:196] TabletServer@127.21.63.1:0 shutdown complete.
I20260709 13:12:56.419332 21756 tablet_server.cc:179] TabletServer@127.21.63.2:0 shutting down...
I20260709 13:12:56.447655 21756 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:12:56.448374 21756 tablet_replica.cc:333] T ac170d34df16448ea4c965890a7d22a1 P bbcbed9750ae416bab78287348e9e196: stopping tablet replica
I20260709 13:12:56.449224 21756 raft_consensus.cc:2243] T ac170d34df16448ea4c965890a7d22a1 P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:56.449810 21756 raft_consensus.cc:2272] T ac170d34df16448ea4c965890a7d22a1 P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.451804 21756 tablet_replica.cc:333] T 976112eb8da2479e9a559399bee97dfc P bbcbed9750ae416bab78287348e9e196: stopping tablet replica
I20260709 13:12:56.452443 21756 raft_consensus.cc:2243] T 976112eb8da2479e9a559399bee97dfc P bbcbed9750ae416bab78287348e9e196 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:56.453218 21756 raft_consensus.cc:2272] T 976112eb8da2479e9a559399bee97dfc P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.455183 21756 tablet_replica.cc:333] T fe6a0e06494f416e80a6ebbec27396bd P bbcbed9750ae416bab78287348e9e196: stopping tablet replica
I20260709 13:12:56.455749 21756 raft_consensus.cc:2243] T fe6a0e06494f416e80a6ebbec27396bd P bbcbed9750ae416bab78287348e9e196 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:56.456539 21756 raft_consensus.cc:2272] T fe6a0e06494f416e80a6ebbec27396bd P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.458482 21756 tablet_replica.cc:333] T 8e4161719e214f1a9836f352271dc985 P bbcbed9750ae416bab78287348e9e196: stopping tablet replica
I20260709 13:12:56.458972 21756 raft_consensus.cc:2243] T 8e4161719e214f1a9836f352271dc985 P bbcbed9750ae416bab78287348e9e196 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:56.459719 21756 raft_consensus.cc:2272] T 8e4161719e214f1a9836f352271dc985 P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.461768 21756 tablet_replica.cc:333] T 249a9a4a171d491b9f64f9b263bbb91a P bbcbed9750ae416bab78287348e9e196: stopping tablet replica
I20260709 13:12:56.462352 21756 raft_consensus.cc:2243] T 249a9a4a171d491b9f64f9b263bbb91a P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:56.462720 21756 raft_consensus.cc:2272] T 249a9a4a171d491b9f64f9b263bbb91a P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.464483 21756 tablet_replica.cc:333] T 7ae6ea4d73e243a19c55a262c620bb7d P bbcbed9750ae416bab78287348e9e196: stopping tablet replica
I20260709 13:12:56.465009 21756 raft_consensus.cc:2243] T 7ae6ea4d73e243a19c55a262c620bb7d P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:56.465399 21756 raft_consensus.cc:2272] T 7ae6ea4d73e243a19c55a262c620bb7d P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.467217 21756 tablet_replica.cc:333] T 0be8303fce99487c9cb34506b3181363 P bbcbed9750ae416bab78287348e9e196: stopping tablet replica
I20260709 13:12:56.467716 21756 raft_consensus.cc:2243] T 0be8303fce99487c9cb34506b3181363 P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:56.468081 21756 raft_consensus.cc:2272] T 0be8303fce99487c9cb34506b3181363 P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.470052 21756 tablet_replica.cc:333] T af54d168945b40d7b514668ce80da735 P bbcbed9750ae416bab78287348e9e196: stopping tablet replica
I20260709 13:12:56.470515 21756 raft_consensus.cc:2243] T af54d168945b40d7b514668ce80da735 P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:56.470897 21756 raft_consensus.cc:2272] T af54d168945b40d7b514668ce80da735 P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.472857 21756 tablet_replica.cc:333] T 9d35b87a150449d5b2adefeb4b241fd2 P bbcbed9750ae416bab78287348e9e196: stopping tablet replica
I20260709 13:12:56.473326 21756 raft_consensus.cc:2243] T 9d35b87a150449d5b2adefeb4b241fd2 P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:56.473687 21756 raft_consensus.cc:2272] T 9d35b87a150449d5b2adefeb4b241fd2 P bbcbed9750ae416bab78287348e9e196 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.475577 21756 tablet_replica.cc:333] T 57aa843159154feb87f476bac96b2dba P bbcbed9750ae416bab78287348e9e196: stopping tablet replica
I20260709 13:12:56.476038 21756 raft_consensus.cc:2243] T 57aa843159154feb87f476bac96b2dba P bbcbed9750ae416bab78287348e9e196 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:56.476769 21756 raft_consensus.cc:2272] T 57aa843159154feb87f476bac96b2dba P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.478626 21756 tablet_replica.cc:333] T fd271e857f3f41c6bcc5d54aae05f985 P bbcbed9750ae416bab78287348e9e196: stopping tablet replica
I20260709 13:12:56.479094 21756 raft_consensus.cc:2243] T fd271e857f3f41c6bcc5d54aae05f985 P bbcbed9750ae416bab78287348e9e196 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:56.479794 21756 raft_consensus.cc:2272] T fd271e857f3f41c6bcc5d54aae05f985 P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.481917 21756 tablet_replica.cc:333] T 250a7a602e924c57b99ecbd2bf900bcc P bbcbed9750ae416bab78287348e9e196: stopping tablet replica
I20260709 13:12:56.482622 21756 raft_consensus.cc:2243] T 250a7a602e924c57b99ecbd2bf900bcc P bbcbed9750ae416bab78287348e9e196 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:56.483698 21756 raft_consensus.cc:2272] T 250a7a602e924c57b99ecbd2bf900bcc P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.486330 21756 tablet_replica.cc:333] T 4bf47f99c081476eabfaeb56515e0d79 P bbcbed9750ae416bab78287348e9e196: stopping tablet replica
W20260709 13:12:56.486384  1237 consensus_peers.cc:597] T af54d168945b40d7b514668ce80da735 P efe0ab937a564199977b910104824f30 -> Peer bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515): Couldn't send request to peer bbcbed9750ae416bab78287348e9e196. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:56.486907 21756 raft_consensus.cc:2243] T 4bf47f99c081476eabfaeb56515e0d79 P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:56.487391 21756 raft_consensus.cc:2272] T 4bf47f99c081476eabfaeb56515e0d79 P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.489403 21756 tablet_replica.cc:333] T 5e059664b8d64cc49d217bfcddfc2232 P bbcbed9750ae416bab78287348e9e196: stopping tablet replica
I20260709 13:12:56.489929 21756 raft_consensus.cc:2243] T 5e059664b8d64cc49d217bfcddfc2232 P bbcbed9750ae416bab78287348e9e196 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:56.490628 21756 raft_consensus.cc:2272] T 5e059664b8d64cc49d217bfcddfc2232 P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.492403 21756 tablet_replica.cc:333] T 98d5edd9b01941898accf5cf97879ab4 P bbcbed9750ae416bab78287348e9e196: stopping tablet replica
I20260709 13:12:56.492954 21756 raft_consensus.cc:2243] T 98d5edd9b01941898accf5cf97879ab4 P bbcbed9750ae416bab78287348e9e196 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:56.493340 21756 raft_consensus.cc:2272] T 98d5edd9b01941898accf5cf97879ab4 P bbcbed9750ae416bab78287348e9e196 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.495649 21756 tablet_replica.cc:333] T 89400744bdfc474a805d19fe866d401b P bbcbed9750ae416bab78287348e9e196: stopping tablet replica
I20260709 13:12:56.496186 21756 raft_consensus.cc:2243] T 89400744bdfc474a805d19fe866d401b P bbcbed9750ae416bab78287348e9e196 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:56.496642 21756 raft_consensus.cc:2272] T 89400744bdfc474a805d19fe866d401b P bbcbed9750ae416bab78287348e9e196 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.499131 21756 tablet_replica.cc:333] T 8de23e2629b2451b900a09bfe12dbf86 P bbcbed9750ae416bab78287348e9e196: stopping tablet replica
I20260709 13:12:56.499632 21756 raft_consensus.cc:2243] T 8de23e2629b2451b900a09bfe12dbf86 P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:56.500065 21756 raft_consensus.cc:2272] T 8de23e2629b2451b900a09bfe12dbf86 P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.502053 21756 tablet_replica.cc:333] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P bbcbed9750ae416bab78287348e9e196: stopping tablet replica
I20260709 13:12:56.502628 21756 raft_consensus.cc:2243] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:56.503178 21756 raft_consensus.cc:2272] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P bbcbed9750ae416bab78287348e9e196 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:56.506580  1237 consensus_peers.cc:597] T ac170d34df16448ea4c965890a7d22a1 P efe0ab937a564199977b910104824f30 -> Peer bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515): Couldn't send request to peer bbcbed9750ae416bab78287348e9e196. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:56.532490 21756 tablet_server.cc:196] TabletServer@127.21.63.2:0 shutdown complete.
I20260709 13:12:56.565781 21756 tablet_server.cc:179] TabletServer@127.21.63.3:0 shutting down...
I20260709 13:12:56.594050 21756 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 13:12:56.594807 21756 tablet_replica.cc:333] T ac170d34df16448ea4c965890a7d22a1 P efe0ab937a564199977b910104824f30: stopping tablet replica
I20260709 13:12:56.595516 21756 raft_consensus.cc:2243] T ac170d34df16448ea4c965890a7d22a1 P efe0ab937a564199977b910104824f30 [term 2 LEADER]: Raft consensus shutting down.
I20260709 13:12:56.596324 21756 raft_consensus.cc:2272] T ac170d34df16448ea4c965890a7d22a1 P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.598335 21756 tablet_replica.cc:333] T fe6a0e06494f416e80a6ebbec27396bd P efe0ab937a564199977b910104824f30: stopping tablet replica
I20260709 13:12:56.598897 21756 raft_consensus.cc:2243] T fe6a0e06494f416e80a6ebbec27396bd P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:56.599325 21756 raft_consensus.cc:2272] T fe6a0e06494f416e80a6ebbec27396bd P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.601169 21756 tablet_replica.cc:333] T 976112eb8da2479e9a559399bee97dfc P efe0ab937a564199977b910104824f30: stopping tablet replica
I20260709 13:12:56.601696 21756 raft_consensus.cc:2243] T 976112eb8da2479e9a559399bee97dfc P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:56.602110 21756 raft_consensus.cc:2272] T 976112eb8da2479e9a559399bee97dfc P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.604053 21756 tablet_replica.cc:333] T 8e4161719e214f1a9836f352271dc985 P efe0ab937a564199977b910104824f30: stopping tablet replica
I20260709 13:12:56.604801 21756 raft_consensus.cc:2243] T 8e4161719e214f1a9836f352271dc985 P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:56.605408 21756 raft_consensus.cc:2272] T 8e4161719e214f1a9836f352271dc985 P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.607765 21756 tablet_replica.cc:333] T 249a9a4a171d491b9f64f9b263bbb91a P efe0ab937a564199977b910104824f30: stopping tablet replica
I20260709 13:12:56.608486 21756 raft_consensus.cc:2243] T 249a9a4a171d491b9f64f9b263bbb91a P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:56.608978 21756 raft_consensus.cc:2272] T 249a9a4a171d491b9f64f9b263bbb91a P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 13:12:56.609010  1237 consensus_peers.cc:597] T 98d5edd9b01941898accf5cf97879ab4 P efe0ab937a564199977b910104824f30 -> Peer bbcbed9750ae416bab78287348e9e196 (127.21.63.2:34515): Couldn't send request to peer bbcbed9750ae416bab78287348e9e196. Status: Network error: Client connection negotiation failed: client connection to 127.21.63.2:34515: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 13:12:56.610968 21756 tablet_replica.cc:333] T 7ae6ea4d73e243a19c55a262c620bb7d P efe0ab937a564199977b910104824f30: stopping tablet replica
I20260709 13:12:56.611562 21756 raft_consensus.cc:2243] T 7ae6ea4d73e243a19c55a262c620bb7d P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:56.611982 21756 raft_consensus.cc:2272] T 7ae6ea4d73e243a19c55a262c620bb7d P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.613878 21756 tablet_replica.cc:333] T 9d35b87a150449d5b2adefeb4b241fd2 P efe0ab937a564199977b910104824f30: stopping tablet replica
I20260709 13:12:56.614398 21756 raft_consensus.cc:2243] T 9d35b87a150449d5b2adefeb4b241fd2 P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:56.614795 21756 raft_consensus.cc:2272] T 9d35b87a150449d5b2adefeb4b241fd2 P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.616788 21756 tablet_replica.cc:333] T 57aa843159154feb87f476bac96b2dba P efe0ab937a564199977b910104824f30: stopping tablet replica
I20260709 13:12:56.617372 21756 raft_consensus.cc:2243] T 57aa843159154feb87f476bac96b2dba P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:56.617771 21756 raft_consensus.cc:2272] T 57aa843159154feb87f476bac96b2dba P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.619745 21756 tablet_replica.cc:333] T fd271e857f3f41c6bcc5d54aae05f985 P efe0ab937a564199977b910104824f30: stopping tablet replica
I20260709 13:12:56.620296 21756 raft_consensus.cc:2243] T fd271e857f3f41c6bcc5d54aae05f985 P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:56.620754 21756 raft_consensus.cc:2272] T fd271e857f3f41c6bcc5d54aae05f985 P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.622618 21756 tablet_replica.cc:333] T 250a7a602e924c57b99ecbd2bf900bcc P efe0ab937a564199977b910104824f30: stopping tablet replica
I20260709 13:12:56.623131 21756 raft_consensus.cc:2243] T 250a7a602e924c57b99ecbd2bf900bcc P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:56.623549 21756 raft_consensus.cc:2272] T 250a7a602e924c57b99ecbd2bf900bcc P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.625486 21756 tablet_replica.cc:333] T 4bf47f99c081476eabfaeb56515e0d79 P efe0ab937a564199977b910104824f30: stopping tablet replica
I20260709 13:12:56.625990 21756 raft_consensus.cc:2243] T 4bf47f99c081476eabfaeb56515e0d79 P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:56.626389 21756 raft_consensus.cc:2272] T 4bf47f99c081476eabfaeb56515e0d79 P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.628167 21756 tablet_replica.cc:333] T 89400744bdfc474a805d19fe866d401b P efe0ab937a564199977b910104824f30: stopping tablet replica
I20260709 13:12:56.628639 21756 raft_consensus.cc:2243] T 89400744bdfc474a805d19fe866d401b P efe0ab937a564199977b910104824f30 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:12:56.629427 21756 raft_consensus.cc:2272] T 89400744bdfc474a805d19fe866d401b P efe0ab937a564199977b910104824f30 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.631582 21756 tablet_replica.cc:333] T 5e059664b8d64cc49d217bfcddfc2232 P efe0ab937a564199977b910104824f30: stopping tablet replica
I20260709 13:12:56.632102 21756 raft_consensus.cc:2243] T 5e059664b8d64cc49d217bfcddfc2232 P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:56.632525 21756 raft_consensus.cc:2272] T 5e059664b8d64cc49d217bfcddfc2232 P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.634404 21756 tablet_replica.cc:333] T 98d5edd9b01941898accf5cf97879ab4 P efe0ab937a564199977b910104824f30: stopping tablet replica
I20260709 13:12:56.634897 21756 raft_consensus.cc:2243] T 98d5edd9b01941898accf5cf97879ab4 P efe0ab937a564199977b910104824f30 [term 3 LEADER]: Raft consensus shutting down.
I20260709 13:12:56.635660 21756 raft_consensus.cc:2272] T 98d5edd9b01941898accf5cf97879ab4 P efe0ab937a564199977b910104824f30 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.638024 21756 tablet_replica.cc:333] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P efe0ab937a564199977b910104824f30: stopping tablet replica
I20260709 13:12:56.638523 21756 raft_consensus.cc:2243] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:56.638968 21756 raft_consensus.cc:2272] T 8ad7ce1d2ca94ba2bc0a82302a9eee4e P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.640854 21756 tablet_replica.cc:333] T 8de23e2629b2451b900a09bfe12dbf86 P efe0ab937a564199977b910104824f30: stopping tablet replica
I20260709 13:12:56.641367 21756 raft_consensus.cc:2243] T 8de23e2629b2451b900a09bfe12dbf86 P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:56.641779 21756 raft_consensus.cc:2272] T 8de23e2629b2451b900a09bfe12dbf86 P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.643671 21756 tablet_replica.cc:333] T 0be8303fce99487c9cb34506b3181363 P efe0ab937a564199977b910104824f30: stopping tablet replica
I20260709 13:12:56.644157 21756 raft_consensus.cc:2243] T 0be8303fce99487c9cb34506b3181363 P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 13:12:56.644568 21756 raft_consensus.cc:2272] T 0be8303fce99487c9cb34506b3181363 P efe0ab937a564199977b910104824f30 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.646394 21756 tablet_replica.cc:333] T af54d168945b40d7b514668ce80da735 P efe0ab937a564199977b910104824f30: stopping tablet replica
I20260709 13:12:56.646853 21756 raft_consensus.cc:2243] T af54d168945b40d7b514668ce80da735 P efe0ab937a564199977b910104824f30 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:56.647630 21756 raft_consensus.cc:2272] T af54d168945b40d7b514668ce80da735 P efe0ab937a564199977b910104824f30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.677325 21756 tablet_server.cc:196] TabletServer@127.21.63.3:0 shutdown complete.
I20260709 13:12:56.709596 21756 master.cc:562] Master@127.21.63.62:41855 shutting down...
I20260709 13:12:56.731873 21756 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 31065877c4af477d9b2e726230936891 [term 1 LEADER]: Raft consensus shutting down.
I20260709 13:12:56.732422 21756 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 31065877c4af477d9b2e726230936891 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 13:12:56.732793 21756 tablet_replica.cc:333] T 00000000000000000000000000000000 P 31065877c4af477d9b2e726230936891: stopping tablet replica
I20260709 13:12:56.754612 21756 master.cc:584] Master@127.21.63.62:41855 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (11470 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (24378 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (347215 ms total)
[  PASSED  ] 8 tests.
I20260709 13:12:56.808596 21756 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 379 messages since previous log ~14 seconds ago
I20260709 13:12:56.808892 21756 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 379 messages since previous log ~14 seconds ago
I20260709 13:12:56.809092 21756 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 372 messages since previous log ~14 seconds ago
I20260709 13:12:56.809294 21756 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~33 seconds ago
I20260709 13:12:56.809497 21756 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~33 seconds ago
I20260709 13:12:56.809691 21756 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/master/catalog_manager.cc:4755: suppressed but not reported on 1 messages since previous log ~74 seconds ago
I20260709 13:12:56.809906 21756 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 9 messages since previous log ~172 seconds ago
I20260709 13:12:56.810158 21756 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 4 messages since previous log ~0 seconds ago
I20260709 13:12:56.810454 21756 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 26 messages since previous log ~6 seconds ago
